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Preface 


Welcome  to  ED-MEDIA  99  in  Seattle,  USA.  This  year  sees  the  conference  back  in  the  USA  after 
two  successful  years  in  Calgary  and  Freiburg.  ED-MEDIA  99  represents  the  11th  time  this 
conference  has  run  and  this  year's  conference  promises  to  be  as  successful  as  those  which  have 
preceded  it. 

The  Conference  is  organised  by  the  Association  for  the  Advancement  of  Computing  in  Education 
(AACE)  and  over  the  years  has  developed  a strong  international  following.  The  Conference  has 
grown  hand  in  hand  with  the  emergence  of  new  learning  technologies  and  now  represents  the 
premier  event  for  this  field  in  the  world.  Each  year  sees  an  increasing  number  of  delegates  and 
participants  and  showcases  the  best  efforts  of  the  major  advocates  and  researchers  of  learning 
technologies. 

The  strong  international  participation  of  the  conference  is  demonstrated  by  the  depth  and 
breadth  of  submissions  we  have  received  this  year  for  presentation  at  ED-MEDIA.  We  received 
over  600  proposals  of  full  and  short  papers,  workshops,  tutorials  and  posters  from  nearly  50 
different  countries.  Each  proposal  was  generally  reviewed  by  3 referees  from  our  panel  of 
expert  reviewers  on  the  Program  Committee.  Those  selected  for  presentation  represent  work  of 
the  highest  quality.  Our  program  of  keynote  and  invited  speakers  once  again  reflects  the 
international  flavour  of  the  programme,  and  we  are  confident  that  these  people  will  inspire  you 
by  challenging  your  assumptions  and  extending  your  thinking. 

In  our  roles  as  Program  Chairs,  we  have  been  very  interested  to  explore  the  types  of 
presentations  that  people  choose  to  make  at  ED-MEDIA.  We  are  constantly  looking  for  ways  to 
bring  new  technologies  into  the  mainstream  of  educational  practices.  Our  reviews  of  the  forms 
of  research  and  content  reflected  overall  in  the  ED-MEDIA  programme  makes  interesting  reading 
and  provides  some  scope  for  reflection. 

To  aid  us  in  this  respect,  we  designed  an  analysis  tool  in  the  form  of  a matrix  to  enable  us  to 
classify  the  full  papers,  short  papers  and  posters  that  were  submitted  to  the  Program  Committee. 
The  matrix  considers  on  one  axis  the  stage  of  the  development  process  on  which  a paper  reports. 
On  this  axis  we  see  4 main  stages: 

1.  The  development  and  description  of  a concept; 

2.  Prototype  development; 

3.  Prototype  implementation  and  evaluation 

4.  Mainstream  implementation 

The  second  axis  of  the  matrix  considers  the  domain  of  the  technology.  We  chose  to  classify 
papers  into  the  following  categories: 

1.  Integrated  WWW  sites,  eg.  courses  on-line 

2.  WWW  Delivery  systems,  eg.  WebCT,  TeleTop 

3.  WWW  resources,  eg.  Java  Applets,  assessment  tools 

4.  Communications  technologies,  eg.  NetMeeting 

5.  CD-ROM  applications 

6.  Novel  environments  eg.  VR 

7.  Others  eg.  development  processes,  project  management 

When  submissions  were  classified  using  this  schema,  interesting  patterns  emerged.  Our  initial 
explorations  indicated  that  the  bulk  of  the  research  and  development  being  imdertaken  is  in 
Web-related  areas  and  in  particular  in  the  development  and  evaluation  of  courses  and 
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programmes.  Consider  for  example  the  results  of  using  this  schema  to  classify  the  papers 
considered  for  best  paper  awards  at  ED- MEDIA  99.  These  were  the  papers  that  achieved  an 
average  score  of  4.5  or  greater  out  of  a maximum  of  5 in  the  review  process. 


concept 

prototype 

evaluation 

mainstream 

Integrated  WWW 
sites 

5 

3 

8 

2 

WWW  delivery 
systems 

2 

1 

WWW  resources 

1 

6 

2 

Communications 

technologies 

2 

CD-ROM 

applications 

3 

Novel  environ- 
ments eg.  VR 

1 

Other  eg.  project 
development 

3 

2 

1 

Classification  of  the  papers  considered  for  a Best  Paper  Award,  ED-MEDIA  99 

In  terms  of  the  stages  of  development,  most  papers  tend  to  report  prototype  development  and 
evaluation.  There  appear  to  be  few  ideas  that  go  beyond  this  stage.  Very  few  researchers  are 
reporting  on  the  mainstream  use  of  their  products  and  ideas.  The  work  seems  to  be  stopping  at 
an  early  stage  and  not  being  taken  to  the  final  stage  where  it  can  really  have  an  influence.  We 
intend  to  explore  these  trends  further  and  hope  to  be  able  to  report  back  to  you  on  our  findings. 

As  Conference  Co-Chairs  we  have  been  aided  considerably  by  a number  of  volimteers  and 
helpers  who  have  given  tirelessly  of  their  time.  In  particular  we  must  thank  the  Conference 
Steering  Committee  of  Gary  Marks,  Ivan  Tomek,  Tom  Reeves  and  Erik  Duval  who  led  and 
coordinated  the  conference  this  year.  We  would  like  to  thank  the  Chairs  of  the  various  Program 
Committees,  Niki  Davis,  Panels  Chair;  Sam  Rebelsky,  Workshops /Tutorials  Chair;  and  Martyn 
Wild,  Posters/Demos  Chair.  We  would  also  like  to  thank  the  crew  behind  the  scenes  from 
AACE  for  their  tireless  efforts  in  coordinating  and  managing  the  huge  munber  of  administrative 
functions  which  a conference  like  this  necessarily  involves. 

We  look  forward  to  meeting  with  you  during  the  conference  and  trust  that  you  will  take  the  time 
to  renew  old  acquaintances,  meet  new  friends,  be  inspired  by  peers  and  take  home  some  new 
ideas  and  activities  to  report  back  at  ED-MEDIA  2000  in  Montreal,  Jime  26-July  1. 


Betty  CoUis,  University  of  Twente,  The  Netherlands  (B.A.CoUis@edte.utwente.nl) 
Ron  Oliver,  Edith  Cowan  University,  Western  Australia  (r.oliver@cowan.edu.au) 
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Invitation  to  Join 

The  Association  for  the  Advancement  of  Computing  in  Education  (AACE)  is  an 
international,  non-profit  educational  organization.  The  Association's  purpose  is  to  advance 
the  knowledge,  theory,  and  quality  of  teaching  and  learning  at  all  levels  with  information 
technology.  This  purpose  is  accomplished  through  the  encouragement  of  scholarly  inquiry 
related  to  technology  in  education  and  the  dissemination  of  research  results  and  their  applications 
through  AACE  sponsored  publications,  conferences,  and  other  opportunities  for  professional  growth 

AACE  members  have  the  opportunity  to  participate  in  topical  and  regional  divisions/societies/chapters,  high-quality 
peer-reviewed  publications,  and  conferences. 

Join  with  fellow  professionals  from  around  the  world  to  share  knowledge  and  ideas  on  research, 
development,  and  applications  in  information  technology  and  education.  AACE's  membership 
^ ^ includes  researchers,  developers,  and  practitioners  in  schools,  colleges,  and  universities; 

administrators,  policy  decision-makers,  professional  trainers,  adult  educators,  and 
other  specialists  in  education,  industry,  and  government  with  an  interest  in  advancing 
knowledge  and  learning  with  information  technology  in  education. 


Membership  Benefit  Highlights 

H Gain  professional  recognition  by  participating  in  AACE  sponsored 
international  conferences 

H Enhance  your  knowledge  and  professional  skills  through  interaction 
with  colleagues  from  around  the  world 

H Learn  from  colleagues ' research  and  studies  by  receiving  AACE's 
well-respected  journals  and  books 

H Receive  a subscription  to  the  Professional  Member  periodical 
Educational  Technology  Review 

H Receive  discounts  on  multiple  Journal  subscriptions,  conference 
registration  fees,  proceedings  books  & CD-ROMs 


PO  Box  2966  ■ Charlottesville  VA  22902  USA 

www.aace.org 


Advancing  Knowledge  and  Learning  with  Information  Technology! 


□□BB  Journals 

Abstracts  for  all  journal  issues  are  available  at  www.aace.org/pubs 


Educational  Technology  Review 

International  Forum  on  Educational  Technology  Issues  & Applications 

(ETR)  ISSN#  1065-6901 

AACE's  member  journal  is  the  focal  point  for 
AACE  members  to  exchange  information 
between  disciplines,  educational  levels,  and 
information  technologies.  It's  purpose  is  to 
stimulate  the  growth  of  ideas  and  practical 
solutions  which  can  contribute  toward  the 
improvement  of  education  through  informa- 
tion technology.  All  AACE  Professional  and 
Student  Members  receive  ETRds  a benefit  of 
membership. 


WebNet  Journal 

Internet  Technologies,  Applications  & Issues 

ISSN#  1 522-1 92X  Quarterly 

Focused  on  WWW,  Internet,  and  Intranet- 
based  technologies,  applications,  research, 
and  issues,  the  WebNet  Journal  is  intended 
to  be  an  innovative  collaboration  between 
the  top  academic  and  corporate  laboratory 
researchers,  developers,  and  end-users. 
Columnists  offer  how-to  articles  and  expert 
commentary  on  the  latest  developments. 


Journal  of  Educational  Multimedia  and  Hypermedia  Journal  of  Computers  in  Mathematics  & Science  Teaching 

(JEMH)  ISSN#  1055-8896  Quarterly  (JCMSJ)  ISSN#  0731-9258  Quarterly 


Designed  to  provide  a multidisciplinary  forum  to 
present  and  discuss  research,  development  and 
applications  of  multimedia  and  hypermedia  in  edu- 
cation. The  main  goal  of  the  Journal  is  to  con- 
tribute to  the  advancement  of  the  theory  and  prac- 
tice of  learning  and  teaching  using  these  powerful 
and  promising  technological  tools  that  allow  the 
integration  of  images,  sound,  text,  and  data. 


JCMST  ls  the  only  periodical  devoted  specifically 
to  using  information  technology  in  the  teaching  of 
mathematics  and  science.  The  Journal  otters  an 
in-depth  forum  for  the  interchange  of  information 
in  the  fields  of  science,  mathematics,  and  com- 
puter science. 


Journal  of  Interactive  Learning  Research 

(JILR)  ISSN#  1093-023X  Quarterly 


Journal  of  Technology  and  Teacher  Education 

(JTATE)  ISSN#  1059-7069  Quarterly 


The  Journals'  published  papers  relate  to  the  underly- 
ing theory,  design,  implementation,  effectiveness, 
and  impact  on  education  and  training  of  the  following 
interactive  learning  environments:  authoring  systems, 
CALL,  assessment  systems,  CBT,  computer-mediat- 
ed communications,  collaborative  learning,  distrib- 
uted learning  environments,  performance  support 
systems,  multimedia  systems,  simulations  and 
games,  intelligent  agents  on  the  Internet,  intelligent 
tutoring  systems,  micro-  worlds,  virtual  reality  based 
learning  systems. 


A forum  for  the  exchange  of  knowledge  about  the 
use  of  information  technology  in  teacher  educa- 
tion. Jotyra/ content  covers  preservice  and  inser- 
vice teacher  education,  graduate  programs  in 
areas  such  as  curriculum  and  instruction,  educa- 
tional administration,  staff  development,  instruc- 
tional technology,  and  educational  computing. 


International  Journal  of  Educational  Telecommunications 

(IJEQ  ISSN#  1 077-91 24  Quarterly 

IJET  serves  as  a forum  to  facilitate  the  interna- 
tional exchange  of  information  on  the  current 
theory,  research,  development,  and  practice  of 
telecommunications  in  education  and  training. 
This  Journal  is  designed  for  researchers,  devel- 
opers. and  practitioners  in  schools,  colleges,  and 
universities;  administrators,  policy  decision- 
makers, professional  trainers,  adult  educators, 
and  other  specialists  in  education,  industry,  and 
government. 


Information  Technology  in  Childhood  Education  Annual 

(ITCE)  ISSN#  1043-1055 

A primary  information  source  and  forum  to  report 
the  research  on  and  applications  of  using  infor- 
mation technology  in  the  education  of  children— 
early  childhood,  preschool,  and  elementary.  The 
annual  is  a valuable  resource  for  all  educators 
who  use  computers  with  children. 


/n formation  Technology 
in  Chiidhood  Education 
Annual 
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Conferences 


Details  for  conferences  are  available  at  www.aace.org/conf 


The  exchange  of  ideas  and  experiences  is  essential  to  the  advancement  of  the  field  and  the  professional  growth  of  AACE  mem- 
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Abstract:  In  traditional  education,  students  belong  to  only  one  school  and  passively  learn 
from  teachers  through  information  transmission  such  as  lectures  and  classroom  demonstrations, 
along  with  a few  opportunities  for  active  participation.  Traditional  approaches  to  education  are 
not  effective  enough  to  prepare  citizens  to  cope  with  the  complicated  problems  of  the  changing 
societies  of  the  21st  century.  Students  need  to  develop  the  skills  that  will  enable  them  to 
become  lifelong  learners. 

Using  information  and  communications  technology,  we  could  overcome  borders  of 
countries,  universities,  schools  and  academic  subjects,  and  also  barriers  of  even  time  and 
location.  Each  student  could  select  the  best  course  or  even  the  best  school  or  university  in  the 
world,  according  to  his/her  own  interests,  by  means  of  internet,  satellite  and  multimedia 
technology.  Information  and  communications  technology  is  reforming  education.  Recent 
examples  of  internet  use  in  schools,  satellite  use  in  universities,  new  types  of  multimedia 
development  will  be  described. 


1.  Culture  Characterized  by  a Private  Room  and  Culture  Characterized  by  a Living 
Room 

Some  twenty  years  ago,  when  a British  friend  of  mine  and  I were  driving  through  an  old  neighborhood  in 
London,  he  pointed  toward  the  sky  and  said,  "Look  at  the  skyline  over  there.  You  can  see  small  chimneys  on  the 
roof  of  each  house.  Four  chimneys,  six  chimneys  there  s a roof  with  eight.  Each  one  of  them  connects  to  a 
fireplace  in  a private  room  with  a locked  door.  Most  of  these  children  have  their  own  rooms.  Many  of  them,  boys 
and  girls  both,  will  leave  their  homes  after  high  school  and  live  independently  in  their  own  flats." 

After  this  conversation,  I began  to  pay  closer  attention  to  this  ubiquitous  element  of  so  many  Western 

skylines,  although  today,  of  course,  more  chimneys  serve  as  a vent  or  decorative  element  than  as  an  actual 

conduit  for  smoke. 

Which  brings  me  to  the  question:  How  are  Japanese  houses  structured?  We  can  begin  by  saying  that  Japanese 
homes  have  a completely  different  structure  than  their  Western  counterparts.  A typical  Japanese  house  features  a 
central  living  room  surrounded  by  a number  of  other,  smaller  rooms.  These  other  rooms  are  partitioned  by 
simple  paper  screens  or  papered  sliding  doors  called  fusuma.  Because  there  are  no  locks  or  keys,  anyone  can 
open  these  partitions.  Within  the  living  room  itself,  this  strategy  of  free-flowing  division  is  often  continued  by 
means  of  free-standing  panels  or  folding  screens.  In  this  sense,  Japanese  homes  are  extremely  flexible  and  open. 
It  is  possible  that  this  difference  between  Japanese  and  Western  homes  derived  from  climatic  conditions. 
Western  homes  must  bear  the  severe  cold  of  winter,  but  Japanese  homes  must  bear  the  heat  and  humidity  of 
summer.  But  what  I most  want  to  emphasize  here  is  that  the  respective  openness  and  containment  of  the 

Japanese  and  Western  architectural  styles  seems  to  have  strongly  influenced  the  way  the  members  of  these 

cultures  think  and  behave. 

In  Europe,  individuals  are  of  primary  importance.  They  join  to  form  a group  as  equal  participants.  Brothers 
and  sisters  call  each  other  by  their  first  names.  In  one  case,  I even  know  of  secondary  school  students  who  call 
their  parents  John  and  Mary.  Parents,  brothers,  and  sisters  hold  seemingly  equal  positions. 

In  Japan,  on  the  other  hand,  younger  brothers  and  sisters  often  call  their  elder  brothers  and  sisters  simply 
"brother"  or  "sister,"  rather  than  using  their  first  names.  Children  almost  never  address  their  parents  by  their  first 
names.  When  sitting  around  the  fire,  the  head  of  the  household  sits  in  a central  position,  and  the  others  hold  their 
respective  positions  in  the  order  of  seniority,  men  and  women.  There  has  been  a clear  hierarchy  among  family 
members.  In  terms  of  privacy,  even  if  children  are  given  private  rooms,  their  parents  are  able  to  open  the  paper 
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screens  or  papered  sliding  doors  at  any  time.  The  solidarity  of  the  group  is  thus  considered  much  more  important 
than  the  exertion  of  one’s  individuality.  In  the  center  of  the  house  is  a living  room  with  a fire.  Private  rooms,  if 
they  exist  at  all,  are  only  temporary  constructions.  In  Western  homes,  the  living  room  is  a place  for 
communication  where  family  members  demonstrate  their  own  individuality  while  respecting  the  individuality  of 
the  other  members.  Schools,  which  are  designed  as  an  extension  of  the  home,  are  often  equipped  with  a central 
open  space.  Just  as  at  home,  children  realize  their  own  individualities  within  this  educational  space,  while 
remaining  careful  to  respect  the  individualities  of  their  peers.  In  the  past,  as  an  extension  of  the  home-fire 
setting,  schools  in  Japan  were  often  called  classroom  kingdoms,  with  children  studying  together  under  a teacher 
who  served  as  "king." 

Today,  however,  with  the  exception  of  a number  of  large,  country  homes,  a more  west-European  style  of 
living  prevails  throughout  Japan.  Apartments  increasingly  consist  of  private  rooms  with  a key.  In  this  sense, 
people's  life  styles  are  steadily  shifting  to  include  an  assertion  of  privacy. 

A key  signifies  one's  domination  over  what  is  locked.  Without  a key,  no  one  can  get  into  a domicile,  or  take 
anything  out.  In  the  past,  then,  a key  was  a security  measure  by  which  wealthy  people  protected  their  property. 
In  an  environment  where  people  trom  different  ethnic  groups  and  social  classes  lived  together,  a key  was  a 
protective  necessity.  In  earlier  Japan,  however,  keys  were  largely  superfluous.  Villagers  helped  each  other  and 
treasured  common  property,  private  ownership  of  which  was  forbidden.  Personal  belongings  were  acknowledged 
by  other  group  members  and  thus  did  not  have  to  be  locked  up.  This  was  the  case  in  which  group  members  are 
homogeneous. 

In  societies  of  this  type,  individuals  obeyed  the  group  to  which  they  belonged.  Dissenting  viewpoints  were 
unwelcome  and  were  considered  to  disturb  the  collective  harmony.  This  system  worked  quite  well  when  the 
group  leader  was  well-intentioned  and  capable.  But  in  the  absence  of  effective  leadership,  the  results  of  this 
paradigm  could  be  tragic.  Then,  group  members  who  had  very  good  ideas  that  might  have  improved  the  situation 
were  hardly  allowed  to  propose  them,  and  people  generally  withheld  their  creative  ideas  for  reform  or  their 
initiatives  for  problem  solving. 

This  Japanese  model  of  society  tended  to  work  best  under  peaceful  conditions  and  with  a group  of  reliable 
members  from  the  same  ethnic  background.  In  the  modem  age,  it  has  become  increasingly  difficult  for  a society 
to  exist  in  such  relative  autonomy,  and  the  necessity  of  contact  with  different  cultural  backgrounds  has  made  a 
purely  homogeneous  group  impossible. 

In  Western  culture,  individuals  are  not  only  permitted,  but  often  encouraged  to  assert  their  own  ideas.  But 
their  contemporaries  must  be  persuaded  of  the  logical  efficacy  of  a particular  idea  before  it  will  be  implemented. 
Failure  to  persuade  others  can  lead  to  the  dissolution  of  a group.  Thus  a culture  characterized  by  a "private 
room"  mentality  finds  it  difficult  to  maintain  the  group's  common  purposes  and  solidarity  while  simultaneously 
encouraging  individuals  to  display  their  creativity  and  problem-solving  abilities.  On  the  other  hand,  a culture 
characterized  by  the  "living  room"  paradigm  is  effective  at  engendering  solidarity  and  sharing  property,  but  less 
effective  at  maximizing  the  individual  strengths  of  its  members. 

Education  in  the  21st  Century  will  continue  to  develop  along  with  internationalization,  advances  in  science 
and  information  technology,  and  an  increasing  population  of  elderly  and  decreasing  population  of  children. 
Cultures  characterized  by  private  rooms  and  those  characterized  by  living  rooms  will  meet  more  frequently 
everywhere.  And  the  question  of  whether  to  adopt  a "living  room"  culture  while  developing  a "private  room" 
culture,  or  to  adopt  a "private  room"  culture  while  developing  a "living  room'’  culture  is  now  in  the  critical  point. 

Education  in  Europe  and  North  America  is  shifting  away  from  the  "private  room"  model  and  placing  new 
emphasis  on  characteristics  of  a "living  room"  culture,  such  as  collective  purpose  and  common  standards. 
Conversely,  education  in  Japan  is  shifting  toward  a new  focus  on  respect  for  individuals,  and  in  opening  up  the 
classroom  "kingdom"  in  an  attempt  to  communicate  with  different  cultures  while  maintaining  student 
individuality.  In  this  way,  both  Europe/North  America  and  Japan  are  attempting  to  recognize  and  assimilate  the 
strengths  of  one  another’s  systems. 

Japanese  education  in  the  21st  Century  will  also  attempt  to  further  the  open  and  flexible  way  of  Japanese 
living  so  as  to  expand  the  "living  room"  culture  from  the  home  to  the  social  unit,  from  the  social  unit  to  the 
nation,  and  from  the  nation  to  the  world.  It  is  important  that  this  expansion  also  respects  the  independence  of 
individual  students  by  means  of  flexible  temporary  partitions  made  up  of  free-standing  panels,  folding  screens, 
paper  screens,  and  papered  sliding  doors,  as  well  as  by  a "latch  of  the  heart," 

Learning  to  exploit  advanced  information  and  communications  technologies  is  important  whether  the  aim  of 
education  is  to  develop  a cooperative  mind  with  an  emphasis  on  individuality,  or  an  individualist  mind  with  an 
emphasis  on  cooperation.  It  is  important  because  such  tools  as  multimedia  content,  home  pages,  databases,  and 
network  communications  not  only  allow  access  to  the  inspired  ideas  of  many  individuals,  but  can  also 
nonetheless  be  applied  to  achieving  common  goals. 
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2.  Changes  in  Learning  Styles 


In  the  realm  of  2 1 st  Century  education,  the  learning  styles  of  children  everywhere  will  change. 

Since  antiquity,  human  beings  have  tended  to  acquire  knowledge  in  a largely  passive  manner.  That  is, 
knowledge  has  been  passed  on  from  men  of  wisdom  and  authority.  The  Analects  of  Confucius  and  the  sermons 
of  Buddha  and  Christ  are  typical  examples.  People  listened  to  the  speeches  of  these  seminal  instructors  in  order 
to  gain  instruction  for  their  lives.  Even  after  the  invention  of  the  printing  press,  learning  meant  passive  reading  of 
the  writings  of  knowledgeable  and  authoritative  authors.  In  Japan,  such  wisdom  was  usually  imparted  in  the 
course  of  repeated  silent  readings. 

The  introduction  of  radio  and  television  did  not  change  this  model  of  passive  learning.  Even  in  school 
broadcasts,  the  University  of  the  Air,  and  other  types  of  distance  education,  the  ordinary  style  of  learning  is  to 
learn  passively  through  a teacher's  presentation  and  a textbook.  Those  who  appear  on  these  broadcasts  are  still 
men  or  women  of  wisdom  and  authority,  and  their  students  continue  to  be  largely  passive  recipients. 

Even  this  passive  learning  environment,  however,  sometimes  involves  more  constructive  activities  in  the 
form  of  submitting  reports  and  taking  examinations.  And,  with  the  growing  emphasis  being  placed  on  searching 
skills  and  active  learning,  investigative  and  constructive  activities  have  become  important.  This  has  resulted  in  a 
wide  range  applications,  from  scientific  investigations  or  an  observational  diary,  to  a general  emphasis 
particularly  in  elementary  school  on  careful  articulation  of  opinions. 

This  type  of  active  learning  will  continue  to  expand  along  with  the  refinement  of  the  various  media  which 
support  it.  One  notable  aspect  of  active  learning  concerns  the  selection  of  pertinent  information.  Recently,  huge 
volumes  of  educational  information  have  been  made  available  to  students,  including  textbooks,  illustrated 
guides,  newspapers,  magazines,  printed  materials,  web  pages,  and  TV,  radio,  video,  and  computerized  teaching 
materials.  Learners  must  positively  select  the  information  appropriate  for  their  learning.  If  learners  take  in 
information  without  a critical  view,  they  will  find  themselves  drowning  in  a sea  of  information.  They  must  also 
compare,  process,  edit,  and  compile  the  selected  information  in  order  to  produce  a finished  report  or 
presentation.  This  process  is  an  excellent  example  of  learning  through  creating.  Such  engaged  learning  is  critical, 
because  learners  understand  content  more  deeply  when  creating  something  on  their  own. 

Multimedia  and  the  Internet  are  powerful  instruments  to  support  this  process.  Learners  extract  information 
from  the  web  pages  of  universities,  museums,  companies,  etc.,  or  take  digital-camera  photos  of  their  community, 
neighbors,  or  local  parks,  and  then  gather  all  this  information  into  a computer,  edit  it,  and  complete  a multimedia 
work.  They  can  show  the  finished  product  to  friends  around  the  world  via  e-mail  or  a web  page.  But  the 
opportunity  for  learning  does  not  stop  here.  Friends  and  anonymous  enthusiasts  around  the  world  are  likely  to 
respond  to  the  student's  work  with  questions,  supplemental  material,  or  corrections.  And  these  responses  will 
further  motivate  the  child  to  reinvestigate,  review  and  modify  his  or  her  works.  This  process  for  deepening  the 
learning  experience  is  not  possible  via  conventional  teaching  methods. 

What  is  particularly  important  is  that  new  tools  for  expression,  such  as  cameras,  videos,  word  processors,  and 
computers  have  appeared  and  spread.  As  a result,  the  communications  world  that  once  focused  mainly  on 
receiving  information  has  now  evolved  into  a world  of  transmission  as  well.  Ordinary  people  can  easily  transmit 
words,  voice,  sound,  still  images,  and  moving  video  images.  Today,  business  people  routinely  create  and 
transmit  information  using  multimedia  tools  and  word  processors.  Even  children  can  draw  pictures  on  their 
computers  and  win  competitions  in  contests  with  middle  school  students  or  older  competitors.  In  one  instance,  a 
first  grader  in  primary  school  created  a multimedia  work,  stored  it  on  a floppy  disc,  entered  a contest,  and  won 
the  Ministry  of  Education's  Prize.  And  there  is  currently  a primary-school  boy  who  is  creating  "multimedia 
poems." 

Originally,  when  people  drew  pictures,  sang  songs,  played  musical  instruments,  or  recited  sentences,  they 
were  transmitting  messages  in  the  same  way  that  they  communicated  spoken  words.  Often,  it  was  a process 
which  required  considerable  time  and  effort.  With  a computer,  however,  one  can  draw  pictures  and  compose 
music  easily.  The  pictures  drawn  and  music  composed  on  a computer  can  be  modified  or  duplicated  an  unlimited 
number  of  times.  Mistakes  can  be  instantly  undone  with  the  click  of  a mouse.  In  the  case  of  real  paintings  or 
musical  scores,  however,  because  erasing  is  so  difficult,  the  expression  of  radical  new  ideas  is  less  likely.  Also, 
with  the  current  prevalence  of  word  processors,  an  "information  sending"  society  in  which  everyone  can  be  a 
sender  of  information  is  rapidly  approaching.  Today,  if  a school  is  linked  to  a network,  one  individual  can 
disseminate  a message  all  over  the  world. 

In  this  way,  written  media  as  a source  of  information  has  expanded  to  multimedia.  We  can  selectively  receive 
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information  from  the  media  through  reliable  databases.  We  can  also  send  multimedia  information  of  our  own. 
We  are  in  the  midst  of  a great  change  in  the  history  of  civilization. 

In  the  21st  Century,  the  understanding,  exploitation,  creation,  and  transmission  of  multimedia  information 
will  become  nothing  special.  Information  integrating  text,  still  images,  moving  video  images,  voice,  and  sound  is 
expected  to  be  routinely  delivered  on  networks. 

Learners  in  the  coming  advanced  information  society  will  have  to  possess  not  only  the  skills  of  reading, 
writing,  and  understanding  information  delivered  from  numerous  media,  including  TV,  but  also  the  skills  to 
retrieve  and  use  multimedia  information,  and  to  create  and  transmit  multimedia  works. 

In  receiving  information  from  the  media,  we  must  have  the  ability  to  distinguish  authentic  information  from 
inauthentic  or  biased  information,  and  we  must  be  able  to  make  this  distinction  independently,  as  individual 
agents.  We  also  must  be  careful  not  to  lose  touch  with  the  physical  world  around  us  through  an  excessive 
reliance  on  information.  When  sending  information,  we  must  be  careful  not  to  invade  the  privacy  of  others.  We 
should  not  use  or  create  biased  or  erroneous  information.  We  must  respect  the  intellectual  property  rights  of 
information  created  by  others.  To  act  responsibly,  learners  must  be  aware  of  the  importance  of  information  for 
life  in  society. 


3.  Educational  Reform  by  Information  and  Communications  Technology 

Information  and  communications  technology  are  dramatically  changing  many  aspects  of  education,  including 
the  essential  functions,  goals  and  contents  of  education,  the  methods,  teaching  materials,  management, 
administration,  and  teacher  education  used  to  realize  these  goals,  and  the  partnership  between  education  and  the 
local  community. 

- Essence  of  Education 

Education  has  always  had  two  major  functions.  The  first  is  to  hand  on  to  successive  generations  the  academic, 
cultural,  technological,  and  traditional  heritage  of  our  ancestors.  The  second  is  to  help  foster  desirable 
characteristics  in  the  citizens  of  the  next  generation,  and  thus  of  the  societies  of  the  future.  Elements  to  be 
fostered  include  creativity,  imagination,  the  skills  to  express  oneself,  sensitivity,  ethics,  motivation,  physical 
strength,  coordination,  collaboration,  cooperation,  and  independence. 

The  essence  of  education  has  not  changed  even  in  the  multimedia  and  Internet  age.  However,  other  aspects  of 
education  are  likely  to  be  affected  by  the  requirements  of  particular  times. 

- Goals  and  Contents  of  Education 

In  the  past,  the  goals  of  education  have  tended  to  focus  on  the  first  of  the  functions  described  above,  the 
acquisition  of  knowledge  as  an  inheritance  from  previous  generations.  Today,  however,  new  emphasis  is  being 
placed  on  the  second  function,  i.e.,  the  evocation  of  student  interest  and  motivation,  and  the  fostering  of 
intelligence,  judgment,  and  self-expression.  Stronger  emphasis  is  being  placed  on  developing  the  ability  to  take 
the  initiative  in  creating  a future  society,  than  on  inheriting  past  assets.  In  Japan,  the  goal  of  the  new  education  is 
to  foster  a "zest  for  living." 

The  emergence  of  multimedia  and  the  Internet  has  dramatically  altered  the  goals  of  education.  Knowledge 
and  skills  of  information  technology,  an  understanding  of  the  role  and  importance  of  information  technology  in 
the  society,  and  information  ethics  are  new  goals  of  education.  And  as  the  goals  of  education  change,  the 
educational  contents  change  with  them.  To  educate  students  to  achieve  an  equal  academic  level  nationwide 
requires  consistent  curricula  throughout  primary  and  secondary  education. 

- Methods  of  Education 

The  most  drastic  educational  change  resulting  from  the  advent  of  multimedia  and  the  Internet  is  in  the 
educational  methods  themselves.  Newly  introduced  teaching  methods  include  individual  learning  that  exploits 
multimedia  instructional  materials,  creation  of  multimedia  works  (e.g.,  multimedia  illustrated  guides  that 
integrate  text,  music  and  pictures),  exchange  of  presentations  with  other  children  on  the  Internet,  collection  of 
necessary  material  from  databases,  global  simultaneous  computation  of  records,  and  classroom  exchanges  via 
satellite  and  telecommunication  network. 

- Instructional  Materials 

Multimedia  instructional  materials  in  the  form  of  CD-ROM  and  DVD-ROM  have  been  added  to  such 
conventional  instructional  materials  as  print  media,  models,  slides,  and  OHP.  Instructional  materials  are  directly 
collected  from  various  sources  around  the  world  via  the  Internet.  Lists  of  links  to  instructional  material  sources 
are  also  available.  In  recent  years,  so-called  tool  software  has  been  used  widely.  And  application  software  itself 
designed  to  help  children  collect,  process,  edit,  create,  and  transmit  information  is  sometimes  used  as  an 
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instructional  material. 

- Facilities  and  Equipment 

Equipment  includes  peripheral  devices  for  computers  and  multimedia  tools,  network  systems,  and  basic 
software.  Facilities  need  to  be  expanded  to  install  these  devices  and  systems.  In  particular,  recent  years  have  seen 
the  spread  of  communications  satellites,  videoconferencing  systems,  and  high-speed  communications  networks. 

- Management  of  Education 

Because  schools  are  connected  to  the  whole  world  through  networks,  one  can  instantly  gain  access  to 
worldwide  education-related  information.  Distant  schools  and  experts  can  carry  out  team  teaching,  and  networks 
and  communications  satellites  will  help  make  conferences,  meetings,  and  training  courses  more  efficient. 

- Administration  of  Education 

The  introduction  of  multimedia  and  the  Internet  is  also  changing  the  administration  of  education.  This  will 
mean  changes  in  all  aspects  of  education  administration,  including  governmental  policy  making,  budgeting,  and 
dissemination  activities  to  facilitate  the  exploitation  of  information  technology  for  education. 

- Teacher  Education 

Because  multimedia  and  the  Internet  are  new  to  the  world  of  education,  many  teachers  still  have  difficulty 
using  them.  Not  surprisingly,  multimedia  and  the  Internet  are  recurrent  themes  at  the  training  events,  workshops, 
and  academic  meetings  of  teachers  worldwide.  Through  such  training,  teachers  will  be  able  to  acquire  new 
abilities,  including  the  skills  to  analyze  and  develop  instructional  materials  and  to  use  multimedia  and  the 
Internet. 

- Partnership  with  the  Local  Community 

By  making  multimedia  equipment  and  the  Internet  available  to  the  local  community,  schools  can  assert 
themselves  as  centers  of  culture  and  civilization  in  less  developed  regions.  In  developed  regions,  constant 
interactions  between  schools  and  the  local  community  will  be  possible  on  networks. 


4.  Exploitation  of  the  Internet  for  Education 

It  is  interesting  to  speculate  what  might  happen  if  cross-cultural  communications  continue  to  increase  through 
the  Internet  and  videoconferencing  systems.  With  cameras  and  videocameras,  children  will  be  able  to  collect  and 
edit  the  information  surrounding  them,  e.g.,  information  on  nature,  society,  customs,  languages,  cultural  heritage, 
and  the  people  of  their  community,  and  use  this  information  to  create  a multimedia  work.  They  will  then  present 
their  project  on  a home  page  or  send  it  to  friends  in  distant  places  via  e-mail.  Their  friends  will  reciprocate  with 
similar  multimedia  works.  Children  will  ask  questions  about  and  exchange  opinions  on  these  works.  Using  this 
information,  they  will  create  better  works  and  send  them  to  friends  in  distant  places.  For  example,  children  in 
snowy  regions  might  tell  children  in  the  South  Pacific  about  their  own  customs,  practices,  lives,  and  play  in  the 
snow-filled  winter.  Conversely,  children  in  the  South  Pacific  might  share  information  about  tropical  foods  and 
the  habitats  of  tropical  fish  in  a coral  reef.  Children  in  urban  cities  might  admonish  children  in  farming  villages, 
"You  should  not  use  agrichemicals."  Children  in  farming  villages  might  counter  them  by  saying,  "We  cannot 
produce  good-looking  agricultural  products  without  agrichemicals.  Will  people  in  urban  cities  buy  poorly  shaped 
vegetables  and  fruits?"  In  this  way  children  will  understand  that  others  may  have  different  positions. 

Utilizing  the  Internet  in  this  way,  children  will  be  able  to  better  understand  their  own  cultures  by  cross- 
comparison with  other  cultures  worldwide.  They  will  begin  to  appreciate  that  acknowledging  and  respecting 
different  cultures  is  integral  to  protecting  and  fostering  their  own.  It  may  not  be  too  much  to  hope  that,  as 
children  continue  to  interact  with  different  cultures  on  the  Internet,  some  of  the  ethnic  conflicts  currently 
underway  throughout  the  world  will  begin  to  disappear.  In  this  way,  the  educational  use  of  the  Internet  will  truly 
be  an  effective  means  of  engendering  peace. 

The  so-called  “100-School  Networking  Project”  launched  in  FY  (Fiscal  Year)  1995  with  the  participation  of 
111  schools  completed  its  tenure  of  two  years.  A new  100-School  Networking  Project  is  now  in  progress. 
Diversified  activities  are  being  carried  out  at  schools  and  other  educational  organizations  in  and  outside  Japan. 
These  activities  include  the  gathering,  transmission  and  exchange  of  information,  collaborative  learning  and 
research,  collaborative  production,  and  network  conferencing. 

Furthermore,  the  “KONET  Plan”,  designed  to  link  1,014  schools  via  the  Internet  and  a videoconferencing 
system,  has  already  been  launched.  Many  activities  such  as  interaction  via  e-mail,  information  gathering,  and 
homepage  creation  are  being  conducted. 

In  addition  to  these  projects,  such  areas  as  Gifu,  Kochi,  Saga  and  Osaka  prefectures,  and  Mitaka  City  have 
actively  been  introducing  the  Internet.  Non-profit  institutions  such  as  Media  Kids  have  also  contributed  to 
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increased  internet  use  by  students. 

In  FY  1997,  classroom  teaching,  linking  a classroom  in  a hospital  and  a master  classroom  in  a school  via 
optical  communications,  was  started  in  order  to  teach,  at  a distance,  children  suffering  from  an  illness. 

The  Ministry  of  Education  has  also  selected  20  districts  to  study  the  effects  of  an  information  and 
communications  network,  designating  six  schools  in  each  site.  As  part  of  the  R&D  project  to  study  the  use  of 
advanced  information  and  communications  facilities  for  schools  in  rural  areas,  the  Ministry  linked  schools  in 
rural  areas  with  schools  in  urban  areas.  These  schools  are  connected  via  two  sets  of  satellite  communications 
systems,  ten  sets  of  optical  communications,  and  19  sets  of  digital  communications  networks.  In  many  cases, 
children  in  rural  areas  benefit  from  acquiring  vivid  information  from  urban  schools.  It  is  expected  that  children 
in  rural  areas  will,  in  turn,  provide  those  in  urban  areas  with  information  on  their  rich  natural  environment  and 
traditional  culture,  thus  contributing  to  each  other’s  learning. 

The  FY  1998  supplementary  budget  included  a plan  to  connect  a total  of  118  schools  by  optical 
communications.  These  schools  comprise  two  schools  from  each  of  47  prefectures  and  12  cities  designated  by 
government  ordinance.  Approximately  1,100  schools  in  30  local  districts  are  also  connected  through  either 
CATV,  WLL  (Wireless  Local  Loop),  DSL  (Digital  Subscriber  Line),  or  satellite  internet  communication 
technologies. 


5.  Effects  of  Information  Technology  on  Education 

The  research  on  the  effects  of  computer-aided  instruction  (CAI)  in  enhancing  educational  experiences  has 
been  undertaken  by  many  studies.  In  fact,  as  computers  are  increasingly  used  for  problem-solving,  material 
production,  information  retrieval  and  transmission,  etc.,  their  effectiveness  is  no  longer  in  question.  Computer- 
assisted  graphs,  composition,  arrangement,  drawings,  and  multimedia  works  are  ubiquitous,  and  many  of  these 
projects  have  placed  first  in  distinguished  contests.  Such  prize-winning  works  could  not  have  been  created 
without  computers. 

Certainly,  computers  have  made  a substantial  contribution  to  education,  if  for  no  other  reason  than  because 
students  cannot  easily  gather,  retrieve,  create,  and  transmit  information  without  a computer  or  the  Internet.  But 
we  must  ask  ourselves,  do  computers  offer  any  benefits  other  than  convenience?  More  specifically,  do 
computers  help  develop  children's  ability  to  collect  information,  solve  problems,  edit  existing  material,  create 
new  things,  think  logically,  and  fully  express  what  they  think  or  feel?  Now  that  the  Internet  and  multimedia 
computers  have  been  introduced  to  most  schools,  studies  to  answer  these  questions  are  extremely  important. 

For  many  years  now,  our  group  has  been  investigating  the  effects  of  computers  and  the  Internet  on  education. 
The  earlier  outcome  of  our  study  showed  that,  as  in  Figure  1,  fifth  graders  playing  video  games  at  home  showed 
lower  levels  of  creativity,  motivation,  and  sympathy  with  others  than  those  not  playing  games.  In  contrast,  fifth 
graders  using  a word  processor  at  home  showed  higher  levels  of  creativity  and  motivation  than  those  not  using  a 
word  processor.  The  children  who  both  learned  computer  programs  at  school  and  used  a word  processor  at  home 
showed  higher  levels  of  creativity  than  the  children  who  did  only  one  of  these  activities.  Not  surprisingly,  those 
children  who  engaged  in  neither  activity  demonstrated  the  lowest  creativity  scores.  In  analyzing  these  results,  it 
is  noteworthy  that  the  children  who  both  played  games  and  used  a word  processor  at  home  showed  high 
creativity  scores.  Our  expectation  was  that  children  who  participated  in  both  these  activities  would  demonstrate 
median-levels  of  creativity,  intermediate  between  the  low  scores  of  those  who  only  played  video  games  and  the 
high  scores  of  those  who  only  engaged  in  word  processing.  However,  the  children  who  did  both  showed  the 
highest  creativity  scores.  Although  it  is  difficult  to  interpret  this  result  conclusively,  it  seems  reasonable  that 
these  children  realized  a synergistic  effect  between  the  constructive  skills  gained  from  word  processing  and  the 
instantaneous  judgment  skills  gained  from  playing  video  games. 

In  this  study,  creativity  was  assessed  by  means  of  questionnaires.  We  subsequently  devised  a test  to  measure 
thinking  ability  more  accurately  and  conducted  a six-month  panel  survey  of  the  same  subject  group  of  fifth 
graders. 


Figure  1.  How  does  a computer  affect  the  development  of  children’s  abilities? 
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Figure  2 shows  the  results  of  the  panel  survey.  Those  fifth  graders  who  took  many  computer  lessons  showed 
relatively  higher  scores  in  three  of  the  five  tests,  i.e.,  the  planning,  deduction,  and  space  cognition  tests,  than 
those  who  received  fewer  lessons.  Six  months  later,  however,  when  the  same  children  were  in  sixth  grade,  those 
who  received  more  computer  lessons  did  relatively  better  in  all  five  tests:  planning,  deduction,  space  cognition, 
induction  and  meta  cognition.  Thus,  for  the  fifth  graders,  the  results  of  the  induction  and  meta  cognition  tests 
were  not  affected  by  the  number  of  computer  lessons  taken.  In  fact,  those  fifth  graders  receiving  fewer  computer 
lessons  actually  showed  the  same  or  better  scores  for  the  two  parameters.  But  by  the  sixth  grade,  the  results  of  all 
five  tests  were  dramatically  improved  by  the  use  of  more  computer  lessons. 

We  conducted  another  creativity  test  by  using  the  commercial  Guilford  Test.  This  survey  is  a panel  conducted 
at  eight-month  intervals.  The  Path  analysis  showed  that  computer  use  improves  the  deductive  ability  of 
elementary  school  boys.  Unfortunately,  the  same  effect  was  not  obtained  for  junior  high  school  students  and 
elementary  school  girls.  Rather,  computer  use  lowered  the  creativity  of  elementary  school  boys.  We  may  say  that 
junior  high  school  students  and  elementary  school  girls,  who  grow  faster  than  boys,  are  too  old  to  develop  their 
inductive  ability  by  use  of  computers.  Because  elementary  school  boys  use  computers  based  on  certain  patterns, 
they  develop  their  deductive  ability  to  apply  their  way  of  thinking  to  different  cases.  On  the  other  hand,  it  was 
assumed  that  they  were  unable  to  develop  creativity  which  required  them  to  organize,  edit,  and  reconstruct 
information  by  themselves. 

We  also  conducted  a similar  panel  to  discuss  whether  computer  use  would  affect  children's  motivation  to 
learn  and  create  at  a later  date.  The  panel  was  for  fourth-  and  fifth-grade  boys,  and  we  found  that  those  who 
originally  possess  creativity  come  to  use  computers  more  effectively,  rather  than  that  computer  use  develops 
creativity.  This  indicates  that  the  use  of  computers  is  likely  to  widen  the  intellectual  gap  between  those  with 
higher  creativity  and  those  with  less  creativity.  In  this  sense,  a computer  functions  as  a gap  amplifier,  not  an 
equalizer. 

We  conducted  another  panel  to  investigate  how  the  exploitation  of  the  Internet  influences  children's 
information  literacy.  The  panel  showed  that,  for  junior  high  school  students,  use  of  the  Internet  enhanced 
knowledge  and  skills  relating  to  the  Internet.  In  terms  of  information  literacy,  it  was  shown  that  the  use  of  E-mail 
helps  develop  the  skills  of  retrieving  and  selecting  appropriate  information. 

We  also  conducted  research  activities  from  a slightly  different  perspective,  which  included  work  on  the  effect 
of  MUD  (Multi  User  Dungeon)  on  education.  We  found  that  when  we  let  introversive  students  behave 
extroversive  in  the  MUD,  the  introversive  students  looked  at  or  spoke  to  strangers  more  frequently  than  those 
who  were  given  a different  computer  task  for  the  same  number  of  hours.  This  study  indicates  that  the  experience 
of  using  a computer  affects  even  the  personality  dimensions  of  the  human  being. 
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Figure  2.  How  does  the  number  of  computer  lessons  affect  children’s  intellectual  abilities? 


DdfeETce  of  AdiEvementii  tfe  Resuts  dPJannrg,  Dedrtcn,  Ixirttn,  Space  Cogiirn,  and 
M eta  Ccgr±ixi  Ifeste  anti  eng  3 G lOLps 


W 

y 

□ NDlfisscns 

□ Mary  Lessens 

□ Ftew  Lessons 


Ptaibg  Dedctin  Iiirtrn  Space  Oogrutm  M eta  Cognttn  Total 


FVe -Month  litervalC hange  ii  the  Percentage  of  C orrect Answers  in  P3annTig  Tests 


u- 


46 


Pi/e-M  onth  Jiterval  Change  n the  Peinentage  of  Correct  Answers  n iiduction  Tests 


Fi^e-M  onth  IitervalChange  ii  the  Percentage  of  Correct  Answers  in  Space  Cognidon  Tests 


o 

ERIC 


47 


6.  R&D  Activities  at  NIME 


NIME  also  conducts  research  into  how  information  and  communications  technologies  should  be  exploited  to 
reform  higher  education. 

To  analyze  and  evaluate  broadcast  programs,  we  have  developed  a system  that  automatically  produces  a 
series  of  still  images  taken  at  specified  time  intervals  from  the  scenes  of  a given  program.  We  also  produce  600 
45-minute  programs  annually  for  the  University  of  the  Air  at  NIME  studios.  The  patterns,  photographs, 
illustrations,  graphs,  etc.,  which  were  used  for  these  programs,  are  incorporated  into  a still  image  database. 

A recent  team  investigated  the  physiologic  effects  of  multimedia  by  taking  measurements  on  the  brain 
bloodstream  and  EEG.  The  results  showed  that  certain  acoustic  waves  of  inaudibly  high  frequency  positively 
influence  human  physiology.  Based  on  this  finding,  an  industry  has  released  even  CDs  containing  inaudible 
sound. 

Other  NIME  activities  include  the  use  of  DVD  as  a new  medium  to  develop  the  world’s  first  DVD  video 
instructional  material.  At  present,  NIME  is  developing  web-based  DVD-ROM  instructional  materials.  But  the 
largest  activity  of  NIME  is  to  operate  the  Satellite  Communications  System  (SCS),  an  Inter-university 
educational  exchange  system  using  a communications  satellite. 


7.  Japanese  Universities  Linked  in  Space 

In  July  of  1998,  a meeting  of  the  Committee  for  Implementing  the  Space  Collaboration  System  (SCS)  project 
was  held.  For  the  meeting,  the  National  Institute  of  Multimedia  Education  (NIME)  functioned  as  a HUB  station 
and  76  VSATs  in  national  universities  across  the  nation  from  Hokkaido  University  to  The  University  of  Ryukyu 
were  connected  via  a communications  satellite  with  about  five  participants  at  each  of  the  VS  AT  stations.  I 
assume  that  approximately  400  people  attended  the  meeting  as  a whole. 

At  present,  1 10  national  universities  and  higher  education  institutions,  and  10  private  universities  have  VS  AT 
stations.  Some  universities  are  provided  with  more  than  one  VS  AT  station.  For  example,  Kyoto  University  has 
four  VS  AT  stations,  The  University  of  Tokyo,  Hokkaido  University,  Tohoku  University,  Nagoya  University,  and 
Osaka  University  have  three  each. 

VS  AT  stations  are  generally  equipped  with  two  large  display  screens:  one  showing  the  image  of  a 
communicating  station  and  the  other  showing  one’s  own  image  or  the  image  of  a third  station  now  in  contact. 
When  a close-up  of  a speaker’s  face  is  shown  on  the  screen,  a slight  time  lag  occurs  between  the  movement  of 
the  mouth  and  the  voice  heard,  which  may  be  disconcerting  to  some  people  compared  with  an  ordinary  TV 
screen.  However,  as  it  is  a negligible  lag,  one  can  get  used  to  it  soon. 

An  operating  desk  is  equipped  with  simple  terminal  devices,  which  are  easily  operated  by  a teacher  or  an 
attendant.  Every  station  can  transmit  images  and  host  events  such  as  lectures,  guidance,  and  conferences  as  an 
organizer.  An  organizing  university/institution  plays  the  role  of  moderator  and  can  call  up  a station  onto  the 
screen.  When  someone  at  a receiving  station  wants  to  express  an  opinion,  that  station  Just  makes  a request  to 
speak  by  touching  a panel.  However,  if  that  station  is  not  called  up  by  the  organizer,  its  opinion  is  not  heard  by 
others.  In  this  sense,  the  function  of  the  organizer/moderator  is  very  influential. 

We  are  approaching  an  era  in  which  undergraduate  and  postgraduate  students  select  and  take  courses  by  their 
choice  from  among  courses  offered  by  universities  through  multimedia  all  across  Japan.  Those  universities 
having  many  faculty  members  chosen  by  many  undergraduate  and  postgraduate  students  all  around  Japan  will 
gain  a high  repute.  Those  universities  whose  teachers  are  not  chosen  by  other  universities’  students  will  be 
eclipsed  as  a result.  An  increasing  number  of  undergraduate  and  postgraduate  students  now  retrieve  information 
from  various  sources  around  the  world  through  the  Internet  for  their  studies.  They  become  free  from  ’’passive 
learning”  in  which  students  passively  osmose  their  professors’  lectures.  They  search,  create,  and  send  out 
information  on  their  own  via  networks,  receive  responses  from  people  around  the  world,  and  thereby  deepen  the 
level  of  their  studies  further.  This  style  of  learning  is  beginning  to  develop  as  we  enter  the  age  of  multimedia  and 
distance  education. 

University  reform  has  literally  been  initiated  in  space. 


8.  Significance  of  Exploiting  Information  Technology  for  Education 

Systems  Function 
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(1)  Equalizer  and  Amplifier 

Functionally,  information  technology  provides  everyone  with  an  equal  opportunity  of  education.  On  the  one 
hand,  information  technology  works  as  an  equalizer  because  it  enables  everyone  to  obtain  necessary  information 
from  all  over  the  world.  On  the  other  hand,  information  technology  works  as  an  amplifier  because  people  can 
collect  a larger  amount  of  information  in  less  time  by  exploiting  information  technology,  rather  than  by  reading 
books  as  in  the  past  However,  there  is  a potential  danger  that  information  technology  might  create  a wide  gap 
between  those  who  can  collect  a large  amount  of  information  and  those  who  cannot,  and  between  those  who 
learn  fast  and  those  who  learn  slowly.  " 

(2)  Generalization  and  Specialization  \ 

Information  technology  enables  everyone  to  share  information,  thus  contributing  to  the  generalization  of 

common  education.  Moreover,  it  facilitates  the  systematic  collection  of  specialized  information,  thereby 
contributing  to  specialization  of  education. 

(3)  Diversification  and  Individualization 

Information  technology  diversifies  as  well  as  individualizes  education.  It  facilitates  the  collection  of 
diversified  infbrmation  on  law,  economy,  literature,  science,  medical  science,  art,  and  so  on.  It  also  enables 
learners  to  learn  selectively  and  develop  their  own  individuality. 

Exploitation  of  Information 

(4)  Effective  Exploitation  of  Information  Sources 

Each  information  source  provides  unique  and  distinctive  information.  Accessing  such  sources  enables  us  to 
collect  information  which  would  previously  have  been  very  difficult  to  obtain.  Schools  and  educational 
institutions  provide  local  cultural  information  or  specialized  information. 

(5)  Approach  to  Excellence 

A wide  range  of  excellent  information  is  available.  If  learners  are  selective  enough,  they  can  approach 
excellence. 

(6)  Elimination  of  Redundant  Activities 

At  the  same  time  that  information  technology  enables  one  to  gather  broad  information,  it  also  allows  one  to 
quickly  pinpoint  and  eliminate  duplicated  information.  Thus  concise  information  can  be  produced  even  in 
distributed,  cooperative  efforts. 

(7)  Publication  of  School  Information 

Schools  can  publicize  their  distinctive  features  worldwide  via  school  home  pages.  This  is  expected  to  be  far 
more  effective,  and  more  ecologically  advisable,  than  distributing  brochures  within  limited  areas. 

(8)  The  Ability  to  Search,  Express,  and  Transmit  Information 

The  repeated  experiences  of  collecting,  selecting,  editing,  creating,  and  transmitting  information  by  use  of 
information  technology  are  expected  to  improve  students’  abilities  to  search  for,  express,  and  transmit 
information. 

(9)  Development  of  Information  Literacy 

Through  the  rich  experiences  of  information  handling,  an  information  literacy  can  be  developed. 

Educational  Systems 

(10)  Emergence  of  Distinctive  Networked  Educational  Systems  Open  to  Everyone 

When  all  education-related  facilities  are  connected  worldwide  through  networks  and  provide  unique 
information  in  the  form  of  databases,  home  pages,  or  other  means  of  direct  transmission,  the  whole  world  will  be 
incorporated  into  a single  networked  educational  information  system.  Systems  for  admission,  graduation  or 
qualification  acquisition  will  therefore  become  more  flexible.  A society  where  people  are  free  to  maximize  their 
full  capabilities  for  learning  will  be  realized. 

Now  that  the  dissemination  of  information  and  communications  technology  enables  people  to  gather 
information  from,  and  send  information  to,  locations  all  over  the  world,  one  needs  to  judge  how  much  and  what 
type  of  information  are  really  necessary.  To  this  end,  information  senders  must  make  sure  that  their  information 
is  always  correct,  fair,  unbiased,  and  previously  unreported,  and  must  respect  the  intellectual  property  rights  of 
others.  Information  receivers,  too,  must  select  information  which  is  correct,  fair,  unbiased,  new,  and  reliable. 
And  while  ideally  an  international  rule  would  be  set  to  allow  the  transmission  of  only  desirable  information,  in 
practice  such  a restriction  is  not  feasible.  Not  only  would  such  a law  inevitably  be  broken,  but  its  very  existence 
would  raise  questions  in  regard  to  the  freedom  of  communications. 

It  is  therefore  desirable  that  a reliable  public  organization  consolidate  information  useful  for  education  and 
provide  links  to  such  information  sources.  The  services  provided  by  such  an  organization  could,  at  very  least. 
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help  keep  undesirable  information  out  of  the  hands  of  children. 

In  this  sense,  such  an  organization  could  serve  as  both  a global  broker  or  an  editor  of  excellence  and  wisdom. 
If  organizations  of  this  sort  were  established  in  many  locations  worldwide  in  order  to  moderate  the  transmission 
of  global  information,  and  if  these  organization  were  linked  together  to  work  as  a team,  a global  educational 
information  network  could  be  built  Such  a network  would  be  connected  to  education-related  home  pages  of 
governmental  bodies,  universities,  schools,  educational  facilities,  enterprises,  and  many  other  organizations 
around  the  world.  In  the  hope  of  one  day  realizing  such  a vast  educational  resource,  1 encourage  all  such 
organizations  to  develop  their  own  education-related  home  pages. 
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Abstract:  During  the  past  three  decades,  hundreds  of  research  studies  have  been 
conducted  to  investigate  interactive  learning  in  a variety  of  forms  ranging  from  the 
earliest  days  of  mainframe-based  computer-assisted  instruction  to  contemporary 
multimedia  learning  environments  accessible  via  the  World  Wide  Web.  In  light  of  this 
body  of  research,  some  researchers  believe  that  we  are  on  the  verge  of  developing  a true 
instructional  science  whereas  others  conclude  that  we  simply  cannot  pile  up 
generalizations  fast  enough  to  adapt  our  interactive  designs  to  the  myriad  variables 
inherent  in  human  learning.  In  this  paper,  I summarize  what  we  know  and  what  we  don’t 
know  about  interactive  learning,  describe  the  strengths  and  weaknesses  of  various 
approaches  to  interactive  learning  research,  and  conclude  by  proposing  a new 
developmental  research  agenda  for  the  first  decade  of  the  new  millennium. 


Introduction 

WHEN  I heard  the  learn'd  astronomer; 

When  the  proofs,  the  figures,  were  ranged  in  columns  before  me; 

When  I was  shown  the  charts  and  the  diagrams,  to  add,  divide,  and  measure  them; 

When  I,  sitting,  heard  the  astronomer,  where  he  lectured  with  much  applause 
in  the  lecture-room. 

How  soon,  unaccountable,  I became  tired  and  sick; 

Till  rising  and  gliding  out,  I wander'd  off  by  myself. 

In  the  mystical  moist  night-air,  and  from  time  to  time. 

Look'd  up  in  perfect  silence  at  the  stars. 

Walt  Whitman,  Leaves  of  Grass. 

Unlike  Walt  Whitman’s  “leam’d  astronomer,”  I offer  no  proofs,  no  figures,  and  nary  a chart  in  this  paper. 
Instead,  the  paper  primarily  consists  of  a logical  argument.  I desire  to  persuade  you  with  my  argument,  and 
although  the  phenomenon  of  interactive  learning  is  hardly  as  wondrous  as  the  stars,  I hope  to  leave  you 
with  a greater  appreciation  for  the  importance  and  complexity  of  researching  interactive  learning.  More 
specifically,  I attempt  to  outline  what  we  know  about  interactive  learning  as  well  as  what  we  don’t  know, 
and  to  suggest  some  directions  for  a renewed  research  agenda  in  the  early  years  of  the  new  millennium. 
Before  proceeding,  however,  I must  reveal  some  of  the  biases  I bring  to  this  argument  and  also  clarify  what 
I mean  by  interactive  learning. 

Almost  every  field  of  inquiry  today  is  beset  with  dichotomous  controversies.  Consider  biology  where  one 
camp  of  scientists  is  laboring  mightily  to  explain  the  nature  of  human  behavior  on  the  basis  of  genetic 
mapping  whereas  another  camp  argues  that  human  behavior  will  ultimately  be  explained  more  completely 
by  the  effects  of  nurture  and  culture.  If  research  on  interactive  learning  can  be  regarded  as  a field  (or  at 
least  a tiny  plot)  of  inquiry,  then  it  too  has  its  controversies.  One  of  the  most  obvious  is  between  those  who 
view  this  enterprise  as  a branch  of  science  or  technology  and  those  who  regard  it  as  more  akin  to  a type  of 
craft  or  even  art  (Clark  & Estes,  1998).  I must  confess  that  I have  grown  increasingly  skeptical  about  the 
science  of  designing  interactive  learning  and  more  attracted  to  the  craft  or  art  of  this  activity.  However,  my 
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skepticism  concerning  learning  sciences  and  educational  technology  does  not  preclude  a strong 
commitment  to  developmental  research  and  evaluation  as  necessary,  but  insufficient,  methods  for 
collecting  information  to  guide  the  decisions  that  must  be  made  when  designing  (crafting)  interactive 
learning  environments. 

What  is  my  definition  of  interactive  learning?  Burdened  by  a history  of  failed  technology-based 
innovations  (e.g.,  programmed  instruction,  teaching  machines,  and  computer-assisted  instruction),  the  latest 
buzzwords  for  interactive  learning  (e.g.,  interactive  multimedia,  the  World  Wide  Web  (WWW),  and  virtual 
reality)  attract  both  ebullient  enthusiasm  (Perelman,  1992)  and  serious  skepticism  (Postman,  1995). 
Ultimately,  all  learning  is  interactive  in  the  sense  that  learners  interact  with  content  to  process,  tasks  to 
accomplish,  and/or  problems  to  solve.  However,  in  this  paper,  I refer  to  a specific  meaning  of  interactive 
learning  as  involving  some  sort  of  technological  mediation  between  a teacher/designer  and  a learner.  In  my 
view,  an  interactive  learning  system  requires  an  electronic  device  equipped  with  a microprocessor  (e.g.,  a 
computer)  and  at  least  one  human  being  (a  learner).  The  adult  school  dropout  developing  basic  literacy 
skills  via  a multimedia  simulation,  the  high  school  student  surfing  the  WWW  for  archival  material  about 
indigenous  people  to  prepare  a class  presentation,  and  the  three-year  old  practicing  color-matching  skills 
with  Big  Bird  with  a Sesame  Street  CD-ROM  program  are  all  engaged  in  interactive  learning. 

What  We  Know  and  Don’t  Know 

There  are  two  major  approaches  to  using  interactive  learning  systems  and  programs  in  education, 
(Although  many  of  the  ideas  expressed  in  this  paper  may  apply  within  training  contexts,  this  paper  will  be 
limited  to  research  from  and  implications  for  education.)  First,  people  can  learn  “from”  interactive  learning 
systems  and  programs,  and  second,  they  can  learn  “with”  interactive  learning  tools.  Learning  “from” 
interactive  learning  systems  is  often  referred  to  in  terms  such  as  computer-based  instruction  or  integrated 
learning  systems  (ILS),  Learning  “with”  interactive  software  programs,  on  the  other  hand,  is  referred  to  in 
terms  such  as  cognitive  tools  and  constructivist  learning  environments. 

The  foundation  for  the  use  of  interactive  learning  systems  as  “tutors”  (the  “from”  approach)  is  "educational 
communications  theory,"  or  the  deliberate  and  intentional  act  of  communicating  content  to  students  with 
the  assumption  that  they  will  learn  something  "from"  these  communications.  The  instructional  processes 
inherent  in  the  “from”  approach  to  using  interactive  learning  systems  can  be  reduced  to  four  simple  steps: 

1)  exposing  learners  to  messages  encoded  in  media  and  delivered  via  an  interactive  technology, 

2)  assuming  that  learners  perceive  and  encode  these  messages, 

3)  requiring  a response  to  indicate  that  messages  have  been  received,  and 

4)  providing  feedback  as  to  the  adequacy  of  the  response. 

The  findings  concerning  the  impact  of  interactive  learning  systems  and  programs  can  be  summed  up  as; 

• Computers  as  tutors  have  positive  effects  on  learning  as  measured  by  standardized  achievement  tests,  are 
more  motivating  for  students,  are  accepted  by  more  teachers  than  other  technologies,  and  are  widely 
supported  by  administrators,  parents,  politicians,  and  the  public  in  general. 

• Students  are  able  to  complete  a given  set  of  educational  objectives  in  less  time  with  CBI  than  needed  in 
more  traditional  approaches. 

• Limited  research  and  evaluation  studies  indicate  that  integrated  learning  systems  (ILS)  are  effective 
forms  of  CBI  which  are  quite  likely  to  play  an  even  larger  role  in  classrooms  in  the  foreseeable  future. 

• Intelligent  tutoring  systems  have  not  had  significant  impact  on  mainstream  education  because  of 
technical  difficulties  inherent  in  building  student  models  and  facilitating  human-like  communications, 

• Overall,  the  differences  that  have  been  found  between  interactive  learning  systems  as  tutors  and  human 
teachers  have  been  modest  and  inconsistent.  It  appears  that  the  larger  value  of  these  systems  as  tutors 
rests  in  their  capacity  to  motivate  students,  increase  equity  of  access,  and  reduce  the  time  needed  to 
accomplish  a given  set  of  objectives. 

The  foundation  for  the  use  of  interactive  learning  systems  as  “cognitive  tools”  (the  “with”  approach)  is 
"cognitive  psychology,”  Computer-based  cognitive  tools  have  been  intentionally  adapted  or  developed  to 
function  as  intellectual  partners  to  enable  and  facilitate  critical  thinking  and  higher  order  learning. 
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Examples  of  cognitive  tools  include:  databases,  spreadsheets,  semantic  networks,  expert  systems, 
communications  software  such  as  teleconferencing  programs,  on-line  collaborative  knowledge  construction 
environments,  multimedia/  hypermedia  construction  software,  and  computer  programming  languages.  In 
the  cognitive  tools  approach,  interactive  tools  are  given  directly  to  learners  to  use  for  representing  and 
expressing  what  they  know  (Jonassen  & Reeves,  1996).  Learners  themselves  function  as  designers,  using 
software  programs  as  tools  for  analyzing  the  world,  accessing  and  interpreting  information,  organizing  their 
personal  knowledge,  and  representing  what  they  know  to  others. 

The  basic  principles  that  guide  the  use  of  interactive  software  programs  as  cognitive  tools  for  teaching  and 
learning  are: 

• Cognitive  tools  will  have  their  greatest  effectiveness  when  they  are  applied  within  constructivist  learning 
environments. 

• Cognitive  tools  empower  learners  to  design  their  own  representations  of  knowledge  rather  than  absorbing 
representations  preconceived  by  others. 

• Cognitive  tools  can  be  used  to  support  the  deep  reflective  thinking  that  is  necessary  for  meaningful 
learning. 

• Cognitive  tools  have  two  kinds  of  important  cognitive  effects,  those  which  are  with  the  technology  in 
terms  of  intellectual  partnerships  and  those  that  are  of  the  technology  in  terms  of  the  cognitive  residue 
that  remains  after  the  tools  are  used. 

• Cognitive  tools  enable  mindful,  challenging  learning  rather  than  the  effortless  learning  promised  but 
rarely  realized  by  other  instructional  innovations. 

• The  source  of  the  tasks  or  problems  to  which  cognitive  tools  are  applied  should  be  learners,  guided  by 
teachers  and  other  resources  in  the  learning  environment. 

• Ideally,  tasks  or  problems  for  the  application  of  cognitive  tools  will  be  situated  in  realistic  contexts  with 
results  that  are  personally  meaningful  for  learners. 

• Using  multimedia  construction  programs  as  cognitive  tools  engages  many  skills  in  learners  such  as: 
project  management  skills,  research  skills,  organization  and  representation  skills,  presentation  skills,  and 
reflection  skills. 

• Research  concerning  the  effectiveness  of  constructivist  learning  environments  such  as  micro  worlds, 
classroom-based  learning  environments,  and  virtual,  collaborative  environments  show  positive  results 
across  a wide  range  of  indicators. 

In  summary,  thirty  years  of  educational  research  indicates  that  various  interactive  technologies  are  effective 
in  education  as  phenomena  to  learn  both  “from”  and  “with.”  Historically,  the  learning  “from”  or  tutorial 
approaches  have  received  the  most  attention  and  funding,  but  the  “with”  or  cognitive  tool  approaches  are 
the  focus  of  more  interest  and  investment  than  ever  before.  Preliminary  findings  suggest  that  in  the  long 
run,  constructivist  approaches  to  applying  media  and  technology  may  have  more  potential  to  enhance 
teaching  and  learning  than  instructivist  models  (Jonassen  & Reeves,  1 996).  In  other  words,  the  real  power 
of  interactive  learning  to  improve  achievement  and  performance  may  only  be  realized  when  people  actively 
use  computers  as  cognitive  tools  rather  than  simply  interact  with  them  as  tutors  or  data  repositories. 

At  the  same  time,  there  is  a paucity  of  empirical  evidence  that  interactive  learning  technologies  are  any 
more  effective  than  other  instructional  approaches.  This  is  because  most  research  studies  confound  media 
and  methods.  Sixteen  years  ago,  Richard  E.  Clark  ignited  a debate  about  the  impact  of  media  and 
technology  on  learning  with  the  provocative  statement  that  “media  do  not  influence  learning  under  any 
conditions”  (Clark,  1983,  p.  445).  He  clarified  this  challenge  by  explaining  that  media  and  technology  are 
merely  vehicles  that  deliver  instructional  methods,  and  that  it  is  instructional  methods,  the  teaching  tasks 
and  student  activities,  that  account  for  learning.  Clark  maintained  that  as  vehicles,  interactive  technologies 
such  as  computer-based  instruction  do  not  influence  student  achievement  any  more  than  the  truck  that 
deliver  groceries  changes  our  nutrition.  Clark  (1994)  concluded  that  media  and  technology  could  be  used  to 
make  learning  more  efficient  (enable  students  to  learn  faster),  more  economical  (save  costs),  and/or  more 
equitable  (increase  access  for  those  with  special  needs). 

Robert  Kozma  challenged  Clark  in  the  debate  about  the  impact  of  media  and  technology  on  learning  by 
arguing  that  Clark’s  separation  of  media  and  methods  creates  “an  unnecessary  and  undesirable  schism 
between  the  two”  (Kozma,  1994,  p.  16).  He  recommended  that  we  move  away  from  the  questions  about 
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whether  technologies  impact  learning  to  questions  concerning  the  ways  can  we  use  the  capabilities  of 
interactive  technology  to  influence  learning  for  particular  students  with  specific  tasks  in  distinct  contexts. 
Kozma  recognized  that  although  interactive  technologies  may  be  essentially  delivery  vehicles  for 
pedagogical  dimensions,  some  vehicles  are  better  at  enabling  specific  instructional  designs  than  others. 

Both  Clark  and  Kozma  present  important  ideas.  It  is  evident  that  the  instructional  methods  students 
experience  and  the  tasks  they  perform  matter  most  in  learning.  In  addition,  1 maintain  that  the  search  for 
unique  learning  effects  from  particular  interactive  technologies  is  ultimately  futile.  After  all,  fifty  years  of 
media  and  technology  comparison  studies  have  indicated  no  significant  differences  in  most  instances. 
Whatever  differences  are  found  can  usually  be  explained  by  differences  in  instructional  design,  novelty 
effects,  or  other  factors.  However,  even  though  technologies  may  lack  unique  instructional  effects,  some 
educational  objectives  are  more  easily  achieved  with  interactive  learning  than  in  other  ways.  Revealing 
effective  implementations  of  interactive  learning  for  various  types  of  learners  and  discrete  learning 
objectives  and  content  is  an  important  goal  for  educational  researchers  and  evaluators. 

A Renewed  Research  Agenda 

The  fact  that  educational  research  is  not  highly  valued  by  educational  practitioners  is  widely  recognized.  A 
large  part  of  the  problem  can  be  attributed  to  the  fact  that  the  interests  of  academics  who  conduct  research 
and  those  of  administrators,  teachers,  students,  parents,  and  others  involved  in  the  educational  enterprise 
are  often  quite  different.  Tanner  (1998)  reminds  us  that  educational  research  should  be  focused  on  the 
mission  of  enhancing  educational  opportunities  and  outcomes: 

Unfortunately,  much  that  is  taken  for  social  research  serves  no  social  purpose  other 
than  to  embellish  reputations  in  the  citadels  of  academe  and  sometimes  to  even 
undermine  the  democratic  public  interest...  Early  in  this  century,  John  Dewey  warned 
that  educational  practices  must  be  the  source  of  the  ultimate  problems  to  be  investigated 
if  we  are  to  build  a science  of  education.  We  may  draw  from  the  behavioral  sciences,  but 
the  behavioral  sciences  do  not  define  the  educational  problems.  The  faculties  of  the 
professional  schools  draw  on  the  basic  sciences,  but  their  mandate  is  mission-oriented, 
not  disciplined  centered,  (p.  348-349) 

As  noted  in  the  previous  section,  research  reveals  that  students  learn  both  from  and  with  interactive  learning 
technology.  Computer-based  instruction  and  integrated  learning  systems  have  been  demonstrated  to  be 
effective  and  efficient  tutors,  and  there  is  considerable  evidence  that  learners  develop  critical  thinking  skills 
as  authors,  designers,  and  constructors  of  multimedia  or  as  active  participants  in  constructivist  learning 
environments.  Unfortunately,  the  level  of  our  knowledge  about  interactive  learning  is  somewhat  analogous 
to  what  health  practitioners  know  about  the  functions  of  vitamins  and  herbs  in  supporting  good  health. 
There  is  general  agreement  within  the  healthcare  professions  that  most  vitamins  and  many  herbs  have 
health  benefits,  but  there  is  considerable  disagreement  about  the  proper  dosages,  regimens,  and  protocols 
for  using  various  products.  Similarly,  in  education,  while  we  can  and  do  generally  agree  that  interactive 
learning  is  good,  we  know  veiy  little  about  the  most  effective  ways  to  implement  interactive  learning.  In 
fact,  the  need  for  long-term,  intensive  research  and  evaluation  studies  focused  on  the  mission  of  improving 
teaching  and  learning  through  interactive  learning  technology  has  never  been  greater.  Both  government  and 
commercial  interests  are  pushing  interactive  learning  in  various  forms  from  preschool  through  lifelong 
learning,  and  major  decisions  are  being  made  about  these  technologies  based  upon  habit,  intuition, 
prejudice,  marketing,  politics,  greed,  and  ignorance  rather  than  reliable  and  valid  evidence  provided  by 
research  and  evaluation. 

As  we  enter  the  new  millennium,  I maintain  that  our  research  and  evaluation  efforts  should  be  primarily 
developmental  in  nature,  i.e.,  focused  on  the  invention  and  improvement  of  creative  approaches  to 
enhancing  human  communication,  learning,  and  performance  through  the  use  of  interactive  learning 
technologies.  The  purpose  of  such  inquiry  should  be  to  improve,  not  to  prove.  Further,  developmental 
research  and  evaluation  should  not  be  limited  to  any  one  methodology.  Any  approach,  quantitative, 
qualitative,  critical,  and/or  mixed  methods,  is  legitimate  as  long  as  the  goal  is  to  enhance  education. 
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My  recommendation  to  engage  and  invest  in  developmental  research  and  evaluation  overlaps  somewhat 
with  advice  emanating  from  policy-makers  in  the  USA  where  the  Panel  on  Educational  Technology  of  the 
President’s  Committee  of  Advisors  on  Science  and  Technology  (1997)  established  three  priorities  for  future 
research: 

1.  Basic  research  in  various  learning-related  disciplines  and  fundamental  work  on 
various  educationally  relevant  technologies. 

2.  Early-stage  research  aimed  at  developing  new  forms  of  educational  software,  content, 
and  technology-enabled  pedagogy. 

3.  Empirical  studies  designed  to  determine  which  approaches  to  the  use  of  technology 
are  in  fact  most  effective,  (p.  38) 

The  second  of  these  priorities  reflects  my  call  for  development  research  issued  above.  However,  I believe 
that  the  President's  Committee  of  Advisors  on  Science  and  Technology  (1997)  has  placed  too  much  faith  in 
the  ability  of  large-scale  empirical  studies  to  identify  the  most  effective  approaches  to  using  interactive 
learning  in  schools.  In  the  final  analysis,  the  esoteric  and  complex  nature  of  human  learning  may  mean  that 
there  may  be  no  generalizable  “best”  approach  to  using  interactive  learning  technology  in  education.  The 
most  we  may  be  able  to  hope  for  is  more  creative  application  and  better  informed  practice. 

Conclusion 

Salomon  (1991)  describes  the  contrast  between  analytic  and  systemic  approaches  to  research  that 
transcends  the  "basic  versus  applied"  or  "quantitative  versus  qualitative"  arguments  that  so  often  dominate 
debates  about  the  relevancy  of  educational  research.  Salomon  concludes  that  the  analytic  and  systemic 
approaches  are  complementary,  arguing  that  "the  analytic  approach  capitalizes  on  precision  while  the 
systemic  approach  capitalizes  on  authenticity"  (p.  16).  Salomon’s  critique  remains  relevant  because  much 
of  the  research  and  evaluation  of  the  effectiveness  of  CBI  and  other  forms  of  interactive  learning  continues 
to  be  plagued  by  fundamental  flaws  that  render  much  of  this  literature  little  more  than  pseudoscience 
(Reeves,  1993). 

One  reason  for  this  deplorable  state  of  affairs  is  that  there  has  long  been  great  disagreement  about  the 
purpose  and  value  of  educational  research.  For  example,  Nate  Gage,  a past  president  of  the  American 
Educational  Research  Association  (AERA),  has  been  a staunch  defender  of  the  notion  that  the  goal  of  basic 
research  in  education  is  simply  "more  valid  and  more  positive  conclusions"  (Farley,  1982,  p.  12)  whereas 
another  past  president  of  AERA,  Robert  Ebel,  proclaimed: 

....the  value  of  basic  research  in  education  is  severely  limited,  and  here  is  the  reason. 

The  process  of  education  is  not  a natural  phenomenon  of  the  kind  that  has  sometimes 
rewarded  scientific  investigation.  It  is  not  one  of  the  givens  in  our  universe.  It  is  man- 
made, designed  to  serve  our  needs.  It  is  not  governed  by  any  natural  laws.  It  is  not  in 
need  of  research  to  find  out  how  it  works.  It  is  in  need  of  creative  invention  to  make  it 
work  better.  (Farley,  1982,  p.  18,  EbeTs  italics). 

Should  researchers  and  evaluators  seek  to  establish  immutable  laws  akin  to  those  found  in  the  harder 
sciences?  Or  should  we  be  focused  on  finding  out  how  to  improve  education  with  different  types  of 
students  in  specific  places  at  particular  times  of  their  development?  These  questions  reflect  an  on-going 
struggle  between  those  who  view  our  field  as  a science  and  those  who  regard  it  as  a craft.  The  questions 
also  reflect  the  so-called  “paradigm  wars”  among  educational  researchers.  Despite  the  increased  acceptance 
in  some  educational  circles  of  qualitative  alternatives  to  the  experimental  methods  that  have  dominated 
educational  research  in  the  past,  there  are  signs  that  some  powerful  policy-makers  are  still  pushing  for  more 
classically  empirical  approaches.  The  aforementioned  Panel  on  Educational  Technology  of  the  President’s 
Committee  of  Advisors  on  Science  and  Technology  (1997)  listed  as  one  of  its  six  major  strategic 
recommendations  that  the  government  “initiate  a major  program  of  experimental  research.... to  ensure  both 
the  efficacy  and  cost-effectiveness  of  technology  use  within  our  nation’s  schools”  (p.  5).  1 contend  that  a 
wiser  course  would  be  to  support  more  development  research  (aimed  at  making  interactive  learning  work 
better)  using  a wider  range  of  quantitative,  qualitative,  critical,  and  mixed  methods  and  less  empirical 
research  (aimed  at  determining  how  interactive  learning  works)  using  experimental  designs. 
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this  paper  when  I actually  give  the  keynote  address.  As  Michelangelo  said,  “I  am  still  learning.” 
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Abstract:  This  paper  overviews  the  programming  and  facilities  at  a large  research  University 
that  is  designed  to  assist  faculty  in  maximizing  their  effective  use  of  educational  technologies. 
Different  programs  developed  by  a university-wide  "new  media  center"  are  described  and 
critiqued.  The  paper  overviews  the  challenges  that  the  University  has  in  utilizing  "disruptive 
technologies"  and  in  developing  models  of  enhanced  teaching  and  learning  that  are  cost  and 
learning  effective. 


Introduction 

This  paper  overviews  the  experiences  and  reflections  of  one  employed  by  a major  research 
university  as  a change  agent,  tasked  with  making  sure  the  University  does  something  about  all  this 
"education  technology  and  distance  education  stuff’!  In  1994  I was  hired  as  one  faculty  member  in  a 
University  with  over  1800  full-time  tenure-track  academics  charged  to  "change  the  world"  - my  only 
weapon  a "title"  - I was  the  Alternative  Delivery  Specialist.  Fortunately,  no  one  in  1994,  nor  today,  knew 
what  an  alternative  delivery  specialist  is  supposed  to  do,  leaving  me  with  much  freedom.  In  fact,  the  only 
ones  who  seemed  to  have  a clear  conception  of  what  an  alternative  delivery  specialist  was,  were 
obstetricians  in  the  Faculty  of  Medicine  who  thought  1 represented  yet  another  brand  of  mid-wife 
attempting  to  usurp  their  monopoly  in  the  child  birth  business. 

Five  years  later,  some  progress  has  been  made  and  the  sky  has  not  fallen,  the  University  has  not 
gone  bankrupt,  has  not  been  deserted  by  students,  nor  has  it  replaced  all  the  instructors  with  teaching  bots. 
Yet  it  is  paradoxical  that,  as  organizations  based  in  information  and  knowledge  acquisition  and 
development,  universities  have  not  flourished  in  the  so-called  Age  of  Information.  In  fact  I hear  as  much 
doom  and  gloom  and  regrets  for  the  passing  of  the  good  old  days  as  1 hear  optimistic  excitement  about  the 
opportunities  available  to  us  at  the  start  of  a new  millennium. 

Why  have  information  and  communications  technologies  had  so  little  impact  on  teaching  and 
learning  within  the  mainstream  university?  I believe  that  this  is  the  case  because,  unlike  the  sustaining 
technologies  used  to  support  and  enhance  the  research  and  administrative  components  of  the  University, 
instructional  technologies  are  disruptive  to  the  current  culture  and  economics  of  higher  education. 
Therefore,  1 believe  that  divergent  strategies  must  be  employed  to  ensure  that  the  University  adopts  and 
exploits  not  only  the  sustaining  technologies  that  will  help  us  improve  on  our  present  practices,  but  also  the 
disruptive  technologies  that  will  move  us  forward  into  the  emerging  new  paradigm  of  instruction  and 
learning  in  higher  education. 

In  this  paper  1 will  trace  the  development  of  programs  and  staffing  in  a university  "new  media 
center"  (http://www.atl.ualberta.ca)  and  try  to  identify  those  factors  in  programming  that  succeed  in 
enticing  faculty  and  administration  to  make  both  adaptive  and  proactive  changes.  Finally,  I'll  suggest 
strategies  and  means  by  which  the  central  core  of  the  academic  community  can  be  induced  to  use 
information  and  technology  tools  to  protect  and  advance  the  fundamental  values  of  the  academic 
community. 

Context  and  Challenges 

Reid  (1997)  situates  in  time  the  information  and  communications  context  in  which  we  live. 
Although  universities  were  in  the  forefront  of  development  of  the  Net  in  the  "before  commercialization" 
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and  "novelty”  stages,  we  have  been  relatively  less  active  as  a presence  in  the  “utility”  stage  and  will  be 
even  less  conspicuous  in  the  “ubiquity”  phase  as  commercial  and  public  entertainment  uses  far  eclipse  our 
development  and  application  endeavors  on  the  Net.  We  are  great  inventors  and  tinkerers;  our  challenge  is  to 
leverage  this  tinkering  into  the  core  business  of  our  academy.  Greg  Baroni,  manager  of  KPMG's  global 
education  practice,  states  that  "Universities  are  sitting  at  the  gateway  of  the  knowledge  economy.  The 
question  is,  will  they  leverage  their  intellectual  capital  or  become  a vestige  of  the  past,  talking  about  the 
way  things  used  to  be?"  (from  Woody, 1998).  There  are  many  who  think  we  do  not  have  the  drive,  speed,  or 
determination  to  recreate  ourselves  for  a networked  learning  environment.  I disagree.  While  we  have  many 
barriers  to  overcome,  there  are  options  open  to  us  through  which  we  can  leverage  and  support  our 
institutions  and  our  faculty  to  overcome  these  barriers. 

The  list  of  challenges  and  threats  to  the  modem  University  is  large  and  growing  -more  students, 
widening  disparity  in  prerequisite  training  and  skills,  increased  demands  for  access  and  lifelong  learning, 
increasing  expectations  from  business,  higher  government  expectations  of  reduced  operating  costs, 
productivity  gaps  - the  list  goes  on  and  on.  It  has  become  apparent  that  the  University  cannot  be  all  things 
to  all  people.  Vest  1995  quotes  an  American  university  president's  description  of  the  modem  research 
university  as  " Overextended,  Under-focused;  Overstressed,  Under-funded  " (Vest,  in  Clark  1995.p.l46). 
These  pressures  result  in  a shortage  of  qualified  academics  willing  to  take  on  the  challenge  of  leading  our 
institutions  at  a time  when  that  leadership  is  most  needed,  a tendency  to  develop  a siege  mentality,  and  a 
search  for  a technological  quick  fix  to  solve  our  complex  problems. 

When  I talk  to  my  colleagues  on  campus  I have  learned  to  expunge  from  my  vocabulary  words 
like  market-share,  customer  and  of  course  the  profit  word.  But  I hope  you'll  allow  me  to  describe  the 
market  context  for  universities  in  Alberta,  and,  I suspect,  similar  market  contexts  in  most  of  North 
America.  In  Alberta  last  year  there  was  an  increase  of  4.8%  in  full-time  undergraduate  enrollments  - and  an 
increase  of  10.3  % part  time  enrollments  (Statistics  Canada,  Center  for  Educational  Statistics,  Nov.  10, 
1 998).  Among  the  G8  industrialized  countries,  Canada  enrolls  the  highest  percentage  of  its  citizens  in 
higher  education  We  have  obviously  moved  from  systems  designed  for  the  elite  to  mass  systems  of  higher 
education  for  life-long  learners. 

In  the  USA,  Clifford  Aelman  (1999)  develops  scenarios  based  upon  increased  numbers  of  high 
school  graduates,  increasing  numbers  of  older,  part  time  students,  and  increased  participation  by 
traditionally  disadvantaged  population  groups,  all  of  which  translate  to  an  increase  in  enrolment  of  between 
23%  and  40%  during  the  next  decade. 

Our  context,  then,  is  not  one  of  a great  threat  of  having  no  students  at  the  front  door  - quite  the 
opposite,  they  will  be  lined  up  beating  on  that  door.  Our  challenge  is,  while  helping  our  institutions  to 
increase  capacity  using  traditional  on-campus  delivery  systems,  to  get  them  to  pay  attention  to 
opportunities  to  increase  capacity  through  development  of  non-traditional  and  off  campus  programming. 
Why  worry  about  students  struggling  to  get  in  at  the  back  door,  when  there  are  line  ups  at  the  front  door? 

A critical  accomplishment  of  my  first  year  at  the  University  was  the  initial  funding  to  establish  a 
multi-media  production  studio.  This  facility  was  designed  to  assist  faculty  in  developing  innovative, 
technology  based  enhancements  to  their  teaching,  (see  http://www.atl.ualbcrta.ca).  It  soon  became  apparent 
that  we  needed  to  develop  a set  of  pedagogical,  economic  and  technical  models  to  guide  the  development 
of  educational  content  and  professional  development  programming  in  our  unit.. 


Models  of  Courseware  Development 

There  are  at  least  3 theories  of  educational  software  production.  The  first  I’ll  call  the  "drop  off  your 
ideas  and  come  back  in  a few  months  and  pick  up  the  CD"  model.  In  this  model  academics  are  used  as 
subject  matter  experts  and  the  school  or  business  lines  up  a team  of  experts  including  instructional 
designers,  graphic  artists,  video  producers  and  programmers.  This  is  a model  that  commercial  software 
houses  use  and  with  very  few  exceptions  have  been  losing  money  at  for  the  past  5 years.  The  production 
quality  is  generally  excellent,  some  have  even  paid  attention  to  pedagogical  principles  and  to  interface 
design.  What  the  model  fails  to  address  is  the  customer  - in  this  case  the  busy  faculty  member  with  the 
power  to  assign  the  purchase  of  the  product  to  hundreds  of  undergraduates.  This  model  strives  to  create 
such  superior  product  that  faculty  will  be  compelled  to  assign  it  as  compulsory  tool  in  every  class.  But  it 
doesn’t  address  the  issues  of  trailability,  observability,  and  compatibility  that  are  critical  to  adoption  of  any 
innovation  (Rogers,  1995).  How  are  professors  to  use  a $80.00  CD  when  they  already  have  a good  text 
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book,  their  lab  or  seminar  topics  all  mapped  out,  and  a niggling  fear  that  if  they  do  assign  the  CD  only  a 
handful  of  students  will  actually  purchase  it  and  worse  maybe  they  will  decide  not  to  come  to  the  lectures. 

The  second  model  was  labeled  by  my  friend  Tony  Bates  from  the  University  of  British  Columbia 
as  the  ''Lone  Ranger  and  Tonto"  model.  In  this  model  a single  faculty  member,  with  a trusty  graduate 
student,  embarks  solo  on  the  development  task.  Single  handedly,  they  master  not  only  the  content  and 
instructional  design  of  the  discipline,  but  also  the  complete  production  process,  including  sales,  marketing, 
finance  and  eventual  retirement  to  a condo  in  Maui.  We’ve  played  with  this  model  and  had  some  limited 
success.  One  faculty  member  developed  a CD  of  rotateable,  high  resolution  images  of  his  mineral 
collection,  which  allows  students  to  zoom  in  examine  a host  of  minerals  without  having  to  touch  or  even 
breath  on  his  lifelong  rock  collection.  Unfortunately,  he  hasn’t  got  the  cash  for  the  condo,  yet  as  every 
potential  publisher  I introduce  to  him,  walks  away  with  at  least  an  order  of  magnitude  difference  in 
assessment  of  the  appropriate  amount  of  royalty  payments  that  should  flow  to  the  professor.  Another 
relative  success  (morally  if  not  economically)  is  the  Healthy  Student  Shareware  collection  that  we’ve 
developed.  This  is  a serious  of  seven  Authorware  based  tutorials  on  issues  such  as  managing  Stress, 
Nutrition,  Booze  and  sexually  transmitted  diseases.  We've  installed  these  programs  in  kiosks  around 
campus  and  given  them  away  via  FTP  all  across  the  world.  This  model  can  work,  but  requires  a 
commitment,  skill  set  and  entrepreneurial  bent  that  is  only  rarely  found  among  academics. 

The  third  model  I’ll  call  the  "teach  'em  to  fish  model"  in  which  instructional  and  technical  experts 
guide  and  assist  faculty  members  with  the  special  skills  needed  and  through  a scaffolding  and  training 
model,  help  faculty  to  create  and  maintain  their  own  teaching  resources.  We  implement  this  model  through 
an  annual  competition  for  seats  in  our  production  studio.  We  pay  for  full  course  release  for  a faculty 
members  and  ’’immerse”  them  in  the  collaborative  production  of  HTML,  instructional  design,  www  sites, 
Authorware  tutorials,  problem  based  learning  scenarios  and  other  innovative  instnictional  approaches.  We 
continue  this  program  today  because  we  see  this  as  a necessary,  long-term  investment  in  our  faculty. 

Besides  these  differing  models  of  content  development,  we’ve  also  developed  various  professional 
development  programs  during  the  past  5 years  which  I will  classify  according  to  those  that  have  worked, 
those  that  have  not  and  those  we're  not  sure  about. 


Initiatives  That  Work 

Partners  Program  This  is  our  full  teaching  release,  "come  live  in  the  production  center"  model  of 
faculty  development.  The  opportunity  to  work  along  side  skilled  peers  and  professional  developers  for  an 
extended  period  of  time  is  one  of  the  few  ways  that  we  have  seen  truly  transformative  change  in  approaches 
to  teaching  and  learning  among  our  faculty.  Although  we  realize  that  the  long  term  answer  to  faculty 
empowerment,  lies  in  the  use  and  development  of  these  tools,  not  in  central  development  centers,  but  in  the 
faculties,  we  also  realize  the  benefits  of  immersion  in  a community  of  developers  and  the  older  tradition  of 
providing  a sanctuary  for  academics  to  develop  their  best  work. 

Training,  Infrastructure  and  Empowerment  System  (TIES):  Mike  Szabo  has  led  a two  year  project 
at  our  university  that  empowers  task  forces  at  the  department  level  to  create  a vision  and  skillfully  chart  a 
plan  of  action  for  their  faculty,  while  engaged  in  a learning  process  focussed  not  only  on  technical  skill 
development,  but  also  on  the  critical  skills  of  change  management 

(see  http://www.quasar.ualberta.ca/DRMIKE/ties.html).  Since  the  real  business  of  the  University  -teaching 
and  research  happens  at  the  departmental  level,  it  is  here  that  real  change  and  innovation  must  be  embraced. 
If  the  department  based  units  ignore  or  oppose  innovative  developments  then  life  at  the  University  proceeds 
un-affected.  "For  change  to  take  hold,  one  department  and  faculty  after  another  needs  itself  to  become  an 
entrepreneurial  unit,  reaching  more  strongly  to  the  outside  with  new  programs  and  relationships  and 
promoting  third-stream  incomes”  (Clark,  1 997 .p. 7) 

Decentralized,  Discipline  Related  Support  Centers:  The  support  of  faculty  based  development 
and  support  centers  is  a challenging  task  for  the  director  of  a centrally  funded  unit.  However,  1 think  the 
change  process  is  so  large,  so  complicated  and  in  many  ways  specific  to  the  context  of  the  discipline  that  it 
is  only  when  individual  departments  and  faculties  grapple  with  the  issues  themselves  that  real  change 
happens.  Our  challenge  is  developing  a central  unit  that  coordinates,  trains,  and  communicates  amongst 
these  units  and  maintains  very  specialized  tools  and  expertise  that  can  be  used  cost  effectively  to  support 
these  distributed  development  centers. 
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Student  Guides  And  Tech  Helpers:  I don’t  think  we  have  been  as  successful  as  we  could  have  been 
in  employing  students  as  technical  assistants,  but  I agree  with  Steve  Gilbert  (Gilbert,  1997)  who  writes  that 
one  of  the  few  economical  solutions  to  the  "crisis  of  support"  in  technology  assisted  higher  learning  is  in 
mobilizing  our  student  body  to  provide  assistance  not  only  to  other  students  but  to  faculty  as  well. 

Pushed  Mailing  lists:  The  single  most  effective  tool  we  have  developed  to  engage  and  inform 
faculty  has  been  our  unmoderated  Majordomo  discussion  list.  We  now  have  over  500  faculty  on  ATLNet, 
providing  a forum  for  not  only  announcements  but  also  for  healthy  debate  and  discussion.  I've  learned 
many  lessons  through  management  of  this  list  including  the  need  to  limit  membership  to  our  faculty,  to 
archive  messages  in  WWW  format  (http://www.ualberta.ca/htbin/lwgate/atlnet/archives/)  and  unobtrusively 
regulate  the  traffic  so  as  to  reduce  superfluous  messages. 


Things  That  Didn’t  Work  Very  Well 

High  Cost,  Full  Scale,  Commercial  Multimedia  Content  Development:  We  are  in  the  midst  of  a 
very  large  scale  multimedia  development  that  is  excellent  technical  and  pedagogical  product.  However,  I 
find  it  difficult  to  rationalize  the  inordinate  amount  of  support  that  is  required  to  sustain  these 
developments.  We  have  not  been  as  successful  as  I would  have  liked  in  attracting  support  of  major 
publishing  houses.  Despite  Bill  Gates  assertion  that  "content  is  king"  the  core  business  of  the  University  is 
in  creating  the  context  for  learning,  not  in  sustaining  high  risk,  large  scale  content  development.  There  may 
be  "big  money"  in  this  type  of  development  in  the  future,  but  it  is  too  much  like  buying  lottery  tickets  as  a 
retirement  planning  scheme  for  my  liking. 

Lone  Ranger  Production  Of  Multi-Media  Content:  As  I mentioned,  we  have  had  some  pretty 
spectacular  results  from  talented  lone  rangers  (for  example  David  Miall’s  excellent  CD-ROM  on 
Romanticism  http://www.ualberta.ca/'-dmiall/ROMCDINF.HTM)  but  this  is  model  that  only  supports  the 
early  adopters,  and  we  are  after  much  more  profound  change  within  the  academy  then  can  be  engineered 
through  the  heroic  efforts  of  individual  faculty  members. 

On-Going  Professional  Development  Computer  Conferences:  I like  computer  conferencing  for 
courses  and  short  term  virtual  conferences,  in  fact  I lay  modest  claim  to  having  organized  the  first 
international  virtual  conference  organized  on  the  Internet  in  1992  (Anderson  and  Mason,  1994).  However, 
the  medium  is  inherently  a pull  technology,  that  relies  on  busy  academics  to  make  a special  effort  to  adapt 
to  and  frequently  logon  to  another  environment.  Our  conferencing  system,  is  full  of  conferences  that 
petered  out  days  or  weeks  after  the  enthusiastic  startup  by  our  training  or  development  groups. 

Production  Of  Print  Based  "Training  Manuals'' : We  joined  in  a consortium  to  produce  a series  of 
27  training  manuals  for  faculty  development 

http://www.gmcc.ab.ca/users/imd/etpdp/sample/programmap.htm.  The  quality  was  high,  the  format 
professional  - but  our  faculty  just  wasn’t  interested.  We're  now  converting  them  to  WWW  however  1 still 
don’t  expect  faculty  to  systematically  go  through  the  materials.  At  least  on  the  WWW  they  will  be  available 
"just  in  time". 

Individual  Faculty  Member  Based  Distance  Education  Initiatives  I know  this  will  sound  very 
naive  to  distance  educators,  but  in  our  desire  to  support  the  innovation  of  individual  faculty,  we  have 
sometimes  funded  development  of  projects  that  died  instantly  when  the  individual  faculty  member  got 
tired,  retired,  relocated  or  died.  Education,  and  especially  that  designed  for  distance  delivery  requires  a 
support  system,  starting  at  the  chair  and  dean  and  moving  right  through  to  the  registrar  and  all  the  way  to 
the  janitorial  staff. 

Forced  Change:  Ever  tried  to  herd  3 cats  - how  about  1 800?  Enough  said. 


Ones  I’m  Not  Sure  About 

Large  Volume,  Scattergun  Staff  Training  Programs  During  the  past  term  our  staff  hosted  a 
training  or  professional  development  event  every  day  (see  http://www.atl.ualberta.ca/training/ ) Some  were 
well  attended,  some  weren't.  We  succeeded  in  burning  out  some  staff  members,  attracting  many  of  the  same 
faces  day  after  day,  but  made  progress  towards  eliminating  the  two  of  the  top  eight  barriers  to  adoption 
identified  by  faculty  in  our  campus  needs  assessment  survey  (Anderson,  Vamhagen  and  Campbell,  1998)  - 
those  being  related  to  lack  of  knowledge  and  exposure  to  new  instructional  technologies. 
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WWW  Contests  We  have  run  contests  for  the  best  web  supported  courses  in  a variety  of  categories 
during  the  past  two  years  (see  http://www.atl.ualberta.ca/cyc/home.cfm).  We  had  over  70  entrants  this  year 
and  gave  away  $8,000  in  prizes,  but  1 think  we  intimidated  the  majority  faculty  leaving  the  spoils  of  victory 
exclusively  to  the  early  adopters  - no  longer  our  major  target  group. 

Strategic  Planning  And  Long  Term  Technology  Integration  Plans  I've  spent  far  too  many 
meetings  this  past  year  working  on  university  and  departmental  Technology  Integration  Plans.  1 find  it  very 
difficult  to  develop  plans  that  do  not  result  in  requests  to  the  president  for  funding  an  order  of  magnitude  or 
two  above  the  capacity  of  the  institution  - thus  creating  documents  with  high  potential  for  dust  gathering. 
However,  1 remain  convinced  that  planning  that  supports  and  allows  for  innovation  is  of  critical  importance 
to  the  evolving  University. 

I've  tried  to  outline  the  challenges  and  accomplishments  and  failures  of  the  past  5 years  at  our 
University  and  hope  that  you  can  draw  some  reflective  parallels  to  your  own  context.  1 invite  you  to  check 
out  the  URLs  above  to  investigate  any  of  these  innovations  in  greater  detail.  I'd  like  to  turn  now  to  a bit 
more  philosophical  analysis  of  the  change  process. 


The  Teaching  Professional 

Many  of  you  have  probably  read  the  article  Digital  Diploma  Mills  by  David  Noble  (another  powerful 
writer  and  Canadian  scholar  and  academic  most  definitely,  not  in  the  tradition  of  Marshall  McLuhan). 
Noble  and  others  neo-Luddites  lament  the  "commodification  of  knowledge"  the  displacement  of  control 
and  ownership  of  intellectual  property  coupled  with  a supposed  de-professionalism  of  academics  as 
teachers.  I'll  return  to  the  issue  of  economics  later,  but  let  me  comment  on  academics  as  professional 
teachers.  I’m  not  sure  who  first  coined  the  axiom  that  "any  teacher  who  can  be  replaced  by  a machine 
should  be"  but  let  me  tell  you  that  such  cliches  do  not  go  over  well  in  our  Faculty  Club.  Many  see  machines 
as  de-skilling  professors  and  as  I mentioned  leading  to  the  de-professionalisation  of  academics.  What  does 
it  mean  to  be  a professional?  Applebaum  & Lawton  (1990)  define  a profession  as  having  the  following 
characteristics: 

♦ Organized  to  serve  a specialized  body  of  knowledge  in  the  interests  of  society 

♦ A set  of  skills  proficiencies,  techniques  and  competencies 

♦ Standards  of  excellence,  self  regulation,  training  of  new  members 

♦ Established  means  of  professional  communications  (journals  and  meetings) 

Are  professors  at  institutions  of  higher  education  professional  teachers?  Do  professors  at  research 
universities  maintain  a "specialized  body  of  knowledge  in  the  interests  of  society"  Of  course  they  do,  but  it 
is  the  knowledge  of  a discipline,  of  research,  of  knowledge  creation,  it  is  not  a specialized  knowledge  of 
teaching  and  learning.  Do  we  have  standards  of  excellence,  training  of  new  members,  established  means  of 
professional  communications?  Again  certainly  within  the  discipline,  but  not  with  the  profession  of 
teaching. 

1 challenge  you  to  undertake  a piece  of  research  when  you  get  back  to  your  campus.  Go  to  your 
library  catalog  and  do  a search  within  any  of  the  subject  matter  domains.  Search  for  those  periodicals, 
journals,  meetings  notes,  training  manuals  or  any  other  documentation  supporting  the  development  and 
growth,  not  of  research  within  the  discipline,  but  of  research  relating  to  the  teaching  of  that  discipline. 

Let  me  tell  you  what  I found  from  researching  our  own  Faculty  of  Engineering,  which  is  arguably 
one  of  Canada’s  finest  faculties  - based  upon  reputation,  awards  and  success  of  graduates.  This  faculty  has 
over  190  tenure  or  tenure  track  academics  and  over  4,000  students  at  undergraduate  and  graduate  levels.  1 
looked  up  how  many  Journals  or  periodicals  we  subscribed  to  that  focused  on  the  science,  specialized 
knowledge,  self  regulation  or  training  of  faculty  to  the  profession  of  engineering  teachers.  1 found  that 
there  is  a listing  for  Engineering  Education  the  Journal  of  the  American  Society  for  Engineering 
Education.  We  have  some  volumes  going  back  to  1898  in  our  holdings,  but  we  dropped  our  subscription  in 
1991!  We  also  have  a few  old  volumes  of  the  European  Journal  of  Engineering  Education  but  dropped  the 
subscription  in  1991  as  well.  Let's  put  this  in  perspective.  Our  same  library  catalog  lists  207  periodical 
publications  under  chemical  engineering  and  even  24  periodical  titles  under  "low-temperature  engineering". 
1 realize  that  living  in  Northern  Canada  might  justify  a special  interest  in  low-temperature  engineering.  But 
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we  subscribe  to  24  periodicals  on  low-temperature  engineering  and  not  a single  active  subscription  to  the 
core  business  of  the  Engineering  faculty  - that  of  teaching  and  promoting  learning  - hardly  professional. 

What  do  1 conclude?  There  is  absolutely  no  danger,  as  Noble  and  others  claim,  that  technology 
will  deprofessionalize  our  teaching  faculty.  The  fact  is  they  are  not  professional  teachers  and  probably 
never  have  been-  professional  researchers,  scientists  yes  - but  teachers  no. 

So  what  you  might  ask?  It  is  interesting  that  Phoenix  University,  the  fastest  growing  and  1 believe 
first  University  to  be  publicly  traded  on  the  Nasdec  exchange,  with  1997  profits  to  share  holders  of  $33 
million,  boasts  that  none  of  their  faculty  (99.4%  of  whom  are  part  time  adjunct)  are  professional 
researchers  - they  are  trained,  guided  and  evaluated  as  professional  teachers  (Winston,  1999,  p.  13).  We 
boast  that  over  90%  of  our  teaching  are  full  time  academics  and  researchers.  Somewhat  of  a disconnect,  I 
would  say.  Many  of  my  colleagues  argue  that  a research  university  doesn't  need  to  worry  about  this,  in  that 
we  focus  on  research,  which  defines  our  market  niche.  We  are  finding  now  in  international  education  that 
we  are  competing  with  real  professional  teachers  and  researchers  and  I fear  that  our  ability  to  compete  will 
be  compromised  unless  we  find  ways  of  rewarding  faculty,  within  the  research  university,  for  expending 
the  necessary  energy  and  time  to  development  excellence  as  professional  teachers.  Murry  Turoff,  one  of  the 
earliest  developers  of  on-line  courses,  makes  the  case  that  ""Competition  in  education  on  an  international 
and  national  basis  will  become  the  principle  determinant  of  the  success  or  failure  of  institutions  in  the  next 
decade"  (Turoff,  1999  p.30) 

Why  is  it  so  difficult  for  us  to  embrace  the  technologies  that  are  specifically  designed  to  aid  the 
core  business  of  our  Universities  in  teaching  and  learning?  A key  to  understanding  this  is  to  differentiate 
these  technologies  between  those  that  are  sustaining  and  those  that  are  disruptive  of  the  social  context  in 
which  our  institutions  were  conceived  and  have  evolved. 


DISRUPTIVE  AND  SUSTAINING  TECHNOLOGIES  from  (Archer,  Garrison  and 
Anderson,  in  press) 

Universities  currently  enjoy  a dominant  position  in  the  post-secondary  education  “industry.” 
However,  this  “industry”  is  entering  a period  of  rapid  technological  change  - the  sort  of  period  in  which  the 
leading  firms  in  an  industry  may  rather  suddenly  be  eclipsed  by  new  players.  In  other  words,  the  next  few 
years  could  see  a sudden  change  for  the  worse  in  universities’  position  in  an  educational  marketplace  being 
transformed  by  new  technologies. 

According  to  Christensen  (1997),  the  main  reason  that  successful  and  apparently  well  run 
organizations  can  and  do  flounder  is  that  they  fail  to  recognize  the  distinction  between  sustaining 
technologies  and  disruptive  technologies.  Sustaining  technologies  are  those  that  improve  the  performance 
of  established  products.  They  are  often  developed  by  successful  companies,  the  leaders  in  their  fields,  for 
and  in  close  collaboration  with  their  most  important  and  lucrative  clients.  In  other  words,  failures  are  often 
the  result  of  successful  firms’  following  the  commendable  business  practice  of  listening  closely  to  their 
customers. 

In  contrast  to  sustaining  technologies,  which  improve  the  performance  of  established  products, 
dismptive  technologies  often  result  in  worse  product  performance  in  the  mainstream  market^  at  least  in  the 
short  run.  But  they  have  other  features  that  fringe  (and  generally  new)  customers  value.  Products  based  on 
disruptive  technologies  are  typically  cheaper,  simpler,  smaller,  and,  frequently,  more  convenient  to  use.” 
Christensen,  (p.  xv)  Generally  disruptive  technology  based  delivery  systems  are  used  to  address  new  types 
or  sectors  of  learners.  Often  professional  development  groups,  or  students  not  generally  in  a position  to  take 
advantage  of  residence  based  courses. 

The  innovator’s  task  is  to  ensure  that  this  innovation  - the  disruptive  technology  that  doesn’t  make 
sense  - is  taken  seriously  within  the  company  without  putting  at  risk  the  needs  of  present  customers  who 
provide  profit  and  growth,  (p.  xxiv)  This  requires  that  the  development  of  niche  markets  that  are  protected 
from  the  demands  and  constraints  of  the  main  business  activity  of  the  day.  It  also  requires  a research  and 
development  capacity  within  the  organization  that  can  test,  fail  and  recover  very  quickly  from 
experimentation  with  the  disruptive  technology  in  small  scale,  but  exemplar 

So  how  should  one  direct  the  development  focus  and  energy  of  a modem  research  university?  In 
an  interesting  paper,  Dan  Surry  (1997),  differentiates  between  developers  with  a determinist  philosophy 
related  to  technology  and  those  who  view  development  from  an  instrumentalist  lens  (Table  1 .).  Determin- 
istic developers  are  convinced  that  technological  change  is  inevitable  and  driven  by  the  superior  product 
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and  process  that  results  in  education  from  the  use  of  new  technologies.  They  focus  on  the  product  itself, 
seeking  the  "killer  ap"  that  either  increases  speed,  effectiveness  or  efficiency  of  the  teaching/leaming 
process.  They  see  institutions  as  largely  disorganized  and  ill-structured  obstacles  to  the  types  of  supportive 
environments  necessary  for  faculty  to  develop  and  use  technologies  effectively.  They  seek  to  minimize  the 
deterious  effects  of  the  context  so  as  to  allow  the  inherent  technological  superiority  of  the  product  to 
manifest  itself 


GOAL 


PHILOSOPHY 

Systematic  Change 

Product  Utilization 

(Macro) 

(Micro) 

Developer 

(Determinist) 

Focus  on  the  structure  and  establishment 
of  a effective  organizational  framework 

Focus  on  process  of 
developing,  and  evaluating 
instnictional  products. 

designing 

effective 

Adopter 

(Instrumentalist) 

Focus  on  the  social,  political  and 
professional  environment  in  specific 
organizations 

Focus  on  the  needs  and  opinions  of 
potential  adopters  and  characteristics  of 
the  adoption  site. 

Table  1.  Overview  of  Instructional  Technology  Diffusion  Theories  showing  diffusion  goal  and 
philosophical  view.  From  Surry  (1997) 

Instrumentalist  developers  on  the  other  hand  are  philosophically  focussed  not  on  the  technology, 
but  rather  on  an  analysis  of  the  needs,  opinions,  characteristics  of  the  actors  within  the  adoption  site.  To 
instrumentalist  agents,  the  power,  advantage  and  efficacy  of  the  tool  is  of  much  less  importance  than  the 
match  between  the  tool  and  the  context  within  which  it  must  operate.  These  developers  realize  that  there 
are  many  innovations  that  will  not  and  cannot  succeed  within  the  context  because  of  the  inherent  conflict 
between  attributes  of  the  innovation  and  the  environment  in  which  they  must  operate.  The  long  history  of 
educational  technology  that  have  not  been  adopted  convinces  me  that  an  instrumentalist  approach  is  the  one 
most  likely  to  lead  to  systematic  change. 

Where  we  have  failed  to  date  in  our  attempt  to  develop  a successful  dynamist  culture,  within  the 
academy,  is  not  in  constraining  the  freedom  of  faculty  members  to  set  their  own  teaching  agendas,  nor  is  it 
in  actively  discouraging  innovation  and  adoption.  Where  we  fail  is  in  developing  a culture  that  supports  a 
ruthless  evolutionary  cultural  of  survivalism.  Effective  innovations  are  allowed  to  grow,  but  so  are  the 
ineffective.  We  have  no  "bottom  line"  to  easily  measure  our  quarterly  performance  and  we  steadfastly 
refuse  to  put  our  best  minds  towards  the  task  of  developing  means  and  measures  to  evaluate  the  impact  of 
our  innovation  on  teaching  and  learning.  This  is  a problem  for  both  traditional  and  technology  enhanced 
instruction.  David  Noble  (1998)  asks  "if  students  are  taking  courses  which  are  just  experiments,  and  hence 
of  unproven  pedagogical  value,  should  students  be  paying  full  tuition  for  them"?  I would  ask  with  equal 
stridency,  ''if  students  are  taking  course  to  which  no  empirical  evaluation  in  regards  to  teaching  or  learning 
efficacy,  nor  any  corporate  or  individual  effort  is  expanded  on  research,  development  or  enhancement  of 
pedagogy,  relevance  or  applicability  has  been  undertaken  for  the  past  century,  should  they  be  paying  full 
tuition  either?  Our  challenge  is  to  enhance  our  evaluation  models  and  plans  with  every  project  so  that  if  we 
fail  - we  do  so  cheaply  and  quickly.  If  we  succeed,  we  can  document  the  context  and  actions  that 
precipitated  that  success. 


More  With  More  Model 

All  of  this  is  background  to  my  concluding  remarks  about  how  technology  fits  into  the  mix.  The 
way  it  does  not  fit  seems  to  be  the  way  technology  is  advocated  on  many  of  our  campuses  - 1 call  it  the 
"More  With  More  Model".  It  generally  works  as  follows.  Early  adopters,  visionaries,  and  missionaries  like 
myself,  plead  with  central  administrators  to  give  us  more  money,  more  technology,  more  teaching  release 
time,  more  professional  development,  more  student  assistants,  more  technicians,  more  ’smart  classrooms’ 
and  more  tools  and  in  return  we  will  promise  to  deliver  more  high  quality  and  more  accessible  education. 

The  University  of  Alberta  currently  spends  over  $ 1 million  a year  on  our  30  public  computer  labs 
on  campus  and  is  still  not  meeting  the  demand  for  ubiquitous  student  access.  We  have  a proposal  before  us 
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to  increase  the  number  of  smart  classrooms  on  campus  from  20  to  70  and  keep  them  "evergreen"  at  a cost 
of  $2million  per  year.  However,  70  smart  classrooms  on  our  campus  will  hardly  meet  the  ever  increasing 
desire  of  our  teaching  staff  to  use  even  simple  tools  like  Powerpoint  presentations.  Clearly,  the  current 
funding  structure  and  model  of  teaching  will  not  sustain  a strategy  based  upon  "more  with  more". 

Polley  McClure  from  the  University  of  Virginnia  published  a simple  formula  to  help  us  connect 
these  variables.  She  notes  that  Productivity  is  a product  of  the  amount  of  Learning  and  Access  to  the 
instruction,  divided  by  the  Cost  to  provide  this  education.  P =(LXA)/C.  The  More  with  More  model 
addresses  only  the  Learning  and  increases  the  Cost.  But  in  the  end  P,  the  productivity  is  constant  or  more 
likely  to  go  down,  unless  we  can  very  clearly  demonstrate  enhanced  learning  outcomes  due  to  the  use  of 
technology.  1 believe  that  we  will  see  very  significant  increases  in  learning,  but  they  will  be  slow  in 
arriving.  Further,  Thomas  Russel  shows  us  with  the  No  Significant  Difference  site, 
http://teleeducation.nb.ca/nosignificantdifference/  that  there  are  no  guarantees  that  the  use  of  any  mediated 
form  of  delivery  will,  by  itself,  increase  learning.  The  distributed  nature  of  much  networked  based  learning, 
begins  to  provide  a small  light  at  the  end  of  the  tunnel.  Note  that  in  the  formula  access  is  multiplied  by 
learning  - thus  the  effort  in  enhancing  technology  through  distribution  of  our  programming  to  new 
audiences  through  distributed  education  and  by  providing  time  shifted  alternatives  for  on-campus  students 
provide  a means  to  improve  upon  the  "more  with  more"  model.  To  achieve  the  goal  of  increasing  learning 
opportunities  while  containing  costs,  we  have  to  be  able  to  change  major  components  of  our  teaching 
programs.  We  need  to  be  able  to  develop  appropriate  combinations  of  independent  study  formats, 
industrialized  learning  systems  and  intense  personal  learning  communities  using  both  face-to-face  and 
mediated  instructional  environments.  The  correct  combination  considers  student  learning  and  faculty 
teaching  needs,  but  also  is  formulated  with  the  costs  and  benefits  of  each  adaptation  are  calculated  as 
precisely  as  possible. 


Conclusion 

Universities  cannot  and  should  not  extradite  themselves  from  the  rich  political  and  social  history 
nor  from  the  rich  cultural  heritage  of  scholarship  that  we  carry  forward  through  the  centuries.  But  neither 
have  we  the  right,  nor  the  means,  to  freeze  the  evolution  of  our  institutions  of  higher  learning.  We  are 
called  as  scientists,  thinkers  and  teachers  to  empirically  and  philosophically  test  our  assumptions  and 
practice  against  the  goals  so  nobly  enshrined  in  our  Latin  mottoes  and  celebrated  in  the  eye  of  our  students 
and  supporters. 

We  need  to  challenge  our  colleagues  not  in  the  techno-Babel  of  machine  features  or  attributes  that 
are  outside  of  their  perceived  need  or  vocabulary.  Rather  we  must  challenge  our  colleagues  and 
administration  in  the  language  of  scholarship  - of  testing  hypothesis,  critically  evaluating  claims, 
substantiating  our  hunches  and  debating  not  only  the  results  but  also  the  costs  of  our  innovations.  We  also 
need  to  seriously  examine  the  values  and  commitments  that  should  be  driving  us  in  a relentless  search  for 
better  ways  to  teach  and  learn.  These  are  lofty  goals.  Goals  that  will  ensure  the  survival  of  the  academy  in 
a form  that  embraces  any  tool  that  provides  long  term  assistance  in  the  task  of  creating  human  beings  wise 
enough  to  warrant  the  stewardship  of  this  planet. 

Posterel  (1998)  celebrates  the  freedom,  the  invigoration  and  motivation  in  play.  And  I'll  conclude 
with  a call  for  celebration  of  the  opportunity  provided  to  us  to  play  in  the  emerging  field  of  educational 
technology.  My  time  as  a change  agent  has  been  marked  by  a few  successes,  a number  of  things  that  might 
have  happened,  but  didn't,  and  a growing  list  of  things  1 really  do  want  to  get  at.  We  are  challenged  by  the 
opportunity  to  play  in  this  field  that  is  redefining  the  university  but  also  the  lives  of  each  of  us  lucky 
enough  to  be  actors  on  this  stage.  We  are  each  entrusted  with  the  honor  and  responsibility  of  carrying 
forward  the  traditions  of  scholarship,  inquiry,  reflection  and  teaching  into  the  next  century. 
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Abstract:  Which  official  initiatives  and  which  efforts  are  made  regarding  the  implementation  of 
ICT,  including  the  use  of  telecommunication  in  the  Danish  school  system?  Taking  the  starting- 
point  in  different  pedagogical  scenarios  we’ll  describe  an  approach  to  implementing  ICT  - 
specifically  teleprojects  carried  out  since  1987  within  the  framework  of  European  Schools  Project. 
Which  educational  materials  have  been  produced  to  meet  the  need  for  and  the  demands  from 
teachers  entering  transnational  projects?  How  do  we  at  RDSES  meet  the  challenges  in  the  use  of 
ICT  in  in-service  training,  whether  that  be  ordinary  or  distance  learning  courses?  How  do  we 
collaborate  between  the  European  countries?  Which  initiatives  are  taken  by  the  EU  as  far  as  the  use 
of  ICT  and  telecommunication  in  education? 


Introduction 

The  authors  work  as  senior  consultants  at  RDSES  1).  RDSES,  which  provides  further  education  for  teachers  of  the 
Folkeskole  2)  and  for  lecturers  at  colleges  of  education,  can  trace  its  historical  roots  back  to  1856.  The  graduate 
studies  form  part  and  parcel  of  the  provision  of  further  education  for  teachers.  Alongside  their  teaching,  the  staff  at 
the  eight  faculties  and  several  centres  - of  which:  'Research  Centre  for  Education  and  ICT'  3)  is  one  - carry  out 
research.  In  this  way  an  essential  part  of  the  basis  of  the  teaching  in  courses  and  graduate  studies  is  brought  into 
being.  An  increasing  number  of  the  research  projects  are  carried  out  in  co-operation  with  researchers  abroad,  many 
supported  by  e.g.  the  Nordic  Council,  EU,  OECD. 


Which  challenges  and  barriers  do  teachers  meet  when  they  want  to  use 
telecommunication? 

At  a ’Council  of  Europe’  Conference  in  Stockholm  in  1993  colleagues  from  different  European  countries  addressed 
us  and  the  audience  by  giving  brief  accounts  of  the  situation  concerning  telecommunication  in  their  respective 
countries.  Anne  Villems  from  Estonia  described  her  view  as  to  which  problems  or  barriers  the  teachers  had  to 
overcome: 

W. 

Here  is  Anne  Villems  from  Tartu,  Estonia,  This  term  we  started  with  our  own  projects.  As  practice  has  shown  it's 
extremely  difficult  for  teachers:  - to  do  too  many  things  at  a time,  - to  start  participating  in  an  international  project, 
- to  start  to  use  computer  and  communication,  - to  connect  projects  with  excisting  curriculum,  - to  integrate 
activities  with  other  subjects,  - to  use  foreign  languages,  etc '. 

The  observations  made  very  well  expresses  how  teachers  then  and  today  are  called  to  respond  to  new  pedagogical, 
scientific  and  technological  developments. 
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In  a ’Communication  Document’  from  a conference  ’ School  Networks  and  Teachers  Training’,  Athens,  September 
1998  6),  the  participants  conclude:  ’The  rapid  evolution  of  new  technologies  in  combination  with  the  continuous 
societal  and  economic  changes  pose  the  need  for  on-going  teachers’  prof essionlisation  and  updating’.  Conclusions 
similar  to  ours. 


ICT  in  the  Education  System 

Action  Plan  for  1998-2003  - The  Danish  Ministry  of  Education  1998  5) 

In  1998  the  Danish  Ministry  of  Education  published  the  combined  strategies  and  action  plan  for  ICT  for  the  period 
1998-2003.  The  plan  describes  five  central  areas:  - the  pupils  and  ICT,  - the  teachers  and  ICT,  - the  subjects  and 
ICT,  - equal  and  flexible  access  to  lifelong  education,  - and  coordination  of  ICT-based  research  and  education.  It’s 
emphasized  that  ’It  is  not  the  objective  to  engage  in  massive  investments  in  ICT  in  education  just  for  the  sake  of  ICT 
..ICT  is  to  improve  the  teaching  and  learning  process,  and  the  students  are  to  attain  the  basic  ICT  qualifications, 
which  are  necessary  in  the  society  of  today  and  of  the  future  ’. 

Furthermore  it’s  emphasized  that:  All  educational  institutions  are  to  be  connected  electronically  in  order  to  ensure 
access  to  knowledge  and  cooperation  across  the  education  sector.  Moreover,  Danish  digital  teaching  materials  are 
to  be  developed:  materials  which  are  based  on  Danish  pedagogical  principles. 

European  Schools  Project  7) 

European  Schools  Project  is  a network  organisation  which  supports  a community  of  schools  worldwide  to  explore 
applications  of  educational  telematics.  The  ESP  aims  at  improvement  and  innovation  of  educational  activities  and  its 
organisation.  It  offers  educational,  organisational  and  technical  support  and  provides  opportunities  for:  - electronic 
communication  between  individuals,  schools  and  educational  networks,  - the  design,  performance  and  evaluation  of 
teleprojects,  - the  creation  and  usage  of  information  resources,  - to  enhance  better  learning  and  teaching. 

The  first  teleprojects,  under  the  name  ’Schools  in  Network’  were  launched  at  RDSES  in  1987.  In  1989  the  project 
began  to  co-operate  with  ESP,  initiated  in  1988  by  the  University  of  Amsterdam,  NL.  The  activities  in  ESP  now 
include  + 26  countries  in  Europe  and  beyond.  Annual  teachers’  conferences  take  place  in  the  participating  countries 
- in  1998  in  Copenhagen,  Denmark,  in  1999  in  Tartu,  Estonia  1 1)  and  in  2000  in  Haugesund,  Norway.  Although  the 
members  of  ESP  are  connected  by  telematics  throughout  the  year,  the  participating  teachers  at  the  conferences  find 
these  to  be  an  important  forum  for  discussing  pedagogical  and  didactical  issues.  The  personal  face-to-face  contacts 
have  also  proved  to  be  an  important  extra  and  very  valuable  element  for  designing  successful  curriculum 
collaborations. 

Teleprojects 

Teleprojects  are  co-operative  distance  learning  projects  in  which  telematics  and  foreign  languages  are 
communication  tools.  Teachers  mutually  determine  a central  topic  of  the  curriculum  around  which  students  execute 
local  (research)  activities.  Results  are  discussed  and  exchanged  via  Internet. 

Materials  to  two  of  the  many  ESP-teleprojects,  an  environmental  project  Bionet  8)  and  a language  teleproject  The 
Image  of  the  Other  9)  can  be  found  on  the  Internet. 

The  Image  of  the  Other 

’The  Image’  material,  which  is  a result  of  collaboration  between  Danish  and  Dutch  teachers,  is  meant  for  the 
teaching  of  English  as  a foreign  (second)  language  for  beginners  (elementary  stage).  In  this  virtual  classroom 
students  and  teachers  experience,  how  they  by  using  the  foreign  language  and  telecommunication  as  communication 
tools  with  their  counterparts,  acquire  new  knowledge  and  new  understanding  not  only  about  their  peers,  but  also 
about  themselves,  as  ’’they  are  mirroring  their  own  sitatuation  in  the  others’  ”.  The  teachers  have  experienced  how 
students  improve  their  language  skills,  that  the  engagement  in  the  project  arise  their  cultural  awareness  as  well  as  it 
is  highly  motivating  and  meaningful  to  take  part  in  such  a project.  We  thus  observe  a clear  shift  in  education  from 
teaching  to  learning,  where  the  latter  is  strengthened  and  the  education  as  such  is  enhanced.  The  Internet  now  thus 
gives  all  a chance  - regardless  of  where  you  live  - to  have  access  to  the  same  educational  material. 
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Teachers’  competences  in  ICT-genre  didactics  and  curriculum  related  didactics 

Teachers  engaged  in  teleprojects  all  have  had  a traditional  curriculum  related  didactical  competence  before  they 
start.  But  many  have  lacked  - at  various  degrees  - the  didactical  competences  related  to  each  of  the  current  ICT 
genres.  It’s  of  vital  importance  that  a teacher  before  engaging  in  a teleproject  masters  both  competences.  Bent  B. 
Andresen,  director  at  the  ’Research  Centre  for  Education  and  ICT’  at  RDSES,  has  in  the  following  model  illustrated 
how  the  ICT-genre  didactics  and  the  subject  related  didactics  complement  each  other.  1 0) 


Teachers  that  acquire  these  ICT  competences  have,  according  to  Bent  B.  Andresen,  access  to  new  pedagogical 
methods,  whereas  teachers  who  don’t  have  these  competences  lack  this  freedom  in  their  educational  practice. 

Importance  of  clearly  formulated  common  objectives 

Although  teachers,  may  have  access  to  the  same  material  (e.g.  the  Image)  some  fail  to  describe  the  common 
objectives  for  their  collaboration  clearly.  The  emphasis  could  thus  be  put  on  the  ICT-genres  for  one  partner, 
whereas  the  curriculum  related  content  was  of  major  importance  for  the  other.  This  invariably  had  an  impact  on  the 
result  of  the  collaboration  and  thereby  an  effect  on  the  learning  processes. 

Why  European  in-service  training  courses? 

ESP  national  co*ordinators  for  quite  some  time  discussed  the  possibility  of  setting  up  European  in-service  training 
courses.  Pedagogical,  scientific  and  technological  developments  as  well  as  the  EU  programme,  Socrates,  and  similar 
programmes  gave  scope  for  implementing  the  ideas  in  the  late  nineties. 

What  we  finally  had  were:  - a common  and  far  more  user-friendly  communication  platform  ( i.e.www)  - access  to 
financial  support  from  the  EU,  SOROS,  Nordic  Council  of  Ministers  etc)  to  conduct  in-service  training  courses  - 
valuable  pedagogical  and  organisational  experience  in  the  teleprojects  as  a basis  for  the  contents  of  the  courses  - and 
a European  team  of  moderators. 

Co-ordination  between  international  and  national  initiatives  and  in-service  courses 

The  ESP  co-operation  and  the  courses  running  under  the  Socrates/  Comenius  programme  are  most  often  bottom-up 
projects.  It’s  vital  that  the  courses  match: 

- national  ICT  initiatives,  - national  educational  reforms  for  schools  or  for  teacher  education,  - top-down  European 
projects  such  as  EUN  (European  SchoolNet) 
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’The  project  makers  - in-service  training  for  educational  key  personnel  on  how  to  make  educational  projects,  find 
partners  and  develop  projects’. 

The  Norwegian  and  the  Danish  ESP-coordinators,  together  with  a Scottish  colleague  ran  two  courses  in  1997  and 
1998  respectively  with  participants  from  15  different  European  countries. 

Experiences  from  these  courses  have  resulted  in  a new  Comenius  course  TRANSPRO  with  our  Czech  ESP- 
coordinator  representing  the  fourth  country  in  the  collaboration.  The  TRANSPRO  courses  are  planned  to  run  in 
2000. 

The  Nordic-Baltic  Language  Project 

A project  for  teachers  of  foreign  languages  sponsored  by  the  Nordic  Council  of  Ministers  and  planned  and 
coordinated  - among  others  - by  the  Estonian,  Danish,  and  Finnish  ESP  co-ordinators  have  run  during  the  academic 
year  of  1 998  - 1 999. 

The  aims  of  the  introductory  course  to  the  transnational  projects  were  to  focus  on: 

- the  cultural  dimension  in  language  teaching,  - introduction  to  and  designing  of  transnational  projects  based  on: 
’Das  Bild  der  Anderen’  (Material  to  be  used  in  teleprojects  in  the  German  language)  and  ’The  Image  of  the  Other’- 
evaluation  of  the  15  teleprojects  nmning  during  1998-99. 

Learning  processes  and  didactics 

We  have  learned  that  the  following  preconditions  for  successful  in-service  training  in  ICT  should  be  bom  in  mind, 
if  you  aim  at  running  successful  teleprojects:  - emphasis  should  be  placed  on  training  processes  rather  than 
products/materials  -A  major  priority  should  be  the  change  of  teachers’  attitudes,  since  the  teacher  is  the  actual  agent 
of  innovation  and  change  in  schools,  - Teacher  training  should  be  supported  with  the  dissemination  and  analysis  of 
models  of  ’good  practice’,  which  can  be  used  as  stimuli  of  discussion,  reflection  and  change,  - Teachers  should  be 
involved  in  evaluation  of  existing  learning  materials  and  this  reflective/evaluative  activity  could  form  part  of  their 
training  and  professional  development 

National  in-service  training  courses 


The  set  up  for  in-service  courses  differ  from  country  to  country.  Personally  we  have  run  an  ODL  course  in 
introducing  how  to  mn  teleprojects  over  the  past  8 years  with  participants  from  both  Denmark  and  the  other  Nordic 
countries.  The  Danish  Ministry  and  others  have  introduced  a large  scale  in-service  training  course  on  the  use  of  ICT 
in  all  subjects  - the  so-called:  ’ICT-licence’  course  - will  be  launched  in  the  autumn  of  1999. 

Conclusion 

Some  of  the  observations  and  recommendations  put  forward  at  the  Athens  6)  state,  what  we  already  have 
experienced  during  the  past  decade  or  so  that:  ’There  is  obviously  a trend  towards  ’process-oriented’ pedagogy 
rather  than  ’product-oriented’  or  ’ knowledge-centered’ pedagogy.  We  have  to  be  aware  that  there  is  a shift  ’from 
’development  of  content  ’ to  ’development  of practices  ’from  text  ’ to  ’context  ’....In  other  words,  the  teacher  herself 
is  called  to  reflect  on,  evaluate  and  participate  in  the  development  of  training  content  and  materials  ’6)  iCT  forces 
us  to  accept  a different  concept  of  learning  as  well  as  the  fact  that  the  teacher’s  role  is  undergoing  a change. 

When  looking  upon  Anne  Villems’s  letter  from  1993  we  observe,  that  much  has  been  accomplished  during  the  past 
decade  regarding  teachers’  pre-service  and  in-service  training  in  ICT,  nationally  as  well  as  internationally,  but  there 
is  still  a strong  need  for  ’on-going  teachers’  professionlisation  and  updating’.  As  for  examples  of  good  practice 
much  more  ought  to  be  accessable  on  the  web.  Further  (action)  research  is  needed  regarding  technological  and 
educational  issues  also  with  much  emphasis  and  stress  on  the  evaluation  of  using  ICT  in  order  to  ensure  a high 
quality  of  the  projects  and  thereby  enhance  education. 
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Abstract:  This  paper  focuses  on  the  influences  of  multimedia  and  hypermedia  technologies 
on  the  academic  practices  of  a higher  education  unit.  It  traces  the  ways  in  which  these 
technologies  penetrate  its  research,  development  and  teaching.  Multifaceted  uses  of  Web  are 
illustrated:  as  environment  for  course  design  and  delivery,  as  research  project  support 
environment,  as  product  integrator,  as  facilitator  of  dissemination  of  research  outcomes  and 
support  for  their  implementation.  Examples  of  on-line  courses,  methodology  and  tools  for 
Web-based  course  development,  and  specialized  educational  Web  sites  are  presented.  Pros 
and  cons  of  this  Web-orientated  approach  are  discussed  and  current  problems  are  identified. 
Multimedia  developments  for  children  - an  authoring  tool  and  applications,  developed  by  a 
European  consortium  - are  also  reported.  Finally,  reflections  on  the  overall  dimension  of 
change  are  made. 


Introduction 

We  are  witnesses  and  active  participants  in  a vast  process  of  globalization  of  education,  educational 
communications  and  professional  collaboration  due  to  the  remarkable  developments  in  the  area  of  distributed 
hypermedia  systems  and  their  applications.  This  tendency  calls  for  adequate  changes  in  the  traditional  activities 
of  higher  education  institutions  to  effectively  integrate  these  innovations.  Challenged  by  the  technology  rich 
reality,  many  higher  education  units  innovate  their  practices  - either  revolutionary  restructure  or  evolutionary 
enrich  and  re-engineer  (Collis  1997)  a broad  range  of  activities.  This  paper  focuses  on  the  impact  of  multimedia 
and  hypermedia  technologies  on  the  academic  practices  of  the  Department  of  Information  Technologies  at  the 
Faculty  of  Mathematics  and  Informatics,  University  of  Sofia.  First  the  ways  in  which  these  technologies 
penetrate  research,  development  and  teaching  are  traced.  Then  examples  of  on-line  courses  and  methodology  and 
tools  for  Web-based  course  design  and  delivery  are  presented.  Next  the  utilization  of  Web  as  facilitator  of 
dissemination  of  research  outcomes  and  support  for  their  implementation  is  discussed.  Current  multimedia 
developments  - an  authoring  tool  for  children  and  applications,  developed  by  a European  consortium  - are  also 
reported.  Finally,  pros  and  cons  of  this  hypermedia-orientated  approach  are  discussed,  problems  are  identified, 
and  the  dimension  of  the  overall  change  is  estimated. 


‘‘Webbing”  Research,  Development  and  Teaching:  An  Overview  of  Adopted  Web  Uses 

1 would  define  “academic  practices”  for  our  Department  as  including  teaching,  research  and  development  and 
naturally  embedding  communication  and  collaboration  - the  staff,  with  students,  with  other  professionals. 

Dissemination  of  research  outcomes  and  implementation  of  our  developments  supplement  our  activities  as  well. 
All  components  of  our  work  have  been  significantly  influenced  by  the  developments  in  the  networked 
multimedia  and  hypermedia.  To  understand  how,  let  us  start  from  the  Department’s  Homepage  (http://www- 
it.fmi.urii-sofia.bg/)  which  is  in  itself  a small  evidence  of  these  influences  - we  are  citizens  of  the  cyberspace 
now. 

Through  the  sections  “About  us”  and  “Staff’  the  Department’s  profile  and  each  of  us  are  now  accessible  on-line, 
which  eases  our  contacts  with  colleagues  and  students  and  helps  expanding  our  local  and  international 
collaboration. 
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“Teaching”  is  a major  section  here.  The  Department  is  responsible  for  the  informatics  part  of  the  B.Sc.  program 
“Mathematics  and  Informatics”,  which  prepares  teachers  in  mathematics  and  informatics.  We  also  run  an  in- 
service  teacher  training  center  and  offer  two  M.Sc  programs:  “Information  and  Communication  Technologies” 
and  “Artificial  Intelligence”,  and  run  informatics  courses  for  an  MBA  program.  Information  about  these 
programs  is  now  available  on-line,  and  contacts  with  responsible  persons  are  directly  possible.  This  enhances  the 
efficiency  of  our  information  provision  and  two-way  communication.  One  major  change,  which  will  be  discussed 
in  more  details  later,  is  the  orientation  to  on-line  teaching.  Most  of  the  on-line  courses  offered  by  the 
Department  are  accessible  from  here  too. 

Another  dimension  of  change  refers  to  the  nature  of  our  teaching.  Since  1984,  when  the  unit  was  established,  we 
have  gone  through  different  phases  where  different  models  of  instruction  have  dominated.  Almost  skipping  the 
typical  for  those  days  in  Bulgaria  highly  structured,  teacher-centered  classroom  instruction  (relying  mostly  on 
the  teachers  and  textbooks  as  sources  of  information),  we  went  through  less-guided,  learner-centered, 
exploratory  style  instructional  environments;  arriving  at  constructivist  principles  and  stimulating  active  learning 
(Campbell  1998).  Today  our  teaching,  especially  in  the  graduate  courses,  is  flexible  and  highly  individualized.  It 
widely  embeds  the  project  approach  (Nikolova  & Sendova  1995)  and  strongly  relies  on  the  “active  learner” 
assumption  and  her/his  access  to  information  and  knowledge,  both  locally  and  virtually.  This  wouldn’t  have  been 
possible  without  technological  facilitation:  availability  of  multimedia  learning  environments,  Web,  Intranets, 
Internet  tools  for  communication  and  collaboration.  The  changes  in  the  nature  of  the  teaching  can  be  summarized 
as:  from  teacher-centered  to  learner-centered  pedagogy,  from  classroom-oriented  to  more  individualized 
teaching,  from  tool-oriented  (as  far  as  computer  use  is  concerned)  to  technology  mediated  teaching  and 
learning. 

The  “Research”  section  links  to  homepages  of  a number  of  national  and  international  research  projects.  Within 
them  Internet  is  used  for  disseminating  information  - to  announce  our  research  to  others;  for  communication 
among  partners,  sharing  project  resources  and  collaborative  development  of  project  outcomes  during  the 
lifetime  of  the  project  (e.g.  MATCh  [1]:  homepage,  Workspace,  Resource  bank;  MALL2000);  /or  sharing 
research  outcomes  and  products  with  users  (VALUE,  GEOMLAND),  In  this  way  project  participation  and 
international  collaboration  are  greatly  facilitated. 

The  “Activities”  section  links  to  Web  pages  of  conferences,  workshops  and  seminars  organized  by  the 
Department.  While  for  earlier  conferences  we  used  to  only  provide  information,  recent  conference  Web  site 
embed  more  interactivity  and  allow  on-line  registration  (e.g.  EUR0L0G099),  which  is  a step  towards  a higher 
level  of  organization  and  time  saving  for  participants  and  organizers. 

The  “Student  project”  area  is  used  to  expose  good  master’s  projects  and  to  store  temporarily  some  students’ 
course  projects  - for  peers  to  view  and  share.  This  transparency  of  their  work  creates  additional  motivation. 

The  “Electronic  Catalogues”  section  contains  topics-oriented  collections  of  references  to  on-line  resources.  They 
have  been  developed  for  students  by  students  and  serve  as  a virtual  extension  of  the  student’s  library.  In  this  way 
peer  support  is  facilitated. 

Discussion 

Building  our  Web  presence  was  an  early  grass-root  initiative  of  our  Department.  It  was  later  promoted  at 
administrative  level  and  all  departments  are  encouraged  now  “to  go”  on  the  University  Web  site.  We  are  aware 
that  there  is  much  room  for  improvement  and  enrichment  of  our  Web  site,  e.g.  to  build  more  interactivity  and 
services,  to  more  frequently  update  the  information.  Nevertheless  and  despite  that  being  on  the  Web  is  trivial 
today  for  most  academic  units  in  the  developed  countries,  we  should  acknowledge  that  this  “small”  step, 
facilitated  by  the  utilization  of  hypermedia  technologies,  really  made  a difference  in  our  professional  lives. 


[1  ] URLs  are  provided  in  the  reference  section  as  “on-line  resources” 
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Teaching  On-line 


During  the  past  few  years  there  has  been  a definite  tendency  in  the  Department  to  integrate  Web  as  a support  tool 
in  the  teaching  and  learning  process,  to  design  Web-based  courses  (even  for  on-site  students)  and  transfer 
existing  courses,  partly  or  fully,  to  the  Web. 

Why? 

We  went  on-line  for  a number  of  reasons: 

we  were  challenged  by  some  research  projects,  in  which  we  participated,  and  gained  initial  experience  from; 
this  is  an  appropriate  solution  for  our  local  environment,  as  a Web-based  course  provides  the  flexibility 
which  our  students  need  and  we  want  to  offer  them; 

access  to  course  resources  and  related  international  publications  is  easier  for  the  students  in  this  way  (it  may 
sound  paradoxically,  but  it  is  true  in  our  case,  where  new  books  in  English  hardly  reach  students’  libraries) 
it  extends  the  scope  of  our  teaching  geographically  and  in  the  number  of  students  taught 
it  eases  the  reuse  of  course  materials 
it  eases  sharing  courses  and  resources  among  staff 

Some  examples  of  On-line  Courses 

Most  of  the  courses  developed  at  the  Department  are  supported  by  course  Web  sites,  built  around  the  virtual 
university  and  classroom  metaphors  (Nikolov  & Nikolova  1996;  Dicheva,  Djakova  & Bachvarova  1998).  The 
site  integrates  course  materials  and  links  to  on-line  resources,  possibilities  for  interaction  between  the  instructor 
and  students  and  among  students  themselves.  Some  course  Web  sites  support  group  discussions  through  WWW 
boards,  some  allow  on-line  testing  with  automatic  assessment.  In  most  cases  the  courses  are  taught  in  a 
“simulated  distance”  mode  — with  some  face-to-face  sessions,  but  most  of  the  students  activities,  assignments 
and  communication  are  performed  at  student’s  convenience  within  a given  period  of  time.  The  results  are 
delivered  to  the  instructor  via  e-mail  or  by  uploading  to  the  Web  site.  A few  course  examples  are  presented 
below. 

Telematics  and  Distance  Education 

An  elective  graduate  course  from  the  “ICT”  M.Sc.  program  (an  early  version  of  this  course  was  the  first 
experiment  for  teaching  on-line  in  the  Department).  Developed  as  a self-initiative  of  the  instructor,  without  any 
financial  or  administrative  support.  It  was  inspired  by  the  course  “Tele-learning”  (Collis  1997)  and  is  supported 
by  a Web  site  (Fig.  1)  which  embeds  course  information  and  study  materials,  allows  personal  and  group  e- 
mailing  and  group  discussion.  The  course  is  organized  in  units,  each  unit  presuming  self  study  of  preparation 
material,  face-to  face  session,  off-class  individual  learning  activities,  and  assignments  submitted  electronically. 
Feedback  is  provided  either  individually  - by  e-mail  or  face-to-face  - or  through  a Web  discussion  board.  All 
students  develop  course  projects,  individually  or  in  groups  of  two.  Project  guidelines  and  support  are  provided 
through  the  Web  site  too. 

Business  English 

Developed  in  the  frames  of  a European  project.  The  Web  site  (Fig.  2)  is  built  around  the  virtual  classroom 
metaphor  (Stefanov,  Dicheva,  Nikolov  & Djakova  1998):  there  are  \mlls  (for  course  description  and  syllabus), 
message  board  (for  instructor’s  guidelines),  shelf  (for  frequently  used  ‘books’/sources),  corners  (for  self-study 
activities),  gateways  (to  the  Post  Office,  Student  Centre,  Conference  Centre,  Video  Centre,  Library,  and  the 
Cafe).  The  units  include  pre-reading,  reading  and  post-reading  activities  and  interactive  assignments. 
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Figure  1:  “Telematics  and  Distance  Education”  course 


Figure  2:  “Business  English”  course 


Business  on  the  Internet 

Developed  in  the  frames  of  a European  project  Intended  for  graduate  students  in  Economics  and  Business 
administration  and  clients  from  SME,  Delivered  through  a virtual  learning  environment  (Nikolov  & Stefanov 
1998)  providing  access  to  lectures  and  guest  lectures,  resource  bank,  tests  and  essays,  group  discussion. 

Discussion 

The  experience  gained  so  far  has  outlined  both  positive  and  negative  aspects  of  Web-based  teaching  (Nikolova 
& Collis  1998;  Dicheva,  DJakova  & Bachvarova  1998),  According  to  students’  views  among  the  advantages  are: 
“flexible  in  space  and  time,  which  is  very  convenient:  allows  better  concentration  when  studying  on  your  own;  allows  your 
own  pace  through  the  course  material,  which  enhances  motivation;  on-line,  non-subjective  testing;  stimulates  regular 
learning  during  the  semester’',  while  some  drawbacks  are:  “problems  with  speed  of  access  and  network  stability  - 
alternative  ways  of  offering  the  course  material,  where  possible,  is  desirable  to  minimize  the  on-line  reading  time;  more 
intensive  feedback  from  the  instructor  is  necessary  when  the  course  is  fully  on-line 

One  major  specifics  of  the  on-line  courses  is  that  they  rely  very  much  on  the  self-motivation  and  self-control  of 
the  learners,  on  the  one  hand,  and  appropriate  technological  infrastructure,  on  the  other.  Before  a wider 
exploitation  is  considered,  more  in-depth  evaluation  is  still  needed  and  stronger  administrative  support  is  desired. 
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Methodology  and  Tools  for  Web-Based  Course  Design  and  Delivery 

The  Department  has  to  deliver  instruction  in  the  field  of  ICT  to  different  target  groups  - pre-service  and  in- 
service  teachers,  master  students  in  ICT,  AI  and  BA.  The  tendency  to  employ  hypermedia  technologies  and  the 
experience  gained  from  the  first  Web-based  courses  called  for  a more  general  solution  in  order  to  offer  more 
flexibility  for  the  students  while  at  the  same  time  being  more  efficient  in  course  design  and  development.  A 
method  for  designing  flexible  instructional  modules  was  developed  (Nikolova  1996)  to  assist  in  designing 
modules,  which  are  adaptable  to  different  learner’s  needs  and  different  delivery  platforms.  Later  two  software 
tools  were  developed  to  support  the  implementation  of  the  method:  Course  Wizard  - a desktop  database-driven 
application,  and  Course  Developer  - a multi-user  Web-integrated  database-driven  system.  The  method  and  the 
tools  are  briefly  presented  below. 

The  Method  for  Flexible  Instructional  Modules  Development 

The  method  is  rooted  in  the  theory  of  flexible  learning  (Van  den  Brande  1993;  Collis,  Vingerhoets  & Moonen 
1995).  Flexibility  is  defined  as  offering  the  learner  choices  with  respect  to  a number  of  flexibility  dimensions, 
related  to  course  time,  content,  entry  requirements,  instructional  approaches  and  learning  materials,  delivery  and 
logistics  (Collis  1996).  Telematics  has  an  essential  role  to  play  when  planning  for  more  flexibility.  For  example, 
adding  a tele-learning  dimension  to  a traditional  course,  one  can  distinguish  two  further  stages  (Collis,  1996): 
pedagogical  enrichment  - when  the  components  of  the  course  and  their  balance  are  preserved,  but  the  nature  of 
activities  changes  due  to  the  new  possibilities  offered  by  tele-learning;  pedagogical  re-engineering  - when  the 
course  changes  both  in  structure  and  nature  of  the  activities. 

The  method  is  developed  for  an  instructional  provider  (the  Department,  in  our  case)  who,  while  given  limited 
staff  time  and  resources,  has  to  design  instruction  for  a range  of  learners’  groups  who  share  similar  needs  in  a 
particular  subject  domain.  It  prescribes  to  construct  a generic  module  which  permits  adaptations  (variants  for 
each  group),  the  overall  effort  in  terms  of  staff  time  and  resources  being  less  than  when  developing  the  variants 
separately.  The  method  encourages  team  course  design. 

The  main  concepts  employed  by  the  method  are:  generic  module,  adaptation  and  resource  bank.  The  generic 
module  is  designed  for  a particular  subject  domain  and  for  a defined  range  of  target  groups.  It  is  a backbone  in 
terms  of  content,  pedagogical  profile  and  instructional  materials.  It  is  not  a ready-to-use  instructional  module  in 
itself,  but  a resource,  which  purpose  is  to  facilitate  the  production  of  adaptations.  An  adaptation  (Fig.  3)  is  a 
ready-to-use  instructional  module,  which  derives  from  the  generic  module  and  is  designed  for  a particular  target 
group.  At  certain  moments  more  than  one  adaptation  will  exist  around  one  generic  module.  The  resource  bank 
(Fig.  4)  is  a set  of  resources  (stored  in  a database)  which  can  be  used  to  produce  adaptations  and  to  revise  the 
generic  module.  Elements  are  added  to  the  resource  bank  during  the  whole  process  of  method  application. 


Other  Adaptations 
Generic  Module 
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New 

Adaptation 
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Resource  Bank 
O ther  sources 


Figure  3:  How  an  adaptation  derives 
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The  Course  Model 

A course  is  a set  of  units.  Each  unit  can  have  its  own  structure  (Fig.  5),  which  is  a subset  of  a predefined 
component  set  (default  pedagogical  profile).  The  latter  includes:  lecture  or  demonstration,  communication  b/n 
instructor  and  learner,  group  discussions,  individual  study  and  preparing  exercises,  individual  or  group  project, 
student’s  testing,  feedback  from  the  instructor  (Collis  1997).  To  each  component  one  or  more  instructional 
materials  can  be  assigned.  The  instructional  materials  are  part  of  the  resource  bank  and  are  stored  in  the  course 
database.  The  database  also  stores  information  (as  links)  about  the  structure  of  the  course  and  its  units. 


Course  Wizard:  A Desktop  Program  for  Course  Design  and  Course  Web  Site  Generation 

Course  Wizard  is  a desktop  application  (Nikolova  & Pelovski  1998),  which  leads  the  designer  through  the 
process  of  building  a generic  module  and/or  an  adaptation.  Throughout  this  process  the  course  database  is 
created  - the  designer  provides  course  information,  schedule  and  structure,  assigns  materials.  Then  the  course 
Web  site  is  automatically  generated  by  using  an  existing  template  and  asking  the  designer  to  specify  additional 
information  to  customise  the  layout  (Fig.  6).  The  Web  site  of  the  “Telematics  and  Distance  Education”  course 
(Fig.  1)  is  an  illustration  of  the  result.  Course  Wizard  eases  the  multiple  uses  and  updates  of  a course.  The 
program  is  implemented  in  Borland  Delphi  and  uses  Borland  Interbase  Server. 
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Figure  6:  Course  Wizard:  Specifying  the  course  homepage 


Course  Developer:  A Web-based  Course  Support  System 

Course  Developer  (Nikolova,  Boyanov  & Dimitrov  1999)  is  an  ancestor  of  Course  Wizard  in  terms  that  it 
supports  the  design  of  courses  of  the  above  model  and  generates  a Web  site  to  mediate  the  course  delivery  It  has 
additional  features  and  functionality  though,  which  make  it  a powerful  course  support  system.  It  has  a Web 
interface  (Fig.  7)  and  enables  team  development/update  of  a course  over  the  network.  It  allows  different  types  of 
users  - administrator,  designer,  tutor,  student  and  guest  - with  different  access  permissions.  Supports  not  only  the 
process  of  course  design  and  development,  but  also  course  delivery  by  offering  registration  of  tutors  and 
students,  access  to  course  materials,  internal  course  mail,  discussion  center  and  information  on  student  progress 
and  tutor’s  tasks.  A module  for  automatic  test  generation  and  student  assessment  is  also  available.  Course 
Developer  is  a Web-integrated,  database-driven  system,  which  uses  dynamic  pages  based  on  the  ASP 
technology. 
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Figure  7:  Course  Developer’s  homepage 
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Discussion 


Web-based  course  environments  require  powerful  infrastructure  and  are  expensive  to  produce.  Many 
professional  systems  for  Web-based  course  support  exist  nowadays  (http://www.online.uillinois.edu/oakley/): 
Mallard,  CyberProf,  Web  Course  in  a Box,  TopClass,  WebCT,  Lotus  Learning  Space  and  many  more.  Our 
Course  Developer  system  is  a result  of  the  research  effort  of  a small  group  of  people  and  was  developed  with 
almost  no  financial  support.  In  this  sense  it  can  hardly  compete  with  the  above  systems.  The  advantages  are  that 
it  is  tailor-made  - it  closely  fits  our  needs,  and  the  work  on  it  allowed  us  to  not  only  be  users,  but  also  engineers 
of  such  environment,  which  has  an  added  value  for  better  understanding  the  anatomy  of  the  process,  tools  and 
outcomes.  If  we  move  to  a large-scale  on-line  teaching,  we  will  probably  choose  companies’  products,  but  the 
experience  gained  will  still  be  valuable  and  helpful. 


Sharing  Research  and  Development  Outcomes  and  Providing  Support  to  Users 

A significant  part  of  the  Department's  research  and  development  outcomes  is  directly  useful  for  schools  and 
teachers,  but  there  is  neither  appropriate  infrastructure,  nor  organizational  and  administrative  support  for 
bringing  it  to  them.  It  is  quite  cumbersome  to  manage  the  flow  of  individual  and  group  requests  coming  from  all 
over  the  country,  organize  in-service  teacher  training  and  follow-up  activities,  as  a voluntary  supplement  to  one’s 
regular  activities.  Still  we  continue  doing  it  as  teachers  inquire  us  and  we  feel  we  can  help.  But  it  is  very 
important  to  be  able  to  do  it  with  minimum  time,  effort  and  distraction  from  our  direct  work.  Some  examples  of 
employing  the  Web  to  ease  the  transition  from  research  labs  to  potential  users  are  presented  below. 

VALUE  - A Virtual  Almanac  for  Logo  Users  and  Educators 

The  project  “Learning  by  developing  with  Comenius  Logo”,  initiated  by  the  Teacher  Training  Center  run  by  the 
Department,  aims  to  bring  the  spirit  of  Logo  and  Logo  educational  philosophy  to  the  Bulgarian  schools 
(Nikolova  1997).  Comenius  Logo  (Blaho  & Kalas  1998)  - an  attractive  and  sophisticated  environment  with 
considerable  conceptual  and  operational  power  - was  chosen  as  software  platform.  We  adapted  the  system  to  the 
Bulgarian  language,  developed  Bulgarian  on-line  help,  and  a whole  range  of  reference,  teaching  and  learning 
materials  (lessons,  problems,  projects,  microworlds)  for  teachers  and  students.  In  this  sense  the  project  is  a good 
example  for  initiating  and  supporting  educational  innovations  (Fullan  1991).  We  used  the  Web  as  a 
supplementary  channel  for  delivering  our  production  to  users,  getting  feedback  and  establishing  collaboration 
with  motivated  teachers.  VALUE:  A Virtual  Almanac  for  Logo  Users  and  Educators  was  created  (Nikolova  & 
Ginkulova  1998).  We  integrated  there  the  developed  materials  and  embedded  communication  and  discussion 
(Conference  Room  metaphor),  event  announcement  (Message  Board),  building  virtual  Logo  community  (Guest 
Book  metaphor).  For  easier  orientation  within  the  site  a site  map  (Fig.  8)  is  available.  A lot  of  users  benefit  from 
VALUE,  which  lowers  significantly  our  load  and  contributes  to  the  effectiveness  of  the  implementation  process. 


Figure  8:  VALUE:  Homepage  and  Site  map 
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GEOMLAND  - A Laboratory  for  Mathematics  Explorations  in  Logo  Style 


GEOMLAND  (Sendov  & Sendova  1997)  is  a powerful  Logo  style  learning  environment  for  mathematics 
explorations,  developed  at  the  Department,  Its  dissemination  and  school  implementation  is  accompanied  with 
similar  problems  as  the  ones  mentioned  above,  A recently  developed  Web  site  (Fig,  9)  makes  it  convenient  to 
provide  users  with  what  they  need  - the  software  is  downloadable  from  there,  research  and  methodology  papers, 
demonstrations  and  examples  are  available.  The  time  spent  until  recently  on  personally  advising  users,  providing 
information  and  demonstration  for  them  can  be  now  allocated  to  other  tasks. 


Figure  9:  A GEOMLAND  Web  page 
School  Resource  Bank  - A Subject-Oriented  Web  Resource  Catalog 

Our  students  make  field-oriented  Web  searches  and  develop  useftil  collections  of  annotated  on-line  resources. 
One  example  is  the  School  Resource  Bank  (Fig,  10),  which  is  a subject-oriented  educational  catalog.  It  is 
included  in  the  Department’s  Web  site  and  offered  also  to  the  Bulgarian  I*EARN  section  for  use  by  schools  with 
Internet  access. 


Figure  10:  School  Resource  Bank 
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Discussion 


Maintaining  educational  Web  sites  does  not  only  save  time  and  help  reach  more  users.  Feedback  from  users  is 
facilitated,  two-way  communication  channel  and  mutual  collaboration  are  established.  Another  positive  effect  is, 
that  offering  meaningful  material  and  helpful  interactions  to  teachers  in  such  a way,  they  get  used  to  using 
Internet  for  educational  resources  and  motivated  to  make  their  own  Web  developments. 

MATCH:  A Multimedia  Authoring  Tool  for  Children 

A recent  European  project  with  partners  form  Slovakia,  Hungary,  Holland,  UK,  Greece  and  Bulgaria  was 
devoted  to  developing  an  environment  for  multimedia  authoring  for  children  (Triantafilou  et  al  1997). 

MATCH  Overview 

The  system  employs  the  constructivist  learning  philosophy  and  supports  creative  thinking  and  expression  by 
means  of  multimedia  technologies.  It  consists  of  five  tools  — Frame,  Sound,  Animation,  Story  and  Web  editors. 
In  the  Story  editor  the  user  can  create  applications  (stories)  by  combining  multimedia  resources  - either  already 
available  or  self-produced  by  the  Frame,  Sound  and  Animation  editors.  The  Story  editor  can  be  entered  at  three 
levels  (Fig.  1 1):  beginner's,  intermediate,  advanced,  and  two  modes:  run  (to  play  a story)  and  edit  (to  create  a 
story).  The  Web  editor  can  export  (with  some  restrictions)  a story  into  a Web  presentation.  During  the  school 
testing  the  environment  was  well  accepted  by  the  teachers  and  students  and  proved  to  be  an  appropriate 
component  of  today’s  technology  rich  classroom.  Especially  the  Story  editor,  with  the  ease  of  use  and 
attractiveness  of  the  final  result,  was  valued  as  a motivating  and  rich  context  for  active  learning  and  creative 
expression.  An  important  requirement  for  the  effective  use  of  the  system  is  the  availability  of  rich  libraries  of 
multimedia  resources  for  the  children  to  start. 
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Figure  11:  Story  Editor:  Entry  Screen 


Applications  Developed  by  the  Bulgarian  Team 

Several  MATCh  applications  were  developed  to  demonstrate  the  power  of  the  environment  and  serve  as  system 
testbeds.  Some  of  them  are  quite  complex  and  have  an  educational  value  on  their  own. 
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Multimedia  Screen  Cards 


A multimedia  screen  card  is  a moving,  speaking,  singing  card,  the  user  can  play  with  (click  on  certain  elements 
to  make  something  happen),  print,  modify,  export  into  a Web  page.  Each  card  has  its  own  scenario,  i.e.  is  a short 
multimedia  story.  The  application  faces  children  with  most  popular  European  traditions  and  festivals  and  with 
personal  occasions,  when  people  interact  with  each  other  by  sending  greetings,  expressing  good  wishes  or 
appologies.  It  tends  to  stimulate  children  for  a more  social  style  of  behavior  and  interaction  by  using  new  means 
of  expression  - multimedia  technologies.  Sixteen  sample  cards  for  different  occasions  were  developed  (for 
Christmas,  St.  Valentine’s  Day,  Easter,  Mother’s  Day,  Birthday,  Get  Well,  Apology,  etc.)  and  built  into  a 
consistent  Screen  Card  World  with  a well  designed  visual  navigation.  The  user  is  taken  by  hand  by  the  main 
character,  the  Duck,  who  tells  a story  about  each  occasion  (Fig.  12).  Then  the  sample  cards  for  this  occasion  can 
be  played  with  (fig.  13).  Not  only  can  the  child  enjoy  the  Screen  Cards  World  - its  cognitive  value,  intriguity, 
variety,  brightness,  dynamics  and  interactivity.  There  is  more:  she  can  use  the  rich  collection  of  resources 
developed  for  this  application  - actors,  images,  sounds,  music,  recorded  speech,  backgrounds  - and  combine 
them  in  her  own  way,  thus  designing  her  own  cards  and  multimedia  stories. 


Figure  12:  Multimedia  Screen  Cards  application 
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Save  the  Animals 


This  application  illustrates  the  possibility  to  design  multimedia  games  with  Match.  It  is  a story  where  the  child 
plays  the  role  of  a Zoo  Detective,  trying  to  help  an  animal  in  danger  (Fig.  14).  The  application  has  a modular 
structure  and  allows  modification  and  extension.  A rich  collection  of  resources  was  developed  for  it:  animated 
actors  and  animals,  media  files  and  backgrounds. 


Figure  14:  Save  the  Animals  (Editing  a page  of  the  story) 


Children  Projects 

While  testing  the  system  and  applications  in  school,  children  made  their  own  production  with  MATCh  - 
individual  screen  cards  and  class  projects:  “Meet  our  class”  (Fig.  15)  and  “Class  album”.  This  work  kept  them 
very  much  involved.  An  interest  to  different  types  of  information  was  provoked  and  children  were  motivated  to 
learn  more  about  using  and  processing  texts,  pictures,  movements,  sounds  and  music. 


E&  DIET  A?/ B are  1 0 slU^ents. 

Click  <an  the  plclures  in  meet  ub 


Figure  15:  Meet  Our  Class  project 
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Discussion 


Why  multimedia  authoring  for  children?  To  provoke  children’s  thinking,  imagination,  expression,  active 
learning  (Druin  & Solomin  96).  To  enable  children  taste  the  process  of  creating  multimedia  and  personalize  the 
basic  concepts  of  the  multimedia  world  surrounding  them.  To  create  a context,  in  which  each  child  can 
contribute  to  an  enjoyable  collaborative  activity  with  what  s/he  is  talented  for  - choosing/creating/mixing  music, 
telling  a story,  creating  an  interesting  scenario,  drawing,  producing  animation,  designing  interface.  There  are 
various  roles  as  various  the  children’s  preferences  can  be. 

Reflections 

The  dimension  of  change  illustrated  by  the  above  is  not  only  a technological  one.  It  is  much  broader,  concerns 
attitudes  and  culture  and  refers  to  both  professional  and  personal  level.  The  overall  tendency  of  virtualization  and 
globalization  of  our  professional  lives  is  sometimes  in  conflict  with  the  intuitive  reaction  to  defend  ourselves 
from  an  intra-personal  Internet  invasion.  On  the  other  hand,  armed  with  the  modem  technologies,  we  were  able 
to  establish  a meaningful  context  for  challenging  and  rewarding  professional  activities  within  an  environment, 
which  is  still  suffering  from  social,  economic  and  organizational  deficiencies. 
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Abstract;  Described  in  this  paper  is  a case  study  of  evaluation  conducted  in  the  instructional  & 
performance  technology  (IPT)  department  at  Boise  State  University  (BSU).  The  IPT  department 
offers  a distance  education  (DE)  option  in  its  Master's  degree  program.  This  case  study  describes 
the  effective  interventions  that  the  educational  organization  used  to  reduce  the  drop  out  rate.  The 
drop  out  rate  during  1989  and  1996  was  44%.  The  IPT  DE  program  designed  and  implemented 
interventions  to  reduce  the  drop  out  rate  during  the  three  semesters  in  1997.  Within  a year,  they 
found  that  significantly  positive  results  were  obtained  from  their  interventions.  The  purpose  of  this 
paper  is  to  help  audience  understand  how  to  design,  implement,  and  evaluate  interventions  to 
reduce  high  dropout  rates.  Several  instructional  design  models  and  an  evaluation  model  such  as  the 
ARCS  model,  the  Organizational  Element  Model  (OEM),  and  Kirkpatrick's  evaluation  model  are 
discussed  in  the  paper. 


1.  Introduction 

1.1  Adult  Education 

The  U.S.  Department  of  Education  defines  adult  education  as  the  teaching  of  adults  via  any  education 
activities,  except  full-time  enrollment  in  higher  education  credential  programs.  According  to  the  Digest  Education 
Statistics  published  by  the  U.S.  Department  of  Education  (1997),  the  number  of  adult  education  participants  among 
1 17,826,000  employed  persons  during  1996-1997  was  59,734,000. 

What  motivates  adults  to  be  involved  in  continuous  formal  education?  Houle  (1971)  conducted  a 
qualitative  study  from  which  he  identified  three  types  of  adult  learners:  goal-oriented  participants,  activity-oriented 
participants,  and  learning-oriented  participants  (cited  in  McCreary,  1990).  Verduin  and  Clark  (1991)  categorized 
three  main  types  of  adult  education  programs:  adult  basic  education  (ABE)  programs  (to  acquire  ABE),  leisure  and 
enrichment  education  programs  (to  increase  enrichment  in  adult  life),  and  career  education  programs  (to  prepare  or 
upgrade  their  job-related  knowledge  and  skills).  Examples  of  adult  education  activities  include  part-time  college 
attendance,  classes  or  seminars  given  by  employers,  classes  taken  for  adult  literacy  purposes,  adult  basic  education 
or  English  as  a second  language,  or  courses  for  recreation  and  enjoyment. 

1.2  Adult  Distance  Education 

Distance  education  is  defined  as  "any  formal  approach  to  learning  in  which  a majority  of  the  instruction 
occurs  while  educator  and  learner  are  at  a distance  from  one  another"  (Verduin  & Clark,  1991,  p.8).  Distance 
education,  due  to  its  time  and  geographic  flexibility,  has  appealed  to  working  adult  learners  who  work  full-time  yet 
want  to  seek  for  continuous  education.  Many  adult  learners  attempt  to  achieve  their  goal  of  adult  learning  via 
distance  learning  options.  Distance  education  institutions  use  various  distant  learning  technologies  such  as  audio  and 
video  conferencing  devices,  the  Internet,  or  computer-mediated  communication  systems. 

According  to  a survey  conducted  by  National  Center  for  Education  Statistics  in  1995,  out  of  about  14.3 
million  students  enrolled  in  higher  education  institutions  in  fall  1994,  about  758,640  adult  students  formally  enrolled 
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in  distance  education  courses  in  academic  year  1994-95.  Eighty  one  percent  of  institutions  reported  that  they 
offered  courses  designed  for  undergraduate  students;  thirty  four  percent  for  graduated  students;  and  thirteen  percent 
for  professional  continuing  education.  Among  the  distance  education  institutions,  thirty  nine  percent  of  them 
targeted  professionals  who  were  seeking  recertification,  and  49  percent  targeted  other  workers  who  looked  for  skill- 
updating or  retraining  as  potential  audiences. 

1.3  Adult  Distance  Education  in  Boise  State  University 

The  Instructional  & Performance  Technology  (IPT)  Department  at  Boise  State  University  offers  a Master's 
degree  program  via  distance  education  (DE).  The  IPT-DE  program  is  intended  to  prepare  adult  students  at  distance 
for  careers  in  the  areas  of  instructional  design,  job  performance  improvement,  human  resources,  organizational 
redesign,  training,  and  training  management.  The  majority  of  students  attend  the  IPT’s  distance  program  not  only  to 
earn  a master’s  degree  in  IPT  but  also  to  upgrade  their  professional  knowledge  and  skills.  The  majority  of  the  adult 
distance  learners  who  enroll  in  the  IPT-DE  program  are  classified  as  goal-oriented  and  learning-oriented  learners. 
The  asynchronous  computer-mediated  communication  (CMC)  environment  that  the  IPT-DE  program  provides  in  the 
DE  classes  has  been  helping  the  adult  learners  engage  in  interactive  and  dynamic  discussions  among  the  participants 
and  achieve  their  goals  of  higher  education  through  the  collaborative  learning  experiences. 

2.  A Problem  in  Adult  Distance  Education 

Although  distance  education  has  appealed  to  working  adult  students  who  work  full-time  and  seek  for 
continuous  education  as  part-time  students  and  allow  them  to  attempt  to  achieve  their  goal  of  adult  learning  via  DE 
options,  a problem  in  DE  has  been  a high  turnover  in  enrollment.  Retention  of  DE  students  is  usually  lower  than 
retention  of  on-campus  students  (Verduin  & Clark,  1991).  Reasons  for  dropouts  are  found  to  be  various.  Adult  DE 
students  tend  to  drop  out  of  DE  programs  when  they  perceive  that  their  interests  and  course  structure  are  not 
matched  (Fenner,  1998),  that  they  are  not  confident  enough  in  learning  processes  (Chacon-Duque,  1987),  and/or  that 
they  have  achieved  what  they  wanted  (Holmberg,  1989  cited  in  Verduin  & Clark,  1991). 

The  dropout  problem  in  DE  programs  was  found  in  the  IPT-DE  program  at  Boise  State  University  as  well. 
Between  the  fall  semester  of  1 989  and  the  fall  semester  of  1 996,  44%  of  DE  students  dropped  out  of  the  IPT-DE 
program  by  their  third  course  (Fenner,  1998). 

3.  Systemic  and  Systematic  Interventions 

In  order  to  solve  the  dropout  problem,  it  is  important  to  approach  to  a solution  in  a systemic  and 
systematical  way.  It  is  critical  to  conduct  a careful  analysis  on  the  causes  to  the  problem  first,  then  to  seek  for  the 
most  appropriate  solutions  to  the  problem,  to  implement  the  solutions,  and  evaluate  the  effectiveness  of  the 
solutions.  The  ARCS  model,  the  Organizational  Element  Model  (OEM),  and  Kirkpatrick's  evaluation  model  guide 
such  processes. 

First,  according  to  Keller's  ARCS  model  (Keller,  1987),  there  are  four  factors  that  influence  the  degree  of 
learners'  motivation  to  learn:  Attention,  relevance,  confidence,  and  satisfaction.  Learners  lose  their  motivation  to 
learn  and  quit  learning,  especially  when  they  do  not  perceive  the  instruction  as  interesting  or  relevant  to  their  goal. 
They  also  lose  motivation  to  learn  when  they  are  not  confident  in  learning  processes,  and/or  they  are  not  satisfied 
with  the  instructional  processes.  Therefore,  it  is  critical  to  design  DE  instruction  based  on  the  ARCS  model  in  order 
to  prevent  dropouts. 

According  to  the  OEM  (Kaufman  & Thiagarajan,  1987),  there  are  four  elements  in  a results  chain: 
Organizational  inputs,  processes,  products,  and  outputs.  Effective  instructional  inputs  and  processes,  possibly 
designed  based  on  the  ARCS  model  and  delivered  by  an  educational  organization,  will  result  in  successful  learning 
outcomes,  which  is  one  of  the  educational  organization's  products.  Learners  who  experience  successful  learning 
outcomes  will  more  likely  continue  to  be  motivated  to  learn;  that  is,  there  will  be  a high  retention  rate  in  enrollment, 
which  is  one  of  the  educational  organization's  positive  outputs. 

In  order  to  evaluate  the  effectiveness  of  the  educational  organization's  inputs  and  processes,  products,  and 
outputs,  Kirkpatrick's  model  of  evaluating  training  programs  can  be  used  (Kirkpatrick,  1 996).  The  level  1 evaluation 
is  to  measure  learners’  reaction  to  the  educational  organization's  instructional  inputs  and  processes.  Since  the 
instruction  was  designed  based  on  the  ARCS  model,  it  is  appropriate  to  evaluate  learners'  reaction  to  the  instruction 
based  on  the  four  ARCS  elements.  That  is,  measured  are  learners'  perceptions  on  their  attention  levels  during  the 
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instruction,  relevancy  of  the  instruction  to  their  interests,  confidence  levels  in  learning,  and  satisfaction  levels 
toward  the  instruction  as  well  as  the  instructor.  The  level  2 evaluation  is  to  measure  the  educational  organization's 
product:  i.e.,  the  learning  outcomes.  The  level  3 evaluation  is  to  measure  the  educational  organization's  output:  i.e., 
whether  the  dropout  rate  has  decreased  or  not.  A summary  of  the  systemic  and  systematic  interventions  and 
evaluations  of  the  effectiveness  of  the  interventions  conducted  by  the  IPT-DE  program  at  BSU  is  illustrated  in 
Figure  1. 


Help  Learners  Develop 
ppsiiivH  Reaction 
Attcntbn 

Relevancy  Improve 

Confidence  Learning 

Sairsfaciion  Outcomes 


Increase 
Retention  Rate 
Decrease 
Drcocij;  Rate 


l-xti'ilf'iOl  lO 
Oroanii^aljcn 


Level  3 

Figure  1 . Systemic  and  Systematic  Interventions  and  Evaluations 


4.  Cause  Analysis 

In  order  to  find  out  the  causes  of  the  dropout  problem,  the  IPT-DE  associate  program  developer  conducted 
interviews  with  the  students  who  dropped  out  of  the  program  as  well  as  those  who  were  continuing  the  program 
between  1989  and  1996.  From  the  interviews,  it  was  concluded  that  their  satisfaction  levels  during  the  first  or 
second  courses  were  the  major  factor  that  determined  their  decisions  to  continue  or  not  to  continue  the  program. 
Forty  two  percent  of  the  students  who  dropped  out  of  the  program  expressed  'dissatisfaction  with  the  learning 
environment’  as  a reason  for  their  dropout.  More  specific  reasons  for  dropouts  included: 

□ discrepancies  between  their  professional  or  personal  interests  and  the  course  structure 

□ low  confidence  levels  in  distance  learning 

□ doubts  in  successful  online  communication 

□ incompetence  in  using  the  DE  software  as  an  effective  learning  tool 

□ feelings  of  being  overwhelmed  by  the  advanced  knowledge  and  overloaded  information 

□ de-personalized  learning  environment. 

5.  Intervention  Plan 

From  the  cause  analysis,  it  was  obvious  that  a solution  to  the  dropout  problem  must  be  to  help  new  distance 
students  be  satisfied  with  instruction  and  improve  their  performance  as  well  as  their  confidence  levels  during  the 
first  and  second  courses.  During  three  semesters  (spring,  summer,  and  fall)  in  1997,  the  IPT-DE  program  addressed 
how  to  design  instruction  to  be  more  attractive  enough  to  keep  new  DE  students  motivated  in  learning,  to  be  more 
relevant  to  their  professional  interests,  and  to  increase  individual  students'  confidence  and  satisfaction  levels  toward 
the  instruction.  The  ARCS  model  and  the  OEM  were  used  to  guide  the  instructional  design  and  developmental 
processes.  Especially  the  instructor  of  the  first  DE  course  aimed  to  help  her  students  develop  self-regulated  learning 
skills  in  the  DE  environment.  In  doing  so,  she  used  various  instructional  strategies,  including: 

□ to  administrate  a pre-knowledge  assessment  to  measure  students'  previous  knowledge  levels  at  the  first  week  of 
the  semester 

□ to  interact  with  individual  students  as  much  as  possible  and  encourage  them  to  interact  with  each  other  as  well 

□ use  all  types  of  data  presented  by  individual  students  in  order  to  know  the  students  better 

□ to  break  down  the  instruction  into  small  weekly  modules  and  help  them  master  each  module  at  a time 
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□ to  inform  students  of  the  goals  and  objectives  of  the  weekly  instruction  and  encourage  them  to  self-evaluate 
their  weekly  learning  process  as  well  as  weekly  learning  outcomes 

□ to  provide  students  with  clear  criteria  of  expected  performance  levels 

□ to  provide  specific  help  to  the  students  who  have  low  pre-knowledge  levels  and  provide  more  challenging  work 
to  the  students  with  high  pre-knowledge  levels 

□ to  deliver  instruction  via  multiple  media  such  as  PowerPoint  slide  show,  the  WWW,  and  an  electronic  bulletin 
board  when  appropriate 

□ to  provide  meaningful  examples  and  analogies  to  help  students  learn  new  concepts 

□ to  modify  instruction  based  on  students'  background  when  it  is  possible  and  appropriate  in  order  to  help  them 
see  the  instruction  relevant  to  their  professional  and  personal  interests 

□ to  conduct  formative  evaluations  during  the  semester  in  order  to  obtain  information  about  the  effectiveness  of 
the  interventions  and  individual  students’  perceptions  toward  instruction 

□ to  monitor  individual  students’  performance  and  provide  immediate^  frequent,  regular  (weekly),  and 
personalized  feedback 

□ to  give  positive  reinforcement  to  students  when  they  collaboratively  work  toward  achieving  the  instructional 
objectives 

In  addition  to  the  instructional  strategies  presented  above,  another  important  intervention  is  an 
administrative  strategy  - personalized  advisement  throughout  the  program.  Prior  to  the  registration,  the  DE 
associate  program  developer  and  the  faculty  members  provided  advisement  to  individual  DE  students,  especially 
regarding  the  most  appropriate  sequence  of  courses  for  them  to  take.  It  is  in  order  to  prevent  the  DE  students  from 
taking  advanced  courses  too  early  and  having  unnecessary  experiences  of  failure  or  to  help  them  take  courses  that 
are  more  relevant  to  their  professional  and  personal  interests. 

6.  Evaluation  of  Reaction,  Learning  Outcomes,  and  Dropout  Rate 

6.1.  Reaction  (Level  1) 

At  the  end  of  each  of  the  three  semesters  in  1997,  a course  evaluation  questionnaire  was  administered  to 
measure  students’  reaction  to  the  quality  of  the  course  and  the  instructor.  The  response  scale  used  was  A: 
Outstanding,  B:  Very  Good,  C:  OK,  D:  Improvement  Needed,  and  E:  Unsatisfactory.  As  coding  their  responses,  the 
number  one  was  assigned  to  a response  to  ’A’;  2 to  ’B’;  3 to  'C;  4 to  'D';  and  5 to  'E*.  The  data  of  average  scores 
revealed  that  students  were  more  satisfied  with  the  quality  of  the  course  as  well  as  the  quality  of  the  instructor  as  the 
implementation  of  interventions  became  matured  semester  after  semester  (see  Table  1). 


Semester 

Attention 

Relevance 

Confldence 

Satisfaction  toward  Course 
& Instructor 

Spring,  1997 

M=  1.41 

M=  1.92 

M=  1.83 

M=  1.92&  1.42 

Summer,  1997 

M=  1.17 

M=1.63 

M=  1.75 

M=  1.50  & 1.00 

Fall,  1997 

M=  1.25 

M=  1.47 

M=  1.56 

M=  1.31  & 1.06 

Table  1.  Level  1 Evaluation:  Attention,  Relevance,  Confidence,  and  Satisfaction  Levels 

6.2.  Learning  Outcomes  (Level  2) 

In  each  of  the  three  semesters  in  1997,  a pretest  and  a posttest  were  administered  to  measure  and  compare 
between  the  pre-knowledge  levels  and  the  learning  outcomes.  All  t- tests  revealed  that  the  posttest  scores  were 
significantly  different  from  the  pretest  scores  (see  Table  2). 


Semester 

Means  (Learning  Outcomes) 

t-test  results 

Spring,  1997 

M (pre)  = 24.58,  M (post)  = 36.83 

t (1 1)  = -20.61,  p<. 01 

Summer,  1997 

M (pre)  = 25.59,  M (post)  = 34.78 

t(16)  = -7.64,p<.01 

Fall,  1997 

M (pre)  = 25.38,  M (post)  = 34.81 

t(15)  = -9.25,p<.01 

Table  2.  Average  Pretest  and  Posttest  Scores  and  t-test  Results 
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6.3.  Dropout  Rate  (Level  3 and  4) 


Between  the  fall  semester  of  1989  and  the  fall  semester  of  1996,  the  average  dropout  rate  by  the  third 
course  was  44%.  In  other  words,  44%  of  DE  students  dropped  out  of  the  program  by  their  third  course.  At  the  end 
of  1 997,  a year  after  the  interventions  were  implemented,  the  dropout  rate  decreased  down  to  22%  (Fenner,  1 998). 
Twenty-two  percentage  of  the  dropout  rate  from  44%  is  a significant  decrease. 

Among  those  who  dropped  out  of  the  program,  three  students  dropped  out  after  the  first  week  of  the  course 
due  to  hardware  and  software  incompatibility  problems.  Six  students  cited  that  they  decided  not  to  continue  the 
program  because  their  professional  goals  and  the  degree  program  did  not  match.  Other  reasons  cited  were  a health 
problem  and  time  constraint. 

7.  Conclusions  and  Educational  Implications 

This  case  study  revealed  how  systemic  and  systematic  ID  models  such  as  the  OEM  and  the  ARCS  model, 
and  Kirkpatrick’s  evaluation  model  guided  throughout  the  processes  of  solving  the  dropout  problem  in  the  IPT 
distance  education  program  at  Boise  State  University.  The  impact  was  significantly  positive  in  terms  of  the  benefits 
to  the  educational  organization  itself  as  well  as  to  the  learners  themselves.  Due  to  the  educational  organization's 
efforts  of  designing  and  implementing  effective  interventions  based  on  the  OEM  framework  and  the  ARCS  model, 
the  DE  students  perceived  the  delivered  instruction  as  more  interesting  and  relevant  to  their  professional  and 
personal  goals.  They  were  confident  in  learning  and  satisfied  with  the  distance  learning  environment.  Due  to  the 
effective  instructional  inputs  and  processes,  DE  students  achieved  significant  learning  outcomes.  Due  to  the  positive 
reaction  to  the  instructional  inputs,  learning  processes,  and  successful  learning  outcomes,  they  more  likely  decided 
to  continue  to  learn  via  distance  education.  As  a result,  the  educational  organization  significantly  decreased  the 
previous  dropout  rate.  It  is  cost-efficient  to  decrease  dropout  rates  to  maintain  the  number  of  enrollment,  instead  of 
trying  to  recruit  new  students.  This  case  study  revealed  how  effective  means  lead  to  desirable  ends  within  an 
educational  organization. 
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Abstract:  The  increasing  utilization  of  the  World  Wide  Web  in  higher  education  allows 

instructors  to  re  examine  pedagogical  strategies  and  explore  ways  of  taking  advantage  of  the 
Web's  potential  to  provide  for  learning  experiences  that  go  beyond  that  possible  in  the  traditional 
classroom  environment.  Assumptions  on  how  this  enhances  the  learning  experience  for  students 
require  examination.  This  paper  discusses  a study  which  examines,  among  other  issues,  student 
perceptions  of  the  learning  experience  when  asynchronous,  Web-based,  collaborative  tutorial 
activities  are  utilized  within  an  undergraduate  health  education  subject.  Analysis  of  the  study 
data  demonstrates  that  students'  perceptions  of  the  effectiveness  of  the  Web-based  tutorials  lie  in 
their  appreciation  of  opportunities  for  flexibility,  reflection,  and  self-directed  investigation  within 
the  learning  environment. 


Introduction 

The  increasing  utilization  of  the  World  Wide  Web  in  higher  education  allows  instructors  to  re  examine 
pedagogical  strategies  and  explore  ways  of  taking  advantage  of  the  Web's  potential  to  provide  for  learning 
experiences  that  go  beyond  that  possible  in  the  traditional  classroom  environment.  Our  assumptions  on  how  this 
enhances  the  learning  experience  for  students  require  examination. 

In  an  effort  to  investigate  the  use  of  Web-based  instruction  in  the  delivery  of  health  education  at  the 
undergraduate  level,  the  Faculty  of  Education  at  the  University  of  Wollongong  implemented  Web-based  learning 
activities  as  part  of  the  tutorial  component  within  its  Health  and  Health  Behaviour  subject  (a  core  subject  for 
students  enrolled  in  a pre-service  teaching  Bachelor  of  Physical  and  Health  Education  degree  program).  This  study 
examined,  among  other  issues,  the  perceptions  of  the  learning  experience  for  students  engaging  in  these  Web- 
based  collaborative  tutorial  activities. 

This  paper  discusses  the  design  and  development  of  a Web-based  learning  environment  with  embedded 
collaborative  tutorial  activities;  methodology  and  implementation  process,  and  preliminary  findings  of  student 
perceptions  of  the  Web-based  learning  experience. 


Theoretical  Background 


Important  components  of  the  learning  process  are  discourse  and  learner  interaction.  Duffy  and 
Cunningham  (1996)  define  learning  as  a "social,  communicative,  and  discursive  process,  inexorably  grounded  in 
talk"  (p.  181).  Dialogue  and  discourse  encourages  the  higher  order  thinking  skills  of  cognitive  conflict  and 
resolution  in  providing  context  and  a mechanism  for  explanation,  justification  and  reason  (Oliver,  Omari  & 
Herrington,  1997).  Learners’  interactions  with  instructors  and  other  learners  "give  them  perspective,  place  them 
within  a community  of  learning,  and  contribute  to  their  mastery  of  concepts  and  skills"  (Price  & Petre,  1997,  p. 
869). 
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This  essential  component  of  learner  interaction  and  discussion  in  the  learning  process  focused  our  attention  on 
learner  discourse  through  collaborative  activities.  The  questions  that  guided  the  study  included: 

• How  do  learners  participate  in  and  contribute  to  collaborative  health  education  learning  activities  within  the 
class  and  Web  environments? 

• What  knowledge,  attitude  and  behaviour  change  occurs  when  students  engage  in  collaborative  health 
education  learning  activities  within  class  and  Web  environments? 

• What  are  learners'  perceptions  of  the  effectiveness  of  class  and  Web  environments  for  health  education? 


Learning  Environment  Design  and  Development 

Phase  one  of  the  study  involved  the  development  and  formative  evaluation  of  collaborative  learning 
activities  and  a prototype  Web  learning  environment. 

For  the  purposes  of  investigating  the  impact  of  implementation  of  Web-based  learning  within  the  subject, 
six  online  learning  activities  were  developed.  During  the  first  half  of  session,  concurrent  class  and  online  tutorials 
focused  on  issues  related  to  HIV/AIDS. 

The  literature  suggests  the  pedagogical  benefits  of  asynchronous  communication  includes  the  opportunity  for 
learner  reflection,  self  directed  learning,  and  learner  participation  in  topic  negotiation  and  communications  control 
(i.e.,  subject  matter  discussed,  number  of  topics  discussed,  and,  the  speed  at  which  communication  about  topics 
occurs)  (Laurillard  1993;  Romiszowski  & Mason,  1996).  With  this  in  mind,  Web-based  learning  activities  were 
designed  for  asynchronous  participation. 

All  Web-based  and  class-based  learning  activities  were  structured  to  allow  for  small  group  discussion  and 
collaboration.  The  general  format  for  the  structure  of  each  learning  activity  was  to  provide  a stimulus  to  discussion 
in  small  groups  (e.g.,  a survey  to  be  answered,  questions  to  be  considered,  etc.)  with  each  group  arriving  at  some 
consensus  or  shared  understanding  that  could  be  communicated  to  the  larger  class  during  the  subsequent  lecture.  In 
Duffy  and  Cunningham’s  (1996)  terms,  the  learning  strategy  involves  a problem  which  provides  stimulus  for  an 
authentic  activity  where  knowledge  is  developed  through  working  on  the  problem. 

Once  the  learning  activities  were  designed,  the  Web  learning  environment  was  designed  and  developed.  A 
guiding  factor  in  the  design  of  the  Web-based  learning  environment  was  consideration  of  the  computer  and  Web 
experience  of  the  students  enrolled  in  the  subject.  Based  on  history  with  similar  cohorts,  it  was  expected  that  there 
would  be  a range  of  computer  experience  among  the  students  with  the  majority  having  little  experience  with 
computers.  Additionally,  it  was  expected  that  there  would  be  a range  of  Web  use  experience  with  the  majority  of 
the  students  having  either  none  or  little  experience  in  using  the  Web.  Brown  and  Thompson  (1997)  argue  that 
"interface  design  must  provide  ease  of  navigation,  a sense  of  human  interaction,  helpfulness  and  responsiveness  to 
the  needs  of  learners  studying  in  an  information  rich,  self-directed  medium."  (p.  78)  As  such,  particular  attention 
was  paid  to  constructing  a Web  environment  with  a simple  structure  with  embedded  communication  tools  that  were 
easy  to  use. 

The  Web  site  structure  included  four  main  components:  (1)  the  subject  outline  which  provided 
information  such  as  the  rationale,  objectives,  content,  presentation,  assessment,  and  participation  expectations  for 
the  subject;  (2)  the  subject  schedule  which  provided,  in  table  format,  the  week-by-week  timetable  of  lecture  topics 
and  tutorial  activities;  (3)  resources  and  Web  links  which  included  links  to  a number  of  Australian  and 
international  Web  sites  related  to  the  topics  covered  in  the  subject;  and,  most  importantly,  (4)  the  Activity  Centre 
which  facilitated  the  learning  activities  for  the  students  who  were  participating  in  Web-based  tutorials. 

The  Activity  Centre  was  designed  using  a structure  based  on  the  suggestion  that  Web-based  learning  environments 
should  put  minimal  cognitive  load  on  the  student  and  that  templates  be  utilized  in  screen  design  which  "promotes 
understanding  by  allowing  the  reader  to  focus  on  new  information  rather  than  devoting  time  and  energy  to 
variations  of  format"  (El-Tigi  and  Maribe  Branch,  1997,  p.25). 

The  initial  structure  of  each  activity  consisted  of  two  or  three  Web  pages.  The  Things  To  Do  page  outlined 
the  activity  and  provided  stimulus  to  the  group  discussion.  Examples  of  discussion  stimuli  utilized  in  the  learning 
activities  included:  a questionnaire  (e.g..  In  what  situations  can  HIV  be  transmitted?)  to  be  answered  and  discussed 
by  the  group  and  a narrative  that  provided  an  example  for  groups  to  develop  their  own  story  (e.g.,  for  risk  factors  of 
HIV/AIDS).  In  cases  where  an  online  questionnaire  provided  stimuli  to  the  collaborative  learning  activity,  results 
of  the  individual  group  members  were  provided  on  a separate.  Results,  page.  This  facilitated  the  sharing  of  ideas 
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among  group  members  and  provided  a basis  for  their  discussion.  The  Discussion  Area  for  each  activity  was 
designed  to  be  the  focal  learning  area.  The  comments  submitted  to  this  page  were  viewable  only  by  group  members 
and  were  displayed  in  chronological  order  on  the  page. 


Formative  Evaluation 

The  formative  evaluation  of  the  learning  environment,  with  embedded  activities,  involved  a multi-faceted 
approach  including:  review  by  experts  in  the  areas  of  health  education  and  instructional  technology;  review  by 
instructional  technology  postgraduate  students;  and  testing  by  undergraduate  teacher  trainees. 

After  using  the  Web  site,  the  health  education  and  instructional  technology  experts  were  interviewed  and 
instructional  technology  postgraduate  students  were  surveyed  regarding:  Web  environment  interface  issues;  clarity 
and  quality  of  information  and  external  links;  and,  perceived  pedagogical  effectiveness  of  the  learning  activities. 
Undergraduate  students  were  asked  to  fill  out  a one-page  survey  of  questions  regarding  Web  site  interface  issues 
and  usability. 

The  Web  learning  environment  was  revised,  based  on  analysis  of  the  data  collected  during  the  formative 
evaluation,  with  the  majority  of  changes  to  the  Web  site  focused  on  the  individual  learning  activities  within  the 
Activity  Centre.  Specific  reviewer  suggestions  to  describe  learning  activities  and  tasks  in  more  detail  resulted  in  a 
clarification  of  the  structure  of  each  learning  activity  and  thus,  a proliferation  of  pages  for  each  activity. 

In  the  revised  version  of  the  learning  environment,  each  learning  activity  was  associated  with  five  main 
areas.  The  This  Week's  Task  Into  page  introduced  the  student  to  the  activity,  related  the  activity  to  the  specific 
lecture,  provided  additional  conceptual  information  and  stimulus  into  the  group  activity  by  asking  the  student  to 
respond  to  some  initial,  topical  questions.  Once  the  student  submitted  their  response  to  these  initial  questions,  they 
were  automatically  moved  to  the  Group  Tasks  page.  Here,  the  student  could  read  a detailed  description  of  the  group 
task  for  that  particular  learning  activity.  The  student  also  had  the  opportunity  to  view  their  responses  and  the 
responses  of  the  other  members  in  their  group  to  the  initial,  topical  questions  by  accessing  the  Individual 
Responses  page.  The  earlier  version  of  the  Discussion  Area  was  maintained,  however,  based  on  reviewers 
suggestions,  an  abbreviated  list  of  the  group  tasks  was  added  to  the  top  of  the  page.  A Submit  Page  form  was  also 
added,  to  the  structure  of  each  activity,  as  the  area  in  which  the  group  submitted  their  final  product.  All  five  areas 
of  the  individual  activities  were  available  to  the  student  at  all  times  (i.e.,  students  are  not  forced  to  follow  a 
particular  page  sequence  when  engaging  in  the  activity). 


Study  Methodology  and  Implementation  Process 

The  study  was  implemented  in  July  1998  within  the  Health  and  Health  Behaviour  subject.  Sixty-two 
students  were  enrolled  in  the  subject  with  the  majority  of  students  (57)  registered  in  their  first  year  of  the  Faculty’s 
Bachelor  of  Education  in  Physical  and  Health  Education  degree  program.  The  subject  was  conducted  over  a 14 
week  schedule. 

During  implementation  of  the  study,  a crossover  factorial  design  was  utilized  where  students  were 
randomly  assigned  into  two  tutorial  groups  and  each  group  then  randomly  assigned  to  one  of  two  learning 
environments  (class  or  Web)  for  the  first  half  of  session.  In  the  second  half  of  session,  groups  crossed  over  to 
engage  in  their  tutorials  within  the  alternate  learning  environment.  Both  tutorial  groups  were  further,  randomly, 
divided  into  working  groups  (seven  working  groups  in  each  of  the  two  tutorial  groups)  consisting  of  four  or  five 
students. 

All  tutorial  sessions  (conducted  either  in  the  class  or  in  the  Web-based  environment)  were  associated  with 
an  assessable  activity  task  for  which  each  working  group  was  required  to  submit  a final  group  product  prior  to  the 
next  lecture. 

During  the  first  week  of  session,  students  were  advised  of  the  subject  Web  site  and  its  location 
(http://www.immll.uow.edu.au/UG/EDUP144/).  At  that  point,  all  students  could  access  the  subject  information  and 
resources  but  could  not  access  the  Activity  Centre  where  Web-based  tutorials  were  located. 

The  first  Web-based  tutorials  were  implemented  in  the  fourth  week  of  session  while  the  second  set  of 
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Web-based  tutorials  were  implemented  in  the  eleventh  week  of  session.  This  time  frame  allowed  students  to 
develop  a working  relationship  with  their  tutorial  work  group;  become  accustomed  to  the  process  of  completing 
and  submitting  their  group  work  within  the  weekly  time  frame;  and,  complete  three  weeks  of  Web  skilling 
exercises  within  a Information  Technology  subject.  During  the  week  prior  to  commencing  Web-based  tutorials, 
students  were  provided  with  a brief  presentation  as  an  introduction  to  the  Web  site  and  the  structure  of  the  learning 
activities.  At  this  time,  students  were  also  given  an  individual  username  and  password  to  access  the  Activity 
Centre. 

Pre-tests  and  post-tests  for  knowledge,  attitude,  and  behaviour  related  to  the  health  topics  covered  by  the 
learning  activities  (HIV/AIDS  and  nutrition),  and  computer  literacy  and  computer  comfort,  as  well  as  a post-test 
which  included  items  related  to  perception  of  the  learning  environments  were  administered  to  students.  The 
collaborative  learning  activities  were  recorded  (via  audio  tape  and  electronic  Web  logs). 

Once  learning  activities  were  complete,  a representative  of  each  working  group  was  chosen,  at  random, 
and  asked  to  engage  in  an  in-depth  interview  regarding  their  experience  and  perceptions  of  the  class  and  Web 
learning  environments.  Specifically,  the  interview  sessions  gathered  information  from  the  students  regarding:  how 
their  group  process  was  managed  during  weeks  when  tutorials  were  held  in  the  classroom  and  when  they  were 
conducted  within  the  Web-leaming  environment;  their  perceptions  of  engaging  in  group-based  tutorials  within  the 
Web  learning  environment  in  general  and  the  individual  learning  activities  specifically;  their  perceptions  of  the 
effectiveness  of  the  Web-based  tutorials  compared  to  class-based  tutorials  in  terms  of  their  own  learning;  and,  their 
experience  in  terms  of  group  interaction  in  both  the  Web  and  classroom  environments. 


Findings 

An  analysis  of  the  in-depth  interviews  conducted  with  students  who  engaged  in  the  Web-based  learning 
environment  during  the  first  half  of  session  (seven  students  representing  each  of  seven  tutorial  working  groups)  is 
presented  as  preliminary  findings. 

All  students  expressed  positive  opinions  on  their  experience  of  engaging  in  the  Web-based  tutorials. 
While  one  student  had  previously  experienced  a subject  which  was  supported  with  Web-based  learning  materials, 
none  of  the  students  had  previously  engaged  in  collaborative  group  learning  activities  using  the  Web.  They  felt  it 
was  a "novelty”;  a "...buzz  to  sit  down  and  talk  to  three  other  people  in  different  places”;  and  that  ”...  it  intrigued 
you  because  it  was  new  and  because  it  was  different.”  All  the  students  mentioned  the  benefit  of  the  "convenience" 
and  "flexibility"  of  engaging  in  the  Web-based  tutorials. 

Most  students  (six  of  seven)  felt  that  the  Web-based  tutorials  were  more  effective  in  terms  of  facilitating 
their  own  learning  than  the  classroom-based  tutorials.  Some  suggested  that  this  was  due  to  the  fact  that,  in  the  Web 
environment,  they  had  time  to  research  the  information  (either  in  texts  or  via  external  Web  sites)  and  reflect  on  the 
group  task.  One  student’s  explanation  of  the  experience  is  indicative  of  that  expressed  by  many  of  her  peers,  “... 
when  you’re  all  rushed,  you  won’t  concentrate  . But  ,when  you've  got  time  for  it,  you  can  sit  down,  think  about  it  . 
A couple  of  times  I’d  go  off  and  read  some  books  that  I had  at  home  to  help  me  a bit  and  make  me  think  more 
about  the  answer.  Then  I could  come  back  to  it  whereas,  when  you  are  in  the  class  you’ve  got  to  know  straight 
away,  right  there.  I mean.  I’d  go  onto  the  Internet  and  try  to  find  some  information  to  give  me  some  ideas  and  then 
I'd  go  back  on  and  write  my  answers...  so  I thought  that  was  good.” 

Each  student  was  asked  to  comment  on  and  compare  the  participation  and  interaction  among  their  work 
group  members  in  both  the  class-based  and  the  Web-based  tutorials.  Only  two  students  reported  that,  during  class- 
based  weeks,  their  group  actually  worked  on  the  entire  task  as  a whole  group.  The  other  five  students  reported  that 
their  group  tended  to  separate  the  task  into  small  parts  and  assign  those  parts  to  individuals.  Then,  one  member  of 
the  group  would  be  responsible  for  pulling  together  the  individual  parts  into  a final  product. 

This  did  not  seem  to  be  the  case  for  the  weeks  in  which  the  tutorials  were  conducted  in  the  Web-based 
environment.  All  students  noted  that,  during  these  weeks,  all  group  members  contributed  by  completing  the 
individual  introduction  page  and  interacting  within  the  Discussion  Area  on  all  aspects  of  the  task.  To  complete  the 
group  Submit  Page,  three  of  the  seven  students  noted  that  their  group  met  around  one  computer  and  completed  the 
page  together.  The  other  four  students  noted  that  their  group  delegated  the  responsibility  of  synthesizing  the  group 
discussion  to  one  group  member. 

In  comparing  the  nature  of  the  interactions  that  took  place  in  the  two  environments,  most  students  (five 


out  of  seven)  felt  that  more  detailed  contributions  were  made  within  the  Web-based  environment.  One  student 
noted  that,  “1  think  it  makes  you  express  what  you  think  more  because  you're  not  talking.  When  we  would  meet  to 
talk...  one  person  would  say  what  they  think  then  the  others  would  just  say,  'yeah,  I agree’.”  Another  student  felt 
this  detailed  contribution  in  the  Web-based  tutorials  was  due  do  the  fact  that  the  Web  takes  away  the  pressure  of 
the  face-to-face  environment,  “Some  people  would  be  sort  of  afraid  to  come  out  and  say  things.  But  on  the  Web, 
people  aren't  directly  looking  at  you  and  you're  not  the  focus  of  attention.  On  the  Web,  in  great  detail,  you  can  tell 
someone  about  something.  But  in  the  classroom  you  don't  tend  to  go  into  a lot  of  detail...  I think  for  people  who  are 
reluctant  to  let  everything  out  in  the  classroom,  [the  Web]  is  a really  good  opportunity,  you  don't  have  any  pressure 
on  you.” 

This  student  went  on  to  explain  that  she  noticed  a difference  in  her  own  contribution,  “I  think  I 
contributed  a lot  more  on  the  Web  than  I do  in  the  classroom,  1 tend  to  sit  back  and  listen  and  just  throw  something 
in  every  now  and  then...  In  the  classroom,  if  you've  got  very  talkative  people  in  your  group,  chances  that  you'll  sit 
back  and  let  them  run  the  show.  But  when  you're  on  the  Web  you're  free  to  make  a huge  contribution  if  you  want 
to,  so  you  write  as  much  as  you  want ...  you  say  a lot  more  on  the  Web  than  you  would  in  the  class.” 

When  asked  what,  if  anything,  the  classroom  tutorials  provide  that  was  unavailable  in  the  asynchronous 
Web-based  environment,  some  students  mentioned  the  ability  to  ask  questions  of  the  tutor  or  another  student  "there 
and  then". 

The  students  were  asked  to  think  about  their  own  skills  in  using  computers  and  the  Web  and  the 
appropriateness  of  the  introduction  to  the  Web-based  environment  which  they  had  received.  Four  of  the  seven 
students  interviewed  said  that  they  felt  competent  about  using  computers  and  the  Web  prior  to  engaging  in  the  Web 
tutorials.  These  students  noted  that  the  brief  introduction  to  the  Web  site  was  enough  to  get  them  started  on  the 
Web-based  tutorials.  They  all  felt  that  the  structure  of  the  Web-based  tutorials  was  very  easy  to  understand. 

The  other  three  students  who  were  novice  computer  and  Web  users  also  felt  that  the  structure  of  the  Web 
environment  was  very  easy  to  understand  and  that  they  were  adequately  equipped  by  the  prior  Web  skilling 
exercises  in  the  Information  Technology  subject  to  be  able  to  engage  in  their  Web-based  tutorials.  These  students 
felt  that  the  experience  of  engaging  in  the  Web-based  tutorials  not  only  increased  their  computer  and  Web  skills, 
but  also  allowed  them  to  feel  more  "relaxed"  using  the  technology.  One  of  the  students  expressed  her  feelings  in 
this  way,  “1  was  a bit  nervous,  1 mean,  1 know  computers  are  going  to  be  important  so  I've  wanted  to  do  it,  but  I 
haven't  had  the  confidence...  actually  using  them  for  something  constructive.  I've  got  the  confidence  now...  a big 
positive.” 

Students  commented  that  felt  they  would  benefit  from  suggestions  on  how  to  better  manage  the  group 
process  when  engaging  in  Web-based,  asynchronous  tutorials.  Some  students  mentioned  instances  when  they  were 
unsure  when  their  work  group  members  would  be  logging  on  to  the  Web  site  or  how  the  final  group  product  was 
going  to  be  organized.  However,  it  is  interesting  to  note  that  many  students  also  reported  that  they  experienced 
similar  process  issues  in  the  class-based  tutorials. 


Discussion 

Analysis  of  this  sample  of  data  demonstrated  a degree  of  perceived  effectiveness  for  the  implementation  of 
the  Web-based  collaborative  learning  activities  within  an  undergraduate  health  education  subject. 

The  majority  of  students  perceived  Web-based  tutorials  to  be  more  effective  for  learning  than  class-based 
tutorials.  They  suggested  that  contributing  factors  to  their  learning  process  in  the  asynchronous  Web-based 
environment  was  the  opportunity  to  research  information  using  print  and  electronic  resources  and  reflect  on  their 
ideas  and  the  contributions  of  their  work  group  peers.  This  supported  the  literature  (Laurillard  1993;  Romiszowski 
& Mason,  1996)  on  the  pedagogical  benefits  of  learner  reflection  and  self-direction  afforded  by  asynchronous 
learning  environments  . 

The  students'  perceived  that  the  quantity  and  quality  of  participation  and  interaction  among  their  work 
group  was  higher  in  the  Web-based  environment.  This  reinforced  the  literature  on  computer-mediated 
communications  (CMC)  which  suggests  that  collaborations  within  this  environment  can  be  more  powerful  than 
face-to-face  encounters  (Romiszowski  & Mason,  1996). 

Full  quantitative  and  qualitative  measurement  of  the  effectiveness  of  the  of  the  Web-based  tutorials  in 
terms  of  learning  will  be  substantiated  with  comparisons  of  the  pre  and  post  knowledge,  attitude  and  behaviour 


tests  and  with  in-depth  content  and  discourse  analysis  of  the  recorded  learner  interactions  in  both  class  and  Web 
environments. 

Final  analysis  of  the  data  from  the  study  are  expected  to  provide  a fuller  understanding  of:  effective  Web-based 
teaching  and  learning  strategies  for  health  education  at  the  tertiary  level;  the  efficacy  of  the  World  Wide  Web  in 
delivering  health  education;  how  discourse-based  knowledge  construction  and  attitude  development  is  supported  in 
a Web-based  learning  environment;  and,  design  issues  for  Web-based  learning  environments. 
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Abstract:  Computer-Supported  Cooperative  Learning  (CSCL)  is  a research  area 
that  studies  how  technology  can  support  the  learning  processes  promoted  through 
collaborative  efforts  among  students  working  in  a given  task.  The  objective  of 
this  work  is  to  trace  a profile  with  the  main  issues  related  to  CSCL  environments. 
For  this,  a framework  is  proposed  to  organize  these  issues,  and  a report  is  made, 
including  available  environments  and  their  classification  in  agreement  with  the 
framework. 


1.  Introduction 

Cooperative  learning  is  a technique  with  which  students  are  supported  in  learning  process,  working 
together  with  other  students  and  the  teacher.  Cooperative  learning  should  be  based  on:  (1)  individual 
responsibility  for  the  information  gathered;  (2)  positive  interdependence,  so  that  the  students  feel  that 
nobody  will  be  well  succeeded  , unless  everybody  is;  (3)  better  form  of  understanding  a given 
material;  (4)  development  of  interpersonal  skills,  which  will  be  necessary  in  future  life;  (5) 
development  of  skills  to  analyze  the  dynamics  of  a group  and  to  work  about  problems;  (6)  a way  to 
increase  involvement  of  students  in  proposed  activities;  and  (7)  an  interesting  and  entertaining  focus. 

Computer-supported  cooperative  learning  (CSCL)  aims  to  effort  the  learning  process,  using  systems 
that  implement  a cooperation  environment  playing  an  active  role  in  analysis  and  control  of  it.  The 
collaborative  technologies  allow  production  of  shared  knowledge  and  new  community  practices. 
Several  subjects  of  educational  and  technological  order,  related  to  each  other,  are  involved  in  the 
construction  and  implementation  of  such  environments.  For  example,  to  determine  the  kind  of 
communication  technology  to  be  used  in  an  environment  will  depend  on  the  educational  objectives 
pointed  by  the  learning  theory  adopted  by  it.  So,  the  set  of  features  of  an  environment  will  establish 
its  application  and  effectiveness. 

In  this  context,  a framework  for  CSCL  environments*  analysis  appears  as  important  conceptual  maps, 
supplying  guidelines  for  research  and  development.  However,  available  frameworks  are  incomplete 
and  do  not  consider  important  dimensions  of  this  kind  of  environment.  The  present  work  gathers  the 
main  aspects  of  CSCL  environments  and  proposes  a framework  for  its  classification. 

2.  Issues  Related  to  Computer-Supported  Cooperative  Learning 

In  this  section,  we  discuss  the  most  important  issues  that  should  be  observed  in  CSCL  environments. 


er|c 


98 


^ Learning  Theories  - One  of  the  most  important  factors  that  regulates  the  collaboration  is  the 
learning  theory  on  which  the  cooperative  interaction  will  be  based.  The  learning  theories  aim  to 
recognize  the  dynamics  involved  in  the  acts  of  teaching  and  learning,  and  try  to  explain  the 
relationship  between  the  pre-existent  knowledge  and  the  new  knowledge.  Learning  is  not  supposed  to 
be  just  intelligence  and  knowledge  construction,  but  basically  personal  identification  and  relationship 
through  the  interaction  with  other  people.  CSCL  environments  should  provide  both  dimensions  and 
bring  other  pertinent  factors  in  the  human  mediation  through  the  technology.  Several  theories 
contribute  to  the  understanding  of  cooperative  learning.  These  theories  have  in  common  the 
assumption  that  individuals  are  active  agents  in  the  search  and  knowledge  construction  within  a 
significant  context.  In  Table  1,  the  main  characteristics  of  learning  theories  related  to  cooperation, 
are  summarized. 


Learning  Theories 

Characteristics 

Genetic  Epistemology  of  Piaget 

Central  point:  cognitive  structures  of  the  individual. 

Development  facilitated  by  the  offer  of  challenges. 

Social  interaction  and  interchanges  among  individuals  stimulate  the 
process  of  knowledge  acquisition. 

Constructivist  Theory  of  Bruner 

Leaner  is  active  in  the  process  of  knowledge  acquisition. 
Determination  of  more  effective  sequences  of  material  presentation. 
Contemporary  theory:  to  create  closer  learning  communities  of 
collaborative  practice  in  the  real  world.  (Kearsley,  1998) 

Partner-cultural  Theory  of  Vygotsky 

Cognitive  development  is  limited  to  a certain  potential  for  each  range 
of  ages  (Proximal  Zone  of  Development). 

Complete  cognitive  development  requests  social  interaction. 

Problem  Based  Learning/  Anchored 
Instruction 

Learning  begins  with  a problem  to  be  solved  (anchor  or  focus). 
Centered  in  the  apprentice  and  the  context. 

Distributed  Cognition 

Interaction  among  individuals,  environment  and  cultural  engines. 
Reciprocal  Teaching. 

Important  role  of  the  technology. 

Theory  of  the  Cognitive  Flexibility 

Knowledge  restructuring  as  answer  to  context  demands. 
Re-visitations  to  the  instructional  material.  (Spiro  at  al.,  1992) 
Activities  should  contain  multiple  representations  of  the  content. 
Sources  of  interconnected  and  distributed  knowledge. 

Situated  Cognition 

Learning  happens  in  function  of  the  activity,  context  and  culture  of 
the  social  environment  in  which  is  inserted. 

Social  interaction  and  collaboration  are  critical  components  for 
learning  (community  of  practice).  (Lave,  1988) 

Self  regulated  Learning  / 

Metacognition 

Control  and  monitor  of  the  own  cognition  by  the  individual. 
Self-observation,  self-judgment,  self-reaction. 

Table  1:  Learning  Theories 


'^Models  of  Cooperation  or  Kinds  of  Tasks  for  Cooperative  Learning  - The  computational 
support  in  learning  process  can  be  used  to  treat  the  cooperation  in  different  kinds  of  tasks.  These  tasks 
are  one  of  the  factors  that  will  determine  the  cooperation  model  proposed  by  the  Environment,  and 
they  can  be  enumerated  independently  of  the  subject  domain  that  is  being  taught.  Kumar  identifies 
three  kinds  of  cooperative  tasks:  concepts  learning,  problems  solving  and  projects*  development 
(Kumar,  1996).  Besides  these  kinds,  an  environment  can  be  based  on  Forums  of  Discussions,  where 
the  subjects  refer  to  the  participants*  behavior,  such  as,  “When  do  the  students  read  notes?”,  “When 
do  they  write  notes?”,  “What  is  the  participation  level  in  the  class?”. 

'^Domains  - Cooperative  learning  is  generally  more  effective  in  domains  where  people  are  engaged 
in  skills’  acquisition,  categorization,  group  planning  and  tasks  that  request  construction  of  collective 
memory  (Kumar,  1996).  Studies  proposing  cooperative  learning  to  help  students  to  understand 
complex  subjects  in  specific  domain*s  environments  (e.g.,  scientific  thought)  present  good  results.  In 
procedural  tasks  that  do  not  involve  deep  understanding,  becomes  more  difficult  to  observe  the 
occurrence  of  conceptual  changes,  suggesting  that  there  are  “more”  and  “minus”  shared  domains. 

'^Technologies  - Two  kinds  of  technologies  can  be  used  in  cooperative  learning  environment: 
asynchronous  and  synchronous  communication.  The  use  of  one  or  other  or  of  their  combination  will 
determine  the  interaction  degree.  The  need  of  different  communication  possibilities  will  depend  on 
the  model  proposed  by  the  environment. 
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Interaction  degrees  can  be  small  (there  is  a low  degree  of  interaction  among  involved  participants. 
For  example,  interaction  happens  only  in  the  information  sharing  activities);  medium  (the  interaction 
degree  is  a bit  bigger,  but  it  is  already  far  from  the  effects  of  face-to-face  interactions.  For  example: 
interaction  happens  when  people  use  e-mail  for  communication);  and  big  (involved  people  need  to 
know  more  deeply  the  other  participants  and  with  this  purpose  they  do  social  changes.  For  example, 
interaction  happens  when  people  use  synchronous  video-conferencing). 

v^Activities  of  Cooperative  Work  - Depending  on  the  learning  environment  proposal,  it  can  support 
a wide  range  of  activities,  such  as:  coordination  of  activities;  negotiation  and  decision  making; 
representation  of  group  knowledge;  sharing  of  a database  (group  memory);  perception  of  the  presence 
and  the  other  participants'  actions  (awareness). 

v^Role  Assignment  - Some  systems  explicitly  assign  different  roles  for  the  users,  which  will  have 
distinct  rights  and  specific  tasks  to  carry  on  in  the  learning  process.  For  example,  teacher/tutor  role 
and  student/leamer  role. 

Implementation/Platform  - The  platform  and  resources  used  in  a system  development  define  for 
example,  its  type  of  users. 


3.  Framework  for  Study  of  CSCL  Environments 

Starting  from  the  discussed  issues,  a framework  is  proposed  for  the  study  and  classification  of  CSCL 
environments. 


Issues 

Possibilities 

Theory  of  Learning 

Genetic  Epistemology  of  Piaget 
Constructivist  Theory  of  Bruner 
Partner-cultural  Theory  of  Vygotsky 
Problem  Based  Learn ing/Anchored  Instruction 
Distributed  Cognition 
Theory  of  the  Cognitive  Flexibility 
Situated  Cognition 

Self-regulated  Learning  / Mctacognition 

Model  of  Cooperation  or  Kind  of  Task 

concepts  learning 
problem  solution 
project  development 
knowledge  construction 
forum  of  discussions 

Domain 

development  of  the  scientific  critical  thought 
ecological  models 
scientific  texts 
non  specific  domain 

Kind  of  Interaction 

asynchronous 

synchronous 

Quality  or  Degree  of  Interaction 

small 

medium 

big 

Activities  of  Cooperative  Work 

coordination  of  activities 
negotiation  and  decision  making 
knowledge  representation 
group  memory 
awareness 

Role  Assignment 

tcachcr/instructor 

student/leamer 

Implementation/Platforms 

UNIX 

Windows  NT 

Macintosh 

WWW 

Table  2:  Framework  for  Study  of  CSCL  Environments 


4.  Computer-Supported  Cooperative  Learning  Environments;  Brief  State-of- 
the-Art 
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A study  of  available  Computer-Supported  Cooperative  Learning  Environments  took  place,  aiming  to 
classify  them  according  to  the  proposed  framework. 


4.1.  N.I.C.E.:  Narrative,  Immersive,  Constructionist/Collaborative  Environments 

NICE’S  objective  is  the  construction  of  virtual  learning  environments  for  children,  based  on  narrative 
theories,  construcionism  and  collaboration.  The  system  was  projected  to  be  executed  in  CAVE,  which 
is  a virtual  reality  environment,  room-sized  where  several  people  can  move  freely,  both  physic  and 
virtually  (Roussos  at  al.,  1997).  A theoretical  framework  that  combines  ideas  of  the  constructivist 
learning  theory,  narrative  techniques  and  collaboration  provides  the  basis  for  it.  Considering  the 
theory  of  Piaget  and  in  the  ideas  of  Dewey,  the  constructivism  is  related  to  the  ways  that  students 
acquire  knowledge  by  participating  in  activities  or  tasks  where  they  are  stimulated  to  build,  to 
manipulate,  and  to  explore  objects. 

In  N.I.C.E.,  it  is  possible  to  accomplish  virtual  construction  blocks  which  contain  characteristics  that 
physical  toys  or  learning  tools  do  not  have:  the  children  can  catch  heavy  or  big  objects,  transfer  them 
to  another  children  remotely  located,  combine  them  in  new  objects,  or  simply  observe  modifications 
in  their  attributes  as  time  goes  by.  All  objects  and  representations  are  VRML  models,  that  can  be 
moved,  increased  or  decreased  by  the  children  in  real  time. 

One  of  the  construction  activity  products  in  NICE  is  the  narrative:  both  the  composed  stories  and 
those  created  by  the  children  who  participated  in  an  interaction  with  the  system  (Roussos  at  al., 
1997).  All  the  actions  that  took  place  within  the  environment  are  added  to  a continually  composed 
story,  even  when  they  do  not  represent  children's  interactions.  The  sequence  of  the  story  goes  through 
a parser,  that  changes  some  words  for  their  iconic  representations  and  publishes  it  in  a WWW  page. 
The  collaboration  in  NICE  is  emphasized  by  the  interaction  of  virtual  communities  (students 
geographically  separated)  and  physic  ones  (students  in  the  same  physical  space),  and  it  could  involve 
verbalization,  collective  decisions,  conflicts  resolution,  reciprocal  teaching  (all  facilitated  by  virtual 
reality  techniques).  NICE’S  main  objectives  are  learning  from  multiple  perspectives;  learning  how  to 
collaborate  with  other  people;  learning  by  controlling  and  exploring  active  variables  of  the 
environment;  programming  for  demonstration;  exploration  of  structures  of  stories;  and  creation  of  a 
final  product. 

4.2.  CSILE;  Computer-Supported  Intentional  Learning  Environments 

CSILE  is  a collective  database  that  supports  students'  ideas  in  textual  or  graphic  format,  available  for 
all  the  participants.  In  this  multimedia  set,  the  students  generate  nodes  with  ideas,  or  part  of  important 
information  to  a topic  in  study.  The  data  are  indexed  and  organized  in  such  a way  that  can  be 
accessed  through  a series  of  channels,  allowing  students  that  are  studying  a topic  in  a certain  domain, 
to  access  information  related  to  it  but  presented  in  another  domain.  The  students  produce  information, 
formulate  subjects,  provide  feedback  and  evaluation,  and  organize  the  knowledge  in  the  database. 
According  to  Gay  (1996),  CSILE  is  based  on  three  research  lines:  intentional  learning  (attempt  to 
actively  reach  an  objective,  different  from  simply  try  to  do  school  activities);  specialization  process 
(process  of  progressive  problems  solution,  moving  besides  the  current  competence  limits);  and 
restructuring  of  Schools  as  communities  of  knowledge  construction.  The  emphasis  of  CSILE  is  in  the 
cooperative  learning,  in  spite  of  the  teacher’s  responsibility  for  this  experience.  The  teacher  and  all 
the  students  can  monitor  the  learning  of  the  other  ones,  answer  the  ideas  of  the  other  ones,  request 
information,  and  make  comments. 

4.3.  CLARE:  Collaborative  Learning  And  Research  Environment 

CLARE  is  a distributed  learning  environment  supported  by  computer,  where  the  objective  is  to 
facilitate  learning  through  the  collaborative  knowledge  construction.  For  this,  CLARE  provides  a 
semi-formal  representation  language  called  RESRA  and  an  explicit  process  model  called  SECAI 
(Wan  and  Johnson,  1994).  RESRA  (Representational  Schema  of  Research  Artifacts)  is  a semi- 
structured  knowledge  representation  language  specifically  projected  to  facilitate  collaborative 
learning  of  scientific  texts.  Wan  and  Johnson  (1994)  describes  the  three  premises  in  which  RESRA 
language  is  based:  (l)the  human  knowledge  can  be  represented  in  terms  of  a small  number  of 
primitive  nodes  and  links;  (2)the  use  of  these  primitives  to  characterize  scientific  artifact  and 
subsequent  activities  in  group  is  significant  to  learning  process,  because  the  learners  should  make 
themselves  many  deep  level  questions  (such  as,  what  hypothesis  is  being  made?.  With  regard  to  what 


problem?,  Is  this  theme  a hypothesis  or  a theory?);  and  (3)different  learners  should  generate  different 
representations  based  on  the  same  artifact,  and  comparing  these  representations,  the  similarities  and 
differences  in  these  points  of  view  can  be  discerned.  A map  of  the  knowledge  structure  reflecting 
students  mental  model  about  the  author’s  of  the  text  intention  will  be  build.  SECAI  (Summarization, 
Evaluation,  Comparison,  Argumentation,  and  Integration)  defines  an  explicit  model  for  a 
collaborative  scientific  texts  learning  process  (Wan  and  Johnson,  1994).  Metaphorically, 
collaborative  learning  with  SECAI  “pulls”  the  learners  from  the  external,  isolated  and  individual 
position  to  the  intern  perspective,  integrated  and  collaborative  in  an  artifact.  As  the  learners  go  by  the 
proposed  activities  in  the  SECAI  model,  the  level  of  collaboration  grows  and  at  the  same  time  a 
knowledge  base  is  formed. 

4.4.  CaMlLE:  Collaborative  and  Multimedia  Interactive  Learning  Environment 

CaMILE  supports  asynchronous  collaboration  on  then  Web  with  the  objective  of  fostering  learning.  It 
is  situated  in  the  CSCL  approach  described  by  Guzdial  (1997)  - analysis  in  a high  level  of 

aggregation:  forums  of  discussions  with  multiple  groups  or  a whole  class.  All  the  accesses  to  the 
system  are  accomplished  through  a Web  browser  that  accesses  a single  server.  System  interface  is 
based  on  forms,  and  it  is  the  same  for  all  the  users. 

The  discussions  in  CaMILE  are  contextualized  as  in  a newsgroup,  even  so,  the  context  is  persistent, 
and  it  is  always  available  for  the  users,  not  “disappearing”  after  the  visualization.  Similar  to  CSILE, 
CaMILE  provides  a facility  in  which  the  students  are  requested  to  identify  the  kind  of  collaboration 
they  are  presenting  (e.g.,  a subject,  a new  idea,  a refutation,  etc.)  and  productive  initial  sentences  are 
suggested  to  be  used  in  each  one  of  these  kinds  of  notes.  The  notes  in  CaMILE  can  contain 
everything  that  a Web  page  can  contain.  In  agreement  with  Guzdial,  an  important  difference  between 
newsgroups  and  CaMILE  is  that  this  environment  supports  anchored  collaboration,  which  means  that 
each  individual  note  can  only  be  referenced  through  a Web  browser  (Guzdial,  1997).  So,  the  notes 
direct  addresses  allow  that  Web  pages  contain  hyperlinks  for  a discussion  context  CaMILE.  The 
anchors  function  as  indexes  and  remainders  of  what  students  discussed  on  a certain  context. 

4.5.  Belvedere 

Belvedere  helps  learners  in  critical  discussion  of  scientific  theories,  based  on  the  collaborative 
paradigm.  It  can  be  summarized  as  a groupware  for  the  construction  of  logical  and  rhetorical 
relationships’  representations  in  a debate,  using  an  interface  that  looks  like  a graphic  editor.  Belvedere 
provides  the  students  with  concrete  forms  of  representing  abstract  components  and  relationships 
between  theories  and  arguments.  Ideas  and  relationships  are  represented  as  objects  that  can  be 
pointed,  tied  up  to  other  objects  and  discussed.  Belvedere  can  be  used  by  students  that  are  physically 
close  to  each  other  working  simultaneously  (synchronous);  students  sharing  arguments  in  different 
times  (asynchronous);  and  students  working  simultaneously,  but  remotely  located  to  each  other. 
According  to  Suthers  (1996),  the  environment  combines  three  approaches  for  learning:  collaborative 
learning,  guided  learning,  and  problem  based  learning.  Each  one  of  these  aspects  is  covered  by  a 
category  of  educational  software  within  the  environment:  groupware  for  learning;  intelligent  tutor; 
and  simulation.  The  software  provides  argument  diagrams  that  dispose  different  geometric  forms  for 
different  kinds  of  arguments  and  their  components,  with  positive  and  negative  links,  multiple  forms 
of  connections,  and  possibilities  of  attaches  to  accommodate  complex  arguments.  Belvedere  also 
provides  means  for  authoring  knowledge  on-line.  This  work  can  be  accessed  and  copied  by  all  the 
students.  The  environment  was  extended  to  serve  as  a WWW  browser,  allowing  authors  to  use 
existent  HTML  tools. 

5.  Classification  of  the  Environments 

The  analyzed  environments  were  classified  in  agreement  with  the  proposed  framework. 


Issues 

Environments 

NICE 

CSILE 

CLARE 

CaMILE 

Belvedere 

Theory  of 

Learning 

Constructivism 
Theory  of  Piaget 

Constructivism 

Constructivism 
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explicit  in 

references. 

Problem  Based 
Learning 

Model  of 

Cooperation  or 
Kind  of  Tasks 

Project  development 
Knowledge 
Construction  through 

Knowledge 

construction 

Knowledge 
construction 
through  SECAI 
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discussions 

Knowledge 
construction. 
Problem  solving 
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narrative  of  stories 

model 

Concepts  learning 

Domain 

Ecological  Model 

Not  specific 

Scientific  texts 

Not  specific 

Discussion  of 

scientific  theories 

Kinds  of 

Interaction 

Synchronous  and 

Asynchronous 

Asynchronous 

Synchronous  and 
Asynchronous 

Asynchronous 

Synchronous  and 
Asynchronous 

Quality /Degree 
of  Interaction 

Big 

Medium 

Big 

Small 

Big 

Activities  of 

Cooperative 

Work 

Decisions  Making 
Representation  of 

knowledge 
Group  memory 
Awareness 

Representation  of 
knowledge 
Group  memory 

Representation 
ofknowledge 
Group  memory 

Group 

memory 

Awareness 

Representation  of 
knowledge 
Group  memory 
Awareness 

Role 

Assignment 

None 

None 

None 

None 

None 

Platforms 

Virtual  Reality 

CAVE  Environment 

Macintosh, 

UNIX 

UNIX/ 

X- Windows 

WWW 

WWW 

Table  3:  Classification  of  CSCL  Environments  according  to  Framework 


From  the  above  Table,  we  can  stand  out  some  observations: 

• Most  of  the  environments  privilege  the  collaborative  construction  of  some  kind  of  knowledge, 
being  based  on  theories  of  learning. 

• The  environments  that  develop  elaborated  models  allow  interactions  of  higher  degree  and 
combine  another  research  areas'  technique. 

• The  group  memory  and  knowledge  representation  are  present  features  in  most  of  the  systems. 

6.  Conclusions  and  Future  Perspectives 

Computer- supported  cooperative  learning  is  a relatively  recent  area  of  researches  and  there  are  not 
still  complete  theoretical  frameworks  that  describe  and  structure  its  whole  dimensions.  In  this  paper,  a 
framework  that  organizes  the  main  aspects  related  to  CSCL  environments  was  discussed,  in  order  to 
facilitate  the  study  and  analysis  of  them.  It  also  aims  to  give  directions  for  new  environments 
building. 

The  work  is  inserted  in  the  context  of  a wider  study  on  CSCL  area,  whose  objective  is  to  define  a 
more  general  framework  embracing,  besides  the  classification  of  learning  environments,  other  themes 
related  such  as:  software  environments  for  development  of  cooperative  learning  applications;  distance 
education,  and  computer-supported  cooperative  learning  in  organizations. 
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Abstract:  This  study  investigated  (1)  the  difference  of  language  performance  and  confidence  of  the 
participants  who  used  computer-mediated  communication  (CMC)  and  who  did  not,  (2)  the  content 
and  appropriateness  of  CMC  in  distant  learning,  (3)  the  opinions  of  students  and  the  instructor 
towards  the  use  of  CMC  in  instruction,  and  (4)  the  problems  the  participants  encountered  in  the  use 
of  CMC.  The  participants  were  college  students  in  the  USA  learning  Spanish  and  communicating 
with  college  students  in  Mexico  via  e-mail  for  ten  weeks.  Data  collection  included  scores  of  cloze 
tests,  written  reports,  and  oral  examinations.  Data  also  consisted  of  e-mail  messages,  survey,  and 
interviews.  The  study  revealed  that  CMC  was  beneficial  for  distant  learning  and  that  the  students 
and  the  instructor  were  in  favor  of  the  use  of  CMC  in  instruction.  A follow-up  study  was 
conducted  one  year  after  the  initial  study  was  completed.  The  results  of  the  follow-up  study 
supported  the  findings. 


Introduction 

Recent  developments  in  electronic  technologies  have  dramatically  altered  global  communication.  The 
technologies  have  changed  how  people  communicate  and  also  influenced  how  they  learn.  The  Internet,  which 
transcends  international  boundaries,  allows  people  to  communicate  with  audiences  afar  within  a short  time.  It  also 
allows  everyone  on  the  globe  to  join  one  big  learning  environment.  E-mail,  a computer-mediated  communication 
(CMC)  technology  that  relies  on  the  Internet,  has  become  the  most  common  and  the  least  expensive  way  to 
communicate  and  learn  at  a distance. 

Is  CMC  appropriate  for  learning?  Should  educators  encourage  the  use  of  CMC  in  instaiction?  Before 
answering  these  questions,  one  needs  to  understand  the  nature  of  CMC  and  social  presence.  Most  CMC  today  is 
based  on  text  and  lacks  nonverbal  cues,  such  as  facial  expressions  and  gestures.  Text-only  e-mail  communication  is 
an  example  of  CMC  where  the  reader  can  not  access  the  sender’s  emotions  unless  the  sender  expresses  them  in  the 
text.  Social  presence  is  the  degree  to  which  a person  feels  "socially  present”  in  a mediated  situation.  People  feel 
intimate  and  share  more  if  they  feel  socially  present.  However,  when  social  presence  is  lacking,  people  recognize 
the  impersonal  environment  and  share  less.  Social  presence  is  an  important  factor  in  distant  learning  and  one  of  the 
four  theoretical  constructs  in  distance  education  (Mclsaac  & Gunawardena,  1996).  Many  scholars  have  addressed 
the  topics  of  CMC  and  social  presence.  Two  perspectives  have  emerged  in  the  literature  of  CMC. 

One  perspective  is  that  the  lack  of  social  cues  in  CMC  is  problematic.  Rice  (1984)  and  Trevino,  Lengel, 
and  Daft  (1987)  noted  that  the  absence  of  social  cues  affected  users’  perception  of  communication  context  and 
constrained  users’  interpretation  of  messages.  Many  scholars  noted  that  it  is  less  appropriate  to  use  CMC  for 
personalized  interactions  which  are  needed  in  resolving  disagreement,  getting  to  know  someone,  or  negotiation 
(Hiltz,  Johnson,  & Agle,  1978;  Rice  & Case,  1983;  Rice,  1984;  Steinfield,  1986).  An  argument  can  be  made  that 
since  interactions  are  crucial  to  the  learning  process,  CMC  is  inappropriate  for  learning.  Research  studies  provide 
more  specific  information  about  the  topic.  According  to  Short,  Williams,  and  Christie’s  (1976)  study,  when  fewer 
nonverbal  codes  were  available  in  a medium,  the  users  paid  less  attention  to  the  presence  of  other  social 
participants.  Hackman  and  Walker  (1990)  studied  students  in  an  interactive  television  setting,  and  they  found  that 
cues  given  to  students  such  as  encouraging  gestures,  smiles,  and  praise  were  social  factors  that  enhanced  students’ 
learning.  Without  such  cues,  CMC  can  be  "less  friendly,  emotional,  or  personal  and  more  businesslike,  or  task- 
oriented"  than  other  communication  media  (Rice  & Love,  1987,  p.  88).  Appelbaum  and  Enomoto’s  (1995)  study 
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revealed  that  individual  differences  could  not  be  highlighted  via  CMC.  Ruberg,  Moore,  and  Taylor  (1996)  noted 
that  CMC  limited  student  participation  and  interaction.  Because  of  CMC’s  nature,  researchers  have  determined  that 
CMC  is  not  appropriate  for  communication  and  learning. 

The  other  perspective  that  emerges  from  the  literature  is  that  CMC  is  appropriate  for  learning. 

Gunawardena  (1995)  stressed  that  although  CMC  contained  few  social  context  cues,  it  could  be  perceived  as 
interactive,  active,  interesting,  and  stimulating  by  participants.  She  suggested  that  student  perceptions  of  the  social 
qualities  of  a medium  depended  upon  the  social  presence  created  by  the  instructors  (or  the  moderators)  and  the  on- 
line community.  Many  scholars  have  described  advantages  of  using  CMC  in  education.  They  noted  that  CMC  users 
adapted  to  the  medium  and  developed  "on-line  communities"  (Hiltz  <&  Turoff,  1978;  Kerr  & Hiltz,  1982)  and 
that  CMC  messages  were  friendly  and  personal.  An  argument  can  be  made  that  since  CMC  messages  are  personal 
and  foster  interactions  which  are  crucial  to  the  learning  process,  CMC  is  appropriate  for  learning.  Some  studies 
provide  insight  into  the  underlying  process  for  CMC  in  education.  Hiltz  and  TurofP s (1 978)  study  presented  cases 
where  friendship  and  warm  relations  developed.  Rice  and  Love  (1987)  found  a significantly  greater  percentage  of 
socioemotional  content  in  a computer  "bulletin  board"  than  had  been  previously  reported  in  experimental  findings. 
Steinfield  (1986)  found  that  the  need  for  communication  across  locations  was  associated  positively  with  CMC 
messages.  Johansen,  DeGrasses,  and  Wilson  (1978)  reported  that  messages  among  participants  on  a network 
reflected  participants'  interests  and  attitudes  and  that  network  communication  turned  into  friendships  over  time. 
Foulger  (1990)  reported  that  experienced  computer  users  rated  several  text-based  media,  such  as  e-mail,  as  "rich" 
media.  Ruberg,  Moore,  and  Taylor  (1996)  found  that  CMC  encouraged  discussion  among  students.  A study  by 
Olaniran,  Savage,  and  Sorenson  (1996)  indicated  that  students  generated  more  ideas  using  CMC.  Anderson  and 
Kanuka  (1997)  concluded  that  CMC  was  beneficial  for  communication. 

In  summary,  the  literature  provides  support  for  two  perspectives:  that  the  lack  of  social  cues  of  CMC  is 
problematic  and  that  CMC  messages  are  personal  and  foster  interactions.  The  former  have  not  supported  the 
appropriateness  of  CMC  in  communication  and  learning  while  the  latter  have.  The  study  presented  in  this  article 
extends  our  knowledge  of  CMC  appropriateness  in  a distance  learning  environment  by  examining  foreign  language 
learners  and  by  involving  two  nations.  This  study  investigated  (1)  the  difference  of  language  performance  and 
confidence  of  the  participants  who  used  CMC  and  who  did  not,  (2)  the  content  and  appropriateness  of  CMC  in 
distant  learning,  (3)  the  opinions  of  students  and  the  instructor  towards  the  use  of  CMC  in  instruction,  and  (4)  the 
problems  the  participants  encountered  in  the  use  of  CMC. 


The  Study 

Student  Samples 

The  participants  were  United  States  (US)  students  who  were  enrolled  in  a fifth-semester  Spanish 
Conversation  and  Composition  course  offered  by  the  Department  of  Languages  and  Literatures  at  a large  public 
university.  Students  in  two  classes  participated  in  the  study.  They  were  taught  by  the  same  instructor  on  the  same 
days.  One  class  learned  and  used  e-mail  during  the  semester,  and  the  other  one  did  not. 

The  e-mail  group  communicated  with  students  in  Mexico  via  e-mail.  The  e-mail  communication  was 
conducted  in  Spanish  and  lasted  for  10  weeks.  A total  of  18  US  students  were  involved  in  the  CMC  and  each  had 
two  Mexican  pen-pals.  They  were  encouraged  but  not  required  to  communicate  with  their  pen-pals. 

The  pen-pals  were  Mexican  students  taking  a university  mathematics  class  in  Institute  Tecnologico  y de 
Estudios  Superiores  de  Monterrey  in  Mexico.  The  Mexican  professor  encouraged  the  students  to  communicate  with 
the  students  in  the  United  States  of  American  (USA)  and  to  find  out  how  students  in  the  USA  learn  mathematics. 

Research  Questions 

This  study  asks  the  following  questions: 

1 . Does  the  reading,  writing,  and  speaking  performance  of  the  students  using  e-mail  in  class  differ  from  the 
performance  of  those  students  not  using  e-mail  in  class? 

2.  Does  the  confidence  of  the  students  using  e-mail  in  class  differ  from  the  confidence  of  those  students  not  using 
e-mail  in  class? 

3.  What  does  the  content  of  the  students’  e-mail  messages  involve? 

4.  What  are  the  opinions  of  the  students  and  instructor  about  using  e-mail  in  foreign  language  learning? 

5.  What  problems  do  students  encounter  during  communication  with  their  pen-pals  via  e-mail? 
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Procedures 


At  the  beginning  of  the  semester,  the  researcher  explained  the  project  to  the  students  of  the  e-mail  group, 
introduced  the  possible  use  of  e-mail  in  foreign  language  learning,  and  expressed  the  hope  that  the  students  would 
choose  to  participate.  The  subjects  were  told  that  they  were  not  required  to  participate  in  the  study.  If  they  decided 
to  get  involved  in  it,  they  had  the  freedom  to  stop  it  at  any  time.  They  had  the  choice  of  writing  a lot  using  their  e- 
mail  accounts,  or  writing  little,  or  even  not  writing  at  all.  Students  were  told  that  they  would  write  their  e-mail 
outside  of  class  so  that  the  writing  would  not  take  up  any  class  time.  If  they  decided  not  to  write,  it  would  not  affect 
their  course  grade. 

To  encourage  the  students  to  write  more,  the  instructor  agreed  to  give  extra  points  up  to  ten  percent  of  the 
total  score  for  participating.  The  extra  points  were  based  on  the  number  of  the  e-mail  messages  sent  to  the  pen-pals 
and  the  amount  written  in  the  e-mails.  To  get  extra  points,  the  students  were  told  that  they  should  send  copies  of 
their  e-mail  messages  to  the  researcher.  After  the  explanations,  all  students  volunteered  and  signed  the  consent 
letter.  They  filled  in  a demographic  data  sheet  and  an  attitude  survey  form,  and  they  participated  in  a cloze  test. 

The  researchers  repeated  similar  procedures  to  the  non-e-mail  group.  The  students  also  signed  the  consent 
letter,  filled  in  the  demographic  data  sheet  and  the  attitude  survey  form,  and  took  the  cloze  test.  Since  they  did  not 
use  e-mail,  their  extra  points  were  based  on  the  number  of  articles  they  read  outside  of  class  and  the  amount  of 
summaries  they  wrote. 

Throughout  the  semester,  both  the  e-mail  and  non-e-mail  classes  received  the  same  encouragement  from 
their  instructor  to  conduct  the  out-of-class  work  for  extra  points.  The  project  lasted  for  ten  weeks. 

At  the  end  of  the  semester,  the  students  in  both  the  e-mail  and  non-e-mail  classes  answered  the  same  cloze 
test  and  the  attitude  survey  as  at  the  beginning  of  the  project.  The  e-mail  group  filled  in  an  additional  questionnaire 
to  share  their  opinions  about  using  e-mail  in  their  classroom.  They  were  informed  that  a telephone  interview  would 
be  conducted  to  clarify  the  responses  on  the  questionnaire.  During  the  following  days,  all  subjects  were 
interviewed. 

After  the  final  examinations,  the  instructor  provided  the  researcher  with  scores  of  both  classes,  including 
scores  of  all  written  reports  and  two  oral  examinations.  Finally,  the  instructor  was  interviewed  to  share  his  opinions 
about  the  use  of  e-mail  in  his  class. 

Data  Analysis 

Both  quantitative  and  qualitative  research  methods  were  used  to  analyze  the  data.  The  data  analysis 
consisted  of  three  major  parts. 

The  first  part  is  to  answer  the  first  two  research  questions.  It  included  the  use  of  statistical  analysis  to 
compare  the  language  performance  and  confidence  of  the  students  of  the  e-mail  group  with  those  characteristics  of 
the  non-e-mail  group.  Both  pre-e-mail  and  post-e-mail  scores  were  obtained.  Pre-e-mail  scores  included  scores  of 
the  cloze  test  administered  at  the  beginning  of  the  research  study,  the  first  three  written  reports,  and  the  first  oral 
examination.  Post-e-mail  scores  consisted  of  scores  of  the  cloze  test  administered  at  the  end  of  the  research  study, 
the  final  three  written  reports,  and  the  final  oral  examination.  A t-test  was  first  used  to  evaluate  whether  the  pre-e- 
mail scores  of  one  group  were  significantly  different  from  those  scores  of  the  other  group.  If  there  was  no 
significant  difference,  a t-test  was  used  to  compare  the  post-e-mail  scores  of  the  two  groups.  If  there  was  a 
significant  difference,  an  Analysis  of  Variance  (ANOVA)  was  conducted. 

Due  to  the  lack  of  a reliable  language  test,  the  evaluation  of  language  performance  of  this  study  was  based 
on  the  scores  of  the  cloze  test  (reading),  the  students’  written  reports  (writing),  and  the  oral  examinations 
(speaking).  The  cloze  test  consisted  of  paragraphs  requiring  students  to  fill  in  twenty  key  words.  A total  of  100 
points  were  available.  The  reliability  of  the  cloze  test  was  checked  by  consulting  with  two  Spanish  experts.  The 
students’  writing  and  oral  skills  were  graded  by  their  instructor  who  used  grading  criteria  agreed  upon  by  all  the 
instructors  teaching  the  Spanish  course  and  by  the  coordinator  of  the  course.  Because  the  instructors  were  Spanish 
experts,  the  reliability  of  those  language  scores  was  high. 

The  analysis  of  the  students’  confidence  was  based  on  the  attitude  survey.  It  consisted  of  1 0 questions 
asking  students  to  rate  their  confidence  on  their  Spanish  language  skills  and  their  knowledge  of  Spanish  cultures. 

For  each  individual,  the  rating  from  each  question  was  summed  together  into  a total  rating  ranging  from  5 to  50. 

The  mean  of  the  rating  numbers  of  the  e-mail  group  was  compared  with  the  mean  of  the  non-e-mail  group.  The 
reliability  of  the  attitude  survey  was  checked  by  consulting  with  three  reputable  researchers.  The  internal  reliability 
of  the  attitude  survey  was  also  checked  by  SAS.  The  Cronbach  Coefficient  alpha  value  of  the  attitude  survey  of  the 
e-mail  group  before  the  e-mail  communication  was  0.95,  and  the  alpha  value  after  the  communication  was  0.93. 
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The  values  for  the  non-e-mail  groups  were  0.85  and  0.81 . The  values  were  high,  and  the  items  of  the  survey  were 
reliable. 

The  second  part  is  to  answer  the  third  research  question.  It  involved  the  use  of  qualitative  research  methods 
to  analyze  the  e-mail  messages.  The  content  topics  in  the  e-mail  messages  were  coded  and  categorized  according  to 
topics  in  the  previous  literature  (Barson  et  al.,  1993;  Cononelos  & Oliva,  1993;  Kern,  1995).  Topics  in  the  literature 
included  culture  (e.g.,  values,  beliefs),  language  itself  (grammar,  syntax),  course  management  (due  date, 
assignment),  social  activity  (planning  for  a movie,  a party),  and  so  forth.  In  the  process  of  coding  and  analyzing, 
other  topics  emerged,  too.  Further,  the  e-mail  messages  were  analyzed  in  depth.  The  messages  which  were  warm 
and  friendly  and  which  fostered  learning  and  communication  were  coded  and  collected  into  specific  folders. 

The  third  part  is  to  answer  the  fourth  and  fifth  research  questions.  It  consisted  of  reviewing  and  analyzing 
the  data  collected  by  survey  questionnaires  and  interviews  conducted  in  the  e-mail  group.  The  survey  consisted  of 
10  questions  with  a scaling  from  one  to  five  and  three  essay  questions.  The  survey  was  designed  to  find  the 
participants’  opinions  towards  the  use  of  e-mail  in  the  study.  The  internal  reliability  of  the  questionnaire  was 
checked  by  SAS.  The  Cronbach  Coefficient  alpha  value  was  0.82.  This  value  was  fairly  high,  and  the  items  of  the 
questionnaire  were  reliable.  Interviews  were  conducted  to  clarify  any  ambiguous  information. 

Results 

The  quantitative  results  of  the  study  indicated  that  there  was  no  significant  difference  on  reading,  writing, 
and  speaking  performance  between  the  two  groups.  The  students’  confidence  of  one  group  also  did  not  significantly 
differ  from  that  of  the  other  group. 

The  qualitative  results  of  the  study  revealed  that  CMC  provided  the  participants  with  good  language 
learning  environment,  motivated  the  learners,  fostered  learning,  and  encouraged  communication.  The  content  of  the 
e-mail  messages  consisted  of  discussions  about  cultures  (e.g.  values,  customs,  and  life  styles)  and  Spanish  language 
use,  such  as  syntax  and  semantics.  As  several  CMC  researchers  mentioned,  many  e-mail  messages  of  the  study  also 
revealed  that  the  participants  developed  on-line  communities.  Their  messages  were  warm  and  friendly,  and  the 
participants  became  friends  via  CMC. 

The  results  of  the  survey  and  the  interviews  indicated  that  the  students  in  the  e-mail  group  and  the 
instructor  were  in  favor  of  using  e-mail  in  language  learning.  The  students  had  strong  desire  of  using  CMC  while 
the  research  study  was  conducted  and  after  the  study  was  completed  as  well.  Many  e-mail  messages  at  the  end  of 
the  semester  appeared  to  be  discussions  about  how  to  continue  the  CMC.  The  students  unanimously  supported  the 
use  of  e-mail  in  foreign  language  learning  and  thought  that  e-mail  was  a good  addition  to  their  class. 

Problems  occurring  during  the  semester  were  systematically  recorded  by  the  researcher.  Some  problems 
were  successfully  handled  by  the  researcher  and  the  professor  in  Mexico;  for  example,  (1)  the  students’  initial  mail 
was  returned  as  undeliverable  and  (2)  some  students  did  not  receive  their  pen-pals’  responses.  Problems  addressed 
in  the  study  can  be  of  interest  to  researchers  or  practitioners  who  wish  to  conduct  similar  projects.  The  following 
problems  surfaced,  for  example,  (1)  several  students  had  limited  access  to  a computer,  (2)  some  pen-pals  lacked 
commitment  of  writing,  and  (3)  the  students  could  not  use  foreign  characters  in  their  e-mail  system. 


The  Follow-Up  Study 

The  follow-up  study  was  conducted  one  year  after  the  initial  study.  E-mail  was  sent  to  each  of  the  students 
in  the  e-mail  group.  The  students  were  asked  to  answer  the  following  questions:  (1)  Did  the  student  continue 
writing  to  the  pen-pals  after  the  research  project  was  completed?  (2)  Was  the  student  still  communicating  with  the 
pen-pals  via  e-mail?  and  (3)  How  did  the  project  affect  the  student  over  time?  Four  students  answered  the  questions 
by  e-mail,  and  ten  students  answered  the  same  questions  through  the  phone.  Four  students  could  not  be  located. 

One  third  (four  students)  of  the  participants  involved  in  the  follow-up  study  said  that  they  have  continued 
writing  to  their  pen-pals  after  the  project  was  over.  The  communication  lasted  for  a few  weeks  for  two  of  the 
students  and  lasted  for  three  months  for  one  student.  The  fourth  student  was  still  communicating  with  her  pen-pal 
when  the  follow-up  study  was  conducted.  Other  students  did  not  continue  writing  to  their  pen-pals  because  they 
were  too  busy  to  write  or  because  their  pen-pals  did  not  reply. 

All  of  the  students  reiterated  that  using  e-mail  in  foreign  language  learning  was  a great  idea  and  should  be 
integrated  into  instruction.  Several  students  reported  that  the  project  has  positively  affected  them.  Three  students 
described  exactly  how  the  project  influenced  them. 
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One  of  the  students  mentioned  that  although  his  experience  with  his  pen-pals  in  the  Spanish  class  was 
"short  lived,"  the  project  motivated  him  to  make  friends  with  people  in  Brazil  and  helped  him  to  "remain  in  touch 
with  the  culture  there."  Another  student  stated  that  his  pen-pals  in  the  class  were  not  "very  responsive."  However, 
he  found  other  pen-pals  and  was  writing  to  them  weekly.  He  said,  "Your  study  got  me  interested  in  writing  to 
people,  and  I have  found  several  lists  [newsgroups]  of  Spanish  pen-pals  on  the  Internet."  Another  student 
mentioned  that  she  was  helping  her  daughter's  school  to  connect  with  people  in  Mexico.  She  said  that,  without  the 
experience  with  the  research  project,  she  could  not  help  the  school. 


Conclusion 

The  research  study  revealed  that  e-mail  (CMC)  is  beneficial  for  learning  and  communication.  Although  the 
participants  lacked  social  cues,  such  as  gestures,  the  participants  developed  on-line  community  for  communication. 
The  students  used  CMC  to  discuss  cultural  issues  and  language  use  with  their  pen-pals  (Spanish  native  speakers)  at  a 
distance.  CMC  provided  the  participants  with  good  environment  for  friendship,  learning,  and  communication.  The 
students  and  the  instructor  who  participated  in  the  study  were  in  favor  of  the  use  of  e-mail.  The  students  had  strong 
desire  to  use  CMC  not  only  while  the  research  study  was  conducted  but  also  after  the  study  was  completed.  They 
unanimously  agreed  that  CMC  was  a good  addition  to  foreign  language  learning. 

The  results  of  the  follow-up  study,  conducted  one  year  after  the  research  study,  revealed  the  consistent 
positive  opinions  of  the  participants  towards  the  use  of  CMC.  They  addressed  again  that  CMC  is  beneficial  for 
foreign  language  learning  and  should  be  integrated  into  foreign  language  instruction. 
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Reusing  Web  Documents  in  Tutorials  With  the  Current-Documents 
Assumption:  Automatic  Validation  of  Updates 


Abstract:  This  paper,  based  on  the  first  author's  Ph.D.  dissertation  describes  an 
experimental  system  for  authoring  and  delivering  tutorials  on  the  web  under  the 
assumption  that  the  system  should  deliver  the  latest  versions  of  the  selected 
documents,  even  when  they  are  not  controlled  by  the  tutorial  author.  The  great 
richness  of  materials  available  on  the  web  encourages  reuse  of  web  documents  in 
tutorials.  However,  the  dynamic,  sometimes  chaotic  nature  of  the  World  Wide  Web 
poses  a new  challenge  for  tutorials  that  depend  on  links  to  various  people's  or 
institutions'  web  pages.  Therefore  our  system  includes  provisions  for  anticipation  of 
various  kinds  of  changes  to  documents  or  their  availability.  At  time  of  tutorial 
delivery,  our  system  validates  web  documents  by  evaluating  differences  between 
document  profiles  stored  at  authoring  time  and  new  profiles  based  on  the  current 
versions.  If  appropriately  constructed,  the  tutorials  can  then  adapt  to  the  changed 
circumstances  of  availability  or  reliability. 


Introduction 

Motivation 

The  educational  potential  for  the  Internet  has  aspects  that  can  be  categorized  under  several  headings: 
online  library  services  and  reference  material,  communication  and  conferencing,  and  online  tutorials  and 
courseware.  These  functions  are  interrelated  and  can  easily  be  combined.  For  example,  tutorials  can  make  use 
of  online  reference  material  by  including  links  to  it.  The  Internet  enhances  opportunities  for  authors  to  reach 
distant  and  diverse  audiences,  and  it  makes  it  easy  to  combine  and  integrate  resources  found  on  the  World  Wide 
Web  into  tutorials. 

This  paper  explores  some  of  the  issues  involved  in  authoring  and  delivering  tutorials  via  the  World 
Wide  Web  that  reuse  documents  found  on  the  web.  The  World  Wide  Web  is  somewhat  chaotic:  web  sites  come, 
go,  and  evolve.  Documents  referred  to  in  tutorials  may  change.  The  student's  access  to  various  web  pages  may 
not  always  be  timely.  One  solution  to  these  difficulties  would  be  to  encapsulate  all  documents  used  in  a tutorial 
on  a CD-ROM  and  distribute  the  tutorial  on  CD-ROM.  This  solution  is  appropriate  for  many  situations,  but  it 
has  some  disadvantages.  The  encapsulated  documents  may  become  obsolete  or  if  not  obsolete  then  inferior  to 
improved  versions  that  get  posted  by  their  creators.  There  may  also  be  legal  restrictions  placed  upon  the  use  of 
the  documents  that  forbid  their  copying  and  distribution  on  CD-ROM.  Finally,  distribution  by  CD-ROM  may 
not  be  as  convenient  as  just-in-time  downloading.  Therefore  it  is  sometimes  appropriate  to  adopt  the  following 
assumption:  documents  referenced  in  the  tutorial  should  be  the  latest  available  versions  and  should  be  retrieved 
when  needed,  without  copying  other  than  the  normal  caching  performed  by  the  browser  and  network  relay  sites. 
We  call  this  the  current-documents  assumption.  The  current-documents  assumption  together  with  ubiquitous 
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reuse  of  documents  is  the  primary  consideration  that  changes  the  nature  of  Web-based  tutorial  authoring  from 
conventional  tutorial  authoring. 

After  we  describe  our  experimental  tutorial  system,  Wtutor,  we  discuss  one  of  the  technical  problems 
that  arise  in  web-based  tutorials  with  the  current-documents  assumption:  ascertaining  to  a reasonable  degree  of 
certainty  that  the  information  contained  in  a current  document  is  consistent  enough  with  that  of  the  original 
document  (the  document  at  the  same  URL  at  the  time  of  authoring)  that  the  tutorial  will  not  suffer  any  loss  of 
effectiveness.  We  call  this  the  validation  problem.  We  provide  a solution  to  this  problem  and  describe  our 
experience  with  the  experimental  system. 


Related  Work 

Tutorial  systems  for  reuse  of  World  Wide  Web  documents  have  been  created  by  several  groups.  The 
primary  concern  among  these  efforts  has  been  providing  means  for  navigating  through  the  web  of  documents. 
The  '’Walden’s  Paths"  system  is  one  such  program  using  this  approach  (Shipman  et  al.  1996).  It  uses  a proxy 
server  and  CGI  scripts  to  translate  requests  for  tutorial  documents  into  requests  for  documents  from  the  World 
Wide  Web.  Paths  are  stored  as  linear  lists  of  URLs.  The  path  server  not  only  fetches  the  documents  from  the 
web  but  also  modifies  them  before  they  are  displayed  by  the  user's  browser.  The  modifications  include  the 
insertion  of  new  navigation  links  that,  when  activated,  take  the  user  one  or  more  steps  forward  or  backward 
along  the  path.  A related  system  is  Footsteps  (Nicol  et  al.  1996).  This  system  also  uses  CGI  scripts  to  request 
and  then  modify  incoming  documents;  however,  rather  than  offering  links  to  several  documents  along  the  path 
in  the  vicinity  of  the  current  document.  Footsteps  provides  a link  to  an  index  to  all  of  the  documents  in  the  path. 

Another  tutorial  system  for  the  web  is  Brio  (Roscheisen  et  al.  1995).  The  BRIO  system  makes  use  of 
special  documents  called  annotations.  The  tutorial  author  creates  one  annotation  for  each  web  document  and 
specifies  a connection  structure  by  linking  annotation  documents  to  each  other.  A special  browser  is  used  that 
provides  two  panes  for  the  user:  one  shows  a list  of  links  to  annotated  documents  in  the  tutorial.  The  other 
shows  the  documents  with  their  corresponding  annotations  superimposed. 

These  systems  provide  facilities  for  gluing  web  documents  together  into  tutorials.  However,  they  do 
not  provide  any  support  for  meeting  the  challenges  of  the  dynamic  nature  of  the  World  Wide  Web. 


The  Wtutor  System 

Our  experimental  system  for  delivering  web-based  tutorials  is  called  Wtutor  (Johnson  1997).  Its  two 
fundamental  components  are  a tutorial  control  program  and  a modified  web  browser.  The  tutorial  control 
program  is  written  in  Common  Lisp.  It  holds  a representation  of  the  tutorial  in  terms  of  URLs  and  how  they 
relate  to  one  another  as  parts  of  the  tutorial.  The  control  program  receives  messages  from  the  browser  and  also 
instructs  the  browser  to  load  particular  web  documents.  The  modified  browser  serves  as  the  interface  between 
the  student  and  the  tutorial  material  as  well  as  an  interface  between  the  student  and  the  control  program. 

Each  tutorial  is  represented  using  a structure  we  call  the  tutorial  map.  The  map  consists  of  a set  of 
URLs  (World  Wide  Web  document  addresses  the  adhering  to  the  Uniform  Resource  Locator  syntax),  together 
with  information  about  the  relationships  among  the  documents  for  purposes  of  the  tutorial.  Documents  are 
ordered  and  can  have  alternatives  specified.  The  author  of  a Wtutor  tutorial  can  specify  that  URLs  belong  to 
groups  of  URLs.  The  groups  can  be  used  as  elements  of  precedence  relationships,  implicitly  specifying 
alternatives.  The  groups  can  also  be  labelled  as  "concurrent,"  meaning  that  all  URLs  in  the  group  should  be 
displayed  simultaneously  in  different  portions  of  the  screen.  For  example,  this  permits  commentary  written  by 
the  tutorial  author  to  appear  in  synchrony  with  a "reused"  web  document  from  a third-party  web  site.  Tutorial 
maps  represent  the  allowable  flows  of  visitation  through  the  portion  of  the  World  Wide  Web  of  interest  to  the 
tutorial.  They  also  contain  information  that  supports  effective  validation  of  the  documents  used  in  the  tutorial 


The  Validation  Problem 

As  we  have  already  discussed,  the  validation  problem  refers  to  the  challenge  of  making  sure  that  URLs 
included  in  the  tutorial  continue  to  serve  their  purposes  in  the  tutorial  even  though  the  documents  they  point  to 
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may  have  been  removed  or  may  have  been  changed  by  their  authors.  Given  the  dynamic  nature  of  the  web,  this 
is  an  important  consideration  in  reusing  web  documents  under  the  current-documents  assumption  that  students 
see  only  documents  just  downloaded  from  their  original  sites. 

Documents  placed  on  the  World  Wide  Web  typically  undergo  changes  after  the  times  they  are  first 
posted.  Sometimes  these  changes  are  trivial-minor  adjustments  to  formatting  or  the  correction  of  a few  spelling 
errors.  In  other  cases,  significant  portions  of  text  are  added  or  removed.  Sometimes  document  are  removed 
either  because  they  were  parts  of  temporary  web  sites  (e.g.,  college  student  web  pages  that  get  removed  when 
the  student  graduates)  or  because  they  were  found  to  violate  some  copyright.  In  many  and  perhaps  most  cases, 
documents  remain  on  the  web  and  remain  effective  for  their  use  in  a tutorial.  However,  the  challenge  is  to  have 
a means  for  the  tutorial  delivery  system  to  automatically  make  a reasonably  reliable  judgment  about  whether  a 
document  is  still  effective  for  the  tutorial’s  purpose. 


Approach  to  Validation 

There  are  many  ways  in  which  a tutorial  system  could  anticipate  the  kinds  of  changes  listed  above  and 
respond  to  them.  The  simplest  way  would  be  to  store  a complete  copy  of  the  original  document.  This  would 
allow  comparisons  to  be  done  with  existing  tools  such  as  dijf,  which  could  provide  a detailed  list  of  the  physical 
changes  to  the  document.  Given  the  list  of  physical  changes,  it  would  still  be  necessary  to  infer  the  significance 
of  the  changes. 

This  approach  has  the  disadvantage  of  consuming  large  quantities  of  storage  space  and  consuming 
large  amounts  of  bandwidth  either  at  tutorial  map  retrieval  time  or  at  document  validation  time.  A tutorial  could 
easily  contain  hundreds  of  megabytes  of  data,  which  would  have  to  be  either  stored  on  the  student’s  local 
storage  medium  or  on  a central  storage  medium.  The  space  requirements  could  be  partially  mitigated  by  the  use 
of  compression  techniques  on  the  document  text,  but  this  would  still  lead  to  significant  space  and  transmission 
requirements.  What  is  needed  is  a document  signature  that  significantly  reduces  storage  requirements  while  still 
providing  enough  information  about  the  document  that  useful  comparisons  can  be  made. 

Rather  than  storing  copies  of  the  tutorial  documents,  Wtutor  creates  digital  signatures  of  each 
document  when  the  document  is  first  added  to  the  tutorial  and  stores  these  signatures.  When  a reader  is  taking  a 
tutorial,  Wtutor  retrieves  the  current  versions  of  the  tutorial  documents,  calculates  the  current  signature  and  then 
compares  the  stored  signature  to  the  current  to  infer  the  level  of  change  in  the  document. 


Wtutor  signatures 

Wtutor  takes  advantage  of  the  structural  information  provided  in  HTML  documents  to  extract 
meaningful  document  signatures.  In  Wtutor,  document  signatures  are  composed  of  three  elements:  the 
checksums  of  the  paragraphs  of  the  document,  the  headings  of  the  document,  and  any  keywords  the  tutorial 
author  chooses  to  specify.  A sample  signature  is  shown  in  (Fig.  1).  Before  extracting  the  signature,  Wtutor 
normalizes  the  document  by  converting  all  the  text  to  lower  case  and  reducing  each  contiguous  group  of 
whitespace  characters  to  single  space  characters. 


URL  http://www.cs.washington.edu/research/metip/tutor.form.image.html 

Paragraphs  (19316  47247  21696  47247  21696  33122  37527  38582) 

Headings  (quantization  sampling  rate  applying  linear  filters  tutorial  image  request  form) 

Keywords  (NIL) 

Figure  1 Sample  Wtutor  Signature 

Other  possibilities  were  considered  for  inclusion  in  the  signature,  such  as  the  title  of  the  document  or 
the  links  contained  within  the  document.  The  paragraph  checksums,  headings,  and  keywords  were  chosen  as  the 
elements  that  best  provide  information  about  the  document. 


Wtutor  signature  comparison  method 
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Wtutor  stores  document  signatures  in  the  tutorial  map.  The  tutorial  author  can  run  a procedure  at  any 
time  to  update  the  signature  of  any  URLs  in  the  tutorial  map.  When  a student  is  using  the  tutorial,  Wtutor 
validates  each  URL  by  comparing  the  digital  signature  stored  in  the  map  with  the  document’s  current  digital 
signature.  The  comparison  yields  a numeric  distance  value,  which  indicates  how  much  the  document  has 
changed. 

For  each  element  of  the  signature,  Wtutor  infers  whether  there  have  been  additions,  changes,  or 
deletions.  Once  the  number  of  changes,  additions,  and  deletions  in  each  category  have  been  determined,  they 
are  combined  according  to  the  function  in  (Fig,  2).  To  allow  for  documents  of  different  lengths,  the  function 
normalizes  by  the  number  of  items  of  each  type  within  the  original  document.  This  normalization  recognizes 
that  the  scale  of  changes  matters — adding  five  paragraphs  to  a document  that  originally  had  two  paragraphs  is 
generally  much  more  significant  than  adding  five  paragraphs  to  a document  that  originally  had  fifty  paragraphs. 


dist  = checksum  dist  + heading  dist  + keyword  dist 

checksum  dist  = checksum  weighting  factor  * 

(#chgs  * chg_weight  + #adds  * add_weight  + #deletes  * del_weight)  / #checksums 

heading  dist  = heading  weighting  factor  * 

(#chgs  * chg  _weight  + #adds  * add_weight  + #deletes  * del_weight ) / #headings 

keyword  dist  = keyword  weighting  factor  * 

#deletes  * del_weight  / #keywords 

Figure  2 Document  signature  distance  function 

The  distance  function  also  recognizes  that  some  types  of  differences  can  be  more  significant  than 
others  by  including  a weighting  factor  for  each  type  of  difference.  For  example,  it  is  probably  more  significant 
if  a heading  has  been  deleted  from  a document  (implying  that  some  segment  of  information  is  no  longer 
provided  by  the  document)  than  if  a heading  has  been  added  (additional  information  is  generally  not  a bad 
thing,  unless  so  much  is  added  that  the  original  content  is  overwhelmed  by  the  new). 

The  distance  function  also  allows  for  the  different  elements  of  the  signature  to  be  weighted  differently. 
A tutorial  author  may  feel  that  the  headings  of  a document  are  more  significant  than  the  paragraph  checksums, 
since  the  headings  provide  more  of  an  indication  of  the  semantic  content  of  the  document.  Another  tutorial 
author  may  feel  that  no  changes  of  any  sort  should  be  allowed,  and  choose  to  weight  all  elements  equally.  The 
Wtutor  distance  function  provides  the  author  the  flexibility  of  choosing  these  weights. 

Once  the  distance  for  a document  has  been  calculated,  Wtutor  compares  that  distance  to  two  defined 
levels:  a warning  level  and  a reject  level.  If  the  distance  from  the  original  signature  to  the  current  signature  is 
less  than  the  warning  level,  the  document  is  simply  displayed  to  the  student.  If  the  distance  is  greater  than  the 
warning  level  but  less  than  the  reject  level,  the  document  is  displayed  and  a warning  message  is  provided  to  the 
student  indicating  that  the  document  has  changed  since  its  inclusion  in  the  tutorial.  If  the  distance  is  greater  than 
the  reject  level,  Wtutor  first  checks  whether  the  document  has  an  alternative.  If  an  alternative  has  been  specified 
in  the  tutorial  map,  that  document  is  retrieved  and  displayed  instead  of  the  rejected  document.  If  there  is  no 
alternative  document,  the  rejected  document  is  displayed  to  the  student  along  with  a message  that  warns  the 
student  that  the  document  has  changed  significantly  and  may  not  be  appropriate  for  the  tutorial. 


Testing  the  Validation  Technique 

The  default  values  used  in  Wtutor  for  weighting  the  different  types  of  changes  in  document  signatures 
were  selected  after  a set  of  testing  with  a small  set  of  selected  documents  found  on  the  World  Wide  Web.  A set 
of  artificial  changes  was  applied  to  each  of  these  documents,  and  then  the  signature  comparison  algorithm  was 
used  to  compare  the  before  and  after  versions  of  each  document.  Once  the  default  weights  had  been  determined 
and  the  warning  and  reject  levels  had  been  established,  the  validation  technique  was  tested  on  examples  of  real 
changes  to  existing  documents  found  on  the  World  Wide  Web. 
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Document  life  cycle 

The  first  test  case  is  a Web  page  describing  Wtutor.  There  were  five  different  versions  of  this 
document.  The  initial  version  (version  0)  was  a brief  (72-line)  description  of  Wtutor.  Version  1 is  essentially  the 
same  document,  with  a bit  of  rewording  done.  Version  2 adds  introductory  paragraphs  to  version  1 . Version  3 is 
a User’s  Guide  to  Wtutor,  which  includes  most  of  the  material  in  version  2.  This  document  was  significantly 
larger  than  the  earlier  versions.  Finally,  version  4 of  the  document  goes  back  to  version  2,  but  adds  links  to 
sections  of  the  User’s  Guide  to  Wtutor. 

The  distance  measures  for  each  of  the  versions  of  the  Wtutor  document  are  shown  in  (Tab.  1).  These 
distance  measures  are  the  result  of  comparing  the  signatures  of  the  versions  to  the  signature  of  version  0.  The 
distance  measures  do  not  exceed  the  warning  level  for  versions  1,  2 and  4.  This  is  a desirable  result,  as  the 
changes  in  these  versions  are  minor  and  do  not  significantly  change  the  semantic  content  of  the  document. 
Version  3 exceeds  the  reject  level.  Again,  this  is  a desirable  result.  While  the  semantic  content  of  the  earlier 
versions  is  included  in  version  3,  the  document  contains  so  much  more  material  that  it  is  not  really  comparable 
to  the  other  versions. 


Version 

Distance 

Description 

0 

0.000 

Original  version;  brief  description  of  Wtutor 

1 

0.145 

Slight  rewording  of  version  0 

2 

0.209 

Addition  of  introductory  material  to  version  1 

3 

2.718 

User’s  Guide  to  Wtutor 

4 

0.245 

Back  to  version  2,  with  addition  of  links  to  User’s  Guide 

Table  1 Distance  measures  for  versions  of  Wtutor  document 


Sample  tutorial 

The  second  case  in  which  Web  documents  were  tracked  over  time  was  a sample  tutorial  about  Dr. 
Martin  Luther  King.  Although  the  tutorial  was  created  primarily  as  an  illustration  of  Wtutor’s  capabilities,  it 
also  served  as  a good  test  of  the  validation  approach,  because  several  of  the  documents  that  were  selected  for 
inclusion  in  the  tutorial  changed  after  the  tutorial  was  originally  created. 

About  a month  after  the  creation  of  the  tutorial,  a running  of  the  validation  report  for  the  tutorial 
showed  the  chronology  document  as  having  a distance  measure  of  0.007.  This  small  distance  is  well  below  the 
warning  level,  so  it  did  not  have  an  impact  on  the  tutorial.  Upon  investigation,  the  reason  for  the  distance  was 
found  to  be  a correction  of  one  of  the  dates  in  the  chronology. 

Two  weeks  later,  another  running  of  the  validation  report  showed  very  large  distance  measures  for 
several  documents.  All  the  documents  from  one  site  had  distances  in  the  1.9-2. 3 range,  well  above  the  reject 
level.  Further  investigation  revealed  that  the  owner  of  the  documents  had  moved  to  a different  site.  The  original 
site  provided  a note  that  the  documents  had  moved,  but  did  not  provide  direct  links  to  the  new  locations.  This 
required  updating  the  tutorial  map  to  point  to  the  new  locations  for  the  documents. 

In  both  of  these  cases,  the  validation  method  worked  as  desired.  In  the  first  case,  the  change  to  the 
document  was  recognized  as  small  enough  that  the  student  was  not  even  informed  of  the  difference.  In  the 
second  case,  Wtutor  recognized  that  the  documents  returned  by  the  original  server  were  not  at  all  what  the 
tutorial  author  wanted  and  notified  the  student  of  that  fact.  Without  the  validation,  the  student  would  not 
necessarily  known  that  the  documents  returned  were  not  what  the  tutorial  author  intended. 

Two  weeks  later,  a running  of  the  validation  report  showed  one  document  with  a distance  measure  of 
0.62,  again  above  the  reject  level.  Upon  inspection,  the  document  was  found  to  have  been  updated  by  the 
owner.  In  the  course  of  correcting  several  typos  in  the  document,  the  author  had  significantly  reformatted  the 
document.  In  this  case,  the  validation  method  returned  a “false  positive”.  The  text  portion  of  the  document  had 
not  changed  significantly,  but  the  differences  in  markup  were  interpreted  by  the  signature  comparison  as 
significant  document  changes.  A student  going  through  the  tutorial  would  have  received  a warning  message  that 
the  document  might  no  longer  be  appropriate  for  the  tutorial,  but  the  document  would  still  have  been  displayed 
to  the  student. 
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Author  volunteered  examples 

In  the  final  case  of  tracked  Web  documents,  Web  document  authors  were  solicited  via  Usenet  postings 
for  participation  in  the  experiment.  Prospective  participants  were  asked  if  they  had  Web  documents  that  they 
were  planning  to  update.  Dr.  Hubert  Parti  of  the  Universitaet  flier  Bodenkultur  in  Vienna,  Austria  volunteered 
to  participate  and  provided  a list  of  the  URLs  for  his  documents.  Dr.  Parti’s  documents  make  up  an  HTML 
handbook. 

A tutorial  map  that  contained  each  of  the  URLs  was  created  and  document  signatures  were  taken. 
Copies  of  each  document  were  saved  at  the  same  time  as  the  document  signatures  were  taken  so  that  actual 
changes  to  the  documents  could  be  tracked.  During  a six  month  study,  periodic  validation  reports  were  taken  to 
test  whether  the  documents  had  changed  since  the  initial  document  signatures  were  taken.  At  the  same  time  as 
the  validation  reports  were  done  the  current  versions  of  the  documents  were  retrieved  and  compared  to  the 
original  copies,  so  that  the  actual  changes  could  be  reviewed  and  the  appropriateness  of  the  resulting  distance 
measure  could  be  evaluated.  The  URLs  for  the  volunteered  set  of  documents  are  listed  in  (Tab.  2),  along  with 
two  distance  measures  for  each  document.  The  intermediate  distances  are  from  compares  done  about  two 
months  into  the  measurement  period;  the  final  distances  are  from  the  compares  done  at  the  end  of  the 
measurement  period. 


URL 

Intermediate 

Distance 

Final  Distance 

http://www.boku.ac.at/htmleinf/hein.html 

0.22 

0.39 

http://www.boku.ac.at/htmleinf/hein  1 .html 

0.30 

0.44 

http://www.boku.ac.at/htmleinf/hein2.html 

0.28 

0.50 

http://www.boku.ac.at/htmleinf/hein3.html 

0.21 

0.29 

http://www.boku.ac.at/htmleinf/hein4.html 

0.10 

0.43 

http://www.boku.ac.at/htmleinf/hein5.html 

0.27 

0.44 

http://www.boku.ac.at/htmleinf/hein6.html 

0.18 

0.21 

Table  2 Volunteered  WWW  document  distance  measures 


How  well  do  these  distance  measures  reflect  the  changes  that  were  made  to  the  documents  during  the 
measurement  period?  The  first  observation  is  that  only  one  version  of  the  documents  exceeds  the  default 
rejection  level.  This  is  a desirable  result,  as  most  of  the  changes  to  the  documents  fell  into  the  categories  of 
markup  changes,  spelling  changes  or  corrections,  rewordings  for  clarity,  or  minor  additions.  For  most  of  the 
documents,  the  final  versions  exceed  the  warning  level.  This  is  primarily  due  to  the  number  of  minor  changes 
that  were  made  to  these  documents.  The  final  version  of  hein2.html  just  reaches  the  reject  level.  The  reason  that 
this  document  exceeds  the  reject  level  is  that  it  includes  some  deletions  of  headings  and  paragraphs. 

The  distance  measures  for  the  intermediate  and  final  versions  of  these  documents  seem  appropriate  for 
the  differences  between  these  versions  and  the  original  versions  of  the  documents.  Overall,  the  Wtutor  approach 
to  validating  documents  provides  a useful  method  of  verifying  that  documents  still  fit  within  the  tutorial. 
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Abstract;  Rapid  advances  in  information  technology  and  telecommunications  (IT  and  T) 
are  impacting  on  a changing  world  in  which  boundaries,  whether  they  are  at  local,  national 
and  international  levels  are  now  less  clearly  marked.  The  so-called  information 
superhighway  is  reaching  and  linking  many  regional,  rural  and  remote  communities  so  that 
the  world  itself  has  become  a small  global  village.  This  leads  to  the  assumption  that  the 
power  of  information  technology  and  telecommunications  is  inescapable  and  unresistable. 
A logical  response  to  a changing  and  challenging  world  is  to  prepare  “global  villagers”  for 
acculturation  into  a new  IT  and  T discourse  community.  However,  potential  IT  and  T users 
are  human  not  virtual  beings  and  fear,  anxiety,  confusion,  resistance  and  inspiration  are 
part  of  the  acculturation  process  towards  an  IT  and  T discourse  community.  In  this  paper 
the  concepts  of  acculturation  and  discourse  community  are  discussed  and  applied  to  IT  and 
T discourse  generally  and  educational  multimedia  in  particular.  A paradigm  shift  is 
occurring  in  education  with  the  introduction  of  IT  and  T,  and  strategies  have  to  be 
developed  to  facilitate  acculturation  of  new  users  to  this  paradigm  to  prevent  hostility  and 
rejection  of  the  advantages  they  bring  to  regional,  rural  and  remote  communities.  Three 
aspects  of  acculturation  in  education  are  presented  and  discussed.  The  implementation  of 
acculturation  in  the  context  of  IT  and  T is  explored  through  a description  of  the  provision  of 
rural  health  education  and  training  opportunities  in  the  small  Island  State  of  Tasmania. 


Acculturation  as  A Psychological  Process 

Mead  in  Williams  (1972)  defined  socialisation  as  “a  process  by  which  human  children  bom 
potentially  human,  are  able  to  function  within  the  societies  in  which  they  are  bom”.  However,  socialisation 
is  a rather  broad  concept.  Williams  (1972)  discussed  the  fine  distinction  between  socialisation  and 
acculturation,  arguing  that  the  former  refers  to  the  process  of  transmitting  human  culture  whereas  the  latter 
is  the  process  of  transmitting  a particular  culture.  However,  in  an  interactive  world  in  which  people  of 
different  cultures  are  engaged  inter  culturally,  the  question  of  cultural  identity  needs  to  be  addressed.  In  an 
intercultural  interaction,  acculturation  is  a significant  phenomenon.  It  is  a process  in  which  people  of  a 
different  culture  have  adapted  the  new  culture  (Brown  1994).  It  is  not  a smooth  process  with  feelings  of 
confusion,  resistance,  and  reluctance  and  sometimes  suffering  evident  at  all  stages. 

In  education,  acculturation  takes  place  when  learners  from  a different  culture  learn  a new  language 
or  face  a new  educational  paradigm  that  requires  a partial  or  whole  orientation  to  a new  cultural  discourse 
and  become  members  of  its  discourse  community.  The  acculturation  process  normally  consists  of  the 
following  stages,  cognitively  and  affectively: 

• Conflict:  This  is  a shock  stage  as  a new  culture  is  rejected  as  a whole,  due  to  negative  attitudes  towards 
the  new  culture. 

• Confusion:  This  is  the  time  when  negative  attitudes  toward  the  new  culture  are  challenged  by  other 
people  or  events,  or  when  the  new  culture  becomes  a part  of  that  person’s  life. 

• Exploration:  Accidentally  or  intentionally,  a few  ‘things’  about  the  new  culture  are  discovered. 

• Familiarisation:  Learning  more  about  the  new  culture  in  order  to  function  well  in  it. 

• Acceptance:  The  new  culture  has  been  accepted.  The  crisis  has  disappeared. 
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These  stages  are  not  necessarily  sequential. 


Acculturation  in  an  IT  and  T Discourse  Community 

Swales  (1990)  presented  six  defining  characteristics  that  are  necessary  for  identifying  a group  of 
individuals  as  a discourse  community: 

• A discourse  community  has  a broadly  agreed  set  of  common  public  goals  that  may  be  formally  inscribed 
in  documents  or  be  more  tacit. 

• A discourse  community  has  mechanisms  of  intercommunication  among  its  members.  The  participatory 
mechanisms  will  vary  according  to  the  community:  meetings,  telecommunications,  correspondence, 
newsletters,  conversations  etc. 

• A discourse  community  uses  its  participatory  mechanisms  primarily  provide  to  information  and  feedback. 

• A discourse  community  utilises  and  in  turn  possesses  one  or  more  genres  in  the  communicative 
furtherance  of  its  aims.  A discourse  community  has  developed  and  continues  to  develop  discoursal 
expectations.  These  may  involve  appropriateness  of  topics,  the  form,  function  and  positioning  of 
discoursal  elements,  and  the  roles  texts  play  in  the  operation  of  the  discourse  community.  In  so  far  as 
genres  are  how  things  get  done,  when  language  is  used  to  accomplish  them,  these  discoursal  expectations 
are  created  by  the  genres  that  articulate  the  operations  of  the  discourse  community. 

• In  addition  to  owning  genres,  a discourse  community  has  acquired  some  specific  texts.  This 
specialisation  may  involve  using  lexical  items  known  to  the  wider  speech  communities  in  special  and 
technical  ways,  as  in  information  technology  discourse  communities,  or  using  highly  technical 
terminology  as  in  medical  communities. 

• A discourse  community  has  a threshold  level  of  members  with  a suitable  degree  of  relevant  content  and 
discoursal  expertise. 

Swales’  concept  of  discourse  community  can  be  applied  to  IT  and  T discourse.  It  is  argued  that  IT 
and  T does  not  exist  in  a cultural  vacuum.  It  has  become  “a  cult  with  its  own  esoteric  language  and  rites,  Mn’ 
groups  and  ‘out’  groups,  and  above  all  faith  in  the  intrinsic  value  of  electronic  communications”  (Moran 
1997).  Aoki,  Fase  & Stowe  (1998)  introduced  the  term  ‘virtual  micro  culture’  as  a unique  group  dynamics, 
which  arises  as  a result  of  frequent  interaction  and  collaboration  toward  a common  goal.  It  has  its  own 
culture  and  anyone  who  is  exposed  to  it  will  go  through  a ‘socialisation  experience’,  namely  acculturation. 
The  stages  described  above  reflect  well  the  acculturation  process  experienced  by  its  users  when  IT  and  T are 
introduced. 

A review  of  the  literature  on  the  role  of  educational  multimedia  in  the  past  decade  supports  the  view 
that  acculturation  is  a powerful  phenomenon  inherent  in  various  experiences  involving  educational 
multimedia.  While  the  business  world  has  embraced  IT  and  T wholeheartedly,  educators,  in  comparison, 
have  been  rather  slow  in  accepting  them  in  teaching  and  learning.  In  some  cases  technological  resistance 
occurs  due  to  passive  or  active  resistance  to  adopting  technology  (Allen  1997).  Though  the  concepts 
‘teleteaching’,  ‘virtual  classroom’,  ‘Internet-based  teaching’  have  been  introduced  during  the  past  decade 
and  occur  in  various  curriculum  discussions,  in  reality  the  full  adoption  of  IT  and  T into  practice  is  still  a 
long  way  off  Teaching  is  still  conducted  in  a predominantly  non-virtual  context,  in  which  the  traditional 
resources  are  teachers,  books  and  blackboards  (or  overhead  projectors).  But,  significantly,  there  has  been  a 
mixture  of  feelings  such  as  hostility,  rejection,  fear,  and  confusion  among  educators  in  relation  to  the  use  of 
IT  and  T in  teaching  and  learning.  According  to  a study  by  Holt  & Thompson  (1996),  old  attitudes,  held  by 
long-time  serving  staff,  limited  vision  of  what  was  possible  in  the  flexible  learning  mode.  To  facilitate  the 
experience  of  acculturation,  it  is  desirable  to  start  with  a relatively  clean  cultural  slate  than  to  inherit  a fully 
mature,  but  unreceptive  culture. 

There  are  three  aspects  of  acculturation  in  education.  Firstly,  IT  and  T experts  are  contributing 
factors  to  the  negative  attitudes  among  IT  and  T prospective  users.  The  worst  case  is  when  such  experts  hold 
the  assumptions  that  IT  and  T is  the  magic  solution  to  education  and  that  teachers  are  secondary  in  the 
teaching  and  learning  process.  IT  and  T are  therefore  the  magic  wand,  which  can  effectively  replace 
teachers.  The  curriculum  concept  ‘learner-centred’  is  sometimes  confusing  in  the  sense  that  it  mis-signals  a 
shift  of  significance  from  teacher  as  a primary  source  and  learners  as  a secondary  source.  In  other  words, 
some  may  interpret  learner-centred  curriculum  as  role-oriented  rather  than  process-oriented.  This  creates  a 
misconception  that  IT  and  T can  replace  the  teacher.  However,  if  ‘learner-centred’  is  interpreted  as 
‘learning-centred’,  the  emphasis  is  not  on  the  comparison  (whether  implicit  or  explicit)  of  students’  and 
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teachers’  roles,  but  on  the  process  of  learning  itself  In  this  way,  IT  and  T can  play  a significant  role  in 
improving  learning  in  general  and  in  enhancing  positive  attitudes  towards  IT  and  T in  particular. 

Secondly,  a paradigm  shift  is  occurring  in  education.  According  to  Roberts  (1997)  this  is  “the 
rejection  of  one  set  of  values  and  ideas  about  education  and  the  adoption  of  a new  set  with  regards  to  what 
constitutes  effective  pedagogy”.  This  paradigm  shift  is  occurring  worldwide  but  faster  in  some  parts  than 
others  depending  on  the  availability  of  resources,  existing  infrastructure  and  the  stage  of  development 
reached, 

A new  paradigm  should  not  be  regarded  as  old  wine  in  a new  bottle  as  it  is  not  only  the  technology 
that  counts  but  a new  spirit  with  a new  discourse  community  that  has  emerged.  Therefore  strategies  have  to 
be  planned  carefully  to  facilitate  acculturation  of  current  and  prospective  users  to  a new  educational 
paradigm  created  by  IT  and  T.  Otherwise  the  acculturation  experiences  can  be  painful  and  sometimes 
destructive.  As  a result,  hostility  can  be  seen  in  headlines  in  the  media  such  as  ‘students  damaged  computers 

It  is  important  to  involve  users  in  the  curriculum-decision  making  process  in  their  acculturation  into 
an  unknown  or  less  familiar  territory.  In  the  past,  IT  educators  have  tried  to  reverse  the  direction  of 
discourses  by  placing  IT  in  the  discourse  of  education  rather  than  placing  education  in  the  discourse  of  IT, 
For  instance.  Moos’  work  has  influenced  the  development  and  use  of  instruments  to  assess  the  qualities  of  the 
classroom  learning  environment  from  the  perspective  of  the  learner  (Waldrip  & Fisher  1997),  Though  IT 
and  T can  create  a perfect  virtual  learning  environment,  learners  should  not  be  treated  as  virtual  beings. 
They  bring  to  a virtual  class  and  the  acculturation  process  their  personal,  social  and  cultural  backgrounds, 
which  shape  their  learning  styles,  educational  attitudes,  and  interpersonal  relationship. 

Thirdly,  there  should  be  a good  harmony  between  learning  IT,  learning  about  IT  and  learning 
through  IT,  Otherwise,  the  perception  of  IT  and  T in  the  curriculum  will  be  challenged. 

Stage!:  Learning  IT: 

This  is  a hands-on  practical  knowledge  about  IT  and  T such  as  word-processing,  emailing.  Web- 
navigation,  scanning. 

Stage  2:  Learning  about  IT: 

This  involves  concepts  and  issues  relating  to  IT  and  T such  as  interactivity,  ethical 
consideration,  security  etc. 

Stage  3:  Learning  through  IT: 

This  involves  literature  search,  conferencing  with  teachers  and  peers,  publishing  etc. 

These  three  aspects  should  not  be  treated  in  a linear  progression.  In  other  words,  the  syllabus 
should  not  proceed  rigidly  from  stages  1 to  2 and  3,  In  an  interactive  dynamic  learning  process,  users  may 
start  with  their  need  to  share  with  others  their  views  on  certain  issues  (learning  through  IT),  Then  they 
proceed  with  how  to  use  the  Web  for  interacting  with  others  (learning  IT),  While  doing  this,  they  are  aware 
of  the  use  and  limitations  of  the  Internet  (learning  about  IT), 


Rural  Health  Education:  an  Example  of  Acculturation 

We  have  discussed  the  concept  of  acculturation  in  relation  to  the  culture  of  IT  and  T,  Now  the  focus 
is  on  the  implementation  of  acculturation  in  the  context  of  IT  and  T and  rural  health  education  in  Tasmania, 

Thomson  and  Walker  (1997)  demonstrated  the  role  of  communications  technology  in  health  care 
provision  through  identification  of  various  communications  techniques  and  analysis  of  these  in  relation  to 
health  care  delivery  and  health  professional  education.  They  showed  how  communications  technology 
provides  a logical  link  between  education  and  health  care  delivery  by  acting  as  a tool  for  change  to  the 
knowledge,  skills  and  attitudes  of  health  care  professionals  towards  rural  health. 

Walker  (1998a)  noted  that  the  level,  quality  and  cost  of  telecommunications  infrastructure  for  rural 
and  remote  populations  in  Australia  is  being  addressed  via  the  Regional  Telecommunications  Infrastmcture 
Fund  (RTIF),  known  as  ‘Networking  the  Nation’.  The  aim  is  to  reduce  the  gaps  in  the  quality  and  cost  of 
telecommunications  services  between  metropolitan  and  rural  Australia.  She  also  explained  how  the 
Commonwealth  Government  in  Australia  is  establishing  and  funding  a number  of  university  departments  of 
rural  health.  In  Tasmania,  Australia’s  small,  rural  island  state,  a University  Department  of  Rural  Health  was 
set  up  in  late  1997.  It  acts  as  a catalyst  for  the  development  of  a comprehensive  and  co-ordinated  approach  to 
rural  health  education,  training  and  research  across  the  State  and  across  the  range  of  relevant  disciplines  and 
professions.  The  aim  of  the  Department  is  to  provide  access  to  education,  training  and  support  opportunities 
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for  rural  health  workers  and  trainees  through  an  open  and  flexible  learning  framework  supported  by  an  IT 
and  T base.  It  is  built  on  a principle  of  partnership  with  health  service  providers  and  communities  and  is 
part  of  Tasmania’s  developing  Telehealth  Network  to  provide  access  to  health  services  and  rural  health 
education  services  using  technology  as  a tool.  The  Department  has  implemented  a series  of  strategies  to 
enhance  the  implementation  of  information  technology  and  telecommunications  in  rural  health.  A top 
priority  is  acculturation  into  the  new  IT  and  T discourse  community. 

People  who  are  involved  in  rural  health  care  are  drawn  from: 

• health  professionals,  directly  responsible  for  patient  care  e.g.,  doctors,  nurses,  dentists,  pharmacists, 
radiologists,  pathologists  and  technicians; 

• administrative  and  support  staff  involved  in  health  care  administration  e.g.,  business  managers,  service 
development  managers,  patient  information  management  officers  and  corporate  strategists. 

Increasingly,  many  of  these  people  do  not  work  in  rural  areas  but  are  based  in  urban-based  large 
hospital  environments.  Through  Telehealth,  services  are  provided  at  a distance  and  rural,  locally  based 
health  care  providers  have  a vital  role  to  play  in  ensuring  successful  delivery  of  services  and  enhanced  health 
outcomes.  However,  most  health  care  workers  do  not  have  a sound  understanding  of  the  fundamentals  of 
Telehealth  and  the  use  of  IT  and  T for  education  purposes  (Hasman  & Albert  1997).  Many  exhibit  feelings 
of  hostility,  rejection,  fear  and  confusion  in  relation  to  use  of  IT  & T in  health  care  provision  and  access  to 
education  and  training  opportunities.  This,  in  tune,  reflects  negatively  on  consumers  of  rural  health  services. 
It  makes  it  difficult  for  practitioners  to  make  decisions  about  the  usefulness  of  applications  and  to  express 
their  needs  in  terms  that  can  be  understood  by  informaticians  and  IT  and  T personnel.  Education  and 
training  in  the  place  and  use  of  IT  & T is  needed  at  a variety  of  educational  levels,  in  different  modes,  for 
various  professions  in  health  care  and  with  different  types  of  specialisation. 

In  Tasmania,  the  Department  of  Rural  Health  and  the  State  Department  of  Health  and  Human 
Services  have  embarked  on  a collaborative  project.  This  will  see  the  progressive  roll  out  of  Telehealth  and  IT 
and  T education  and  training  to  meet  an  increasing  demand  for  information  management  and  the  use  of 
information  and  telecommunications  technologies  to  enhance  health  services,  to  access  education  and 
training,  and  to  assist  in  the  acculturation  process. 

A major  objective  is  to  assist  the  change  process  to  a health  information  management  and 
information  technology  and  telecommunications  environment  through  training,  including  the  use  of  IT  and  T 
to  deliver  training  that  is  customised  to  workplace  needs.  It  requires  strategic  planning  across  a range  of 
services  and  collaboration  in  use  of  shared  resources.  It  requires  training  that  is  based  on  the  recognition  of 
the  role  that  acculturation  plays  in  a new  discourse  community  (Walker  1998b). 

At  the  same  time,  the  University  Department  of  Rural  Health  is  establishing  a network  of  Rural 
Health  Teaching  Sites  (RHTS)  in  collaboration  with  the  Telehealth  Network.  The  Department  is  facilitating 
rural  health  education  and  training  programs  for  undergraduates  (medical,  nursing  pharmacy  and  allied 
health  students)  in  targeted  rural  areas  and  is  building  on  the  intellectual  capital  of  these  rural  areas  by 
providing  on-going  teaching,  research  and  service  development  support  to  resident  health  care  workers. 
Each  Site  has  a defined  catchment  area  and  provides  accommodation  and  learning  facilities  with  computing, 
fax,  internet  access,  audio  and  videoconferencing.  An  important  part  of  the  program  is  provision  of  training 
in  the  use  of  the  technologies. 

The  three  aspects  relating  to  the  role  of  IT  and  T have  been  taken  into  consideration:  Learning  IT, 
Learning  about  IT  and  learning  through  IT.  Through  the  RHTS  rural  users  have  sound  and  reliable  access  to 
IT  and  T.  Teleteaching  and  teleinteraction,  with  awareness  and  sensitivity  to  rural  health  issues  and  rural 
users,  have  been  carefully  planned  and  introduced.  The  Department’s  IT  strategies  have  targeted  rural  health 
workers  who  have  been  disadvantaged  by  their  isolation  from  the  IT  and  T discourse  community  in  rural 
health.  For  example,  the  Australian  Physiotherapists’  North-East  Journal  club  runs  internet  access  lunch  time 
get-togethers  to  share  the  meaning  making  process  about  IT  and  T with  local  health  staff  and  students.  This 
event  is  organised  and  co-ordinated  by  the  University  Department  of  Rural  Health.  Without  this  effort 
acculturation  into  the  IT  and  T discourse  community  will  never  take  place. 


Conclusion 

IT  and  T is  permeating  deeply  into  various  aspects  of  society.  The  IT-based  superhighway  is 
travelling  to  many  remote  parts  of  the  world  and  has  brought  with  it  the  technological  power  to  make  human 
communication  dynamically  interactive  and  effective.  It  has  certainly  brought  people  in  the  world  much 
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closer.  However,  the  power  of  IT  and  T is  often  viewed  without  taking  into  consideration  the  potential 
conflict  between  IT  culture  and  the  acculturation  experienced  by  those  who  are  introduced  to  it.  While  IT 
and  T can  create  many  virtual  experiences,  we  must  remember  that  potential  IT  and  T users  are  human  not 
virtual  beings  and  fear,  anxiety,  confusion,  resistance  and  inspiration  are  part  of  the  acculturation  process 
towards  an  IT  and  T discourse  community. 
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Abstract: 

This  paper  reports  the  findings  of  a study  of  how  university  teachers  approach 
teaching  with  one  form  of  telecommunications  media  - cross-campus  video 
conferencing.  Typically,  the  assumption  underlying  training  for  teachers  in  the 
use  of  telecommunications  media  is  that  teaching  will  improve  as  a result  of 
improved  use  of  the  technology.  However,  this  study  reveals  that  factors  other 
than  the  role  of  the  technology  are  crucial  in  determining  the  way  teachers 
conduct  their  teaching.  The  study  shows  that  a more  pedagogically-oriented 
understanding  of  teaching  with  telecommunications  media  is  needed  to  inform 
faculty  development  programs  intended  to  improve  teachers’  approaches  to 
teaching.  The  findings  of  this  study  have  direct  implications  for  the  use  of 
other  forms  of  educational  telecommunications  media  in  university  and  other 
educational  settings. 


No  medium,  however  useful,  can  solve  educational  problems.  Media  cannot  alter  the  way  teachers 
understand  teaching  (Ramsden,  1992,  p,  161), 


Introduction 

The  shift  towards  more  student-centred,  open  and  flexible  approaches  in  university  education,  with  their 
consequent  reduction  in  on-campus  attendance,  is  largely  based  on  the  capacity  of  telecommunications 
media  to  provide  for  various  forms  of  real-time  and  delayed  interaction  (Mason,  1994).  In  light  of  this 
development,  institutions  face  the  significant  task  of  supporting  on-campus  teachers  experienced  in  face-to- 
face  delivery  in  the  development  of  effective  approaches  to  teaching  with  one  or  more  forms  of 
telecommunications  media. 

The  research  reported  in  this  paper  was  undertaken  to  inform  the  design  of  faculty  development  programs 
relating  to  teaching  conducted  by  one  form  of  telecommunications  media  - cross-campus  video 
conferencing.  Typically,  telecommunications  media  are  used  to  break  down  the  classroom-student  nexus 
by  reducing  contiguous  teaching  and  learning  situations  (e.g.  online  courses). 

However,  cross-campus  video  conferencing  teaching  stands  apart  from  most  other  uses  of 
telecommunications  media  in  a fundamental  way  and  presents  teachers  with  an  enormous  conundrum.  As 
with  on-campus  education,  it  consists  of  a scheduled  class  in  a fixed  classroom  location  and  involves 
group-based  learning.  The  arrangement  of  the  facilities  and  technology  most  commonly  presumes  a 
teacher-led  lesson  where  the  control  of  the  technical  facilities  is  placed  at  the  teacher’s  disposal  at  the  front 
of  the  classroom.  Yet,  this  situation  combines  the  teacher-student  arrangement  and  synchronous  interaction 
of  the  regular  classroom  with  the  teacher-student  separation  characteristic  of  distance  education.  Generally, 
it  involves  a group  of  students  in  attendance  at  the  same  site  as  the  teacher  forming  a regular  face-to-face 
class  whilst  the  teacher  is  also  physically  separated  from  the  other  student  group(s)  gathered  at  a similar 


classroom  facility.  As  in  traditional  distance  education,  often  the  teacher  may  have  no  direct,  personal 
contact  with  any  of  the  distant  students  throughout  the  duration  of  the  course. 

The  literature  on  teaching  by  video  conferencing  has  largely  emphasised  the  primary  role  of  the  technology 
in  the  teaching  process.  The  key  assumption  made  by  researchers  and  authors  of  training  manuals  alike  is 
that  the  technology  is  at  the  core  of  teaching  in  these  situations.  From  this  perspective,  teaching  (and 
learning)  will  be  improved  by  enhancing  the  teacher’s  skill  in  working  with  the  technology  (Cyrs,  1989; 
Ostendorf,  1995).  Schiller  and  Mitchell  (1993)  claim  that  video  conferencing  ‘requires  a different  teaching 
methodology  from  any  that  lecturers  have  used  previously’  (p  50).  Overall,  the  emphasis  in  the  literature 
and  guides  on  how  to  teach  by  video  conferencing  is  on  how  teaching  happens. 

Does  a focus  on  the  technology  and  how  it  is  used  result  in  the  most  effective  approaches  to  teaching  ? Is 
the  technology  the  critical  element  of  teaching  by  video  conferencing?  The  present  study  is  concerned  with 
the  way  teachers  understand  teaching,  in  other  words  the  meaning  it  has  for  them,  when  it  takes  places  in 
the  cross-campus  video  conferencing  setting.  It  commences  from  the  assumption  that  how  teachers  make 
sense  of  the  context  within  which  they  act  is  integrally  linked  to  how  they  approach  their  teaching. 

Considerable  research  has  been  conducted  into  university  teaching  and  learning  over  the  past  three  decades 
(Entwistle  & Ramsden,  1983;  Marton  et  al.,  1997).  One  of  the  key  ideas  to  emerge  from  this  research  is 
that  the  content  of  student  learning  {what  is  taught)  logically  precedes  the  method  of  teaching  content  {how 
it  is  taught)  (Ramsden,  1992).  In  other  words,  without  a content  to  teach,  there  is  no  teaching  method. 
From  this  perspective,  at  the  core  of  the  teaching  process,  including  when  it  involves  the  use  of  any  form  of 
telecommunications  media,  is  the  way  the  students  are  brought  into  engagement  with  the  specific  content  of 
their  learning  by  the  teacher.  The  teacher’s  critical  role  concerns  the  way  in  which  the  content  is 
represented  to  the  students,  and  how  they  structure  the  student  learning  experience.  Previous  research  into 
teaching  with  telecommunications  media  has  paid  little  or  no  attention  to  these  elements  by  narrowly 
conceptualising  the  teacher  as  the  user  of  a technology  through  which  the  teaching  is  conducted. 

Providing  an  overview  of  the  pedagogically-focused  research,  Ramsden  (1992)  says  effective  university 
teachers: 

• engage  the  student  at  the  appropriate  level  of  their  understanding; 

• focus  on  key  concepts,  and  students’  misunderstandings  of  them,  rather  than  covering  the  ground; 

• enthuse  students  to  make  the  material  being  taught  interesting  and  stimulating; 

• explain  the  material  in  simple,  concise  ways; 

• use  methods  and  set  tasks  which  require  students  to  learn  actively  and  collaborative ly; 

• provide  high-quality  feedback  on  student  work;  and 

• use  valid  assessment  methods. 

Researching  teachers’  experience  of  teaching  by  video  conferencing 

One  of  the  most  influential  approaches  to  research  into  teaching  and  learning  in  higher  education  has  been 
undertaken  within  phenomenography.  This  qualitative  approach  to  research  developed  in  Sweden  in  the 
early  1970s  and  grew  out  of  a concern  with  the  relationship  between  the  ways  in  which  students 
approached  learning  and  the  learning  outcome.  Phenomenography  assumes  that  the  world,  or  aspects  of  it, 
do  not  exist  in  isolation  from  the  individual  experiencing  it.  Rather,  the  only  world  that  can  be  known  to 
the  individual  (and  the  researcher)  is  an  experienced,  and  therefore  an  interpreted,  world. 
Phenomenographic  studies  focus  on  individuals’  descriptions  of  their  experiences  of  a specific 
phenomenon.  The  aim  is  to  describe  the  qualitatively  distinct  ways  in  which  a phenomenon  has  been 
experienced  or  understood  (phenomenography  uses  these  terms  interchangably)  by  a particular  group  of 
people. 

Phenomenography  contends  that  some  ways  of  understanding  a phenomenon,  or  aspects  of  the  world,  are 
more  sophisticated  than  others;  and  for  this  reason  it  has  been  used  overwhelmingly  in  educational  studies. 
In  more  than  two  decades,  phenomenography  has  developed  a rich  heritage  of  studies  primarily  concerned 


with  aspects  of  student  learning  and,  more  recently,  teachers’  approaches  to  teaching  (Marton  & Saljo, 
1976;  Trigwell  & Prosser,  1997). 


From  the  phenomenographic  perspective,  teachers  do  not  simply  teach  differently  by  cross-campus  video 
conferencing.  Rather,  teachers  experience  or  understand  the  phenomenon  of  teaching  by  cross-campus 
video  conferencing  in  fundamentally  different  ways.  The  variation  in  how  teachers  approach  their 
teaching,  in  other  words  which  aspects  of  the  situation  they  are  most  concerned  with,  reflects  the  different 
ways  they  experience  or  understand  teaching  in  that  situation.  With  phenomenographic  data  drawn  from 
empirical  research,  it  is  possible  to  conduct  faculty  development  programs  which  aim  to  shift  an  individual 
teacher’s  experience  or  understanding  of  the  phenomenon  of  teaching  by  cross-campus  video  conferencing 
to  one  which  is  more  pedagogically  sound. 


The  present  study 

The  present  study  was  concerned  with  the  teachers’  perspectives  of  their  experience  of  teaching  by  video 
conferencing.  It  aimed  to  identify  which  aspects  of  the  teaching  process  teachers  were  most  concerned 
with,  and  which  most  affected  teachers’  approaches  to  teaching,  in  the  cross-campus  video  conferencing 
classroom  situation.  The  study  asked  the  question:  How  do  teachers  experience,  and  make  sense  of, 
teaching  by  video  conferencing?  It  involved  ten  examples  of  university  teaching  by  cross-campus  video 
conferencing  and  included  twenty-three  teachers. 

The  study  was  based  primarily  on  open-ended,  semi-structured  interviews.  Teachers  were  interviewed 
prior  to  a specific  class.  Based  on  the  view  of  teaching  underpinning  this  study,  the  interviews  initially 
centred  on  the  content  to  be  taught,  how  the  teachers  intended  students  to  learn  that  content,  whether  their 
teaching  by  video  conferencing  differed  to  teaching  in  the  regular  classroom  and,  if  so,  how?  These  aspects 
of  the  teaching  (and  learning)  process  were  not  conceptually  separated  from  the  teacher’s  use  of  the 
technology,  but  were  seen  to  provide  the  rationale  for  its  use.  The  class  was  subsequently  observed,  and  a 
post-class  interview  was  conducted  to  discuss  what  had  occurred. 

What  is  at  the  core  of  teaching  by  cross-campus  video  conferencing? 

The  teachers  in  the  present  study  focused  on  a number  of  critical  elements  of  the  teaching  process  when 
teaching  by  cross-campus  video  conferencing.  In  contrast  with  the  research  literature’s  fixation  on  the 
technology,  the  study  showed  that  teachers  had  a primary  focus  on  one  of  three  broad,  distinct  aspects,  with 
critical  distinctions  in  each  category.  They  were: 


The  teaching  environment. 

Several  teachers  were  predominantly  concerned  with  the  physical  context  of  the  video  conferencing 
classroom,  including  the  technology,  the  method  of  communication  between  linked  sites,  and  the 
arrangement  between,  and  the  governance  of,  the  physically  separated  classrooms. 

Given  its  critical  role,  it  is  not  surprising  that  the  technology  was  the  primary  concern  for  some  teachers. 
From  the  phenomenographic  perspective  of  this  study,  these  teachers  can  be  said  to  have  the  least 
sophisticated  understanding  of  teaching  by  cross-campus  video  conferencing.  They  were  preoccupied 
with  the  affect  of  the  technology  on  their  own  role  as  teacher.  Any  difficulties  found  in  the  teaching 
environment  were  viewed  in  terms  of  the  inconvenience  caused  to  the  teacher,  rather  than  in  terms  of  the 
impact  on  student  learning.  In  particular,  the  teachers  felt  a massive  dependence  on  the  technology  which 
related  to  the  reliability  of  its  operation,  rather  than  how  teachers  would  use  it.  Prior  to  each  lesson 
teachers  were  uncertain  whether  or  not  the  class  would  be  convened  until  the  video  conferencing 
technology  established  the  link  between  distant  sites. 


Furthermore,  some  teachers’  understandings  of  what  constituted  the  technology  extended  to  the  technical 
support  staff  who  were  the  embodiment  of  the  teachers’  dependency.  Teachers  felt  that  they  had  little  or 
no  control  over  the  operation,  maintenance  and  update  of  technical  facilities.  Unlike  the  regular 
classroom,  where  the  teachers  felt  they  had  considerable  autonomy  over  their  teaching,  the  video 
conferencing  classroom  was  predominantly  the  domain  of  the  technical  support  staff  where  teachers  were 
relegated  to  a secondary  role.  On  another  level,  the  teaching  and  learning  environment  was  also  formed  by 
the  medium  of  communication  provided  by  video  conferencing.  Some  teachers  spoke  of  the  barrier  it 
created  for  communicating  between  participants  compared  with  the  regular  classroom;  particularly  the 
difficulty  in  reading  facial  expressions  and  problems  in  hearing  student  responses. 

A further  problem  related  to  the  two  or  more  classroom  sites  joined  electronically  and  the  problems 
teachers  faced  in  managing  student  activity  and  measuring  student  responses  to  set  tasks  undertaken  at 
distant  locations. 

How  does  a focus  on  the  teaching  environment  affect  a teacher  *s  approach  to  teaching?  Because  of  the 
comparative  complexity  of  the  video  conferencing  classroom  and  its  impact  on  their  own  role,  these 
teachers  had  an  extremely  negative  view  of  teaching  by  video  conferencing.  The  teachers  adopted  a very 
didactic,  information-delivery  approach  which  relegated  students  to  passive  transcribers  of  a presentation 
made  by  the  teacher  at  the  front  of  the  classroom.  The  students  in  these  classes  were  presented  with  no 
opportunity  to  influence  any  aspect  of  the  teaching  and  learning  process. 

Importantly,  these  teachers  treated  the  physically  separated  student  groups  as  distinct  classes  Joined 
electronically,  rather  than  as  a single,  unified  class.  As  a consequence,  those  student  groups  remote  from 
the  teaching  site  received  relatively  less  attention  from  the  teacher  and  were  reduced  to  distant  observers  of 
a process  which  centred  on  the  teacher’s  immediate  physical  location. 

Teachers  in  this  category  made  no  attempt  to  work  collaboratively  with  teaching  colleagues  in  the  design 
or  delivery  of  their  teaching  programs.  Instead,  the  teaching  input,  was  largely  governed  by  each  teacher’s 
physical  location  and  can  best  be  described  as  sequential,  with  various  teachers  making  individual 
contributions  to  specific  sections  of  the  program.  Equally,  these  teachers  made  no  effort  to  co-operate 
with  technical  support  staff  to  ensure  a more  effective  use  of  the  facilities. 


The  teaching  process. 

Teachers  in  this  category  had  a more  sophisticated  view  of  teaching  by  cross-campus  video  conferencing 
than  those  with  a focus  on  the  teaching  environment.  They  were  primarily  concerned  with  their  own  role 
as  teacher  in  the  video  conferencing  classroom.  These  teachers  either  focused  on  replicating  their  previous 
teaching  practice  in  the  regular  classroom  in  the  new  situation;  or  were  intent  on  improving  their  current 
practice  to  become  better  video  conferencing  teachers. 

Although  they  were  not  concerned  with  the  impact  of  the  environment  on  their  own  performance,  they 
remained  fixed  on  what  they  would  do  as  teachers  in  the  situation,  rather  than  the  implications  this  may 
have  for  the  quality  of  the  student  learning  experience.  Teachers  with  this  focus  did  not  make  a direct  link 
between  the  teaching  process  and  student  learning. 

How  does  a focus  on  the  teaching  process  affect  a teacher's  approach  to  teaching?  Some  teachers  in  this 
category  simply  attempted  to  transfer  previous  classroom  teaching  practice  into  the  video  conferencing 
environment.  They  saw  the  context  as  one  which  accommodated  their  existing  practice.  However,  in 
doing  so  the  teachers  failed  to  take  into  account  the  specificity  of  the  circumstances  (ie.  multiple  student 
groups;  more  than  one  teacher;  different  requirements  for  responding  to  distant  students;  the  impact  of  the 
telecommunications  media  on  interaction)  due  to  their  concern  with  their  own  teaching  role. 

Other  teachers  in  this  category  did  make  adjustments  for  peripheral  elements  of  the  new  environment  (ie. 
changing  forms  of  text  and  graphics  to  comply  with  presentation  methods).  However,  in  these  instances 
the  changes  were  similarly  driven  by  a primary  concern  with  their  own  teaching  role,  rather  than  the 
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potential  to  enhance  student  learning.  Although  in  relative  terms  they  worked  in  closer  co-operation  with 
teaching  colleagues,  these  teachers  did  not  seek  to  maximise  the  potential  for  working  collaboratively  with 
other  teachers  who  were  also  engaged  in  the  subject,  or  with  technical  support  staff 


Student  learning. 

From  the  perspective  underpinning  this  study,  a student  learning  focus  is  the  most  sophisticated  and 
preferred  way  of  experiencing  or  understanding  teaching  by  cross-campus  video  conferencing.  In  this 
category,  teachers  were  overwhelmingly  concerned  with  the  student  learning  experience,  rather  than 
environmental  factors,  or  their  own  role  as  teacher.  In  addition,  some  teachers  saw  in  the  new  situation  the 
prospect  for  improving  student  learning  by  enhancing  the  technical  system,  or  by  incorporating  video 
conferencing  to  enable  additional  chances  for  student  interaction  and  participation. 

How  does  a learning  focus  affect  a teacher's  approach  to  teaching?  Teachers  with  a learning  focus  placed 
the  student  learning  experience  at  the  heart  of  their  practice.  The  video  conferencing  classroom  became  a 
setting  in  which  they  aimed  to  actively  involve  students  in  their  own  learning.  Teachers  with  this  focus 
viewed  the  situation  very  positively,  and  sought  the  advantages  it  offered  to  students.  Their  teaching  was 
conducted  as  facilitation,  in  which  their  role  was  to  maintain  momentum  in  a class  which  was  driven 
overwhelmingly  by  the  students’  contributions  and  interests.  As  a result,  these  teachers  recognised  the 
value  of  close  collaboration  in  the  teaching  process  through  which  students  would  gain  access  to  a wider 
range  of  expertise  and  knowledge  regardless  of  their  physical  proximity  to  an  individual  teacher. 

Teachers  with  a learning  focus  were  aware  that  the  situation  of  multiple  student  groups  provided  an 
opportunity  to  engage  students  in  ways  which  would  enhance  the  learning  experience  of  the  total  group. 
Most  importantly,  teachers  with  a learning  focus  were  the  only  ones  to  encourage  and  expect  students  to 
operate  the  technology  of  video  conferencing,  thereby  further  reducing  the  role  of  the  teacher  in  that 
classroom  setting.  These  teachers  emphasised  what  student  would  do  in  the  context,  rather  than  their  own 
operational  or  presentational  skills,  and  worked  closely  with  technical  support  staff  to  ensure  the  most 
efficient  and  effective  environment  for  students. 


Implications  for  improving  teaching  by  cross-campus  video  conferencing 

The  concept  of  what  teaching  is  in  the  video  conferencing  classroom  needs  to  be  seen  by  researchers  and 
faculty  developers  as  a pedagogical  process  rather  than  a technical  one.  Faculty  development  programs 
relating  to  telecommunications  media  have  concentrated  on  improving  teachers’  operational  and 
presentational  skills  for  one  simple  reason:  it  has  been  assumed  that  this  is  the  most  important  and 
distinctive  feature  of  teaching  in  these  contexts.  The  findings  of  this  study  give  cause  to  reconsider  the 
design  of  faculty  development  programs  relating  to  teaching  by  video  conferencing,  and  the  principal 
assumption  underlying  such  programs. 

Teaching  by  video  conferencing  should  not  be  conceived  primarily  as  the  act  of  using  a 
telecommunications  medium  effectively.  Although  teacher  proficiency  with  the  technology  will  assist 
student  learning,  the  teacher’s  response  to  other  key  aspects  of  the  teaching  process  has  a much  greater 
impact  on  their  approach  to  teaching.  Where  the  technology  is  the  primary  concern,  this  study  shows  that 
teachers’  understanding  of  the  technology  is  much  more  complex  than  the  literature  generally 
acknowledges  (ie.  it  creates  the  method  and  form  of  interaction). 

Clearly,  some  ways  of  seeing  the  act  of  teaching  by  video  conferencing  are  more  preferable  than  others. 
From  a broad  phenomenographic  perspective,  faculty  development  programs  should  be  based  upon  the 
present  findings  that  teachers  have  a range  of  primary  concerns,  or  ways  of  experiencing  teaching  by  video 
conferencing.  The  aim  of  faculty  development  programs  firstly,  should  be  to  establish  how  individual 
teachers  understand  teaching  in  that  context,  rather  than  assume  that  the  technology  is  the  predominant 
concern  for  all.  Then,  teachers  should  be  made  aware  how  other  teachers  in  the  program  understand 
teaching  by  video  conferencing,  thereby  introducing  each  teacher  to  the  range  of  understandings,  and  the 
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implications  for  how  teachers  approach  their  teaching.  Ultimately,  the  intention  should  be  to  shift  the 
teacher’s  understanding  to  a more  sophisticated  one.  Faculty  development  programs  should  aim  to  produce 
a student  learning  focus  where  the  teacher  is  fundamentally  directed  to  how  students  will  be  brought  into 
engagement  with  the  lesson  content  in  the  relative  complexity  of  the  video  conferencing  classroom. 

Faculty  development  programs  should  also  address  several  of  the  key  issues  to  emerge  from  the  present 
study  which  are  directly  related  to  how  teachers  approach  teaching  by  video  conferencing.  For  instance, 
there  is  a clear  need  to  promote  a view  of  the  separate  student  groups  as  a single,  unified  class.  Such  an 
approach  emphasises  the  value  of  the  group*based  learning  experience,  which  is  the  essence  of  the  video 
conferencing  classroom,  where  the  aim  should  be  to  generate  interaction  between  all  students.  In  group- 
based  learning  contexts,  interaction  should  be  the  intention,  and  the  group  members  viewed  as  sources  of 
knowledge  and  experience  to  be  utilised  for  the  common  good. 

Finally,  the  study  demonstrates  that  teachers  need  to  work  collaboratively  in  the  design  and  teaching  of  a 
program,  with  all  teachers  assuming  responsibility  for  the  learning  of  the  whole  student  group.  Faculty 
development  programs  should  directly  address  the  concept  and  process  of  team  teaching  rather  than 
assuming  that  collections  of  teaching  colleagues  will  necessarily  work  effectively.  In  order  to  have  the 
greatest  impact  of  student  learning,  faculty  development  programs  should  be  directed  at  course  or  subject 
teams  rather  than  individual  teachers.  The  concept  of  collaboration  should  also  extend  to  the  role  of 
technical  support  staff  who  have  a major  role  in  assisting  teachers  to  function  effectively  in  the  video 
conferencing  classroom,  and  who  should  be  regarded  as  members  of  the  teaching  team  in  development 
programs. 

In  broader  terms,  the  approach  to  conceptualising  and  researching  teaching  by  video  conferencing  reported 
in  this  paper  also  has  major  implications  for  other  forms  of  telecommunications  media  as  well  as  other 
educational  sectors. 
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Abstract:  The  purpose  of  this  research  was  to  assess  the  impact  of  a school  district 
technology  initiative  on  the  adoption  of  computer  technology  as  an  innovation  for  teaching 
and  learning.  A review  of  the  literature  suggests  that  instructional  technology  is  not  widely 
implemented  in  the  educational  setting.  The  Board  of  Trustees  of  a large  school  board  in 
Western  Canada  operated  a district  Technology  Incentive  Program  (TIP)  from  1996  to 
1999.  This  study  assessed  the  impact  of  the  first  year  of  TIP.  The  data  were  collected 
through  quantitative  and  qualitative  methodologies.  The  results  showed  that  there  was  an 
increase  in  TIP  participants'  perception  of  the  level  of  expertise  in  using  a computer  and 
usage  of  computers  for  teaching  and  non-teaching  purposes.  Results  were  discussed  in 
terms  of  theories  of  change  and  innovation. 


Introduction 

The  purpose  of  this  research  project  was  to:  (a)  examine  the  effects  of  the  Technology  Incentive 
Program  (TIP)  on  selected  aspects  of  the  implementation  of  computer  technology  by  teachers,  and  (b)  make 
recommendations  to  school  districts  to  encourage  teachers  in  adopting  computer  technology. 

In  Western  Canada,  one  large  school  district  initiated  a multi-year  Technology  Incentive  Program 
during  the  school  year  1996-1997,  for  teachers  who  were  novice  users  of  computer  technology,  to  gain 
experience  in  using  computers.  The  Board  of  Trustees  approved  a three  year  district  project  which  provided 
teachers  with  computers  and  in-service  training  to  encourage  them  to  adopt  computer  technology  in  teaching. 

With  the  advance  of  computer  technology,  educators  are  feeling  pressure  to  use  computers  in  the 
classrooms.  Some  parents  are  demanding  changes  in  the  curriculum.  They  want  computer  technology  to  be 
integrated  into  the  curriculum  so  that  their  children  will  be  ready  to  meet  the  challenges  of  an  information 
society  once  they  graduate.  Means  and  Olson,  (1994),  Szabo  & Schwarz  (1997),  and  the  U.S.  Congress,  (1995) 
have  shown  that,  although  computers  are  generally  available  in  schools,  a substantial  number  of  teachers  are 
either  not  using  computers  as  tools  for  teaching,  or  the  use  of  computers  is  at  low  level  (e.g.,  word  processing). 
Some  school  districts  are  looking  for  ways  to  support  teachers  so  they  will  adopt  computer  technology  into  the 
instructional  process. 


Research  Question 

This  study  was  designed  to  address  the  question:  What  impact  did  the  first  year  TIP  have  on  teachers 
with  regard  to  computer  usage?  The  research  question  was  broken  down  into  ten  sub-questions  so  that  the 
impact  of  the  first  year  of  the  TIP  could  be  examined  in  detail. 

Sub-questions  dealt  with  (a)  changes  in  skill  and  knowledge  in  computer  usage,  (b)  usage  of  computers 
for  teaching,  (c)  usage  of  computers  for  non-teaching  purposes,  (d)  the  impact  of  having  a computer  for  one 
year,  (e)  teachers’  attitude  towards  computers,  (f)  TIP  participants’  purchase  of  computer  as  result  of  the  TIP,  (g) 
the  TIP  training  program,  (h)  interest  shown  in  computer-related  activities,  (i)  other  factors  which  influence  the 
use  of  computers,  and  (j)  computer  usage  and  teachers'  perception  of  what  they  need. 


Review  of  the  Literature 


The  review  of  the  literature  shows  that  the  implementation  of  computer  technology  should  be  looked  at 
as  an  embedded  innovation  problem  within  the  process  of  change.  Research  is  needed  in  the  adoption  and 
implementation  of  computer  technology  and  on  the  impact  of  computer  technology  programs  at  district  and 
local  levels.  The  review  and  research  drew  heavily  from  the  work  on  change  and  innovation  of  Fullan  (1991, 
1992),  Rogers  (1995)  and  Szabo  (1996). 


Study  Methods 

Data  were  gathered  from  a survey  of  TIP  participants  for  an  ex  post  facto  study.  To  add  depth  of 
understanding  to  the  study,  seven  TIP  participants  were  also  interviewed. 

Sample  of  Study 

The  target  population  consisted  of  teachers  who  were  novice-users  of  computer  technology  and  would 
volunteer  to  participate  in  programs  similar  to  the  TIP.  The  accessible  population  was  comprised  of:  (a)  teachers 
who  participated  in  the  TIP  in  a large  school  district  in  Western  Canada,  in  1996/1997  and  (b)  teachers  who 
applied  for  the  program  but  were  not  selected  (non-TIP). 

There  were  148  (64%)  TIP  teachers  and  20  (14%)  non-TIP  teachers  who  responded  to  the  survey. 
Seven  TIP  participants  were  chosen,  based  on  grade  levels  taught  to  gain  an  insight  into  the  impact  of  the  TIP  at 
the  different  grade  levels,  to  be  interviewed. 

The  impact  of  the  program  was  defined  by  the  (a)  change  of  perception  of  teachers  in  the  skill  and 
knowledge  or  level  of  expertise  with  computer  usage,  (b)  change  in  the  number  of  hours  of  computer  use  for 
teaching  purposes,  (c)  change  in  the  number  of  hours  of  computer  usage  for  non-teaching  purposes,  and  (d) 
attitude  toward  the  use  of  computers  shown  by  surveyed  teachers  . 

Hypotheses 

It  was  hypothesized  that  TIP  participants  exhibit  a difference  before  and  after  the  TIP,  with  regard  to 
(a)  their  perception  of  their  skill  and  knowledge  in  using  a computer,  (b)  the  number  of  hours  in  computer  usage 
for  teaching  purposes,  and  (c)  the  number  of  hours  in  computer  usage  for  purposes  other  than  teaching. 

It  was  hypothesized  that  Non-TIP  teachers  exhibit  a difference  before  and  after  TIP,  with  regard  to  (a) 
their  perceptions  in  the  level  of  knowledge  and  skill  in  using  a computer,  (b)  the  number  of  hours  they  used  the 
computer  for  teaching  purposes,  and  (c)  the  number  of  hours  they  used  the  computer  for  purposes  other  than 
teaching. 

The  Technology  Incentive  Program 

In  July,  1996,  a large  school  board  in  Western  Canada  approved  $753,000  to  operate  a TIP  in  which 
230  teachers  were  given  a computer  and  software  productivity  suite  for  their  personal  use  for  one  year  and 
approximately  three  weeks  and  four  Saturday  sessions  of  training  in  its  use.  Due  to  the  funding  limit,  the 
administrator  of  the  school  was  asked  by  the  district  to  nominate  some  of  the  teachers  who  volunteered  for  the 
program.  Of  the  371  novice  computer  users  nominated  by  their  schools,  230  were  successful  in  applying  for  the 
program. 

The  summer  workshops  included  sessions  on:  (a)  unpacking  and  setting  up  the  computer  system,  (b) 
the  operating  systems  of  either  Windows  95™,  with  MS  Office™  or  Macintosh™  system  7.5,  with 
ClarisWorks™,  (c)  how  to  use  word  processing  software;  (d)  electronic  presentation;  (e)  CD-ROM  resources 
such  as  Encarta  and  Canadian  Encyclopedia,  and  (f)  the  Internet.  Saturday  workshops  were  offered  in  language 
arts,  mathematics,  science,  and  social  studies  to  teachers  in  Divisions  One  and  Two  (grades  one  to  six). 
Workshops  on  word  processing  across  the  curriculum,  spreadsheets,  database  were  offered  to  teachers  of 
Divisions  Three  and  Four  (grades  seven  to  twelve). 
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Data  Analysis 

Response  Summary 


Five  TIP  teachers  were  not  accessible  due  to  moving  or  leaves  of  absence.  Of  the  1 50  surveys  returned 
by  the  TIP  teachers,  148  (64%)  were  usable  for  data  analysis.  Of  the  21  surveys  returned  by  non-TIP  teachers, 
20  of  them  (14%)  were  usable  for  data  analysis.  The  majority  (76%)  of  the  TIP  participants  were  female  and 
that  69%  of  the  TIP  participants  were  elementary  teachers  (43%  taught  in  Division  One  and  26%  in  Division 
Two). 

Analysis  of  Data 

A comparison  of  the  data  was  made  by  using  paired  t tests  on  the  level  of  expertise  with  regard  to 
computer  technology  the  TIP  participants  perceived  they  had,  before  and  after  the  TIP  training.  Comparison  of 
the  data  was  also  made  by  using  paired  tests  on  the  number  of  hours  the  TIP  participants  used  computers,  for 
teaching  and  non-teaching  purposes,  before  and  after  the  TIP  training.  The  same  procedure  was  used  for  the 
both  TIP  and  non-TIP  participants.  A comparison  across  TIP  and  non-TIP  teachers  was  not  made  due  to  the 
vastly  different  sample  sizes. 


Presentation  of  Findings:  Survey  of  TIP  Participants 

Results  for  TIP  Participants:  Level  of  Expertise  and  Computer  Usage 

Results  of  t tests  showed  that  the  TIP  seems  to  have  made  a significant  impact  on  the  participants  with 
regard  to  their  self-reported  knowledge  and  skill  in  using  a computer.  The  TIP  also  seemed  to  have  made  a 
significant  impact  on  the  participants  with  regard  to  the  use  of  computers  for  teaching  and  non-teaching 
purposes. 

Results  for  TIP  Participants:  Attitude  and  Confidence  in  Computer  Usage 

The  TIP  participants  reported  being  more  confident  in  using  a computer.  They  also  reported  being 
more  proficient  in  using  a computer  and  exhibited  a more  favorable  attitude  toward  computer  technology  for 
teaching  and  learning.  An  overwhelming  majority  of  TIP  participants  held  the  opinion  that  computers  should  be 
used  for  learning  (89%)  and  that  the  computer  is  an  effective  tool  for  teaching  and  learning  (92%).  However, 
only  half  of  the  teachers  agreed  that  they  were  confident  with  using  computer  technology  in  delivering 
instruction. 

Impact  of  TIP  Training  Program  and  Interest  in  Computer-related  Activities  (TIP  Participants) 

The  TIP  was  reported  to  have  made  positive  impact  on  its  participants  regarding  the  use  of  the  Internet, 
clip  art  and  word  processing.  It  neither  made  a great  impact  on  integrating  computer  with  content  area  nor  with 
the  computer  applications  of  spreadsheets,  data  bases  or  transferring  data  between  applications.  The  TIP 
appears  to  have  increased  participation  in  computer  related  activities  such  as  reading  books,  articles  or 
magazines,  and  attending  conferences  on  computer  technology.  As  well,  the  TIP  computer,  a component  of  the 
TIP  program,  appears  to  have  influenced  some  participants  toward  the  purchase  of  a computer. 

Factors  of  Usage  (Other  Than  TIP)  for  TIP  Participants 

When  asked  about  influential  factors  other  than  the  TIP,  TIP  participants  identified  a necessity  to  learn 
to  use  computers  for  report  card  purposes  and  needs  of  students  along  with  help  from  family,  colleagues  and 
friends  and  having  access  to  computers  in  school  or  classroom. 

Perceived  Needs  of  Teachers  (TIP  Participants) 


Participants  in  the  TIP  indicated  requirements  for  (a)  people  support,  (b)  help  in  applying  technology  in 
classroom,  (c)  access  to  hardware  and  technical  support,  (d)  training  strategies,  and  (e)  school  and 
administrative  support. 


Summation  of  Discussion  of  Data  from  TIP  Participants 

Findings  from  Quantitative  Data 

The  findings  of  the  impact  of  the  TIP  were  generally  compatible  with  the  theories  of  change  and 
innovations  as  suggested  by  the  review  of  literature.  Teachers  adapted  to  the  process  of  change  in  their  early 
attempts  to  adopt  computer  technology  as  an  innovation  as  a result  of  the  impact  of  the  training  from  the  TIP. 
However,  there  are  many  other  factors  in  a complex  social  structure,  such  as  a school  district,  which  affect  the 
change  process.  While  it  was  not  possible  to  completely  isolate  the  effects  of  the  TIP  from  this  milieu  of 
complexity,  the  TIP  seems  to  have  had  a not  insignificant  impact. 

Findings  From  Interview  Data 

Interviewees  had  different  reasons  for  their  participation  in  the  TIP.  Overcoming  of  the  fear  of 
technology,  a change  to  a positive  attitude  towards  using  technology  and  an  increase  in  teachers’  use  of 
computers  were  some  of  the  significant  outcomes  of  the  TIP.  Teachers  perceived  that  they  need  help  in  the 
implementation  of  technology  with  curriculum.  The  interviewees  identified  barriers  which  prevented  them  from 
the  successful  implementation  of  computer  technology  in  their  classrooms.  In  general,  the  findings  support  the 
conclusions  made  by  Szabo  and  Schwarz  (1997)  which  stated  that  in  addition  to  training,  teachers  require  a 
broad  range  of  support  (infrastructure)  and  the  ability  to  adopt  the  innovation  to  unique  needs  (a  form  of 
empowerment)  (Tyack  & Cuban,  1995). 

Discussion  of  Results  of  Non-TIP  Group 

Non-TIP  teachers  reported  increases  in  knowledge  and  skill  in  using  a computer  for  personal  and 
teaching  purposes.  They  seemed  to  have  gained  more  confidence  in  using  computer  technology  and  acquired  a 
more  favourable  attitude  towards  computer  technology  for  teaching  and  learning. 

Non-TIP  teachers  appeared  to  participate  in  computer  related  activities  such  as  reading  books,  articles 
or  magazines,  and  attending  conferences  on  computer  technology.  They  also  seemed  to  have  increased  the 
usage  of  word  processing.  However,  they  did  not  seem  to  show  much  increase  in  their  use  of  the  multimedia 
software  and  the  application  programs  of  transferring  data  between  applications,  spreadsheet  or  database. 

The  non-TIP  group  identified  usage  of  computer  for  report  card  as  one  of  the  factors  which  increased 
computer  usage.  They  identified  lack  of  time  as  a reason  for  prevented  them  from  increasing  their  use  of 
computers. 

Non-TIP  teachers  reported  that,  in  order  to  increase  the  use  of  computers  with  their  students,  they 
needed  (a)  people  support,  (b)  help  in  applying  technology  in  classroom,  (c)  access  to  hardware  and  technical 
support,  (e)  training  strategies,  and  (f)  easier  scheduling  of  facilities. 

Summation  of  Discussion  of  Findings  of  Data  From  Non-TIP  Teachers 

Of  the  21  surveys  returned  by  non-TIP  teachers,  20  of  them  (14%)  were  usable  for  data  analysis.  Since 
the  sample  of  Non-TIP  teachers  was  low,  it  cannot  be  assumed  that  the  sample  of  the  twenty  teachers  is 
representative  of  the  Non-TIP  population  of  teachers.  Therefore,  the  comparison  of  the  two  group  of  teachers, 
TIP  and  Non-TIP  teachers,  was  not  made.  The  validity  of  the  findings  of  the  Non-TIP  group  is  also 
questionable,  due  to  the  small  sample. 

There  are  several  factors  to  suggest  the  gains  made  by  the  non-TIP  teachers.  A close  examination  of 
the  returned  surveys  from  the  Non-TIP  respondents  revealed  that  three  of  the  respondents  received  training, 
support  and  had  access  to  hardware  similar  to  what  the  TIP  participants  had  received.  These  three  Non-TIP 
respondents  constituted  to  15%  of  the  sample  for  the  Non-TIP  group.  The  fact  that  the  three  teachers  were  given 
all  the  support  they  needed  could  be  a confounding  factor  because  of  the  ‘rippling  effect’  of  the  TIP  program. 
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Since  principals  knew  about  the  existence  of  the  TIP  program,  they  might  be  trying  hard  to  keep  up  with  what 
was  happening  in  the  district  by  giving  teachers  who  volunteered  for  TIP  but  were  rejected  by  TIP  as  much 
support  as  they  could  afford  to  give.  This  could  very  well  have  been  a confounding  factor  to  the  findings  of  the 
Non-TIP  group. 

There  are  several  factors  to  suggest  the  gains  made  by  the  non-TIP  teachers.  Since  the  non-TIP  group 
originally  volunteered  for  participation  in  TIP,  their  motivation  to  adopt  computers  as  an  innovation  was  likely 
as  high  as  the  TIP  participants.  The  concomitant  popularity  of  the  Internet  might  have  been  a catalyst  in  causing 
the  non-TIP  teachers  to  try  to  learn  more  about  computers  on  their  own.  The  fact  that  over  one  third  of  the 
teachers  identified  the  usage  of  computers  for  report  cards  showed  that  there  was  a need  for  the  non-TIP 
teachers  to  adapt  to  the  changing  process  of  using  computer  technology  in  their  classrooms.  Another  possible 
factor  is  the  fact  that  money  and  training  were  committed,  which  was  likely  interpreted  as  a signal  of  the 
importance  of  technology  to  the  senior  administrators  (Szabo,  1996).  This  is  in  stark  contrast  to  a common 
reaction  of  "There's  not  enough  money  in  the  budget  for  computers." 

All  the  above  mentioned  factors  combine  to  suggest  why  the  non-TIP  group  seemed  to  have  gained 
confidence  in  skill  and  knowledge  in  their  ability  to  use  the  computer  and  have  shown  a tendency  to  adopt 
computer  technology  as  an  innovation  for  teaching  and  learning. 


Conclusion  and  Recommendations 

This  study  was  concerned  with  the  research  question,  “What  impact  did  the  TIP  have  on  teachers  with 
regard  to  computer  usage?”  The  TIP  impacted  teachers’  usage  of  computers  in  a significant  way  because  it 
addressed  the  needs  of  the  teachers  in  the  implementation  of  computer  technology  by  supporting  them  in  their 
adaptation  to  the  process  of  change.  The  TIP  was  the  beginning  of  a process  in  which  teachers  began  the 
adoption  of  computer  technology  in  their  classrooms.  An  analysis  of  the  TIP,  from  findings  of  quantitative  and 
qualitative  data  of  this  study,  showed  that  the  main  reason  for  the  success  of  the  TIP  was  that  the 
implementation  of  computer  technology  was  looked  at  as  an  embedded  innovation  problem  in  which  teachers 
had  to  be  adapting  to  the  process  of  change.  Teachers  were  involved  with  the  change  process.  They  knew  what 
the  innovation  was.  Their  training,  organization  and  resource  needs  were  beginning  to  be  met  within  the  TIP. 
They  were  given  the  time  to  try  the  innovation  for  a one  year  period  so  they  could  make  informed  decisions  as 
to  whether  to  adopt  or  reject  the  innovation.  More  teacher  input  into  decisions  about  the  content  of  the  inservice 
program  for  the  inservice  sessions  would  be  helpful  for  future  programs  modeled  on  the  TIP. 

As  TIP  participants  demonstrated  the  characteristics  of  adopters  of  the  innovations  of  computer 
technology  for  teaching  and  learning,  the  challenge  of  institutionalization  lies  ahead  for  the  school  district 
which  initiated  the  TIP.  As  recommended  by  Fullan  (1992),  the  ultimate  adoption  of  an  innovation  for 
implementation  was  the  institutionalization  of  the  innovation.  Institutionalization  of  an  innovation  is  a process 
in  which  the  innovation  is  fully  implemented  by  members  of  the  group  with  support  from  the  institution  or 
community  in  which  the  members  are  involved.  The  major  contribution  of  the  TIP  was  the  impetus  which 
initiated  the  implementation  of  computer  technology  as  an  educational  innovation  in  a large  school  district. 

Recommendations 

Szabo  (1996)  proposed  a model  which  begins  with  a vision  from  the  senior  administrative  level, 
followed  by  the  development,  training  and  support  of  strategically  placed  leadership  teams  who  are  empowered 
to  make  the  vision  a reality  in  their  own  schools.  Szabo’s  model  could  be  considered  for  the  implementation  of 
computer  technology  in  the  classroom. 

From  a case  study  of  implementation  of  new  educational  technologies  in  Ontario  schools,  Fullan 
(1992)  suggested  that  teachers’  requirement  for  organizational,  resource  and  training  support  must  be  met  in 
order  for  them  to  successfully  implement  technology  as  an  educational  innovation.  In  conclusion,  the  first 
Technology  Incentive  Program,  though  significant  in  its  impact  as  to  the  influence  it  had  on  teachers  with 
regard  to  their  computer  usage,  is  but  a first  step  in  the  right  direction.  The  school  district  must  accept  the 
challenge  of  supporting  teachers  with  the  further  implementation  of  computer  technology  by  helping  to 
institutionalize  the  new  technology  for  the  school  system. 

Significance  of  the  Study 
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The  significance  of  this  study  lies  in  (a)  its  contribution  to  classroom  practice  by  gathering  data  on  how 
teachers  attempted  to  implement  computer  technology  in  their  classrooms  within  the  TIP,  and  (b)  its 
contribution  to  theory  by  relating  a district  initiative  technology  program  with  the  theories  of  change  and 
innovation  in  education. 
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Abstract:  This  study  presents  the  rationale,  model  and  results  of  the  first  year  of  operation  of 
an  operational  change  model  based  in  innovation.  The  change  model  focuses  upon 
professional  development  of  faculty  in  the  area  of  alternative  delivery  systems.  Tlie  two 
main  components  of  TIES  are  the  development  of  and  commitment  to  a vision  for  change 
using  alternative  delivery  systems  and  a leadership  team  training  approach  whereby 
faculties  may  interpret  and  develop  the  vision  according  to  their  own  unique  needs  and 
goals.  Results  of  the  first  year  of  operation  in  terms  of  vision  development  and  leadership 
team  training  and  support  are  presented. 


Overview 

The  philosophy  of  TIES  is  that  the  process  of  change  is  equally  or  more  important  than  training  or 
resources  to  faculty  use  of  alternative  delivery  systems.  While  numerous  theories  of  change  have  been  put 
forth  in  the  literature,  there  are  few  working  examples  of  comprehensive  application  of  the  principles  which 
derive  from  the  theories. 

The  purpose  of  TIES  was  to  design  and  test  a system  for  change  in  a university  environment.  The 
system  is  based  upon  1)  innovation  change  theory,  2)  a workable,  not  just  theoretical  approach,  3)  content 
taught  is  alternative  delivery  systems,  4)  change  methodology,  5)  both  top-down  and  bottom-up  operation, 
6)  professional  development  and  7)  empowerment  through  leadership  training. 

There  are  two  major  intended  goals  of  TIES.  The  first  is  that  the  chief  academic  officers  identify  a 
vision  for  alternative  delivery  systems  of  instruction  for  the  university,  publish  that  vision  widely,  and 
demonstrate  their  commitment  to  it  in  a clear  and  convincing  fashion.  Secondly,  departments  within  the 
university  create  leadership  task  forces  to  interpret  the  vision  for  their  unit  and  prepare  colleagues  to 
implement  the  shared  vision. 

TIES  was  devised  in  1996  by  the  Project  Director  during  a sabbatical  leave  and  through  LEE  funding, 
implementation  began  in  1997.  The  first  description  of  TIES  was  presented  at  an  International  Conference 
in  1996  and  can  be  viewed  at  http://www.atl.ualberta.ca/papers/change/tie.html/  and  Appendix  A.  Note 
that  all  appendices  can  be  viewed  at  http://www.quasar.ualberta.ca/edmedia/TIES/TIES  Repoit.html 

Deliverables 

The  first  deliverable  is  an  outline  of  a Visioning  workshop  for  Chief  Academic  Officers  (CAO)s  of  an 
institution.  This  describes  what  will  transpire  in  a workshop  to  create  a vision  for  the  institution  (Appendix 
B).  The  second  deliverable  is  a workshop  outline  for  the  training  of  TIES  Leadership  Task  Forces  (TLTF)s 
(Appendix  C).  The  third  deliverable  is  a set  of  TIES  training  modules  in  Web-Based  Instruction  (WBI) 
format.  The  list  of  modules  is  located  in  Appendix  D.  A sample  module  (L7:  Dealing  with  Mental 
Models)  has  been  provided  in  Appendix  E.  All  modules  are  available  at 

http://www.quasar.ualberta.ca/edmedia/TrES/Ties  home.html/ 
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Project  Planning  and  Implementation 


Problems/Issues 

The  driving  force  behind  TIES  stems  from  the  conviction  that  bringing  people  to  actually  change  their 
practices  with  respect  to  new  teaching  approaches  is  extremely  difficult.  Critics  of  education,  of  which 
there  is  no  short  supply,  often  contend  that: 

• students  are  poorly  prepared  (for  the  work  force), 

• education  has  become  bureaucratized,  politicized  and  centralized, 

• the  current  classroom-instructor  based  model  and  old  learning  theories  are  more  consistent  with 
preparing  students  for  the  industrial  model  of  work,  rather  than  for  work  in  the  information  age, 

• education  is  slow  to  change, 

• school  districts  undertake  innovation  under  circumstances  of  unclear  goals,  unpredictable  technology, 
and  uncertainty  in  general  (Morris,  1997). 

There  are  major  advantages  and  disadvantages  to  Instructional  Technology  (Appendix  F).  The 
University  has  a rich  history  in  computer-based  instruction  and  the  environment  for  Alternative  Delivery 
Systems  (ADS)  there  is  described  in  Appendix  G.  Furthermore,  ADS  is  an  innovation.  It  cannot  be 
understood  or  made  to  happen  without  a strong  understanding  of  what  an  innovation  is  or  is  not.  See 
Appendix  H for  pertinent  findings  from  educational  innovation. 

Assumptions  of  TIES 

Almost  every  system  rests  upon  a set  of  assumptions  which,  if  not  met,  may  render  the  system  inoperable 
before  it  begins.  The  assumptions  of  TIES  are  as  follows. 

• ADS  is  stimulated  MORE  by  a strong  vision,  the  willingness  to  share  that  vision,  and  an  unequivocal 
commitment  to  that  vision,  THAN  by  policies  and  procedures  (which  should  follow).  Normal 
pressures  for  institutional  reorganization,  such  as  falling  student  numbers,  competetion,  and  customer 
dissati faction  are  not  yet  apparent  in  Alberta  where  expansion  needs  are  being  met  using  old  models. 

• Emphasis  is  placed  FIRST  on  realistic  goals  of  improving  formal  learning,  such  as  increased 
achievement,  decreased  learning  time,  increased  accessibility  to  instruction  or  cost  control,  and 
SECOND  on  the  use  of  the  technology  of  ADS. 

• To  succeed  in  reforming  an  institution,  implementation  of  ADS  must  be  widespread  and  pervasive 
throughout  the  institution  and  not  confined  to  the  small  percentage  of  'early  adopters.' 

• ADS  requires  ownership  (commitment,  not  just  compliance)  by  those  who  will  ultimately  implement 
it.  This  is  the  “Empowerment”  phenomenon. 

• ADS  cannot  be  implemented  in  a shotgun  approach.  It  must  be  strategically  leveraged.  This  does  not 
mean  there  is  one  correct  model  or  plan-thus  the  emphasis  on  departmental  teams  making  local 
decisions  within  the  University. 

• Implementation  of  ADS  requires  sacrifices  at  all  levels  of  the  institution.  The  sacrifices  should  not  fall 
unevenly  on  specific  groups  or  individuals.  This  was  problematic  in  that  the  University  of  Alberta  is 
currently  recovering  from  a 21+%  reduction  in  government  funding. 

• No  one  can  predict  the  future,  especially  in  the  highly  changing  area  of  instructional  technology.  The 
best  way  to  predict  the  future  is  to  invent  it  though  experimentation,  keeping  the  good  ideas  and 
shelving  those  that  don't  work. 

• The  three  sequential  stages  of  innovation  are  play,  use,  and  creativity.  Support  for  and  evaluation  of 
ADS  needs  to  be  different  for  each  of  these  three  stages. 

Phases 

Phase  I:  Vision  Building 

The  ultimate  goal  of  Phase  I is  the  creation,  dissemination  support  building  for  a vision  of  what  the 
instructional  program  of  the  institution  will  look  like  in  the  future;  a vision  generated  by  the  Chief 
Academic  Officers  (CAOs). 

A retreat  involving  the  CAOs  and  key  support  staff  who  would  be  provided  with  stimuli,  resources  and 
a mechanism  to  capture  and  edit  their  deliberations  was  chosen  (Appendix  B).  At  the  retreat,  CAOs  would 
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be  presented  with  examples  of  the  use  and  impact  of  IT  from  within  the  institution  as  well  as  a sampling  of 
the  best  practices  from  around  the  world. 

Next  comes  a brainstorming  session  to  create  the  University  of  Alberta  Vision  for  Alternative  Delivery 
Systems.  The  basic  question  to  stimulate  these  outcomes  is:  “Suppose  it  is  now  10  years  into  the  future 
and  your  vision  has  succeeded  beyond  your  wildest  expectations.  If  you  were  to  walk  around  the  campus, 
city,  and  province,  what  evidence  would  you  see  of  the  success  of  this  vision?” 

Finally,  discussion  focuses  on  “What  are  the  major  barriers  that  stand  between  us  and  the 
accomplishing  this  vision?”  This  session  encourages  the  CAOs  to  identity  ways  these  barriers  could  be 
overcome.  After  a suitable  time,  it  is  expected  that  the  CAOs  publicly  present  the  results  of  their 
deliberations  with  emphasis  upon  the  vision  and  presenting  a convincing  case  for  their  commitment  to 
having  it  become  a reality. 

Phase  II:  Identification  of  Departments 

Deans  of  faculties  and  chairs  of  departments  were  approached  to  describe  the  vision  of  the  institution  and 
the  role  of  the  TIES  Leadership  Task  Force  (TLTF)  to  meet  that  vision,  and  extend  an  invitation  to 
participate.  The  authors  identified  six  departments  that  appeared  to  be  prepared  to  participate  and  in  which 
there  were  active  colleagues  from  previous  work  in  ADS.  One  had  other  more  pressing  agenda  items  and 
declined  to  pursue  the  option  further.  Deans  and  Chairs  were  informed  that  the  funding  would  provide 
$7,500  for  each  team  and  requested  to  provide  additional  resources  in  support  of  the  work  of  each  team. 

Phase  III:  Development  of  TIES  Workshop  and  Modules 

The  modules  to  be  created  had  to  provide  basic  instruction  in  ADS  and  change  methodology,  model  ADS, 
and  incorporate  adult  learning  strategies  (andragogy).  The  driving  philosophy  behind  TIES  is  to  make 
certain  all  TLTF  members  have  a basic  competence  in  the  following  areas  of  ADS:  presentation  and 
interactive  digital  technology,  professional  development,  leadership,  curriculum  and  instruction,  evaluation, 
and  related  issues  (See  Appendix  D)  for  a list  of  the  38  modules  by  topic. 

In  keeping  with  adult  learning  principles,  it  was  recognized  that  1)  not  all  participants  would  need  to 
study  all  the  modules,  2)  there  was  not  enough  workshop  time  to  complete  all  modules  needed  by  each 
individual,  3)  some  selection  was  needed  by  individuals,  and  4)  learning  rates  would  require  an  allowance 
for  self-pacing,  all  modules  were  developed  in  an  individualized,  self-paced  web-delivered  instnictional 
format,  complete  with  diagnostic  and  prescriptive  self-assessment  tools.  The  workshop  also  had  the  goal  of 
building  a team  orientation  to  complete  the  workshop  tasks  and  the  activities  of  the  coming  year 

Phase  IV:  TLTF  Training 

The  workshop  was  conducted  during  a four  day  break  in  the  second  term  of  the  academic  year. 

Phase  V:  Follow  Up  Support 

TLTFs  were  informed  that  the  training  team  would  be  available  to  assist  in  whatever  way  possible  during 
the  coming  12  month  period.  The  support  provides  and  models  the  importance  of  on-going,  local  and  just- 
in-time  support.  A Web  conference  site  was  set  up  to  facilitate  communication  within  and  between  teams 
and  trainers. 

Evaluation  Results 

Phase  I:  Vision  Building 

Initially  a retreat  to  develop  a vision  could  not  be  scheduled.  There  was  some  difference  of  opinion  as  to 
whether  the  vision  should  be  developed  by  the  CAOs  themselves  (the  author’s  opinion)  or  should  be 
developed  by  key  people  in  the  university  and  presented  to  the  CAOs  for  their  approval. 

During  a retreat  with  CAOs,  limited  progress  was  made  toward  the  goals  of  examining  current 
noteworthy  applications,  creating  a vision  for  the  university,  and  identifying  the  key  barriers  and  their 
solutions  to  making  the  vision  happen.  One  key  outcome  was  to  begin  a dialogue  on  the  role  of  ADS  with 
the  deans  of  the  university.  Another  outcome  was  the  development  of  an  updated  version  of  the  institution’s 
technology  integration  plan  (Learning  First,  1998).  However,  this  document  and  subsequent  versions  at  the 
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University  and  Faculty  levels  do  not  possess  the  essential  vision  component.  It  appears  that  using  a vision 
model  does  not  mesh  well  with  current  adminstrative  planning  models. 

Phase  II;  Identiflcation  of  Departments 

After  extensive  discussion  and  probing,  five  teams  from  across  the  university  agreed  to  participate  in  the 
year  long  program,  starting  with  an  intensive  workshop. 

One  Dean  had  begun  work  on  a technology  integration  plan  and  invited  the  author  to  discuss  TIES  and 
asked  to  nominate  a TLTF  (TLTF  B).  Due  to  the  small  size  of  the  departments  in  this  faculty,  they 
requested  to  participate  as  a faculty  level  team  rather  than  as  a department  level  team. 

TLTF  C was  formed  from  an  instructional  technology  team  which  had  already  been  created  and  was 
grappling  with  how  to  proceed.  Their  plan  was  to  convert  a large  undergraduate  course  to  ADS-based 
instruction  and  use  that  to  encourage  departmental  colleagues  to  become  involved.  This  team  had  the  most 
questions  to  ask  about  TIES  and  the  author  met  with  them  repeatedly  and  answered  numerous  questions  in 
person  and  electronically. 

Discussion  with  another  faculty  resulted  in  the  formation  of  TLTF  D at  the  faculty  level  since  the 
departments  were  quite  small.  The  Dean  was  enthusiastic  and  involved  a faculty  member  who  was  heavily 
involved  in  developing  IT  course  materials. 

When  a presentation  on  TIES  was  made  to  the  Associate  Dean  of  another  faculty,  the  enthusiasm  was 
quite  high  and  a meeting  with  the  Dean  was  quickly  arranged,  followed  by  a presentation  to  the  Dean’s 
Council.  TLTF  E was  formed  from  this  faculty  and  once  again,  it  was  a faculty- wide  endeavor  as  the 
departments  involved  are  small  and  engage  in  many  joint  projects. 

Due  to  the  complexity  of  the  TIES  and  its  innovative  nature  (attempting  to  deal  with  change  and 
reform  rather  than  simply  training  or  resources),  an  extensive  schedule  of  meetings,  discussion,  writing, 
explanation,  answering  questions  and  negotiation  had  to  be  carried  out 

Phase  III:  Development  of  TIES  Workshop  and  Modules 

Content:  The  driving  philosophy  behind  the  TIES  training  program  is  to  make  sure  all  TLTF  members 
attain  a basic  competency  in  several  different  areas  of  ADS  and  change.  In  the  area  of  ADS,  modules  were 
designed  to  deal  with  digital  presentation  technology  (7),  digital  interactive  technology  (5)  and  distance 
delivery  (3). 

In  response  to  instructors’  requirements  to  know  how  to  apply  IT  to  courses  and  curricula,  a set  of 
modules  on  curriculum  and  instruction  (6)  and  evaluating  students  (4)  were  developed. 

On  the  assumption  that  teaching  bright,  articulate,  and  motivated  instructors  to  develop  ADS  is 
relatively  easy  but  motivating  people  to  change  behaviors  is  considerably  more  complex,  a series  of  eight 
modules  designed  to  promote  leadership  skills  aimed  at  resistors  was  developed.  The  content  of  the 
modules  draws  heavily  from  the  work  on  change  of  Senge,  Kotter,  and  Covey  among  others,  and  their 
writings  were  employed  in  the  learning  activities. 

Finally,  a series  of  5 modules  on  key  related  topics  (e.g.,  professional  development,  copyright,  and 
effectiveness  of  IT)  were  created.  These  modules  are  open  for  viewing  at  the  TIES  web  site  listed  at  the 
end  of  this  article. 

Methodology  to  Acquire  Content : Each  module  was  designed  to  be  completed  in  a self-paced  format  with 
several  optional  objectives  and  linked  activities  to  choose  from.  Each  module  has  its  own  self-assessment 
pre  and  posttest,  the  expected  objectives,  and  learning  activities  to  meet  those  objectives.  In  keeping  with 
the  team-building  theme,  several  modules  required  completion  by  either  the  departmental  team  or  a cross- 
team but  small  group.  In  order  to  make  the  instruction  available  on  an  extensive  basis,  i.e.,  beyond  the 
workshop,  the  modules  were  created  in  a web-based  instruction  format.  A diagnostic  pretest,  covering  all 
38  modules,  was  developed  using  the  preassessment  items  from  the  modules 

Methodology  to  Foster  Team-Building  : Professors  may  or  may  not  have  experience  in  substantial  team 
efforts,  in  spite  of  the  fact  they  may  have  worked  together  in  the  same  department  for  substantial  lengths  of 
time.  Activities  were  included  which  were  aimed  specifically  at  dealing  with  cooperative  efforts  with 
respect  to  providing  leadership.  These  were  couched  in  the  context  of  ADS. 


Phase  IV:  TLTF  Training 

The  TLTF  training  took  place  during  a 4 day  school  break  in  February  1998.  The  entire  TLTF  C 
participated,  including  the  department  chair  and  a faculty  member  who  was  officially  on  sabbatical  leave. 
TLTF  A,  on  the  other  hand,  was  a no-show.  No  one  from  the  team  participated  and  the  chair  did  not  offer 
the  courtesy  of  contacting  the  workshop  coordinators  that  they  had  decided  not  to  participate,  although  one 
participant  had  indicated  a conflict  due  to  a conference  scheduled  the  same  time 

TLTF  B consisted  of  a single  individual  who  had  recently  returned  from  his  faculty  from  several  years 
at  another  institution.  This  individual  was  well- versed  in  the  WWW  component  of  IT,  keenly  interested  in 
the  promoting  IT  in  general  and  had  strong  support  from  the  dean. 

TLTF  D did  not  translate  the  specified  team  structure  into  reality.  It  consisted  of  one  full  time  faculty 
member  who  was  actively  developing  a WBI  course  for  the  faculty,  three  part  time  sessional  instructors, 
and  no  faculty  administrators.  TLTF  E was  also  at  the  faculty  level  and  consisted  of  a seasoned  IT  person, 
also  just  returned  from  a sabbatical  leave,  an  Associate  Dean,  and  a sessional  instructor.  They  also 
participated  in  the  entire  workshop  and  began  to  develop  ideas  for  working  with  their  faculty  colleagues. 

During  the  six  months  following  the  training  of  TLTFs,  the  teams  returned  to  their  respective  academic 
homes  to  1)  increase  personal  use  of  alternative  delivery  technology,  2)  identify  and  meet  training  needs  of 
colleagues  in  the  home  academic  unit,  3)  provide  or  arrange  for  support  for  colleagues  in  the  unit  and  4) 
develop  a long  range  plan  for  the  home  academic  unit  which  would  begin  to  address  the  future  direction 
and  requirements  for  alternative  delivery  systems.  The  latter  activity  was  particularly  timely  as  the 
university  administration,  with  stimulation  from  ADEM,  requested  a "Technology  Integration  Plan"  from 
each  Faculty. 

Evaluation  Results 

TLTF  "A” 

It  is  not  clear  why  this  task  force  did  not  bother  to  attend  the  workshop  or  to  inform  workshop  leaders  of 
this  decision.  One  member  previously  indicated  inability  to  attend  due  to  a conflict  with  a previously 
scheduled  conference.  Perhaps  the  lack  of  interest  in  ADS  evidenced  by  the  lack  of  a clear  vision  statement 
for  ADS  by  either  the  university  or  the  dean  sent  a signal.  Perhaps  a clear  understanding  of  TIES  and  its 
goals  and  objectives  was  not  conveyed  to  those  involved. 

TLTF 

Team  "B"  consisted  of  a single  individual  from  a faculty  whose  Dean  had  previously  and  in  several  ways 
expressed  a strong  interest  in  ADS.  They  too  chose  to  participate  as  a faculty  rather  than  department  due  to 
small  size.  The  individual  participated  extensively  and  expressed  appreciation  for  the  empowerment 
philosophy  of  TIES,  having  recently  come  from  an  environment  where  technology  had  been  ruthlessly 
enforced  on  a faculty  with  major  negative  implications.  After  the  workshop,  the  Dean  reassigned  this 
individual  to  a full  time  ADS  coordinator  who  then  formed  an  ADS  team  that  embarked  upon  developing 
and  implementing  a long-range  technology  integration  plan  for  the  Faculty. 

TLTF  ”C" 

This  team  came  the  closest  to  meeting  the  TLTF  selection  criteria:  several  key  opinion  leaders,  the 
department  chair,  and  several  members  already  active  in  ADS.  Before  agreeing  to  participate,  they  asked 
many  important  questions  about  the  workshop  and  commitment  expected.  They  participated  actively 
during  the  workshop  and  by  its  end  had  developed  the  structure  of  a long  range  technology  integration  plan. 
At  the  time  of  this  writing,  the  Project  Director  has  not  had  the  opportunity  to  interview  the  team  for  more 
specific  details  of  their  progress. 


TLTF 

TLTF  D was  driven  by  a single  individual,  rather  than  a team.  Steve  (not  his  real  name)  identified  a need 
for  training  his  colleagues  in  the  basic  use  of  computers  and  contracted  two  individuals  to  create  training 
materials  in  these  areas  which  would  meet  the  unique  needs  of  this  Faculty. 
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Steve  personally  conducted  several  two  day  workshops  and  provided  personal  support  to  colleagues. 
The  major  activity  that  consumed  Steve’s  time,  in  addition  to  carrying  an  above  average  (for  the  University) 
teaching  load,  was  the  development  of  a web-based  course. 

The  experience  of  TLTF  D is  highly  representative  of  the  individual  entrepreneur  approach  to  change 
(see  Team  Section  of  this  report  above).  Note  that  no  administrators  or  other  faculty  opinion  leaders 
(beside  Steve)  participated  in  TIES  training  week.  It  might  be  concluded  that  the  experience  of  TLTF  D 
was  solid,  positive,  but  quite  limited  in  the  scope  and  depth  of  addressing  alternative  delivery  systems  in 
the  unit.  It  depended  upon  the  good  will  of  Steve.  However,  Steve  observed  that  his  work  in  alternative 
delivery,  particularly  in  the  development  of  a web  course  was  cited  as  a positive  factor  in  his  receipt  of 
tenure. 

TLTF 

TLTF  "E"  participated  as  a faculty,  rather  than  as  a department.  Historically,  this  faculty  had  recently  been 
reorganized  through  a merger  of  disparate  departments  that  did  not  possess  a great  deal  of  affinity  with  one 
another.  This  task  force  came  close  to  meeting  the  selection  expectations,  with  an  Associate  Dean  and 
several  key  opinion  leaders.  It  also  included  several  sessional  instructors  and  research  personnel.  Their 
participation  in  the  workshop  was  quite  strong  and  they  too  began  to  develop  their  technology  integration 
plan.  Subsequent  to  the  workshop,  this  team  developed  and  submitted  its  TIP  to  the  faculty  as  a whole, 
developed  several  successful  ADS  funding  proposals,  hired  a course  developer  and  hired  an  instructional 
designer.  They  have  also  - begun  discussions  with  other  units  on  campus  regarding  joint  professional 
development  masters  degrees  to  be  delivered  by  ADS. 

Lessons  Learned  or  Revisited 

• TIES  is  relatively  successful  in  meeting  its  goals,  given  the  limitation  the  several  of  its  critical 
assumptions  were  not  met. 

• People  are  interested  in  quick  and  easy  solutions  to  complex  problems  which  involve  change,  reform, 
or  different  ways  of  operating. 

• Communicating  the  requirements  for  change  is  extremely  difficult,  both  in  the  reason  change  is  needed 
and  how  change  can  be  effectively  implemented. 

• The  concept  of  making  the  necessary  commitments  to  build  internal  expertise  to  provide  leadership 
through  a task  force  does  not  fit  within  the  normal  operational  mandate  of  educational  institutions. 

Task  forces  are  rarely  explicit  about  professional  development,  don't  understand  the  need  for 
commitment  by  senior  leaders,  and  are  often  viewed  as  'the  solution’  rather  than  as  a means  to  the 
solution. 

• The  effects  of  the  absence  of  a strong  vision  from  chief  academic  officers  and  other  administrators 
ADS  appears  to  have  been  substantiated. 

• The  importance  of  the  entrepreneur  or  keeper  of  the  dream  to  the  functioning  of  the  TLTF  has  been 
substantiated. 
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Abstract:  On  June  26-27,  1998,  doctors  at  UCSF  Stanford  Health  Care  consulted  via  the  Internet  with  pediatricians  at  Xi'an  Medical 
University  Hospital  in  Xi'an,  China,  on  the  cases  of  two  critically  ill  children.  Stanford  Medical  Information  Technologies  (rncdlT),  in 
cooperation  with  Sun  Microsystems  and  others,  developed  an  architecture  that  allowed  the  doctors  to  use  Internet  video,  audio,  whiteboard, 
and  browser  applications  to  confer  and  review  the  medical  cases;  and  to  webcast  the  entire  session  across  the  Internet  via  RealMedia. 
Secretary  of  State  Madeleine  Albright  and  other  ranking  members  of  the  Clinton  Delegation  to  China  attended  the  session. 


Introduction 

Background;  Intcmct-bascd  telemedicine,  though  in  its  early  stages  of  development,  is  of  great  importance  in  less-developed  countries,  where  advanced  medical  facilities 
and  expertise  arc  concentrated  in  major  cities,  and  inaccessible  to  large  segments  of  the  population  who  reside  in  outlying  areas.  Telemedicine  systems  can  allow 
practitioners  in  regional  and  rural  hospitals  and  clinics  to  draw  on  the  resources  of  major  medical  centers.  Many  hospitals  in  China,  such  as  Peking  Union  Medical  College 
Hospital,  have  acquired  video-based  teleconferencing  systems  and  are  delivering  telcmcdical  services.  These  services  arc  far  too  expensive  for  the  use  of  the  average  Chinese 
citizen.  Intcmct-bascd  consultation  systems  have  been  used  to  distribute  telemedicine  at  extremely  low  cost.  In  the  spring  of  1995,  the  case  of  a patient  with  heavy  metal 
poisoning  was  sent  from  Beijing  to  the  U.S.  Respondents  around  the  world  sent  suggestions  for  treatment  over  the  Internet  by  e-mail.  However,  text-based  e-mail  docs  not 
provide  the  capability  for  radiological  imaging,  transmission  of  sound  files  (such  as  the  output  of  a stethoscope),  and  in  particular  the  real-time  direct  interaction  of  medical 
colleagues,  which  is  seen  to  be  crucial  in  facilitating  consultation.  The  Internet  now  holds  the  promise  of  providing  this  type  of  interaction  at  much  lower  cost  than 
proprietary  videoconferencing  systems. 

Goals  of  the  Demonstration;  The  high-end  telcmcdical  technology  systems  that  arc  being  used  in  the  U.S.,  U.K.,  and  Israel,  cannot  be  deployed  widely  in  China,  or 
operated  on  a routine,  day-to-day  basis.  What  arc  needed  are  low-end  systems  that  match  the  capacities  of  existing  network  infrastructure  and  computers,  such  as  the 
Intcmct-bascd  applications  that  were  demonstrated  at  Xi'an  Medical  University  and  UCSF  Stanford  during  this  demonstration.  The  challenge  is  to  deliver  benefits  of 
modem  medicine  to  locations  that  arc  last  in  line  to  receive  them,  using  applications  that  will  work  reliably  and  on  a day-to-day  basis  in  the  most  difficult  environments. 
Making  use  of  the  publicity  surrounding  the  Clinton  visit  to  China,  it  was  hoped  that  a convincing  demonstration  of  real-time  Intcmct-bascd  telcmcdical  consultation  (the 
first  of  its  kind)  would  pave  the  way  and  lead  to  future  advances. 

The  Event:  On  June  26-27,  1998,  doctors  at  Xi'an  Medical  University  (western  China)  and  at  UCSF  Stanford  Health  Care  (California,  USA)  participated  in  the  first 
real-time  medical  consultation  between  China  and  the  United  States  over  the  Internet,  on  behalf  of  two  critically  ill  children.  The  demonstration,  which  coincided  with 
President  Clinton's  visit  to  China,  was  designed  to  highlight  U.S.-China  cooperation  in  medicine  and  technology  and  underscore  the  role  that  telemedicine  can  play  in 
improving  patient  care.  The  consultation  used  Internet  technology,  which  allows  for  real-time  communication  and  sharing  of  information,  including  images  of  the 
participants,  photographs,  medical  records  and  other  graphics,  and  voice  communications.  The  nonprofit  organization  Bridge  to  Asia  conceived  and  organized  the 
demonstration,  which  was  hosted  by  UCSF  Stanford  Health  Care.  The  technology  was  installed  and  managed  by  Stanford  medlT  (in  the  U.S.),  and  engineers  from  XMU 
(in  China),  with  support  from  Sun  Microsystems. 

Participants;  In  China,  the  event  was  presided  over  by  Secretary  of  State  Madeleine  Albright  and  Secretary  of  Commerce  William  Daley,  both  of  whom  addressed  the 
audiences  in  California  and  Xi'an  with  speeches  praising  the  technological  cooperation  and  emphasizing  the  importance  of  medical,  educational,  and  cultural  exchanges 
between  the  U.S.  and  China.  "This  is  indeed  a very  impressive  set-up  and  an  impressive  achievement,  and  as  we  inaugurate  this  link  between  Xi'an  and  The  Stanford  Medical 
Center,  we  use  technology  the  way  it  should  be  used-to  save  and  improve  people’s  lives,"  said  Albright.  "It  is  a sign  of  how  quickly  the  newest  information  technology  is 
coming  to  China  and  how  far  reaching  its  effects  will  be." 

The  other  U.S.  dignitaries  were  Senators  John  D.  Rockefeller,  Daniel  Akaka,  and  Max  Baucus;  and  Representatives  Edward  J.  Markcy,  John  Dingcll,  and  Lee  Hamilton. 

The  UeSF/Stanford  physicians  providing  the  consultation  were  Dr.  Daniel  Bernstein,  Chief  of  Pediatric  Cardiology  and  Associate  Professor  of  Pediatrics,  Stanford;  Dr. 
David  Stevenson,  Nconatologist,  Stanford;  Dr.  Joseph  Kittcrman,  Professor  of  Pediatrics  and  Nconatologist,  UCSF;  and  Dr.  David  Tcitcl,  Professor  of  Pediatrics  and  Chief 
of  Pediatric  Cardiology,  UCSF.  Said  Dr.  Tcitcl,  on  the  subject  of  telemedicine,  "I  think  it's  clear  that  patients  gain  a tremendous  amount  when  physicians  arc  able  to 
communicate  and  share  critical  diagnostic  information  and  images  in  real  time." 

The  Patients:  The  doctors  discussed  treatment  for  what  had  been  diagnosed  as  viral  myuocarditis,  an  inflammation  of  the  muscular  walls  of  the  heart,  in  a 12-ycar-oId  girl 
who  had  been  suffering  from  shortness  of  breath,  fatigue,  and  abnormal  heart  rhythms.  They  also  reviewed  treatment  for  a male  infant  who  was  bom  by  Cesarean  section  on 
June  13  and  was  suffering  from  brain  damage  due  to  a lack  of  oxygen  and  decreased  blood  flow  at  birth. 

Technology  and  Architecture 

Overview:  There  were  two  technological  challenges  to  the  telemedicine  demonstration.  One  was  facilitating  the  actual  conference,  with  video,  audio,  shared  whiteboard,  and 
image  viewers  (for  radiological  imagcs)-idcally  using  lightweight  and  widely  available  applications.  The  second  was  capturing  all  of  the  audio  and  video,  digitizing  it  into 
RealMedia,  and  streaming  it  out  live,  so  people  world-wide  could  witness  the  event  online.  The  final  architecture  was  a bit  complex,  but  very  robust  and  functional.  Two  Sun 
workstations  were  installed  on  each  end,  running  a suite  of  freeware  Internet  conferencing  applications,  browsers,  image  viewers,  and  supporting  software.  The  full  video 
output  from  one  of  the  conference  workstations  was  captured  with  a high-end  scan  converter  and  streamed  into  an  onsite  PC  running  Windows  95  and  RcalPublishcr;  this 
machine  converted  the  video  (and  audio,  captured  from  a conference  phone)  into  RealMedia,  which  it  streamed  across  the  local  network  to  another  PC  (in  the  next  building) 
running  RealServer.  From  there,  the  audio/vidco  stream  was  sent  out  over  the  Internet.  The  video  feed  was  also  sent  to  a large  display  device  in  the  U.S.  conference  room,  for 
the  edification  of  live  participants  as  well  as  captured  directly  by  the  various  camera  crews  in  attendance  for  broadcast  on  news  programs. 
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Hardware  & Software  (for  the  videoconference):  On  each  side,  two  Sun  Ultra  5s  (one  with  24-bit  graphics  capability),  both  running  SunOS  5.6,  were  utilized  for  the 
conferencing  applications.  Sun  desktop  cameras  and  microphones,  as  well  as  standard  VHS  video  camcorders,  were  used  for  audio  and  video  capture.  The  workstations  on 
both  ends  ran  a suite  of  TCP/IP  conferencing  applications  from  Berkeley  Labs:  vie  (video),  vat  (audio),  and  wb  (shared  whiteboard  space).  [Documentation  available  at 
<littp;//www-nrg.cc.lbl.gov/>‘.]  All  machines  ran  the  Sun  HoUava  web  browser  and  the  RealPlayer  streaming  media  player  from  RealNetworks.  Unix  talk  was  utilized  for 
some  technical  interchange. 

Hardware  & Software  (for  the  webcast):  A HyperConverter  1024  Scan  Converter  was  used  to  capture  the  full  video  feed  from  the  main  conferencing  workstation. 
[Information  on  this  product  available  at  <http://w'ww.pcvidco. com/1 280ds.htm>.]  A high-end  table-top  conference  phone,  with  direct  RCA  audio  out,  was  ultimately  brought 
in  for  the  audio  portion.  (The  TCP/1  P-based  audioconference  link  was  Judged  too  spotty  (sec  below)  for  use  in  the  actual  event,  and  was  supplanted  by  telephony.)  A 
Windows  95  box  onsite  ran  the  RealPublishcr  5.0  software  to  capture,  and  convert  in  real  time,  the  audio  and  video  streams.  These  streams  were  run  across  the  LAN  to  a 
Windows  NT  server  running  RealServer  5.0,  with  a 100-stream  license,  and  out  to  the  Internet.  A link  to  this  RealMedia  stream  was  provided  on  the  official  web  site. 
[Information  on  RealPublishcr  and  RealServer  available  at  <httpy/ww\v.rcal.com/>.] 

Network:  The  machines  on  the  Stanford  end  were  connected  via  10  megabit  ethemet  to  the  SUNct  backbone,  and  through  BBNPlanct  to  the  Internet.  In  Xi’an,  the  machines 
were  connected  to  CMINET  (the  Chinese  Medical  Information  Network)  to  CERNNET  (the  principle  backbone  in  China)  to  the  Internet.  Two  satellite  hops  and  a 
microwave  link  were  involved  in  getting  Internet  traffic  from  the  U.S.  to  Xi’an  (in  rural  western  China). 

Hurdles 

Infrastructure:  One  of  the  catch  phrases  of  the  event  was  was  to  show  "the  Internet  in  China  as  it  is,  not  as  we  wish  it  to  be."  In  numerous  tests  over  a period  of  several 
weeks  leading  up  to  the  event,  attempts  were  made  to  secure,  and  make  best  use  of,  bandwidth  into  Xi'an  Medical  University.  It  was  particularly  critical  to  experience  some 
success  in  our  testing,  as  a number  of  the  "dry  run"  sessions  were  attended  (frequently  without  advance  warning)  by  members  of  the  U.S.  State  Department  advance  team; 
their  impressions  would  dictate  whether  this  consultation  was  included  on  the  schedules  of  the  visiting  U.S.  delegation.  During  one  State  Department  visit,  a camera  went 
out;  during  another,  one  of  the  workstations  in  China  crashed  and  wouldn't  come  back  up.  These  situations  were  handled  smoothly  on  a casc-by-casc  basis.  But,  overall, 
experiences  of  limited  bandwidth,  packet  loss,  and  low  video  frame  rates  were  acute  in  numerous  tests.  Compression  of  the  video  was  optimized,  but  It  was  ultimately 
decided  that  the  spottincss  of  the  audio  stream  would  be  too  frustrating  for  the  dignitaries,  and  (to  a lesser  extent)  the  physicians.  As  a result,  we  reluctantly  decided  to  use  a 
telephone  conference  call  for  the  audio  portion  of  the  live  event. 

The  total  bandwidth  into  XMU  was  extremely  limited,  so  some  steps  were  taken  in  advance  to  address  this.  The  microwave  link  from  Xi'an  to  Beijing  was  overheating  in  the 
sun  during  the  day,  so  the  dish  was  upgraded  a few  days  before  the  event.  Preparations  were  also  made  to  cut  off  all  other  Internet  traffic  from  XMU  during  the  period  of 
the  demonstration.  Additionally,  during  testing  the  night  before,  we  experienced  interference  in  the  video  stream  from  China  which  was  unrelated  to  the  Internet  transport. 

We  determined  that  this  was  caused  by  on-off  cylcing  of  air  conditioning  units  at  XMU;  consequently,  people  at  XMU  were  cut  off  from  their  climate  control  (as  well  as  the 
Internet)  for  a short  time. 

The  Intcmct-bascd  video,  shared  whiteboard,  and  shared  image  viewing  eventually  went  over  extremely  well.  Although  video-taped  welcomes  and  introductions  were  not 
shipped  to  China  in  time  for  the  event,  digitized  RealMedia  versions  of  these  materials  were  transferred  electronically,  an  unexpected  coup  for  the  Internet. 

Arcliitectiirc  for  the  webcast:  One  of  the  main  architectural  challenges  was  how  to  capture  various  video  sources  (two  conferencing  windows,  the  whiteboard,  images  in  a 
browser,  etc.),  as  well  as  audio  (complicated  by  the  use  of  telephony),  and  convert  it  all  into  one  audio/vidco  stream  for  RcalCasting.  Several  strategics  were  considered,  but 
ultimately  we  concluded  we  had  to  pull  the  full  video  feed  from  the  main  workstation  (on  its  way  to  the  monitor).  The  HyperConverter  1024  scan  converter,  with  its  13W3 
Sun  video  connector,  solved  that  problem  for  a relatively  modest  price  (S  1,395).  The  direct  RCA  audio  out  on  the  conference  phone  solved  our  problem  of  capturing  both 
ends  of  the  dialogue.  The  scan  converter  and  the  phone  were  both  run  directly  to  the  PC  running  RealPublishcr,  resulting  in  a very  good  quality  RealMedia  stream. 

U.S.  vs.  China:  Time  zone,  language,  and  cultural  differences  were  issues  to  be  dealt  with-but  none  proved  serious.  Most  of  the  live  tests  were  conducted  in  the  evening  on 
the  U.S  West  Coast,  to  coincide  with  morning  in  Xi'an.  The  great  majority  of  the  Chinese  technicians  and  physicians  had  a very  good  command  of  English-though  on  some 
occasions  it  was  helpful  to  have  a bilingual  participant  who  was  familiar  with  medical  terminology  (James  Bair,  the  Director  of  International  Medical  Services  for  UCSF 
Stanford  Health  Care).  Finally,  the  cultural  differences  were  inconsequential,  and  the  engineers  definitely  spoke  the  same  language. 

Results 

On  the  evening  of  the  event,  introductory  comments  were  made  by  Secretaries  Albright  and  Daley,  and  the  Internet  link  was  established.  The  U.S.  physicians  consulted,  in 
pairs,  on  the  two  different  cases  (neonatology  and  pediatric  cardiology)  for  about  two  hours.  They  viewed  CT  and  lung  x-ray  images  in  an  image  viewer  and  browser,  asked 
questions,  and  provided  opinions  to  their  colleagues  in  China.  They  also  exchanged  e-mail  addresses  (using  the  whiteboard  application)  to  facilitate  follow-up. 
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Several  multimedia  materials  (a  weleome  and  introduetion  of  the  physieians,  a video  tour  of  pediatrie  faeilities  at  UCSF/S  tan  ford)  were  prepared  but  not  used  in  the  aetual 
event  (they  are  availaWe  on  the  site).  The  live  webeast  of  the  aetual  event  was  not  ultimately  approved  by  the  People's  Republie  of  China,  or  the  U.S.  State  Department,  a 
mildly  disappointing  turn  of  events.  The  initial  eonsultation  and  preparatory  session  of  the  night  before  was  sueeessfully  webeast,  proving  the  viability  of  the  architeeture. 
But  the  main  event,  with  the  dignitaries,  was  blaeked  out  from  the  Net,  though  not  from  traditional  media. 

The  event  rceeived  eoverage  from  the  New  York  Times,  the  L.A.  Times,  the  San  Jose  Mercury  News,  the  China  Press,  and  Sing  Tao  Daily;  as  well  as  CNN  International  and 
San  Franeiseo  area  loeal  broadeast  news. 


The  Main  Sereen  of  the  Event.  Counterclockwise  from  top  left:  the  consulting  physicians  in  the  U.S.;  a chest  x-ray  of  the  infant;  a web  browser  with  a 
screen  from  the  official  site;  in  China,  Secretary  of  State  Madeleine  Albright  using  the  system  (assisted  by  Robert  Yung,  CTO  Sun  Microsystems  Asia). 


The  Web  Site 

A complete  informational  site  was  created  in  conjunction  with  the  event.  Information,  images,  multimedia,  and  future  plans  continue  to  be  available  on  the  web  at 
<http://tclcmcd.stanford.edu/tclcmed/>. 
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Becoming  a Student  in  an  Asynchronous,  Computer-Mediated  Classroom 
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Abstract:  Graduate  instruction  is  traditionally  delivered  in  face-to-face  classrooms  in  university 
settings.  However,  with  the  decentralizing  of  corporations  and  workforces  in  the  late  20th  century, 
potential  students  have  been  distributed  far  from  traditional  academic  centers.  Distance  education 
is  a viable  option  for  these  persons.  Asynchronous  learning  networks  (ALNs)  are  an  increasingly 
available  option  for  distance  education.  However,  ALN  environments  impose  unusual  constraints 
on  the  way  individuals  perceive  themselves,  their  fellow  students  and  their  teacher,  and  how  they 
interact  as  students  and  teachers.  The  transition  from  face-to-face  classroom  to  ALN  classroom  is, 
therefore,  tricky  for  students  and  teachers  alike.  This  paper  describes  characteristics  of  competent 
students  in  a face  to  face  classroom  and  then  contrasts  them  with  problems  typically  experienced  by 
students  in  an  ALN.  Solutions  for  these  problems  are  described  and  used  to  create  a model  of 
instruction  for  helping  students  become  competent  and  collaborative  learners  in  an  ALN. 


Introduction 

Graduate  instruction  is  traditionally  delivered  in  face-to-face  classrooms  in  university  settings.  However,  with  the 
decentralizing  of  corporations  and  workforces  in  the  late  20th  century,  potential  students  have  been  distributed  far 
from  traditional  academic  centers.  Faced  with  this  situation,  persons  wishing  to  pursue  continuing  professional 
education  have  been  forced  to  choose  between  employment  and  leaving  a job  to  attend  school  full-time. 

Distance  education  has  emerged  as  a viable  option  for  these  students.  Instead  of  moving  to  the  education,  a student 
can  have  educational  opportunities  sent  to  himself  or  herself  Students  can  enroll  in  courses,  participate  in  them,  and 
graduate  with  degrees  without  every  actually  "being  with"  their  instructors  or  classmates. 

Many  delivery  options  exist  for  distance  education.  For  example,  correspondence  courses,  compressed  video  or 
satellite  television  and  computer  based  asynchronous  learning  networks  (ALNs)  are  common  distance  education 
delivery  systems.  ALNs  have  the  advantage  of  permitting  students  to  participate  in  educational  experiences  in  a 
"time  shifted"  environment.  In  other  words,  because  students  and  teacher  are  not  required  to  meet  at  the  same  time, 
ALNs  permit  students  from  different  time  zones,  and  with  different  work  schedules  to  interact  in  the  same 
"classroom." 

The  skill  of  being  a competent  classroom  student  is  a status  and  skill  learned  implicitly  over  many  years  of 
experience  in  traditional  face  to  face  classrooms.  However,  ALNs  impose  unusual  constraints  on  the  way 
individuals  perceive  themselves,  their  fellow  students  and  their  teacher,  and  how  they  interact  as  students  and 
teachers.  The  transition  from  face-to-face  classroom  instruction  to  ALN  instruction  is,  therefore,  tricky  for  students 
and  teachers  (Harasim,  Hiltz,  Teles  & Turoff,  1995). 

A Theoretical  Model  of  the  Competent  Classroom  Student 

Hymes  (1974)  indicates  that  there  are  several  skill-sets  required  for  competent  interaction.  First  is  a mastery  of  the 
behavioral  elements  of  interaction.  Second  is  a mastery  of  the  linguistic  components  of  interaction,  and  third  is  skill 
at  mixing  the  first  two  - knowing  how  to  behave  in  interaction,  and  saying  appropriate  things  at  the  right  times. 
Hymes  (ibid.)  indicates  that  this  third  skill  is  what  makes  a person  a functional  and  contributing  member  of  a culture 
or  community.  However,  before  this  skill  may  be  practiced,  the  first  and  second  elements  must  be  mastered.  Each 
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of  the  three  interaction  skills  is  necessary  for  competent  classroom  interaction,  but  none  is  sufficient  for  competent 
classroom  interaction. 

Hymes’  (ibid.)  constructs  are  intended  to  describe  competent  members  of  interactive  social  communities.  However, 
his  three-part  model  is  readily  applicable  to  classroom  settings  - whether  they  are  instantiated  in  traditional  face-to- 
face  classrooms,  or  asynchronous  classrooms.  This  is  especially  true  in  highly  interactive,  or  cooperative 
classrooms.  Highly  interactive  classrooms  are  sometimes  very  desirable,  for  example  in  a design-based  engineering 
classroom  where  ideas  and  technologies  are  discovered,  tested  and  refined.  In  fact,  Salomon  (1993)  and  Schrage 
(1990)  explicate  that  interactive  or  cooperative  communities  provide  a richer  environment  in  which  to  share  and 
develop  ideas  and  engage  in  learning,  than  do  more  didactic  instructional  environments.  When  members  of  a 
classroom  possess  and  practice  all  three  of  Hymes  constructs,  cooperative  learning  communities  can  exist  and  thrive. 

The  Task  of  Becoming  an  Asynchronous  Student 

Problems.  Distance  education  in  general,  and  ALNs  in  particular  permit  access  to  educational  opportunities  to 
persons  who  might  not  otherwise  have  it.  However,  there  are  several  roadblocks  to  competent  classroom 
membership  in  the  ALN  classroom.  The  first  can  occur  immediately  upon  entry  to  the  distance  education 
environment  and  is  associated  with  using  the  technological  tools  of  the  distance  classroom. 

In  the  asynchronous  classroom,  students  normally  use  some  form  of  E-mail  to  send  comments  to  classmates  and  the 
teacher,  and  to  receive  messages  from  them.  If  a student  is  unfamiliar  with  the  particular  communications  tool  being 
used  in  class,  he  or  she  lacks  the  first  of  Hymes’  constructs.  There  is  a paradox  associated  with  this  problem.  In 
order  to  be  a contributing  member  of  the  classroom,  a student  must  have  mastered  the  tools  being  used  to  interact, 
and  the  ways  they  may  be  used.  However,  the  novice  ALN  student  must  know  how  to  use  the  communication  tools 
before  interaction  becomes  possible. 

The  second  roadblock  is  related  to  the  first.  It  is  not  enough  to  simply  ’’know  how”  to  use  the  required 
communication  technologies.  The  student  must  also  know  when  to  use  these  tools  in  a way  that  will  facilitate  the 
learning  process.  For  example,  a student  may  know  how  to  send  and  receive  classroom  messages  (fulfilling  part  of 
Hymes’  first  competency  construct),  but  if  he  or  she  does  not  know  and  practice  the  skill  of  filing  messages  in  a way 
that  facilitates  retrieval  (for  example,  by  a keyword,  or  in  a particular  folder  arrangement)  - and  then  use  their 
system  to  organize  efficient  study  of  the  material,  learning  may  not  be  accomplished. 

The  third  roadblock  to  competent  classroom  membership  effects  the  second  of  Hymes’  interactive  skills,  mastery  of 
the  linguistic  components  of  interaction.  While  most  students  attending  a distance  class  will  be  capable  speakers  of 
the  language  used  in  class  (for  example,  spoken  and  written  English),  speaking  and  writing  in  the  distance  classroom 
is  subtly,  but  importantly,  different  then  what  is  familiar  in  a face  to  face  classroom.  For  example,  in  a traditional 
face  to  face  classroom,  interactants  make  use  of  many  non-language  signals  to  create  messages  and  to  interpret  and 
understand  what  is  going  on.  For  example,  voice  inflections,  facial  expressions,  hand  motions,  linguistic  devices 
(for  example,  sarcasm),  etc.,  are  used  by  speakers  to  embellish  their  messages.  Once  these  paralinguistic  devices  are 
known  and  understood,  they  also  assist  listeners  in  understanding  the  message  as  intended.  However,  because  the 
primary  communication  channel  in  the  asynchronous  classroom  is  written  text,  none  of  these  para-linguistic  devices 
may  be  used  in  their  conventional  form.  However,  because  they  are  so  common  in  face  to  face  classroom 
interaction,  it  is  very  common  for  novice  asynchronous  students  to  use  sarcasm,  a cliche,  or  "insider  knowledge,"  to 
embellish  a written  comment.  However,  without  the  benefit  of  familiar  paralinguistic  signals,  these  comments  are 
often  misunderstood  - and  may  result  in  unintentional  insult  or  embarrassment.  Together,  these  three  roadblocks 
may  create  an  incorrigible  problem  for  students  in  an  asynchronous  classroom.  Therefore,  interactive  learning 
communities  may  never  develop. 

Solutions.  It  has  been  observed,  however,  that  when  the  instructor  and  students  foster  an  open  and  interdependent 
exchange  of  instructional  and  social  dialog  in  the  asynchronous  distance  education  classroom,  students  can 
simultaneously  overcome  both  of  these  roadblocks  (Winiecki,  1997).  This  process  is  similar  to  that  described  by 
Wittgenstein’s  "language  games"  (1958).  Sociolinguists  have  argued  that  language  games  facilitate  the 
development  of  pidgin  languages  and  eventually  permit  individuals  with  vastly  different  backgrounds  and  languages 
to  develop  a mutually  acceptable  language.  "Language  games"  are  characterized  by  episodes  of  showing  what  is 
being  spoken  of,  speaking  of  it,  and  negotiating  understanding  of  a task  that  involves  it,  at  the  same  time.  The  goal 
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is  to  communicate  not  only  the  concepts  being  discussed,  but  also  to  negotiate  a mutually  understandable  and 
acceptable  way  of  communicating  about  it,  at  the  same  time. 

For  example,  the  following  conversation  occurred  between  two,  first-semester  graduate  students  in  the  ALN  course, 
"Introduction  to  Instructional  & Performance  Technology,"  offered  the  Instructional  & Performance  Technology 
department  of  Boise  State  University’s  College  of  Engineering. 

[start  of  transcript  segment] 

Rebecca  (9/28/97,  9:20PM):  1 can’t  find  last  week’s  assignment!  I remember  reading  it,  but  now 


Instructor  (9/29/97,  8:15AM):  Hi  Becky.  You  should  be  able  to  find  it  in  your  database  of  messages. 

Go  into  your  database,  and  search  for  all  messages  sent  on  September  22nd.  It  will  be 
easy  to  pick  out  of  that  list. 

Rebecca  (9/30/97,  6:23PM):  What  database!  I don’t  have  any  database  - all  1 see  is  a bunch  of 
messages  from  everyone  to  everyone!!  Nobody  seems  to  be  talking  about  the  same 
thing!!!  This  is  very  confusing!!!! 

George  (9/30/97,  7:44PM):  Hey  Rebecca,  did  you  follow  the  setup  instructions  in  the  computer 
conferencing  guide?  It’s  really  easy  and  it  tells  you  all  about  creating  and  using  your 
messages  database. 

Rebecca  (9/30/97,  8:03PM):  It  might  be  easy  for  you!!!!!  1 tried  to  follow  those  instructions  but  I 
can’t  understand  them  - 1 must  not  be  computer  literate  enough  to  understand  it  all.  Will 
you  send  me  the  assignment?! 

Instructor  (10/01/97,  8:27AM):  Hi  Becky,  I’ll  send  you  the  assignment  again,  but  we’ve  got  to  get 
your  messages  database  set  up.  Try  this:  From  the  "message  packets"  screen,  click  the 
"Import"  button,  and  then  answer  "Yes"  when  the  software  asks  you  if  you  "want  to  import 
all  messages."  The  software  will  then  build  your  messages  database  (you’ll  see  a scrolling 
list  of  every  message  you’ve  ever  received,  as  it  is  inserted  into  the  database).  When  it’s 
done  you’ll  see  a new  icon  titled  "Messages  database."  You’ll  be  able  to  search  through 
this  to  find  any  message. 

Rebecca  (10/01/97,  7:10PM):  Thanks  for  your  help,  but  how  do  1 know  which  screen  is  the 
"message  packets"  screen?! 

George  (10/02/97,  8:44AM):  Oh,  sorry  about  that...  The  "message  packets"  screen  is  the  one  with 
the  gray  background,  and  has  a list  of  many  message  "packets"  in  a blue  rectangle. 

Rebecca  (10/02/97,  7:37PM):  Hey!  1 just  did  it  - and  now  I’ve  got  a messages  database.  Should  1 
keep  all  of  the  other  files  that  are  listed  on  this  "message  packets"  screen? 

George  (10/03/97,  8:48PM):  Super!  It’s  a real  feeling  of  accomplishment  when  you  prove  to  your 
computer  who’s  boss  - isn’t  it!?  1 remember  when  1 got  my  first  BASIC  program  to  run.  1 
felt  like  jumping  up  and  "high-fiving"  everybody! 

Rebecca  (10/03/97,  9:15PM):  Hey!  You  must  be  some  kind  of  computer  expert  - writing  programs 
and  all!  ... 

[end  of  transcript  segment] 

In  this  interchange,  Rebecca  was  unable  to  understand  the  language  contained  in  a booklet  that  described  how  to 
perform  some  task  in  the  communications  software.  Another  student  and  the  instructor  attempted  to  communicate 
with  her  but  found  that  she  was  not  able  to  interpret  their  language  either.  George  then  engaged  in  a "language 
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game"  with  her  (shown  as  boldface  in  the  transcript,  above)  complete  with  descriptive  references  to  the  software 
screens  and  how  she  should  proceed.  This  language  game  was  used  between  the  students  to  negotiate  what 
“database,”  and  “message  packets  screen”  meant  before  they  could  actually  solve  Rebecca’s  problem. 

Rebecca’s  inability  to  solve  the  problem  is  not  unusual.  There  are  as  many  "Rebecca"  students  as  there  are 
"George"  students.  Technical  documentation  is  not  enough  to  support  all  ALN  students.  Occasionally,  it  is 
necessary  to  engage  in  an  interactive  "language  game"  to  find  out  what  is  not  understood,  and  how  to  solve  the 
problem.  In  some  instances  telephone  calls  between  students  or  between  the  instructor  and  students  are  necessary, 
to  work  beyond  problems  - especially  if  the  difficulties  prevent  asynchronous  interactions. 

Ability  to  control  communications  software  is  only  a small  part  of  interacting  and  learning  in  an  ALN.  Hymes 
reminds  us  that  it  is  also  necessary  to  know  how  to  communicate  to  other  learners.  Students  in  an  ALN  classroom 
are  very  familiar  with  classroom  interactions  - they  have  all  been  students  in  conventional,  face-to-face  classrooms 
in  the  past.  However,  "talking"  in  an  asynchronous  classroom  as  if  it  were  a face-to-face  classroom  can  lead  to 
unexpected  results,  as  can  be  seen  in  the  following  transcript. 

[start  of  transcript  segment] 

Anna  (10/12/97,  9:08PM):  Hey,  Wanda  - old  girl!  How  did  your  performance  appraisal 
interview  go  at  work  yesterday? 

Wanda  (10/14/97,  7:23PM):  Anna,  I know  we’ve  been  working  on  a group  project  this  week,  but  I 
don’t  know  how  to  understand  your  comment  from  last  Sunday.  Have  I been  too  slow  to 
finish  my  part  of  the  project? 

Anna  (10/14/97,  8:10PM):  Wanda,  I had  to  go  back  to  my  message  from  Sunday  to  find  out 
what  I said.  I apologize,  I didn’t  mean  to  imply  that  you’re  moving  slowly  on  the  project 
(or  anything  for  that  matter).  I was  just  teasing  a little  bit  because  we  seem  to  be  doing  so 
well  as  a group. 

[end  of  transcript  segment] 

In  this  dialog,  Anna  appears  to  have  been  offended  by  Wanda  referring  to  her  as  "old  girl!"  After  these  comments, 
Anna  and  Wanda  appeared  to  remain  somewhat  uncomfortable  with  each  other  and  their  group  work  and  classroom 
interaction  suffered.  A telephone  interview  by  the  researcher,  with  both  Anna  and  Wanda  disclosed  that  this 
incident  was  troubling  to  both  of  them,  and  that  neither  was  able  to  "forget  about  it."  However,  following  this 
experience,  both  Wanda  and  Anna  reported  that  they  were  much  more  careful  about  how  they  worded  messages  to 
classmates. 

Nevertheless,  sometimes  it  will  facilitate  interaction  if  the  "speaker"  and  "listeners"  in  an  ALN  are  able  to  discern 
the  attitude  with  which  a certain  comment  is  being  voiced.  For  example,  sarcasm  and  hyperbole  are  important 
linguistic  mechanisms  with  which  to  highlight  misperceptions  in  a Socratic  dialog.  In  some  situations,  comedy 
relief  can  be  used  to  change  the  course  of  a discussion  or  to  defuse  tensions  in  a prolonged  discourse.  However,  as 
displayed  above,  all  of  these  linguistic  techniques  are  easily  misunderstood  if  the  "listeners"  are  not  savvy  to  the 
"speaker’s"  intention.  On  the  other  hand,  these  techniques  lose  their  impact  if  the  speaker  is  too  obvious  in 
"telegraphing"  his  or  her  plan. 

Interactants  in  online  "chat"  groups,  MOOs,  and  MUDS  (multiplayer,  computer-based  role  playing  games)  have 
developed  a technique  that  increases  the  possibility  that  these  linguistic  techniques  will  be  recognized  appropriately, 
but  that  does  not  reduce  their  potential  impact.  This  technique  is  called  "emoting"  and  is  accomplished  by 
embedding  posture  or  attitudinal  information  in  a message  (Turkle,  1995).  Similar  techniques  have  been  called 
"embodied  actions"  by  other  researchers  (Jarmon,  1995). 

For  example,  to  display  embarrassment,  and  a childhood  innocence,  a message  sender  might  write: 
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<Don  looks  at  his  feet,  and  toes  the  sand>  Gosh... 

To  display  an  almost  fanatical  devotion  to  an  idea  being  discussed,  the  researcher  once  typed  the  following  message: 

<Don  stands  on  his  desk,  faces  East,  thrusts  his  right  fist  into  the  air  and  shouts>  Long  live  B.  F.  Skinner! 

To  exhibit  approval  for  a student’s  effort,  but  also  to  encourage  him  to  press  on  despite  persistent  difficulties,  the 
researcher  once  wrote: 

<Don  emerges  from  a mist,  and  whispers  in  a solemn  voice>  use  the  force,  Aaron. 

Interestingly,  the  technique  of  "emoting"  was  spontaneously  adopted  by  several  of  the  researcher’s  students  in  their 
own  messaging.  Through  telephone  interviews  with  these  students,  I learned  that  my  "emoting"  comments  elicited 
laughter,  reflection  and  surprise  - but  in  all  cases,  the  students  described  that  the  use  of  "emotes"  led  to  a much 
faster,  and  deeper  understanding  of  the  comment  than  they  felt  could  have  been  realized  otherwise. 

Serendipitous  Side  Effects 

The  social  processes  and  the  cooperative  development  of  technical  skills  and  communication  techniques  described 
above  has  also  had  the  serendipitous  side  effect  of  engendering  a cooperative  and  interdependent  learning  culture 
among  students  (Winiecki,  1997).  Soon  after  the  emergence  of  "language  games"  and  "emoting,"  students  increased 
their  mutual  problem  solving  and  help  seeking  activities.  They  became  much  less  reliant  on  the  teacher  as  a source 
of  answers,  and  more  reliant  on  each  other  and  their  individual  areas  of  expertise,  as  the  following  set  of  messages 
displays. 

[start  of  transcript  segment] 

Rebecca  (11/12/97,  12:14AM):  George,  I haven’t  been  able  to  figure  out  what  Gagne  is  talking 
about  in  chapter  4.  Have  you  been  able  to  make  sense  of  it? 

George  (1 1/13/97,  7:18PM):  Hi  Becky,  I’ve  been  away  on  a business  trip  and  haven’t  had  time  to 
reread  the  chapter.  But  if  I remember  correctly,  he  was  trying  to  classify  the  different 
kinds  of  things  that  people  usually  have  to  learn.  I understood  verbal,  psychomotor  and 
concrete  concepts,  but  I don’t  think  I really  know  what  he  means  by  defined  concepts  or 
rules.  Is  this  where  you’re  getting  stuck  also? 

Rebecca  (1 1/13/97,  7:45PM):  Yes,  George,  I’m  "not  getting"  what  he  means  by  "defined  concept." 
Every  time  I think  that  I understand,  I go  back  and  read  pages  59-61  and  I start  doubting 
myself  again. 

Aaron  (11/13/97,  8:27PM):  Rebecca  and  George,  I think  I’ve  got  an  example  of  the  difference 
between  a concrete  concept  and  a defined  concept... 

George  (1 1/13/97,  8:36PM):  Hi  Aaron.  Your  example  got  me  thinking.  Do  you  suppose  that ... 

Rebecca  (11/13/97,  9:17PM):  Hmmm  <rebecca  scratches  her  head,  and  looks  confused>.  I’m  still 
not  sure  - but  I think  that  I’m  getting  closer  to  knowing  what  this  stuff  means.  I don’t 
know  what  I’d  do  without  you  guys.  Here  it  is  almost  9:30  on  a Thursday  night  and  we’re 
still  working  on  this  stuff  Do  you  think  we’ll  figure  it  out  before  the  test  next  week? 

[end  of  transcript  segment] 

Aaron,  George  and  Rebecca  appeared  to  be  the  most  active  cohort  in  the  class.  Their  interactions  spanned  both 
curricular  and  personal  matters.  On  several  occasions,  Rebecca  referred  to  future  interactions  with  her  classmates. 
For  example,  as  the  semester  neared  its  end,  she  sent  a message  to  the  class:  "when  we  graduate,  we’re  going  to  have 
to  have  a big  party  together!" 
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Since  the  semester  when  these  transcripts  were  captured,  Rebecca,  Aaron  and  George  have  not  taken  another  ALN 
course  together.  However,  they  still  correspond  in  a public  area  of  the  ALN,  called  (aptly  enough),  the  "Student 
Union.”  Their  messages  still  refer  to  their  experiences  as  classmates  and  they  still  task  about  taking  other  courses 
together.  The  experiences  they  shared  were  instrumental  in  facilitating  an  ongoing,  and  an  interdependent  learning 
relationship. 


This  paper  describes  tacit  practices  that  are  necessary  for  the  successful  accomplishment  of  "competent  studentship" 
in  a face-to-face  classroom.  It  then  described  how  these  same  practices  are  sometimes  difficult  to  realize  in 
asynchronous  learning  networks  (ALNs).  Finally,  methods  for  aiding  the  ALN  student  to  accomplish  these  tacit 
practices  are  described,  and  examples  were  given.  While  the  concepts  and  techniques  described  in  this  paper  are 
remarkably  "unremarkable,"  their  mundane  appearance  belies  their  importance  in  educational  interactions. 
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Abstract:  An  issue  concerning  school-based  learning  is  the  difficulty  many 
students  experience  in  applying  what  they  have  learned  to  everyday  situations.  Authentic 
learning  environments  such  as  problem-based  learning  have  been  proposed  as  a way  to 
provide  students  an  opportunity  to  engage  in  authentic  activities  in  authentic  contexts.  This 
research  study  examines  the  effects  of  expert  stories  on  students’  achievement  and  problem 
solving.  Two  sixth  grade  science  classes  were  randomly  assigned  to  the  treatment  conditions 
in  which  the  learning  environment  provided  support  in  the  form  of  expert  stories  or  expert 
non-stories.  Measures  of  factual  knowledge  recall,  near  and  far  transfer  were  administered. 
On  the  measure  of  factual  knowledge  recall,  no  significant  difference  was  found  between  the 
story  and  non-story  conditions.  However,  when  asked  to  solve  near  transfer  and  far  transfer 
problems,  students  in  the  story  treatment  did  significantly  better  than  students  in  the  non- 
story condition.  This  finding  suggests  that  expert  stories  can  scaffold  student  learning.  In 
particular,  it  appears  that  expert  stories  may  help  students  transfer  learning  to  novel 
situations. 


1.  Research  Framework 

An  issue  concerning  school-based  learning  is  the  difficulty  many  students  experience  in  applying 
what  they  have  learned  in  school  to  everyday  situations.  Many  schools  are  concerned  with  the  transfer  of 
abstract,  decontextualized  concepts  (Brown,  Collins,  & Duguid  1989).  Increasing  amounts  of  research 
indicate  that  the  inability  of  students  to  apply  concepts  learned  in  formal  contexts  is  due  to  the  abstraction  and 
decontextualization  of  learning  (Cognition  and  Technology  Group  at  Vanderbilt  1992).  Spiro  and  Jehng 
(Spiro  & Jehng  1990)  argue  that  the  decontextualization  of  knowledge  creates  inflexible  understandings.  But 
it  is  not  the  abstraction  of  knowledge  as  such  that  distracts  learners,  but  that  the  abstractions  are  not 
illuminated  with  examples  in  context.  Understanding  is  a product  of  the  context  and  activity  (McLellan 
1993). 

Context  provides  a framework  which  guides  and  supports  the  learner.  The  context  naturally 
structures  knowledge  in  a way  that  suggests  its  proper  use.  Traditional  instructional  design  attempts  to 
simplify  learning  by  dividing  the  content  into  components  and  teaching  them  separately.  This  reductionsist 
approach  most  likely  will  result  in  students  resorting  to  rote  memorization  rather  than  engaging  in  meaningful 
learning.  In  contrast,  situated  cognition  argues  that  learning  is  simplified  by  embedding  concepts  in  the 
context  in  which  they  will  be  used  (Brown  & Duguid  1993).  The  context  provides  the  much-needed  support 
when  working  at  the  edge  of  one’s  ability.  Yet  authentic  context  alone  is  not  enough  to  support  student 
learning.  Situated  cognition  argues  that  learners  must  engage  in  authentic  tasks  as  well  (Winn  1993). 
Authentic  activities  are  the  ordinary  practices  of  the  experts  in  that  domain  (Brown  et  al.  1989).  The  learning 
activity  is  an  integral  part  of  what  is  learned.  One  activity  in  which  all  domains  engage  is  the  solving  of 
problems.  Experts  utilize  tools  to  support  problem-solving  in  context.  Active  use  of  a culture’s  tools  provide 
learners  the  opportunity  to  view  knowledge  from  the  experts’  perspective  (Brown  et  al.  1989).  Active  learning 
in  the  context  of  a domain’s  culture  allows  the  gradual  “fleshing  out”  of  concepts  over  time. 

There  has  been  a growing  body  of  research  on  authentic  and  situated  learning  environments  utilizing 
the  problem-based  approach  to  learning  (Spiro  & Jehng  1990).  Problem-based  learning  (PBL)  emphasizes 


solving  authentic  problems  in  authentic  contexts.  It  is  an  approach  where  students  are  given  a problem, 
replete  with  all  the  complexities  typically  found  in  real  world  situations,  and  work  collaboratively  to  develop  a 
solution.  Problem-based  learning  provides  students  an  opportunity  to  develop  skills  in  problem  definition  and 
problem  solving,  to  reflect  on  their  own  learning,  and  develop  a deep  understanding  of  the  content  domain 
(Lajoie  1993)  (Jacobson  & Spiro  1995).  This  approach  was  developed  in  the  fifties  for  medical  education,  and 
has  since  been  used  in  various  subject  areas  such  as  business,  law,  education,  architecture  and  engineering. 
Most  recently,  there  is  a growing  interest  among  educators  to  use  problem-based  learning  in  the  K-12  setting, 
and  a growing  need  for  problem-based  educational  software  to  facilitate  the  development  of  higher  order 
thinking  skills  via  technology. 

Though  literature  supports  the  efficacy  of  problem-based  learning  environments,  little  research  exists 
which  investigates  the  types  of  tools  or  features  that  are  effective  in  supporting  students  working  in  PBL 
environments.  Many  researchers  have  found  that  providing  tools  for  students  to  use  while  working  in  a 
learning  environment  can  result  in  positive  learning  outcomes  (Lajoie  1993). 

With  an  increased  awareness  of  the  efficacy  of  learning  in  context,  there  is  a renewed  interest  in  the 
use  of  stories  as  an  information  resource  (Bell,  Gold,  & Kaplan  1998)  (Burke  1993).  Students  are  familiar 
with  the  narrative  format,  therefore  stories  are  a useful  vehicle  for  providing  information  (Edelson  1993). 
Stories  are  usually  memorable  since  they  provide  rich  detail  within  a specific  context  (Schank  1990).  Stories 
of  an  expert’s  experiences  can  provide  valuable  insight  into  the  application  of  knowledge  in  varied  contexts 
(Danzig  1997).  These  stories  can  also  provide  a considerable  amount  of  information  about  the  teller’s  beliefs 
(Burke).  All  of  these  factors  indicate  that  stories  may  be  a useful  mechanism  for  providing  information  to 
students  when  engaging  in  authentic  problem  solving. 

In  order  to  design  an  effective  computer-supported  PBL  learning  environment,  it  is  important  to 
understand  how  student  learning  can  be  supported.  It  is,  therefore,  the  purpose  of  this  study  to  examine  how 
the  use  of  first  person  narratives  related  by  experts  affect  student  learning  while  immersed  in  a problem-based 
learning  environment. 


2.  Research  Questions 

This  study  investigates  the  effects  of  expert  stories  on  middle  school  students’  learning  of  science 
concepts.  Specifically,  the  study  asked  the  following  questions:  (1)  How  do  expert  stories  affect  students’ 
acquisition  of  astronomy  concepts?  and  (2)  How  do  expert  stories  affect  students’  ability  to  transfer  learning? 


3.  Design  of  Study 

3.1  Sample 

The  participants  of  the  study  (N  = 101)  were  students  enrolled  in  sixth  grade  science  at  a middle 
school  located  in  a medium-sized  city  in  the  southwestern  United  States.  Four  intact  classes  participated  in  the 
study.  The  participants  in  the  study  consisted  of  73%  white,  17%  Hispanic  American,  7%  African  American, 
and  2%  Asian  American  students.  The  age  of  the  students  ranged  from  12  - 14  years.  There  were 
approximately  an  equal  number  of  male  and  female  students. 


3.2  Treatment 

For  this  research,  a hypermedia-supported  authentic  learning  environment  (HALE)  was  developed  in 
the  content  area  of  astronomy.  The  learning  goals  of  the  HALE  were  for  students  to  be  able  to: 

• Plan  and  implement  procedures  for  solving  complex  problems 

• Identify  relevant  information  needed  in  solving  a complex  problem 

• Identify  the  characteristics  of  objects  in  our  solar  system 
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• Describe  components  that  comprise  probes  used  for  astronomical  research 

• Rationalize  the  design  of  a probe  in  regards  to  it’s  intended  mission 

• Analyze  data  and  draw  conclusions  from  astronomical  data 

Two  versions  of  the  HALE  were  developed  corresponding  to  the  two  treatment  conditions:  stories  and 
non-stories.  All  versions  of  the  program  contain  the  same  content  and  tools  differing  only  in  the 
implementation  of  the  information  provided  by  the  expert  tool.  The  Stories  version  has  an  expert  tool 
containing  short  first  person  narratives  (video)  related  by  an  expert.  The  Non-stories  version  provides  the 
same  content  as  the  Stories  version,  except  the  expert  relates  factual  information  with  no  story  elements. 


3.3  Dependent  Measures 

Of  primary  interest  in  this  study  is  how  expert  stories  affect  student  learning.  Therefore,  measures  of 
knowledge  recall  and  transfer  were  administered.  In  addition  to  knowledge  recall  and  transfer  measures, 
interviews  were  conducted  with  the  participants  and  the  classroom  teacher.  The  triangulation  of  the 
quantitative  and  qualitative  data  sources  will  help  to  answer  the  research  questions  with  richer  and  more 
detailed  information. 


3.3.1  Recall  Measure 

The  recall  measure  evaluated  the  amount  of  declarative  knowledge  students  have  concerning  the 
astronomy  concepts  being  taught  in  the  study.  The  measure  consists  of  multiple  choice  and  fill-in-the-blank 
items.  A panel  of  reviewers  were  asked  to  verify  the  validity  of  the  measure.  Items  were  revised  until 
agreement  was  reached.  The  recall  instrument  was  administered  as  a pretest,  posttest,  and  retention  measure. 
Due  to  the  factual  and  objective  nature  of  the  measure,  it  was  scored  by  the  principal  investigator.  The  recall 
measure  was  used  to  gauge  the  degree  to  which  students  acquire  an  understanding  of  astronomy. 


3.3.2  Transfer  Measures 

As  noted  in  the  literature  review  section,  students  are  often  unable  to  apply  what  they  have  learned  in 
problem  solving  situations.  The  review  of  literature  put  forth  that  this  is  due  to  the  decontextulization  of 
learning.  Therefore,  in  order  to  evaluate  student  learning  it  is  not  sufficient  to  only  measure  acquisition  of 
concepts.  But  one  must  also  measure  the  ability  to  apply  that  knowledge  in  context.  Hence,  near  and  far 
transfer  measures  have  been  included  in  the  design  of  this  research. 


3.3.2. 1 Description  of  Near  Transfer  Measure 

In  order  to  evaluate  to  what  degree  students  are  able  to  apply  their  knowledge  in  solving  problems 
similar  to  the  one  received  in  the  treatment,  participants  were  given  a scenario  in  textual  form  describing  a 
problem  similar  to  the  one  received  during  the  study.  The  near  transfer  measure  was  given  as  a posttest  and 
used  to  provide  information  on  the  effects  of  the  expert  stories  on  student  learning.  After  reading  the  problem 
scenario,  participants  were  asked  to  provide  the  following  information: 

1 . Hypothesis  of  a solution  to  the  problem. 

2.  Rationale  for  hypothesis  which  includes  supporting  information  learned  while  using  the  environment. 

3.  Indication  of  other  information  that  would  need  to  be  gathered  to  support  the  answer. 

4.  Description  of  which  scientific  tools  could  be  used  to  gather  information  to  test  the  hypothesis. 

3.3.2.2  Description  of  Far  Transfer  Measure 
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In  order  to  evaluate  to  what  degree  students  are  able  to  apply  their  knowledge  in  solving  problems 
that  they  have  not  encountered  before,  students  were  asked  to  solve  a problem  unlike  those  encountered  during 
the  treatment.  The  far  transfer  measure  was  given  as  a posttest  and  used  to  gauge  the  degree  to  which  students 
are  able  to  apply  what  they  have  learned  to  a novel  situation.  After  reading  the  problem  scenario,  participants 
were  asked  to  provide  the  following  information: 

1 . Hypothesis  of  a solution  to  the  problem. 

2.  Rationale  for  hypothesis  which  includes  supporting  information  learned  while  using  the  learning 
environment. 

3.  Indication  of  other  information  that  would  need  to  be  gathered  to  test/support  the  hypothesis. 

4.  Description  of  which  scientific  tools  could  be  used  to  gather  information  to  test/support  the  hypothesis 


3.3.2.3  Scoring  of  Near  and  Far  Transfer  Measures 

Since  scoring  the  near  and  far  transfer  measures  is  subjective  in  nature,  responses  from  the  near  and 
far  transfer  measures  were  evaluated  by  three  trained  graders  who  will  be  blind  to  the  subjects’  treatment 
conditions.  Due  to  the  large  number  of  responses  to  evaluate,  the  responses  will  be  randomly  divided  between 
the  three  graders.  In  order  to  ensure  interrater  reliability,  each  grader  was  given  an  additional  20%  of  the 
essays  from  each  of  the  other  two  graders.  These  additional  essays  were  randomly  selected,  scored,  and 
compared  with  the  scores  from  all  graders.  Each  essay  was  scored  in  the  following  manner: 

L A score  from  0 to  15  was  given  for  the  essay  on  the  following  criteria: 
(a)  Quality/plausibility  of  the  hypothesis  and  supporting  rationale,  (b)  Level  of  incorporation  of 
details  and  facts  in  the  essay,  (c)  Thoroughness  of  analysis  of  needed  information  to  support  answer, 
(d)  Number  of  tools  cited  and  the  appropriate  use  of  them,  (e)  overall  originality  of  ideas,  (f)  overall 
quality  of  the  essay. 

2.  Scores  from  each  of  the  criteria  were  collapsed  into  one  overall  score  for  the  response. 


3.4  Procedures 

Four  intact  classes  were  randomly  assigned  to  either  the  Stories  condition  or  the  Non-stories 
condition  resulting  in  two  classes  in  each.  Because  the  treatment  was  included  as  a part  of  the  regular  science 
class,  complete  random  assignment  was  not  possible  in  this  case.  The  assignment  resulted  in  49  students  in 
the  Stories  condition  and  52  students  in  the  Non-stories  condition.  Within  each  class,  students  worked  in 
heterogeneous  groups  of  3-4  students  created  by  the  classroom  teacher.  Students  were  presented  with 
information  about  the  nature  of  the  learning  environment.  They  were  informed  that  at  the  end  of  the  activity 
they  would  be  asked  for  input  concerning  the  aspects  of  the  environment  they  found  useful  and  those  they  did 
not.  Prior  to  treatment,  students  were  asked  to  complete  the  recall  measure.  After  all  groups  had  completed 
the  activity,  participants  were  asked  to  complete  the  recall  measure,  near  transfer  measure,  and  far  transfer 
measure.  Three  weeks  after  the  completion  of  the  study,  the  recall  measure  was  administered  once  again  to  all 
of  the  participants  to  collect  retention  data. 


4.  Analysis,  Results  and  Discussion 

In  order  to  examine  the  effects  of  expert  stories  on  students’  declarative  knowledge  (research  question 
one),  a simple  ANOVA  with  repeated  measure  was  conducted  using  the  data  collection  points  pre,  post,  and 
retention  on  the  factual  knowledge  measure  with  the  treatment  conditions  as  factors.  The  results  show  there 
was  no  significant  effect  for  data  collection  point  on  achievement  from  pre-treatment  to  post-treatment  to 
retention  when  considering  the  interaction  between  the  treatment  condition  and  the  measure:  F(2,94)  = 1.34, 
p = .266  (STORY  MEAN  p„  = 6.96;  STORY  MEAN  post  = 8.22;  STORY  MEAN  retention  = 7.87;  NON-STORY 
MEAN  pre  = 5.96;  NON-STORY  MEAN  post  = 7.27;  NON-STORY  MEAN  retention  = 8.12).  That  is,  students 
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who  received  expert  stories  performed  similarly  to  students  who  received  expert  non-stories  on  the  measure  of 
factual  knowledge. 

The  unexpected  lack  of  a significant  difference  between  the  story  and  non-story  treatment  conditions 
may  be  attributed  to  a number  of  factors.  The  HALE  used  for  this  research  had  many  features  to  support 
student  learning.  The  only  apparent  variation  between  the  two  treatment  conditions  was  that  the  story 
treatment  received  expert  support  in  the  form  of  stories  while  the  non-story  condition  received  the  same 
information,  but  in  a format  that  was  not  a story.  Since  the  environment  was  designed  with  many  features  to 
support  authentic  context  and  activity,  the  additional  support  provided  by  expert  stories  may  not  have  been 
strong  enough  to  make  a difference  in  student’s  declarative  knowledge.  Especially  since  both  treatments 
conditions  received  the  support  “just-in-time”.  In  other  words,  other  features  in  the  learning  environment  may 
have  supported  students  in  the  non-story  condition.  Similarly,  stories  and  non-stories  provide  the  same  factual 
information  differing  only  in  the  presentation.  Therefore,  acquisition  of  factual  knowledge  may  not  have  been 
influenced  since  the  factual  content  was  equivalent  in  the  two  treatment  conditions. 

In  order  to  measure  the  effects  of  expert  stories  on  near  transfer  (research  question  two),  a simple 
ANOVA  was  calculated  with  story/non-story  conditions  as  a between-subjects  independent  variable  and  the 
near  transfer  measure  as  the  dependent  variable  (post-treatment  only).  The  results  indicate  there  was  a 
significant  difference  between  the  treatment  groups:  F(l,52)  = 631,  p = .015  (MEAN  story  = 10.6;  MEAN  non- 
story=  6.9).  That  is  to  say,  students  who  received  the  expert  stories  performed  significantly  better  in  solving  a 
near  transfer  problem  than  students  which  received  non-stories.  In  order  to  explore  the  effects  of  the  expert 
stories  on  far  transfer  (research  question  two),  a simple  ANOVA  was  calculated  with  story/non-story 
conditions  as  a between-subjects  independent  variable  and  the  far  transfer  measure  as  the  dependent  variable. 
The  results  show  a significant  difference  between  the  story  and  non-story  treatments:  F(l,52)  = 4.6,  p = .037 
(MEAN  story  = 7.8;  MEAN  non-story  =6.1).  In  other  words,  students  who  received  the  expert  stories  performed 
significantly  better  in  solving  a far  transfer  problem  than  students  who  received  non-stories. 

These  findings  support  the  hypothesized  benefits  of  situated  cognition  in  a number  of  respects.  Recall 
that  situated  cognition  argues  that  it  is  essential  that  learning  occur  in  an  authentic  context,  that  students 
engage  in  authentic  activities,  and  that  students  learn  about  the  culture  of  the  knowledge  domain  (Brown, 
Collins,  Duguid  1989)  (Choi  & Hannafin  1995)  (McLellan  1993).  The  ability  of  students  to  solve  complex 
problems  is  often  cited  as  one  of  the  most  important  goals  of  education.  However,  traditional  methods  of 
teaching  often  fall  short  of  this  elusive  goal.  Students  frequently  have  difficulty  solving  such  problems  and 
often  need  support.  This  research  hypothesized  that  expert  stories  might  help  scaffold  student  problem 
solving.  Data  collected  for  this  research  indicate  that  students  who  received  expert  stories  did  significantly 
better  on  solving  near  and  far  transfer  problems  than  students  who  received  the  same  information  but  not  in 
story  form. 

It  can  be  argued  that  stories  can  help  provide  a meaningful  context  for  student  learning  through  the 
rich  detail  included  in  them.  Stories  are  also  capable  of  illustrating  knowledge  in  context,  hence  making 
learning  more  meaningful.  It  appears  that  the  authentic  context  supported  by  expert  stories  can  facilitate 
students’  ability  to  transfer  learning  to  new  situations.  Research  by  the  Cognition  and  Technology  Group 
(Cognition  and  Technology  Group  at  Vanderbilt  1992)  supports  this  finding  that  providing  an  interesting 
relevant  context  for  problem  solving  can  facilitate  transfer  to  new  problem  solving  situations.  Other 
implementations  of  learning  environments  which  emphasize  the  importance  of  providing  a rich  context  for 
problem  solving  such  as  Schank’s  goal-based  scenarios  (Schank  & Edelson  1990)  and  Spiro’s  cognitive- 
flexibility  hypertexts  (Spiro  & Jehng  1990)  (Jacobson  & Spiro  1995)  further  support  this  finding. 

Stories  can  also  support  student  engagement  in  authentic  activities.  Scientists  often  share  information 
in  the  form  of  stories.  The  listening  to  stories  is  itself  an  authentic  activity.  Mentors  often  share  stories  with 
apprentices  in  order  to  illustrate  some  point.  Both  theory  and  practice  can  be  shown  by  the  use  of  expert 
stories  (Danzig  1997).  Hence,  the  stories  help  students  engage  in  authentic  activities  thereby  facilitating  better 
performance  on  problem  solving.  Research  on  anchored  instruction  and  problem-based  learning  support  this 
finding  that  engaging  in  authentic  activities  can  facilitate  transfer  of  learning  to  new  problem  solving 
situations  (Cognition  and  Technology  Group  at  Vanderbilt  1992). 


4.  Conclusions 


As  problem-based  learning  becomes  more  popular,  instructional  designers  must  find  new  ways  to 
utilize  technology  in  order  to  support  problem-based  learning.  For  this  research  an  innovative  problem-based 
learning  environment  was  created  which  provided  the  context  for  learning  and  authentic  activities  for  students 
to  engage.  Just  as  a mentor  provides  stories  to  an  apprentice  while  working  in  context,  this  software  attempted 
to  provide  stories  to  students  as  they  were  working.  Constructivism  and  situated  cognition  provide  a useful 
framework  for  realizing  that  it  is  not  enough  to  rely  on  the  affordances  of  media,  but  that  the  media  must  be 
utilized  to  facilitate  the  engagement  of  students  in  meaningful  activities.  In  order  to  make  learning 
meaningful,  students  should  be  placed  in  learning  situations  which  allow  them  to  engage  in  the  same  types  of 
activities  experts  would  in  a context  that  is  rich  and  interesting.  Hypermedia-supported  authentic  learning 
environments  (HALE)  can  be  designed  to  accomplish  this  goal.  HALEs  bring  together  the  valuable  elements 
of  hypermedia  (i.e.  audio,  video,  animation,  non-linear  access  to  information)  and  problem-based  learning 
environments  (authentic  context  and  authentic  activities)  to  create  rich  contexts  for  learning. 

The  confluence  of  hypermedia  technology,  constructivism,  situated  cognition,  and  problem-based 
learning  provide  an  opportunity  for  instructional  designers  to  create  authentic  learning  environments. 
Hypermedia-supported  authentic  learning  environments  can  extend  beyond  the  school  walls  by  providing  a 
series  of  authentic  experiences  which  can  direct  students  along  the  path  of  life-long  learning  (Dewey  1938). 
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Abstract:  The  exploration  of  social  networks  is  essential  to  find  capable  collaborators  who  can  help  problem- 
solving and  to  augment  cooperation  between  users.  This  paper  describes  PeCo-Mediator-II  for  seeking  for  a 
collaborator  with  the  chain  of  personal  connections  (PeCo)  in  distributed  organizations.  Moreover,  this  system 
helps  gathering,  exploring,  and  visualizing  social  networks.  The  experimental  results  show  that  the  system 
facilitates  that  learners  encounter  collaborators  and  develop  a new  helpful  relationship  beyond  the  classroom. 


Introduction 

Recently,  opportunities  for  communication  and  collaboration  via  computer  networks  have  immensely 
been  increased  in  networked  organizations  (Sproull  and  Kiesler,  1991).  A fundamental  problem  is  how  to 
encounter  people  who  can  help  problem-solving.  We  are  focusing  on  the  problem  of  discovering  such  people 
through  social  networks.  Social  networks  are  at  least  as  important  as  the  official  organizational  structures  for 
tasks  ranging  from  immediate,  local  problem  solving  (e.g.,  fixing  a piece  of  equipment),  to  primary  work 
functions,  such  as  creating  collaborative  groups  (Kautz  et  al.,  1997). 

In  CSCW  (Computer  Supported  Cooperative  Work),  researchers  are  interested  in  the  role  of  social 
networks  between  organizational  members.  Clement  stated  that  users  developed  informal  collaborative  networks 
to  know  how  to  use  a new  software  (Clement,  1990).  Then,  private  networks  are  important  for  workers  to  solve 
problems  by  providing  helpful  information.  A number  of  studies  have  shown  that  one  of  the  most  effective 
channels  for  gathering  information  and  expertise  within  on  an  organization  is  its  informal  networks  of 
collaborators,  colleagues  and  friends.  The  networks  of  helping  relationships  are  called  ''Help  Network" (EvQlmd 
et  al.,  1 994).  However,  the  networks  are  not  collected  and  generally  follow  work  group  alignments  rather  than 
technical  specialization.  Therefore,  it  is  significant  to  use  members’  interpersonal  connections  effectively  in  their 
activities. 

In  CSCL  (Computer  Supported  Collaborative  Learning),  one  common  component  of  collaborative 
learning  is  the  "informal  peer-help  networks".  This  notion  is  compatible  with  Wenger's  communities  of  learners 
(Wenger,  1996),  where  people  who  share  learning  goals  within  an  authentic  learning  environment  can  develop 
ties  that  reinforce  learning  outcomes.  From  this  viewpoint,  Greer  et  al.  (1998)  proposed  PHelpS  (Peer  Help 
System)  that  supports  workers  as  they  perform  their  tasks,  offers  assistance  in  finding  peer  helpers  when 
required,  and  mediates  communication  on  task-related  topics.  On  the  other  hand,  our  approach  focuses  on  how  a 
system  can  support  both  storing  and  exploring  "Personal  Connection"  (PeCo)  in  a collaborative  learning 
environment. 

W propose  PeCo-Mediator-Il  (Ogata  et  al.,  1996a;  Ogata  et  al.,  1998)  for  gathering,  seeking,  and 
visualizing  social  networks  in  a networked  organization.  PeCo-Mediator-II  is  a distributed  system  with  a 
personal  database  (PeCo-Collector)  and  a software  agent  (PeCo-Agent).  Every  user  has  the  two  softwares  on  the 
respective  site.  PeCo-Collector  incrementally  gathers  information  on  its  user's  acquaintances  and  the 
relationships  through  watching  the  exchanges  of  e-mail.  PeCo-Agent  moves  to  colleagues'  sites  and  negotiates 
with  other  agents  and  users  to  find  collaborators.  Although  the  users  of  both  NetNews  and  e-mail  lists  are 
passive  to  find  answers,  our  system  can  actively  discover  collaborators  with  the  chain  of  personal  connection 
from  the  user  and  the  collaborators. 

Overview  of  PeCo-Mediator-II 

Our  initial  system  called  PeCo-Mediator  (Ogata  et  al.,  1995)  is  a groupware  that  allows  sharing  of  PeCo 
in  a group  and  to  search  for  connections  between  the  user  and  targets.  The  users  need  to  share  PeCo  with  the 
common  database  of  PeCo-Mediator.  Although  the  system  was  very  available  in  some  small  groups,  it  was 
reluctant  in  terms  of  users  offering  their  private  information  like  PeCo  into  the  common  database.  Also,  it  is  hard 
for  the  users  to  entry  personal  data  of  their  friends. 

When  a computer  network  connects  people  or  organizations,  it  is  a social  network.  Just  as  a computer 
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network  is  a set  of  machines  connected  by  a set  of  cables,  a social  network  is  a set  of  people  connected  by  a set 
of  social  relationships,  such  as  friendship,  co-working,  or  information  exchange  (Garton  et  al.,  1997).  Computer 
Mediated  Communication  (CMC)  systems  also  reduce  the  transaction  costs  of  initiating  and  maintaining 
interpersonal  ties  (Pickering  and  King,  1992).  Weak  ties  created  by  CMC  expand  the  channels  of  information 
sources  for  the  individual  and  have  potential  for  strong  ties. 

PeCo-Mediator-II  is  combined  PeCo-Mediator  and  on-line  social  networks.  It  consists  of  the  two  systems; 
PeCo-Collector  and  PeCo-Agent  (see  Figure  1).  Every  organizational  member  has  the  two  softwares  on  the 
respective  site.  PeCo-Collector  gathers  information  on  its  user's  acquaintances  and  the  relationships  through 
watching  the  exchanges  of  e-mail.  PeCo-Agent  moves  between  members'  sites  to  find  a partner  in  the 
community.  The  user's  PeCo  is  a starting  point  for  the  exploration.  The  user's  acquaintance  acts  as  a liaison 
between  the  user  and  the  partner  in  this  situation.  In  this  figure,  user  X requests  user  Y to  help  the  problem 
solving,  and  user  Y introduces  user  Z.  After  that,  user  Z can  help  user  X by  request. 

The  characteristics  of  this  system  are: 

1)  Accumulation  of  on-line  and  off-line  social  networks:  Mainly,  our  system  deals  with  PeCo  based  on  the 
exchange  of  e-mail.  PeCo-Mediator-II  automatically  stores  relationships  based  on  e-mail  tags  (Ogata  et  al., 
1996).  In  addition,  the  user  can  provide  on-line  relationships;  e.g.,  based  on  the  exchange  of  name-cards. 

2)  Measurement  of  PeCo  strength:  The  strength  of  PeCo  is  estimated  with  the  frequency  of  e-mail  exchange. 
This  degree  is  very  useful  for  deciding  the  receivers  of  the  request  (Ogata  et  al.,  1996). 

3)  Privacy  protection:  PeCo-Mediator-II  manages  individual  ties  with  a distributed  personal  database  in  the 
user's  own  site.  Personal  data  is  safer  in  a personal  database  than  in  a common  database.  Therefore,  it  is 
easy  for  this  system  to  protect  user's  privacy  and  to  be  accepted  in  a large-scale  organization. 

4)  Compatibility:  The  architecture  of  PeCo-Mediator-II  is  compatible  with  existing  e-mail  mechanisms. 
Compatibility  reduces  user  overhead  in  taking  advantage  of  the  e-mail  tools. 

5)  Scalability:  Even  if  the  number  of  users  increase,  this  system  can  work  robustly  because  of  an  agent  based 
distributed  system  architecture. 

6)  Parallel  exploration  assisted  by  agents:  PeCo-Agent  supports  the  user  to  search  for  a collaborator  through 
social  networks  while  negotiating  with  other  users  and  PeCo-Agents.  Moreover,  the  user  can  visually 
understand  the  current  status  of  the  exploration  and  easily  control  that  process. 

7)  Mitigation  of  collaborators'  overload:  The  questions  are  possibly  concentrated  on  a part  of  users  (experts). 
This  system  provides  a common  database  of  answers  and  navigates  the  questions  with  strategies  on 
educating  the  secondary  collaborators  and  on  spreading  the  answers. 


Exploration  Model  of  Social  Networks 

PeCo  Exploration  Diagram 

Figure  2 shows  the  data  diagram  of  exploring  social  networks.  User  A is  a sender  of  e-mail  and  user  B is 
its  receiver.  Each  node  denotes  the  state  during  the  exploration,  and  it  moves  to  the  next  state  if  the  user  acts  on 
the  activity  of  the  arc.  The  starting  point  is  the  state  one  and  the  ending  point  is  the  state  four,  five  and  six.  The 
state  2'  shows  the  exploration  is  continued  from  the  state  2 after  changing  the  receiver.  The  sender  has  three 
options:  request,  cancel  and  remind.  The  receiver  has  five  options:  read,  not-read,  accept,  forward,  and  reject. 
For  example,  if  user  A sends  a request  to  user  B,  the  receiver  either  reads  it  or  does  not  read  it.  If  user  B does  not 
read  it,  user  A may  remind  user  B to  read  it.  If  user  B reads  it,  s/he  either  accepts  the  request,  rejects  it,  or 
forwards  it  to  his/her  friends.  Until  someone  accepts  the  request  or  user  A cancels  it,  the  exploration  of  social 
networks  is  continued.  Finally,  user  A acknowledges  to  user  B. 


156 


Table  1.  Taxonomy  of  users  with  the  exploration  history. 

Usert]^8  Conditioii 

Collaborator  ^ccept>  ctand  ^ccept>  reject  and  accept>  forward ’siA  accept>  reqi^t 
Mediator  forward > aziA  forward > reject  and  forward>  accept 

Requestor  reqeust>  aziA  req{\ssi>  and/pqift?pf>  Awarrf 

Non-collaborator  reject>  a and  reject  > accept  and  reject>  forward 

Semi-collaboiator  recejw_accept>  adiiA  ■rprpjvp_acrep^  > accept 

(Italic  vord  shovs  the  number  of  the  action,  o:  is  a constant  given  by  the  user.) 


Taxonomy  of  Users 

Based  on  the  above  history,  we  divide  users  into  the  five  types  (see  table  1): 

1)  Collaborator:  The  collaborator  is  a user  who  usually  accepts  the  request  during  this  system  use.  The 
collaborator  is  often  an  expert  about  the  request. 

2)  Semi -collaborator:  The  semi-collaborator  is  a user  who  potentially  has  the  capability  for  cooperation  about 
the  request.  We  assume  that  a semi -collaborator  receives  the  answer  from  others  rather  than  accepting 
requests. 

3)  Mediator:  The  Mediator  is  the  user  who  usually  forwards  the  request  to  his/her  friends. 

4)  Requestor:  The  requestor  is  a user  who  asks  a question  and  s/he  becomes  a stating  point  of  exploration  of 
social  networks. 

5)  Non-collaborator:  The  non-collaborator  is  a user  who  almost  rejects  the  request. 

6)  Unknown  user:  If  a user  has  never  received  or  sent  a request,  the  user  is  unknown  for  the  system. 

PeCo- Agent  understands  the  users’  capability  through  watching  the  exchanges  of  questions  and  answers.  We 
represent  the  capability  of  the  user  and  his/her  acquaintances  with  the  keywords  in  the  e-mail.  For  example,  a 
friend  is  a collaborator  about  C programming  language  although  the  friend  is  a non-collaborator  about  Tcl/Tk. 

Supporting  PeCo  Exploration  with  History 

PeCo-Agent  helps  a user  in  the  following  situation: 

1)  Request  support:  When  the  user  decides  a receiver  of  the  request,  this  system  shows  the  user  the  following 
information  about  the  receiver: 

a)  User's  type  (a  collaborator,  semi-collaborator,  mediator,  requestor,  non-collaborator,  or  unknown  user): 
If  the  receiver  is  a collaborator,  the  user  may  find  the  answer  easily. 

b)  Strength  of  relationship:  If  the  connection  between  the  user  and  the  receiver  is  strong,  it  is  easy  to  ask 
for  cooperation. 

c)  The  number  of  requests  left  unattended:  If  the  receiver  has  many  requests,  his/her  answer  may  be  late. 

d)  System  usage:  If  the  receiver  uses  this  system  at  that  time,  the  user  can  obtain  the  response  as  soon  as 
possible. 

2)  Acceptance  support:  If  the  user  accepts  a request,  the  system  provides  the  reply  of  the  request  to  the  user. 
The  user  edits  the  past  results  to  answer  it. 

3)  Forward  support:  PeCo-Agent  shows  the  user  the  possible  acquaintances  that  can  help  the  user  with  the 
results  of  the  past  exploration.  If  the  user  has  a friend  who  is  a collaborator,  semi-collaborator  or  mediator, 
PeCo-Agent  recommends  them  as  the  receiver. 

4)  Reject  support:  PeCo-Agent  automatically  rejects  the  request,  if  the  relationship  strength  between  the  user 
and  the  request  sender  is  lower  than  the  given  value  by  the  user. 

Reducing  the  overload  of  collaborators 

Collaborators  are  often  burdened  with  the  requests  from  others.  To  mitigate  that,  we  propose  educating 
semi-collaborators  in  this  system.  If  semi-collaborators  are  educated  and  reach  a level  of  collaborators,  the 
number  of  collaborators  increases.  Therefore,  PeCo-Agent  recommends  a collaborator  to  send  the  request  to 
semi-collaborators  and  reduces  the  overload  of  collaborators.  The  collaborator  sends  the  answer  to  the  requester 
after  checking  and  correcting  the  answer  from  the  semi-collaborators.  Next  time,  PeCo-Agent  recommends  that 
the  requester  sends  the  request  directly  to  the  semi-collaborator.  After  that,  the  semi-collaborator  becomes  a new 
collaborator  and  the  requester  becomes  a new  semi-collaborator.  Moreover,  the  free  rider  issue  (Salomon,  1 992), 
which  is  a user  who  obtains  information  without  giving  any,  might  be  settled  also. 
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Implementation 

System  Configuration 

We  developed  a prototype  system  on  a workstation  with  Tcl/Tk  (Ousterhoult,  1994). 

(1)  PeCo-Collector:  This  system  has  two  components:  data  management  and  E-mail  handler.  All  the  data  is 
managed  by  TRIAS  (Triple  Associative  System)  and  the  e-mail  tool  is  TkMH  based  on  MH  (Peek,  1994), 

(2)  PeCo-Agent:  The  characteristics  of  PeCo-Agent  are: 

1)  To  represent  capability  of  users  with  keywords  about  e-mails; 

2)  To  obtain  the  capability  of  users  from  the  user  and  other  agent; 

3)  To  move  around  the  Internet  and  communicate  with  other  users  and  agents; 

4)  To  find  the  candidates  of  partners  concurrently. 

In  PeCo-Mediator-II,  a user  communicates  and  negotiates  with  others  through  e-mail.  In  the  same  way,  PeCo- 
Agent  communicates  with  other  agents  with  structured  e-mail  (Malone,  1986).  Keywords  are  extracted  with 
Chasen  (Matsumoto,  1997)  that  is  a Japanese  morphological  analysis  tool,  PeCo-Agent  calculates  the  similarity 
between  the  given  question  and  the  stored  questions  by  matching  nouns  elicited  from  Chasen  filter. 

Interface 

Figure  3 shows  the  interaction  after  "aiso"  asks  a question  to  his  PeCo-Agent,  In  the  window  (A)  "aiso"  writes 
the  request  message.  In  the  window  (B),  the  user  sets  time  out  for  seeking  social  networks,  the  minimum 
strength  of  PeCo  and  the  maximum  steps  between  "aiso"  and  the  receiver,  PeCo-Agent  finishes  the  exploration 
according  to  this  setting.  In  the  window  (C),  PeCo-Agent  assists  "aiso"  to  decide  who  is  the  better  receiver  of  his 
acquaintances  and  the  user  agent  provides  information  about  the  candidates  of  the  receivers.  The  window  (D) 
displays  the  list  of  the  requests  that  the  user  has  sent.  The  window  (E)  shows  the  flow  of  the  exploration  from  he 
user  graphically.  This  tree  is  the  result  of  traveling  with  the  connections  of  "aiso".  The  icons  except  "aiso"  denote 
the  candidates  of  partners.  The  shorter  the  distance  between  two  icons,  the  stronger  the  relationship  they  have. 
While  the  dotted  line  denotes  the  receiver  has  not  read  the  message  yet,  the  solid  line  shows  the  receiver  has 
already  read  it.  The  black  icon  means  the  user  has  rejected  the  request.  The  node  icon  shows  the  user  has 
forwarded  the  message  to  his/her  friends.  The  leaf  and  white  icon  means  the  user  has  accepted  the  cooperation. 
In  this  figure,  "mendori"  refused  aiso's  request,  and  "ogata",  "abe",  and  "kawasaki"  agreed  to  his  request,  "akagi" 
has  not  read  the  message  yet.  If  the  user  reminds  the  reply  to  the  request  from  this  window,  PeCo-Agent  of 
"akagi"  tells  him  to  read  the  message.  From  this  result,  "aiso"  is  the  most  familiar  to  "goji"  and  can  easily  access 
the  collaborator  "ogata"  through  the  mediation  of  "goji". 
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Figure  3:  Screen  shot  of  PeCo  exploration  with  PeCo-Mediator-II. 


Experimental  Use 

Users  and  Tasks 


In  this  experiment,  we  arranged  13  master  course  students  (group  A)  and  94  undergraduate  students  (group 
B)  who  had  no  relationship  with  the  members  of  group  A at  the  first  stage  of  the  experiment.  Only  one  person, 
teacher  VI,  knows  all  the  members  of  group  A and  B.  They  used  the  prototype  system  during  nine  weeks  in  a 
class  of  programming  language  C.  We  divided  nine  weeks  into  three  terms.  Teacher  VI  gave  group  B some 
homework  every  week,  for  example,  making  a program  of  data  sorting. 

Term  1 : In  the  first  three  weeks,  the  system  gathered  their  usual  ties.  Each  group  member  communicated  among 
the  internal  group  members  without  the  contact  of  the  other  group. 

Term  2;  We  allowed  group  A and  B to  communicate  and  collaborate  with  each  other  to  solve  problems.  The 
users  solved  the  given  problems  through  this  system  without  supporting  PeCo  exploration  with  history. 

Term  3:  In  this  period,  we  evaluated  the  function  for  supporting  PeCo  exploration  with  the  history  that  was 
stored  in  the  term  2. 


Experimental  Results 

Figure  4 shows  the  social  networks  between  the  users  after  six  weeks  from  the  beginning  of  this 
experiment.  While  the  user  of  group  A is  indicated  by  a circle,  the  user  of  group  B is  shown  by  a square.  The 
thick  arrows  denote  the  requested  messages  from  the  sender  to  the  receiver.  The  thin  arrows  represent  the 
forwarded  messages  over  one  time.  The  weight  of  the  arrow  shows  how  many  times  e-mail  was  exchanged  from 
the  sender  to  the  receiver.  The  user  VI  was  a central  person  and  acted  as  a liaison  between  group  A and  B.  As 
shown  in  this  figure,  group  A and  B learned  to  communicate  with  each  other  through  the  introduction  of  user  VI, 
although  they  did  not  have  connections  beyond  the  group.  Moreover,  most  of  the  requests  from  group  B 
concentrated  on  user  VI  and  III,  and  the  collaborators  were  almost  fixed  at  six  persons  of  group  A.  In  this  case, 
there  was  no  collaborator  in  group  B. 
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Figure  4.  Network  forming  in  the  term  2.  Figure  5.  Network  forming  in  the  term  3. 


In  the  previous  experiment  (Ogata  et  aL,  1998),  we  compared  this  system  with  e-mail,  mailing  list  and 
NetNews  during  four  weeks.  This  experiment  was  executed  in  the  same  class.  Both  mailing  list  and  NetNews 
were  not  often  used  for  getting  answer,  because  the  student  hesitated  to  ask  a question.  On  the  other  hand,  both 
this  system  and  e-mail  were  frequently  used.  In  this  case,  social  networks  were  stable  because  direct  and  explicit 
relationships  were  used  to  get  collaborative  help.  Likewise,  (Yamakami,  1995)  describes  the  interaction  patterns 
of  e-mail  and  bulletin  board  are  stable  from  the  long-term  usage  observation. 

Figure  5 depicts  the  message  flow  in  the  term  3.  Some  of  group  B became  collaborators  because  our 
system  lead  the  users  to  reduce  the  incipient  collaborators'  load.  For  example,  the  system  recommended  user  VI 
to  forward  the  request  from  user  32  to  user  17  who  was  a semi-collaborator.  After  that,  user  32  directly  requested 
user  17  to  cooperate  with  problem  solving. 
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Figure  6.  Rate  of  collaborators  in  the  term  3. 


Figure  6 shows  the  comparison  of  the  collaborators 
in  the  term  2 and  3.  In  term  2,  the  user  VI  and  II 
accounted  for  78%  of  all  collaborators,  and  there  was 
no  collaborator  in  group  B.  In  term  3,  the 
cooperation  rate  of  the  user  VI  and  111  decreased 
69%  and  some  of  group  B became  collaborators. 
This  experimental  result  seems  to  show  that  the 
facility  of  this  system  is  available  to  prevent  the 
collaborators  from  being  fixed  and  to  facilitate 
mutual  cooperation.  In  the  term  3,  the  users  provide 
their  knowledge  into  the  shared  repository. 


Conclusion 

This  paper  proposed  PeCo-Mediator-II  as  a support  to  find  capable  collaborators  with  the  chain  of  personal 
connections  in  a collaborative  learning  environment.  This  system  helps  gathering,  seeking,  and  visualizing  social 
networks  of  organizational  members.  PeCo-Mediator-II  is  an  agent-based  system  to  deal  with  e-mail.  This 
system  consists  of  PeCo-Collector  as  a personal  database  and  PeCo-Agent  as  a user's  assistant.  PeCo-Mediator-lI 
was  experimentally  tested  and  evaluated  in  a C programming  language  course.  The  results  showed  the  system 
could  help  the  user  to  encounter  a collaborator  and  developed  the  new  relationship  with  the  collaborator. 
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Abstract;  The  design,  development,  implementation,  and  evaluation  of  Web-based 
courseware  for  teaching  and  learning  are  important  issues  in  higher  education. 
However,  available  evidence  suggests  evaluation  of  much  courseware  is  minimal. 

This  paper  reports  a meta-analysis  of  the  adoption,  usage,  and  implementation  of  a 
generic,  customisable,  online  survey  / evaluation  software  tool — Learning  Evaluation 
Online  (LEO).  A customisable  evaluation  tool  derived  from  the  need  to: 

• customise  each  survey  to  suit  each  particular  project, 

• collect  data  from  widely  geographically  separated  evaluators,  and 

• decrease  the  time  and  expense  of  data  collection  and  evaluation. 

LEO  was  also  designed  to  be  customisable  by  individuals  with  minimal  computing 
experience.  Templates  for  basic  evaluation  question  types  are  provided.  The  use  of 
LEO  was  logged  by  the  LEO  software  engine  and  our  LEO-on-LEO  survey.  An 
analysis  of  the  quantitative  and  qualitative  data  generated  is  presented  together  with 
recommendations  for  improving  implementation  of  similar  tools. 


Introduction 

In  the  recent  past,  the  evaluation  of  courseware  has  often  occurred  when  the  project  is  nearing  the  end  of  the 
software  development.  However,  this  is  changing  as  the  perception  of  the  value  of  formative  evaluation  has 
improved  (Barker,  1995;  Beattie,  1994).  Laurillard  (1994,  p.  287)  suggested  that  "formative  evaluation  has  a 
vital  role  to  play  in  the  progress  of  multimedia  because  it  is  the  means  by  which  we  build  up  our  knowledge  of 
what  the  medium  can  do". 

Learning  Evaluation  Online  (LEO)  was  originally  developed  as  a survey  / evaluation  tool  in  response  to  a need 
for  an  asynchronous  Web-based  evaluation  mechanism,  that  could  be  easily  modified  to  suit  a variety  of 
potential  respondents  (e.g.,  peers,  students,  multimedia  developers,  interface  designers,  etc.),  and  courseware 
from  various  content  domains.  We  also  wanted  to  decrease  the  time  and  expense  of  data  collection  and 
evaluation.  Other  systems  have  been  reported  (Recker  & Greenwood,  1997).  However  these  systems  lacked  the 
ability  to  be  used  easily  by  non-programmers.  A system  such  as  LEO  had  the  potential  to  make  evaluation  of 
courseware  less  problematic — in  terms  of  programming,  deployment,  and  adaptation  for  a range  of  evaluation 
or  survey  needs. 

The  LEO  software  environment  is  a template  using  the  first  author's  OXYGEN  (Object  extensible  analYsis  and 
Generation  of  Education  coNtent)  software  engine.  The  authoring  capability  of  the  OXYGEN  software  engine 
is  extremely  flexible,  making  it  very  adaptable  for  a wide  variety  of  potential  for  uses,  and  users.  Initially,  LEO 
was  developed  to  meet  the  needs  of  evaluating  interactive  Web-based  learning  tools  then  under  development  at 
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Multimedia  Education  Unit,  at  the  University  of  Melbourne  (Kennedy  & Fritze,  1998;  Kennedy,  Ip,  Eizenberg, 
& Adams,  1998).  LEO  has  a number  of  purpose  built  features  that  support: 

• the  creation  of  the  survey  objectives  as  a hyperlink, 

• the  entry  of  the  purpose  of  the  survey, 

• the  generation  of  common  demographic  details,  and 

• seven  different  pre-defmed  question  styles,  including  Likert  scales,  true/false  items,  free  text  responses  and 
a custom  style. 

A more  complete  description  of  the  basic  functionality  of  LEO  may  be  found  in  (Kennedy  & Ip,  1 998). 

A Generic  Survey  Tool  versus  a Specific  Measuring  Instrument 

LEO  was  designed  to  meet  a specific  need — evaluation  of  Web-based  courseware.  However,  it  is  a tool,  not  an 
instrument.  Most  published  survey  or  evaluation  instruments  are  in  paper  form  (e.g.,  the  Course  Experience 
Questionnaire  (Ramsden,  1991)).  An  instrument  is  designed  to  measure  specific  parameters  and  therefore 
requires  validity  and  reliability  studies.  Any  survey  developed  with  LEO  would  have  to  be  rigorously  analysed 
and  validated  to  ensure  reliability  and  validity. 

However,  LEO  has  been  implemented  as  a generic  tool,  making  little  (if  any)  assumptions  about  the  objective  of 
any  particular  survey  or  the  information  the  survey  author  may  be  seeking.  No  specific  items  have  been  defined 
except  a limited  number  relating  to  the  demographic  details  of  the  respondents — those  too  have  been 
customised  in  response  to  requests  from  survey  authors.  In  summary,  LEO  is  a tool  that  provides: 

• ease  of  question  authoring  (we  believed), 

• a simple  method  of  deployment  to  potential  respondents,  and 

• a mechanism  for  the  survey  author  to  retrieve  the  data  and  export  it  to  a statistical  package  for  further 
analysis. 

In  this  paper,  we  shall  use  the  term  "lecturer"  to  refer  to  the  survey  author.  The  authors  of  this  paper  have  used 
LEO  to  formatively  evaluate  two  Web-based  interactive  learning  tools  (Kennedy  & Fritze,  1998;  Kennedy  et 
ai,  1998).  However,  the  wider  use  of  LEO  by  academics  may  also  provide  insights  into  the  uptake  of  generic 
software  tools  of  this  type  in  higher  education. 

We  hypothesised  that  the  value  of  LEO  to  the  wider  academic  community  would  be: 

• its  ease  of  authoring, 

• customisability  by  the  lecturer, 

• short  developmental  time  between  authoring  and  actual  delivery  of  the  survey,  and 

• the  return  of  any  data  generated  in  electronic  form. 

We  shall  report  on  these  factors  in  later  sections  of  this  paper.  Both  qualitative  and  quantitative  data  was 
generated  by  a LEO  survey  (using  LEO,  of  course)  and  further  quantitative  data  captured  by  the  usage  log  of 
LEO  (the  actual  content  of  each  survey  is  confidential). 

Using  LEO 

LEO  is  entirely  web-based.  The  LEO  web  site  (http://www2.meu.unimelb.edu.au/OXYGEN/LEO/),  asks 
potential  users  to  fill  in  a simple  web-based  form  with  the  user’s  name  and  email  address,  and  proceed  to  the 
authoring  environment . An  email  is  returned  to  the  lecturer  to  her  or  his  email  address  with: 

• the  assigned  survey  ID; 

• a password,  and  the  name  of  survey  author — required  to  author  the  survey  and  later  to  collect  survey  data; 

• the  uniform  resource  locator  (URL)  to  which  the  respondents  may  be  directed; 

• a URL  for  collecting  survey  data  in  table  form,  and 

• a URL  for  collecting  survey  data  in  comma  delimited  form. 
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In  addition  to  the  password  generated  (provided  automatically  by  the  LEO  system),  one  more  level  of  security 
can  be  obtained  by  using  a random  letter  user  name. 


Authoring  a LEO  survey  or  questionnaire 

Once  the  survey  author  has  entered  the  LEO  authoring  environment  with  their  password  the  lecturer  may 
choose  to  enter  the  following  information: 

• a title  for  the  survey, 

• a brief  introduction  to  describe  the  purpose  of  the  survey  for  potential  respondents, 

• the  objectives  of  the  survey,  which  will  be  available  to  the  respondents  as  a hypertext  link, 

• the  demographic  details  the  lecturer  wishes  to  collect,  and 

• the  first  survey  item. 


After  clicking  on  the  Submit  button,  the  lecturer  will  be  prompted  to  provide  subsequent  items.  The  question 
style  of  each  item  is  chosen  from  a list.  The  process  continues  until  the  Finish  button  is  clicked.  In  order  to 
facilitate  more  complex  formatting  (for  authors  who  require  it),  all  data  entry  fields  are  html  (hypertext  markup 
language)  active.  Text  with  html  tags  may  be  entered  directly  into  the  data  entry  fields  (e.g.,  <B>My 
survey</B>  would  result  in  My  survey  in  bold).  In  Figure  1 (Fig.  1)  the  basic  question  entry  template  is  shown 
with  a bipolar  Likert  style  question  as  the  selection  in  the  popup  menu  used  to  decide  the  question  type. 
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Figure  1 : Choosing  a question  template  with  LEO 


Survey  delivery 

The  LEO  system  incorporates  a database  to  store  the  data  generated  by  the  survey.  Respondents  access  a LEO 
survey  by  clicking  on  a hypertext  link  in  a web-page.  The  LEO  server  creates  the  survey  in  a window  in  the 
html  browser  of  the  respondent.  This  process  enables  lecturers  to  create  an  online  survey  without  the  need  for 
programming  skills,  knowledge  of  html,  or  server  management  experience.  The  LEO  project  web  site  is 
currently  hosted  by  the  experimental  web  server  at  MEU,  the  University  of  Melbourne  and  is  available  24  hours 
7 days  a week,  the  database  being  backed  up  daily  at  about  midnight,  Melbourne  time. 

Results  retrieval 

The  lecturer  accesses  the  results  of  a survey  in  the  form  of  a html  table  or  comma  delimited  text  file.  The 
comma  delimited  text  files  can  be  imported  to  most  statistic  packages  and/or  a spreadsheet  with  minimal 
difficulty. 
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BEST  COPY  AVAILABLE 


LEO  on  LEO 


We  have  created  a LEO  survey  on  our  project  web  site  to  evaluate  the  authoring  process  and  the  use  of  the  tool. 
The  stated  purpose  of  LEO-on-LEO  survey  is  to,  improve  the  functionality  of  LEO,  determine  the  uses  to  which 
LEO  is  being  put,  determine  the  potential  requirements  for  more  pre-defined  question  styles  for  survey  authors, 
and  improve  the  performance  aspects  of  LEO  (e.g.,  time  for  the  survey  to  load  to  the  respondent’s  browser,  or 
response  to  a survey  submission). 

Evaluation  of  LEO 

The  data  we  are  able  to  generate  from  LEO  users  are: 

• who  and  how  many  surveys  have  been  constructed  by  individual  users  (from  name  and  email  address); 

• the  number  of  respondents  who  complete  each  survey; 

• the  time  period  over  which  the  survey  operates; 

• the  number  of  surveys  that  are  constructed  as  part  of  the  learning  process  of  generating  a LEO  survey; 

• the  lead  time  between  survey  creation  and  data  collection;  and 

• feedback  from  the  LEO-on-LEO  survey. 

From  the  usage  log,  we  found  that  between  March,  1998  and  October,  1998,  LEO  has  collected  20780  data 
points  from  599  respondents  with  the  235  potential  surveys  created.  However  this  data  includes  the 
demonstration  surveys  we  did  for  academics,  and  some  test  surveys  generated  during  minor  modifications  of 
the  functionality  of  LEO.  In  fact,  the  first  author  has  created  56  surveys  and  only  two  are  in  real  use.  The 
second  author  has  created  43  surveys  with  4 in  actual  use.  One  of  the  235  surveys  is  the  LEO  on  LEO  survey 
mentioned  above. 

The  log  shows  that  there  were  75  users  other  than  the  authors.  From  the  email  addresses  supplied,  there  are  3 
users  from  outside  Australia.  Demonstrations  of  LEO  in  the  United  Kingdom,  Sweden  and  Germany  during 
1998  on  computers  with  Macintosh  and  PC  operating  systems  did  not  result  in  any  significant  time  delays  in 
comparison  to  surveys  developed  and  deployed  locally.  Of  the  75  individuals  who  investigated  LEO,  26 
individuals  created  two  or  more  surveys  (excluding  the  authors  of  this  paper).  In  our  experience  most  people 
accessing  the  LEO  site  did  so  to  have  a brief  look  at  the  tool.  At  the  time  of  writing,  only  one  lecturer  (beside 
the  authors)  has  produced  more  than  one  survey  for  active  use. 

The  time  between  the  creation  of  the  survey  and  the  first  use  of  the  survey  varied  from  2 minutes  to  63  days. 
Generally,  the  longer  the  time  delay,  the  more  likely  the  survey  was  intended  to  be  used  for  genuine  data 
collection. 

Out  of  the  235  potential  surveys,  only  30  surveys  have  more  than  2 respondents.  Table  1 shows  the  distribution 
of  respondents  to  the  number  of  surveys  that  have  that  number  of  respondents.  This  data  suggests  that 
academics  are  interested  in  a LEO-like  tool.  However,  in  many  cases  they  were  not  yet  in  the  position  to  use 
LEO.  Typically,  developing  online  courseware  requires  a significant  lead  time  and  many  academics  indicated 
they  would  be  using  LEO  in  the  future,  once  the  courseware  had  been  developed.  Indicative  of  the  process  of 
creating  an  active  survey  are  the  surveys  with  the  larger  response  rates  (58,  63,  and  1 12  respondents).  These 
surveys  required  2,  7,  and  3 attempts  respectively.  Some  of  the  surveys  are  still  generating  data  (Feb.  1999). 
The  lecturer  who  required  7 attempts  customised  the  survey  substantially  using  the  html  facility  of  the  LEO 
template. 

This  data  confirms  that  LEO  is  easy  to  learn  using  the  templates  provided,  but  customising  any  particular 
survey  (using  html)  will  require  a greater  investment  in  time.  From  initial  data  we  estimate  the  learning  curve 
to  be  about  3 to  5 attempts  in  order  to  get  a survey  into  a format  satisfactory  to  the  lecturer.  We  have  no 
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evidence  to  indicate  which  is  the  limiting  factor — authoring  using  the  LEO  templates,  or  constructing  the 
survey. 


Respondents 

Surveys 

Remarks 

2 

5 

Most  likely,  these  are  test  surveys. 

3 

2 

4 

4 

6 

2 

7 

4 

One  of  the  survey  here  is  our  "LEO  on  LEO"  survey 

11 

1 

For  surveys  with  respondents  number  in  this  range,  we  suspect  that  they  may  be  course 
evaluation  surveys  representing  about  30%  to  40%  response  rate 

15 

1 

17 

1 

26 

1 

42 

1 

44 

1 

52 

2 

58 

1 

63 

1 

112 

1 

An  survey  in  the  medical  faculty  (by  the  second  author) 

Table  1 : Number  of  respondents  and  survey  numbers 

Informal  discussions  and  feedback  from  the  LEO  on  LEO  survey  have  indicated  that  few  survey  authors  access 
the  FAQs  in  order  to  find  out  the  best  way  to  approach  the  task.  Most  seem  to  prefer  to  try  it  and  see.  From 
our  LEO  on  LEO  survey,  one  lecturer  commented  that  she  was: 

Rather  fussy  about  layout  and  (the  lecturer)  wanted  the  survey  to  look  as  good  as  a paper  one 
would  be. 

The  lead  time  (the  time  of  first  registering  for  a survey  and  the  time  the  first  response  was  collected)  ranges 
from  8 minutes  to  147  days.  Generally,  surveys  with  a short  lead  time  (in  the  order  of  minutes)  are  test  surveys. 
The  147-day  lead  time  survey  was  later  identified  (by  the  course  advisor)  to  be  a well  planned  formative 
evaluation  for  a new  web-based  course  at  The  University  of  Melbourne. 

The  LEO  on  LEO  survey  only  generated  7 respondents  (a  disappointing  result — not  unlike  the  problems  with 
paper-based  evaluations,  not  all  potential  respondents  complete  the  survey).  Two  respondents  used  LEO  for 
formative  evaluation,  one  for  a questionnaire  to  obtain  contributions  from  students  and  teachers  to  facilitate  the 
construction  of  a resource  database,  two  for  formative  and  summative  evaluation,  one  for  evaluation  of  student 
use  of  a new  anatomy  CD-ROM  and  one  as  end-of-course  evaluation.  Comments  supporting  LEO  and  the  LEO 
authoring  interface  included  the  simplicity  of  the  authoring  interface,  ease  and  speed  of  deployment  of  the 
survey,  and  ease  with  which  surveys  for  a range  of  potential  respondents  may  be  constructed.  Actual  comments 
from  the  LEO  on  LEO  survey  include: 

That  it  is  Web  deliverable,  relatively  easy  to  use  once  you've  got  the  hang  of  it  and  quick.  I 
could  never  have  got  a paper  survey  out  in  such  a short  time. 

Once  you  know  how  to  use  it,  putting  a survey  online  is  so  quick. 

It’s  easy  for  a non-computer  person  to  handle. 

The  lecturers  identified  the  following  weaknesses  of  the  current  implementation,  including: 

• a need  of  more  Likert  tags  (the  descriptors  used  to  discriminate  the  level  of  response); 

• limited  editing  capability  of  items  of  the  survey  once  submitted 

• no  choice  of  background  graphics  at  present;  and 

• the  need  to  manually  modify  the  data  collected  before  exporting  to  a spreadsheet. 
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Future  Directions 


In  seven  months  of  operation,  LEO  collected  over  20,000  pieces  of  data  for  75  users  from  a range  of  different 
surveys.  It  has  been  extremely  reliable.  We  estimate  that  LEO  has  saved  over  1 00  hours  of  transcription  time, 
postage,  and  maybe  a tree.  Lecturers  recognise  the  value  of  access  to  a tool  such  as  LEO  and  we  expect  many 
more  users  next  semester,  once  the  online  courseware  currently  under  development  are  in  use.  There  are  a 
number  of  areas  in  which  the  current  iteration  will  be  improved.  We  are  working  in  the  following  areas: 

• more  templates  variations  to  cater  to  broader  needs, 

• more  secure  means  of  delivering  the  survey, 

• faster  response  from  the  underlying  OXYGEN  engine, 

• exporting  the  survey  data  into  Excel  spreadsheet  with  built-in  frequency  counts,  barchart  generation  and 
cross-tabulation  for  preliminary  data  analysis  (almost  complete  at  the  time  of  writing). 

The  experience  with  LEO  provides  evidence  that  designing  online  tools  that  are  generic  rather  than  prescribed, 
customisable  rather  than  fixed,  and  easily  adapted  by  non-programmers,  overcomes  some  of  the  issues  of  the 
'not  invented  here  syndrome'  prevalent  in  much  software  use  (Australian  Vice  Chancellors'  Committee,  1996). 
One  of  the  most  satisfying  outcomes  our  evaluation  of  LEO  has  been  the  discovery  that  lecturers  are  using  the 
tool  in  ways  we  didn't  imagine. 
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Abstract:  This  paper  describes  the  transformation  of  a multimedia  program,  designed  to 
teach  research  skills  to  business  students,  from  one  based  on  an  ‘instructivist’  model  to  one 
underpinned  with  a constructivist  philosophy.  The  revised  program  uses  the  theory  of  situated 
learning  as  a framework  for  the  instructional  design,  and  introduces  elements  such  as 
authentic  context,  an  authentic  activity,  collaboration,  and  opportunities  for  articulation  and 
reflection,  into  the  learning  environment. 


Introduction 

Over  the  past  decade  or  more,  there  has  been  a substantial  theoretical  shift  from  a ‘behavioral  to  cognitive  to 
constructivist’  learning  perspectives  amongst  educators  (Ertmer  & Newby,  1993,  p.  50;  Jonassen,  1991;  von 
Glasersfeld,  1995).  Little  credence  is  now  given  to  learning  theories  that  propose  that  learning  is  no  more  than 
the  transmission  of  a body  of  knowledge  from  teacher  to  student.  No  matter  how  much  teaching  methods  in  the 
classroom  have  changed,  the  theoretical  foundations  of  interactive  multimedia  programs  are  frequently  found  to 
be  based  on  behavioral  traditions  inherited  from  educational  technology.  As  Jonassen  puts  it:  ‘The  roots  of 
behaviorism  extend  deeply  into  1ST  [instructional  systems  technology]  practice’  (Jonassen,  1991,  p.  6).  It  is 
possible,  even  today,  to  find  many  examples  of  multimedia  learning  environments  which  use  the  same 
instructional  design  as  the  early  programmed  instruction  texts  of  the  50s. 

It  is  one  such  multimedia  program  which  we  describe  in  this  paper — a program  firmly  in  the  behaviourist 
mould,  which  attempted  to  take  the  subject  of  research  methodology  and  transmit  that  knowledge  in  a linear 
fashion  through  eight  modules  and  26  lessons.  The  program  was  based  on  a pedagogical  philosophy  described 
by  Reeves  and  others  (Reeves,  1993;  Duffy  and  Jonassen,  1991)  as  ‘instructivist’  where  little  emphasis  is 
placed  on  the  learner  ‘who  is  the  passive  recipient  of  instruction’  (Reeves,  1993,  p.  4).  The  program  had  been 
under  construction  for  about  two  years,  and  had  already  swallowed  up  a substantial  amount  of  funding.  The 
Faculty  was  committed  to  the  program’s  completion.  But  we  both  envisaged  a far  better  way  to  create  a 
learning  environment  to  teach  research  methodology.  We  planned  to  design  a constructivist  shell  to  provide  a 
meaning  to  the  lessons  that  comprised  the  original  program. 

Some  educators,  such  as  Squires  (1996)  have  spoken  of  programs  designed  from  a constructivist  philosophy 
being  used  in  very  non -constructivist  settings.  Young,  Nastasi  and  Braunhardt  (1996)  relate  their  experience  of 
implementing  ‘a  constructivist  design  in  a constructivist  manner’  (p.  121).  Clearly,  the  software  itself  is  but  one 
aspect  of  an  interrelating  group  of  influences  which  may  determine  whether  learning  is  successful.  Our  plan 
was  to  incorporate  a ‘non-constructivist  design’  into  a learning  environment  based  upon  a constructivist 
philosophy. 
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An  Instructivist  Learning  Tool 


The  original  program  comprised  a multimedia  package  for  teaching  statistics  and  research  methods  in  a 
business  degree,  traditionally  taught  by  lectures  and  tutorials.  As  in  many  similar  courses,  mastering  the  theory 
took  most  of  the  time,  leaving  little  room  to  explain  its  application  and  the  practicalities  of  using  it  in  business. 
A first  attempt  to  improve  the  course  recognized  the  power  of  multimedia  to  present  technical  material  in  a 
self-paced  format  (Figure  1),  using  multiple  choice  or  other  closed  format  exercises  to  help  students  test  their 
understanding  at  each  stage.  This  approach  was  based  on  the  typical  transmission  model  of  learning,  made 
interesting  with  multimedia  features  like  graphics,  animations,  and  interactive  exercises.  However,  unlike  a 
lecture,  students  could  also  repeat  sections  of  the  course  until  the  concepts  were  mastered. 


Introduction  To  Inferential  Statistics 


^Hypothesis  Tests 

~ • eWhile  a confidence  interval  predicts  the  likeiy 
;range  in  which  p faiis,  a hypothesis  test  is  about  a 
i-specific  value  of  the  population  mean  which  is 
critical  to  a theory  or  a practical  decision.  A 
,^j|researcher  would  get  a hypothesis  about  the  value 
Jpfp  from  previous  research.  fromthe'ory  or  even 
Troma'gu  estimate'. 


■^For  example,  suppose  that  in  another  country  the 
{mean  height  of  drivers  has  been  found  to  be 
<:l  85cm.  You  can  use  a car  design  from  that  country 
ifthis  Is  the  correct  height  for  your  population.  You 
take  a random  sample  of  drivers  and  find  a mean 
80cm.  But  could  this  sample  actually  come 
^sfrom  a population  with  a mean  of  1 85cm? 


Figure  1:  Instruction  screen 

The  resulting  ‘electronic  textbook’  had  some  advantages  over  traditional  lectures  and  tutorials,  but  pilot  testing 
showed  it  did  not  really  solve  a major  problem  of  the  course:  that  the  technical  material  was  boring  when 
studied  in  isolation  from  its  application.  Students  lacking  work  experience  could  not  make  the  connection,  and 
business  students  are  renown  for  their  resistance  to  teaching  that  is  not  apparently  relevant.  The  attractions  of 
self-pacing,  constant  feedback  and  animations  were  acknowledged,  but  the  trial  revealed  no  improvement  in 
students’  motivation  to  work  through  the  many  steps  needed  to  understand  the  material.  In  hindsight,  self- 
pacing  and  constant  feedback  can  be  seen  as  reducing  the  quality  of  teaching:  students  could  not  turn  off  when 
the  material  was  difficult,  nor  use  their  considerable  skills  at  answering  tutorial  questions  with  very  partial 
knowledge.  A solution  designed  to  increase  learner  control  worked  on  one  level,  but  had  the  reverse  effect  on 
another  and  was  actually  judged  worse  or  no  better  than  traditional  methods.  The  following  sections  describe 
the  philosophy  we  adopted  and  how  we  designed  a more  constructivist  learning  environment  to  incorporate  this 
original  program. 


169 


BESTCOPY  AVAILABLE 


Situated  Learning 


In  designing  our  ‘constructivist  shell’,  we  wanted  to  provide  a real-life  context  and  meaning  to  the  learning  that 
the  students  were  required  to  do  as  they  worked  with  the  program.  In  so  doing,  we  drew  heavily  on  situated 
learning  theory  (Brown,  Collins,  & Duguid,  1989;  Lave  & Wenger,  1991;  McLellan,  1996)  which  emphasises 
the  notion  that  the  learning  process  cannot  be  divorced  from  the  context  of  the  problem,  and  that  the  learner 
will  use  information  and  clues  from  the  situation  rather  than  apply  formally  taught  problem-solving  techniques. 
Resnick  (1987)  contends  that  school  and  university  learning  is  fundamentally  different  to  everyday,  practical 
learning  in  that  it  provides  too  little  engagement  with  genuine  situations,  and  too  much  emphasis  on  theoretical 
perspectives.  Above  all,  we  wanted  the  learning  environment  to  more  closely  resemble  the  way  learning  occurs 
in  real  life  than  it  does  in  formal  school  and  university  settings. 

From  the  literature  on  situated  learning,  it  was  possible  to  produce  a list  of  nine  critical  criteria  (Herrington  & 
Oliver,  1995)  that  could  be  used  to  guide  the  design  and  development  of  the  interactive  multimedia  which  was 
to  comprise  the  constructivist  shell.  The  program  needed  to  provide: 

1.  Authentic  context  that  reflects  the  way  the  knowledge  will  be  used  in  real  life 

The  program  needed  to  be  able  to  encompass  a physical  environment  which  reflected  the  way  the  knowledge 
would  ultimately  be  used,  and  a large  number  of  resources  to  enable  sustained  examination  from  a number  of 
different  perspectives  (Brown  et  al.,  1989;  Honebein,  Duffy,  & Fishman,  1993).  The  context  chosen  was  a 
large  research  company,  with  all  its  resources  and  infrastructure,  where  the  students  gain  temporary 
employment. 

2.  Authentic  activities 

The  learning  environment  needed  to  provide  ill-defined  activities  that  have  real-world  relevance,  and  which 
present  a single  complex  task  to  be  completed  over  a sustained  period  of  time  (Bransford,  Vye,  Kinzer,  & 
Risko,  1990;  Brown  et  al.,  1989).  The  authentic  activities  in  the  program  include  three  research  ‘jobs’  on  the 
students’  desk  when  they  start  work,  each  of  which  require  2-4  weeks  of  a semester  to  complete. 

3.  Access  to  expert  performances  and  the  modelling  of  processes 

In  order  for  the  learning  environment  to  provide  expert  performances,  the  program  needed  to  provide  access  to 
expert  thinking  and  the  modelling  of  processes  (Brown  et  al.,  1989;  Collins,  Brown,  & Newman,  1989;  Lave  & 
Wenger,  1991).  The  program  provides  examples  of  completed  research  projects  to  enable  students  to  examine 
professional  research  reports. 

4.  Multiple  roles  and  perspectives 

In  order  for  students  to  be  able  to  investigate  the  learning  environment  from  more  than  a single  perspective,  the 
program  needed  to  provide  different  perspectives  on  the  topics  from  various  points  of  view  (Collins  et  al., 
1989;  Honebein  et  al.,  1993;  Spiro,  Feltovich,  Jacobson,  & Coulson,  1991).  In  each  of  the  research  scenarios, 
the  students  need  to  interview  a number  of  different  employees  within  organisations  to  obtain  information,  and 
to  consult  a variety  of  other  resources. 

5.  Reflection 

In  order  to  provide  opportunities  for  students  to  reflect  on  their  learning,  the  program  needed  to  provide  both  an 
authentic  context  and  non-linear  navigation  to  enable  them  to  return  to  any  element  of  the  program  (Boud, 
Keogh,  & Walker,  1985;  Collins  & Brown,  1988;  Kemmis,  1985).  Unlike  the  original  research  methodology 
program,  which  followed  a linear  form,  the  new  shell  enabled  students  to  navigate  at  will  within  the  resource. 

6.  Collaborative  construction  of  knowledge 

The  learning  environment  needed  to  provide  the  opportunity  for  students  to  collaborate,  and  while  this  could 
not  be  incorporated  within  the  software  itself,  it  is  recommended  that  students  work  on  the  program  in  small 
groups  to  enable  them  to  problem-solve  together  (Brown  et  al.,  1989;  Collins  et  al.,  1989;  Hooper,  1992). 

7.  Articulation 

The  learning  environment  needed  to  ensure  that  students  were  free  to  discuss  the  task  as  they  use  the  program 
to  enable  them  to  learn  to  speak  the  language  of  the  discipline  and  community  of  practice  (Collins  et  al.,  1989; 


Lave  & Wenger,  1991).  Unlike  the  original  program  which  was  designed  for  individual  students  working 
silently  on  their  own,  the  new  learning  environment  encouraged  the  students  to  articulate  their  growing 
understanding  of  the  research  methodology  with  their  partners. 

8.  Coaching  and  scaffolding 

The  learning  environment  needed  to  ensure  that  the  teacher  was  available  to  provide  required  coaching  and 
scaffolding  for  students  as  they  used  the  program  (Collins  et  al.,  1989;  Greenfield,  1984). 

9.  Authentic  assessment 

In  order  to  provide  assessment  of  student  learning  compatible  with  the  situated  learning  model,  the  program 
needed  to  provide  assessment  that  was  seamlessly  integrated  with  the  activity  (Linn,  Baker,  & Dunbar,  1991; 
Reeves  & Okey,  1996;  Wiggins,  1993).  Students  are  assessed  on  the  research  report  that  is  the  purpose  of  their 
work  at  the  research  agency,  rather  than  on  the  separate  quizzes  and  tests  of  the  original  program. 


A Functional  Learning  Tool 


While  this  constructivist  style  of  multimedia  is  not  as  simple  to  create  as  the  instructivist  electronic  textbook,  it 
has  many  advantages:  it  shows  the  relevance  of  theory  to  application;  learning  of  theory  is  driven  by  the  need 
to  use  it,  rather  than  an  artificially  imposed  pedagogical  framework;  and  field  experiences  closer  to  professional 
realities  than  simplified  student  projects  can  be  simulated.  The  context  is  modeled  on  the  experience  of  a 
student  employed  as  an  apprentice  in  a summer  job  with  Acumen  Research  to  undertake  research  for  a client,  a 
large  bank.  In  their  office  at  Acumen  (Figure  2)  are  various  resources,  including  information  on  office 
procedures  (principally  on  navigating  in  the  simulation)  and  a folder  containing  information  on  the  project. 


Figure  2:  The  office  at  Acumen 
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The  books  on  research  and  statistics  are  the  electronic  texts  described  earlier:  students  do  not  now  have  to  use 
them  except  as  they  need  to  supplement  learning  from  other  sources  to  complete  the  project.  Two  general 
resources  available  throughout  the  simulation  are  a phone  that  provides  context  sensitive  help  (e.g.,  hints  on 
what  to  do  next),  and  a notepad  primarily  for  taking  notes  or  storing  information.  No  instructions  on  using  the 
program  are  needed  as  navigation  involves  only  clicking  on  objects  or  people  to  perform  activities  as  they 
would  be  done  in  the  real  situation.  After  familiarizing  themselves  with  the  resources,  students  are  directed  to 
the  client’s  office.  There  they  meet  the  manager  commissioning  the  research  and  have  a preliminary 
‘discussion’  on  the  nature  of  the  problem  as  he  sees  it.  This  involves  a statement  by  the  manager,  followed  by 
the  opportunity  to  choose  questions  from  a list  in  the  notebook  (the  manager’s  statement  and  responses  are 
video  clips).  Students  are  expected  to  choose  questions  with  some  thought  to  the  situation:  there  is  not  time  to 
answer  all  questions,  and  some  are  largely  irrelevant  to  the  problem.  Students  work  in  pairs,  and  are 
encouraged  to  discuss  their  choices. 

The  manager  subsequently  invites  the  student  to  discuss  the  project  with  a senior  staff  They  learn  in  these 
discussions  that  the  problem  has  other  angles  and  must  be  reframed  to  meet  the  needs  of  the  various 
organizational  members.  Apprentices  are  told  beforehand  that,  as  in  the  real  world,  there  may  be  no  one  right 
way  of  construing  the  problem,  but  that  some  views  may  be  more  politically  important  than  others.  These  are 
often  unexpected  and  interesting  issues  to  students,  and  not  readily  available  in  the  traditional  teaching 
approach.  After  constructing  their  version  of  the  problem,  students  design  a questionnaire  by  selecting 
questions  from  a large  bank  of  items  used  in  previous  surveys.  Next,  they  choose  the  sampling  design  from  a 
set  of  alternatives;  tradeoffs  between  scientific  accuracy  and  costs  invite  students  to  see  theoretical  principles  in 
the  context  of  real  world  constraints.  On  finishing  the  research  design  tasks,  students  receive  their  data,  which 
has  been  ‘collected’  by  a group  of  telephone  surveyors  employed  by  the  research  agency.  Finally,  the  data  are 
downloaded,  analyzed  in  a standard  statistics  package,  and  written  up.  The  report  can  be  evaluated  by  the 
lecturer,  authentically,  in  the  same  way  its  real-world  counterpart  would  be. 


Behavioral  to  Constructivist 

In  designing  a constructivist  shell  for  a behavioral  interactive  multimedia  program,  we  deliberately  sought  to 
adopt  an  approach  which  saw  learning  as  an  active  process  rather  than  the  result  of  a transmission  of 
knowledge  from  program  to  student.  The  tasks  that  were  designed  for  the  program  were  global,  complex  and 
sustained,  rather  than  clearly  defined  tasks  and  subtasks,  neatly  broken  up  into  lessons  and  modules.  These 
tasks  were  placed  within  a full  and  authentic  context  rather  than  fragmented  tasks  and  pre-determined 
instructional  sequences.  The  new  program  required  and  encouraged  students  to  explore  the  learning 
environment,  rather  than  be  captive  to  the  presentation  of  a number  of  linear  lessons.  It  presented  a complexity 
which  required  more  reflective  thought  than  the  step-by  step  lessons  of  the  original  program.  Students  were 
required  to  select  relevant  data  and  material  from  a wealth  of  sources,  rather  than  try  to  absorb  a pre- 
determined and  well-defined  body  of  knowledge.  In  so  doing,  they  were  required  to  reflect  about  their  path 
rather  than  progress  automatically  through  it.  The  interface  of  the  new  program  used  an  ecological  approach 
(Pejtersen,  1 993)  which  presented  real  world  metaphors  and  objects  rather  than  buttons  as  navigational  devices. 
Students  would  need  to  associate  the  meaning  of  an  object  with  its  destination  rather  than  click  upon  the 
ubiquitous  forward  and  back  arrows  of  the  original  program. 

In  its  implementation,  the  learning  environment  moved  from  a purely  individual  and  solitary  pursuit  to  a 
collaborative  one,  supported  by  a teacher  who  could  provide  coaching  and  scaffolding  at  appropriate  times. 
Assessment  of  learning  became  integrated,  authentic  and  inseparable  from  the  activities  themselves,  as  opposed 
to  the  separate  tests  and  quizzes  designed  to  assess  decontextualised  packets  of  learning. 

In  the  real  world,  unlike  the  classroom,  practitioners  actively  construct  their  understanding  of  a problem,  and 
design  a sequence  of  problem-solving  steps  based  on  both  the  textbook  principles  and  contextual  factors.  The 
program  described  here  provides  an  understanding  of  how  the  theory  behind  research  functions  in  the  face  of 
ambiguous  and  contradictory  information,  practical  limits  on  time  and  budget,  and  social  agendas.  In  so  doing, 
it  provides  students  with  a learning  environment  no  longer  based  on  the  instructivist  models  of  the  50s,  but  one 
firmly  grounded  in  recent  theory  and  research. 
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Abstract:  The  need  for  a higher  integration  of  training  and  work  is  becoming  increasingly 
apparent  in  countries  such  as  Greece  where  the  costs  for  higher  education  have  lead  to  a 
permanent  development  process  of  the  educational  culture.  In  this  paper  we  describe  our 
ongoing  work  on  an  educational  system  which  properly  exploits  the  recent  advances  in 
information  & communication  technologies  and  addresses  efficiently  the  above  need.  This 
effort  aims  to  the  development  of  the  necessary  infrastructure  and  services  in  order  to 
establish  a complete  “Virtual  Education  System”  offering  tele- training  services  based  on 
WWW  technology. 


Introduction 

Internet  has  became  an  integrated  set  of  inter-networking  resources  and  services  based  on  open,  de 
facto  standards  and  offered  by  an  array  of  competing  providers  in  a commercial  environment  now  exhibiting 
many  of  the  features  of  a commodity  market.  The  World  Wide  Web  (Web)  and  its  attendant  browsers,  with 
their  origins  also  in  the  research  and  academic  communities,  catapulted  the  Internet  to  its  current  revolutionary 
status  as  an  educational  phenomenon  [Bertrand  et  al.  1995]. 

Today,  training  in  Greece  is  characterised  by  classroom  seminars  where  computer  technology 
(multimedia  systems)  is  used  occasionally.  But  the  latest  progress  in  the  field  of  computer  networks  has  not 
been  taken  into  account  yet.  Many  educational  institutions  own  World  Wide  Web  servers  thus  providing 
information  on  their  interest  areas,  their  achievements  and  goals.  In  the  Internet,  a large  number  of  digital 
libraries  containing  a wide  amount  of  knowledge  are  able  to  provide  information  but  they  are  not  oriented  for 
on-line  training.  Moreover,  in  the  last  few  years,  Greek  universities  offer  off-line  tele-training  via  the  Web. 

The  Virtual  Education  System  described  in  this  paper  is  an  on-line,  web  based  educational  platform 
which  aims  to  enable  the  user  to  exploit  the  capabilities  and  tools  offered  today  due  to  the  progress  of  computer 
science,  in  order  to: 

• access  multimedia  information  about  relevant  educational  subjects 

• communicate  with  other  users  sharing  the  same  interests  and  exchange  ideas  end  knowledge 

• develop  new  skills 

• keep  in  touch  with  the  recent  advances  that  affect  working  methods 

The  system  addresses  the  need  of  restructuring  traditional  vocational  training  concepts  in  order  to  support 
current  educational  requirements  such  as  effective  integration  of  work  & training,  flexibility,  in  place-training 
needs  and  wide  exchange  of  experience  and  problem  solving  methodologies. 

In  this  work  we  present  the  overall  architecture  of  the  system  which  currently  exists  as  a prototype. 
The  paper  is  organised  as  follows:  section  2 describes  the  main  objectives  of  the  system.  Section  3 presents 
certain  issues  that  are  posed  when  developing  distance  education  systems  over  the  Internet.  Section  4 presents 
the  general  architecture  and  the  technological  aspects  of  the  system  and  finally,  section  5 discusses  future 
directions. 


Main  Objectives  of  the  Proposed  System 
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The  educational  concept  of  the  system  is  characterised  by  the  transformation  of  theory  into  practice. 
The  course  modules  are  separate  entities,  and  play  the  role  of  plug-ins  to  the  system.  The  Virtual  Education 
System  plays  the  role  of  a customised  borwser  and  collaboration  tool,  which  fully  facilitate  the  Greek  language 
providing  a simple  and  friendly  user  interface  [Tritsch  1995].  This  feature  allows  even  novice  users  to  operate 
complex  services  (like  video  conferencing)  with  little  effort. 

Each  course  module  deals  with  a specific  subject  (e.g.  computer  science  or  algebra)  and  is  constructed 
using  special  WYSIWYG  authoring  tools.  This  way,  the  training  material  is  system  independent  and  every 
educational  author  can  construct  his  own  courses.  The  only  limitation  is  that  certain  lesson  architecture  rules 
should  be  followed  in  order  for  the  module  under  construction  to  be  compatible/readble  by  the  system.  Each 
subject  is  not  taught  point  by  point  but  as  an  integral  whole  and  through  action  oriented  methods.  This  serves  to 
promote  the  process  of  “natural  learning”  working  in  teams  and  projects  and  is  oriented  on  future  personnel 
development  conceptions  [Goldberg  1996]. 

The  users  can  communicate  over  virtual  education  centres  with  tutors,  experts  as  well  as  other  users 
(see  figure  1).  The  design  of  these  virtual  systems  has  the  aim  to  link  all  functions  under  a uniform  surface.  All 
functions  can  be  used  single  or  combined  which  means  that  they  can  be  used  for: 

^ communication 

> learning 

> any  combination  of  the  above 


Figure  1.  The  topology  of  the  system. 

The  Virtual  Educational  System  will  use  all  recent  advances  in  the  fields  of  information  & 

communication  technologies,  in  order  to  achieve: 

• Integration  of  working  and  training.  The  process  of  learning  will  change  in  accordance  to  the  developments 
of  work  processes.  Working  and  learning  will  be  more  and  more  linked  together  so  that  the  education  and 
working  processes  of  the  future  will  show  an  increased  degree  of  integration. 

• Realistic  training  for  the  users.  Being  connected  with  modem  working  environments,  they  will  have  the 
opportunity  to  be  trained  according  to  the  latest  demands  of  the  market. 

• Lifelong  and  autonomous  learning:  The  user  has  the  capability  to  learn  at  home  and  has  the  opportunity  to 
communicate  with  tutors,  other  users  or  experts. 

• Flexibility  with  regard  to  time,  place  and  intensity  of  learning:  The  user  has  the  choice  of  place,  intensity 
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and  duration  of  study. 

• High  training  level  with  the  use  of  new  technologies  like  the  integration  of  multimedia  technology  systems 
(offering  images,  sound  and  video)  and  computer  networks  which  allow  the  user  to  access  information  from 
digital  libraries  of  other  institutions  (open  and  distance  learning  and  computer  based  training). 

• The  exchange  of  experiences,  knowledge  and  ideas  between  people  separated  by  large  distances. 


3.  Issues  for  Distance  Education  Applications  that  use  the  Internet 

During  the  design  period  of  the  system  we  faced  several  topics  that  needed  to  be  addressed  before 
being  able  to  offer  a complete  Virtual  Education  System  over  the  Internet.  These  range  from  technical 
considerations  to  sociological  aspects  of  learning  without  the  existence  of  a physical  classroom.  Much  study  has 
already  been  done  in  distance  education.  While  distance  education  can  provide  efficient  utilisation  of  scarse 
resources,  more  fair  distribution  of  knowledge,  and  timely  dissemination  of  new  knowledge,  limitations  in  its 
effective  use  exist.  Student  motivation  is  critical.  On-line  instructors  may  need  to  take  more  time  for  the 
electronic  courses.  Limits  to  the  number  of  students  may  be  required  to  maintain  quality  interaction.  While 
generalisations  in  the  effectiveness  of  distance  education  are  dangerous,  a common  set  of  advantages  and 
disadvantages  are  emerging  [Rossman  1992]. 

The  web  solves  some  problems  of  traditional  distance  education  and  brings  new  issues  to  the  forefront. 
During  our  work  we  adressed  the  following  issues: 

> Target  Audience.  With  traditional  university  courses,  attendance  is  limited  by  the  physical  size  of  the 
training  facility  (about  50  students).  Normally  this  means  that  the  attendees  are  users  and  potential  users  of 
our  facilities  -researchers  whose  backgrounds,  computing  experiences,  and  expectations  are  somewhat 
similar.  With  a Virtual  Education  System,  the  size  of  each  class  will  be  limited  by  other  non  physical 
factors  such  as  staffing  requirements  for  account  administration,  limited  consulting  support  and  CPU 
resources  required  for  the  provision  of  on-line  services.  While  we  may  now  be  able  to  directly  impact  a 
broader  group  of  people,  this  group  will  be  likely  much  more  diverse  in  its  background  and  computer 
experience.  Having  in  mind  the  above  problems  the  following  issues  are  posed: 

• the  policy  on  the  individual  computer  accounts 

• the  way  that  novice  users  will  be  supported  by  e-mail  services  and  newsletters 

• the  security  policy  that  must  be  followed  since  the  application  should  be  protected  from  outside 
intruders. 

> Prerequisites.  The  use  of  the  system  will  require  some  basic  knowledge  of  Windows,  the  main  operating 
system  used  by  the  majority  of  Greek  users.  With  traditional  workshops  held  in  training  facilities  the  type 
of  computer  hardware,  software  and  network  connection  is  known.  Although  many  of  these  parameters 
(such  as  network  speed)  are  known  in  a University  environment,  the  users  coming  from  outside  (e.g. 
logging  in  from  their  homes)  have  unknown  properties.  This  situation  could  cause  a number  of  problems. 
For  example,  some  users  may  find  that  limited  network  bandwidth  (a  common  problem  in  Greek  public 
networks)  does  not  provide  a satisfactory  learning  environment.  A simple  solution  would  be  to  decrease  the 
volume  of  the  material  (e.g.  limit  the  use  of  multimedia)  but  this  would  mean  reduced  quality  of  the 
educational  process.  Since  our  main  purpose  was  to  have  a fully  interactive  and  media  rich  system,  the  use 
of  multimedia  objects  was  extensive  even  though  some  users  would  be  eventually  excluded  from  the 
learning  process. 

> Learning  Environment.  Extensive  feedback  from  attendees  at  traditional  workshops  indicated  that  there 
were  two  benefits  to  learning  in  a classroom  setting  which  were  independent  of  the  material  covered  in  the 
course.  First,  students  find  the  ability  to  get  away  form  their  other  responsibilities  and  concentrate  on 
learning  the  course  material  greatly  increases  the  amount  that  they  learn  and  retain.  Second,  the  interaction 
with  other  students  and  instructors,  whether  this  was  side  discussion  or  simply  listening  to  questions  asked 
by  others,  often  resulted  in  rethinking  of  their  own  techniques  and  suggest  new  solutions  to  their  problems. 
Our  system  should  have  the  same  attributes  in  order  to  ensure  that  students  find  the  experience  worthwhile 
and  complete  the  course.  This  need  can  be  satisfied  only  when  the  learning  environment  provides  advanced 
interaction  between  instructors  and  students  with  the  ability  of  whole  classrooms  to  participate  in  questions 
and  answers. 

Technological  Aspects  of  the  System 
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The  project  aims  to  develop  tools  in  order  to  combine  the  classical  education  methods  into  a complex, 
but  efficient,  system  based  on  new  technological  achievements  in  the  field  of  information  & communication 
technologies.  The  technical  basis  of  this  educational  system  is  formed  by  a virtual  education  centre  that 
provides  interactive  educational  courses  over  a data  network.  Precondition  for  the  virtual  education  centre  is  an 
integrated  software  environment  which  makes  possible  that  several  users,  tutors  and  experts  use  all  different 
capabilities  of  the  educational  system  at  the  same  time  [Repenning  1993], 

New  technologies  are  contributing  to  major  transformations  in  the  way  people  live,  learn,  work  and 
interact  around  the  globe.  Important  new  technologies  such  as  networking  / internetworking,  tele- 
communication, presentation  / authoring,  collaboration,  visualisation/imagery  analysis,  conferencing  and 
multimedia  are  getting  more  closely  related  to  daily  life.  Consequently,  a phenomenal  alteration  is  changing  the 
environments  surrounding  as  such  as  virtual  workplace,  virtual  classroom,  virtual  library,  virtual  school  and 
virtual  community. 

The  basis  for  the  implementation  of  the  system  will  be  an  integration  of  existing  technologies 
(multimedia  systems,  computer  networks).  Images,  text,  sound  and  video  will  be  sent  via  the  network  to  the 
users.  But  the  development  of  the  system  requires  more  that  a simple  integration  of  existing  technologies.  The 
training  has  to  be  «on-line».  This  demand  sets  new  standards  for  the  system,  creates  research  activities  and  tasks 
to  be  done  and  produces  challenging  demands  for  the  implementation  [Brunner  1995],  The  services  to  be 
developed  will  be  presented  to  the  user  through  a friendly  user  interface  [EU  1996],  From  a technical  point  of 
view,  these  services  can  be  classified  into  the  following  categories: 

Basic  services 

• Teaching  materials  (course  modules)  can  be  sent  to  the  users  by  e-mail.  The  updates  would  be  sent  over  the 
Internet 

• Electronic-Mail-Service\  Among  other  things  the  users  will  be  able  to  receive,  transmit  and  draw  messages 
and  pass  them  to  other  users  or  groups.  In  addition  to  modify  the  teaching  materials,  teachers  may  store 
messages  in  the  server. 

• Notice  Board:  An  electronic  notice  board  its  also  offered  on  which  the  user  can  write  messages  and  each  of 
the  participants  can  read  them.  This  function  allows  an  exchange  of  information  and  questions  between  the 
participants. 

• Electronic  Questionnaire:  A list  of  questions  to  be  answered  by  the  users  electronically  will  give  important 
response  for  the  tutor. 

• Question- Answer-Service:  With  the  electronic  questionnaires  the  users  can  ask  questions  in  multiple  choice, 
which  can  be  evaluated  automatically  by  a process  which  calculate  the  score,  based  on  the  given  answers. 
This  service  will  help  the  user  to  control  the  learning  process. 

Advanced  services 

• On-line  communication:  This  system  will  make  possible  a visual  and  acoustic  communication  between  the 
students  or  between  the  tutor  and  remotely  sited  experts,  by  using  an  integrated  camera  in  the  PC  and  a 
microphone.  This  will  activate  tele-coaching  and  a virtual  discussion  in  the  teaching  groups. 

• Screen  Sharing:  The  screens  of  the  students  and  tutors  or  experts  will  be  coupled.  There  will  also  be  an 
acoustic  connection  via  a telephone  line.  The  tutor  may  put  his  screen  on  the  monitor  of  the  student  and  vice 
versa.  So  it  is  possible  to  go  through  an  error  analysis  or  to  demonstrate  alternative  solutions,  online  and  in  a 
participative  manner. 

• Catalogue  Service:  There  will  be  a catalogue  service  in  the  server  with  a digital  library  and  extensive  search 
possibilities. 

• Information  Service:  At  any  time  the  user  can  achieve  world  wide  thematic  topic  information  with  menu 
controlled  links  to  external  services  and  information  sources. 

• Application  Service:  With  this  application  several  students  will  have  access  to  shared  application  programs, 
at  the  same  time.  This  system  is  a necessary  precondition  for  virtual  partners,  teamwork  and  for  tutorial 
support. 

The  main  technological  challenge  that  we  face  during  the  development  period  of  the  system  is  the  use 
of  multimedia  over  the  Internet  [Encamacao  1993],  Different  network  based  multimedia  platforms  provide 
different  types  of  audio  devices  and  services.  Nevertheless,  all  of  them  offer  a common  set  of  functionalities, 
such  as  record,  store  and  playback.  Online  audio  exchange  for  communication  purposes  was  realised  based 
upon  these  basic  functionalities.  In  order  to  provide  cross-platform  usability,  a common,  intermediate  exchange 
format  was  defined  and  a number  of  online  bi-directional  converters  supporting  different  platform-dependent 
audio  formats  were  implemented.  Video  communication  for  learning  environments  requires  a fast  codec 
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algorithm  that  can  be  performed  in  real-time  on  either  low-end  hardware  or  by  software.  Since  the  real-time 
issues  for  video  as  a communication  channel  were  more  important  than  high  quality  pixel  and  color  resolution 
we  decided  to  use  the  JPEG  (Joint  Photographic  Expert  Group)  codec  standard  for  our  video  sequences,  referred 
to  as  M-JPEG  (Motion-JPEG).  Within  the  system,  a multipoint  audio/video  communication  environment  allows 
real-time  discussions  between  groups  of  up  to  six  teachers  and  students.  Its  control  elements,  are  embedded  in 
the  GLS  application.  Within  the  WWW,  online  communication  is  supported  by  using  the  M-Bone  technology 
which  is  driven  by  the  same  idea  of  having  people  communicate  in  a more  natural  way. 


Conclusions 

In  this  paper  we  described  our  on  going  work  on  a Virtual  Education  System  which  enables  on-line  training 
courses  in  a collaborative  manner  using  state  of  the  art  technologies.  It  enables  a high  level  of  instructor-student 
interaction  using  network,  database  and  multimedia  technology.  It  is  designed  by  applying  strict  pedagogical 
principles  so  as  to  achieve  the  greater  acceptance  as  an  educational  tool.  We  believe  that  the  aspiration  of 
Virtual  Educational  Systems  will  be  further  approached  with  this  effort. 
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Abstract:  In  this  paper  the  development  of  structural  learning  environments  for  use  over  the 
Internet  is  discussed.  Many  educational  programs  are  designed  by  specialist  designers  in 
consultation  with  educators.  This  can  often  be  inconvenient,  time-consuming  and  costly.  The 
implications  of  this  system  will  be  discussed  and  a new  template-based  model  proposed.  With 
this  new  model  the  educator  can  be  given  more  control  over  course  structure,  allowed  easier 
modification  of  course  material  and  have  the  option  to  embed  teaching  and  learning  strategies 
within  the  program.  Ultimately,  templates  will  be  produced  by  multimedia  designers  and  the 
educators,  who  use  the  templates  to  develop  their  educational  programs,  will  then  be  able  to 
work  independently  and  at  their  own  time,  location  and  pace. 


Introduction 


Since  automated  teaching  machines  were  first  developed  by  Sidney  Pressey  in  the  1920's  (Sloffer, 

1995) ,  the  development  of  educational  programs  has  steadily  advanced.  The  progress  followed  on  from  those 
early  teaching  machines,  Eastman’s  educational  films  of  the  late  1920's  (McReynolds,  1995),  military  training 
films  (Shlechter,  1991),  computer-based  instruction  (Reeves,  1991),  interactive  multimedia  (Jones,  1998)  to  the 
current  and  wide-spread  use  of  the  Internet  and  the  World  Wide  Web  for  educational  purposes  (McManus, 

1996) .  Increasingly,  the  Internet  is  being  used  as  a vehicle  for  delivery  of  educational  material.  This  can  be  of 
particular  relevance  in  the  field  of  distance  education  (Carswell,  1997).  Today  it  is  possible  for  a student  to 
complete  an  entire  degree  course  through  this  medium. 

Currently,  design  of  Internet-based  educational  programs  is  often  produced  by  a "project  team",  which 
consists  of  multimedia  designers,  a coordinator  and  the  educator(s).  The  project  team  will  use  a set  framework  as 
the  basis  for  design,  producing  a number  of  pages  with  common  attributes.  A good  layout  will  incorporate 
consistent  design  features  which  include: 

• buttons  and  entry/exit  points  which  retain  page  position 

• constant  background  pattern  or  colour 

• uniform  menu  items 

In  this  way  the  program  can  retain  a user-friendly  quality.  The  user  has  fewer  navigational  problems 
and  is  not  required  to  learn  new  navigational  skills  for  each  page,  section,  lesson  or  subject. 

In  this  paper,  the  focus  is  on  the  design  of  templates  for  Internet-based  educational  programs  which 
allow  for  the  incorporation  of  teaching  and  learning  strategies,  often  overlooked  by  more  conventional  methods. 
A new  template  model,  which  applies  the  ideas  put  forward  in  this  paper,  is  presented  and  discussed. 


Templates  in  Internet-based  Educational  Programs 

Templates  have  been  used  across  a broad  spectrum  of  applications  and  fields  of  study  for  many  years.  A 
template  is  basically  a model  of  some  type,  which  allows  the  user  to  add  individual  design  features.  Many  people 
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will  be  familiar  with  the  plastic  templates  used  to  prepare  programming  flowcharts,  chemical  diagrams  and 
geographical  maps.  More  recently,  word-processing  templates  have  emerged.  These  templates  provide  a pre- 
delineated area  to  be  filled  out  in  order  to  complete  a document.  Apart  from  computer-based  templates,  there  are 
many  other  types  made  of  a variety  of  materials  for  use  in  a wide  range  of  applications.  The  important  thing  to 
remember,  and  the  unifying  quality  of  a template,  is  that  it  is  a “pattern”  and  it  is  “reusable”  (Chia,  1996). 

Templates  can  offer  the  same  basic  elements  of  design  as  provided  by  the  project-team  “set  framework” 
approach.  Pages  can  be  produced  with  common  attributes  incorporating  the  consistent  design  features  of  buttons 
which  retain  page  position,  constant  background  pattern  or  colour  and  uniform  menu  items.  In  addition,  when 
using  the  template  method,  the  educator  is  able  to  both  create  and  design  the  instructional  program  to  be 
delivered  via  the  Internet,  leading  to  higher  productivity  and  more  efficient  time  management  and  cost 
expenditure  (Jo  & Jones,  1998).  Current  aspects  of  template  use  include  uniform  formatting,  reuse  of  materials, 
easy  modification  and  flexibility  (Rebelsky,  1997).  Much  of  the  current  template  research  and  development 
results  in  interactive  courseware  products,  such  as  CourseWeaver  (Rebelsky,  1997),  ASML:  Web  Authoring 
(Owen  & Makedon,  1997)  and  DISCourse  (Mispelkamp  & Sarti,  1994).  These  template-based  programs  have 
many  advantages,  yet  there  is  a tendency  to  concentrate  on  development  from  the  designer’s  perspective  rather 
than  that  of  the  novice  user. 


Incorporation  of  Teaching  and  Learning  Strategies 

The  current  approaches  to  template-based  design  may  satisfy  the  needs  for  design  simplification, 
clarity,  cost  effectiveness  and  time  efficiency.  However,  the  enhancement  of  learning  performance,  a crucial 
aspect  of  effective  teaching,  appears  to  take  low  priority.  The  issue  of  interactivity  is  also  often  overlooked. 
Interactivity  helps  the  student  remain  focused  and  interested  in  the  subject.  A student-centred  approach  based  on 
the  learning  theories  of  behaviourism  and  cognitivism,  should  play  a significant  role  in  the  preparation  of  any 
educational  multimedia  product  (Jones  & Jo,  1998a).  People  tend  to  learn  initially  by  behavioural  methods.  As 
they  start  to  recognise  strategies  and  patterns,  cognitive  skills  are  established  (Lefrancois,  1988).  It  is  interesting 
to  note  that  Pressey’s  teaching  machines,  although  primitive,  incorporated  the  theory  of  behaviourism,  using  the 
stimulus-response  method  of  teaching. 

The  fundamental  principles  of  behaviourism  can  be  implemented  using  the  stimulus-response- 
reinforcement  process.  This  does,  of  course,  depend  on  the  target  audience,  age  group  being  an  important  factor. 
The  subject  and  purpose  of  the  lesson  should  also  be  considered.  For  a young  target  audience,  the  use  of 
stimulus-response-reinforcement  can  be  useful  in  obtaining  and  keeping  the  students’  attention,  whereas,  for  an 
older  group  it  is  perhaps  more  suitable  to  be  used  only  in  the  feedback  section  of  the  course.  Various  stimuli, 
such  as  visual  and  audio  prompts,  can  instigate  a response  from  the  user  and  suitable  positive  or  negative 
reinforcement  can  be  initiated.  This  may  be  in  the  form  of  a reward  or  remark,  such  as  a pop-up  comment,  an 
offer  to  continue  into  the  next  phase  of  the  lesson,  or  relevant  links  to  other  pages  that  may  help  with  student 
understanding. 

In  an  educational  computer  program,  as  in  everyday  life,  users  are  at  an  advantage  if  they  are  able  to 
apply  previous  knowledge.  In  this  way  they  are  using  their  cognitive  skills  (Lefrancois,  1988),  as  well  as 
constructivism  by  forming  new  knowledge  from  the  learning  experience  (McMahon,  1997).  The  processes  used 
include  recognition,  perception  and  information  processing.  This  is  initiated  by  using  “reminders”  in  the  form  of 
different  coloured  text,  video  clips  or  visual  prompts  to  remind,  represent  or  clarify  certain  points,  and 
incorporating  quizzes  into  the  feedback  structure.  It  is  the  belief  of  the  authors  that  these  approaches  should  be 
considered  when  designing  templates. 

Incorporation  of  teaching  and  learning  strategies  within  the  template  can  be  achieved  by  embedding 
visual  and  audio  prompts,  time-out  detectors,  and  a variety  of  user  options,  such  as  colour  specifications.  The 
techniques  used  are  implemented  by  using  HTML,  DHTML,  JavaScript  and  CGI  scripts. 


Template-based  Educational  Model 

A number  of  templates  may  be  necessary  to  suit  the  many  and  varied  fields  of  study  and  age  groups. 
The  templates,  designed  by  professional  multimedia  designers,  need  only  be  designed  once  thus  reducing  the 
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cost  of  production.  The  educator,  who  uses  the  template  for  web  page  design,  can  enter  course  content  material 
through  a Web  browser,  then  review  and  refine  the  program  as  it  develops,  making  the  whole  process  user- 
friendly  and  flexible.  The  educator  is  provided  with  more  control  as  the  program  can  also  be  modified  or  extra 
lessons  added  at  a later  date.  The  entire  design  process  is  represented  in  (Figure  1). 

Figure  1 represents  the  step  by  step  processes  involved.  The  multimedia  designer  produces  the  design 
library.  This  large  storage  component  comprises  the  designed  templates  held  in  the  template  library  and  an 
audio/video  database.  The  multimedia  designer  has  access  to  both  the  design  library  and  the  Web  page  generator 
for  creation,  maintenance  and  technical  support  purposes.  When  educators  use  the  program,  a template  is 
retrieved  along  with  any  audio/video  material  they  select  for  inclusion.  They  are  guided  through  the  design 
process  in  easy  to  follow  steps,  with  various  options  including  a default  selection.  Once  the  design  process  is 
complete  the  educators  can  store  their  design  in  their  own  personal  design  database  for  later  use  and 
modification.  This  only  requires  a limited  storage  capacity  as  specified  by  the  designer  on  current  context  and 
preferences. 


The  Learning  Enhancement  Unit  (LEU)  is  an  important  factor  in  offering  the  student  an  interactive 
environment  which  applies  teaching  and  learning  strategies.  The  User  Reaction  Analyser  (URA)  is  used  to 
evaluate  student  performance.  The  Reward  Manager  receives  input  from  the  student/user  and  forwards  this  to  the 
URA,  a specially  designed  system  which  uses  a series  of  “if ...  then”  statements  to  produce  a range  of  suggested 
rewards  or  prompts.  The  Reward  Manager  draws  a suitable  reward,  either  positive  or  negative,  from  the  Reward 
Database,  and  implements  a reaction  within  the  Web  page  (Jones  & Jo,  1 998b) 

Issues  that  could  be  detected  by  this  system  include  student  comprehension,  habits  and  preferences,  as 
well  as  physical  issues  of  hearing  and  vision.  For  example,  the  screen  colour  may  cause  difficulty  for  a colour- 
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blind  student.  Other  events  include  time-out  detectors  and  question-answer  analysers  which  detect  the  need  for 
feedback  to  the  student.  The  multimedia-based  Reward  Manager  and  Reward  Database  work  hand-in-hand  in 
determining  and  dispensing  appropriate  reinforcement.  This  includes  positive  and  negative  support,  in  the  form 
of  bonuses,  penalties  and  encouragement  or  deterrents.  The  reward  system  functions  by  using  pop-up  remarks, 
visual  displays,  sound  alerts,  appraisals  or  the  offer  of  a brief  game  or  new  links  to  other  pages. 


Conclusion 

The  template-based  model  for  producing  educational  programs  has  been  designed  and  developed  using 
HTML,  DHTML  and  JavaScript  Web  languages.  The  template  itself  consists  of  a series  of  easy  to  navigate  web 
pages  which  guide  the  user  through  the  design  process  offering  many  options.  It  is  easy  to  use,  especially  for  the 
novice  user  as  no  programming  experience  is  necessary  or  assumed.  As  the  program  is  written  in  Web 
languages,  the  pages  appear  similar  to  any  web  page  and  are  just  as  easy  to  use. 

With  this  model,  educators  work  independently  at  their  own  time  and  pace.  They  are  provided  with  the 
means  to  design,  produce  and  modify  their  own  instructional  programs  thus  relevant  teaching  and  learning 
strategies  may  easily  be  incorporated  within  a common  design.  This  gives  educators  greater  control  over  course 
presentation  and  learner  performance.  Using  a template,  the  educator  can  effectively  become  the  creator  of  an 
individualised  learning  tool. 
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Abstract 

Information  technology  (IT)  is  critical  to  the  ftiture  success  of  universities.  Those  universities  that 
succeed  in  integrating  IT  will  attract  students  and  resources;  those  that  get  it  wrong  will  suffer 
financial  and  structural  difficulties.  Any  management  scheme  for  IT  in  a university  needs  to 
balance  five  key  factors:  Strategy,  Structure,  Management  processes,  Roles  and  skills  and 
Technology.  Successful  universities  exhibit  a high  level  of  fit  among  these  five  factors.  A poor  fit 
means  that  a university  is  not  functioning  in  a cost-efficient  with  respect  to  IT.  Most  universities 
have  developed  some  form  of  educational  technology  support  structure  centred  on  a media  unit, 
computing  service  or  library.  The  paper  concludes  by  describing  three  approaches  to  the 
management  of  IT  support:  a distributed  approach  (old  universities)  integrated  approach  (new 
universities)  and  parallel  approach  (divisional  universities) 

Engines  for  change 

Technology  is  inexorably  changing  higher  education  as  we  move  into  the  digital  age  of  the  third  millennium 
(Negroponte,  1 995).  Of  course  technology  is  not  the  only  factor  at  work  here,  but  is  one  of  many  forces,  and  they  are 
all  inter- related. 

Mass  tertiary  education. 

In  the  1950s  and  1960s  in  most  developed  countries  8-10%  of  schools  leavers  went  to  university;  in  the  1990s  the 
figure  is  closer  to  40%.  This  has  two  consequences 

■ These  students  can't  all  go  to  the  'old'  universities  which  have  dominated  the  field  for  the  past  1 00  or  more 
years;  new  universities  have  been  established  to  meet  the  needs  of  mass  education 

■ Universities  differ  in  quality:  some  new  universities  have  developed  niche  positions  and  specialisations  in  non- 
traditional  or  cross-disciplinary  fields;  others  are  more  like  American  3-year  colleges. 

■ Distance  and  flexible  learning  is  attracting  n on-traditional  students. 

Learning  theory 

The  past  decade  has  seen  a noticeable  shift  in  learning  theory  from  'top-down'  instruction-based  teaching  to  a focus 
on  the  individual  student's  construction  of  knowledge.  This  is  evidenced  in  changing  curriculum  structures  such  as 
Problem-Based  Learning,  Flexible  Learning,  educational  multimedia  and  Web-based  teaching. 

Technological  imperative 

In  a period  of  1 0 years  there  has  been  a revolution  in  information  and  communication  technologies  that  has  affected 
almost  every  area  of  life,  not  the  least  education.  Some  of  its  manifestations  have  been: 

■ A convergence  of  technologies  whereby  different  and  distinct  media  such  as  print,  video,  graphics,  photography 
and  audio  have  converged  into  a single  technology,  multimedia. 

■ An  information  explosion,  whereby  an  Internet  user  with  affordable  equipment  can  gain  access  to  exponentially 
increasing  volumes  of  information. 

■ The  cost-performance  ratio  of  IT  has  improved  dramatically  with  technologies  such  as  CD-ROM  and  the 
WWW,  which  enable  flexible  delivery. 

Economic  view  of  IT 

Clark  & Sugrue  (1995)  identify  economics,  particularly  cost-benefit,  as  a driving  force  behind  the  adoption  of  IT  in 
education.  The  arguments  can  be  summarised  as: 

■ New  technologies  promise  the  possibilities  of  new  strategic  advantages: 

■ Shift  from  the  constraints  of  place  to  the  freedom  of  space:  learn  anywhere,  any  time. 

■ Shift  from  focus  on  teaching  by  lecturer  to  learning  by  the  student 
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New  paradigms  for  higher  education 

New  paradigms  have  arisen.  The  following  table  is  an  amalgam  of  many  dichotomies  found  on  the  World  Wide 
Web: 


Old 


New 


Students  Students  are  1 8’25 


Culture  University  as  a "city" 


Learning  Books  are  primary  medium 
Listen  & learn 
Teaching  Single  product 


System  Take  what  you  can  get 


Funding  Government  funded 


Faculty  Tenure 


Academic  calendar 
Terminal  degree 


Student  as  a burden 
A 4 years  revenue  source 


Competition  is  other  universities 
Technology  an  expense 


Bricks  and  mortar 
Multi-cultural 
Institution  centric 
University  as  ivory  tower 


Delivery  in  a classroom 
Single  discipline 


Research  is  priority 
Responsible  for  content 


Courses  on  demand 
Year-round  operations 
Lifelong  learning 
Cradle  to  grave 
Student  as  a customer 
Lifelong  revenue  source 
Information  on  demand 
Solve  problems 
Information  reuse 
Delivery  anywhere 
Multi-discipline 
Market  value 
Teaching  is  assessed 
Part  of  a team 
Market  funded 
Competition  is  everywhere 
Technology  as  differentiation 
University  as  an  idea 
Bits  and  bytes 
Global 

Market  centric 

University  as  partner  in  business 


Table  1 


Impediments  to  change 

Not  everyone  agrees  with  this  recent  and  sudden  move  towards  IT  and  education.  Universities,  more  than  most 
organisations,  value  tradition  and  are  institutionally  resistant  to  sudden  change.  In  most  universities  it  is  easy  to 
observe  an  ongoing  struggle  between  advocates  of  university  tradition  and  of  the  new  technologies. 

Conservatism 

Universities,  even  the  new  ones,  act  as  if  they  are  part  of  a long  tradition  going  back  to  the  early  universities  of 
Europe  (gowns,  ceremonies,  arcane  by-laws,  titles,  etc.)  In  this  sense,  universities  are  appropriately  conservative  and 
students  value  the  traditions  they  conserve.  In  contrast,  IT  strategic  planning  often  advocates  major  change  in  the 
delivery  or  form  of  education.  The  guardians  of  a university's  reputation  may,  and  probably  should,  resist  such 
changes  until  they  are  convinced  that  this  is  not  a flash  in  the  pan. 

Culture 

Not  everyone  in  a university  shares  the  same  culture.  This  is  true  in  particular  for  technology  specialists  and 
academics.  They  have  very  different  experiences  and  each  often  sees  the  other  as  uninformed. 

■ Academics  typically  have  long  formal  training,  8-10  years  for  a PhD  (12-15  years  for  a habilitation  in  Europe) 
and,  frequently,  little  experience  outside  higher  education. 

■ Technology  specialists  often  have  less  formal  training  (but  not  necessarily  less  training)  and  typically  have 
experience  in  a number  of  different  industries  but  may  not  fully  understand  the  university's  culture. 

IT  Management 

Rapid  changes  in  IT  and  in  the  higher  education  sectors  generally  have  stimulated  changes  in  IT  management 
processes.  In  some  cases  these  changes  have  provoked  knee-jerk  reactions  or  have  been  grist  to  the  mills  of  internal 
power  plays:  media  centres  have  been  closed,  or  moved  into/out  of  the  library  or  amalgamated  under  a central  body. 


In  most  university  Senate  debates  and  discussion  papers  about  the  role  of  IT  there  are  a number  of  common  key 
questions; 

■ Who  owns  IT? 

■ Where  should  it  sit  in  a university  structure? 

■ How  do  we  get  the  cost  benefits  of  centralisation  as  well  as  the  value-added  benefits  of  devolution  and  user 
control? 

■ Should  IT  report  to  Administration  or  to  a PVC  (Academic)  along  with  other  teaching  and  learning  units? 

■ Is  the  head  of  Flexible  Learning  (an  educator)  at  the  same  level  as  the  head  of  IT  (a  technologist)? 

■ Should  training  about  teaching  and  learning  technologies  be  in  the  academic  professional  development  unit  or 
in  Information  Services? 

■ Should  the  professional  development  unit  be  in  a faculty  or  in  central  administration? 


Planning  change  - the  MIT90  Model 

Reflecting  the  new  paradigms  set  out  in  Table  1 , many  universities  are  turning  to  business  models  in  order  to 
restructure  their  management,  curriculum  and  communications.  The  four  diagrams  below  illustrate  variations  of 
what  is  known  as  the  MIT90  model,  developed  by  Scott  Morton  (1991),  and  quoted  by  Yetton  (1997)  in  an 
important  paper  on  managing  technology  in  higher  education  commissioned  by  the  Australian  Government. 
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Strategic  fit  (Figure  1) 

Yetton  notes  that  successful  organisations  exhibit  a high  level  of  fit  among  five  factors:  strategy,  structure, 
management  processes,  roles  & skills,  and  technology.  "It  is  a principal  task  of  management  to  attain  and  sustain  a 
high  level  of  fit  and,  when  strategic  change  occurs,  to  re-establish  fit,  and  therefore  high  performance  as  soon  as 
possible"  (p.2). 
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Organisations  in  strategic  fit  have  four  characteristics. 

1 . The  effective  integration  of  strategy,  structure,  management  processes,  technology  and  roles/skills  creates  a 
focused  organisational  gestalt  which  simplifies  formerly  complex  organisational  and  management  demands. 

2.  This  simplicity  fosters  clarity  of  understanding  within  the  organisation,  which  reinforces  and  sustains  fit.  This 
means  that  the  organisational  structures,  management  processes  (such  as  reward  and  control  systems),  and  key 
technologies  combine  to  support  the  strategic  intent,  as  well  as  shaping  the  required  task  focus  and  behaviour  of 
managers  and  employees. 

3.  Simplicity  reduces  the  need  for  elaborate  and  complex  coordinating  mechanisms;  thereby  freeing  up  resources 
which  can  be  focused  on  value-generating  activities. 

4.  The  high  levels  of  performance  created  by  this  strategic  fit  serve  to  reinforce  the  process  by  which  it  is  attained, 
and  thus  further  simplify  the  fit  among  the  elements  of  the  organisational  gestalt. 

Such  a tight  fit,  both  internally  among  the  elements  of  the  organisation  and  externally  with  its  environment,  is 
associated  with  high  performance. 

Top  down  management  (Figure  2) 

In  a time  of  change,  when  the  balance  between  these  factors  has  been  altered  (as  described  above,  for  example),  how 
do  we  go  about  re-establishing  fit?  The  most  common  method  is  to  view  strategic  change  as  a three -step  process, 
driven  by  a top  down  strategic  positioning  perspective. 

1 . A new  strategy  is  identified  and  adopted. 

2.  The  organisation  is  restructured  to  support  that  strategy. 

3.  The  IT  strategy  and  the  management  processes  are  redefined  and  aligned  to  the  organisational  strategy  and 
human  resources  trained  in  these  new  skills  as  required. 

This  conventional,  rational  model  of  strategic  change  assumes  that  successful  change  is  the  result  of  a formal 
process  of  strategic  planning,  in  which  IT  is  aligned  to  the  dominant  strategic  design.  In  line  with  business 
organisations  in  other  fields,  many  universities  have  explicitly  adopted  this  top  down  positioning  model  of  IT-based 
strategic  change.  The  most  common  outcome  of  this  approach  has  been  the  drastic  restructuring,  or  in  some  cases 
the  dismantling,  of  existing  educational  technology  services  and  the  creation  of  an  entirely  new  grouping, 
independent  of  teaching  departments.  Its  mission  is  to  speed  up  the  employment  of  IT  in  teaching  and  management 
but  it  often  meets  resistance  from  areas  of  the  university  which  have  not  been  restructured. 

Strategy  vs.  culture  (Figure  3) 

The  MIT90  schema  clarifies  one  of  the  common  problems  of  strategic  change  with  technology,  when  the 
impediments  to  change,  mentioned  above,  come  into  conflict  with  the  vision  of  university  management.  In  Figure  3, 
strategic  IT  planning  is  the  horizontal  arm  to  the  cross.  When  the  organisation  culture,  the  vertical  post  of  the  cross, 
is  compatible  with  the  implicit  assumptions  of  IT  strategic  planning,  there  is  close  fit  and  few  impediments  to  the 
realisation  of  the  strategy. 

Unfortunately,  existing  university  culture  is  frequently  not  supportive  of  IT  strategic  planning.  So  as  the  university 
implements  its  change,  it  shifts  out  of  fit  and  performance  drops  off.  Unless  it  can  bring  its  culture  into  fit  quickly, 
then  performance  loss  is  likely  to  stall  the  change  initiative. 

Technology  as  the  driver  (Figure  4) 

If  we  redraw  the  MIT90s  framework  to  show  IT  at  the  centre  of  the  five  elements,  we  can  visualise  IT's  role  in  both 
top  down  and  bottom  up  management  of  change.  The  diagram  shows  two  triangles  illustrating  the  top  down  and 
bottom  up  dimensions. 

Much  IT-based  strategic  change  involves  altering  the  competencies  of  the  organisation,  and  these  are  found  in  the 
bottom  triangle.  So  as  well  as  considering  the  organisation's  strategic  position  (the  top  triangle),  change  requires 
careful  attention  to  reconfiguring  the  bottom  triangle.  In  fact  an  integrated  top  down  and  bottom  up  management  of 
IT-based  change  is  required  (Craig  & Yetton  1997). 

Staff  roles  & skills 

Of  the  five  factors  described  in  the  models,  the  most  contentious  for  academic  staff  is  the  requirement  for  change  in 
"Individual  Roles  & Skills". 
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• Teachers  need  to  have  more  theoretically  grounded  pedagogical  knowledge,  so  that  they  can  manage  and  direct 
projects  using  IT  for  creating  resources  and  facilitating  processes,  otherwise  they  may  come  to  regard 
technology  as  a panacea. 

• In  the  new  media  of  network  based  learning  the  teacher  is  not  the  primary  source  of  content,  but  rather  a 
resource  person  with  content  expertise.  There  is  a shift  in  the  'power  balance'  between  teacher  and  student.  The 
use  of  contact  time  is  more  determined  by  student  needs  than  teacher’s  agenda. 

• Academic  staff  may  take  up  technical  roles  related  to  media  production  (interface  design,  graphic  design, 
instructional  design,  storyboarding,  programming,  media  selection,  file  conversion  etc.);  there  will  be  a greater 
emphasis  on  process  design  for  teaching  and  increased  emphasis  on  the  roles  associated  with  the  development 
of  distance  materials. 

• IT  experienced  staff  need  to  be  recruited  as  staff  developers  and  trainers  with  respect  to  their  colleagues  in 
relation  to  communicating  innovative  practices  and  assisting  colleagues  to  learn  the  skills  required  to  use  IT  in 
their  teaching 

• More  and  different  'basic  skills'  are  required  for  academic  work.  Teaching  roles  are  becoming  more  information 
technology  intensive,  with  demands  for  straight  technical  knowledge  and  skills  increasing:  word  processing, 
presentation  software,  project  planning,  programming,  graphics,  sound  and  video  are  among  the  areas  coming 
into  contention  for  inclusion  in  the  core  skill  sets  of  teachers. 

• Academics  are  being  required  to  take  on  increased  administrative  work  with  developments  in  networked 
administrative  systems.  Central  data  entry  and  report  generation  in  areas  like  student  administration  are  being 
devolved  to  Faculties  and  Schools,  and  some  of  this  work,  once  done  by  administrative  staff,  is  being  taken  up 
by  academic  staff 

• Heads  of  Schools  who  once  understood  the  technologies  of  teaching  (blackboards,  classrooms  and  laboratories) 
are  now  required  to  make  decisions  about  curriculum  developments  and  equipment  purchases  with  significant 
information  technology  components. 

• The  information  technology  component  of  work  is  increasing  in  all  areas,  and  managers  are  also  having  to  make 
decisions  about  the  development,  funding  and  maintenance  of  information  technology  intensive  activities,  often 
with  a less  than  perfect  grasp  of  the  technical  details  involved. 

Costs  of  production 

A primary  change  brought  about  by  Technology  is  the  cost  equation  of  producing  and  utilising  new  media 
materials: 

• Universities  are  not  accustomed  to  paying  for  the  development  of  curriculum  and  teaching  materials.  Content 
and  quality  are  typically  seen  as  the  private  responsibility  of  teaching  staff.  Teaching  materials  are  rarely 
considered  to  be  university  property. 

• The  use  of  IT-based  teaching  resources  and  processes  demands  much  higher  'production  values'  than  is  the  case 
for  lecture  notes  and  paper  handouts  - more  staff  and  staff  time  in  development,  and  the  use  of  a range  of 
production  services. 

• The  process  of  recognising  the  true  costs  of  teaching  with  technology  based  resources  and  processes,  and  of 
funding  it  appropriately,  involves  shifting  the  issue  from  the  private  concern  of  teachers  to  the  concern  of  the 
institution. 

• This  shift  has  cultural,  industrial  relations  and  legal  implications,  all  of  which  will  have  to  be  addressed. 

Three  models  of  IT  management 

We  do  not  need  to  conclude  that  one  of  the  approaches  described  above  is  right  and  the  others  wrong,  in  which  case 
the  task  would  be  to  discover  the  best  approach.  Rather,  we  can  assume  that  each  university  will  attempt  to  develop 
a different  approach,  based  on  some  combination  of  the  above  models  of  change,  to  discover  a new  effective  fit,  and 
to  satisfy  its  stakeholders'  needs.  The  success  of  the  approach  adopted  by  a university  will  in  part  be  contingent 
upon  the  fit  with  its  current  resource  position,  history  and  capabilities,  as  well  as  how  effective  it  is  in  successfully 
differentiating  itself  and  embedding  the  required  strategic  change  in  the  organisation. 

Old  universities 

Until  comparatively  recent  times  university  education  was  an  elite  activity  and  the  older  universities  offered  degrees 
dominating  the  high  status  professions  of  medicine,  law  and  engineering.  These  universities  are  located  in  or  near 
the  CBDs  of  capital  cities  and  have  long  traditions  of  scholarship,  endowments  and  research.  Many  of  society's 
leaders  are  graduates  of  these  universities. 
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IT  Strategy 

■ A distributed  approach  with  responsibility  for  IT-based  teaching  and  learning  developments  devolved  to 
local  innovators  across  a range  of  faculties  and  units 

■ IT  is  used  to  enrich  the  old  university's  elite  community. 

■ Concentration  on  maintaining  the  established  base  while  funding  independent  new  ventures. 

■ Successful  ventures  feed  the  established  university  community  with  their  innovations  in  teaching,  learning 
& research. 

■ Entrepreneurial  ventures  have  the  ability  to  select  expertise  with  few  constraints  such  as  terms  and 
conditions  of  employment. 

■ Attracts  high  status  academic  risk  takers  and  innovators. 

New  universities 

Many  began  life  as  polytechnics  or  specialised  institutes  and  have  recently  been  'promoted'  to  university  status; 
some  have  been  created  from  scratch.  There  is  little  if  any  tradition  of  research  and  the  emphasis  is  on  establishing  a 
reputation  for  teaching  excellence,  employment-oriented  courses  and  quality  facilities. 

IT  Strategy 

■ An  integrated  approach  with  a central  unit  managing  the  integration  of  teaching  and  learning  with  IT, 
emphasising  support  for  professional  development  in  educational  and  information  technologies  and  linking 
it  to  university  goals. 

■ IT  is  central  to  and  critically  underpins  the  strategic  agenda. 

■ IT  enabled  teaching  & learning  is  designed  to  deliver  quality  and  reliability  to  a large  number  of  students. 

■ Highly  skilled  experts  in  production  and  technology  are  recruited  as  required. 

Divisional  universities 

Typically  large  and  devolved,  often  an  amalgamation  of  a number  of  institutions  with  multiple  campuses  and  a high 
degree  of  specialisation  in  strong  academic  faculties.  There  is  a strong  emphasis  on  applied  research  and 
collaboration  with  industry,  particularly  in  disciplines  such  as  Engineering,  Business,  Health  and  Education. 

IT  strategy 

■ A hybrid,  strategy  that  uses  IT  to  obtain  benefits  of  a low  cost  central  IT  infrastructure,  while  empowering 
innovation  and  student  focus  in  strong  academic  faculties. 

■ Devolved  faculties  enabled  by  a powerful  IT  central  infrastructure  each  has  different  competencies  and 
strategic  foci;  each  division  has  its  own  IT  support  and  develops  its  own  appropriate  set  of  technologies, 
management  processes,  skills  & roles. 

■ Focus  on  each  division's  core  competencies  in  particular  areas  of  applied  research  & teaching 
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Abstract:  Internet  delivered  Continuing  Medical  Education  (ICME)  is  a new  learning 
mode  for  medical  practitioners.  This  technique  allows  the  delivery  of  CME  material  via 
the  Internet,  using  the  World  Wide  Web,  regardless  of  geographic  location.  This  paper 
reports  a case  study  of  GasBone,  an  early  adopter  and  innovator  in  ICME.  The  Introduction 
describes  the  users  and  their  characteristics,  the  educational  goals  addressed  by  the 
ICME  project,  GasBone,  and  the  issues  addressed  by  this  paper.  Study  Methodology 
describes  logfile  and  questionnaire  methodologies  together  with  a description  of  the 
different  presentation  methods  included  in  the  GasBone  project.  The  third  part 
discusses  the  findings  from  the  logfile  analyses  and  questionnaire  results  relating  to  ICME 
lecture  presentations.  Finally  the  results,  conclusions  and  recommendations  for 

organisations  implementing  remote  delivery  via  the  WWW  are  discussed. 


Introduction 

Medicine  requires  its  practitioners  to  update  their  knowledge  and  skills  on  a continuing  basis  in  order 
to  maintain  a license  to  practice.  Continuing  Medical  Education  (CME)  is  a term  used  to  encompass  the 
process  of  lifelong  learning  following  initial  graduation  from  medical  school.  Traditional  delivery  modes  for 
CME  include  self-directed  reading  of  journals  and  books  together  with  attendance  at  conferences,  courses  and 
lectures.  Access  to  group  CME  events  are  frequently  limited  by  geographical  isolation  and/or  the  need  to 
maintain  medical  services  to  the  local  community. 

Medical  practitioners  have  always  undertaken  CME  activities  but  the  pace  of  change  now  limits  the 
effectiveness  of  traditional  practices.  The  advent  of  the  World  Wide  Web  and  ICME  provides  an  important 
additional  access  to  delivery  that  may  solve  some  of  the  problems  of  geographical  isolation  and  inequity  of 
access. 

The  on-line  environment  is  different  from  the  traditional  style  of  lecture.  The  learner  can  read  the 
lecture  anytime  and  anywhere  as  long  as  they  have  a computer  modem  and  Internet  connection.  Even  though 
in  early  1998  Internet  CME  is  still  in  its  infancy,  it  may  become  the  most  common  and  convenient  CME 
learning  mode  in  the  future.  As  an  example,  Malaysia,  a developing  country,  has  already  planned  to  implement 
Internet  CME  under  the  Flagship  of  Telemedicine  in  its  Multimedia  Super  Corridor  (MSC)  project.  The 
relevance  of  this  research  into  Internet  CME  and  its  potential  for  future  development  is  high. 

Suggs,  Rose  and  Mittelmark  (1993,  149)  see  CME  not  as  a supplementary  service  to  health 
professionals  but  as  a necessity.  The  professional  development  of  health  care  personnel  is  crucial  as  they  have 
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to  adapt  to  the  rapid  (even  daily)  changes  in  knowledge  generated  by  research  and  the  technological  advances 
made  in  medicine.  Wallace  (in  Suggs,  Rose  and  Mittelmark,  1993)  lays  the  responsibility  on  the  professions  to 
improve  and  enhance  the  existing  knowledge  and  pass  it  on  to  the  coming  generation.  Miller,  et  al.  (1997,  760) 
and  Carney  et  al.  (1995,  52)  are  in  agreement  that  CME  is  designed  to  improve  quality,  knowledge  and  skills 
in  order  to  maintain  the  competence  and  capabilities  of  physicians. 

This  case  study  examines  the  GasBone  web  project,  an  ICME  site  that  specializes  in  the  delivery  of 
CME  in  the  medical  subspecialty  of  Anaesthesia.  We  wish  to  determine  which  formats  and  modalities  of  ICME 
are  preferred  by  medical  audiences.  Both  retrospective  log  file  analysis  and  prospective  questionnaire  were 
performed  during  the  study  which  was  conducted  over  an  8-month  period  from  late  1997  until  mid  1998.  Web 
server  logfiles  were  analyzed  covering  the  period  June  1996  to  November  1997. 

In  this  research,  GasBone,  a project  of  the  Division  of  Anaesthesiology  & Intensive  Care  of  the 
University  of  Queensland,  Brisbane  in  Australia,  was  investigated  as  an  example  of  ICME.  This  project  was 
originally  conceived  as  a way  of  linking  geographically  separate  centers  in  a statewide  medical  department  but 
grew  into  a major  facility  for  the  provision  of  CME  for  the  international  medical  community.  By  late  1997  the 
GasBone  project  had  been  in  use  for  CME  and  undergraduate  medical  teaching  for  3.5  years  (Palmer  1996a, 
1996b). 

GasBone  provides  a library  of  WWW/Intemet  lectures,  presented  in  real  time  video,  lecture  slides 
with  audio,  lecture  slides,  graphic  images  and/or  plain  text  page.  This  CME  web  site  provides 
educational  material  specifically  for  specialist  anaesthetists,  anaesthetist  trainees  and  GPs  who  practise 
anaesthesia.  The  GasBone  project  was  a proof  of  concept  project  rather  than  a fully  funded  research  project 
so  new  lectures  were  added  as  funding  became  available  during  the  period  under  study. 

The  GasBone  project  includes  42  on  line  CME  lectures  in  various  formats  including:  text  based 
web  pages  (19);  text  based  slides  (10);  video  only  (6);  text  slides  with  sound  track  (3);  slides  with  video  (2); 
and  text  slides  with  audio  and  some  video  (2). 

These  lectures  target  various  knowledge  levels.  Two  thirds  of  the  lectures  are  anaesthetic  CME  for 
practising  anaesthetists  and  trainee  anaesthetists.  The  other  third  of  the  lectures  target  undergraduate 
medicine  and  pre-medicine  students. 

The  GasBone  project  was  an  early  adopter  of  streaming  video  on  the  WWW  with  a VDO  server 
installed  in  1996.  The  web  server  and  video  server  are  separate  software  programs  running  on  a 
common  hardware  platform  (Pentium  120,  128MB  Memory,  Freebsd2.2).  User  requests  for  video  are 
processed  by  the  web  server  but  actual  delivery  of  video  is  handled  by  the  video  server.  The  server  has  a 
permanent  connection  to  the  Internet  via  the  University  of  Queensland  network. 

Study  methodology 

This  case  study  uses  both  retrospective  analysis  of  Web  server  log  files  and  prospective  mail 
questionnaire.  Server  log  files  will  include  data  from  all  users,  both  local  and  international  (Ng,  1998,4-4)  and 
may  also  include  a proportion  of  traffic  from  non-medical  users.  The  mail  questionnaire  was  however 
directed  to  all  specialist  and  non-specialist  anaesthesiologist  trainees  in  Queensland,  Australia  The  log  file 
analysis  gathered  data  on  the  monthly  hit  rate  for  five  modes  of  CME  presentation.  Text  only;  plain  text 
with  graphical  illustration;  graphical  slides  with  a sound  track;  slides  with  sound  and  video  and  video 
only.  Data  was  collected  for  the  monthly  'hits'  data  for  the  first  page  of  a set  of  lectures  from  the  time  they 
became  available  until  the  end  of  the  study  period  for  all  the  presentation  types  available. 

The  mail  questionnaire  was  sent  to  the  1041  specialist  anaesthetists,  anaesthetist  trainees,  and 
rural  and  remote  practising  GPs  who  also  perform  anaesthesia.  These  practitioners  comprise  the  GasBone 
target  group  living  in  Queensland.  Responses  were  received  from  32%  (332)  of  the  target  group.  Only  19% 
(63)  of  them  had  participated  in  Internet  CME  at  the  time  of  our  survey  (February  1998). 

We  asked  respondents  to  rank  their  preference  for  the  following  seven  presentation  types. 

1 . Series  of  20  slides  from  a lecture:  a linear  online  / Internet  lecture,  presented  in  the  form  of  images  similar 
to  a PowerPoint  presentation  or  a set  of  transparencies.  The  user  can  browse  serially  from  one  slide  to 
another  by  using  a mouse  to  click  on  either  the  previous  or  next  button  at  the  bottom  of  each  slide. 

2.  Sequence  of  5 slides  from  a lecture:  same  format  as  (1)  except  for  much  shorter  series  of  slides. 


3.  Slides  with  sound  track:  an  online  lecture  usually  presented  as  just  1 slide  with  the  speaker’s  voice 
recorded.  Text  is  minimal. 

4.  Slides  with  sound  and  video:  an  online  lecture  presented  in  the  form  of  slides  with  embedded  sound  and 
video.  There  are  usually  a number  of  slides  for  this  type  of  lecture.  Each  slide  has  its  own  sound  and  video 
The  user  must  install  VDO  plug-in  software  in  order  to  view  this  type  of  lecture. 

5.  Plain  text  only:  an  online  lecture  presented  as  a web  page  with  only  text.  The  text  is  often  in  only  one  file 
but  there  is  no  indication  of  length  aside  from  the  scroll  button  included  with  the  browser.  The  user 
browses  through  the  whole  lecture  by  using  the  mouse  or  arrow  keys. 

6.  Plain  text  with  at  least  half  pictures:  this  online  lecture  is  similar  to  plain  text  only  lecture  except  almost 
half  of  the  lecture  is  presented  in  the  form  of  static  graphics  or  pictures. 

7.  Video  only:  an  online  pure  streamed  video  in  which  a lecture  is  presented  in  the  form  of  a motion  picture 
but  with  a quarter  screen  picture  and  very  slow  frame  rate.  This  type  of  lecture  also  needs  the  VDO 
software  plug-in  installed  in  order  to  view  it. 


What  Kind  of  Lecture  Presentation  Keeps  Users’  Interest? 

Presenting  web  based  learning  materials  is  of  little  use  if  users  do  not  or  cannot  access  them.  Disuse 
stems  from  a variety  of  problems  such  as  slow  modem  lines,  boring  material,  overly  long  sequences  of 
material,  the  need  for  plug  ins  before  the  material  can  be  used,  sound  either  alone  or  with  video  that  is 
indistinct  or  unintelligible,  even  content  that  is  inappropriate  to  the  user's  learning  needs. 

The  questionnaire  provided  data  on  respondent's  subjective  preferences  for  the  seven  types  of  lecture 
presentation  on  the  Internet.  Note  however  that  less  than  22%  had  direct  experience  of  an  ICME  presentation. 
To  gauge  the  difference  between  preferred  information  delivery  method  and  the  current  practicalities  of 
Internet  CME  this  data  was  analysed  separately  for  respondents  with  and  without  Internet  CME  experience, 
(Tab.  1). 


Types  of  presentation 

Rank  (all 
respondents) 

Rank  (ICME 
experienced) 

Slides  with  Sound  and  Video 

1 

2 

Slides  with  Sound  Track 

2 

1 

Plain  Text  with  Lots  of  Pictures 

3 

3 

Video  Only 

4 

7 

A Series  of  20  Slides  from  a Lecture 

5 

4 

A Sequence  of  5 Slides  from  a Lecture 

6 

5 

Plain  Text  Only 

7 

6 

Table  1:  Comparison  of  rankings  of  Internet  lecture  presentations  between  all 
respondents  and  those  with  ICME  experience. 

Sound,  video  and  pictures  rank  highly  when  the  whole  group  of  respondents  is  considered,  but  video 
drops  in  rank  among  those  with  ICME  experience.  We  can  see  from  the  second  column  that  Video  Only, 
without  the  accompanying  slides,  was  least  preferred  by  this  group.  Video  quality  off  the  Web  is  still  not  good 
enough  in  comparison  to  television  or  even  home  video.  The  network  bandwidth  and  currently  available 
technology  causes  the  download  time  to  be  lengthy,  often  exceeding  user’s  patience. 
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Percentage  of  Hit  Rates  for  the  final  slide  in  a series  compared  mth  the  Total 
Number  of  slides  in  a series 
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Total  Number  of  Slides  in  Lecture 


Figure  1 : Retention  of  user  interest  by  number  of  slides  in  the  presentation. 

Figure  1 plots  the  percentage  of  hits  for  the  final  slide  in  relation  to  hits  for  the  first  slide.  The  total 
number  of  slides  in  each  presentation  varies  from  3 to  5 1 . The  graph  clearly  indicates  the  decline  in  retention 
of  users  as  the  total  number  of  slides  increases  from  3 (39%)  to  15  (10%)  For  lectures  which  consisted  of 
more  than  15  slides  retention  was  generally  well  below  10%  and  fell  as  low  as  5%. 

The  anomalies  were  those  with  content  of  particular  interest  (for  example  the  total  slides  of  21, 
“Preoperative  Assessment  of  Dental  Patients”,  and  44,  “Electronic  Access  to  Medical  Education”)  or  those 
which  contained  elements  which  had  slow  load  times  (total  slides  of  5 and  7 are  lectures  presented  as 
segmented  video  and  requiring  the  user  to  have  plug-in  video  software  in  order  to  start  the  lectures).  Thus 
content  and  accessibility  appear  unsurprisingly  to  affect  retention. 

Comparing  log  file  data  for  plain  text  and  graphic  intense  pages,  we  found  there  was  very  little 
difference  in  the  access  rates  for  the  two  presentation  methods.  This  seems  to  indicate  that  well  chosen 
and  sized  graphics  does  not  preclude  user  acceptance. 

When  the  hit  rates  for  plain  text  were  compared  with  video  pages  we  expected,  but  did  not  see  a 
significant  difference.  However  this  may  be  spurious  as  Internet  video  was  a technical  novelty  at  the  time  of 
the  study  period  and  GasBone  was  an  early  adopter.  The  video  requests  may  therefore  have  been  high  during 
the  first  few  months  of  availability  due  to  interest  in  the  technology  rather  than  the  content.  More  recent 
feedback  from  the  target  audience  indicates  Internet  video  delivery  is  not  as  successful  as  a hybrid  delivery  that 
places  the  video  on  CD-ROM  with  dynamic  links  to  the  web  site  for  updates. 
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Ten  Months  average  hits  for  the  six  types  of  Internet  lecture  presentation 


slides  with  sound  textual  only  slides  plain  text  only(avg  graphic  intense  slides  with  sound  video  only(avg  of 
and  video(avg  of  (avg  of  10)  of  1 0)  track(avg  of  3)  6) 

2) 


Types  of  lecture  presentation 


Figure  2:  Average  monthly  hit  rate  by  type  of  Internet  lecture  presentation 

Figure  2 presents  the  average  monthly  hit  rate  for  each  type  of  presentation.  It  can  be  seen 
that  slides  with  audio  and  video  had  a higher  average  monthly  hit  rate  than  any  other  presentation  type.  We 
believe  that  slides  with  media  provide  textual  backup  to  the  video  or  audio  that  augments  the  user’s 
comprehension. 

Slides  with  audio  and  video  had  the  highest  average  hits  in  the  log  file  data  depicted  in  Figure  2 but 
ranked  second  among  the  ICME  experienced  respondents  in  Table  1 . Appropriate  use  of  video  with  audio  is 
thus  not  just  a curiosity,  although  the  higher  hit  rates  shown  above  can  be  partly  attributed  to  the  novelty 
factor. 

Conclusions  and  Recommendations 

The  nature  of  the  GasBone  project  presents  challenges  to  data  analysis.  Materials  were  placed  on  the 
Web  as  they  could  be  made  available.  As  an  early  adopter  of  streaming  video  on  the  WWW,  the  developers 
experimented  with  video  presenters,  styles  of  presentation  and  modes  of  delivery.  The  apparent  advantage  of 
integrating  video  and  audio  into  an  otherwise  textual  slide  that  orients  the  user  was  indicated  in  this  study  and 
has  proved  successful  in  the  longer  term.  Log  file  data  is  ’noisy'  by  nature  and  in  the  case  of  publicly  available 
Internet  delivered  learning  materials  does  not  necessarily  represent  relative  preference  by  the  target  audience. 

In  the  interest  of  achieving  a high  volume  feedback  from  the  questionnaire,  we  felt  that  difference 
between  perceived  choice  and  what  is  viable  and  usable  over  current  (particularly  remote)  WWW  delivery  was 
not  fully  reflected  in  the  data  gathered.  Indications  of  reality  emerged,  however,  from  separate  analysis  of  the 
responses  of  those  who  had  participated  in  ICME. 

In  sparsely  settled  areas  of  Queensland,  modem  speeds  were  achieving  no  better  than  14.4  kb/s  at  the 
time  of  this  survey.  The  content  of  a streaming  video  would  have  to  be  outstanding  before  busy  people  like 
medical  practitioners  would  be  willing  to  wait  for  it  to  download.  However,  as  indicated  earlier,  more  recent 
efforts  have  shown  hybrid  delivery  using  a CD-ROM  for  video  delivery  to  be  viable.  Slides  with  sound  and 
video  are  another  alternative  and  were  more  frequently  requested  than  were  the  video  only  pages.  There  is  a 
definite  interest  in  video  and  one  must  monitor  advances  in  these  technologies  that  may  allow  wider  delivery  of 
useful  video. 

It  is  also  important  to  honour  the  user’s  comfort  zone.  Medical  practitioners  are  accustomed  to 
learning  from  text.  Providing  long  textual  pages  that  can  be  downloaded  in  one  hit  for  local  printing  are  a 
useful  delivery  mechanism  for  this  target  group.  Moving  decorations  are  much  less  of  interest,  particularly  with 
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the  bandwidth  restrictions.  However,  well  chosen  and  clearly  laid  out  graphics  can  provide  useful  explanatory 
power. 

While  a broader  group  was  not  sampled  in  this  study,  similar  results  would  be  expected  where  the  user 
population  was  well  educated  and  in  a profession  where  continuing  education  is  required  to  maintain 
accreditation.  Inferences  about  the  problems  inherent  with  remotely  delivering  complex  materials  can  also  be 
made.  Further  study  would  be  beneficial  to  track  whether  the  rapid  uptake  in  Internet  participation  is  extending 
into  sparsely  settled  areas  without  the  higher  bandwidth  found  in  urban  centres. 
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Abstract:  The  Web  is  argued  to  be  the  next  great  technological  tool  to  be  used  in  the  delivery  of 
educational  materials.  The  success  of  virtual  classrooms  will  depend  on  a number  of  issues 
including  pedagogy  and  the  appropriate  exploitation  of  technology.  This  paper  describes  the 
building  of  a Web  site  and  experiences  in  using  such  a site,  to  support  collaborative  learning 
between  students,  experts  and  the  course  co-ordinator.  Active  Server  Pages  and  database 
technology  running  on  Microsoft  Internet  Information  Server  was  used  to  create  the  system. 
Students  wrote  assignments  on  the  Web  that  were  reviewed  by  peers,  experts  and  course  co- 
ordinator. The  course,  based  on  modem  educational  principles,  was  found  engaging  by  the  students 
but  was  under-utilised  by  the  peer  and  export  groups.  Also,  the  use  of  the  Internet  as  a mechanism 
to  stimulate  open  discussion  appears  to  change  the  way  in  which  papers  are  written  and  reviewed. 


Introduction 

The  use  of  the  World  Wide  Web  as  a mechanism  to  create  virtual  learning  environments  and  to  deliver  other 
educational  resources  (such  as  lecture  notes)  is  heralded  as  the  next  great  educational  tool  that  will  solve  many 
problems  and  make  education  cheaper  and  more  accessible  to  a larger  community.  It  is  often  argued  that  the  World 
Wide  Web  can  facilitate  learning  as  the  technology  allows  the  easy  integration  of  many  disparate  forms  of  media 
and  communication  technologies  into  a single  delivery  mechanism  (Dwyer,  Barbieri,  & Doerr,  1995).  During  the 
past  few  years  there  has  also  been  an  explosion  of  the  use  of ’'Virtual  Classrooms".  However,  Winn  (1998)  argues 
that  "simply  converting  traditional  university  courses  to  web  pages,  or  worse,  just  putting  lecture  notes  on  a Web 
page,  is  a serious  mistake".  A search  using  Infoseek  (www.infoseek.com)  with  the  key  word  "virtual  classroom" 
returned  13  390  hits.  Many  of  these  relate  to  the  use  of  the  technology  and  not  on  the  use  of  appropriate  learning 
theories  to  support  the  development  and  deployment  of  virtual  classrooms.  Ryder  (1996)  argues  that  "the  challenge 
for  the  postmodern  educator  is  to  discover  the  capabilities  and  natural  constraints  associated  with  distributed 
pedagogy  for  scaffolding  learners  in  the  age  of  information". 

Alexander  (1996)  argued  that  the  Web  could  be  used  as  a platform  for  course  delivery  as  hyperlinks  closely  map  to 
the  structure  of  the  knowledge  that  it  represents  (see  Kearsley,  1988),  allows  interactivity  and  collaborative 
authoring  of  content,  and  also  integrate  the  learning  experience.  However,  that  authors  also  contends  that  new 
adopters  of  technology  often  concentrate  on  the  technology  and  not  on  the  learning.  Presentation  of  content  is 
insufficiency  to  promote  learning  (Winn,  1998). 

In  an  extensive  study,  Saljo  (1979)  found  that  learning  can  be  divided  into  five  categories  that  include,  quantitative 
increase  in  knowledge,  memorisation,  skill  and  methodological  acquisition,  making  sense  or  abstraction  of  idea  and 
interpretation  and  understanding  of  concepts.  Schank  & Cleary  (1995)  argue  that  teaching  frameworks  for  learning 
should  include  simulation,  incidental  learning,  reflection,  exploration  and  include  case-based  scenarios. 
Constructivists  argue  that  knowledge  and  understanding  are  actively  constructed  by  the  learner,  not  passively 
received  from  the  environment,  and  socially  constructed  through  experience  with  the  physical  world  and  social 
interactions  leading  to  consensus  (Piaget  1929,  Ausubel  1968,  1977,  Vygotsky  1978,  Driver  & Erickson  1983,  von 
Glasersfeld  1989). 

The  use  of  Internet  and  database  technologies  allow  for  the  easy  development  of  resources  where  learners  can  easily 
create  Web-based  content,  critically  evaluate  the  submissions  of  other  students  and  interact  using  discussion  fora. 
However,  the  sociological  effect  of  creating  public  content  is  not  clearly  understood. 
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In  this  paper  I will  describe  a system  that  was  created  to  teach  a fourth  year  biology  course  where  students 
investigated  topics  and  reported  their  findings  on  the  Internet,  and  thereafter  discuss  the  use  of  the  system.  The  basic 
question  posed  is:  Can  technology,  and  especially  the  use  of  the  Internet,  be  utilised  in  such  away  that  would  allow 
individuals  to  work  on  theirs  own  yet  have  discussion,  with  and  support  from,  their  peers,  other  postgraduate- 
students,  supervisors  and  experts? 


Course  conception 

The  six-week  course  consisted  of  three  topics  related  to  the  use  of  computers  in  education.  Apart  from  the  students 
and  course  co-ordinators,  four  experts,  external  to  the  University  and  a number  of  postgraduate  students  participated 
in  the  course.  A schedule  outlining  the  topics  and  source  material  was  provided.  Each  topic  included  a theoretical 
component  (writing  of  an  essay),  discussion  period,  a mini  project  and  a face-to-face  final  discussion.  Some  core 
source  material  was  provided  for  each  topic  but  it  was  expected  that  students  would  find  additional  information. 
Essay  content  was  entered  into  database  tables  using  Web-based  forms.  The  content  was  added  as  blocks  of 
information  that  the  writer  could  reorder.  During  the  discussion  period  the  course  co-ordinator,  postgraduate 
students  and  experts  attached  comments  to  the  student  essays  (using  Web-based  forms)  and  the  discussion  forum. 
Mini-projects  were  used  to  put  what  the  students  learned  during  the  theory  component  into  practice.  In  the  face-to- 
face  meeting  students  and  course  co-ordinators  evaluated  the  learning  that  had  taken  place  during  the  two  weeks  and 
attempted  to  integrate  the  different  elements  under  discussion.  The  design  of  the  course  attempted  to  implement  the 
five  learning  activities  reported  by  Saljo  (1979)  and  used  constructivist  methods  where  the  teacher  acted  as  co- 
leamer  with  the  students  constructing  their  own  knowledge  both  as  individuals  and  as  part  of  a larger  group. 

Students  were  evaluated  using  number  of  different  criteria  that  include  finding  of  relevant  information,  quality  of 
their  papers,  interaction  with  peers  and  evaluators,  quality  of  their  Mini-projects,  and  a portfolio  of  work.  The 
portfolio  contained  any  three  pieces  of  work  undertaken  in  the  course,  any  other  information  by  independent 
research,  additional  notes  or  Web  pages  that  they  authored,  a personal  evaluation  of  the  course  and  what  they  had 
learnt  in  the  course. 


Technology 

Database  technology  allied  to  a Web  server  was  used  to  create  resources  required  for  the  course 
(http://www.nu.ac.za/biology/education).  Microsoft  ® Active  Server  Pages  (ASP),  a server-side  scripting 
environment,  can  be  used  to  create  and  run  dynamic,  interactive  Web  server  applications.  With  ASP,  Web 
documents  are  created  from  HTML  pages,  script  commands,  and  ActiveX  components.  This  technology  is  easy  to 
develop,  modify  and  deploy.  The  dynamic  binding  of  database  contents  to  the  creation  of  ASP  pages  uses 
Microsoft  internet  Information  Server  and  Data  Access  Components  (MDAC)  that  integrates  information  from  a 
variety  of  sources  including  relational  (SQL)  and  non-relational.  MDAC  consists  of  ActiveX  '^'^ata  Objects  (ADO) 
and  the  Remote  Data  Service  (RDS),  Microsoft  OLE  DB  Provider  for  ODBC,  Open  Database  Connectivity 
(ODBC),  and  ODBC  drivers.  The  server-side  scripting  environment  uses  VBScript  or  JScript  that  is  executed  on  the 
Web  server.  The  ASP  technology  therefore  allows  delivery  of  pure  HTML  code  to  the  client  that  contains 
information  stored  in  database  tables  on  the  server  and  provides  an  easy  method  to  update  server-based  databases 
from  content  entered  into  forms  on  the  client  Web  browser. 

The  Web  site  for  the  course  was  divided  into  a number  of  sections  that  included  an  Introduction,  Schedule, 
Discussion  Forum,  Data  Entry,  Papers  and  Participant  Information.  All  sections,  except  the  Data  Entry  component, 
were  open  to  all  visitors.  The  Introduction  and  Schedule  section  were  coded  using  Microsoft  FrontPage  while  the 
other  parts  used  ASP  linked  to  a simple  Access  database  (Fig,  1 ). 

The  database  was  used  for  two  activities:  entry  of  data  (essays  content  and  comments)  and  the  discussion  forum.  A 
record,  in  the  Users  table,  was  created  for  each  user  that  included  the  email  address  and  password  used  to  uniquely 
identify  each  user.  The  Group  field  served  to  identify  the  user  as  either  a student,  postgraduate,  expert  or  course-co- 
ordinator. The  other  fields  in  this  table  were  used  to  display  information  on  the  Participant  Information  section. 
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Figure  1.  Database  showing  relationships  between  tables  and  unique  keys  (bold). 


In  the  Data  Entry  section,  users  entered  the  system  using  their  email  address  and  password.  User  could  then  edit 
their  personal  information  or  access  the  papers.  The  Papers  links  worked  differently  depending  on  the  group  to 
which  the  user  belonged.  Student  could  add  new  papers  or  edit  existing  ones.  Papers  were  entered  as  a number  of 
sections  that  could  be  ordered.  Other  members  could  view  contents  of  papers  and  add  comments  to  each  section. 
Records  in  the  Pages  table  (Fig.  1)  represent  papers  created  by  student  users.  Sections  of  papers  were  stored  as 
records  in  the  PaperPart  table.  Therefore,  each  paper  consisted  of  a Page  record  and  a number  of  PageParts  records. 
Comments,  related  to  each  section  of  a paper,  were  stored  in  the  Comments  table. 


The  Discussion  section  of  the  Web  site  used  the  Discussion  table  (Fig.  1).  Here  the  screen  was  divided  into  two 
main  sections.  On  the  left  a list  of  topics  was  displayed  in  a tree  format  using  Java  applet  (diputree,  from  dipu  at 
http://www.dipu.com/root.html).  Instruction  on  how  to  use  the  system,  data  entry  and  display  of  discussion  topics 
were  displayed  on  the  right.  A non-Java  view  of  the  discussion  topics  was  also  available.  The  Discussion  table 
contained  fields  that  allowed  the  records  to  be  sorted  and  grouped  according  to  the  discussion  thread  and  only  one 
level  was  provided  for  each  discussion  topic. 


In  the  Papers  section  both  the  essays  and  associated  comments  were  displayed.  Therefore  the  Papers,  PaperParts  and 
Comments  tables  were  used  to  display  the  HTML  pages  associated  with  each  paper. 


ASP  coding 

To  illustrate  the  ease  of  using  ASP  the  code  for  the  non-Java  view  of  the  Discussion  topics  is  given  in  Fig.  2.  The 
symbols  ‘<%’  and  ‘%>’  denote  the  start  and  stop  of  the  script.  Two  subroutines,  AddLinsLinks  (Fig.  2,  O)  and 
AddLine  (Fig.  2,  ©),  were  used.  The  AddLine  routine  displays  the  Discussion  topic  while  the  AddLinelinks  adds  all 
the  records  associated  with  that  topic.  To  create  the  content  for  the  page  a Record  Set  object  is  created  on  the  server 
(Fig.  2,  ©),  database  opened  using  standard  SQL  and  for  each  unique  record  the  topic  link  and  associated  topic 
threads  generated  using  the  two  sub-routines.  Finally  the  database  is  closed  (Fig.  2,  O)  and  the  end  page  HTML  tags 
added. 
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<%  O Sub  AddLineLinks ( DiscussionID  ) 

set  ra  = Server .CreateObject ( "ADODB .Recordset ” ) 

ra.Open  "select  * from  Discussion;",  "DSN=EDUCATION" , adOpenKeyset,  adLockOptimistic 
ra. Filter  = ' "LinkedToID  = " + CStr (DiscussionID) 
if  (Err. Number  > 0)  then 
Err .Clear 
else 

if  not  (ra . RecordCount  = 0)  then 
while  not  ra.EOF 

% > &nbsp ; &nbsp ; &nbsp ; &nbsp ; &nbsp ; 

<a  href  = "showtopic  .asp?ids=<%-  CStr  (ra  ( "DiscussionID"  ))%>"  ><%=  ra  ( "Title" ) %> 

(<%=ra  ( "Name" ) %>)  </axbr> 

<%  ra.MoveNext 

wend 
end  if 
ra . Close 

set  ra  = nothing 
end  if 
End  Sub 

0 Sub  AddLine  (DiscussionID,  LinkedToID,  Name,  Title,  NewTopic) 
if  isNull (LinkedToID)  then 
LinkedToID  = 0 
NewTopic  = 1 
end  if 

%>  <a  href = " showtopic . asp?id=<%=  CStr (DiscussionID) %>" ><%=  Title%>  (<%=Name%>) </a><br><% 
AddLineLinks ( DiscussionID  ) 

End  Sub 
%> 

<html> 

<head> 

<base  targets "main" > 

</head> 

<body  link="#FF0000"  vlink^"#800000"  alink="#808080"  bgcolor="#FFFFFF"  text="#000000" 

topmargin=" 0 " lef tmargin= "2 " > 

<table  border="0"  cellpadding= " 0 " cellspacing^ " 0 " width="380"> 

<tr> 

<tdxfont  size="2"  f ace  = "VERDANA, ARIAL , HELVETICA" > 

<%  On  ERROR  RESUME  NEXT 

© set  rs  = Server . CreateObject ( "ADODB .Recordset " ) 

rs.Open  "select  * from  Discussion  WHERE  (Discussion . LinkedToId  = 0);", 

"DSN=EDUCATION" , adOpenKeyset,  adLockOptimistic 
if  (Err. Number  > 0)  then 
Err . Clear 
else 

if  not  (rs .RecordCount  = 0)  then 
while  not  rs . EOF 

AddLine  rs ( "DiscussionID" ) , rs ( "LinkedToID" ) , rs("Name"),  rs( "Title"), 
rs ( "NewTopic " ) 
rs . MoveNext 
wend 
end  if 
O rs. Close 

set  rs  = nothing 
end  if 

%>  </Font> 

</td> 

</tr> 

</table> 

</body> 

</html> 

Figure  2.  Example  ASP  code  used  to  generate  non-Java  view  of  Discussion  topics. 
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Results  and  Discussion 


The  objective  of  the  study  was  to  investigate  the  use  of  modem  Internet  technologies  to  build  a virtual  classroom 
where  students  could  submit  papers  for  evaluation  and  discussion  by  peers  and  experts.  Use  of  Active  Server  Pages 
allied  to  a simple  database  was  used  to  create  the  Web  environments  where  students  submitted  papers  for  discussion, 
peers  and  experts  reviewed  the  papers  and  all  groups  had  access  to  a discussion  forum.  Each  of  these  three  aspects 
will  be  discussed  separately. 

Microsoft’s  ASP  technology  and  Internet  Information  Server  provided  an  extremely  powerful  environment  for  the 
development  of  HTML  code  that  interacts  directly  with  databases.  Using  VBScript  and  a simple  Access  database  it 
was  very  easy  to  create  the  resources  required  for  the  course.  While  the  coding  of  ASP  can  be  tedious  at  times,  it  is 
not  difficult  to  master  or  implement.  All  that  is  required  is  a fundamental  understanding  of  database  structures,  basic 
SQL,  VBScript  and  HTML  tags. 

Students  found  it  very  easy  to  gain  access  to  the  site,  and  while  there  were  some  initial  problems  with  the  system, 
were  able  to  use  the  form-based  interface  quickly  to  submit  their  papers.  The  initial  problems  related  to  how  parts  of 
papers  had  to  be  entered  into  the  system,  students  requiring  a clearer  explanation  from  the  course  co-ordinator  and  a 
postgraduate  student  of  how  to  submit  papers.  Students  suggested  that  papers  could  consist  of  more  that  one  level, 
which  would  allow  individual  sections  to  be  submitted  in  smaller  blocks.  While  this  was  a fairly  good  idea  it  was 
rejected  as  it  would  require  major  changes  to  the  database  structure  and  further  complicate  the  system  of  data  entry. 
While  using  the  core  reference  material  to  construct  their  papers,  students  also  used  both  on-line  and  library 
references.  Students  appeared  to  enjoy  the  structure  of  the  course  and  found  the  mini-projects  valuable  and  allowed 
them  to  put  some  of  the  theory  they  had  learnt  into  practice.  The  inclusion  of  a face-to-face  tutorial  for  each  section 
of  the  work  was  valuable.  Here  students  and  the  course  co-ordinator  discussed  and  evaluated  what  the  students  had 
learnt.  These  discussions  also  helped  the  course  co-ordinator  in  identifying  problems  with  the  system  correcting 
student’s  misconceptions. 

The  use  of  the  system  by  postgraduate  students  (peer  review)  was  less  than  expected.  While  some  postgraduate 
students  made  frequent  contributions,  others  did  not  (the  four  students  made  15,  7,  1 and  0 comments  respectively). 
None  of  the  postgraduate-students  experienced  any  problems  in  using  the  system. 

Four  experts  were  invited  to  participate  in  the  course.  Of  these,  two  were  from  South  Africa  (one  at  Durban),  one 
from  the  USA  and  one  from  Australia.  The  input  from  the  experts  was  disappointing.  While  two  submitted 
comments,  they  did  not  use  the  system  but  emailed  them  to  the  course  co-ordinator.  Students  appeared  to  appreciate 
these  comments  and  reported  that  they  further  advanced  their  understanding.  The  lack  of  use  by  experts  of  the 
system  is  interesting.  While  there  may  be  many  reasons,  such  as  time  and  commitment,  in  supporting  a course  for 
which  there  is  no  direct  reward,  the  evaluation  of  student  papers  in  a public  forum  (the  Web)  is  very  different  from 
the  traditional  reviewing  process  that  is  normally  only  between  teacher  and  student.  Also,  the  course  co-ordinator 
found  the  process  of  reviewing  of  papers  on  the  Web  very  different  from  grading,  or  evaluating,  a term  paper.  Such 
comments  could  no  longer  be  briefly  written  in  the  margins  of  the  paper,  but  full  explanations  had  to  be  carefully 
constructed  and  formulated.  Therefore,  comments  on  paper  on  the  Web  required  more  care  and  additional  time. 

The  Discussion  Forum  was  not  very  successful.  At  first  students  posed  questions  via  email  to  the  course  co- 
ordinator. During  one  of  the  face-to-face  meeting  a student  suggested  that  the  Discussion  Forum  be  used  for  more 
philosophical  discussion  on  the  Web.  While  the  group  agreed  that  this  would  be  a good  idea,  a thread  started  by  the 
one  student  only  contained  one  additional  comments  that  was  made  by  the  co-orrdinator.  Student  did,  however, 
appear  to  refer  to  the  topics,  for  example  dates  for  meetings  submitted  by  the  co-ordinator.  However,  they  were  not 
consistent  in  this  activity  and  consequently  all  important  course  information  was  sent  by  email  to  the  students.  The 
failure  to  use  the  Discussion  Forum  by  the  students  resulted  in  a steady  stream  of  email  message,  from  students, 
postgraduates  and  experts  to  the  course  co-ordinator.  Those  questions  that  appeared  to  be  universal  were  answered 
using  the  Forum  but  many  were  answered  individually  resulting  in  more  time  being  spent  using  email  during  the 
course. 

Collectively,  these  observations  highlight  a number  of  important  issues.  During  the  past  number  of  years  many 
institutions  have  created  virtual  classrooms.  Authors  such  as  Winn  (1998)  and  Ryder  (1996)  argue  that  such  courses 
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will  succeed  only  when  supported  by  appropriate  pedagogy.  However,  the  comment  made  by  Ryder  that  we  need  to 
"discover  the  capabilities  and  natural  constraints"  associated  with  such  courses  delivered  on  the  Web  appears  not  yet 
answered.  A fundamental  question  remains:  "Does  writing  of  course  assignments  and  assessment  on  the  Web,  which 
is  freely  available  to  all  visitors  to  the  site,  fundamentally  change  the  way  in  which  students  approach  their  papers 
and  the  way  in  which  peers  and  experts  evaluate  and  report  on  such  content?".  Students  appear  to  adapt  rapidly  to 
the  use  of  new  technology  and  are  able  to  use  it  successfully  to  assemble  their  papers.  However,  the  involvement  of 
peer  and  expert  review  appears  to  be  problematic.  While  both  of  these  groups  could  make  valuable  contributions  to 
student  learning  in  a distributed  system,  neither  gains  any  direct  reward.  Also,  there  is  a crucial  difference  in 
evaluation  of  papers  on  the  Web:  no  longer  is  the  process  private  and  confidential,  but  open.  Academic  review  has 
for  centuries  been  cloistered  in  anonymity  and  the  use  of  the  Internet  could  change  this.  Such  a basic  shift  in 
evaluation  may  be  required  if  we  are  to  use  all  the  power  of  the  Web  to  teach,  but  as  yet  we  do  not  have  the 
experience  or  wisdom  to  do  this  wisely. 

In  conclusion,  the  development  of  a virtual  classroom  where  students,  peers,  experts  and  course  co-ordinator  can 
interact  is  easy  using  Active  Server  Pages  and  database  technology.  Design  of  a Web-based  course  on  sound 
educational  principles  appears  to  provide  students  with  tools  to  submit  papers  and  receive  comments.  However,  the 
use  of  the  Web  for  open  assessment  changes  the  way  in  which  such  activities  are  conducted.  For  the  Web  to  succeed 
as  a platform  for  the  delivery  of  course  content  and  the  evaluation  of  student  work  we  need  to  understand  more 
clearly  the  move  away  from  anonymity  to  open  truthful  and  insightful  discussion  and  criticism. 
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The  forms  of  support  needed  by  preservice  teachers  in  the  practicum  setting  relate  mainly  to 
access  to  relevant  information,  resources,  guidance  and  support  through  communication  with 
university  instructors  and  competent  peers.  Too  often  these  kinds  of  support  are  lacking  for 
students  on  teaching  practice  in  schools.  This  paper  describes  the  development  of  a web-site 
where  students  can  access  a variety  of  support  structures  which  enable  them  to  reflect  on  their 
role  and  performance  as  teachers  within  a resource-rich  and  fully  supported  online 
environment.  Through  the  website,  student  teachers  can  access  resources  such  as:  lesson 
plans,  samples  of  teaching  methods  and  activities,  links  to  information  sources,  on-line  and 
downloadable  teaching  materials,  and  discussion  facilities  to  enable  students  to  post  problems 
and  queries  to  peers  and  lecturers  and  to  receive  feedback. 


A Reflective  Approach  to  Practicum  Support  for  Preservice  Teachers 

I have  come,  over  the  years,  in  spite  of  all  the  reform  agendas,  to  believe  that  the  best  we  can  do  in 
teacher  preparation  programs,  through  a variety  of  courses  and  clinical  experiences  in  intentionally 
selected  schools,  is  to  help  academically  able  and  socially  committed  students  enter  teaching  with 
constructive  dispositions  and  skills  relating  to  young  people,  curriculum  content,  pedagogy,  and 
the  power  of  collective  thought;  well-developed  habits  of  observation  and  reflection;  reasonable 
confidence  and  an  understanding  that  they  are  entering  a process  of  learning  something  important 
every  day,  working  toward  the  largest  possibilities  they  can  imagine.  (Perrone,  1997,  p.  649) 

In  today’s  overstretched  curriculum,  and  with  society’s  increasing  demands  for  the  teaching  of  a growing  body 
of  new  knowledge,  a competency-based  approach  for  learning  to  teach  is  obsolete.  Students  cannot  be 
competent  in  content  and  skills  that  are  rapidly  changing,  or  may  not,  as  yet,  even  exist.  As  Perrone  (1997) 
argues,  it  is  better  to  provide  students  with  generic  and  reflective  skills  that  assist  them  in  their  continued 
learning  of  any  new  enterprise.  Connecting  these  generic  thinking  skills  to  the  context  in  which  they  occur  is 
essential.  If  university  programs  do  not  do  this  they  are  likely  to  ‘prepare  teacher  technicians  rather  than 
reflective  professional  educators’  (Boyd,  Boll,  Brawner&  Villaumer,  1998,  p.  61). 


Abstract: 


Reflection  is  one  aspect  of  a complex  number  of  interrelated  functions  which  contribute  to  task  performance 
(Ridley,  1992),  an  aspect  which  is  gaining  increased  attention  in  recent  years  after  almost  disappearing  from 
consideration  under  the  influence  of  learning  models  based  on  behaviorism  (von  Wright,  1992).  Boud,  Keogh 
and  Walker  (1985)  define  reflection  as:  ‘those  intellectual  and  affective  activities  in  which  individuals  engage 
to  explore  their  experiences  in  order  to  lead  to  new  understandings  and  appreciations’  (p.  19).  These  authors 
stress  that  such  reflection  must  not  occur  solely  at  the  unconscious  level:  ‘It  is  only  when  we  bring  our  ideas  to 
our  consciousness  that  we  can  evaluate  them  and  begin  to  make  choices  about  what  we  will  or  will  not  do’ 
(p.  19).  Kemmis  (1985)  points  out  that  we  do  not  reflect  in  a vacuum:  ‘We  pause  to  reflect  ...  because  the 
situation  we  are  in  requires  consideration:  how  we  act  in  it  is  a matter  of  some  significance’  (p.  141). 

Many  theorists  see  reflection  as  both  a process  and  a product  (Collen,  1996;  Kemmis,  1985),  and  that  it  is 
action  oriented  (Kemmis,  1985).  Knights  (1985)  contends  that  reflection  is  not  the  kind  of  activity  which  its 
name  suggests — a solitary,  internal  activity — ^but  a two-way  process  with  the  aware  attention  of  another  person: 
‘Without  an  appropriate  reflector,  it  cannot  occur  at  all’  (p.  85).  This  view  is  strongly  supported  in  the  literature 
by  others  who  point  out  that  reflection  is  a social  process  (Kemmis,  1985),  and  that  collaboration  on  tasks 
enables  the  reflective  process  to  become  apparent  (von  Wright,  1 992). 

Another  important  function  of  reflection  is  that  it  enables  the  learner  to  compare  his  or  her  performance  or 
understanding  to  an  expert  in  the  field  (Candy,  Harri-Augstein,  & Thomas,  1985;  Collins,  1988;  Collins, 
Brown,  & Holum,  1991).  Collins,  Brown  and  Newman  (1989)  have  also  pointed  out  that  it  is  important  for 
students  to  be  able  to  compare  their  performance  with  others  at  various  levels  of  expertise.  Access  to  expert 
performances  and  the  modelling  of  processes  has  its  origins  in  the  apprenticeship  system  of  learning,  where 
students  and  craftspeople  learned  new  skills  under  the  guidance  of  an  expert  (Collins  et  al.,  1989).  Important 
elements  of  expert  performances  are  found  in  modem  applications  of  the  apprenticeship  model  such  as 
internship  (Jonassen,  Mayes,  & McAleese,  1993),  and  case-based  learning  (Riesbeck,  1996).  Such  access 
enables  narratives  and  stories  to  be  accumulated,  and  invites  the  learner  to  absorb  strategies  which  employ  the 
social  periphery  (legitimate  peripheral  participation)  (Lave  & Wenger,  1991).  Expert  performances  and  the 
modelling  of  processes,  allow  students  to  observe  and  reflect  upon  a task  before  it  is  attempted.  Such  reflection, 
one  might  argue,  is  only  possible  in  a learning  environment  which  provides  appropriate  supports  and 
communication  channels  to  enable  reflective  learning  to  occur.  And  yet  the  typical  experience  of  preservice 
teachers  on  professional  practice  is  one  of  isolation,  divorced  from  the  support  structures  of  their  university 
environment,  and  without  communication  channels  to  their  peers. 

The  purpose  of  this  paper  is  to  outline  the  development  of  a web-based  resource  that  provides  reflective  support 
and  communication  to  assist  preservice  teachers  learn  about  teaching  in  the  context  of  their  school  practice. 
Early  childhood,  primary  and  secondary  education  student  teachers  enrolled  in  Bachelor  of  Education  and 
Graduate  Diploma  of  Education  courses  at  Edith  Cowan  University  (ECU)  are  required  to  attend  between  10 
and  1 8 weeks  of  school  practice  during  their  training.  These  practices  vary  from  continuous  blocks  of  time  in  a 
school  to  distributed  practice  where,  for  example,  students  attend  a half-day  a week  for  one  school  term.  In  the 
context  of  reducing  University  and  Faculty  budgets,  coupled  with  increased  pressure  on  academic  staff  to 
increase  their  research  output,  there  has  been  a dramatic  reduction  in  university  staff  involvement  in  the 
supervision  of  students  on  school  practice.  This  has  necessitated  developing  alternative  approaches  to  assisting 
students  on  school  practice. 

This  project  set  about  to  provide  an  improved  framework  of  support  for  student  teachers  during  their 
involvement  in  school  practice  by  providing  them  with  a range  of  resources  to  increase  content  and  skill 
knowledge,  and  to  enable  them  to  reflect  on  their  teaching  practice.  Initially,  the  content  focus  centered  on 
teaching  mathematics,  however,  the  generic  nature  of  the  skills  being  developed  meant  that  these  abilities  could 
easily  be  transferred  to  other  areas  of  the  school  curriculum.  To  support  and  develop  generic  teaching  skills  it 
was  felt  that  students  would  benefit  by  having  access  to  rich  sources  of  lesson  ideas,  particularly  those  type  of 
lessons  that  reflected  constructivist  pedagogy;  and  guidance  and  support  through  communication  with  content 
experts  and  their  peers.  In  addition  to  their  supervising  teacher,  students  would  be  adequately  supported  in  their 
practicum  settings  by  having  immediate  access  to: 

• Quality  curriculum  materials  to  guide  lesson  planning; 

• Examples  of  exemplary  teaching  to  enable  students  to  compare  themselves  to  ‘experts’; 
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• Examples  of  lesson  activities  across  a range  of  teaching  strategies  and  content  areas; 

• Templates  with  which  to  build  lesson  plans  and  ideas,  and  teaching  materials  for  use  in  lessons;  and 

• Open  communications  channels  to  discuss  problems  and  difficulties  with  their  peers  and  lecturing  staff. 

Providing  support  along  these  lines  could  be  effected  through  an  appropriately  designed  Internet-based 
database  and  information  delivery  system.  The  process  of  using  the  Internet  to  communicate  between  student 
teachers  on  school  practice  and  university  supervisors  has  been  reported  (Casey  1 994;  Hutchinson  & Gardner 
1997).  Recommendations  from  these  reports  include  extending  the  process  of  communication  to  students’  peers 
and  school  supervisors.  This  project  provides  such  a system  with  the  construction  of  a WWW  site  with  the 
following  components  and  attributes: 

• A database  of  prepared  lesson  materials  accessible  through  a simple  search  engine; 

• Short  video  clips  of  teaching  approaches  and  activities  to  stimulate  students’  ideas  when  creating  and 

preparing  lesson  formats  and  plans; 

• Links  to  the  various  information  sources  useful  to  students; 

• Answers  and  suggestions  regarding  known  problems  faced  by  students  on  school  practice; 

• Online  and  downloadable  teaching  materials;  and 

• Online  discussion  facilities  enabling  students  to  post  problems  and  queries  to  peers  and  to  staff,  and  to 
receive  feedback. 

All  entering  students  at  ECU  are  provided  with  an  email  address  and  access  to  the  Internet  through  the 
university  modem  pool.  The  vast  majority  of  students  will  have  access  to  the  WWW  in  the  schools  in  which 
they  will  be  doing  their  practicums.  The  benefits  provided  by  this  resource  include: 

• Immediate  access  to  required  information; 

• Ability  to  collaborate  in  a virtual  community  during  the  practicum  and  be  relieved  of  the  sense  of 
isolation  so  often  experienced; 

• Ability  to  contribute  to  the  database  by  posting  successful  materials  of  their  own; 

• Ability  to  share  the  information  passed  in  the  discussion  sessions  without  having  to  formally  participate; 

• Problems  and  queries  which  surface  and  are  resolved  during  each  practice  will  be  added  to  the  database 
of  known  problems  and  solutions  for  reference  by  subsequent  students;  and 

• Problems  and  queries  will  provide  a significant  resource  for  lecturers  and  students  in  their  pre-planning 
for  school  experience. 


Using  the  Resource 

To  ensure  the  successful  application  of  the  WWW  resource,  students  are  made  familiar  with  its  information  and 
organisation  before  going  on  school  practice.  They  gain  a sense  of  how  best  the  resource  could  be  used  and  the 
real  advantages  that  could  be  gained  through  its  use.  Student  use  of  the  resource  on  school  practice  can  be 
ensured  through  prior  activities  at  the  University  where  students  are  required  to  explore  the  resource  and  all  its 
features  and  capabilities  and  to  use  the  resource  in  simulated  practicum  conditions  by  preparing  sample  lessons. 
They  are  encouraged  to  add  materials  of  their  own  to  relevant  sections  of  the  database  and  to  learn  appropriate 
procedures  for  online  discussion  and  communication  through  computer  mediation.  School  supervisors  are  also 
informed  of  the  resource  and  encouraged  to  contribute  and  use  the  email  facility  to  communicate  with 
University  supervisors.  This  WWW  site  is  an  extensive  resource  with  the  features  and  attributes  described 
above.  The  site  incorporates  the  following  features: 

L Interface  Design:  The  WWW  site  interface  (see  Figure  1 ) reflects  the  forms  of  information  contained, 
and  provides  an  intuitive  organisational  storage  and  retrieval  structure.  Several  navigation  and  data  retrieval 
strategies  exist  which  include  hierarchical  organisation  based  on  year  groups  and  mathematics  topics  and  a 
keyword  search  engine  to  provide  access  to  the  materials. 

2.  Content  Development:  The  project  has  involved  the  development  of  lesson  activities  (see  Figure  2 for 
an  example)  across  a broad  range  of  topics  and  teaching  strategies.  There  are  at  least  160  such  elements  in  the 
database  which  provides  an  adequate  starting  resource  for  student  teachers  in  schools.  The  project  has  involved 


lecturers  in  the  department  of  mathematics  education  creating  approximately  40  lesson  activities  in  each  of  the 
categories:  pre  K-2;  3-5;  6-8;  9-12  (and  these  will  grow  as  the  students  contribute  their  own  lesson  ideas). 


Figure  1:  The  interface  of  the  web  resource 
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Figure  2;  A sample  lesson  plan 
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The  activities  reflect  the  type  of  pedagogy  encountered  in  the  University  methods  classes  and  the  directions 
advocated  in  such  documents  as  the  National  Statement  on  Mathematics  for  Australian  Schools  (AEG,  1991) 
and  the  National  Council  of  Teachers  of  Mathematics  Curriculum  and  Evaluation  Standards  (NCTM,  1989). 
Many  activities  will  have  accompanying  graphics  and  images,  including  images  from  real  classrooms  where 
the  activities  have  been  trialed. 

5.  Communications  Components:  The  development  of  a customised  discussion  program  for  the  site 
supports  the  forms  of  communication  and  discourse  described  above.  The  communication  capability  of  the  site 
enables  students  to  cross-post  messages  and  documents  providing  the  full  capacity  for  information  and 
materials  exchange.  This  is  an  essential  component  to  enable  the  ‘development  of  a rehetoric  for  interchanges’ 
(Dede,  1996,  p.  168)  which  is  so  important  for  the  effectiveness  of  the  students’  learning  support  systems. 


Outcomes  for  Students 

Student  teachers  using  this  resource  gain  a diversity  of  expected  learning  outcomes  developed  within  the 
context  of  mathematics  education.  These  primarily  relate  to  planning  for  teaching  and  applying  appropriate 
teaching  strategies  and  generic  strategies  developed  using  electronic  forms  of  information,  such  as: 

Generic  planning  skills: 

• Plan  mathematics  lessons  and  program  a series  of  lessons 

• Select  suitable  learning  experiences 

• Select  and  prepare  resources  and  contexts  and  plan  for  all  pupils’  needs. 

Generic  teaching  strategies: 

• Become  aware  of  and  apply  a range  of  teaching  strategies 

• Adapt  strategies  for  individual  pupils’  needs 

• Motivate  pupil  interest. 

Generic  information  processing  strategies: 

• Access,  create  and  evaluate  online  information 

• Communicate  online. 


Evaluation 

The  project  is  being  evaluated  formatively  with  students  throughout  the  entire  period  of  development  in  three 
ways:  by  student  consultation,  trial  implementation  and  focus  group  discussion  with  student  users.  With  a 
dynamic  and  responsive  medium  such  as  the  WWW,  formative  evaluation  is  an  ongoing  feature,  and  necessary 
changes  to  meet  the  growing  needs  of  the  preservice  teachers  using  the  resource  can  be  made  at  any  stage. 
Summative  evaluation  of  the  resource  will  be  conducted,  in  the  first  instance,  with  a qualitative  study  of  a small 
group  of  preservice  teachers.  Students  will  be  observed  accessing  the  resource,  then  interviewed.  The  purpose 
of  the  investigation  will  be  to  assess  the  impact  of  the  resource  on  students’  selection  and  preparation  of  lesson 
material,  the  teaching  strategies  employed  by  students  in  their  teaching  practice,  and  their  use  of  electronic 
media  to  seek  information  and  use  communication  technologies  to  establish  support  networks.  Further  analytic 
studies  will  be  conducted  based  on  the  findings  of  the  interpretive  research. 


Conclusion 

This  paper  describes  the  development  of  an  online  resource  to  be  used  by  preservice  teachers  while  on 
professional  practice  in  schools.  The  resource  has  been  designed  to  support  reflective  practice  by  these  students 
as  they  prepare  and  teach  their  practice  lessons.  The  value  of  reflective  practice  and  its  potential  to  be  assisted 
through  the  web  site  comes  from  several  sources.  Firstly,  the  site  enables  reflective  practice  by  providing  a 
variety  of  resources  from  which  to  gain  alternative  perspectives  on  any  teaching  task.  Secondly, 
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communication  technologies  allow  students  to  establish  communication  in  the  language  of  the  culture,  and  to 
share  stories  and  anecdotes  of  their  experiences.  Thirdly,  the  site  provides  exemplary  performance  and  the 
modelling  of  processes,  enabling  students  to  reflectively  compare  their  own  performance  to  that  of  experts. 

A resource  such  as  this  has  the  potential  to  transform  the  professional  practice  experience  from  an  isolated  and 
anxious  one,  where  students  work  with  minimal  resources  and  supports,  to  one  which  is  dynamic, 
collaborative,  resource-rich,  supportive  and  reflective. 
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Abstract:  At  the  University  of  Twente  many  courses  apply  group  based  learning  as  a 
successful  strategy  for  those  educational  settings  in  which  the  application  and  integration  of 
theory  is  the  main  theme.  Also  the  University  of  Twente  encourages  instructors  to  consider 
the  use  of  information  and  communication  technology  solutions  if  appropriate  (Coll is, 
1996).  “People,  Technology  and  Organisation-2”  is  a course  focusing  on  organisation  theory 
and  its  relevance  for  designing  business  organisations.  Students  work  in  project  groups 
studying  both  theory  and  case-studies. 

To  stimulate  the  reading  of  theory  articles  a Web-site  was  set  up.  The  whole  set  of  24 
articles  is  too  much  for  each  individual  student,  so  each  group  divides  the  articles  among  its 
members.  Every  group  has  to  submit  for  each  of  the  24  articles  sets  of  ‘question  & answer’ 
that  target  the  core  of  the  article.  The  submissions  on  a particular  article  together  form  a 
resource  for  all  groups,  though  only  accessible  for  those  groups  who  submitted  a serious 
contribution  themselves.  In  this  paper  we  report  on  the  problem  analysis  of  the  educational 
setting,  the  web-solution,  and  on  the  evaluation  results  of  the  experiment  which  took  place 
in  the  period  September  - November  1998. 


Educational  setting 

The  University  of  Twente  offers  curricula  in  technical,  social  and  business  sciences.  In  the  second  year  of 
the  School  of  Management  Studies  curriculum,  180  students  enrol  for  the  course  titled  “ People, 
Technology  and  Organisation-2”.  In  this  200  hours  course  the  students  work  in  groups  of  6 students  each. 
Two  instructors  assisted  by  seven  tutors  (senior  students)  take  care  of  lecturing  and  group  support. 

The  learning  goals  of  the  course  are  (Smit  & van  Riemsdijk,  1998): 

• Knowledge  and  insight  in  the  field  of  organisation  theory  and  design. 

• Being  able  to  apply  this  knowledge  and  insight  in  problem  solving  strategies. 

The  course  uses  the  textbook  of  Daft  (1998)  and  a set  of  articles  from  the  research  field.  In  parallel  the 
students  work  on  the  evaluation  of  case-studies.  In  the  course  four  phases  can  be  discerned,  together 
spanning  a period  of  1 0 weeks: 

In  the  first  phase,  the  main  topic  is  an  introduction  into  several  schools  of  thought  in  the  domain  of  human 
resource  management.  Starting  with  a re-examination  of  Taylorism,  students  are  introduced  to  the  work  of 
Lewin,  Me  Gregor,  Trist  & Bamforth.  Parallel,  students  are  asked  to  study  the  Udde valla  Volvo  plant  and 
the  Nummi  (Toyota/GM)  plant  with  respect  to  human  resource  aspects  of  the  production  organisation.  They 
are  invited  to  reflect  on  these  two  plants  using  the  literature.  Finally,  they  are  asked  to  suggest 
(first/preliminary)  redesign  options  for  Morgan  Cars  on  this  topic,  after  analysing  Morgan  Car  Job  design. 

The  second  phase  deals  with  contingency  theory  and  organisation  design.  Technology  and  Organisational 
environment  are  the  main  topics.  Using  studies  by  Perrow,  Thompson,  Lawrence  & Lorsch,  Woodward  etc. 
students  are  again  stimulated  to  analyse  the  Nummi  and  Ude valla  plant  in  this  respect.  The  second  half  of 
this  part  is  again  devoted  to  analysing  and  “redesigning”  Morgan  Cars. 

The  third  phase  deals  in  a similar  way  with  ‘Power,  Politics  and  Decision  making’,  focussing  mainly  on  a 
stakeholder  approach  and  of  course  on  decision  theory.  Again,  Nummi  and  Uddevalla  serve  as  a practical 
illustration  and  practice  material,  Morgan  Cars  as  a serious  effort  for  re-design.  Finally  in  the  fourth  phase 
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of  the  project,  students  are  invited  to  attempt  a full  re-design  of  the  Morgan  factory  on  all  topics  touched  so 
far  and  on  all  other  aspects  of  the  organisation  that  they  think  are  relevant  to  the  task. 

Problem  analysis 

The  first  three  phases  of  the  course  start  with  the  study  of  about  eight  rather  basic  and  theoretical  articles. 
The  concepts  and  ideas  from  these  articles  then  have  to  be  translated  into  practical  ideas  for  changing  a 
case-situation.  In  order  to  be  able  to  do  so,  it  is  essential  that  all  the  articles  have  been  studied  rather 
thoroughly.  Ideally,  students  will  interchange  insights  and  concepts  encountered  in  the  literature.  Students 
are  more  likely  to  raise  their  problem  solving  capabilities  when  working  together  with  fellow  students  with 
comparable  but  slightly  better  capabilities  (Vygotsky,  1978).  They  are  working  in  each  other's  "zone  of 
proximal  development".  This  means  that  in  some  way  or  other,  a situation  has  to  be  created  in  which 
students  that  have  read  the  same  material  can  share  their  findings.  Previously  this  was  attempted  by 
prescribing  that  at  least  two  students  in  every  group  should  read  the  same  articles  (Smit  & van  Riemsdijk, 
1998).  These  students  could  then  discuss  the  articles  amongst  themselves,  and  present  their  findings  to  the 
rest  of  their  group.  It  turned  out  however,  that  students  read  not  two  but  only  one  article  each,  blowing  this 
strategy  of  communication  about  theories  and  concepts  right  out  of  the  water.  Instructors,  as  a consequence, 
found  that  in  workgroup  sessions,  individual  knowledge  of  separate  articles  was  adequate,  but  that 
theoretical  knowledge  had  not  spread  throughout  groups  but  stayed  firmly  located  with  one  ‘expert-student’ 
per  article  per  group.  This  resulted  in  a lack  of  overview  within  the  groups  and  in  sub-standard  knowledge 
and  grasp  of  the  different  theories  and  concepts  involved  by  all  members  of  the  groups.  An  additional 
problem  would  arise  if  individual  students  would  not  have  understood  the  material  sufficiently.  Lacking 
any  possibility  of  communication  with  other  students  within  their  group  on  a specific  article  (no  one  else 
would  have  read  the  same  material)  their  problem  could  not  be  solved  and  would  spread  throughout  their 
whole  group.  Given  these  results,  obviously,  something  had  to  be  done. 

Solution  design 

It  was  concluded  that  the  decisions  on  who  was  to  read  which  article  should  be  left  to  the  responsibility  of 
the  groups  themselves.  To  alleviate  the  problems  mentioned  before  however  a web-site  was  designed.  The 
web-site  was  designed  to  solve  several  of  the  problems  encountered.  First,  to  stimulate  the  reading  of  the 
articles  each  group  was  obliged  to  submit  a contribution  through  the  Web-site  (figure  1).  This  contribution 
should  consist  of  two  relevant  questions  and  their  correct  answers  about  each  article  that  was  read.  The 
submitted  Q&A  would  be  screened  (figure  2)  by  tutors  helping  the  instructors.  They  would  accept  or  reject 
(with  some  comrhents)  the  contributions.  This  gives  the  students  full  flexibility  with  respect  to  the  planning 
and  execution  of  their  tasks,  while  at  the  same  time  making  sure  that  everybody  does  what  is  expected  of 
them. 
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Figure  1.  Web  form  for  submitting  ‘questions  & answers’. 
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Second,  once  acceptable  questions  and  answers  are  submitted,  the  student  gets  access  to  another  part  of  the 
site,  allowing  him/her  to  monitor  all  the  other  positively  assessed  contributions  about  his/her  particular 
article,  submitted  by  students  from  other  groups.  In  this  way  also  a treshold  was  built  in  to  prevent  ‘free 
rider  effects*  when  students  copy  from  other  contributions  (Ronteltap  & Eurelings,  1997).  In  the  last  step 
students  rate  the  group  submissions,  indicating  which  five  contributions  they  like  best.  The  solution  is 
described  in  figure  3. 
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Figure  2:  Tutor  evaluation  module,  also  showing  the  course  menu  on  the  leftside  of  the  screen. 


So,  while  accepting  the  fact  that  students  would  typically  read  only  one  article,  we  tried  to  provide  for  a 
workspace  where  some  form  of  communication  with  other  ‘experts*  would  be  facilitated.  Students  would 
be  ‘rewarded’  for  their  effort  a)  by  being  allowed  to  enter  the  site  and  b)  by  being  able  to  read  other  correct 
questions  and  answers  about  ‘their*  article.  Using  this  broader  knowledge,  they  would  be  more  secure  about 
the  work  they  had  done  and  be  able  to  contribute  more  to  discussions  during  their  group  meetings.  At  the 
same  time,  this  had  the  advantage  that  we  could  be  reasonably  sure  about  the  level  of  knowledge  and 
understanding  that  students  would  have  about  the  different  articles.  This,  of  course,  was  theory. 
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Figure  3.  Main  activities  with  respect  to  the  Web-Repository  of  student  reviews. 
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Technical  set-up 

Active  Server  Page  (ASP)  technology  was  used  to  implement  writing  to  and  querying  from  the  Access 
database  that  contained  all  group  contributions.  To  limit  the  access  to  the  resources  to  only  those  groups 
that  had  made  a proper  contribution,  the  membership  mechanism  of  BSCW  (Basic  Support  for 
Collaborative  Work)  was  used.  After  each  assessment  those  groups  entitled  to  access,  were  granted 
membership  to  those  workspace  folders  containing  the  HTML-file  making  a call  to  the  database  (BSCW, 
1998).  The  groups  could  also  use  BSCW  workspaces  as  a group  archive  if  they  wished  so. 

Evaluation  questions  and  methods 

The  main  goal  of  the  implementation  and  therefor  the  main  question  involves  the  value  of  the  website  in 
improving  discipline  of  reading  and  improving  insight  of  students  in  the  theory  described  by  the  articles 
(Q1,Q3).  We  were  interested  to  see  how  the  web-site  would  support  the  work  of  the  students  (Q2,  Q4),  the 
tutors  when  assessing  the  quality  of  the  questions  and  answers,  and  of  the  instructors  when  having  a look  of 
the  progress  of  the  students,  and  taking  an  overview  with  respect  to  the  quality  of  the  students 
contributions. 

With  respect  to  the  technical  side  we  wanted  to  built  and  implement  the  interactive  website  as  described. 
And,  as  no  special  training  was  given  we  were  interested  in  the  ease  of  use,  and  were  curious  to  know  if  the 
web-site  would  function  well  (Q5).  To  find  answers,  data  are  gathered  in  different  ways  with  involvement 
of  student,  instructors  and  tutors.  Our  research  questions  and  evaluation  methods  are  summarised  in  table  1 . 


Evaluation  methods 
Questions 

Student 

questionnaire 

student 

interview 

Tutor 

interview 

Instructor 

Interview 

Log 

analysis 

Q1  How  good  is  overall  student  insight  in  theory  ? 

* 

* 

* 

Q2  Did  the  intended  use  of  the  Web-site  take  place  ? 

* 

* 

* 

* 

* 

Q3  Did  student  insights  improve  as  a result  of  this  ? 

* 

Q4  Is  the  Web-repository  perceived  as  helpful  ? 

* 

* 

* 

Q5  Did  the  Web-site  technically  function  well  ? 

* 

* 

* 

* 

Table  1.  Research  questions  and  evaluation  methods. 


Results  and  conclusions 

With  only  limited  efforts  (80  hours)  a Web-repository  of  student  reviews  and  group  archives  was  set  up 
using  a database  driven  Web-site  (Q5).  From  the  database  and  logged  data  we  found  information  on  the  use 
of  the  web-repository,  table  2.  The  submission  of  questions  and  answers  was  done  by  the  groups  as  planned 
(Q2).  Most  groups  (23  out  of  28)  submitted  contributions  for  20  or  more  articles  (out  of  24).  The  students 
reported  that  they  used  more  than  30  minutes  to  read  an  article.  They  found  it  difficult  to  formulate  a good 
question  and  answer,  indicating  that  there  was  additional  reflective  effort  needed  after  having  read  the 
article  (Salomon,  1984).  The  instructors  reported  that  students  attending  the  lectures  in  general  were  better 
prepared  compared  with  the  previous  situation  (Q3,  Q4). 


1®*  phase 

2"**  phase 

3"^**  phase 

Overall 

Number  of  submissions 

246 

218 

195 

659 

Positive  assessed  submissions 

81% 

58% 

61% 

68% 

Reading  other  group  submissions 

79% 

39% 

26% 

48% 

Top  5 contributions 

44% 

16% 

n.a. 

n.a. 

Table  2.  Some  statistics  on  the  use  of  the  Web-site  of  the  course.  Mistaken  entries  (66)  in  the  database 
(empty  forms,  duplicates)  are  removed  from  statistics. 
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Overall  68%  of  the  contributions  was  assessed  positively.  The  drop  in  the  percentage  of  positive  assessed 
contributions  in  the  second  phase  was  caused  by  comments  of  one  of  the  instructors  urging  the  tutors  to  be 
a bit  more  strict  in  their  assessment.  During  the  same  meeting,  after  the  first  phase,  the  tutors  indicated  that 
it  is  sometimes  difficult  to  make  a "ok"  / "not  ok"  choice  as  sometimes  a contribution  contains  both  good 
and  bad  parts. 

The  percentages  of  students  reading  other  group  submissions  one  has  to  take  into  consideration  that  access 
was  only  granted  to  those  that  had  submitted  a good  submission  themselves.  Therefor  this  ‘reading  peer 
contributions*  percentage  can  never  be  higher  than  the  ‘positive  assessed  contributions’  percentage.  The  top 
5 competition  was  not  used  very  much.  Reading  other  groups  submission  does  not  have  a high  added  value 
as  perceived  by  the  students.  At  the  time  of  the  submissions  all  learners  are  still  in  the  orientation  phase  and 
most  have  not  yet  reached  expert  level  (Ql).  Also  time  pressure  to  move  on  to  other  deliverables  is  a reason 
for  lack  of  interest  in  this  feature.  Because  of  this  lack  of  interest  the  instructors  decided  to  drop  this  feature 
in  the  third  phase  (Q2). 

Via  a questionnaire  students  (N=83,  response=50%)  were  asked  to  indicate  which  workform  had 
contributed  to  acquiring  knowledge,  improved  insight  and  to  learning  better  to  apply  knowledge.  Compared 
with  group  discussions,  tests  and  working  lectures  that  were  applied  in  the  same  course,  the  website 
repository  received  the  lowest  score,  table  3 (Q4).  It  may  be  that  the  hoped  for  ‘peer-to-peer*  learning 
effects  [Vygotsky,  1978]  across  groups,  are  too  weak  when  the  peer  contribution  is  ‘tunneled’  through 
technology  with  no  opportunity  for  discussion.  Students  report  that  they  used  the  group  discussions  to  share 
their  individual  findings  on  the  articles  within  the  group.  It  may  be  that  working  with  the  Web  repository  is 
a good  preparation  for  this  discussion,  more  than  for  exchange  of  opinions  across  groups. 


Tests  on  theory 

Group  discussion 

Working  lectures 

Web  repository 

Acquiring  knowledge 

2.2 

2.8 

2.2 

4.3 

Improving  insight 

3.2 

2.5 

1.4 

4.2 

Applying  knowledge 

4.1 

2.2 

2.3 

4.1 

Table  3.  Average  student  appreciation  of  work  forms  applied  in  the  course  (l=most  important,  5=least 
important). 


The  web-site  user  interface  was  appreciated  as  easy  to  use  (Q5),  table  4,  even  though  we  feel  that 
improvements  are  possible.  Browsing  through  other  group  contributions  for  instance  now  takes  four 
mouseclicks  per  item.  This  will  be  reduced  to  one  mouseclick  per  item. 


Question 

Score 

It  was  difficult  to  formulate  a good  question  and  answer  set. 

2.8 

If  I wanted  to  enter  data  in  the  Web-site,  it  was  easy  to  do. 

1.9 

I had  a good  overview  of  what  I still  had  to  do. 

2.5 

It  was  fun  to  work  with  the  Web-site. 

3.7 

Table  4.  Student  appreciation  of  the  Web-site  (l=agree  very  much,  5=disagree  very  much). 

The  flexibility  to  submit  from  any  Internet  connected  pc  was  used  not  only  by  students,  but  also  by  the 
tutors  when  assessing  the  contributions.  From  the  responding  students  81%  indicated  that  they  had  used 
their  home  pc  for  some  of  the  submissions.  Analysing  submission  time  stamps  shows  that  47  % of  all 
submissions  arrived  close  before  the  deadlines  on  Mondays.  For  90  % of  the  submissions  the  time  stamp  is 
between  10.00  and  17.00  hours.  For  this  students  mainly  used  their  project  room  pc  or  other  personal 
computers  at  the  Department,  possibly  because  from  their  home  pc  connectivity  costs  are  a treshold. 

The  access  mechanism  for  the  overview  of  all  contributions  involves  the  tutors  granting  access.  This  human 
step  turns  out  to  be  error  prone.  The  failure  rate  of  6%  dropped  after  the  first  phase  towards  occasional 
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mistakes.  Still  the  automation  of  this  step  will  make  the  design  as  a whole  more  robust,  at  the  same  time 
preventing  the  good  students  from  becoming  frustrated  when  they  find  that  access  is  not  granted  by 
mistake. 


Discussion 

It  turns  out  that  the  discipline  of  reading  theory  articles  has  been  enhanced.  As  a result  of  the  need  to 
formulate  ‘question  and  answer’  combinations,  student  reflection  was  also  stimulated.  Of  all  submissions, 
68%  was  of  acceptable  quality.  But  what  is  acceptable  ? We  wanted  people  to  get  to  the  core  of  things,  but 
most  students  did  not  reach  that  level.  It  is  now  seriously  considered  that  the  students  will  have  an  optional 
course  in  their  third  year  in  which  those  interested  in  the  course  topic,  can  further  improve  their  level. 

The  hoped  for  cross-group  exchange  of  expertise  on  each  article,  was  not  adopted  by  most  students.  The 
reasons  have  to  do  with  the  effort  needed  to  read  reviews  from  more  than  20  groups,  which  would  take 
more  time  than  reading  the  article  itself  So  most  students  no  more  than  scanned  the  contributions  of  other 
groups,  resulting  in  almost  random  top  five  scores.  Also  the  mentioned  quality  aspect  of  the  submissions, 
did  not  stimulate  students  to  keep  reading. 

As  our  students  report  that  they  rely  mostly  on  the  exchange  of  expertise  within  their  own  peer-group,  we 
plan  to  redesign  the  web-repository  in  such  a way  that  it  supports  these  group  members  first  of  all.  We  will 
ask  the  students  to  submit  short  summaries  on  the  articles.  Apart  from  basic  quality  assessment  of  the 
submissions,  the  tutors  and  instructors  will  make  a shortlist  of  excellent  submissions,  adding  some  expert 
remarks  per  article  as  a conclusion.  This  way  we  hope  the  Web-repository  will  support  each  group  by 
building  a portfolio  of  article  summaries  per  group,  and  by  offering  an  efficient  shortlist  per  article  for  the 
experts  that  have  read  the  same  article. 
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Abstract 

This  paper  illustrates  how  the  Constructivist  Learning  Theory  can  be  demonstrated  through 
the  use  of  Javascript  in  a web-based  learning  environment.  The  power  of  stand-alone 
multimedia  applications  can  now  be  replicated  on  the  web  through  many  types  of  plug-ins. 
However,  they  are  tedious  and  time  consuming  to  download.  Client-side  Javascript  is  the 
answer  this  research  proposes  as  a fast,  reliable  alternative.  By  embedding  Javascript 
throughout  a web-based  tutorial  system  the  principles  of  Constructivism  guide  the  Learning 
Environment  Designer  (LED)  in  the  development  of  a web-based  tutorial  system. 


Web-Constructivism 

Constructivism  is  the  current  dominant  learning  theory  [Bruner  1966,  Hein  1995].  The  Internet  is  a well  suited 
distance  learning  media  to  support  this  methodology  . This  paper  shows  how  using  Javascript  the  following  sub- 
set of  Constructivist  principles  [Boyle  1 997]  can  be  adapted  to  a web-based  learning  environment: 

1 . Support  the  active  construction  of  knowledge  in  the  mind  of  the  student. 

2.  Embed  learning  in  a social  experience. 

3.  Use  multiple  modes  of  representation  (multimedia). 

4.  Support  the  sense  of  ownership  and  a voice  in  the  learning  process. 

5.  Give  the  opportunity  for  reflection.  (Ideas  must  be  revisited  for  learning  to  occur.) 

6.  Support  interactivity  with  feedback,  thus  promoting  motivation. 

A more  detailed  explanation  of  Constructivism  can  be  found  at:  http://skip.mdx.ac.uk/Report/Chaptcr2.htm 
Two  types  of  interactivity  and  feedback  are  offered  to  the  student  in  this  web-based  tutorial  system;  1)  student 
and  system  2)  student  and  participants.  The  first  is  interaction  between  the  student  and  the  system.  This  can 
result  in  a navigational  movement  through  the  environment  by  clicking  buttons  / links  .Feedback  prompts 
(message  boxes)  can  alert  the  student  about  their  progress  through  the  content  of  the  tutorial.  The  second  form 
of  interactivity  is  between  the  participants  in  the  learning  system.  One  example  of  this  is  collaboration  between 
peers,  while  the  other  is  communication  between  the  student  and  tutor  / subject  expert. 

Javascript  Tools  to  Illustrate  Web-Constructivism 

Although  there  are  many  multimedia  options  on  the  Web,  most  require  specialist  software  which  can  be  a slow, 
time  consuming  process  of  down-loading  plug-in  applications.  The  Minimalist  approach  suggests  using  the 
simplest  method  to  attain  the  desired  goal  [Boyle,  1997].  Using  a client-side  scripting  language,  like  Javascript, 
supports  this  approach  in  web-based  learning  environment  design.  The  basic  specification  for  viewing  these 
examples  is  a Java  capable  browser  that  is  set  to  view  graphics  / animated  .gifs.  An  alternative  method  of 
distribution  would  be  to  place  the  files  on  a CD  with  a copy  of  a browser  that  would  allow  the  information  to  be 
accessed  locally. 

This  research  supports  a problem-solving  seminar  teaching  strategy  which  encourages  an  active  use  of  the 
students  prior  knowledge  to  argue  with  his  peers  to  produce  a resolution.  There  are  several  steps  to  this  protocol 
[Jackson  1996,  Bensusan  1995]: 

1 . A starting  problem  is  introduced  with  its  related  real-world  context. 
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2.  Drawing  upon  prior  knowledge  and  on-line  research  sources  (i.e.-  digital  libraries,  museums, 
universities,  etc.)  help  form  a preliminary  conclusion. 

3.  Collaborate  and  confront  peers  to  reaffirm  or  reassess  that  solution. 

4.  Compare  this  answer  to  a sample  solution  produced  by  the  tutorial  subject  expert. 

5.  Reaffirm  or  reassess  the  answer  based  on  that  information 

The  main  categories  of  functionality  for  using  Javascript  in  a web-based  learning  system  are  discussed 
chronologically  as  they  would  appear  throughout  the  lesson: 

1 . passwords 

2.  personal  name  entry 

3.  multimedia  content 

4.  new  browser  windows 

5.  alert  boxes 

6.  self  assessment 

7.  e-mail  tools 

8.  graphically  representing  feedback 
7.  passwords 

Security  is  an  issue  for  any  on-line  learning  system.  There  are  three  levels  of  password  protection  [11  available 
with  Javascript.  An  Alert  Box  prompts  the  student  for  passwords  to  enter  the  lesson.  The  collaborative  nature  of 
the  web  supports  the  second  Constructivist  principle  (embed  learning  in  a social  experience)  allowing 
participants  on-line  social  exchange  via  e-mail,  news  groups,  computer  mediated  conferencing  (CMC),  Internet 
Relay  Chat  (IRC)  rooms  and  video  conferencing  [Laurillard  1996  & Brunner  1997]. 

2.  personal  name  entry 

To  personalise  the  lesson  introduction  of  the  start  problem  Javascript  can  be  used  to  prompt  the  student  for  their 
name  j^lwhich  is  then  placed  at  the  top  of  the  page  for  each  use.  This  reinforces  ownership  in  the  learning 
process.(figure  1.1) 


J.  multimedia  content 

A number  of  Javascripts  are  available  to  present  the  start  problem  and  related  content  which  make  use  of 
multimedia  to  support  multiple  modes  of  representation.  User  interactivity  with  the  material  encourages 
motivation  from  the  participant  which  supports  the  sixth  Web-Constructivist  principle  listed  above.  Information 
can  be  in  the  form  of  text,  still  images,  animated  .gifs  (slide  show),  .avi  movie  files  converted  to  animated  .gifs 
(using  GIF  Constructor  Set  software  - http://www.mindworkshop.com/  ),  a series  of  .gif  images  taken  to 
simulate  a circular  pan  effect  [ 1 61  and/or  sound  as  .wav  and  midi  files. The  graphics  can  be  image  maps  which 
have  clickable  hyperlinks  to  increase  interactivity.  Other  multimedia  tools  requiring  special  plug-ins  are  not 
recommended  by  this  study  (i.e.  - Quicktime  movies  for  a 360  degree  virtual  reality  effect ). 

4.  new  browser  windows 

There  is  a Javascript  [3J  which  allows  a new  browser  window  to  appear  over  the  original  screen.  This  can  vary 
in  size,  location  and  features.  It  can  be  used  to  reinforce  the  content  by  representing  it  in  another  modality.  This 
'top’  window  can  contain  animation  to  imitate  a media  player.  Sound  can  be  embedded  in  the  HTML  file  to  play 
upon  the  page  opening  or  on  a mouse  click  or  mouse  over  event  handler.  [151  TTie  new  browser  window  can  act 
as  a ’windows-help-file’  by  placing  hypertext  to  offer  glossary  terms  or  how-to  instructions,  (figure  1 .2) 
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Figure  1 . 1 Javascript  prompt  for  username 
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The  Annotation  window  is  a variation  on  this  theme  allowing  the  student  to  bring  up  a on-line  note  pad  of  sorts. 
With  the  new  browser  window  set  to  include  the  title  bar  menu  the  student  has  the  option  to  open  the  window  in 
the  HTML  editor  of  Netscape.  This  allows  them  to  add  text,  images  or  links  to  an  HTML  document  which  can 
then  be  stored  locally  or  FTP'ed  to  a public  site  [J_7].  This  allows  the  student  to  keep  an  on-line  logbook  or 
journal.  This  can  be  used  for  assessed  work  or  as  an  evaluation  methodology  [Mason  1996]. 

5.  alert  boxes 

There  are  three  main  types  of  alert  boxes;  a)  message  b)  feedback  c)  exit. 

a)  message 

Alert  Boxes  (a.k.a.:  message  boxes,  prompt  boxes,  etc.)  are  another  Javascript  tool  to  promote  an  active, 
immediate  response  from  the  student  in  the  learning  process.  There  are  a variety  of  ways  they  may  be  placed 
throughout  the  delivery  of  the  tutorial  content.  For  example,  upon  entering  the  HTML  page  containing  the 
sample  solution  an  Alert  message  form  [4]  can  prompt  the  student  to  confirm  that  they  have  developed  their 
own  solution  before  reading  the  tutor’s  example.  This  Javascript  tools  again  illustrates  the  principle  of 
increasing  learning  through  interactivity. 

b)  feedback 

At  anytime  during  the  delivery  of  the  content  an  immediate  feedback  response  is  available  by  clicking  a button 
which  may  produce  an  Alert  Box  response  [51  or  some  text  on  the  page  [6].  The  text  can  be  a positive/negative 
prompt  ( i.e.-  "Correct!  Well  Done,  or  Sorry!  Try  Again."  ) or  directions  to  the  location  of  supportive  material 
(i.e.  - "Correct:  More  details  can  be  found  at  this  URL:  http ://www.morestuff  com/  or  Sorry!  You  may  wish  to 
review  this  material  before  going  on:  http://www.reviewstuffcom/")[7]  . (figure  1.3) 


F.A.Q.'s  What's  New? 

JavaScript  Forum  "Email- a- Script"! 


Visitor  Survey 
Site  Search  Etxgbe 


Click  a topic  above  for  more  info! 
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Go  There! 


Figure  1.3 

Javascript  formative  evaluation  (feedback) 

c)  exit 

Upon  completing  a section  of  the  tutorial  an  exit  prompt  can  be  issued.  Clicking  a next  or  exit  link  will  bring  up 
a "Do  you  wish  to  leave  this  part  of  the  lesson?"  query  [8J.  These  embedded  Javascripts  prompt  the  student  to 
reflect  upon  their  learning  experience  throughout  the  tutorial  which  supports  the  fifth  principle  (give  the 
opportunity  for  reflection). 
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6.  student  self  assessment 

Student  self  assessment  is  a powerful  method  of  encouraging  reflection,  a necessary  part  of  the  learning  process 
[Laurillard  1994].  This  research  suggests  using  self  assessment  as  a formative  method  of  evaluation  (feedback) 
in  the  learning  system  in  addition  to  the  traditional  summative  (end  exam)  assessment  techniques. 

There  are  three  general  categories  of  self  assessment;  a)  closed  response  b)  open  response  c)  mixed, 
a)  closed  response 

Two  closed  methods  are  multiple  choice  questions  [91  and  matching  [10]  using  Javascript.  Both  give  immediate 
feedback,  thus  supporting  interactivity  between  the  student  and  the  system  illustrating  the  sixth  principle. 
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Figure  1.4  Javascript  self  assessment 


b)  open  response 

There  are  three  methods  for  exploiting  the  e-mail  Javascript  tool;  1)  preformatted  distribution  lists  2)  use 
preformatted  subject  headings  to  filter  responses  3)  preformatted  e-mail  surveys 

1)  preformatted  distribution  lists 

A Javascript  e-mail  distribution  list  illustrates  an  open  response  method,  f 1 11  By  clicking  a button  a 
preformatted  e-mail  message  box  with  an  open-ended  question  can  be  automatically  sent  to  the  group  for 
collaboration  and  discussion.  The  student's  social  experience  ( principle  #2)  is  supported  by  this  tool.  The  tutor 
may  also  receive  a copy  of  the  submission.  The  student  will  not  be  able  to  use  the  e-mail  reply  button  since  the 
message  was  posted  to  their  own  address.  It  is  necessary  to  include  the  original  URL  for  the  distribution  list  link 
in  the  body  of  the  message.  The  student  has  the  flexibility  to  return  there  and  reply  to  the  comments  . 

2)  use  preformatted  subject  headings  to  filter  responses 

Many  e-mail  applications  have  filtering  capability  which  would  allow  the  e-mail  generated  with  a preformatted 
subject  heading  to  be  placed  into  a specific  folder  [Brunner  1997].  This  has  the  benefit  of  helping  the  on-line 
tutor  organise  the  potentially  large  number  of  correspondence.  Alternatively,  the  e-mail  address  used  for  the 
CC:  prompt  can  be  a separate  account  designated  for  that  on-line  class.  Many  free  e-mail  services  are  now 
available  on  the  web  which  can  be  used  to  receive  specific  student  responses  (i.e.  - http://hotmail.com/ ). 

3)  preformatted  e-mail  surveys 

Reflection  can  be  encouraged  after  the  lesson  is  completed.  On-line  surveys  can  be  submitted  as  HTML  forms 
[12]  or  as  preformatted  e-mail  questionnaires  [J_3],  both  of  which  get  mailed  directly  to  the  tutor  for 
interpretation  and  action.  This  strengthens  the  learning  process  by  supporting  the  fifth  Web-Constructivist 
principle,  (figure  1.5) 

The  text  submitted  from  the  HTML  form  will  appear  as  one  continuous  line  of  characters  in  the  body  of  the  e- 
mail  message.  It  is  necessary  to  use  software  (such  as  mail  to  .Formatter  v.  4.01  - rpfries@interacxess.com  ) to 
format  the  text  into  separate  lines. 
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(2)  2I.I  found  this  tutorial  easy  to  use. 


figure  1 .5  e-mail  surveys 


c)  mixed 

A mixed  response  combines  a fixed  reply  and  open-ended  option.  This  form  of  feedback  is  the  strongest 
methodology  for  gaining  both  statistical  data  and  the  'unexpected'  comment.  An  example  of  this  is  a Likert  scale 

survey  which  includes  an  optional  description  ( i.e.  - 1 found  the  tutorial  1)  good  2)  fair  3)  bad  Why 

)(Mason  1992). 

8)  graphically  representing  feedback 

The  data  collected  by  the  web-survey  can  be  inserted  into  a Javascript  chart  builder  to  provide  a graphical 
representation  of  the  student  feedback  for  the  tutor.  fl41  This  can  be  printed  with  the  current  date  on  the  page 
for  record  keeping  purposes.  This  script  supports  the  principle  of  using  multiple  forms  of  representation,  (figure 
1.6) 
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Summary 

Javascript  can  support  Web-Constructivist  principles  through  every  stage  of  the  problem-solving  seminar 
teaching  strategy.  The  learner  makes  active  use  of  the  knowledge  by  interacting  with  the  tutorial  system,  on-line 
resources  and  colleagues  in  the  lesson.  This  social  approach  to  learning  can  be  achieved  using  Javascript  e-mail 
tools.  Information  can  be  presented  in  a number  of  multimedia  formats  using  images,  animation,  sounds  and 
simulated  virtual  reality.  An  alert  box  which  places  the  student's  name  on  the  HTML  page  empowers  a sense  of 
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ownership.  Using  formative  feedback  tools  embedded  throughout  the  tutorial  the  participant  has  the  opportunity 
to  voice  their  opinion,  thus  prompting  change  in  the  tutorial  content  or  delivery  methodology. 

Reflection,  an  important  principle  in  the  learning  process,  is  built  into  the  teaching  strategy  protocol.  At  the 
start  of  the  lesson  the  student  is  prompted  to  reflect  in  order  to  solve  the  starting  problem.  At  the  end  of  the 
lesson  self  assessment  tools  also  allow  review  and  support  reflection.  Web  surveys  prompt  the  student  to  reflect 
on  the  learning  experience  and  provide  the  Learning  Environment  Designer  (LED)  with  the  formative  feedback 
necessary  to  alter  the  tutorial  system  to  meet  the  student's  needs.  Throughout  the  tutorial  the  student  is 
prompted  to  interact  with  the  system,  participants  and  tutor  / LED  to  promote  motivation  through  feedback. 

This  research  demonstrates  that  by  using  a problem-solving  seminar  teaching  approach  supported  by  Javascript 
tools  and  using  the  protocols  developed  in  this  study  a true  Web-Constructivist  Learning  Environment  can  be 
realised.  An  example  of  this  can  be  seen  in  the  research  case  study:  DELBERT  ( Digital  Environment  Learning- 
Based  Evaluation  Response  Theory)  Tutorial  System.  The  URL  for  this  research  project  is: 
http: /Ask  ip.  mdx.ac.uk/ 
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Abstract:  The  times  when  computers  were  literally  used  only  for  computation  have  ended. 
Multimedia  (the  synthesis  of  text,  pictures,  sounds,  etc.)  has  became  a widely  used  term,  as 
information  technology  progressed.  This  is  not  an  exception  in  the  field  of  education,  where  much 
research  is  being  conducted,  to  make  use  of  multimedia  in  Computer  Assisted  Learning  (CAL) 
systems.  This  paper  describes  a Japanese  education  system  using  multimedia  towards  the  non- 
Japanese. 


1.  Introduction 

In  recent  years,  the  number  of  foreigners  trying  to  learn  Japanese  has  increased.  However,  it  is  difficult  to  master 

Japanese  even  after  having  lived  several  years  in  Japan.  The  singularities  of  Japanese,  which  we  will  show,  remain 

the  first  reason. 

• The  Japanese  language,  besides  hiragana  and  katakana  syllables,  uses  Chinese  characters  (kanji).  Hiragana  and 
katakana  are  phonograms,  while  kanji  are  ideograms.  Languages  using  combinations  of  phonograms  and 
ideograms  at  the  same  time,  in  this  way,  are  very  uncommon  in  the  world. 

• On  one  hand,  the  alphabet  used  in  America  and  Europe  has  about  50  letters,  including  capitals.  On  the  other 
hand,  there  are  over  3,000  kanji  characters.  In  Japan,  pupils  are  expected  to  learn  over  1000  kanji  during  their  6 
years  of  elementary  school.  Therefore,  the  speakers  of  native  English  as  well  as  Indo-European  languages  are 
troubled  by  the  complexity  of  kanji:  their  language  does  not  require  intense  writing  practice  nor  need  to 
discriminate  between  numerous  characters. 

• Some  kanji  characters  are  very  complex.  Their  shape  is  sometimes  difficult  to  differentiate.  (Fig.  1) 

• Many  Kanji  characters  have  several  pronunciations,  while  others  share  the  same  pronunciation. 

• Some  words  change  their  meaning  depending  on  the  position  of  the  accent.  For  example  "hashi"  in  Japanese 
means  "chopsticks  " when  the  first  syllable  is  accentuated.  It  means  "bridge"  when  the  accent  comes  on  the  last 
syllable. 

• Calligraphy  (shodo),  an  important  part  of  the  Japanese  culture,  pays  much  attention  to  the  stroke  order  and  the 
shapes  of  the  characters  strokes:  "hold",  "sweep",  "curve  ",  "stop"  etc.  (Fig.2)  These  concepts  are  essential  to 
the  foreigner  who  wants  to  master  Japanese.  They  cannot  be  mastered  without  practice. 
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break 


Figure  1 : Well-used  Kanji  characters  in  Japan. 


Figure  2:  Stop,curve,break  and  leap 
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Thus,  it  is  difficult  for  foreigners  to  master  Japanese.  However,  the  use  of  multimedia  is  effective  in  Japanese 
education.  A sample  animation  movie  can  show  the  stroke  order  and  speed,  while  the  pronunciation  and  the  accent 
of  the  word  are  heard.  Of  course  we  cannot  ignore  the  effect  of  the  interest.  Using  a computer  is  fun,  especially  for 
the  children  who  like  computer  games.  Besides,  finding  a kanji  is  faster  when  using  a multimedia  database. 
Previously,  we  had  developed  "Terakoya  on  the  Web"  (Omae  97)  and  "Terakoya  Kanta"  (Fujita98).  This  time,  we 
present  ’’Terakoya",  an  Integrated  Japanese  Education  System,  which  include  them. 


2.Past  works 

There  are  some  software  products  for  Japanese  education.  They  can  be  classified  into  "dictionary"  and  "drill"  types. 
They  have  the  following  features  and  problems. 


2.1  Dictionary  type 

These  are  electronic  dictionaries,  giving  information  on  characters.  They  allow  quick  reference  to  pronunciation, 
sample  usage,  number  of  the  stroke,  etc.  But  they  cannot  be  used  for  self-teaching  by  themselves.  They  must  be  used 
with  other  materials,  such  as  textbooks  and/or  drill  books. 


2.2  Drill  type 

These  can  be  used  for  study  with  input  devices,  mouse  or  keyboard.  There  are  three  important  skills  to  master  a 
language. 

1.  Reading 

2.  Writing 

3.  Talking  and  listening 

Many  CAL  software  products  for  Japanese  can  teach  only  one  of  these  skills,  which  is  not  enough  complete  to  learn 
comprehensive  language  skills.  Besides  it,  they  tend  to  provide  only  little  information  on  characters  or  words,  and 
do  not  have  much  effect  on  their  own. 

When  these  two  types  of  software  are  used  in  classroom  situations,  there  is  sometimes  a big  difference  between  what 
CAI  provides  and  what  the  teacher  wants  to  teach.  The  teacher  may  think  "There  is  no  Kanji  characters  in  this 
software  that  I would  like  to  teach"  or  "I  don’t  like  to  teach  this  usage  now"  or  "I  want  to  use  this  sentence  as 
example". 

Our  system  has  some  functions  to  solve  these  problems  as  following. 

• We  developed  four  types  of  drill  software.  One  to  learn  reading  skills,  another,  writing  skills  and  at  last,  talking 
skills.  Each  software  is  integrated  closely.  This  system  can  achieve  an  integrated  learning.  Students  can  train 
reading  skills,  writing  skills  and  talking  skills  for  a chosen  kanji. 

• We  have  integrated  database  and  drill  software.  This  approach  can  compensate  the  lack  of  information  in  drill 
type  software  and  the  lack  of  teaching  function  in  a database  type  software. 

• Our  system  can  be  configured  to  fill  up  a gap  between  the  teacher’s  needs  and  the  producing  of  CAL.  This  is  a 
good  help  to  use  CAL  effectively  in  classrooms. 


3.  For  effective  CAL 

Developing  effective  CAL  requires  careful  consideration.  Animation  or  sound  effects,  which  would  make  some 
children  happy,  could  be  only  disturbing  for  adults.  For  the  learner,  who  wants  to  travel  to  Japan  without  difficulty, 
it  may  be  unpleasant  to  be  advised  on  some  details,  like  the  stroke  order  of  characters.  We  must  think  over  the 
situation  in  which  the  system  will  be  used.  Is  it  intended  for  self-teaching  or  will  it  be  used  in  classroom?  We  have 
developed  software  with  an  eye  on  the  following  points. 
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Generation's  gap 

• Children:  Joyful  animations  and  sounds,  easy  comments. 

• University  Students:  efficient  use. 

• Adults:  Restful,  easy  handling. 

Situation's  gap 

• For  self- teaching 

• In  classroom  with  one  computer. 

• In  classroom  with  many  computers 
The  goal  gap 

• So  as  not  to  be  in  trouble  during  the  trip  to  Japan. 

• So  as  not  to  be  in  trouble  living  in  Japan. 

• For  a native  Japanese 

We  have  prepared  some  learning  modes  and  optional  functions,  like  animation.  Our  software  can  compensate  the 
difference  of  the  learner’s  age  or  goal.  Most  software  we  know  is  developed  for  self-teaching  or  without  a general 
idea  of  the  target.  Our  software  can  be  also  used  in  classrooms. 


4.Model  case 

In  this  section,  we  show  an  example  of  effective  usage  of  our  system. 

Situation:  In  the  Japanese  class.  There  is  a computer  in  the  room  and  all  students  can  see  the  monitor  through  the 
projector. 

1 . In  the  class,  the  teacher  selects  the  character  he  wants  to  teach  in  the  textbook  or  another  paper  document  and 
searches  it  in  the  "Kanta"  dictionary  software. 

2.  All  students  study  the  information  of  the  character  on  the  projector:  the  pronunciation,  the  usage  etc, 

3.  One  or  two  of  the  students  practice  the  character  with  the  “Kanta”  drill  software.  Other  students  can  look  at  the 
appearance  on  the  projector. 

4.  “Let’s  practice!”  All  students  practice  the  character  on  the  paper  drill  made  by  the  making  drill  module  of 
Kanta 

5.  At  home,  the  students  can  review  the  lessons  on  the  Internet  with  a PC. 

Of  course,  besides  this  usage,  our  system  can  be  used  for  self-teaching  in  classrooms  with  a computer  for  each 
person. 

Until  now,  we  could  not  find  CAI  software  supporting  the  whole  features,  from  the  classroom  to  self-teaching.  Our 
system  does  it. 


5.  Japanese  integrated  CAI  system  “Terakoya” 

On  this  base,  the  integrated  CAI  system  “Terakoya”  is  composed  of  three  modules. 

5.1  Database  module 

This  module  has  all  the  information,  which  a paper  dictionary  has.  Besides,  it  uses  multimedia  for  the  animation  of 

the  strokes  and  the  pronunciation  of  the  characters.  Our  multimedia  database  has  the  following  good  points 

compared  with  a paper  dictionary. 

• Our  multimedia  database  uses  animation  sequences  to  teach  the  stroke  order  and  rhythm.  It  is  about  “leap”, 

• In  Japanese,  the  meaning  of  some  words  can  changes  depending  on  the  intonation.  For  example,  the  word 
“hashi”  means  “chopsticks”  if  the  accent  is  put  on  the  first  syllable.  It  means  “bridge”  in  case  when  the  musical 
accent  is  put  on  the  last  syllable.  It  is  difficult  to  express  these  differences  with  a paper  dictionary.  We  can  do  it 
easily  with  the  sound  of  our  multimedia  database. 
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• The  desired  information  can  be  found  faster  than  with  a paper  dictionary. 

This  module  is  closely  co-ordinated  with  drill  module  and  the  student  can  practice  the  character  he  encounters. 
Besides,  the  teacher  can  make  paper  drills  by  choosing  the  necessary  data  in  our  database  module  with  the  “making 
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Figure  3:  Database  Module 


5.2  Drill  module 

Our  “drill  module”  includes  two  types  of  software.  The  first  is  dedicated  to  the  learning  recognition  skills  of  the 
character.  The  second  concerns  the  learning  of  writing  skills.  Software  functions  are  co-ordinated  and  the  student 
can  practice  a word  through  different  stages.  Furthermore,  he  can  get  information  on  the  words  through  links  to  the 
“database  module”. 


5.2.1  Drill  for  recognition  skills,  "Terakoya  Web** 

This  software  was  presented  as  a stand-alone  at  the  ED-media97.  The  student  could  get  recognition  skills  by  re- 
constructing a character  randomly  selected  with  a mouse  to  form  a puzzle,  (Fig.4),  motivating  his  study. 

In  Japanese,  some  characters  are  similar  in  shape,  while  others  are  different  in  appearance  from  the  fonts,  (Fig,5) 
The  student  can  be  accustomed  to  recognize  them  by  constructing  their  parts  with  his  hand. 
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Figure  5:  Difference  depending  on  the  font 


5,2.2  Drill  for  writing  skills,  "Terakoya  Kanta" 

This  software  has  also  been  presented  as  a stand-alone  at  the  ED-media  98.  This  software  has  been  tested  at  the 
Richmond  Primary  School  in  Portland,  Oregon  and  Waseda  University  Language  Centre  and  has  got  a good 
evaluation. 

The  learner  can  practice  the  characters  with  a mouse  or  a tablet  in  this  module,  which  advises  and  shows,  like  a real 
teacher,  the  score  for  the  written  character.  Our  software  uses  vector  and  stroke-rhythm  information  and  can  find 
wrong  stroke  order  and  other  close  points  as  "Hold",  "Leap",  "Sweep"  and  "Curve". 

The  learner  can  select  the  learning  mode  depending  on  his  age  or  his  level.  Many  animations  and  happy  sounds  are 
built-in  for  children  as  an  option.  This  system  can  make  the  writing  practice  joyful. 
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Figure  6:  Drill  for  writing  skill 


5.3  Module  for  making  drill. 

As  for  the  past  works,  the  software  of  the  database  module  can  not  be  used  for  self- teaching  and  sometimes  there  are 
some  differences  between  the  contents  of  the  database  and  the  teacher's  needs.  Furthermore,  only  one  person  can  use 
the  software  of  the  drill  module  at  the  same  time  and  schools,  which  provide  each  student  a computer,  are  the 
exception.  Therefore  we  suggest  that  at  first  one  or  two  students  practice  with  the  software  in  the  presence  of  all 
students  and  then,  that  all  students  practice  with  a paper  drill.  A paper  drill  closely  related  to  the  drill  software  will 
be  of  great  help  to  the  students.  To  realize  it,  two  functions  are  necessary. 

The  teacher  can  select  the  contents  that  he  wants  to  teach  from  the  database  module,  such  as  some  pronunciations 
and  usage  of  the  word.  Thereafter,  selected  contents  should  be  shown  in  the  drill  module,  which  students  do. 

The  teacher  can  print  out  the  paper  drill.  That  includes  the  contents,  which  the  teacher  selects. 

Our  system  "Kanta"  has  these  two  functions. 
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6.  Conclusion 

Use  of  multimedia  is  one  most  effective  way  to  leam  a language.  Because  animation  and  sound,  respectively 
samples  of  writing  and  pronunciation,  will  be  a great  help  to  master  a language  rather  than  paper  or  other  media.  Of 
course,  we  cannot  ignore  the  motivation  effect  when  using  a computer.  Besides,  a much  higher  efficiency  is 
expected  when  the  learner  utilizes  the  database  module  for  reference. 

However,  such  a system  can  hardly  satisfy  the  needs  of  the  Japanese  education.  One  reason  is  the  difficulty  to  teach 
Japanese  on  the  base  of  its  singularities.  Most  software  we  have  found  can  train  only  certain  skills,  like  recognition 
skills,  writing  skills  or  talking  skills.  Therewith,  it  is  difficult  to  utilize  them  in  classrooms  because  of  the  difference 
between  what  the  teacher  wants  and  what  the  software  can  produce. 

To  compensate  these  problems,  we  have  integrated  the  database  and  the  drill  modules,  allowing  training  recognition 
skills,  writing  skills  and  talking  skills.  This  system  is  an  integrated  environment  for  mastering  Japanese.  The  learner 
can  find  the  information  of  a certain  word  in  the  database  module,  and  then,  can  practice  the  recognition  of  certain 
words.  At  last  he  can  try  to  write  the  word.  For  the  use  in  class,  the  drill  module  can  be  customized  according  to  the 
teacher's  needs.  The  teacher  can  also  prepare  paper  drills  utilizing  our  database  module. 

Our  system  "Terakoya”  is  developed  on  a WWW  base  so  that  it  can  be  used  on  any  platform.  We  are  preparing 
some  options  to  achieve  effective  learning  irrespective  of  the  learner’s  age  or  aim  to  study  Japanese.  Our  system  can 
be  used  not  only  for  self-teaching  but  also  in  classrooms. 

7.  Future  works. 

A network  version  of  this  system  will  be  tested  at  Richmond  primary  school  in  Portland,  Oregon  and  at  Language 
Centre  of  Waseda  University.  (Stand-alone  versions  have  already  been  tested  and  got  a good  evaluation  there.)  At 
present,  we  have  a plan  to  develop  a training  module  for  idioms  (Jukugo).  We  are  also  considering  applying  this 
system  to  other  languages. 
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Abstract:  The  aim  of  the  this  study  was  to  apply  cognitive  theory  about  two  learning  modes 
to  HCI  and  to  investigate  the  effect  of  feedback  on  user  performance  in  a simulation  based 
learning  environment.  The  effect  of  continuous  and  discontinuous  feedback  was  tested  in  an 
experiment  where  the  subjects  first  learned  principles  of  illumination  ergonomics  with  a 
simulation.  After  the  learning  phase  the  procedural  and  declarative  knowledge  were  tested. 
The  experiment  shows  that  variations  in  feedback  had  no  significant  influence  on  the 
performance  and  behaviour  in  the  learning  phase,  but  feedback  had  an  effect  on  declarative 
knowledge  and  the  users  confidence  in  procedural  knowledge.  The  conclusion  is  that  the 
discontinuous  feedback  condition  increased  the  declarative  knowledge  and  the  confidence  in 
the  procedural  knowledge.  The  differences  between  the  experimental  groups  can  be 
explained,  in  that  the  feedback  conditions  induced  different  learning  modes  in  the  subjects. 


Introduction 

Interface  design  should  be  theory  and  effect  driven.  Theory  should  guide  experimental  research,  rather  than 
phenomenological  interest  in  technological  features.  On  the  practical  side,  knowledge  about  the  effects  of  user- 
interface  features  should  guide  implementation,  rather  than  technological  advances.  Today,  computing  power  is 
not  anymore  a limitation  for  implementation,  the  problem  moves  from  how  to  implement  to  what  to  implement. 
This  paper  shall  give  a better  understanding  of  the  impact  of  theory  for  interface  design,  and  a context  for  how  to 
make  decision  for  the  implementation  of  different  interface  features. 

This  study  combines  both  cognitive  theory  and  software  ergonomics.  The  theoretical  background  is  drawn 
from  the  increasing  evidence  that  people  can  learn  in  two  different  modes  (Reber,  A,S,,  1989;  Hayes  & 
Broadbent,  1988).  An  explicit  mode  is  recognised  by  a conscious  and  selective  attention  towards  the  problem, 
while  in  the  implicit  mode,  the  learning  is  more  unconscious  and  trial  and  error  based.  The  success  of  employing 
each  of  these  modes  depends  on  characteristics  of  the  problem.  Previous  experiments  have  shown  that  different 
user-interfaces  can  induce  the  two  learning  modes,  and  that  different  user-interfaces  influence  how  people  learn 
(Guttormsen  Schar,  1998;  Guttormsen  Schar,  1997a),  By  way  of  example,  a command-based  interface  can 
induce  an  explicit  learning  mode,  and  a direct  manipulation  interface  can  induce  an  implicit  learning  mode 
(Guttormsen  Schar  et.al,  1997b;  Guttormsen  Schar,  1996),  These  studies  showed  a benefit  for  the  explicit 
learning  mode  for  intelligible  learning  tasks,  A study  was  designed  to  investigate  the  effect  of  the  interface  when 
the  differences  between  the  interface  features  become  subtle.  How  subtle  can  variations  in  the  user-interface  be, 
and  still  induce  two  different  learning  modes? 

The  software  ergonomics  in  this  study  address  decisions  for  the  implementation  of  user-interface  features. 
The  effect  of  continuous  and  discontinuous  feedback  of  the  actions  with  a simulation  was  examined.  With 
increasing  processing  power,  continuous  feedback  to  the  user  is  appealing.  The  effect  of  this  kind  of  feedback 
should  be  investigated  in  order  to  gain  knowledge  of  benefits  and  eventual  disadvantages  for  the  user.  An  effect 
of  feedback  in  this  case  would  not  only  be  related  to  aspects  of  the  feedback  itself,  but  would  also  indicate  the 
degree  to  which  users  are  influenced  by  apparently  unimportant  interface  aspects  in  general.  Two  actual 
examples  from  commercial  software  illustrate  different  forms  of  subtle  feedback.  The  two  examples  represent 
continuous  and  discontinuous  methods  of  feedback  respectively: 

1)  A graphics  editor  such  as  Adobe  Photo-shop  offers  filtering  options,  which  are  operated  by  dialogue  boxes.  In 
the  dialogue  box,  different  parameters  can  be  changed  interactively  by  moving  small  handlers  on  a bar.  The 
effect  of  a change  can  often  be  seen  instantly  in  a small  preview  box.  This  gives  the  user  an  immediate 
impression  of  the  effect  of  such  a change. 
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2)  The  use  of  3D  programs  often  involves  the  following  situation:  when  rotating  a three-dimensional  object,  the 
user  is  actually  manipulating  a simple  copy  of  the  object,  i.e.  a 'bounding  box'.  The  real  object  is  first  shown 
when  the  rotation  is  finished. 

The  theoretical  question  was  whether  continuous  and  discontinuous  feedback  of  users  actions  can  induce 
different  learning  modes  in  the  user.  The  software  ergonomics  question  was  whether  continuous  or 
discontinuous  feedback  can  be  recommended  for  simulation  based  learning.  The  following  hypotheses  were 
tested  in  the  experiment: 

1)  An  interface  with  continuous  feedback  induces  an  implicit  learning  mode. 

2)  An  interface  with  discontinuous  feedback  induces  an  explicit  learning  mode. 

3)  Discontinuous  feedback  results  in  a better  learning  effect. 


Method 

Subjects 

24  subjects,  between  20  and  39  years  participated.  The  subjects  were  sampled  from  different  study  directions, 
both  men  (10)  and  women  (14)  were  represented.  All  subjects  were  paid  30  SFr.  None  of  the  subjects  had 
experience  with  the  learning  content  of  the  simulation. 


The  Simulation  and  the  Interaction 

The  experiment  used  an  interactive  simulation  of  illumination  ergonomics  for  the  learning  task.  The  main  mask 
of  the  simulation  is  shown  in  Figure  1 . The  subjects  could  install  and  remove  luminaries  in  a virtual  room,  and 
they  could  move  and  rotate  a working  place.  The  effect  of  the  actual  illumination,  i.e.  the  ergonomics  evaluation 
of  the  illumination  configuration  was  shown  on  six  different  scales  displayed  as  bars:  amount  of  lights  shadows, 
glare,  screen  glare,  work  place  visibility,  and  general  satisfaction.  Hence,  the  scales  gave  feedback  about  how 
the  actual  illumination  and  configuration  of  the  room  influenced  a virtual  person.  The  ergonomics  evaluation 
applied  a scale  from  0 to  10,  and  was  indicated  on  the  bars  by  a ruler.  In  addition,  the  bars  were  colour  coded:  a 
good  configuration  made  the  rulers  appear  within  the  green  area  of  the  bars.  The  red  area  of  the  bars  indicated  a 
bad  illumination  configuration.  The  simulation  was  operated  by  'direct  manipulation',  the  subjects  could  perform 
all  operations  by  using  the  mouse  to  click  on  buttons,  select  and  drag  objects. 


Feedback 

The  feedback  in  this  experiment  was  directly  related  to  the  user  actions  with  the  simulation.  The  calculation  of 
the  ergonomics  evaluations  could  proceed  in  one  of  two  ways:  it  could  be  displayed  continuously  with  tracking 
actions  or  alternatively  the  ergonomics  evaluations  scales  could  be  computed  discontinuously  and  the  scales 
would  be  updated  only  when  the  mouse  button  was  released.  In  the  continuous  condition  the  bars  indicating  the 
actual  value  on  the  ergonomic  scales  would  instantly  move  with  the  tracking  movements  of  an  object.  In  the 
discontinuous  condition  the  subjects  would  only  see  the  bars  change  from  the  last  position  on  the  scale  to  the 
new  position  after  the  object  had  been  moved  to  a new  position. 
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Figure  1:  The  main  mask  of  the  simulation 


Experiment  and  Procedure 

The  experiment  had  a 2 X 5 mixed  design,  with  feedback  as  the  between  variable  and  task  as  within.  The 
independent  variables  were  number  of  actions  per  task,  total  time  per  task,  thinking  time  between  each  action, 
performance  (i.e.  total  scores  on  the  ergonomic  scale),  declarative  knowledge  about  illumination  ergonomics  and 
subjectively  estimated  confidence  in  the  knowledge  from  the  post  learning  tasks. 

Initially  a test  of  the  ability  to  use  a mouse  was  administered.  All  subjects  were  required  to  complete  a 
puzzle,  which  required  precise  positioning  with  a mouse.  The  time  to  complete  the  puzzle  and  thinking  times 
were  recorded  in  a log  file. 

In  the  learning  phase,  the  experimental  task  was  to  create  an  ergonomic  optimal  configuration  of  the 
working  place  and  the  light  sources  (luminaries  and  window).  All  the  subjects  were  required  to  solve  5 tasks. 
The  tasks  reflected  variations  in  the  external  conditions  in  the  room:  the  position  of  the  window,  and  the 
conditions  for  the  movements  of  the  working  place.  The  working  place  was  initially  positioned  differently  for 
each  task.  All  the  actions  with  the  simulation,  the  effects  on  the  ergonomics  evaluation,  and  the  time  per  action, 
were  recorded  in  a log-file. 

After  the  learning  phase,  10  post-leaming-tasks  were  administered  to  test  the  procedural  knowledge  of  the 
subjects.  The  subjects  were  required  to  use  the  simulation  to  create  an  illumination  configuration  in  the  room.  In 
these  tasks  the  subjects  could  not  see  the  ergonomic  effects  of  their  configurations.  In  five  of  the  tasks  the  goal 
was  to  achieve  optimal  illumination  evaluations,  and  in  the  other  five  tasks  the  goal  was  to  achieve  the  poorest 
evaluations.  After  each  task  the  subjects  were  asked  to  indicate  on  a scale  from  1 to  100  how  confident  they  felt 
that  they  had  achieved  the  target,  the  objective  ergonomics  evaluation  were  not  shown  to  them. 

As  a test  of  declarative  knowledge,  all  the  subjects  were  asked  to  write  down  their  answer  to  a question, 
which  was  designed  to  measure  the  declarative  knowledge  about  illumination  ergonomics.  The  answers  were 
evaluated  by  three  independent  experts  on  illumination  ergonomics,  who  were  uninformed  about  the  difference 
in  learning  conditions  during  the  knowledge  acquisition  phase.  The  mean  scores  of  the  experts  were  used  as  the 
measure  of  declarative  knowledge  for  each  subject.  At  the  end  all  the  subjects  were  asked  to  indicate  on  a five- 
point  scale  how  difficult  the  learning  tasks  were,  and  how  pleasant  it  was  to  use  the  simulation. 

The  experiment  was  run  according  to  the  following  procedure:  1)  Introduction  to  the  experiment,  2)  Direct 
manipulation  test,  3)  Introduction  to  the  simulation,  4)  Learning  phase,  5)  Post  learning  tasks  + Confidence 
questions,  6)  Questions  about  procedural  knowledge,  task  difficulty,  and  user-preference  towards  the  system. 


Results 

The  learning  phase  did  not  result  in  any  significant  effects  of  feedback  or  interaction  effects  between  feedback 
and  task.  There  was  also  no  difference  between  the  groups  on  how  difficult  it  was  to  solve  the  tasks  or  on  user 
satisfaction. 

In  the  post  learning  phase,  a main  effect  of  feedback  was  evident  in  the  subjective  confidence  in  achieving 
the  goals  of  the  procedural  tasks,  and  in  the  declarative  knowledge.  These  effects  can  be  seen  in  Table  1,  The 
procedural  knowledge  were  not  different  between  the  groups  (p  = 0,34),  The  reliability  between  the  three 
independent  adjudicators  evaluating  the  verbal  knowledge  was  .86,  which  indicates  an  acceptable  degree  of 
agreement  between  the  adjudicators. 


Feedback 

continuous 

discontinuous 

F 

P 

Confidence 

75,33 

83,56 

20.28 

0.0001 

Knowledge 

1.15 

2.08 

3.43 

0.0783 

Table  1:  Effects  of  feedback  in  the  post-learning  phase 

A main  effect  of  the  post  learning  task  (target  'good'  or  'bad')  was  found  for  satisfaction  and  confidence  in  the 
procedural  knowledge.  This  effect  can  be  seen  in  Table  2, 
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Task 

good 

bad 

F 

P 

Satisfaction 

7.35 

2.68 

179.25 

0.0001 

Confidence 

75.89 

83.00 

15.82 

0.0001 

Table  2:  Effects  of  post  learning  task 

No  sex  dependent  learning  effects  resulted  in  the  learning  phase.  The  mouse  test  revealed  no  significant 
differences  between  the  groups.  The  results  of  the  experiment  can  therefore,  not  be  explained  by  biased 
distribution  between  the  groups  due  to  skills  in  operating  the  interface. 


Discussion 

The  lack  of  effects  in  the  learning  phase  (both  on  the  interaction  style  and  on  the  performance  side)  shows  that 
feedback  only  had  an  effect  on  a deeper  level.  The  effect  of  feedback  is,  evident  by  the  post-learning  tasks.  The 
main  effect  of  task  (target  ’good’  or  ’bad’)  in  the  post  learning  phase  shows  that  the  subjects  succeeded  in 
distinguishing  the  two  targets.  The  differences  in  confidence  of  the  procedural  knowledge  and  the  tendency  of 
better  verbal  knowledge  of  the  illumination  principles,  give  reasons  to  conclude  that  continuous  and 
discontinuous  feedback  can  induce  implicit  and  explicit  learning  modes,  respectively.  Hence,  hypotheses  1 and  2 
are  supported.  The  cognitive  theory  predicts  these  effects.  When  learning  in  an  explicit  mode,  people  are 
expected  to  have  explicit  and  rational  reasons  for  their  actions,  while  in  the  implicit  mode  the  actions  are  more 
intuitive  and  difficult  to  verbalise.  The  negative  effect  of  the  implicit  learning  mode  can  be  explained  in  two 
ways:  either  the  illumination  principles  were  not  understood  or  the  knowledge  was  cognitively  organised  in  a 
way  that  made  retrieval  difficult.  The  latter  explanation  seems  correct  based  on  the  fact  that  there  were  no 
difference  between  the  group  in  the  procedural  knowledge  for  the  post-learning  tasks,  i.e.  both  groups  were  able 
to  employ  the  knowledge  from  the  learning  phase.  Poor  cognitive  organisation  of  the  learning  material  was 
reflected  in  the  difference  in  confidence  in  the  procedural  knowledge  between  the  groups.  The  subjects  in  the 
discontinuous  feedback  group  solved  the  post  learning  tasks  by  relying  more  on  explicit  reasons,  hence  they  also 
reported  greater  confidence  in  achieving  the  target.  The  effect  of  two  modes  of  learning  is  also  supported  by  the 
tendency  of  a difference  in  declarative  knowledge  between  the  groups.  Learning  in  the  implicit  mode  did  result 
in  procedural  but  not  in  declarative  knowledge.  The  knowledge  in  the  implicit  group  could  not  be  verbally 
expressed  as  good  as  when  learned  in  an  explicit  mode. 

Hypothesis  3 was  only  partly  supported  because  discontinuous  feedback  only  showed  to  have  a benefit  for 
declarative  knowledge  and  confidence.  A stronger  result  would  have  been  when  the  effect  of  feedback  was 
evident  for  both  procedural  and  declarative  knowledge. 

On  the  theory  side,  the  results  are  in  line  with  the  conclusions  of  our  previous  experiments  (Guttormsen  et 
al.  1997b  and  Guttormsen,  1997a.)  Guttormsen  et  al.  1997b  applied  an  earlier  version  of  the  simulation  used  in 
this  experiment.  This  experiment  can  therefore  be  seen  as  a replication  of  the  effect  of  an  explicit  learning  mode 
for  tasks  involving  simulation  based  illumination  ergonomics.  When  the  subjects  are  encouraged  to  learn  with 
more  focused  attention, 

they  are  able  to  invent  heuristic  rules  about  the  illumination  principles,  and  to  apply  these  rules.  The  more 
unfocused  attention  recognised  in  implicit  learning,  obscure  the  obviousness  of  such  heuristics. 

On  the  software  ergonomics  side,  the  results  of  the  experiment  may  be  surprising,  because  the  difference 
between  the  feedback  conditions  was  not  large.  There  seems  to  be  a tendency  among  software  developers  to  not 
reflect  much  about  the  effects  of  such  interface  factors.  A common  opinion  seems  to  be  that  it  is  good  for  the 
users  to  get  fast  feedback  as  long  as  computing  power  makes  it  possible.  In  this  experiment  it  is,  therefore,  worth 
reflecting  more  about  which  practical  effects  the  two  feedback  conditions  actually  implied  for  the  subjects.  In  the 
continuous  condition  it  is  possible  that  the  subjects  focused  their  attention  more  on  the  ergonomic  evaluation 
bars  than  on  the  actual  configuration  of  the  objects  in  the  room.  It  was  in  principle  possible  to  select  an  object 
and  drag  it  without  to  look  at  the  objects  itself  Every  mouse  movement  caused  an  instant  updating  of  the 
feedback  bars,  hence  the  subjects  could  have  been  implicitly  shaped  to  keep  the  attention  on  the  ergonomics 
which  navigated  their  movements  of  the  objects  in  the  room.  In  the  discontinuous  condition,  the  ergonomics 
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values  were  only  calculated  and  shown  when  a drag  action  was  finished.  Hence,  the  subjects  could  focus  their 
attention  more  on  the  configuration  of  the  objects  in  the  room.  The  subjects  in  this  category  therefore  have 
learned  more  about  how  to  configure  the  objects  in  the  room  in  order  to  achieve  an  optimal  illumination  and 
ergonomics  evaluation. 

Conclusion 

Two  aspects  were  addressed  in  this  study:  on  the  cognitive  level,  the  applicability  of  cognitive  theory  to  HCI  was 
investigated,  and  on  a software  ergonomic  level  the  question  was  whether  subtle  variations  of  the  user-interface 
also  have  an  effect  on  the  learning  performance.  The  experiment  has  shown  that  implicit  and  explicit  learning 
modes  can  be  induced  in  the  users  by  means  of  continuous  and  discontinuous  feedback  respectively.  The 
cognitive  theory  offers  a background  for  explaining  performance  effects  in  the  HCI  context.  This  results  in  a 
more  differentiated  way  of  understanding  HCI,  as  it  incorporates  the  cognitive  aspects.  Traditionally  in 
comparative  HCI  studies,  the  focus  is  on  phenomenological  aspects  such  as  the  user  preferences  or  task 
completion  times.  Task  completion  time  is  a performance  measure,  which  does  not  incorporate  the  quality  of  the 
achieved  task  solution.  This  study  did  not  reveal  significant  differences  in  user  preferences  or  task  completion 
times.  Hence,  focusing  on  the  cognitive  aspects  of  HCI  have  resulted  in  a picture  of  the  advantages  and 
disadvantages  of  the  user-interface  that  otherwise  would  have  been  ignored.  The  theoretical  approach  to  an 
interpretation  of  user-interface  effects  is  important  in  showing  a more  differentiated  picture  of  the  effects  of 
various  user-interfaces.  The  theoretical  approach  is  particularly  applicable  to  user-interface  design  in  the 
computer  aided  learning  context,  where  the  aim  is  to  acquire  knowledge.  The  aim  of  a program  is  an  important 
aspect,  because  the  effect  of  continuous  vs.  discontinuous  feedback  could  be  quite  different  if  applied  to  another 
context.  Contexts,  which  are  less  focused  on  knowledge  acquisition  and  transfer  of  knowledge  to  different  task 
environments,  may  benefit  more  from  continuous  feedback.  This  may  apply  to  tasks,  which  require  creativity 
and  possibilities  to  explore.  The  fact  that  the  intervention  by  the  user-interface  was  minor,  but  nevertheless 
induced  different  learning  modes  shows  that  users  of  software  are  extremely  sensitive  to  any  effect  imposed  by 
the  user-interface.  The  advice  for  developers  is  therefore  that  even  apparently  minor  user-interface  features  can 
have  significant  effects  on  performance  and  should  be  considered.  All  kinds  of  interface  features  should, 
therefore,  be  evaluated  objectively  in  relation  to  perceivable  advantages  and  disadvantages  related  to  the  task 
before  they  are  implemented.  Further  research  in  HCI  should  continue  to  incorporate  the  cognitive  aspects  of 
using  an  interface,  and  effort  should  be  put  into  investigation  of  differentiating  aspects  of  user-interface 
technology. 
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Abstract:  AIMS  is  a knowledge-based  information  system  whose  main  goal  is  to 
help  users  to  deal  in  the  most  efficient  way  with  information  within  a specific 
domain  in  respect  to  their  preliminary  defined  tasks.  AIMS  provides  the  user  with  an 
access  to  the  whole  information  domain  presenting  to  him/her  information  search, 
exploration  and  navigation  facilities.  AIMS  is  endowed  with  an  explicit  conceptual 
mapping  of  the  domain  knowledge  structures  and  a user  model.  Within  the  general 
system  architecture  of  AIMS  we  will  focus  on  Information  Retrieval  and 
Information  Visualisation  tasks  and  will  present  a solution  to  some  of  the  problems 
in  this  respect. 


Introduction 

Although  the  task  of  searching  and  finding  relevant  information  is  quite  an  old  one,  nowadays  it  becomes  very 
prominent  due  to  the  presence  of  the  World  Wide  Web  and  constantly  increasing  amount  of  dynamic 
information  available  there.  There  are  a number  of  problems  that  are  encountered  in  respect  to  this  current  and 
extremely  large-scale  information  context.  They  are  mainly  related  to  the  searching,  filtering,  navigation  and 
browsing  tasks.  In  this  concern  the  issues  of  how  to  increase  the  effectiveness  of  information  retrieval  and 
information  visualisation  are  becoming  essential  for  the  information  management  in  such  dynamic  and  complex 
environments. 

In  this  paper  we  present  an  approach  to  Information  Retrieval  and  Information  Visualisation  that  employs  the 
use  of  Concept  Maps  for  representing  classification  knowledge.  The  latter  allows  for  more  efficient  information 
retrieval  and  clear  and  attractive  information  visualisation.  We  discuss  a prototype  of  an  Agent-based 
Information  Management  System  (AIMS)  that  implements  the  proposed  approach  applying  the  Intelligent 
Agents  technology.  The  reported  work  is  performed  within  the  framework  of  some  masters  and  PhD  research 
projects  related  to  the  use  of  Intelligent  Agents  technologies  in  Virtual  Study  Environments.  We  believe  that  a 
combination  of  agent  technologies  and  visualisation  techniques  in  the  context  of  concept  mapping  has  a very 
positive  impact  on  effectiveness,  efficiency  and  user  engaging  in  respect  to  information  use,  reuse  and 
maintenance. 

Information  management  systems  are  proposed  as  a way  to  improve  the  usability  and  maintenance  of 
information.  In  our  project  we  envisage  the  management  of  information  as  concerning  two  mutually 
complementing  processes:  these  of  retrieving  the  right  information  and  visualising  it  in  the  right  way.  Thus  the 
focus  of  the  work  is  on  information  search,  retrieval  and  effective  presentation  to  the  user.  The  aim  is  to  support 
the  user  in  accessing,  selecting  and  understanding  the  requested  information.  The  main  idea  was  to  integrate 
several  technologies  that  supplement  to  each  other  in  respect  to  more  effective  information  management  within 
the  framework  of  distributed  environments.  The  technologies  we  consider  include:  intelligent  agents  (I A), 
information  retrieval  (IR),  information  visualisation  (IV),  concept  mapping  (CM)  and  user  modelling  (UM). 
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When  we  speak  about  Intelligent  Agents  we  have  in  mind  software  components  that  support  a user  in 
accomplishing  some  complex  activities  (composed  of  a number  of  tasks)  within  information  environments,  such 
as  offering,  finding,  and  editing  information.  Each  of  the  atomic  tasks  is  performed  by  one  or  more  agents  who 
are  presented  as  autonomous,  goal-driven  and  sensitive  software  entities  with  abilities  to  learn,  co-operate  and 
react  within  complex  environments  (Travers  96).  Intelligent  Agents  are  used  in  the  context  of  Information 
Retrieval  that  involves  search  strategies  over  large  collections  of  documents  and  classification  of  the  available 
documents.  Concept  Mapping  is  one  of  the  main  techniques  used  to  support  domain  ontology  (structures).  It  is  a 
way  of  graphical  representation  of  concepts  and  their  relations  (nodes  - link  formalism)  or  visualisation  of  the 
human-like  knowledge  representation  based  on  schema  and  semantic  memory  models  (Collins,  Quillian  70).  It  is 
not  a new  knowledge  representation  language  but  just  a way  to  use  existing  languages  for  more  flexible  and 
dynamic  knowledge  representation.  An  important  issue  among  the  ones  already  mentioned  is  Information 
Visualisation.  There  are  two  main  goals  for  embedding  information  visualisation  techniques  within  information 
search  and  management.  The  first  is  to  facilitate  users’  comprehension  and  provide  means  for  easy  manipulation 
of  large  amounts  of  information  based  on  ontological  structures  and  ordering  (Welty  98).  The  second  one  is  to 
provide  an  easy  to  manipulate  overview  of  search  results  that  can  make  the  search  process  more  efficient  for  the 
user  and  that  will  support  successive  refined  or  new  searches. 

The  paper  starts  with  a discussion  of  the  information  retrieval  and  information  visualisation  issues  and  their 
implementation  in  AIMS.  In  the  following  section  we  focus  on  a short  description  of  the  AIMS  prototype.  The 
last  part  presents  some  conclusions  and  future  perspectives. 


Information  Retrieval  in  AIMS 

Information  retrieval  involves  two  main  issues: 

- effective  search  strategies  over  large  collections  of  documents; 

- appropriate  classification  of  the  available  documents. 

Search  strategies  concern  efficient  document  retrieval  in  respect  to  relevance  and  recall  of  the  obtained  results 
and  effective  document  retrieval  in  respect  to  computation.  The  efficiency  here  depends  heavily  on  the 
document  description  model.  Classification,  on  the  other  hand,  concerns  the  design  and  implementation  of  a 
framework  for  development  of  multiple  classification  subsystems  used  in  a single  document  retrieval  system. 

The  information  retrieval  approach  we  propose  is  based  on  using  intelligent  thesaurus  represented  by  the 
Concept  Map  technique.  This  thesaurus  plays  the  role  of  an  active  information/knowledge  dictionary  that 
supports  the  domain  ontology.  It  assists  users  in  identifying  similar,  broader  or  narrower  terms  and  expressions 
within  a specific  information  domain  and  obtaining  the  assigned  information.  The  classification  scheme  we  use 
is  based  on  conceptual  mapping  of  terms  and  relations  in  a specific  problem  domain.  Our  choice  is  based  on  CM 
analogy  with  human  mental  structures  and  means  of  handling  information.  Such  classification  facilitates  the 
dynamic  adaptation  of  the  system  to  changes  in  the  domain.  In  addition,  CM-based  classification  correlates 
efficiently  with  an  user  model  presented  also  in  the  form  of  a concept  map  of  user  knowledge,  preferences, 
styles,  and  tasks.  The  use  of  conceptual  representation  of  the  domain  knowledge  and  information  structure 
makes  the  system  dynamic  and  provides  methods  for  automatic  manipulation  of  its  information  and  knowledge. 
Classification  in  IR  concerns  grouping  and  selecting  document  descriptors  by  using  document  text,  statistics, 
user  feedback,  document  interrelation,  collaborative  filtering  or  other  criteria.  The  DoCS  (Document 
Classification  and  Search)  model  used  in  AIMS  is  keyword-based,  where  each  document  is  described  by  a 
correctly  constructed  set  of  keywords.  The  model  is  defined  as  a 6-tuple: 

DoCS  = (D,  K,  T,  Q,  E,  P),  where: 

D is  the  set  of  all  documents  in  the  database:  D = {Dj  | Dj,  ieN[l,n]}; 

K is  the  set  of  indexed  keywords  used  in  the  document  descriptors:  K = {kj  | ieN[l,n]}; 

T (thesaurus)  is  a set  of  keyword-synonyms  pairs,  describing  the  relations  among  the  keywords  in  K\ 

T = {(kj,  Sjj)  I kjeK,  Sj  = {syl  Sy  6 [0,1],  iJeN[l,4]}},  where  Sij  is  the  probability  that  the  user  would  relate  the 
keyword  kj  to 
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Q (user  query)  is  a set  of  coefficients  specifying  the  document  with  its  keyword  relevant  to  the  query:  Q={qi| 
qieN[0,l],  ieN[l,n]},  where  qj=l  when  keyword  ki  is  part  of  the  user  query,  otherwise  ^,=0; 

E (extended  user  query)  is  the  set  of  indexed  coefficients  for  the  calculation  of  the  relativeness  of  a document  to 

a query,  E = {ei|  ie  [l,n],  ei  = 5^j=[i,n]  qjSji,  qjCQ,  SjjG  Si}; 

P is  the  set  of  pairs  (document,  keyword  set):  P = {(Di,  Ki)  | DiG  D,  Ki  = {dj  | dje  R[0,1],  jeN[l,n]}.  It  presents 
set  of  coefficients  (probabilities)  that  relate  the  keyword  kj  to  the  document  D/  by  the  value  dj,  This  is  interpreted 
as  the  probability  with  which  the  user  giving  the  keyword  kj  would  be  satisfied  by  the  document  D,. 

The  matching  value  for  the  user  query  Q and  a document  D/  is  obtained  by  the  formula: 
m(Q,D|)=Zi^Ii,n]  Cidi,  where  eieE,  d[eD\. 


AIMS  document  database  includes  an  extensive  term  thesaurus  that  defines  a special  synonym  relation  in  order 
to  support  more  precise  and  effective  search  approximation  and  selection.  Every  keyword  has  a synonym  or  a 
list  of  synonyms  related  to  it  with  specified  weights.  Thus  if  keyword  Kj  is  related  to  (is  a synonym  of) 
keyword  K2  with  a specified  weight,  this  weight  can  be  interpreted  as  the  percentage  of  the  meanings  of  Kj  that 
match  or  are  near  to  some  meaning  of  K2.  When  a user  gives  a search  expression  the  system  searches  not  only 
for  the  words  in  the  expression  but  also  for  their  synonyms.  Synonym  relation  is  defined  as  a full  relation,  which 
means  that  any  two  keywords  are  considered  linked  but  some  with  zero  weight. 

The  search  procedure  for  a query  includes  calculation  of  the  matching  value  for  each  document  and  sorting  the 
documents  by  these  values  in  descending  order.  The  documents  with  higher  matching  value  are  considered  more 
relevant  to  the  query  than  the  ones  with  lower  matching  value.  This  algorithm  has  a simple  and  efficient 
implementation  by  the  use  of  RDBMS  and  SQL.  A SQL  query  implementing  the  search  procedure  would  look 
like  this: 

select  d.id,  d. title,  d.type,  d.url,  d.description 

from  doc_keyword  dk,  synonym  s,  document  d 

where  (d.id  = dk.did)  and  (dk.kid  = s.kidl) 

and  (s.kidl  in  [qkidl,  qkid2, ...  , qkidm*]) 

group  by  d.id,  d. title,  d.type,  d.url,  d. subtype,  d.description 

order  by  sum  ( dk. weight  * s. weight ) desc 

(*  qkidl  ..m  are  the  keyword  ids  used  in  the  query) 

The  presented  model  supposes  independence  of  the  documents,  the  keywords  and  the  synonyms  at  the  stage  of 
document  search.  This  does  not  affect  the  generality  of  the  solution  since  relations  between  different  objects  can 
be  accounted  at  the  classification  stage.  Some  of  the  advantageous  properties  of  the  presented  model  are  that  it  is 
easy  to  understand  and  easy  to  implement;  the  matching  function  is  intuitive  and  natural;  the  search  procedure  is 
computationally  efficient  in  regard  to  speed  and  storage;  the  refine  procedure  is  efficiently  implemented  by 
adding  new  query  keywords  to  the  old  ones  and  restarting  the  search  procedure,  instead  of  keeping  huge 
quantities  of  data  between  the  refine  queries.  Such  refined  search  is  more  likely  to  produce  relevant  results  then 
the  standard  ones,  which  use  only  the  documents  retrieved  after  the  first  search  was  initiated. 


Information  Visualisation  in  AIMS 

Information  visualisation  supports  user’s  thinking  and  comprehension  processes  by  activating  his/her  visual 
thinking  and  memory.  Visualisation  techniques  come  to  reduce  mainly  user’s  overhead  of  finding  the  needed 
information  and  supports  algorithmic  achievements  in  finding  potential  relevant  items. 

Visual  displays  in  our  work  are  mainly  concerned  with  consolidating  the  search  results  into  a graph  form  that  is 
easily  processed  by  the  user’s  cognitive  abilities  and  provides  him/her  with  an  additional  information  about  the 
context  of  the  retrieved  results.  The  graph  is  presented  with  the  means  of  a concept  map  giving  the  ontology  of 
the  domain  under  consideration.  It  is  the  main  skeleton  for  organising,  structuring  and  presenting  the  documents 
to  the  user.  It  contributes  to  the  effectiveness  of  search  activities  by  supporting  both  relationship-guided  search 
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for  relevant  information  and  its  visual  presentation.  Thus  the  user  is  able  to  navigate  through  the  information 
hierarchy  and  focus  on  specific  portions  of  it  but  still  within  the  context  of  the  domain  as  a whole  (Fig.  1 ). 

Within  the  AIMS  prototype  we  have  defined  a Concept  Map  (a  domain  term  network  of  structured  and  related 
concepts)  that  explicitly  describes  the  domain  structure  and  knowledge  in  the  area  of  Courseware  Engineering. 
All  documents  are  attached  to  the  concepts  (domain  terms).  Each  concept  is  a node  in  this  map  and  is  related  to 
other  terms  with  means  of  labelled  relations.  A customised  set  of  standard,  canonical  labels  defined  by 
(Lambiotte,  Dansereau,  Cross,  Reynolds  89)  is  used,  which  is  proved  to  be  useful  in  mapping  of  most  academic 
and  technical  domains.  The  map  provides  a spatial  arrangement  of  nodes  and  links  to  communicate  about 
concepts  and  to  specify  the  multiple  relationships  among  concepts  in  a given  information  or  knowledge  domain. 
This  helps  the  user  to  process  larger  numbers  of  document  resulting  from  searches  in  shorter  time,  and  to 
develop  a clear  view  (mind  picture)  on  the  information  domain:  all  terms  available  and  their  conceptual  relations 
(Jonassen,  Marra  96).  This  type  of  creative  visualisation  activates  user’s  imagination  by  providing  him/her  with 
an  additional  knowledge,  context,  meaning  to  the  information  used.  Visualisation  of  the  user  search  is  also  quite 
important  part  in  making  the  information  processing  more  effective  and  easy  for  immediate  analysis. 
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Figure  1:  Interface  of  AIMS:  display  of  search  results. 


AIMS:  General  System  Architecture 

In  AIMS  we  take  a social  viewpoint  to  an  information  system,  in  which  several  agents  interact  by  exchanging 
information,  sending  specific  requests,  offering  services,  accepting  tasks,  competing  with  each  other  for  a task  to 
be  accomplished  or  even  helping  each  other.  Following  this,  the  AIMS  prototype  was  developed  as  an  agent- 
based  information  environment  for  intelligent  management,  search  and  usage  of  distributed  information 
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resources.  We  look  at  information  manipulation  in  complex  information  domains  in  a new  way:  we  do  not 
conceive  it  as  a mere  application  of  (simple)  search  engines  to  collections  of  information  materials.  We  rather 
see  it  as  a task  that  necessitates  concise  system  architecture  bringing  together  the  main  functional  modules  for 
manipulating  collections  of  resources  situated  within  a knowledge  and  information-sharing  environment.  A 3- 
dimentional  architectural  framework  is  applied  employing  multi-agent  support  for  intelligent  information 
manipulation  over  a visual  representation  of  the  information  domain,  and  a conceptual  organisation  of 
information  resources  involving  semantic  mapping  techniques  (Aroyo,  De  Diana,  Dicheva  98). 

AIMS  presents  an  attempt  for  integral  solution  for  the  organisational  management  of  resources  as  they  are 
available  in  a dynamic  and  constantly  changing  (electronic)  information  distributed  environment.  From 
architectural  point  of  view  the  system  is  built  as  a multy-agent  system.  Agents  perform  different  tasks,  such  as 
search,  presenting  the  results  to  the  user,  user-modelling,  etc.  Each  agent  task  consists  of  atomic  sub-tasks. 
Agents  have  co-operation  abilities  (communicate  with  each  other,  change  their  behaviour  depending  on  the  user 
action),  communication  ability  (exchange  messages),  activeness  (could  be  active  even  when  there  is  no  clear 
user  request).  The  primary  task  of  the  system  is  retrieving  information  and  presenting  the  results  to  the  user.  The 
agents  perform  this  working  in  parallel  as  separate  but  co-ordinated  processes. 

Three  of  the  AIMS  agents  are  related  to  Document  Search,  Classification  and  Visualisation  task:  the  Search 
Agent,  the  Domain  Expert  (Classification)  Agent,  and  the  Interface  Agent.  The  Search  Agent  employs  the  DoCS 
model  to  perform  intelligent  search  activities  over  the  given  information  domain.  It  receives  messages  that  are 
well-specified  queries  and  finds  the  documents  relevant  to  those  queries. 

The  Domain  Expert  Agent  is  responsible  for  a set  of  domain  specific  tasks,  like  maintaining  the  concept  map  of 
domain  terms  (add,  delete,  check,  reorganise,  etc.)  and  by  doing  this  it  is  responsible  for  the  changes  going  on  in 
the  domain  database.  Further  automation  of  this  agent  module  can  lead  to  more  sophisticated  tasks  in  respect  to 
learning  and  knowledge  building. 

The  Interface  Agent  is  responsible  for  the  communication  with  the  user.  It  visualises  the  information  domain 
structures,  search  activities,  and  search  results  by  presenting  graphically  different  parts  of  the  concept  map.  Its 
main  task  is  to  give  the  user  attractive  and  effective  information  presentation  in  order  to  make  the  system  more 
user-friendly  and  adaptable  to  single  user  profiles.  In  order  to  make  it  more  natural  for  the  user  we  have 
combined  graphical  representation  of  the  concept  map  with  textual  presentation  of  list  of  documents  related  to 
each  domain  term  in  the  concept  map. 


Conclusion 

This  paper  presents  an  approach  to  effective  and  efficient  information  management  by  connecting  together  in 
one  system  intelligent  agents,  information  visualisation  and  information  classification  techniques.  We  discuss  an 
agent-based  information  system  AIMS.  The  core  of  AIMS  is  a knowledge  base  containing  the  domain  model, 
the  user  model  and  a set  of  rules  for  concept  analysis,  pattern  matching  and  decision  making.  The  different  types 
of  knowledge  and  information  in  the  KB  are  presented  and  manipulated  using  the  same  Concept  Map  - based 
knowledge  representation  and  reasoning  techniques.  This  model  allows  organising  the  different  topics  around 
the  key  concepts  (terms)  of  the  domain.  As  a testbed  we  have  developed  a conceptual  model  of  the  domain  of 
Courseware  Engineering. 
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Abstract:  This  paper  documents  the  rationale  of  the  design  of  the  ALICE  virtual 
classroom  to  support  an  undergraduate  course  on  web  literacy  offered  in  1997-1998 
at  National  Tsing  Hua  university  in  Taiwan.  It  describes  the  ALICE  project,  in 
which  students  used  a variety  of  WWW-based  learning  materials  to  supplement 
collaborative  group  learning.  Major  findings  of  this  study  reveal  that 
overwhelmingly  most  of  the  students  were  satisfied  with  the  use  of  the  ALICE 
virtual  classroom.  ALICE  was  perceived  by  the  students  to  be  a very  innovative, 
useful  and  openly  accessible  virtual  classroom  that  met  their  different  needs  and 
allowed  learning  at  their  own  convenience  and  without  limitations  of  time  and 
space.  This  study  also  identifies  the  characteristics  of  those  learners  who  can  get 
most  advantage  from  using  this  new  medium.  Implications  for  web  supplement 
design  and  the  role  of  teachers,  and  recommendations  for  further  research  are  also 
discussed. 


Introduction 

The  advent  of  Internet  technologies  provides  educators  with  exciting  and  alternative  methods  of 
delivering  instruction  to  and  communicating  with  their  students.  Integrating  the  World  Wide  Web  (WWW) 
and  the  Internet  in  higher  education  is  a trend  nowadays,  and  it  has  been  the  focus  of  much  research  on 
instruction  and  learning  over  the  past  few  years  (Wells,  1992;  Lai,  1997;  Collis,  Andemach,  & Diepen,  1997; 
Roussos,  Johnson  & Leigh,  1997;  Freeman,  1997;  Oliver  & Omari,  1998).  The  use  of  the  WWW  and  the 
Internet  in  educational  settings  can  be  presented  in  a number  of  forms,  such  as  information  access,  interactive 
learning,  networked  communication  and  information  presentation  (Oliver  & Omari,  1998)  or  overall  in  a 
virtual  classroom  which  represents  all  possible  class  functions  on  the  Internet  (Porter,  1997).  According  to  a 
study  conducted  by  Wells  (1992),  with  its  technological  capabilities,  the  Internet  has  become  an  increasingly 
effective  way  for  teachers  to  offer  computer-mediated  distance  courses  without  compromising  the  quality  of 
instruction. 

Using  Telecommunication  Technologies  to  Reform  Education 

Over  the  past  few  decades,  constructivism  has  attracted  the  attention  of  educators  in  every  discipline. 
Because  the  hyperlink  techniques  of  the  Internet  support  the  basic  ideas  and  claims  of  constructivism,  the 
advance  of  telecommunication  technologies  and  the  rise  of  constructivism  have  brought  fresh  ideas  to  school 
settings.  Following  constructivist  views,  educators  have  made  schooling  more  student-centered:  learning 
happens  when  students  assimilate  information  based  on  their  individual  experiences,  values,  attitudes  and 
beliefs.  Constructivists  believe  that  learning  occurs  most  effectively  in  contexts;  educators,  therefore,  should 
establish  real-world-like  situations  rather  than  decontextualizing  learning  in  an  isolated  schooling 
environment  (Jonassen,  1991). 

According  to  Gibson’s  theory  of  affordances  (Gibson,  1977;  Gibson,  1979;  Gibson,  1986),  different 
types  of  instructional  materials  encourage  or  afford  different  types  of  activities.  The  affordances  of  the 
advanced  telecommunication  technologies  on  the  Internet  make  a computer-supported  real-classroom  like 
learning  environment  available.  Studies  (Sun  & Chou,  1996;  Freeman,  1997)  indicate  that  the  use  of  Internet 
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technologies  with  the  affordances  of  multimedia  presentation  could  provide  learners  with  a more  creative  and 
innovative  interactive  learning  environment  in  which  flexible  and  alternative  modes  of  content  were  delivered. 
Multimedia-based  Internet  courses  could  accommodate  various  kinds  learners  preferred  mode  of  learning,  such 
as  hands-on  learners,  traditional  learners,  visual  leamersc.  If  the  courses  on  the  Internet  are  well  designed, 
they  could  offer  learners  a wide  range  of  choices,  so  that  learners  could  find  the  right  mix  of  interaction  and 
learning  style  to  enhance  their  individual  capacity  to  learn. 

Description  of  the  ALICE  Project 

ALICE,  an  acronym  for  Asynchronous  Learning  in  Interactive  Computer-network-supported 
Environments,  is  a research-instructional-oriented  project  funded  by  the  National  Science  of  Council  (NSC)  of 
Taiwan.  It  is  basically  a virtual  classroom  created  on  the  Internet.  The  ALICE  system  intends  to  provide  a rich, 
interactive  and  collaborative  setting  used  to  enhance  learning  and  interaction  for  an  undergraduate  course  on 
web  literacy  offered  in  the  Center  for  General  Education. 

Based  on  the  constructivist  views  of  instructional  system  design  in  both  concept  and  construction 
(Lebow,  1993;  Jonassen,  1997),  ALICE  aims  to  be  an  inviting  environment  that  integrates  four  major 
principles  as  follows: 

A familiar  distant  interactive  learning  environment  that  is  similar  to  a traditional  classroom  setting  with  the 
integration  of  computer  networks,  multimedia  presentations,  data  bases,  and  the  like; 

1)  A student-centered,  openly  accessible  channel  that  encourages  and  supports  students'  active  and  self- 
regulated  learning  processes  and  that  accommodates  the  needs  of  their  personal  learning  style; 

2)  Both  a well-structured  and  ill-structured  environment  that  on  the  one  hand  provides  the  learner  with  a 
course  pre-organizer,  such  as  an  on-line  syllabus  and  rich  and  digestible  domain  knowledge  in  web 
applications;  and  on  the  other  hand  allows  the  learner  to  flexibly  hyperlink  and  explore  issues  related  to 
networking  computer  applications  in  learning  and  living; 

3)  An  interactive  and  collaborative  environment  that  provides  settings  for  a community  of  practice  (Lave  & 
Wenger,  1991)  through  engaging  and  dynamic  interactions  in  sharing  information  and  knowledge  between 
students  and  teachers  and  among  students. 

The  ALICE  virtual  classroom  is  secured  with  the  use  of  students  IDs  and  passwords.  Some  pages, 
such  as  Bulletin  Board,  Conference  Area,  and  Assignment,  use  forms  and  appropriate  CGI  scripts  at  the  server 
end  to  manage  the  interactive  messages.  There  are  eight  components  in  the  virtual  classroom,  including: 

1)  Bulletin  Board:  providing  learners  with  the  most  recent  announcements; 

2)  Syllabus:  including  the  rationales  and  goals  of  the  course,  tentative  course  schedule,  weekly  assignments, 
criteria  for  grading,  required  and  suggested  readings; 

3)  Instructors:  presenting  the  instructor's  personal  background  information,  such  as  photos,  degrees  received, 
academic  and  research  interests,  correspondence; 

4)  Students:  connecting  to  individual  and  group  students'  homepages  that  contain  rich  and  various 
presentation  styles; 

5)  Conference  Area:  allowing  students  and  instructors  as  well  as  system  masters  to  communicate  with,  post, 
share,  reflect  and  discuss  what  they  have  browsed  on  the  Internet.  It  expands  the  nature  and  scope  of 
themes  originally  designed  for  the  course; 

6)  Assignments:  allowing  each  group  to  post  their  group  discussion,  reports  and  final  projects; 

7)  On-line  Readings:  uploading  weekly  related  reading  materials  in  text-based,  multimedia-based  or 
powerpoint-based  electronic  forms; 

8)  Network  Resources:  linking  to  related  web  sites  in  order  to  fully  utilize  the  rich  resources  on  the  Internet. 

Methodology 

1.  Purpose  of  the  Study 

The  purpose  of  this  study  was  to  investigate  possible  uses  of  the  ALICE  virtual  classroom  to  support 
learning  and  instruction  on  an  elective  Web  Literacy  course  in  general  education.  The  study  sought  to 
investigate  answers  to  the  following  sets  of  questions: 


238 


1 . What  were  the  perceptions  of  students  to  the  use  of  the  ALICE  virtual  classroom  as  an  information  source 
and  alternative  environment  to  support  learning? 

2.  Are  learning  styles,  access  to  ALICE,  and  prior  computer  knowledge  and  experience  important  factors? 
What  were  the  factors  affecting  the  attitudes  of  the  students  towards  computer-mediated  discussion?  What 
were  the  characteristics  of  learners  who  chose  to  take  this  course? 

3.  Did  the  use  of  the  ALICE  system  support  collaborative  learning? 

4.  What  challenges  could  a teacher  be  confronted  with  in  using  advanced  technologies  to  support  instruction? 

2.  Research  Design  and  Data  Collection 

This  study  was  conducted  using  both  qualitative  and  quantitative  research  methods.  The  primary  data 
collection  methods  included  the  use  of  observations,  formal  and  informal  interviews,  and  two  sets  of  pre-  and 
post-  survey  questionnaires  which  were  administered  to  students  to  solicit  information  on  demographic  data, 
attitudes  and  prior  experiences  and  knowledge  toward  computing  network  technologies.  Observations  were 
conducted  weekly  during  class  time  in  16  weeks.  The  researcher  kept  field  notes  and  reflexive  notes 
immediately  after  each  observation.  Meanwhile,  notes  taken  from  informal  interviews  served  as  an  adjunct 
data  source.  In  addition,  traffic  on  the  web  site  was  logged.  The  Webtrend  program  was  used  to  analyze  the 
traffic  and  provide  summary  access  information  on  all  of  the  files/pages  presented  at  the  site.  Messages  posted 
to  the  Conference  Area  were  analyzed  to  examine  the  patterns  of  communication. 

Triangulation  was  used  to  improve  the  probability  that  findings  and  interpretations  found  would  be 
credible.  Multiple  sources  of  data  verified  that  the  data  collected  in  the  survey  were  consistent  with  what  was 
being  said  in  the  interviews  and  observed  in  classrooms.  Data  were  collected,  transcribed,  reduced,  coded, 
categorized  and  analyzed  in  response  to  the  research  questions. 

3.  Participants 

Participants  involved  in  the  study  were  54  undergraduates,  49  male  and  5 female  students,  enrolled  in 
an  elective  general  education  course  entitled  "Computer  Networks,  Learning  and  Living".  They  were  from  6 
colleges  (Science,  Engineering,  Electrical  Engineering  and  Computer  Science,  Nuclear  Science,  Life  Science, 
Humanities  and  Social  Sciences)  of  a northern  national  university  located  in  Taiwan.  Their  grade  levels  ranged 
from  undergraduate  freshman  to  senior. 

4.  Instructional  Setting  and  Course  Delivery 

"Computer  Networks,  Learning  and  Living",  a 3 credit-hour  course,  is  basically  an  interdisciplinary 
course  designed  for  learners  from  different  disciplines.  The  purpose  of  this  design  is  to  get  students  from  a 
variety  of  backgrounds  to  share  and  contribute  their  thoughts  or  techniques  on  web  literacy  from  various 
perspectives.  In  addition  to  a weekly  class-based  two-hour  seminar,  the  course  places  a great  emphasis  on  the 
use  of  the  ALICE  virtual  classroom,  so  that  participation  on  the  Internet  counts  for  one  credit  hour.  This 
pedagogical  strategy  was  employed  to  increase  students’  involvement  on  the  Internet. 

All  of  the  related  information  and  reading  materials  were  stored  on  the  Internet.  No  handouts  were 
distributed  at  the  weekly  class  seminar.  From  the  outset  of  the  semester,  students  were  requested  to  form  10 
groups  on  a voluntary  basis,  and  a mix  of  academic  backgrounds  and  different  computer  experiences  in  each 
group  was  highly  encouraged.  Since  this  course  was  designed  for  collaborative  learning,  group  scores  were  a 
big  proportion  of  each  individual's  grade.  The  students  were  given  a series  of  questions  and  activities  to  guide 
their  readings  and  inquiry.  Students  were  required  to  use  the  resources  at  the  ALICE  virtual  classroom  and 
other  web  sits  to  answer  the  questions  and  prepare  for  the  group  discussions,  which  formed  the  basis  of  the 
classwork  each  week.  Each  group  leader  was  responsible  for  the  postings  of  the  group  reports  on  the 
"Assignment"  interface  for  critique. 

Furthermore,  students  were  directed  to  seek  out  new  resources  and  were  required  as  part  of  their  course 
to  recommend  10  relevant  web  sites  with  personal  comments  which  represented  a student’s  summary  or 
impression  of  the  information  presented  at  the  site.  At  the  end  of  the  semester,  each  group  was  required  to 
complete  a theme-based  homepage  project.  Collaboration  enabled  them  to  learn  about  homepage  production  by 
HTML,  Photoshop,  GIF  Construction,  or  CGI  script.  Each  student  was  expected  to  have  interactive  hands-on 
learning  experiences  through  a community  of  practice  (Lave  & Wenger,  1991). 
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Preliminary  Data  Analysis  and  Results 


1.  Students’  Perceptions  of  ALICE 

Major  findings  of  this  study  reveal  that  overwhelmingly  most  of  the  enrolled  students  were  satisfied 
with  the  use  of  the  ALICE  virtual  classroom  and  were  positive  in  their  responses  to  questions  regarding  the 
value  of  information  obtained  from  the  links  provided  weekly.  Feedback  from  the  students  indicated  their  love 
of  the  image  of  ALICE  borrowed  from  "Alice  in  Wonderland"  created  by  Lewis  Carroll  in  the  1860s.  They 
enjoyed  the  new  interpretation  as  ALICE  in  Modem  Wonderland-Cyberspace.  Yes.  It  was  expected  that  every 
student  would  become  a happy  ALICE  and  learn  through  the  Internet.  According  to  the  students,  ALICE  was 
a very  innovative,  useful  and  openly  accessible  virtual  classroom  that  met  their  different  needs  and  allowed 
them  to  learn  at  their  own  convenience  and  without  limitations  of  time  and  space.  However,  some  students 
remarked  on  the  frequent  instability  of  university  server  systems  and  difficulties  in  accessing  the  virtual 
classroom  when  traffic  might  be  heavy  in  the  daytime.  From  the  students'  traffic  patterns  collected  from  the 
ALICE  virtual  classroom,  it  was  found  that  students  who  possessed  personal  computers  had  more  convenient 
access  24-hours  per  day  than  those  who  did  not. 

2.  Who  Can  Benefit  from  ALICE  ? 

According  to  the  data  gathered  from  the  questionnaires,  among  the  54  enrolled  students,  49  (91%) 
were  male  students  and  5 (9%)  females.  Referring  to  the  undergraduate  population  of  the  same  academic  year 
of  1997-1998,  the  ratio  between  males  (total  2601)  and  female  (total  1073)  was  2.6  to  1.  Obviously,  female 
students  were  underrepresented  in  this  course.  Their  self-report  data  showed  that  100%  of  the  enrolled 
students  had  prior  computer  experiences,  ranging  from  beginner  to  expert.  Most  of  them  claimed  to  be  at 
intermediate  level.  When  looking  at  students'  academic  backgrounds,  we  found  that  the  distribution  of  their 
College  origins  was  not  even.  52%  of  the  enrolled  students  were  from  the  college  of  Electrical  Engineering 
and  Computer  Sciences  and  the  rest,  48%,  were  from  College  of  Science  (19%),  College  of  Engineering 
(1 1%),  College  of  Nuclear  Science  (9%),  College  of  Life  Science  (7%)  and  College  of  Humanities  and  Social 
Sciences  (2%).  Consequently,  students  (52%)  from  the  College  of  Electrical  Engineering  and  Computer 
Sciences  had  strong  interest  in  taking  this  course,  while  the  lone  student  (2%)  from  the  College  of  Humanities 
and  Social  Sciences  expressed  the  least  interest.  In  several  interviews  with  students  of  College  of  Humanities 
and  Social  Sciences  who  dropped  out  of  this  course,  they  unanimously  expressed  their  opinion  that  the  course 
was  difficult  for  them  because  they  got  stuck  and  frustrated  on  hearing  a lot  of  terminology  about  the  Internet 
at  the  first  two  weeks.  Actually  they  did  not  use  computers  in  their  daily  life.  Lack  of  comfort  and  confidence 
with  the  computing  technologies  was  the  main  reason  given  for  dropping  out. 

3.  Challenges  to  the  Role  of  Instructors 

The  data  collected  from  the  instructor's  field  notes  indicate  that  in  addition  to  the  weekly  two-hour 
class  seminar,  the  instructor  had  to  spend  an  extra  six  to  ten  hours  per  week  at  the  virtual  classroom.  In  order 
to  keep  tracking  students'  weekly  learning  and  their  participation,  the  instructor  spent  large  amounts  of  her 
time  attempting  to  interact  with  students'  email  and  WWW  postings  on  the  Conference  forum  and  read 
through  or  visited  related  materials  or  web  sites  as  recommended  by  the  students.  Ironically,  the  benefits  of  the 
Internet  to  the  students  could  become  an  enormous  burden  to  the  instructor.  Accustomed  to  the  use  of 
telecommunication  technologies,  such  as  e-mail  and  BBS,  younger  generations  are  gradually  conditioned  to 
get  instant  reply  /feedback.  If  timely  replies  were  not  given,  students  expressed  their  impatience. 

Offering  web-supported  courses  is  not  an  easy  job  for  the  instructor  in  terms  of  web  course  design, 
time  involvement,  system  maintenance  and  so  on.  First  of  all,  it  takes  extra  time  and  efforts  to  create  a web 
environment  and  design  multimedia  courses  on  the  Internet.  Along  with  the  attributes  of  the  new  medium, 
such  as  high  efficiency  in  communication  and  open  access  and  interactivity,  using  the  Internet  for  instruction 
is  more  labor  intensive  and  time  consuming.  The  role  of  educators  is  changing  with  the  integration  of  modem 
technologies.  Instructors  in  this  computer-network-mediated  instructional  setting  are  becoming  facilitators  and 
learners  as  well.  Because  the  technologies  advance  at  amazing  speed,  instructors  are  actually  learning  from 
their  students,  particularly  the  technical  aspects  of  advanced  computing  technologies.  Instruction  and  learning 
are  occurring  at  the  same  time. 
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Summary  and  Conclusion 


The  design  and  implementation  of  the  ALICE  virtual  classroom  to  supplement  a conventional  course 
provided  expected  outcomes  from  the  students’  side  but  also  several  unexpected  challenges  to  the  instructors. 
This  study  indicated  the  appropriate  integration  of  the  Internet  could  produce  real  learning  but  not  virtual 
learning  only,  as  claimed  by  Ashworth  (1996).  Originally,  the  purpose  of  this  course  was  to  integrate  the 
ALICE  virtual  classroom  with  teaching  about  web  literacy  for  students  from  diverse  backgrounds.  This  study 
supported  our  contentions  concerning  the  potential  of  ALICE  to  provide  more  innovative,  motivating, 
engaging,  useful  and  convenient  learning  opportunities  for  the  enrolled  students  even  though  at  times  ALICE 
as  an  information  source  proved  frustrating  and  annoying  to  learners  on  several  occasions  when  the  server  was 
down  or  the  information  could  not  be  easily  retrieved.  However,  the  findings  also  revealed  a numbers  of  factors 
which  served  to  hinder  learning  opportunities  for  a certain  group  of  students.  For  example,  female  students 
and  those  students  from  backgrounds  in  Humanities  and  Social  sciences  were  underrepresented  in  this  course. 
They,  in  general,  lacked  of  prior  computer  experiences  and  were  not  confident  and  comfortable  in  taking 
computer-related  courses.  It  is  important  to  take  these  factors  into  consideration  for  future  course  design  and 
offerings. 

Incorporating  advanced  technologies  into  an  existing  learning  environment  can  be  an  exciting  but 
challenging  task.  How  to  help  students  learner  better  and  more  effectively,  supplemented  by  advanced 
technologies,  and  how  to  make  teachers  enjoy  giving  the  course  without  experiencing  too  much  stress— these 
questions  deserve  further  study  from  the  aspects  of  instructional  design,  teaching  load,  intellectual  property  on 
the  Internet,  teacher  training  and  logistical  support. 
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Abstract:  This  paper  explores  how  the  use  ofinultimedia  communications  technologies  to  enable  elements  of  real- 
worldworking  knowledge,  that  are  tacit  andembedded  in  working  practices  to  be  rendered  into  shareable  forms 
forprofessional  learning.  Communications  technology  offers  innovative  ways  forgeographically  distributed  professional 
communities  to  create,  annotate,discuss  and  reflect  upon  multimediaobjects  that  capture  working  practices,  such  as 
problems  of  practice.  Thepaper  reports  on  work-in-progress  on  multimedia  objects  (primarilydigitised  videoclips).  Use  of 
multimedia  in  professional  learning  isrelatively  novel,  and  it  is  surfacing  issues  about  the  requirements  for  creating 
shareable,  effective  representations.Descriptors  for  multimedia  objects  have  been  developed  and  the  paperfocuses  on 
applying  the  descriptors  to  an  example  videoclip.  The  durationof  the  clip,  it's  use  of  artefacts  and  the  social  cues  in  the  clip 
are  considered  key  factors  in  creating  ashareable  representation  for  discussion  and  reflection  in  distributedprofessional 
communities. 


Introduction 

This  paper  concerns  asynchronous  collaborative  multimediaenvironments  used  to  support  professional  development; 
to  enable  key  elements  of  real- world  working  knowledge,  that  are  tacit  and  embedded  in  working  practices,  to 
berendered  into  shareable  forms  for  improving  working  practices  and  forprofessional  learning.  We  believe 
multimedia  communications  technology  can  offer  innovative  ways  of:  a)capturing  rich  examples  of  working 
practices  (and  the  tacit  knowledgetherein);  b)  rendering  the  examples  into  communicable  objects;  c)  sharingand 
subjecting  these  objects  to  ^close  scrutiny  within  a community  of  learners;  and  by  so  doing  d)  create 
authenticexperiential  and  reflective  environments  for  learning. 

Real-world  practice  as  performed  by  professionals  is  highly  complex. Skilled  practitioner  performance  is 
underpinned  by  contextualised  andvocationally-relevant  knowledge  (eg  Schon,  1983;  Lave  & Wenger,  1991).  This 
situated  knowledge,  tied  to  anddeveloped  through  practice,  is  considered  highly  valued  for  improvingactual  working 
practices  (Cervero,  1992;  Koschmann,  1996).  It  is  part  of  what  shapes  a community  of  practice  and  helps  makes  it 
an  authentic’arena  for  learning’  (Barab  & Duffy,  in  press).  However,  since  thisknowledge  is  tacit  and  embedded 
within  practice,  it  can  be  very  difficultfor  practitioners  to  describe  and  share  their  knowledge  (especially  in  a written 
text),  without  making  an  extendedaccount  of  the  context,  tasks,  etc.  Schon  (1987)  has  argued  by  observing  and 
reflecting  on  action,  it  is  possibleto  reveal  the  tacit  'knowing’  implicit  in  ’doing’.  This  paper  describes  a new 
approach  to  sharing  real-world  knowledge  for  learning  aboutprofessional  practice:  an  approach  that  exploits 
asynchronous  multimediacommunication  technologies. 

There  are  three  key  elements  to  this  approach.  Firstly,  multimediatechnology  enables  us  to  capture  (eg  in  video 
clips)  rich  and  vivid  examples  ofpractitioners  engaged  in  real-world  practice.  Secondly,  the 
communicationstechnology  enables  the  captured  examples  (or  multimedia  objects)  to  beshared  and  discussed  among 
distributed  professional  communities.  Not  least,  the  asynchronous  medium  means  the  objectsand  discussions  can 
persist  for  revisiting,  for  reflection,  and  they  can  bemade  available  to  other,  especially  newer  members  within  a 
community. 
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Next,  the  context  for  this  research  is  described  and  the  kinds  of  representation  that  might  be  captured  in  amultimedia 
object  are  presented.  The  paper  focuses  on  the  creation  of  justone  kind  of  representation,  making  use  of  descriptors 
to  classify  and  tohelp  understand  key  aspects  in  the  multimedia  object. 

The  SHARP  and  Voice  Annotation  of  Multimedia  ArtefactsProjects 

This  paper  reports  on  two  discrete  but  related  projects:  (i)  aEuropean-funded  project,  SHARP  (SHAreable 
Representations  of  Practice). SHARP  concerns  pedagogical  and  organisational  aspects  in  the  use  of  asynchronous 
multimedia  conferencing(AMC)  (see  also  SHARP  Project  Team,  1998);  (ii)  an  ESRC  (UK's  Economic  andSocial 
Research  Council)  project.  Voice  Annotation  of  Multimedia  Artefacts. This  project  explores  multimedia 
communications  technologies  used  for  reflective  learning  from  group  basedtasks. 

In  both  projects,  we  have  developed  example  video  clips  to  help  builda  conceptual  model,  and  as  a way  of 
simplifying  and  ordering  the  space  ofpossibilities  within  which  continuing  professional  development  might  occur 
when  using  distributedmultimedia  communications  technologies  (Goodyear  & Steeples,  1998) 

Kinds  of  (video)  Representations  of  Practice 

Three  broad  categories  and  six  potential  'kinds'  of  videorepresentation  of  practice  have  been  identified.  The 
distinction  bQt'weQnkinds  reflects  their  distance  from  the  purity  of  a working  practice,  occurringin  (non-observed) 
normal  conditions.  They  therefore  represent  instancesalong  a continuum  from  near-pure,  through  degrees  of 
abstraction,  to  a wholly  artificial  rendering  of  practice,  (see  alsoGoodyear  & Steeples,  1 998).  The  broad  categories 
include: 

Near  pure  representations  eg  'fly  on  the  wall',  and  'thinkaloud'  or  'concurrent  verbalisation'.  To  illustrate,  we  use 
theterm  'fly  on  the  wall'  for  video  clips  which  are,  in  some  sense,  'raw'representations  of  practice:  clips  generated  by 
techniques  which  TVdocumentary  makers  call  'fly  on  the  wall',  because  the  camera  - and  thefact  that  a 
representation  is  being  created-  are  meant  to  have  no  significant  disruptive  effect  on  the  practice. 

Action  with  commentary  representations  eg  'commenting  whiledoing',  'talking  head',  and  'prepared  script'.  Clips 
which  we  call  'actionwith  commentary'  are  those  in  which  the  practitioner  uses  the  voice  track  to  a video 
representation  of  an  unfolding  practice, as  a way  of  making  that  practice  more  comprehensible  to  others 

While'near  pure'  representations  may  seem  intuitively  appealing  as  ones  mostlikely  to  capture  the  essence  of 
practice,  trial  work  has  revealed  important  difficulties  in  adopting  such  anapproach.  For  any  kind  of  extended  or 
complex  practice,  many  hours  of  videowill  likely  be  produced,  requiring  skilful  editing.  This  suggests  thesekinds  of 
representation  have  a number  of  major  limitations,  not  least  in  terms  of  their  cost  effectiveness.  Incontrast,  'action 
with  commentary'  kinds  appear  to  have  a useful  balance  ofstrengths  over  weaknesses.  They  cafi  still  be  firmly  rooted 
in  real  worldpractice,  but  created  without  unduly  excessive  demands  upon  practitioners. 

Descriptors  for  multimedia  objects 

To  help  make  sense  of  representations,  a set  of  descriptors  can  beapplied  to  multimedia  objects.  A four-way 
partitioning  of  the  descriptorsis  made  under  the  headings  of  'substantive',  'communicative',  'technical'  and  'social' 
aspects.  Figure  1 below  summarises  the  descriptors.  The  descriptors  help  to  make  sense  of  thevariety  of  factors  that 
can  be  applied  when  describing  a multimedia  object.The  descriptors  help  when  looking  in  detail  at  exemplar  (video) 
objects,  to  identify  relationships  betweenthe  descriptors  and  to  look  for  particular  patterns  emerging 
betweendescriptors.  A brief  outline  is  given  below  for  the  descriptors,  using  theterm  'subject'  to  denote  the 
practitioner. 
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Substantive 

Communicative 

Social 

Technical 

• Nature  of  practice 

• Intent 

• Number 

• Method  of  capture 

• Visibility 

• Audience  awareness 

• Gender  | 

• Duration 

Temporal  extent 

• Discoursemoves 

• Role 

•File  size  and  format 

• Involvement  of  artefacts 

• Use  of  artefacts 

• Status 

• Richness  in  form 

• Role/status 

• Completeness 

•Cues  given 

•Output  provision 

Figure  1 .’descriptors  for  multimedia  objects 


SubstantivediS^QZis  include  looking  at  the  nature  of  the  practice  itself:  eg  what  it  isabout  and  whether  there  are  sub- 
activities; who  is  involved;  where  ithappens;  its  frequency,  etc.  Substantive  descriptors  also  include  thevisibility  of 
the  practice  especially  if  it  is  primarily  a cognitive  task, where  little  visible  activity  would  normally  occur.  The 
temporal  extent  of  the  practice  also  needs  to  be  considered. For  example,  the  practice  may  last  just  a few  minutes,  or 
it  may  extendover  weeks,  even  years.  It  may  be  important  too,  to  examine  the  practice'suse  of  artefacts  which  might 
assist  in  the  process  of  referential  anchoring  (Resnick,  1993)  and  thus  may  actas  cognitive  aids  and  help  in 
communication  about  the  practice  (Saljo,1995).  The  subject’s  status/role  in  the  practice  may  also  beidentified. 

Communicative  aspects  are  those  thatmake  the  object  communicable  and  shareable.  'Intent'  concerns  thesubjecfs 
purpose  in  creating  the  representation,  eg  to  show  a practicebelieved  to  be  of  value  to  others,  or  to  seek  help,  or  to 
demonstrate  aspart  of  an  ongoing  discussion.  Communicativeaspects  also  include  the  extent  to  which  the  subject  is 
aware  of  anaudience  when  making  the  video  clip,  and  whether  the  subject  draws  uponartefacts  to  help  give  an 
explanation.  'Discourse  moves'  classify  thelinguistic  form  of  the  representation  eg  as  a question,  a statement,  a 
disagreement,  etc.  We  are  also  interested  inhow  ‘complete’  an  object  might  be,  or  whether  it  is  reliantupon  later 
elaborations,  such  as  in  an  annotation,  to  render  it  into  ashareable  form. 

Social  aspects  are  primarily  suggested  for  objects  involving  more  than  one  person,  where  we  might  identify  aspects 
of  group  dynamics  (such  as  gender  and  status)  in  multimedia  objects.  (Eg,  of  a video  clip  we  might  ask  'what  is  it 
about  the  way  this  group  work  together  and  the  roles  they  adopt  that  helps  create  a thought-provoking  representation 
of  practice?').'Social  cues'  relates  closely  to  communicative  aspects,  but  concerns  theextent  to  which  the  subject  uses 
language,  action  or  voice  tone  to  suggesta  sense  of  community,  eg  by  inviting  others  into  a discussion. 

7ec/z«/ca/aspects  relate  to  the  specifics  of  creating  theobject  including  the  method  and  management  of  capture.. 
Technical  aspectsalso  include  the  object's  file  size  and  its  duration.  Duration  is  likely  tobe  critical  in  determining  the 
usability  by  others  of  a clip.  Technically,  the  clip  can  also  bedescribed  in  terms  of  its  richness  in  form,  ie  whether  it 
includes  movingor  still  images  and/or  voice.  Output  provision  will  describe  thefunctionality  offered  to  users  to 
control  the  play,  eg  whether  the  clip  can  be  replayed,  if  play  can  be  halted, moved  back  and/or  forward,  etc. 

Applying  the  descriptors  to  an  example  multimedia  object 

In  this  section  an  example  video  clip,  created  with  a practitioner  inthe  educational/training  technology  field,  is 
examined  using  the  descriptors  for  multimediaobjects.  The  reader  may  find  it  useful  while  reading  this  paper  to  look 
atthe  clip  on  the  SHARP  website  at: http:// www.lancs.ac.uk/users/edres/research/sharpdemo/mul timed.htm 

Explaining  a problem  of  practice 

This  video  clip  is  used  to  explain  and  reflect  on  an  authenticwork-related  problem.  In  terms  of  kinds  of 
representation,  wesuggest  this  clip  falls  within  the  family  of  ‘action  withcommentary’,  and  is  closest  to  a talking 
head  kind  of  clip.  In  this  case  the  subject,  Sonia,  is  using  a computer  program  todesign  and  develop  a paper-based 
A5  booklet.  Sonia  encounters  this  task  anumber  of  times  each  year,  but  not  sufficiently  regularly  to  expendtargeted 
effort  in  learning  how  to  cope  with  it  in  more  time-efficient  ways.  The  problem  is  usually  met  whenworking  to  a 
deadline,  when  it  is  essential  to  be  working  directly  on  task,rather  than  learning  how  better  to  use  the  tool. 

Sonia  explains  to  camera,  using  a booklet  to  demonstrate  aspects  of  the  problem.  The  clip  provides  a relatively  easy 
way  to  show  asshe  talks.  She  can  draw  upon  (and  show)  an  actual  artefact  in  ways  that  areinherently  easier  than  if 
she  were  to  create  a text  description.  Thecapture  method  assists  the  communication  process.  The  listener/viewer  can 
see  the  artefact  under  discussion  as  well  as  to  hear/watch  the  explanation.  The  audience  can  see  Sonia,  hergestures  or 
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movements  and  hear  her  voice  (including  the  changing  tone  ofher  voice).  The  method  of  capture  therefore  has 
potential  to  assist  the  audience  in  making  sense  of  Sonia’sproblem.  The  clip  is  prepared  with  an  audience  in  mind. 

Analysing  the  clip  for  relationships  between  key  descriptors 

The  example  video  clip  is  a relatively  generic  problem  of  practice, sufficiently  germane  for  widespread  recognition 
by  people  accessing  ourwebsite.  It  is  distinguished  by  at  least  three  key  features,  discussedbelow. 


The  duration  of  the  clip  and  the  richness  of  its  form 

It  took  just  two  minutes  to  prepare  the  clip,  which  lasts  exactly  79  seconds.  Beyond  editing  the  start  and  finish  of 
theclip,  the  recording  was  continuous  and  nothing  has  been  changed.  Ittherefore  represents  a concise  yet  coherent 
account  of  the  problem,  whichhas  taken  little  effort  or  time  in  capture. 

The  duration  over  time  is  an  interesting  factor  to  probe  further.  Inthe  79  seconds,  the  subject  speaks  180  words  and 
makes  5 other  utterances(’ums’),  which  equates  to  2.27  words  per  second.  We  made  a rough  andready,  quick 
comparison  by  creating  anaccount  of  the  problem,  in  a textual  message.  This  was  similarly  done  ”onthe  fly”  by  one 
of  our  research  team  who  was  familiar  with  the  content  ofthe  clip  and  who  uses  a word  processor  daily.  In  the  test,  it 
took  150seconds,  to  create  the  text  and  in  that  time  120  words  were  typed.  No  time  was  spent  on  correcting  spelling 
crediting  the  message.  On  completion  it  was  observed  that  the  text  messagehad  taken  almost  twice  as  long  to 
achieve,  and  though  it  does  create  acomparable  description,  it  fails  tocomment  on  three  aspects  that  are  spoken  of  in 
the  video  clip.  Figure  2 and3  below  illustrate  respectively  the  transcript  of  the  video  clip  and  thetest  (text)  message. 
While  a strict  and  rigorous  comparative  analysis  isnot  suggested  between  the  two,  the  limitations  of  the  text  are 
apparent.  It  fails  to  capture  the  richnessof  the  clip:  we  do  not  see  the  subject  or  artefact,  nor  hear  thesubject’s  voice 
and  cannot  see  her  gestures  and  references  to  the  artefact.  Andit  does  takes  longer  to  create. 


OK  Ihave  a problem  in  producing  documents  in  A5  format  like  this.  I do  thisin  Word. 

I know  there  are  other  packages  that  would  be  better  um  such  asPagemaker,  would  be  better  tools  for  doing  this, 
such  as  Pagemaker  but  forvarious  reasons  I do  this  in  Word. 

I do  it  about  four  times  a year  and  the  problem  is  having  a goodformat  that  enables  me  to  number  the  pages  in  some 
automatic  numbering  way. 

The  way  1 do  it  currently  is  just  to  type  in  the  page  number  and  I workout  by  drawing  diagrams  um  which  page 
needs  to  be  opposite  which  page  so  um  page  2 is  opposite  page  1 1 and  it  carrieson  like  this  in  a way  that  is  not 
immediately  obvious.  And  each  of  thesepages  is  a page  um  within  Word,  divided  into  columns 
Ifs  a two  column  page,  um,  and  if  s a really  clumsy  way  to  do  it  andl  know  ifs  a clumsy  way  to  do  it,  but  I haven’t 

thought  of  a better  way  todo  it.  So  thaf  s my  problem 

Figure  2:  transcipt  of  the  audio  track  on  the  video  clip 

Duration  will  also  becrucial  to  the  audiences  for  multimedia  objects,  such  asvideo  clips.  Since  video  clips  are 
inherently  linear  and  sequential  (inboth  visual  and  aural  terms),  attention  to  their  content  is  likely  morelimited  than 
attention  to  a text  that  can  be  visually  scanned.  Yet,  the  cl  ip’s  persistence  as  a stored  object  and  the  control  and 
playback  provision  provided  couldprove  influential  for  improving  the  viewing  of  multimedia  objects  ieto  enable  new 
forms  of  visual/aural  scanning. 

In  time,  voice  technology  will  enable  rapid,  easier  creation  of  textmessages.  It  is  also  widely  suggested  that  the 
creation  of  text-based  accounts  promotesdiscursive  and  reflective  representations,  considered  beneficial  for’deeper’ 
processing  (eg  Laurillard,  1993;  Mason,  1994).  We  do  not  discountthe  value  of  reflective  text  messages  for  learning, 
but  the  anticipated  ease  of  use  will  not  overcome  the  lack  ofvivid,  rich  content  and  the  support  for  social  cues  that 
areintrinsic  features  of  multimedia,  and  requisite  for  ‘showing  anddoing’  in  professional  learning 
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My  problem  concerns  using  a word  processor,  Word  to  creatbooklets  such  as  A5  booklets. 

I use  word  to  do  this  but  I know  it  would  be  better  using  a desktoppublishing  package. 

The  problme  concersn  pagination  of  the  pages. 

In  Word  I have  to  splkiut  an  A4  pages  into  twocolumns  and  number  the  pages  maually  to  get  the  right  pages  next  to 
eachother  in  the  final  prionted  document 

because  you  need  to  have  an  oder  that  is  notimmediately  obvious  such  as  page  1 9 being  opposite  page  2 so  thats 
myprobmlem  and  I know  I hanbndle  it  in  a clumsy  way 
but  there's  never  seemed  an  easier  way  to  do  it. 

I hope  you  can  help. 

Omissions  from  the  text:  does  not  comment  on  number  of  times  peryear  doing  this  task;  does  not  mention 
Pagemaker;  does  not  mention  drawingdiagrams 

Figure  3:  test  tocreate  the  same  message  using  text 

Use  of  artefacts  as  'cognitive  aids'  and  to  'ground'  the  representation  inpractice 

We  believe  the  use  of  artefacts  in  video  representations  will  beextremely  valued  for  aiding  the  demonstration  and 
unfolding  of  a practiceand  for  ensuring  the  clip  is  faithful  to  real-world  practice  and  not  anabstracted  theoretical 
justification  of  it.  A practitioner  who  is  calledupon  to  describe  their  actions,  without  the  referential  anchor(s)  of 
theartefacts  used  in,  or  produced  from, the  practice  may  well  describe,  in  abstracted  ways,  the  theory,  rules  orideals 
that  are  the  starting  points  for  their  actions.  The  use  of  artefactshelps  the  practitioners'  descriptions  and  explanations 
to  be  grounded  inthe  reality  of  their  actions.  In  the  clip  Sonia  makes  verbal  and  (physical)  visual  reference  to  the 
bookletShe  uses  it  to  help  her  to  explain  the  problem.  The  booklet  is  central  to  her  representation;  toextend  her  use 
of  language.  It  mediates  between  thought  processes  and  herspoken  word.  The  booklet  helps  ground  the  explanation 
in  the  reality  of  thepractice.  The  artefact  is  also  a mediational  tool  for  the  audience:  to  helpthem  understand  about 
and  relate  to  the  represented  practice.  It  is  avisible  object  that  helps  in  creating  a picture  of  a socially  meaningful 
activity,  among  a community's  members. 

The  artefact  can  also  mediate  between  us  and  reality,  in  terms  oftime.  In  this  clip,  the  booklet  (alongside  Sonia's 
explanation)  enablestime  to  be  transcended.  The  clip  shows  the  finished  booklet  and  we  hear  about  the  pagination 
problem:  we  do  not  have  towitness  every  stage/process  in  the  booklet’s  development.  Thisoffers  tiniesaving 
opportunities  by  temporally  delimiting  the  representation. 

Social  cues  to  the  community  of  practice 

Future  working  in  distributed  professional  communities  will  probably  mean  many  of  its  members  will  nothave 
physically  met,  yet  we  believe  it  will  be  important  for  learning  insuch  communities  that  there  is  a sense  of  identity 
and  trust  between  themembers.  Visual  and  aural  aspects  of  the  clip  will  help  establish  the  community.Within  Sonia’s 
clip,  through  the  images  and  sound  track,  we  gain  a sense  about  her.  Wecan  see  what  she  looks  like,  hear  her  voice 
and  gain  a sense  of  her  ways  ofdoing.  She  suggests  trust  by  confiding  about  her  problem-  eg  in  her  reference  to  the 
"clumsy"way  she  currently  performs  this  task.  Sonia  gives  cues  to  her  audience,  toencourage  responses  from  other 
members  of  the  community.  She  is  invitingothers  to  express  their  views  or  to  offer  advice 

The  duration  of  the  clip  and  its  richness  in  form,  its  use  ofartefact  and  the  social  cues  within  the  clip  we  suggest 
distinguish  thisclip.  Yet  there  is  nothing  inherently  unusual  in  it  that  cannot  bereproduced  in  other  clips  of  practice. 
This  suggests  to  us  that  video  representations  do  offer  innovative,vivid  yet  effective  means  for  sharing  in 
professional  communities,  whichwarrant  further  investigation. 

Future  work  on  annotating  representations,  especiallywith  audio 

'Near  pure'  video  representations  seem  intuitively  to  suggest  themselves  for  capturingpractice  in  near-authentic 
ways,  yet  if  we  want  to  stay  closer  eg  to  fly  onthe  wall  kinds  of  clips,  we  see  a rising  importance  to  add 
annotation(s)  tothe  base  representation,  thuscreating  complex  multimedia  objects.  In  our  project  work  we  will 
createvideo,  voice  and  textual  annotations  to  video  clips.  We  are  particularlyinterested  in  the  use  of  stored  voice 
annotations,  to  add  to  and  toelaborate  upon  the  video  clip.  We  believe  the  use  of  persisting  voice  messages  will  offer 
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new  and  effective  ways  toencourage  professionals  to  articulate  ideas  about  their  working  practices(Steeples,  1995). 
The  video  clip  provides  a direct  and  demonstrable  exampleof  practice  in  action,  while  making  a voice  annotation 
allows  the  subject  to  offer  further  (such  asbackground  or  explanatory)  information.  Voice  annotations  will  be  used 
toencourage  practitioners  to  explain  their  practices  and  also  to  reflect  uponthem. 

Summary 

Thedevelopment  of  technologies  that  enable  recording  and  digitising  of  multimedia  artefacts( video  clips  that  can  be 
elaborated  with  video,  aural  or  textualannotations)  offers  a radical  departure  for  the  design  and  use  of  onlineleaming 
environments.  Multimedia  technologiesoffer  innovative  ways  to  allow  practitioners  within  distributedprofessional 
communities  to  create  and  share  representations  of  problemsderived  from  real-world  practices. 

The  next  questions  to  consider  in  our  work  will  include  about  thekinds  of  things  that  come  to  light  when  people  are 
put  in  a position  where  they  have  tomake  translations  between  tacit  and  discursive  forms  of  knowledge.  We  arealso 
concerned  to  understand  the  kinds  of  translations  people  make  whenthey  have  to  talk  about  materials  theyhave 
encountered  in  multimedia/video  rather  than  in  a textual  form.  The  work  outlined  in  this  paper  will  also  be  subjected 
to  further  scrutiny  inforthcoming  user  trials,  and  will  lead  to  a setof  guidelines  on  bestpractice  for  using  multimedia 
communications  technologies  in  continuing  professional  development. 
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Abstract:  This  paper  draws  on  the  ideas  underpinning  a two  year  Australian  Research 
Council  Large  Grant  project  which  explores  the  extent,  nature  and  problematics  of  creating 
and  sustaining  educational  dialogue  through  interactive  television  (ITV)  in  schools.  ITV  has 
been  introduced  in  many  schools  as  a solution  to  the  demands  for  increasingly  complex 
curricula,  high  retention  rates  and  equitable  provision,  all  within  tight  budgets.  The  State  of 
Victoria  in  Australia,  is  at  the  forefront  of  these  initiatives.  There  is  little  research  on  ITV  in 
schooling,  especially  as  a means  of  developing  and  sustaining  forms  of  interactive  learning. 
This  paper  discusses  some  of  the  arguments  concerning  interaction  and  dialogue  that  are 
important  to  understanding  the  use  of  ITV  in  schooling. 


Introduction 

ITV  has  been  introduced  in  schools  of  many  countries  as  a solution  to  the  demands  for  increasingly 
complex  curricula,  high  retention  rates  and  equitable  provision,  all  within  tight  budgets.  Australia,  especially 
Victoria,  is  at  the  forefront  of  these  initiatives  however  there  has  been  little  research  on  ITV,  especially  as  a 
means  of  developing  and  sustaining  forms  of  interaction  which  foster  learning.  Interaction  is  constituted  as  the 
capacity  for  the  students  to  engage  in  contiguous  or  non-contiguous  means  of  communication  with  other  people 
in  order  to  facilitate  their  learning. 

The  use  of  satellite,  cable  and,  increasingly,  digitised  forms  of  ITV  broadcasting  and  ‘narrowcasting’ 
to  schools^  has  become  an  important  feature  of  schooling  in  countries,  such  as  Australia.  In  such  countries, 
geography,  demography  and/or  climate  make  it  difficult  to  provide  efficient  ‘traditional’  schooling  across  the 
range  of  curricula  required  of  schools  from  primary  through  to  secondary  levels.  Canada  and  Scandinavia  are 
two  international  examples  where  ITV  is  becoming  increasingly  significant  in  schooling  (Haughey  & Roberts, 
1996;  Meisalo,  1996).  In  Australia,  Victoria  has  developed  what  is  arguably  the  most  widespread  system-wide 
ITV  provision  of  any  similar  school  system  in  the  world.  The  Victorian  provision  has  spread  to  other  states  in 
Australia,  and  there  are  also  other  examples  of  smaller  ITV  networks  (for  example,  in  Western  Australia  and 
Queensland)  in  operation. 

Whilst,  until  early  this  decade,  broadcast  television  played  a small,  but  important,  part  in  schooling 
through  the  Australian  Broadcasting  Commission  provision,  the  current  emphasis  is  on  school  authorities  using 
new  means  of  television  transmission  to  fulfil  their  mandate  to  provide  schooling  for  all  people  of  school  age. 
However,  unlike  TV  broadcasts,  the  emphasis  in  these  new  school-based  providers  has  been  on  building 
interactivity  into  their  provision;  the  sorts  of  interactivity  which  teachers  and  students  expect  of  schooling,  but 
not  of  television.  Schooling,  especially  primary  schooling,  is  one  of  the  most  interactive  educational  contexts 
and  students  are  expected  to  engage  in  dialogue  in  order  to  learn;  teachers  find  it  difficult  to  imagine  teaching 
otherwise.  Given  that  much  of  the  curriculum  development  in  ITV  has  been  in  languages  other  than  English,  it 
is  no  wonder  that  the  construction  of  ITV  into  a means  of  facilitating  dialogue  within  the  classroom,  and  within 
the  broader  ITV  community  (with  other  schools,  the  program  makers  and  presenters,  etc)  has  been  seen  as  a 
priority.  However,  there  has  been  a struggle  to  understand  the  practical  construction  of  such  forms  of  dialogue 
within  the  ITV  community  (Barty,  1996).  This  project  provides  research  upon  which  policy,  practice  and 
theory  can  be  built  which  addresses  ITV  in  schooling 

Educational  Dialogue 

The  potential  of  ITV  to  be  a more  ‘industrialised’  and  also  highly  centralised  form  of  ‘distance’ 
schooling  sits  uneasily  both  with  student-centred  approaches  to  learning  and  also  with  teachers’  (and  teacher 
unions’  and  subject  associations’)  concerns  for  professional  autonomy.  However,  educational  technologies,  such [*] 


[*]  We  are  focussing  on  schools  and  the  use  of  ITV  with  children  in  this  research.  ITV  has  other,  somewhat 
different,  uses  in  other  education  and  training  sectors. 
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as  ITV,  can  be  used  in  ways  which  encourage  dialogue  and  participatory  forms  of  learning,  if  the  programs  are 
designed  to  foster  such  and  if  the  local  teachers  and  learners  work  accordingly. 

The  ‘dialogic  approach’  is  not  yet  a dominant  paradigm  in  research,  theory  and  practice  in  distance 
education,  educational  technology  and  open  learning  but  it  is  becoming  very  influential  (Buckley,  1992:  81-84; 
Lockwood,  1992:  41-49;  Morgan,  1990;  Paul,  1990:  88-90).  As  part  of  this  approach,  educational  technology 
has  joined  dialogue  as  a central  concept  within  a framework  which  views  teaching  practices  as  mediated 
processes  relevant  to  their  social  and  historical  circumstances.  ‘Educational  technologies  are  not  simply  the 
tools  of  educators — although  this  is  a popular  misconception,  rather  they  are  the  knowledge,  values  and 
practices  which  constitute  the  development  and  use  of  those  tools’  (Evans  & Nation,  1993b:  198-199).  The 
interactivity  of  ITV  suggests  some  useful  lines  of  inquiry  and  analysis  in  terms  of  the  local  adoption  and 
adaptation  of  the  programs  to  local  conditions,  and  even  the  reflection  of  local  issues  and  concerns,  through 
interactivity,  into  the  global  broadcast. 

The  School  Contexts  of  ITV 

The  Victorian  ITV  network  includes  interstate  provision,  which  is  leading  to  a broader  national 
significance,  not  only  for  ITV  itself,  but  also  for  the  proposed  research  project.  The  international  significance  of 
the  ITV  initiative  is  that  the  Victorian  network  is  held  to  be  the  most  comprehensive  state  education,  system- 
wide,  ITV  network  in  the  world.  There  are  many  examples  of  interactive  television  in  educational  contexts 
(Bruning,  et  al  1993;  Zhang  & Fulford,  1994),  however,  the  use  of  ITV  in  schools  is  relatively  unusual  (there 
are  instances,  in  Canada  and  Scandinavia,  for  example  Catchpole,  1993).  Typically,  school  and  other 
educational  sector  uses  of  interactive  television  are  based  on  dual  or  multi-site  ‘video-conferencing’.  Broadcast 
uses  are  generally  not  interactive,  (such  as  the  Open  Learning  Australia  broadcasts  in  Australia,  or  those  of  the 
Open  University  in  the  UK,  or  the  Open  Learning  Institute  in  Hong  Kong).  It  is  in  this  context  that  a study  of 
the  use  of  ITV  in  Victoria  is  internationally  significant,  especially  in  terms  of  the  potential  benefits  of  the 
findings  for  future  government  policy  and  for  educators  and  scholars,  both  in  Australia  and  overseas. 

The  potential  impact  of  ITV  is  significant  for  schools.  Although  current  evidence  suggests  that  usage 
in  schools  is  generally  patchy,  the  pressures  on  schools  to  provide  a wide  and  varied  curriculum,  and  the 
predicted  decline  in  the  numbers  of  teachers — ^particularly  in  specialist  areas  such  as  languages  other  than 
English — ^points  to  ITV,  and  other  communications  and  computer  technologies,  adopting  a higher  profile.  In 
Victoria,  these  trends  are  already  in  evidence,  and  ITV  usage  is  increasing  steadily.  Interstate  usage  is  less 
clear,  but  it  is  understood  that  ITV  is  developing  similarly,  but  is  less  widespread  (Mark,  1995).  Evans  and 
Nation  (1993b)  found  that  the  ways  educational  organisations  initiate  and  foster  change  is  a complex  social 
process  which  is  replete  with  conflicts  and  contests.  Such  conflicts  and  contests  spring  from  the  different  values 
and  interests  of  the  parties  concerned.  Other  work  by  the  researchers  (eg.  Evans  & Newell,  1993;  Evans  & 
Tregenza,  1993;  Stacey,  1995)  points  to  the  importance  of  systematic  support  and  appropriate  resourcing  in 
order  to  facilitate  change.  In  this  case,  it  is  hypothesised  that  Victorian  schools  are  likely  to  reflect  more 
widespread  and  effective  use  of  ITV,  than  interstate  schools,  due  to  the  weaker  support  and  resourcing 
provided. 

Exploring  these  processes  of  change  surrounding  ITV  within  the  context  of  different  schools,  as  well 
as  including  a consideration  of  the  central  players  in  the  government  departments,  is  achieved  using  qualitative 
research  approaches  in  a case-study  form.  Such  approaches  are  particularly  important  where  the  focus  is  on  the 
interactive  elements  of  the  teaching  and  learning  process  as  it  is  mediated  and  shaped  by  ITV.  We  are  more  and 
more  in  a position  where  ITV  and  other  communications  and  computer  technologies  (which  share  many 
common  features  with  respect  to  the  issue  of  shaping  and  mediating  interaction  and  dialogue)  could  become 
substantial  media  for  reshaping  the  ways  schooling  is  managed  and  practised.  However,  this  will  only  occur  to 
the  extent  that  ITV  actively  engages  the  lives  of  teachers  and  children,  and  through  such  change  the  practices, 
management  and  culture  of  schools.  To  achieve  this  end  in  ways  which  are  worthwhile  for  teachers,  students 
and  schools,  it  is  necessary  to  understand  the  practices  and  processes  of  such  facilitated  and  mediated 
interaction  which  ITV  provides. 

ITV  and  Children  as  Learners 

A significant  aspect  of  this  research  focuses  on  the  children  as  learners  through  ITV.  Current 
literature,  inspired  by  Vygotskian  theory,  which  researches  the  significance  to  learning  of  social  interaction  and 
discussion,  has  provided  a framework  for  observing  the  role  and  impact  of  technology  as  a mediator  in  this 
interactive  dialogue  (Stacey,  1997).  Through  observations  and  interviews,  the  research  provides  findings 
concerning  the  ways  in  which  children  interact  with  ITV  and  interact  within  the  classroom  with  their  teachers 
and  peers  as  a result  of  the  strategies  deployed  through  ITV.  This  will  be  particularly  focussed  on  analysing  the 
learning  outcomes  achieved  though  ITV  as  identified  and  described  by  both  the  teachers  and  students.  An 
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important  element  of  the  research  will  be  to  assess  these  outcomes  against  the  students’  relative  ability-levels  in 
their  ITV  program  subject  area  as  compared  with  their  teachers’  expectations  and  other  similar  students’ 
performances  who  do  not  use  ITV.  This  will  make  a very  significant  contribution  to  our  understanding  of  the 
effectiveness  of  ITV,  in  terms  of  both  particular  subject  areas,  and  also  for  particular  age  and  ability  levels. 

Given  the  range  of  evidence  and  experience  on  the  gendered  nature  of  schooling  in  general,  it  is 
important  that  gender  is  considered  in  relation  to  the  use  of  ITV,  especially  in  terms  of  boys’  and  girls’ 
interactions  and  experiences.  In  these  respects  the  research  extends  Evans's  previous  work  in  terms  of  the 
gendered  nature  of  schooling  (1982,  1987a,  1987b,  1988)  and  relates  to  the  work  which  identifies  the  gendered 
nature  of  technology  in  society  more  generally  (for  example,  Wajcman,  1991).  Therefore,  the  research  design 
ensures  appropriate  representation  of  gender  issues,  in  addition  to  particular  observations  of  the  gendered  life  of 
the  ITV  case-studies. 

The  Case-studies 

The  research  uses  qualitative  research  methods  founded  within  educational  ethnography  including 
case-studies,  interviews,  observation  and  document  collection  to  develop  case  studies  of  the  implementation  and 
use  of  ITV  in  the  selected  schools.  Qualitative  methods  best  allow  us  to  explore  the  processes  of  change 
surrounding  ITV  within  the  context  of  different  schools.  Interviews  are  conducted  with  relevant  teaching  staff, 
students  and  Department  of  Education  personnel. 

This  project  explores  the  extent,  nature  and  problematics  of  educational  dialogue  through  interactive 
television  (ITV)  in  primary  and  secondary  schools.  In  particular  it  is  designed  to: 

• investigate,  record  and  analyse,  the  use  of  ITV  in  primary  and  secondary  schools,  especially  in  terms  of  the 
development  of  interactive  and  dialogic  approaches  to  teaching  and  learning. 

• study  the  processes  through  which  teachers  and  students  adopt,  and  adapt  to,  the  interactive  and  dialogic 
elements  of  ITV. 

• analyse  the  findings  in  terms  of  the  changing  nature  and  demands  on  schools,  and  to  relate  these  to  issues  of 
the  social  and  cultural  conditions  of  schooling,  including  gender,  rurality  and  ethnicity. 

• analyse  the  findings  in  terms  of  educational  theories  and  practices  of  dialogue,  and  especially  to  those 
concerning  mediated  forms  of  educational  dialogue,  such  as  in  open  and  distance  education  theory  and 
practice. 

Research  Plan 

The  first  phase  of  the  research  in  1 998  involved  the  selection  of  five  primary  schools  each  being  from 
either  a country,  regional  or  metropolitan  region.  Two  of  the  selected  schools  are  non-government  schools.  In 
each  primary  school  ITV  case-studies  have  been  conducted  simultaneously  each  being  split  between  the  main 
curriculum  areas  of  ITV  (Science  and  Technology,  and  Languages  other  than  English).  Throughout  the  year 
teachers  have  been  invited  to  discuss  the  children’s  learning  with  ITV,  as  well  as  their  own  experiences  of 
teaching  with  and  alongside  ITV.  Typically,  the  ITV  broadcasts  are  only  part  of  the  timetable  for  the  particular 
curriculum  area  and  the  teachers  work  with  the  children  ‘off-air’  in  other  timetabled  sessions.  These  lessons  are 
also  observed  in  addition  to  the  actual  ITV  broadcast  sessions  which  may  be  either  watched  live  by  the  class,  or 
taped  and  played  back  at  another  time. 

The  second  phase  of  the  research  to  be  conducted  in  1999  will  involve  the  nomination  of  three 
secondary  schools  where  six  case-studies  will  be  developed.  The  curriculum  areas  selected  for  study  at  these 
schools  will  be  chosen  from  those  used  via  ITV.  In  addition  to  the  specific  focus  on  the  case-studies,  a sample 
has  been  made  of  other  uses  of  ITV  in  the  schools  and  by  the  Department  of  Education. 

A phase  of  the  research  includes  the  development  of  six  mini  case-studies  across  New  South  Wales 
and  South  Australia  for  the  purposes  of  comparison  and  contextualisation.  This  involved  one  day  visits  to  one 
primary  school  in  each  state  during  1998;  and  two  secondary  schools  in  each  state  during  1999.  These  mini 
case-studies  involve  visits  by  a project  team  member  for  the  purpose  of  observing  ITV  sessions  in  progress  and 
subsequent  follow-up  lessons,  and  to  interview  staff  and  students  about  their  experiences  with,  and  use  of,  ITV. 

ITV  Programs 

The  Victorian  Department  of  Education  regularly  transmits  a number  of  programs  to  schools  through 
the  satellite  learning  network,  the  main  curriculum  areas  being  Science  and  Technology  Education  in  Primary 
Schools  (STEPS)  and  Primary  Access  to  Languages  via  Satellite  (PALS).  Other  ‘special’  programs  are  also 
broadcast  throughout  the  year.  In  1997,  600  hours  of  programs  were  broadcast,  2,500  schools  participated  and 
an  estimated  100,000  students  learned  another  language  via  satellite. 
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STEPS  consists  of  a series  of  programs  that  assist  teachers  in  the  delivery  of  science  and  technology 
education  within  the  framework  of  the  Victorian  Curriculum  Standards  Framework.  Students  are  able  to 
participate  in  each  program  as  it  is  broadcast  live  to  air.  This  provides  opportunities  for  the  students  to  interact 
with  the  teacher  (presenter)  in  the  studio.  Each  school  participating  in  STEPS  is  provided  with  comprehensive 
classroom  support  materials  comprising  a list  of  resources  required  for  each  program,  the  expected  student 
outcomes,  a brief  summary  of  the  program  format  and  detailed  descriptions  of  the  activities  to  be  undertaken. 
The  programs,  either  thirty  minutes  or  one  hour  duration  depending  on  the  year  level,  are  broadcast  fortnightly 
catering  for  the  Prep;  1/2;  3/4  and  5/6  year  levels.  Schools  are  required  to  complete  work  and  activities  to 
prepare  for  the  broadcast  and  also  post  broadcast.  STEPS  also  integrates  the  use  of  multiple  media  through 
their  World  Wide  Web  site  which  provides  general  information,  a teacher  discussion  page  and  a student 
discussion  page.  This  Web  site  complements  and  enhances  the  programs  rather  than  just  duplicating  the 
printed  materials  and  the  transmissions. 

To  facilitate  the  teaching  of  another  language,  PALS  programs  are  broadcast  to  primary  schools 
twice  a week  covering  languages  including  Italian,  Indonesian,  Mandarin,  Japanese,  French  and  German.  A 
similar  concept  has  been  developed  for  the  teaching  of  another  language  at  the  secondary  school  level  called 
SALS  (Secondary  Access  to  Languages  via  Satellite).  As  with  the  STEPS  programs,  teachers  are  provided  with 
comprehensive  classroom  materials  that  support  and  complement  the  language  programs.  These  programs  run 
for  thirty  minutes  and  encourage  students  to  interact  live  with  the  studio  presenters.  Students  are  required  to 
complete  pre  and  post  broadcast  activities  to  facilitate  their  learning. 

One  example  of  a ‘special’  ITV  broadcast  is  Communicating  over  the  catchment.  This  is  a student 
conference  concerned  with  conservation  of  the  environment.  Each  year  it  is  developed  and  broadcast  live  from 
Melbourne  Zoo  with  support  from  the  Murray  Darling  Basin  Commission.  The  annual  satellite  broadcast 
consists  of  two,  one  hour  long  programs  about  conservation  issues,  and  provides  students  with  the  opportunity 
to  interact  live  with  zoo  experts  via  telephone,  fax,  email  or  a special  Web  Chat  facility.  The  broadcast  is  only 
one  aspect  of  the  interactive  program.  There  are  also  preliminary  activities  through  interactive  communication 
through  a Web  site,  and  also  through  use  of  a student  resource  kit  and  professional  development  support 
materials  for  teachers. 

Preliminary  Findings 

The  observational  data  from  the  first  year  of  the  study  has  begun  to  provide  a range  of  emerging 
themes.  Teachers'  strategies  and  planning  are  integral  to  the  best  use  of  the  ITV  programs  and  several 
important  factors  are  beginning  to  appear  in  the  results  of  the  study. 

Interactivity 

As  predicted  in  establishing  the  study,  the  interactive  element  of  the  ITV  programs  is  an  important 
component  of  the  use  of  this  technology  and  it  is  established  in  a number  of  ways.  These  ways  of  interacting 
with  the  program  are  varied  and  the  programs  are  designed  to  include  the  children  in  activities  before  and 
during  the  broadcast  so  that  they  are  not  always  passively  receiving  the  broadcast  but  are  regularly  given  the 
opportunity  during  the  program,  to  participate  in  teacher  managed  interactive  discussion  within  the  classroom. 

These  interactive  points  can  also  be  part  of  the  broadcast  if  the  classroom  teacher  sets  up  the  process  and 
representative  groups  of  children  in  the  classrooms  observed  have  had  opportunities  to  interact  with  the 
program  presenters  through  telephone  links  during  the  live  transmission  of  the  program  or  more  rarely,  as 
participants  in  the  studio  production  and  broadcast.  These  are  heard  and  seen  over  the  broadcast  and  are  a very 
motivating  aspect  of  the  children's  learning  by  this  medium.  All  the  participating  students  have  been  excited  to 
hear  their  school  acknowledged  and  this  motivational  factor,  as  much  as  the  opportunity  to  answer  a question, 
or  report  on  an  activity  has  kept  the  classes  focused  on  and  interested  in  the  programs.  The  students  also 
respond  to  these  interactive  points  with  interest  when  children  from  other  schools  appear,  so  the  peer 
interaction  appears  to  be  an  important  factor  in  the  success  of  the  programs. 

Schools  also  send  work  into  the  program  producers  which  is  also  acknowledged  and  read  out  during  the 
broadcast  and  this  again  provides  students  with  an  motivational  involvement  in  the  program. 

Some  schools  tape  the  programs  and  then  can  stop  the  program  and  interact  through  teacher  managed 
activities  and  discussion  allowing  more  time  and  repetition.  This  has  been  observed  to  be  a useful  repetitive 
interaction  particularly  with  the  language  programs  (PALS)  where  sometimes  the  speed  of  the  new  vocabulary 
use  and  the  difficulty  in  comprehending  the  language  used  in  the  broadcast  requires  help  and  translation  to  be 
effective. 

Management 
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The  different  ways  schools  use  the  programs,  alluded  to  above,  seems  to  make  a difference  to  the 
effectiveness  of  the  programs  for  learning.  If  preparatory  classes  are  held  and  the  content  of  the  program  is 
integrated  across  the  curriculum  of  the  class,  there  is  more  interest  and  involvement  from  the  children  during 
the  broadcast.  In  one  classroom  observation,  the  familiarity  of  the  children  with  the  Indonesian  song  and  their 
physical  placement  with  their  notes  with  the  song's  words  easily  accessible,  meant  they  could  all  participate  in 
singing  together  during  the  program.  However  at  another  site,  the  children  did  not  attempt  to  participate  as 
they  were  less  practised  with  singing  the  songs  and  had  no  easy  access  to  the  words,  making  the  broadcast  less 
interactive  for  them. 

The  flexibility  of  the  whole  school's  approach  to  the  technology  use  impacts  on  this  aspect  of  the 
program's  effectiveness  as  well.  If  an  ITV  program  cannot  be  scheduled  easily  into  a busy  school  program  or 
there  is  inflexibility  in  the  manner  and  timing  of  the  school's  broadcast  of  the  program,  classroom  teachers 
cannot  always  vary  the  live  and  taped  broadcasts  to  suit  the  children's  learning,  and  cannot  customise  the 
progams  to  their  students'  learning  as  effectively. 

The  teachers'  use  of  the  program  notes  in  their  curriculum  planning  depends  on  them  receiving  this 
information  in  sufficient  time.  This  management  issue  was  one  that  has  been  frequently  mentioned  by  teachers 
as  an  important  factor  if  the  broadcast  team's  planning  cycle  does  not  concur  with  the  planning  timeline  of  the 
classroom  teachers. 

Media  Integration 

Greater  integration  of  the  ITV  media  with  electronic  communication  is  a emerging  factor  of 
importance  as  the  Victorian  schools  become  networked  and  provided  with  an  easier  access  to  electronic 
communication.  The  schools  who  watch  the  STEPS  programs  and  access  the  Internet  most  confidently  are  able 
to  use  a wider  range  of  multimedia  activities  provided  through  the  STEPS  World  Wide  Web  site 
(http://www.sofweb.vic.edu.au/steps/)  and  through  emailed  activities  devised  by  the  STEPS  production  team. 

As  Victorian  schools  are  all  included  on  a new  VIC  One  network  by  the  end  of  1998,  this  aspect  will 
begin  to  be  available  to  all  participating  schools  and  will  provide  a new  communicative  aspect  to  ITV  use. 
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Abstract:  Formative  evaluation  and  iterative  prototyping  seem  an  interesting  approach  for 
building  collaborative  learning  systems.  Our  approach  to  create  our  software  system  consisted 
on  iterating  successive  stages  of  analysis,  specification,  implementation  and  evaluation.  At  the 
end,  the  process  turned  out  to  be  a five-pass  cycle,  carried  out  in  two  years,  with  a major 
reimplementation  between  the  third  and  the  fourth  prototype  version.  We  describe  the  main 
features  for  each  version,  as  well  as  the  evaluation  techniques  followed.  The  first  cycle  is 
presented  in  more  detail  and  relevant  aspects  are  summarized  for  the  rest.  Each  evaluation 
includes  the  design  of  a learning  activity,  the  configuration  of  the  prototype  to  carry  out  the 
experience,  the  set-up  and  observation  of  the  activity  performed  with  real  students  and  tutors, 
and  finally  the  drawing  of  conclusions  including  the  improvements  to  consider  for  a next 
design  phase. 

Introduction 

For  a number  of  years  we  have  been  involved  on  research  and  experimentation  with  collaborative  learning  at  a 
distance  in  an  effort  to  identify  and  deploy  appropriate  uses  of  the  new  media  to  support  a distance  learning 
community.  In  this  paper  we  focus  on  the  process  of  designing  a software  environment  to  support  collaborative 
learning  activities  in  an  integrated  way.  The  design  of  collaborative  software  has  proved  to  be  hard.  Theories  and 
metaphors  in  the  CSCW  field  have  emerged  but  are  still  a matter  of  big  debate  and  further  elaboration  is  needed 
to  produce  models  supporting  the  practical  design  of  CSCW  applications.  The  difficulty  comes  from  the 
complexity  of  the  situations  to  be  considered,  where  the  technical  perspective  is  one  part  of  the  picture.  In  a 
specific  work  situation,  technology  should  support  human  activity  taking  into  account  the  social  setting:  an 
organization  with  rules,  structure  and  working  practices. 

Formative  evaluation  and  iterative  prototyping  seem  to  be  an  interesting  approach  for  building 
collaborative  learning  systems.  Conducting  experiments  to  find  out  how  the  prototype  is  used  and  what  is 
learned  from  its  use  provide  insights,  not  only  into  how  to  improve  the  system  but  also  into  how  to  motivate 
further  extensions.  A conceptual  framework  is  still  needed  to  analyze  and  capture  how  the  technical  solutions  in 
each  version  of  your  prototype  will  support  the  group  of  learners  to  attain  their  goals.  The  socio-cultural 
framework  provides  the  concept  of  Activity  (Nardi,  1996)  as  a unit  of  analysis,  with  a rich  internal  structure  to 
make  the  context  of  a situation  explicit,  specially  the  interlinks  between  the  individual  and  social  levels  stressing 
the  role  of  the  tools  as  mediating  artifacts. 

One  of  our  systems  serves  here  as  a case  study  to  illustrate  the  potential  of  combining  user-centered 
approach  (Gould  & Lewis,  1985)  with  Activity  Theory  to  design  collaborative  learning  software.  The  next 
section  describes  our  context,  and  elaborates  on  the  kind  of  learning  tasks  and  collaborative  framework  we  have 
considered.  In  section  three  the  main  features  of  the  iterative  design  are  presented  and  learners’  experiences 
discussed.  For  a detailed  description  of  the  system  itself  the  reader  can  consult  (Barros  & Verdejo,  1998). 

Context,  goal,  and  premises  for  defining  the  learning  approach 

Collaborative  learning  research  has  paid  close  attention  to  studying  pupils  interactions  during  peer-based  work  in 
order  to  analyze  and  identify  the  cognitive  advantages  of  joint  activity.  As  Crook  (1994)  points  out  the  benefit  of 
the  collaborative  approach  for  learning  lies  on  the  processes  of  articulation,  conflict  and  co-construction  of  ideas 
occurring  when  working  closely  with  a peer.  Participants  in  a problem-solving  situation  have  to  make  their  ideas 
explicit  (assertions,  hypothesis,  denials..)  to  other  collaborators.  Disagreements  prompt  justifications  and 
negotiations,  helping  students  to  converge  to  a common  object  of  shared  understanding. 


The  computer  provides  opportunities  to  support  and  enhance  this  approach  in  a number  of  ways  (e.g. 
offering  computer-based  problem  spaces  for  jointly  create  and  exploit  structures  of  common  knowledge  and 
shared  reference).  Furthermore  networks  made  possible  the  opening  of  collaborative  frameworks  to  distributed 
communities,  providing  remote  access  to  these  spaces  as  well  as  computer-mediated  communication  to  support 
interpersonal  exchange  and  debate.  An  increasing  number  of  collaborative  learning  environments  for  open  and 
closed  virtual  groups  have  been  built  for  a range  of  learning  tasks  (Scardamalia  & Bereiter,  1991)  (Edelson  & 
O'Neill,  1994)  (Wang  & Johnson,  1994)  (Suthers  & Jones,  1997),  and  experiences  of  use  are  reported  from 
school  to  university  level  (Bell,  Davis  & Linn,  1995)  (Collis,  1997). 

The  potential  of  collaborative  learning,  within  the  framework  of  a distance  learning  institution,  has  to 
be  explored  taking  into  account  the  traditional  practice  of  individual  and  stand-alone  study  in  distance  education. 
Not  only  faculty  but  also  students  are  reluctant  to  carry  out  collaborative  practice.  Network  facilities  are  widely 
appreciated  as  a way  to  improve  the  efficiency  of  how  course  material  is  delivered  rather  than  an  opportunity  to 
change  patterns  of  teaching  and  learning.  So  the  challenge  for  developing  collaborative  activities  in  a traditional 
distance  learning  university  is  not  only  to  build  appropriate  tools  but  also  to  transform  established  practices  in 
the  community.  For  these  reasons,  it  is  quite  critical  to  look  for  opportunities  of  implementing  collaborative 
learning  experiences  where  collaboration  could  be  perceived  by  students  as  a clear  added-value  to  existing 
learning  practices. 

So  we  want  to  propose  that  our  distance  learners,  geographically  distributed,  work  together  in  small 
groups  to  prepare  an  overview  of  a topic  related  to  their  professional  profile.  There  are  a number  of  ways  in 
which  this  collaborative  activity  can  be  organized,  one  of  the  procedures  that  we  have  implemented  consists  of  a 
phase  of  joint  involvement  in  the  preparation  of  the  work  within  each  group,  followed  by  a period  of  individual 
reading.  Then  participants  engage  in  a dialogue  to  elaborate  collaboratively  a synthesis  according  to  a schema. 
Variants  can  include  a phase  of  planning  and  the  negotiation  of  the  schema  before  to  the  collaborative  writing.  A 
further  step  could  be  mutual  presentation  of  the  final  products  between  groups. 

From  the  learning  point  of  view  the  task  fulfills  the  requirements  of  been  relevant  (it  corresponds  to  a 
real  practice  in  a research  context)  and  co-  constructive  in  the  sense  that  they  have  to  build  their  own  shared 
understanding  of  the  topic.  Furthermore  the  discursive  nature  of  the  procedure  and  the  deferred  mode  for  the 
debate  afford  opportunities  for  the  students  to  reflect  on  the  way  they  perform  the  collaboration  process. 

The  iterative  design  process 

Our  approach  to  create  the  software  system  consisted  of  iterating  successive  stages  of  analysis,  specification, 
implementation  and  evaluation.  At  the  end,  the  process  turned  out  to  be  a five-pass  cycle,  carried  out  over  two 
years,  with  a major  reimplementation  between  the  third  and  the  fourth  prototype  version.  Next  we  will  describe 
the  main  features  for  each  version,  as  well  as  the  evaluation  techniques  followed.  The  first  cycle  is  presented  in 
more  detail  and  relevant  aspects  are  summarized  for  the  rest.  Each  evaluation  includes  the  design  of  a learning 
activity,  the  configuration  of  the  prototype  to  carry  out  the  experience,  the  set-up  and  observation  of  the  activity 
performed  with  real  students  and  tutors,  and  finally  the  drawing  of  conclusions  including  the  improvements  to 
consider  for  a next  design  phase. 

The  first  cycle.  The  prototype 

The  first  prototype,  installed  on  a web  server,  offered  the  same  services  to  all  clients.  The  access  was  remote  and 
the  mode  asynchronous.  Each  user  had  a login  and  a password  and  belonged  to  a group.  The  system  supported  a 
number  of  small  groups  operating  at  the  same  period  of  time. 

The  main  metaphor  in  the  prototype  for  sustaining  a learning  activity  is  the  concept  of  space^  a virtual 
structured  place  with  resources  and  tools  to  perform  a task.  Three  types  of  spaces  were  available:  an  individual 
workspace,  private  for  each  user,  and  two  shared  spaces  for  each  group  (one  workspace  for  debate  and  joint 
construction,  the  other  for  coordination  purposes).  The  information  handled  is  mainly  textual,  so  a variety  of 
editing  tools  and  file  management  facilities  was  available,  as  well  as  links  to  other  relevant  electronic  sources  of 
further  information  for  the  task  at  hand.  The  shared  workspace  provided  support  for  conversation  in  the  form  of 
semi-  structured  typed  messages.  When  learners  express  their  contributions  they  have  to  select  a type  from  a 
predefined  set.  These  types  include:  proposal,  contraproposal,  comment,  question,  and  clarification.  The  system 
dynamically  builds  a representation  of  the  discussion  process  going  on  in  each  workspace,  in  the  form  of  an 
index  of  related  contributions  for  each  subtask.  This  index  was  fully  deployed  and  represented  on  the  left  part  of 
the  workspace  screen. 

The  evaluation  phase 

The  first  evaluation  addressed  the  central  point  of  the  adequacy  of  the  shared  workspace  to  support  a group  of 
learners  to  perform  a discursive  task.  The  design  of  the  learning  activities  was  carried  out  by  our  team,  as  well  as 
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researchers  we  are  also  teachers.  Two  kind  of  learning  situations  were  considered.  We  describe  one  of  them, 
characterized  in  terms  of  Activity  Theory  (Kuuti,  1996)  (Cole  & Engestorm,  1993)  as  follows: 

The  object  of  the  learning  activity  was  to  synthesize  the  state  of  the  art  in  a focused  research  topic.  Two 
topics  were  selected:  one  in  Artificial  Intelligence  (AI),  the  other  on  Educational  Technology.  The  outcome  of 
the  activity  was  an  essay,  pointing  out  the  key  ideas.  The  schema  for  the  essay  was  previously  defined  by  the 
tutor,  and  included  in  the  shared  workspace.  The  community  involved  in  the  activity  were  pairs  of  graduate 
students  in  a Ph.D.  distance  teaching  programme,  in  UNED,  our  university.  Students  were  part  time, 
geographically  distributed,  usually  interacting  in  disjoint  time  slots.  The  subject,  each  individual  student,  had 
either  a technical  or  non  technical  profile,  but  groups  were  formed  by  people  with  the  same  background.  The 
topic  for  the  groups  with  technical  profile  was  in  AI 

Mediational  tools  included:  (i)  humans,  a tutor  and  a technical  assistant  both  from  our  team 
(ii)  artifacts:  the  phone,  a set  of  documents  and  (iii)  the  prototype:  a networked  computer  environment  including 
the  script  of  the  activity,  private  and  shared  structured  task  spaces  for  peer  argumentative  discussion,  coedition 
and  coordination,  as  well  as  access  to  other  electronically  available  sources  of  information.  Rules  for  the  activity 
were  stated  explicitly  and  have  to  be  accepted  before  starting.  They  include  the  commitment  to  finish  the  work, 
the  script  for  the  activity  and  the  protocol  for  the  collaborative  debate.  Some  of  these  rules  were  embedded  in  the 
system,  for  example,  the  conversational  graph  defined  the  way  one  can  contribute  after  another  peer 
contribution.  Others  were  the  full  responsibility  of  the  learners,  such  as  the  way  to  organize  the  discussion,  the 
way  to  agree,  as  well  as  aspects  of  time  management  and  deadlines.  The  only  direction  given  was  to  use  the 
coordination  space  to  deal  with  all  these  matters,  using  the  coedition  workspace  only  for  the  topic  discussion. 
Students  in  each  group  had  the  same  responsibility.  The  task  is  divided  in  subtasks  but  there  were  no  predefined 
roles  for  division  of  labour. 

Performing  the  activity 

Once  the  learning  activity  was  designed  and  the  prototype  ready,  the  experience  went  to  a set-up  phase  to 
organize  the  formation  of  groups,  fix  the  training  strategy  for  the  use  of  the  tool,  prepare  the  support  strategy  and 
establish  the  evaluation  framework.  In  this  case  students  were  recruited  on  a call  for  volunteers.  The  groups  were 
made  up  by  the  tutor.  A handout  for  the  use  of  the  system  was  distributed  to  all  participants,  and  a session  of 
practice  on  their  own  was  recommended  before  starting  the  activity  to  became  familiar  with  the  tool.  Moreover 
an  interactive  training  session,  using  the  phone  and  the  system,  was  offered  and  carried  out  with  one  of  the 
students.  Technical  support  for  the  whole  period  of  the  experience  was  provided,  a person  could  be  reached 
either  electronically  or  by  phone  to  deal  with  questions  or  problems  related  to  technological  aspects. 

For  content  or  pedagogical  related  matters,  a teacher  was  also  available.  Along  the  running  stage  of  the 
experience  her  role  was  to  act  only  on  request,  however  for  evaluation  purposes  she  was  observing  actively  the 
use  of  the  system.  Events  and  data  were  all  recorded.  This  information  together  with  teacher  and  assistant  notes, 
as  well  as  individual  interviews  for  dropping-out  cases,  were  the  main  sources  for  the  assessment  of  the 
prototype. 

Results 

Three  groups  started  the  activity  and  only  one  arrived  at  a completion  state.  All  the  groups  performed  the 
preparatory  phase,  but  one  of  the  groups  dropped-out  early,  before  engaging  in  the  collaborative  step,  another 
student  dropped-out  in  the  middle  and  we  had  to  redefine  the  learning  task  for  the  remaining  student  to  allow 
him  to  continue  alone.  Finally  the  third  group  worked  well  and  accomplished  the  whole  activity,  having  a 
balanced  participation. 

This  50%  ratio  of  drop-out  is  comparable  to  a standard  course.  In  the  first  group  both  students  changed 
their  job  and  were  not  interested  anymore  in  the  course.  Some  students  needed  help  and  technical  assistance  for 
the  communication  infrastructure,  being  the  first  time  they  had  connected  and  used  Internet.  The  system  was 
considered  easy  to  use,  and  students  in  the  third  group  stated  the  experience  as  successful,  appreciating  the 
benefits  of  collaboration,  despite  the  additional  constraints  this  form  of  work  demanded  on  them.  Added-value 
was  perceived  in  three  dimensions:  more  depth  in  content  related  matters,  becoming  familiar  with  new 
technologies,  and  improving  collaboration  abilities. 

The  concept  of  structured  shared  workspace  proved  suitable  for  the  learning  task,  however,  as  expected, 
significant  problems  were  pointed  out:  some  regarding  either  the  interface,  the  functionality  of  the  prototype,  or 
the  accessibility  and  reliability  of  the  server;  others  to  group  related  aspects  of  engagement,  participation  or  tutor 
intervention.  Related  to  the  interface,  the  representation  of  the  discussion  as  an  index  fully  deployed  on  the  left 
part  was  uncomfortable  to  handle  when  it  became  long.  Some  problems  of  consistency  with  buttons  for  editing 
operations  were  also  detected.  Related  to  the  collaborative  task,  students  missed  the  explicit  agreement  option. 
This  was  considered  to  be  a natural  way  to  conclude  a discussion.  An  important  issue  was  the  lack  of  a separate 
representation  for  the  process  and  the  outcome.  The  possibility  to  look  for  what  was  being  achieved  -drafts  of  the 
essay-  at  different  stages  of  the  process,  separated  from  the  discussion  was  perceived  as  useful. 
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The  coordination  space,  a public  list  of  messages,  suffered  from  lack  of  privacy.  Students  wanted  to 
send  messages  not  only  to  groups  but  also  to  individuals,  particularly  the  tutor  or  the  assistant.  Facilities  for 
browsing  and  reading  related  messages  with  different  criteria,  for  instance  topic,  date,  etc.,  were  not  provided. 
The  period  of  the  experiences  lasted  for  the  ’97  spring  term. 

A second  experience  using  the  same  prototype  was  carried  out,  involving  three  groups  of  students  in  a 
master  program  of  another,  non  distance,  university.  In  this  case  students  combined  face  to  face  teaching 
sessions  with  collaborative  discussions  through  the  system.  Groups  consisted  of  three  students  who  knew  each 
other  previously.  The  patterns  and  style  of  the  interactions  were  clearly  different  and  specific  problems  of  mutual 
awareness  and  concurrence  arose  due  to  a more  synchronous  use  of  the  system.  Comments  from  students, 
feedback  from  tutors,  and  improvements  suggested  by  designers  were  discussed  in  the  analysis  phase  of  the 
second  version  carried  out  during  the  summer,  in  order  to  have  a prototype  ready  for  the  97/98  autumn  term. 

The  second  and  third  cycle 

The  main  redesign  for  the  second  version  concerned  the  structure  and  function  of  the  shared  workspace.  Instead 
of  a unique  workspace,  three  related  subspaces  were  defined,  one  for  the  joint  elaboration,  a second  one,  the 
result  space,  to  keep  separately  the  outcome,  and  the  third,  the  version  space,  to  save  the  process  and  the 
outcome  at  different  stages.  The  conversational  graph  included  agreement  as  an  option.  Once  all  participants  had 
performed  this  action  for  a particular  proposal,  the  system  automatically  loaded  the  text  on  the  corresponding 
subtask  section  of  the  result  space.  Thus,  each  subtask  has  to  be  finished  by  explicit  consensus.  The 
conversational  graph,  wired  in  the  first  version,  was  configurable  from  an  external  data  file  so  that  it  could  be 
different  for  different  tasks.  Moreover,  the  number  of  workspaces  was  an  option  to  be  defined  at  the 
configuration  of  the  prototype,  depending  on  the  needs  of  the  learning  activity. 

Related  to  the  interface,  these  were  the  main  new  features: 

• As  the  number  of  spaces  increased,  on  the  upper  part  of  the  screen,  a button  bar  enabled  the  user  to  go 
directly  either  to  the  information  space,  the  individual  space,  the  shared  workspaces  or  the  coordination 
space. 

• All  the  shared  working  spaces  (version,  result  and  elaboration)  were  directly  accessible  from  each  other,  by 
buttons,  and  they  have  a similar  interface. 

• The  representation  of  the  discussion  process  on  the  left  part  of  the  elaboration  workspace  was  redesigned  in 
order  to  allow  the  folding  and  unfolding  of  the  content  index  of  the  contributions  for  each  subtask.  This 
helped  to  reduce  the  amount  of  visible  information,  focusing  on  the  task  at  hand. 

• The  interface  was  rewritten  in  English  to  allow  a wider  use. 

• Messages  in  the  coordination  space  could  be  one-to-one  or  one-to-many. 

As  in  the  first  cycle,  the  evaluation  phase  consisted  of  steps  for  fixing  the  objectives  of  the  evaluation, 
designing,  setting-up  and  performing  the  learning  experience.  In  this  cycle  we  aimed  at  two  different  scenarios  to 
test  the  versatility  of  the  system  regarding  other  communities  and  activities.  The  first  one  was  to  support  the 
discussion  of  a group  of  international  experts,  distributed  all  over  the  world,  previous  to  the  celebration  of  a 
working  conference.  The  second  one  involved  a teacher  and  their  graduate  students,  all  external  to  the  designer's 
team,  using  the  system  as  a complementary  activity.  The  learning  activity  was  fully  designed  by  the  teacher.  In 
both  cases  the  prototype  was  configured  by  a member  of  the  research  team  to  generate  an  environment  adapted 
to  the  activity,  i.e.,  number  of  workspaces,  its  structures,  associated  conversational  graphs,  and  links  to  other 
sources,  as  well  as  roles  and  participation  rules.  All  these  features  were  different  in  each  experience. 

Results 

From  users  the  main  feedback  in  this  cycle  was  the  need  of:  (1)  having  notification  when  new  things  happened 
related  to  the  group  activity;  (2)  seeing  at  a glance  where  new  things  were  placed. 

From  the  monitoring  of  these  and  previous  experiences,  we  also  concluded: 

In  relation  to  collaborative  aspects 

• In  the  case  of  students,  procrastination  is  a problem,  support  for  explicit  time  management  as  well  as 
negotiated  intermediate  deadlines  should  be  considered. 

• For  full  distance  learners,  a preliminary  phase  to  set  up  the  group,  establishing  common  objectives  and 
protocols  is  a must  to  ensure  their  engagement  as  well  as  to  enforce  their  feeling  of  belonging  to  a group. 

• In  the  case  of  larger,  open  groups,  a well-specified  purpose,  shared  responsibility  and  the  role  of  a moderator 
are  key  factors  to  promote  interaction  and  participation.  The  only  existence  of  the  mediational  tool  does  not, 
of  itself,  generate  patterns  of  collaboration, 

• Teacher  intervention  along  the  collaborative  process  remains  an  open  question:  whether  this  should  be 
supporting  the  group  spontaneously  or  on  request,  being  involved  as  a member  of  the  discussions  groups  with 
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an  specific  facilitator  role,  or  entering  as  commentator  at  specific  time  of  the  process.  Students  feelings  were 
quite  controversial  on  that  point. 

In  relation  to  the  system 

• On-line  help,  guidelines  for  each  activity,  and  rules  of  participation  should  be  included  in  the  system,  in 
addition  to  handouts  (i.e.  the  more  integrated  and  self  contained  the  system,  the  better  its  usability).  For 
Spanish  students,  English  for  the  interface  was  accepted,  if  all  the  rest  was  available  in  Spanish. 

• Facilities,  beyond  notification,  to  handle,  monitor  and  analyze  the  collaboration  processes  are  required.  Both 
during  and  after  the  experience. 

• A configuration  mode  should  be  available  to  generate  tailored  web  sites,  as  automatically  as  possible  from 
the  data  provided  by  the  designer  of  the  learning  activity. 

The  fourth  and  fifth  cycle 

At  that  point  we  decided  to  rethink  the  whole  system  architecture,  characterizing  levels  and  generic  components 
and  implementing  the  workspaces  on  a database  framework.  For  further  details  on  this  point  see  (Barros  & 
Verdejo,  1998).  Besides  the  full  reprogramming,  the  improvements  for  the  next  version  concerned  two  aspects: 

• The  definition  of  roles  for  shared  workspaces  allowing  to  distribute  responsibilities  for  a learning  activity. 
For  each  role  a profile  of  features  was  defined,  where  accesses  and  privileges  for  different  uses  of  the  system 
could  be  configured.  A simple  mechanism  of  notification  was  also  introduced  and  could  be  enabled  or 
disabled  in  the  role  profile. 

• The  redesign  of  the  coordination  space  to  provide  not  only  an  enhanced  e-mail  facility  including  a variety  of 
filters  to  browse  the  messages  with  different  criteria,  but  a new  tool,  a group  agenda  to  facilitate  time 
management  matters. 

Evaluation  Phase 

The  evaluation  followed  the  same  steps  as  in  the  previous  cycles.  This  time  the  new  issue  addressed  was  to  test 
how  easy  the  system  could  be  adapted  by  the  users  themselves,  for  instance,  allowing  learners  to  configure 
workspaces  completely  tailored  to  their  needs.  This  possibility  opened  the  range  of  learning  activities  that  the 
system  could  support.  For  instance,  students  performing  design  tasks  or  negotiating  among  themselves  their 
collaboration  schemas.  So  we  planned  and  executed  an  experience  where  the  learners,  using  the  system,  could 
also  participate  and  decide  aspects  of  the  learning  activity  they  have  to  perform  in  collaboration,  such  as  the 
conversational  graph  structure  for  the  elaboration  process,  roles  and  their  distribution,  or  the  schema  for  the 
outcome.  We  launched  an  experience  comprising  three  learning  activities.  The  first  one,  a collaborative  synthesis 
on  an  Educational  Technology  topic.  The  second  one,  the  selection  and  definition  of  a case  study;  and  the  third 
the  elaboration  of  a detailed  proposal  for  the  case  study.  A teacher,  a technical  assistant  and  two  groups  of  three 
part-time  students  following  a Ph.D.  course  worked  together  for  6 months. 

• The  objectives  for  the  first  activity  included  acquiring  knowledge  of  a selected  topic,  and  developing 
collaborative  abilities  as  well  as  becoming  familiar  with  the  use  of  the  system. 

• The  second  activity  was  a preparation  for  the  third  one,  students  had  to  attain  three  goals:  (1)  to  select, 
discuss  and  agree  on  a case  study,  i.e.  a learning  activity,  (2)  to  write  a detailed  plan  to  develop  their 
proposal  including  the  structure  of  the  outcome,  and  (3)  to  fix  the  responsibilities  of  each  one  on 
completing  this  proposal. 

• The  third  learning  activity  included  the  configuration  of  the  shared  workspaces  where  they  could 
develop  the  design  of  their  case  study. 

They  started  to  work  with  the  third  prototype.  This  time  as  the  experience  lasted  for  a longer  period  as 
soon  as  some  problems  were  detected  we  decided  to  work  simultaneously  on  the  system,  so  that  they  started  with 
version  3 but  finished  with  version  4.  New  features  were  included.  For  instance  the  initial  bar  included  all  the 
activities  and  there  was  the  possibility  to  state  temporal  restrictions  between  activities  so  that  spaces  were 
accessible  only  when  the  time  conditions  were  fulfilled.  Agreement  methods  were  extended  and  could  be 
selected  between  consensus  or  majority.  Further  tools  were  also  provided  for  global  and  individual  evaluation: 
the  representation  of  accesses  per  hour,  work  evolution  in  the  experience  period  and  number  of  contribution 
type.  For  global  analysis  there  were  an  option  for  visualizing  the  evolution  of  the  discussion  by  access  and  by 
user  and  the  evolution  of  work  by  section  and  user.  These  results  could  be  visualized  in  graphics  or  textual  form. 
Also  a tool  to  create,  fill,  submit  and,  automatically  process  WWW-forms  was  provided  in  order  to 
systematically  collect  student  feedback. 

Version  four 

A major  extension  in  version  4 was  the  concept  of  an  organizational  learning  memory.  This  was  a structured 
repository  of  collaborative  learning  activities  containing  selected  events  and  outcomes  (Barros  & Verdejo,  1998). 
This  memory  was  defined  and  implemented  with  a range  of  editing  and  searching  facilities.  Initially  we  filled  it 
with  relevant  data  collected  in  previous  evaluation  cycles.  This  resource  can  be  exploited  for  a variety  of 


259 


educational  purposes,  two  of  them,  scaffolding  and  peer  evaluation,  were  tested  on  the  latest  cycle  developed 
during  the  last  three  months  of  the  97/98  academic  year. 

Two  experiments  were  carried  out.  The  first  comprised  two  learning  activities.  One  was  fully  defined 
by  the  teacher,  the  other  only  partially  predefined.  The  learners  once  the  first  activity  was  completed  had  to 
select  outcomes  of  cases  stored  in  the  organizational  memory  to  perform  a peer  review.  The  second  experiment 
followed  a similar  approach  but  this  time  students  could  use  previous  cases  as  guidelines  for  their  own  work  and 
not  only  for  the  peer  evaluation  step. 

Results 

• The  experiences  in  this  cycle,  as  well  as  the  system,  were  more  complex.  Close  monitoring  from  the  teacher 
became  critical,  but  also  a clear  state  of  affairs  was  very  helpful  for  students.  Notifications  provided 
automatically  from  the  system  to  learners  and  the  teacher  about  selected  events  improved  the  “what  is  going 
on”  with  respect  to  the  other  versions,  but  the  solution  to  present  the  “what  is  new”  required  some  further 
analysis.  In  short  this  information  has  to  be  presented  not  only  in  quantitative  terms  but  also  in  a more 
qualitative  flavor. 

• There  were  also  problems  with  the  generation  of  tailored  versions  of  workspaces.  The  configuration 
procedure  demanded  additional  understanding  of  the  system,  and  the  students  were  not  prepared  to  invest 
time  on  that  point.  They  preferred  to  specify  what  they  wanted  and  have  the  technical  assistant  doing  it  for 
them.  This  suggested  that  more  training  on  the  tool  and  better  configuration  facilities  are  needed. 

These  two  points  were  addressed  on  the  next,  current  version,  of  the  prototype.  The  current  architecture 
of  the  prototype  is  organized  in  levels;  configuration  level,  performance  level,  reuse  level  and  reflection  level. 
We  consider  the  tool  robust  and  versatile  enough  to  finish  the  development  phase  and  to  proceed  to  a wider 
dissemination  in  order  to  prepare  a summative  evaluation  phase. 

Concluding  remarks  and  further  work 

Formative  evaluation  has  been  essential  to  guide  the  team  in  tuning  the  tool  and  for  suggesting  directions  for 
further  extensions.  Prototype  testing  with  real  users  helps  not  only  to  reach  technical  validity  but  more 
importantly,  to  face  the  complexity  of  implementing  collaborative  learning.  Social  and  institutional  factors  are 
essential  when  designing  collaborative  learning  activities,  specially  in  a conventional  educational  framework.  In 
this  sense  Activity  Theory  seems  a promising  framework  for  modeling  learning  situations  with  a global 
approach.  Issues  for  the  next  evaluation  are,  on  the  one  hand,  to  look  more  closely  at  pedagogical  and 
collaborative  effectiveness,  and  on  the  other  to  collect  more  experiences.  We  perceive  that  practical  guidelines 
for  designing  and  performing  these  activities,  not  only  looking  at  learning  objectives  but  also  dealing  with  social 
and  organizational  issues,  are  indeed  needed. 
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Abstract:  This  paper  proposes  a method  of  learning  assistance  for  a physics  learning  environment. 
The  learning  objective  of  the  environment  is  to  acquire  ability  of  recognizing  forces  acting  on 
physical  objects.  Because  the  knowledge  of  applying  fundamental  concept  and  rules,  such  as  the 
concept  of  gravity  and  Newton’  law,  is  empirical  knowledge,  learners  need  training  to  acquire  the 
ability.  We  propose  a method  of  triggering  off  reflection  by  comparing  two  physical  systems.  Our 
method  generates  a physical  system  depending  on  the  cause  of  a learner’s  error.  By  comparing  the 
generated  system  and  the  originally  given  system,  the  learner  gets  some  clues  to  recognize  forces 
acting  in  the  original  system,  or  some  clues  to  recognize  inconsistency  of  the  learner’s  solution. 


1.  Introduction 

One  of  important  features  of  intelligent  tutoring  systems  is  to  provide  assistance  that  is  tailored  for  each  individual 
learner.  There  are  different  teaching  strategies,  among  which  is  to  make  learners  reflect  on  their  processes  of 
working  out  solutions  in  order  to  acquire  correct  knowledge  when  they  make  some  mistakes.  Because  assistance 
should  provide  information  that  leads  the  learner  to  aware  inconsistency  about  the  solution,  it  must  be  prepared 
depending  on  the  causes  of  the  learner’s  errors. 

This  paper  proposes  a method  of  providing  feedback  for  a physics  learning  environment  (Takeuchi  & Otsuki  1997). 
The  learning  objective  of  the  environment  is  to  acquire  ability  of  recognizing  forces  acting  on  physical  objects. 
Learners  have  to  apply  fundamental  concept  and  rules,  such  as  the  concept  of  gravity  and  Newton'  law,  on  a given 
physical  system  in  order  to  recognize  forces.  Because  the  knowledge  of  applying  fundamental  concept  and  rules  is 
empirical  knowledge  (Anzai  1986),  they  need  training  to  acquire  the  ability.  The  assistance  of  promoting  reflection 
is,  therefore,  effective  to  activate  thinking.  We  propose  a method  of  triggering  off  reflection  by  comparing  two 
physical  systems.  Our  method  generates  a physical  system  depending  on  the  cause  of  a learner's  error.  By  comparing 
the  generated  system  and  the  originally  given  system,  the  learner  gets  some  clues  to  recognize  forces  acting  in  the 
original  system,  or  some  clues  to  recognize  inconsistency  of  the  learner's  solution.  Section  2 describes  the  overview 
of  the  learning  environment,  and  the  method  of  triggering  off  reflection  by  comparison  is  given  in  section  3. 
Examples  of  assistance  are  given  in  section  4.  Section  5 describes  evaluation. 


2.  Learning  Environment  for  Physical  System 

Fig.  1 shows  the  structure  of  the  learning  environment  for  physical  system  modeling.  Learners  try  to  apply  physical 
rules  on  a given  physical  system  to  recognize  forces,  and  illustrate  forces  in  order  to  express  their  understanding  on 
the  graphical  user  interface.  Fig.  2 shows  a snapshot  of  the  user  interface.  When  they  want  to  study  specific  physical 
systems,  they  are  allowed  to  construct  arbitrary  physical  systems.  Tab.  1 shows  available  physical  elements.  The 
causal  relation  model  deriver  in  Fig.  1 creates  the  causal  relation  model  from  a given  physical  system.  The  causal 
relation  model  represents  relationship  among  forces  and  elements.  The  student's  kinetic  model  deriver  creates  the 
student’s  kinetic  model  from  learner's  illustration  of  forces.  The  student's  kinetic  model  represents  learner’s 
understanding  of  the  given  system.  The  student’s  kinetic  model  has  the  same  structure  as  the  causal  relation  model. 
These  two  models  are  used  as  the  source  of  diagnosis. 
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FIGURE  NOT  AVAILABLE 


Figure  1:  Overview  of  the  Learning  Environment 
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Figure  2:  Snapshot  of  the  User  Interface 


Table  1:  Available  Physical  Elements 


Class 

Kind  of  Element 

Finite  mass 

Block,  Ball 

Zero  mass 

Spring,  String 

Infinite  mass 

Ceiling,  Wall,  Horizontal  plane.  Slope 

The  learning  support  part  detects  learner's  errors  by  comparing  the  student's  kinetic  model  with  the  causal  relation 
model.  There  are  three  kinds  of  erroneous  phenomena  about  forces;  missing  (learners  cannot  recognize  existing 
forces),  extra  (learners  recognize  non-existing  forces),  and  error  (learners  make  some  mistakes  on  attributes  of 
forces).  The  learning  support  part  decides  a method  of  assistance  according  to  causes  of  learner's  errors.  This  paper 
focuses  on  one  of  the  assistance  method;  “presentation  of  a system  for  comparison”. 


3.  Presentation  of  a Physical  System  for  Comparison 

We  propose  “presentation  of  a system  for  comparison”  as  one  of  assistance  methods  for  learner’s  errors.  Physical 
systems  for  comparison  are  generated  by  transforming  the  original  system  based  on  some  rules.  Purposes  of  the 
assistance  are  both  helping  learners  to  recognize  forces  acting  in  the  original  system  easier  and  giving  learners  some 
clues  to  recognize  errors  and  contradiction  by  thinking  about  similarity  and  difference  between  the  original  system 
and  the  generated  system.  Because  the  method  of  presenting  systems  for  comparison  is  an  indirect  instruction  and  it 
doesn't  communicate  knowledge  of  physics  directly,  it  is  based  on  the  assumption  that  learners  have  knowledge  of 
fundamental  physical  rules  but  failed  to  apply  these  rules.  This  kind  of  errors  often  occurs  about  internal  forces,  i.e. 
tension  and  normal  forces.  The  assistance  method  is,  therefore,  effective  in  cases  of  missing  tension,  errors  of 
tension,  missing  normal  forces  and  errors  of  normal  forces. 


3.1.  Generation  Process  of  Systems  for  Comparison 

The  generation  process  of  systems  for  comparison  consists  of  the  following  four  steps. 

1)  To  generate  candidate  systems  by  applying  transformation  rules  to  the  original  system.  The  transformation  rules 
are  described  in  section  3.2. 

2)  To  derive  the  causal  relation  model  for  each  generated  system. 

3)  To  check  whether  each  generated  system  satisfies  conditions  of  effectiveness  as  systems  for  comparison.  The 
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Conditions  are  given  in  section  3.3. 

4)  To  decide  the  order  of  priority,  if  more  than  one  systems  for  comparison  satisfy  the  conditions  at  3rd  step. 
Ordering  rules  are  given  in  section  3.4. 

Fig.  3 shows  a generation  process. 


3.2.  Rules  for  Automatic  Generation 

Candidate  physical  systems  for  comparison  are  generated  by  applying  transformation  rules  to  the  original  system. 
There  are  eight  rules;  deletion  of  a finite  mass  element,  deletion  of  a external  force,  replacement  of  a finite  mass 
element  with  an  infinite  mass  element,  replacement  of  a slope  with  a horizontal  plane,  deletion  of  a string,  deletion 
of  a spring,  and  replacement  of  an  external  force  with  an  infinite  mass  element.  In  general,  these  rules  are  applicable 
repeatedly  on  an  original  system.  However,  we  restrict  the  application  time  only  once,  because  the  resemblance 
between  the  original  system  and  a generated  system  becomes  low  and  it  becomes  difficult  for  learners  to  compare. 
These  rules  reduce  complexity  of  the  original  system  from  the  viewpoint  of  the  number  of  physical  parameters  or 
the  order  of  motion. 


FIGURE  NOT  AVAILABLE 

Figure  3:  Generation  Process  of  Systems  for  Comparison 

3.3.  Conditions  of  Systems  for  Comparison 

Generated  systems  are  effective  as  systems  for  comparison  if  they  satisfy  one  of  the  following  two  conditions. 
Cond.l)  Learner's  wrong  interpretation  of  the  original  system  holds  in  the  generated  system. 

Cond.2)  The  same  physical  causalities  about  which  the  learner  made  an  error  exist  in  the  generated  system,  and 
complexity  of  the  generated  system  is  lower  than  the  original  system.  The  complexity  is  measured  by  the  number 
of  physical  objects  and  the  order  of  motion. 

Examples  that  satisfy  the  conditions  are  as  follows. 

Ex.1-1)  The  force  that  the  learner  has  missed  in  the  original  system  doesn't  really  act  in  the  generated  system.  Or 
the  extra  force  that  the  learner  illustrated  for  the  original  system  acts  in  the  same  way  as  the  learner's  illustration  in 
the  generated  system. 

Ex.  1-2)  The  force  that  is  equal  with  the  learner's  erroneous  force  acts  in  generated  system.  Correct  forces  that  the 
learner  illustrated  for  the  original  system  also  act  in  the  generated  system. 

Effect  of  Ex.1-1  and  Ex.1-2:  The  learner's  interpretation  of  the  original  system  holds  in  the  generated  system. 
Since  forces  actually  acting  in  two  systems  are  different  and  hence  the  motion  of  them  are  different,  the  learner  will 
find  inconsistency  in  his/her  interpretation.  Because  the  generated  system  is  less  complex  than  the  original  system,  it 
is  easier  for  the  learner  to  understand  generated  system  correctly  . 

Ex.2)  The  Learner  missed  a force  in  the  original  system  and  a force  of  the  same  kind  as  the  missed  force  acts  on  the 
same  element  in  the  generated  system  and  the  element  stands  still. 

Effect  of  Ex.2:  Forces  which  act  on  the  element  balances  in  the  generated  system,  because  the  element  stands  still. 
Because  the  generated  system  is  less  complex  from  the  viewpoint  of  the  order  of  motion,  it  is  easier  for  the  learner 
to  recognize  the  missed  force.  Since  the  generated  system  preserves  causalities  between  physical  parameters,  the 
learner  will  recognize  the  causalities  of  the  original  system. 


3.4.  Order  of  Priority  of  Systems  for  Comparison 

In  most  cases,  more  than  one  candidate  systems  for  comparison  satisfy  the  conditions  described  in  section  3.3.  In 
order  to  select  the  most  suitable  system  for  comparison,  the  order  of  priority  is  decided  according  to  the  following 
criteria: 

* It  is  easy  for  learners  to  find  forces,  if  elements  in  a physical  system  stand  still. 

* It  is  easy  for  learners  to  compare  two  physical  systems,  if  they  are  alike  in  appearance. 

The  order  of  priority  is  defined  by  both  motion  of  elements  in  the  system  for  comparison  and  the  degree  of 
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resemblance  between  the  system  for  comparison  and  the  original  system.  Tab.  2 gives  items  to  be  considered  to 
decide  the  order  of  priority  of  the  systems  for  comparison.  As  for  a string,  a spring,  a finite  mass  element  and  an 
external  force,  the  degree  of  resemblance  is  high  when  the  score  in  Tab.  2 is  high.  As  for  motion,  the  score  is  high  if 
a physical  system  stands  still.  The  score  of  each  physical  system  is  calculated  by  summing  up  each  item's  score.  The 
order  of  priority  is  decided  by  total  score  of  each  system. 

Table  2:  Check  Item  For  Order  of  Priority 


Check  Item 

Score 

Motion 

All  of  finite  mass  elements  stand  still 

2 

At  least  one  finite  mass  element  stands  still 

1 

All  of  finite  mass  elements  don’t  stand  still 

0 

External  Force 

The  force  act  in  the  same  element  of  the  original  system 

1 

The  force  act  in  the  different  element  of  the  original  system 

0 

The  force  do  not  act 

0 

The  force  are  replaced  with  an  infinite  mass  element 

0 

Finite  mass  Element  ( for  each  finite  mass  element) 

The  element  exists  in  the  system  comparison 

2 

The  element  is  replaced  with  an  infinite  mass  element 

1 

The  element  is  deleted 

0 

String,  Spring 

String  or  Spring  exists 

1 

String  and  Spring  don’t  exist 

0 

4.  Example 

Fig.  4 shows  an  example  of  candidate  systems  for  comparison  generated  from  a given  system.  Eight  candidate 
systems  are  generated  by  applying  transformation  rules.  Here  we  assume  that  a learner  made  one  of  the  following 
four  mistakes. 

1)  missing  tension  of  both  ends  of  string. 

2)  missing  tension  of  one  end  of  string,  and  the  value  of  tension  of  the  other  end  is  wrong. 

3)  missing  tension  of  one  end  of  string. 

4)  the  value  of  tension  of  the  string  is  wrong. 

Tab.  3 shows  the  result  of  selecting  appropriate  systems  for  comparison.  Numbers  indicate  appropriateness  of  eight 
candidates  for  the  above  errors.  These  numbers  are  the  order  of  priority  for  each  system,  and  the  smallest 

FIGURE  NOT  AVAILABLE 

Figure  4:  Generated  Systems  as  Candidate  for  Comparative  Systems 

one  is  the  most  suitable  system  for  comparison.  Each  number  in  the  parenthesis  is  a total  score  of  each  system. 
Asterisks  mean  that  candidate  systems  don’t  satisfy  the  conditions  of  the  systems  for  comparison. 


Table  3:  Score  (Our  System) 


Error 

System  1 

System2 

System3 

System4 

Systems 

Systemb 

System? 

Systems 

Missing  tension- 1 

1(7) 

6(3) 

5(4) 

* 

3(6) 

1(7) 

^(6) 

* 

Missing  tension-2 

* 

4(3) 

* 

3(4) 

2(6) 

1(7) 

* 



Missing  tension-3 

* 

3(4) 

* 

* 

2(6) 

1(7) 

* 

* 

Tension  error 

* 

* 

* 

_J(4) 

* 

1(7) 

* 

* 

5.  Evaluation 
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We  evaluated  the  result  of  generating  systems  for  comparison.  In  order  to  investigate  which  physical  systems  are 
appropriate  for  comparison,  we  asked  five  university  students  to  select  effective  systems  for  each  error  in  Tab.  3 
from  eight  candidate  systems.  Tab.  4 shows  the  result  of  the  selection.  Numbers  indicate  the  percentage  of  university 
students  who  answered  that  learners  would  notice  the  error  or  contradiction  by  presenting  the  physical  system  when 
learners  made  the  corresponding  error. 


Table  4: Score  ( University  Students)(%) 


Error 

System  1 

System2 

System3 

System4 

Systems 

System6 

System? 

System  8 

Missing  tension- 1 

100 

60 

80 

40 

80 

80 

80 

40 

Missing  tension-2 

0 

40 

0 

60 

80 

100 

0 

60 

Missing  tension-3 

0 

60 

0 

20 

60 

100 

0 

20 

Tension  error 

0 

0 

0 

40 

40 

80 

0 

0 

By  comparing  Tab.  3 with  Tab.  4,  we  confirmed  that  most  of  the  physical  systems  selected  by  the  learning  system 
were  judged  to  be  effective  for  learner's  errors  by  university  students.  And  when  the  order  of  priority  that  our 
method  decides  is  high,  the  percentage  which  they  judged  the  physical  system  to  be  effective  is  high,  too.  From  the 
above  facts,  we  conclude  that  our  method  selects  effective  physical  systems  and  the  order  of  priority  is  proper. 


6.  Conclusion 

We  described  a method  of  learning  assistance  that  presents  physical  systems  for  comparison.  From  the  result  of 
evaluation,  we  confirmed  that  effective  systems  for  comparison  were  chosen  and  the  order  of  priority  is  proper. 
Simulation  based  interactive  learning  environments  are  widely  accepted  in  the  domain  of  physics  (Tiberghien  & 
Mandl  1991).  Simulation  is  useful  to  understand  fundamental  concept  of  physics  by  experiment  and  to  grasp  system 
behavior  by  observation.  Stevens's  innovative  work  (Stevens,-  Roberts  & Stead  1983)  combined  a quantitative 
simulator  with  qualitative  representation  to  emphasize  causality.  Qualitative  simulation  (Kuipers,  1986,  Forbus  & 
Falkenhainer,  1990)  is  also  useful  to  explain  causality  explicitly  and  to  help  learners  to  understand  causal  relations 
of  physical  parameters.  Both  qualitative  and  quantitative  simulations  are  based  on  equations  of  motion.  Learning 
objectives  of  these  methods  are,  therefore,  to  understand  nature  of  constraints  among  system  parameters.  In  contrast 
to  these  learning  methods,  we  focused  on  acquiring  ability  of  modeling  physical  systems  or  ability  of  deriving 
equations  of  motion,  which  provides  justification  of  causality.  The  assistance  method  presented  in  this  paper  aims  to 
activate  learners'  thinking. 

Because  similar  cases  or  examples  are  effective  for  learning,  some  of  intelligent  tutoring  systems  have  mechanism  to 
manage  similar  cases,  (Schult,  1993)  proposed  a similarity  measure  and  (Reimann,  1993)  proposed  a method  of 
solving  problems  by  using  a case-base  in  the  domain  of  physics.  Learners  expand  their  knowledge  by  considering 
similarities.  In  contrast  to  these  systems,  our  method  uses  similar  physical  systems  to  give  learners  clues  to  revise 
their  errors.  (Asami,  Takeuchi  & Otsuki,  1 995)  also  proposed  a method  to  generate  physical  systems  that  give 
learners  to  correct  mistakes.  Their  viewpoint  for  remedy  is  the  same  as  ours,  but  generation  method  and  learning 
situation  are  different . 

Though  each  module  has  been  implemented,  they  have  not  been  integrated  into  the  learning  environment  at  present. 
Validation  of  the  method  in  real  learning  situation  is  future  work. 
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ABSTRACT:  In  this  paper  we  describe  the  design  of  a WWW-based  course-support  environment  for 
a course  in  instructional-design  theories,  give  an  overview  of  how  the  environment  was  used  as  part  of 
the  course  experience,  and  summarize  the  student  evaluation  of  the  course.  Our  conclusion  is  that  such 
a course-support  environment  can  extend  the  teaching  and  learning  process,  if  well  designed,  by 
bringing  added  opportunities  for  communication,  coaching,  and  the  increase  of  student  self- 
responsibility. We  do  not  see  such  a tool  as  replacing  the  instructor,  but  extending  him  or  her.  The 
design  of  the  environment  should  reflect  this.  Although  the  article  describes  the  particular  case  of  a 
course  about  instructional  design,  we  argue  that  the  conclusions  can  be  valid  for  courses  of  a variety  of 
disciplines  and  instructional  approaches. 


Introduction 

Despite  the  sophistication  of  many  computer-related  learning  materials,  we  believe  that  sooner  or  later  a teacher 
is  required  to  motivate  the  students,  to  coach  the  students  in  the  use  of  problem-solving  methods,  to  explain  the 
descriptions  and  interpretations  of  the  subject  matter,  and  to  model  the  learning  and  practicing  of  skills. 
However,  the  typical  classroom  or  course  setting  is  not  particularly  conducive  to  stimluating  communication 
with  individual  students  in  order  to  recognize  their  intellectual  and  motivational  problems,  to  explain  to  them  a 
difficult  part  of  the  subject  matter,  to  provide  clear  tasks,  and  to  coach  the  students’  problem-solving  activity. 
The  often-repeated  criticism  of  the  whole-class  approach  has  done  little  yet  to  change  the  system  and  increase 
personal  contacts  and  coaching  between  the  teacher  and  individual  students. 


We  believe  that  the  integration  of  computer  technology  with  data  communication,  what  is  called  in  Europe 
telematics,  creates  new  possibilities  to  support  and  amplify  the  communication  between  a teacher  and  the 
students  and  among  the  students  themselves  even  in  a large  class;  increases  the  opportunities  for  the  instructor  to 
coach  on  an  individual  basis;  and  provides  the  students  a possibility  to  become  more  self-reliant  in  their 
approach  to  studying,  while  still  retaining  the  benefits  of  the  familiar  face-to-face  class  setting  and  textbook, 
WWW-based  environments  in  particular  can  combine  the  strengths  of  computer-based  learning  with  those  of 
instructor-guided  learning  and  can  help  to  overcome  the  mass-production  aspects  of  the  large  course  in  higher 
education.  But  how  does  one  design  a WWW-based  course-support  environment  to  reflect  this  instructional 
philosophy?  Are  there  instructional-design  principles  to  follow?  We  address  these  questions  via  an  example,  a 
course  about  instructional  design  at  the  Faculty  of  Educational  Science  and  Technology  of  the  University  of 
Twente  in  The  Netherlands.  The  case  we  describe  is  just  one  of  over  30  courses  in  our  faculty  that  are  taking 
advantage  of  WWW-based  course-support  environments  to  enrich  teaching  and  learning,  for  students  in  the 
regular  institutional  setting  as  well  as  students  sometimes  or  even  primarily  at  a distance  from  that  setting  (see 
Tielemans  & Collis,  1998). 

A Description  of  the  Course  Instructional  Design  Theories 

Instructional  Design  Theories  is  a 120-hour  senior  course  in  the  Faculty  of  Educational  Science  and  Technology 
at  the  University  of  Twente.  Students  in  the  course  represent  a mixed  cohort,  in  terms  of  background  relating  to 
the  theoretical  principles  relating  to  instructional  design  and  of  mother  tongue.  The  language  used  in  the  course 
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is  English.  The  textbook  used  for  the  course  is  a scholarly  analysis  of  the  theoretical  foundations  of  instructional 
design  and  of  key  research  approaches  (Tennyson,  Schott,  Seel,  & Dijkstra,  1997).  Moreover  the  meaning  of  the 
theories  for  solving  instructional-design  problems  is  shown  (Dijkstra,  Seel,  Schott,  & Tennyson,  1997).  Because 
the  students  have  different  levels  of  prior  knowledge  coming  into  the  course,  their  learning  needs  are  different. 

Design  of  the  WWW-Based  Course-Support  Environment 

Based  on  this,  and  also  the  key  principles  for  all  students  of  (a)  supporting  and  amplifying  the  communication 
between  the  teacher  and  the  students  and  among  the  students  themselves,  (b)  increasing  the  opportunities  for  the 
instructor  to  coach  students  on  an  individual  basis,  and  (c)  providing  the  students  the  possibility  to  become  more 
self-reliant  in  their  approach  to  studying,  the  following  requirements  were  established  for  a WWW  environment 
to  support  the  course: 

• The  site  should  provide  access  to  prerequisite  knowledge  for  those  who  need  it. 

• Examples  of  how  instructional-design  theories  and  models  are  applied  in  practice  should  be  available 

• The  information  presented  in  the  textbook  should  be  extended  by  showing  different  perspectives  in  the  topics 
discussed 

• The  meaning  of  critical  terms  should  be  clarified 

• The  students  should  be  supported  in  searching  for  additional  resources  and  examples  illustrating  the  course 
concepts  via  the  WWW 

• The  students  should  be  motivated  and  supported  in  discussing  and  sharing  information. 

Using  the  possibilities  made  available  through  the  TeleTOP  database-generated  course-support  system  at  the 
faculty  (see  http://teletop.edte.utwente.nl,  and  also  Tielemans  & Collis,  1998;  De  Boer  & Collis,  1998a,b)  a 
course-support  environment  was  created  with  the  following  basic  components:  (a)  a news/update  area,  for 
general  communication  from  the  instructor  to  the  students;  (b)  a matrix-like  roster,  in  which  the  weekly 
organization  of  the  course  is  given,  and  linked  to  this  extra  study  materials,  links  to  discussions,  opportunities  for 
students  to  enter  assignments  and  new  materials  into  the  course  site,  and  for  the  instructor  and  other  students  to 
give  feedback;  (c)  a communication  area,  supporting  direct  e-mail  among  all  participants  in  the  course;  (d)  a 
section  with  general  information  and  background  about  the  course;  (e)  a shared- workspace  area  where  groups  of 
students  can  work  and  discuss  together  on  topics  in  the  course;  (f)  a glossary,  with  communication  options  added 
so  that  students  can  suggest  new  entries  or  ask  for  clarification  about  terms;  and  (h)  a resources  area  containing  a 
large  number  of  links  to  external  resources  relevant  to  the  course,  some  to  serve  as  prerequisite  material,  some 
for  illustration  of  concepts  and  theories,  and  others  to  serve  as  enrichment  and  examples  of  professional 
interactions  relating  to  instructional  design.  Figure  1 shows  the  roster  of  the  course. 
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Figure  1.  Roster  of  the  Instructional  Design  Theories  course-support  site 
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A closer  look  at  the  roster 


The  roster  works  as  a course  organizer.  For  each  session,  it  shows  the  date  and  place  of  the  session,  gives  links  to 
self-study  activities  that  should  take  place  before  the  session,  links  to  notes  to  accompany  the  face-to-face 
session,  links  to  materials  created  by  the  students  for  presentation  during  the  face-to-face  sessions,  and  follow-up 
activities  after  the  session.  The  session  notes  are  prepared  by  the  instructor  to  help  the  students  deal  with  each  of 
the  chapters  and  topics  in  the  course.  The  notes  also  allow  the  instructor  to  update  the  material,  add  links  to 
views  of  other  specialists,  provide  links  for  further  exploration,  and  to  illustrate  concepts  with  examples  and  real- 
life  applications. 

There  were  three  types  of  follow-up  assignments:  discussion  questions,  assignments  relating  to  the  creation  of 
reports,  and  self-tests.  For  the  discussion  assignments,  students  took  part  in  discussions  relating  to  instmction- 
dcsign  issues  available  via  the  Internet.  This  serves  to  increase  their  sense  of  the  application  of  the  concepts  in 
the  study  materials,  and  also  helps  them  to  articulate  and  defend  their  own  ideas.  They  also  discussed  among 
themselves,  using  the  shared  workspace,  each  entering  some  reflections  and  then  commenting  on  the  reflections 
of  the  other  students  in  their  group.  The  instructor  occasionally  intervened,  to  support  and  motivate,  but  the  main 
focus  was  on  communication  with  others  in  the  field  as  well  as  one's  classmates.  For  the  report-type  assignments, 
students  needed  to  search  the  Internet  to  find  additional  resources  to  support  their  ideas.  Also,  they  posted  their 
reports  on  the  course  site  for  peer-review.  Again,  the  instructor  coached  and  commented  on  an  individual  basis 
when  useful.  The  TeleTOP  course  environment  allowed  the  easy  entry  of  comments  and  files,  as  well  as 
feedback  and  follow-up  to  materials  that  have  been  entered;  all  the  instructor  and  students  need  to  do  to  type  in 
their  responses  or  upload  previously  created  files.  The  entries  are  immediately  entered  into  the  course  database 
and  integrated  into  the  course  WWW  environment.  The  self-testing  was  done  with  Java-enabled  test  items  that 
provide  immediate  feedback  after  the  student  submits  an  answer.  The  students  were  free  to  make  as  much  use  of 
these  resources  as  they  wished.  An  important  part  of  the  course  site  was  the  glossary  section.  In  the  glossary,  all 
terms  in  the  course  are  defined,  and  external  links  provided  to  lead  to  more-detailed  information  about  each  term 
and  concept.  In  addition,  a communication  form  was  provided  so  that  students  could  immediately  enter  a 
suggestion  for  an  addition  to  the  glossary,  or  ask  for  clarification.  The  instructor  would  then  respond,  sometimes 
with  an  email,  sometimes  with  a new  entry  in  the  glossary,  sometimes  by  a further  comment  during  the  next 
face-to-face  session. 

Student  Evaluation  of  the  Course 

At  the  conclusion  of  the  course,  the  students  responded  to  a questionnaire  embedded  directly  in  the  course 
environment.  The  questionnaire  contained  22  multiple-choice  questions  and  1 2 open  questions.  Among  the  main 
results  of  the  evaluation  are  the  following  points: 

• In  general,  the  students  enjoyed  the  site  and  especially  appreciated  the  well-organized  extra  resources. 

• The  students  immediately  contributed  to  the  site  by  adding  their  own  PowerPoint  presentation  materials  to 
support  their  class  presentations.  The  students  indicated  they  planned  to  continue  using  the  resources  for 
other  courses. 

• Although  they  appreciated  the  follow-up  assignments,  they  often  did  not  have  time  to  carry  them  out  if  they 
were  not  given  points  but  only  offered  on  a voluntary  basis. 

• Students  need  an  initial  demonstration  of  the  site,  and  how  to  use  its  features  such  as  the  shared  workspace 
and  the  facilities  for  uploading  and  downloading  files  and  giving  comments.  Students  do  not  read  the  user 
manuals  provided. 

• Students  appreciated  being  able  to  read  the  contributions  of  other  students  in  the  shared  workspace. 

• Students  made  particular  use  of  the  site  when  studying  for  the  examination. 

• Students  appreciated  a convenient  way  to  communicate  among  themselves  and  with  the  instructor. 

The  students  did  make  constructive  critical  comments  about  the  design  of  the  site,  and  the  manner  of  integrating 
it  into  the  course  (i.e.,  its  fit  with  the  instructional  design  of  the  course).  In  particular,  while  they  appreciate  rich 
resources,  they  feel  they  lack  the  time  to  exploit  them  unless  this  time  is  included  in  the  marking  scheme  of  the 
course.  This  is  especially  true  with  discussions;  in  order  to  get  a critical  mass  of  discussion  activity,  students 
appreciate  the  stimulus  of  getting  points  for  their  contributions.  Students  also  commented  that  learning  from  each 
other  also  has  limitations:  Learning  form  the  presentations  of  others  is  not  always  good,  because  they  can  tell 
wrong  things  or  they  cannot  explain  well  When  the  instructor  explains,  it  is  interesting,  but  there  is  often  not 
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enough  time  because  of  student  presentations.  An  evaluation  of  the  1998-1999  cycle  of  the  course  will  be 
presented  at  the  ED-MEDIA  ’99  conference. 

Discussion 

The  site  was  used  as  a support  for  the  course,  to  lead  the  students  to  decide  for  themselves  what  extra  materials 
they  wish  to  select  and  study.  The  goal  is  to  support  their  increased  self-responsibility  for  learning.  But  students 
also  need  a push,  or  a reward;  in  the  following  cycle  of  the  course,  points  will  be  given  to  acknowledge 
participation  in  the  discussions.  The  most  important  function  of  the  course  site  was  to  stimulate  a more-active 
learning  process  among  the  students,  by  providing  valuable  exercises  and  letting  them  participate  in  building  the 
course  site  by  finding  or  creating  resources  and  sharing  them  with  others.  The  purpose  of  the  course  site  is  more 
than  putting  instructional  content  in  a form  ready  for  study.  If  our  aim  as  instructors  is  to  coach  students  to  think 
critically  and  to  reason  about  concepts  and  issues,  as  well  as  having  the  ability  and  motivation  for  learning 
independently,  then  the  instructional  design  of  the  course-support  site  should  reflect  these  aims.  For  example, 
using  the  site  to  search  for  additional,  and  relevant,  information  and  examples,  and  presenting  them  within  the 
site  so  that  the  other  students  could  reflect  on  each  others'  choices  is  a valuable  activity  for  our  aims.  In  the  case 
of  students  all  working  in  a language  which  is  not  their  mother  tongue  (English),  the  asynchronous  nature  of  the 
course  site  allowed  them  the  time  to  construct  their  responses  without  the  difficulties  of  speaking  spontaneously. 


Extending  our  experiences  to  other  types  of  courses 

Although  this  case  study  has  focused  on  a particular  course,  it  is  our  experience  from  working  with  the 
instructors  of  30  courses  within  our  faculty  during  the  past  year  that  our  main  conclusions  are  relevant  to  a large 
variety  of  courses.  With  all  of  these  courses,  the  following  guidelines  have  been  appropriate,  although  the  way 
of  realizing  them  varies  considerably  from  course  to  course: 

• A WWW-based  course-support  environment  should  extend  the  boundaries  of  the  traditional  teaching  that 
occurs  in  the  classroom. 

• It  should  support  while  challenging  students'  thinking. 

• It  should  include  a resource  center  for  the  investigation  of  a variety  of  information,  with  an  emphasis  on  the 
students  evaluating  this  additional  information  in  terms  of  its  meaningfulness  to  the  course. 

• It  should  support  discussion,  exchange  of  ideas,  and  peer  evaluation,  and  these  activities  should  be  part  of  the 
graded  activities  in  the  course. 

• It  should  allow  the  students  to  organize  and  restructure  information  as  well  as  create  and  contribute  their  own 
resources. 

• It  should  present  a convenient  way  for  students  to  participate  in  collaborate  work  and  in  the  communication  of 
ideas  and  questions,  in  ways  that  cannot  take  place  in  the  classroom  due  to  restricted  time  and  the  difficulty  of 
making  reflective  comments  in  a classroom  setting. 


Critical  to  all  of  this  is  a user  interface  that  makes  contribution  to  the  course  site  simple  for  both  the  instructor 
and  students.  Thus  other  major  requirements  in  terms  of  the  instrumentation  of  the  course  site  are: 

• The  course  site  should  allow  students  and  the  instructor  to  enter  comments,  and  upload  and  download  files, 
without  having  to  know  any  technical  aspects  of  the  process. 

• The  access  to  the  course  site  should  be  through  the  usual  WWW  browser,  requiring  no  separate  software  for 
either  instructor  or  students. 

• All  course-support  resources  should  be  integrated  within  a single  WWW  environment. 

• Students  should  be  co-builders  of  the  course  resources,  and  thus  need  to  be  able  to  see  and  give  comments  on 
the  entries  of  their  peers  into  the  course  site.  Thus  the  course  environment  must  grow  during  the  process  of  the 
course,  based  on  student  entries,  and  thus  must  be  flexible  technically  to  handle  this  and  still  maintain  the 
underlying  database  relationship. 

Conclusion:  Toward  an  Instructional  Design  for  Course-Support  Environments 

The  value  of  a WWW-based  course-support  environment  depends  entirely  on  how  the  environment  is  designed 
and  how  it  is  integrated  into  the  rest  of  the  teaching-learning  process.  But  because  of  the  newness  of  the  WWW, 
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instructional-design  models  for  the  whole  instructional  communication  in  which  WWW  sites  are  included  have 
to  be  developed.  Such  models  could  retain  the  skills  and  insights  of  the  personal  instructor  and  at  the  same  time 
make  use  of  the  time  involvement,  specialisation  skills,  sophisticated  tools,  and  task  division  possible  in  the 
professional  development  team  (Collis  & De  Boer,  1998b).  This  is  the  situation  now  at  the  University  of 
Twente.  We  hope  that  the  guidelines  suggested  in  this  article  can  contribute  to  an  instructional-design  model  for 
courses  making  use  of  WWW-based  support  environments. 

Note:  This  paper  is  a shorter  version  and  revision  of  Dijkstra,  Collis,  & Eseryel  (1998). 
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Abstract:  Educational  hypermedia  is  often  seen  as  an  improvement  of  textbooks.  In  this  paper  we 
show  how  educational  hypermedia  can  be  improved  by  understanding  it  as  a completely  new 
paradigm  that  has  quite  few  in  common  with  books.  The  benefits  of  features  such  as  continuous 
media,  user-driven  interaction,  layered  structures,  and  adaptation  can  only  be  fully  exploited  if  they 
are  not  just  seen  as  additional  possibilities  for  textbook  writing. 


A Change  of  Paradigms 

Educational  Hypermedia/Multimedia  is  a completely  new  paradigm  and  should  be  treated  that  way.  This 
especially  means  that  it  can't  be  compared  to  books,  it  does  not  compete  with  books,  and  it  has  almost  nothing  to  do 
with  books.  It  does  not  make  sense  to  read  a novel  from  a computer  screen  as  it  does  not  make  sense  to  learn 
statistics  from  a book  — even  though  millions  of  people  did  so  for  hundreds  of  years. 

Especially  for  educational  and  informational  purposes  Hypermedia/Multimedia  is  the  paradigm  of  choice,  as  it 
allows  the  integration  of  differently  encoded  media,  such  as  text,  graphic,  audio  and  video, 
allows  interactivity 

provides  possibilities  for  non-sequential  and  even  non-hierarchical  structures, 
enables  multi-dimensional  navigation  (hyperlinks,  guided  tour,  glossary)  and 
allows  adaptation  to  the  current  users  prerequisite  knowledge  and  preferred  learning  model. 

In  this  paper  we  describe  some  of  the  experience  we  made  during  the  last  four  years  developing  educational 
hypermedia  within  the  DIALECT  project  {Digital  Interactive  Lectures).  This  experience  is  based  on  an  empirical 
evaluation  (Weiber,  1997),  user  surveys,  user  interviews,  and  user  observations.  We  will  concentrate  on  the  first  four 
aspects,  for  more  detailed  information  about  adaptation  see  (Caumanns,  1 998). 


Multimedia 

Computers  are  not  books  and  information  can  be  provided  by  other  means  than  text  (ask  your  kids!).  We  suggest 
that  the  media  of  choice  for  educational  hypermedia  should  be  animation  and  video.  Animation  in  most  cases  is  the 
best  way  to  make  abstract  knowledge  visible  and  therefor  more  understandable.  Video  stories  help  providing  an 
authentic  context  for  embedding  the  theoretical  part  of  the  application. 

Since  our  first  application  in  1996  (Apostolopoulos  et  al.,  1996)  until  now  we  have  made  a radical  shift  from  text 
to  video  and  animation  as  the  main  means  of  delivering  information.  Our  first  application  about  marketing  had  a rate 
of  about  one  animation  per  seven  screen  pages  and  an  average  of  less  than  40  seconds  of  video  and/or  animation  per 
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screen  page.  The  latest  application  we  completed  was  about  taxes  (Buchmann  & Geukes,  1 998).  It  contained  more 
animations  than  screen  pages  with  an  average  of  more  than  4 minutes  of  video  and/or  animation  per  screen  page. 
The  average  animation  length  nearly  increased  by  a factor  of  two  (from  98  to  167  seconds).  The  amount  of  text 
decreased  as  radical  as  the  amount  of  continuous  media  increased. 


The  Use  of  Text 

Currently  most  educational  hypermedia  applications  that  were  not  particularly  designed  for  kids  are  still  heavily 
based  on  text.  Especially  web-based  educational  systems  still  use  text  as  the  main  media  for  transporting  content. 
Many  of  these  applications  look  as  if  someone  transformed  a book  chapter-by-chapter  into  a set  of  hyperlinked 
HTML  pages,  without  taking  care  of  the  differences  between  paper  and  computer  screens: 

Reading  speed  from  a computer  monitor  is  about  25%  slower  than  from  paper  (Nielsen,  1 997a). 

Only  16%  of  web  users  read  texts  word  by  word  (Nielsen,  1997b).  This  figure  might  be  higher  for  systems  not 
based  on  the  WWW,  but  possibly  even  for  these  applications  it  won't  exceed  50%. 

Images  always  catch  more  attention  than  text.  Whenever  there  is  an  image,  a diagram,  or  a video  on  a page,  text 
is  read  last. 

These  figures  don't  imply  that  text  is  completely  useless  or  waste  of  space,  but  they  suggest  to  restrict  the  use  of 
text  to  where  it  is  best  suited: 
giving  short  summaries, 

structuring  screen  pages,  e.g.  by  headlines  and  subheadings,  and 
listing  items,  keywords,  links,  etc. 

Providing  information  just  by  text  easily  leads  to  inert  knowledge  - knowledge  that  might  help  a student  to  solve 
a test  but  never  to  solve  a real-life  problem.  E.g.  most  textbooks  and  hypermedia  applications  about  statistics  contain 
a sentence  like  "The  median  is  more  robust  than  the  mean  as  it  is  not  influenced  by  outliners. " Pupils  will  remember 
that  (at  least  until  the  test)  but  only  a minority  of  them  will  think  about  it  and  understand  it.  Providing  this 
knowledge  by  an  animation  - e.g.  by  showing  a distribution  with  its  mean  and  median  and  slowly  moving  an  single 
data  point  to  the  left  or  right  - does  not  guarantee  the  students  will  be  able  to  write  such  a well  phrased  sentence 
about  robustness,  but  they  will  understand  its  meaning. 

Especially  rather  abstract  or  more  sophisticated  concepts  can  best  be  mediated  by  animation  or  video,  e.g.  how  a 
superscalar  and  superpipelined  RISC  processor  works,  or  how  certain  distributions  can  be  mapped  to  real-life 
problems. 


Context  and  Authenticity 

The  advantages  of  video  in  general  and  video  stories  in  particular  are  well  known  and  have  been  discussed  in 
detail  by  various  authors  (e.g.  Bielenberg  & Carpenter-Smith,  1996).  Video  is 
more  motivating, 
more  engaging, 
easier  to  remember, 
and  has  a higher  content  bandwidth 

than  any  other  known  media.  Further  on,  video  is  best  suited  for  creating  an  authentic  learning  context  (CTG,  1992). 
Especially  for  constructivistic  approaches  such  as  Situated  Learning  (CTG,  1992),  Digital  Interactive  Lectures 
(Apostolopoulos  et  al.,  1996),  and  Cognitive  Apprenticeship  (Collins,  et  al.,  1989)  a video  story  in  most  cases  is  an 
integral  part  of  the  application. 

For  all  of  our  applications  we  try  to  create  a story  that  not  only  provides  a learning  context  but  also  shows  the 
relevance  of  the  topic.  It  is  very  important  for  the  students  to  see  that  the  knowledge  obtained  by  the  application  can 
be  useful  for  their  private  or  professional  life.  The  main  actor  of  our  stories  always  is  either  a student  or  a young 
employee,  who  is  faced  with  a problem  he  or  she  cannot  solve  on  his/her  own.  Help  to  solve  the  problem  comes 
from  various  sources:  professionals,  fnends,  teachers,  etc.  These  characters  work  as  a bridge  between  the  story  and 
the  theoretical  part  of  the  application,  as  they  are  the  only  ones  who  explicitly  talk  about  theoretical  aspects  of  the 
knowledge  domain. 

One  of  the  drawbacks  with  continuous  media  we  had  to  solve  was  the  hyperlink-problem.  Using  text  with 
educational  hypermedia  is  attractive  as  currently  text  and  images  are  the  only  media  that  can  contain  embedded 
hyperlinks  - either  as  hot  words  or  as  hot  zones.  We  solved  this  problem  by  introducing  hot  frames,  A hot  frame  is  a 
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very  simple  yet  powerful  tool  to  bind  a certain  frame  of  a video  or  animation  to  an  event.  We  used  these  hot  frames 
to  provide  a list  of  hyperlinked  keywords  - whenever  a certain  concept  is  mentioned  in  the  video  an  appropriate 
keyword  is  displayed.  Clicking  this  keyword,  the  user  either  gets  additional  information  about  that  concept  by  a 
popup  window  or  a jump  to  the  screen  page  containing  information  about  the  concept. 


Interaction 

Computers  are  not  books  and  users  should  be  able  to  do  more  with  a hypermedia  applications  than  just  switching 
pages.  Interactivity  is  certainly  one  of  the  most  over-used  buzz-words  in  our  business:  A video  is  called  interactive  if 
it  can  be  stopped;  a graphic  is  interactive  if  it  contains  hot  zones;  a text  is  interactive  if  it  contains  hyperlinks;  a 
screen  page  is  interactive  if  it  contains  a next  page  button;  a diagram  is  interactive  if  it  can  be  scaled;  ... 

Interaction  is  the  most  powerful  tool  we  have  at  hand  to  help  users  learning  new  and  abstract  concepts.  The 
obvious  reason  for  the  strength  of  interaction  is  that  it  inherently  supports  the  idea  of  "learning  by  doing".  The  not  so 
obvious  reason  is  that  it  permits  users  to  check  their  newly  obtained  knowledge. 


Unsupervised  Interaction 

Interaction  in  educational  hypermedia  in  many  cases  is  unsupervised;  there  is  no  right  and  no  wrong.  Typical 
tasks  with  unsupervised  interaction  are  "Change  the  data  and  look  what  happens"  or  "Connect  these  objects  and 
check  the  result".  An  example  in  the  domain  of  statistics  would  be  a raw  data  set  and  a boxplot:  The  user  changes 
the  raw  data,  the  boxplot  changes.  The  system  does  never  say,  that  any  of  the  changes  done  was  right  or  wrong. 

Especially  when  abstract  concepts  can  be  visualized  easily  - e.g.  by  diagrams  - unsupervised  interaction  follows 
the  idea  of  "learning  by  doing".  Users  can  prove  their  own  theories  or  even  develop  new  ones  by  "playing"  with  the 
system. 

In  our  current  application  about  statistics  unsupervised  interaction  is  realized  by  the  Statistics  Laboratory 
(StatLab).  The  Statistics  Laboratory  is  a toolbox  of  visual  and  statistical  objects  including  worksheets,  data 
generators,  various  diagram  types,  text  fields,  and  an  S+  interpreter.  Connecting  these  objects,  students  can  create 
complex  filter  graphs,  where  data  "flows"  from  a source  (e.g.  a worksheet)  through  various  filters  (e.g.  S+ 
Interpreters)  to  a sink  (e.g.  a diagram). 

The  basic  ideas  that  led  us  to  the  Statistics  Laboratory  were 

Immediate  Response:  Whenever  the  user  changes  data  at  the  source  of  the  filter  graph,  these  changes  are 
immediately  visible  in  all  objects  that  are  somehow  connected  to  the  source. 

Unlimited  Possibilities:  StatLab  puts  nearly  no  restrictions  on  either  the  kind  of  data  used  or  on  the 
calculations  performed.  This  allows  users  to  experience  their  own  topics.  They  choose  the  data  and  the  method. 
The  drawback  of  this  flexibility  is  that  any  kind  of  supervision  of  the  users  actions  is  impossible  (see  next 
section). 

Other  examples  for  unsupervised  interaction  in  an  application  about  statistics  can  be  found  in  (Schulmeister,  1 996). 


Supervised  Interaction 

The  latest  application  we  finished  contains  two  kinds  of  interaction:  multiple  choice  questions  and  spreadsheets 
for  solving  some  real-life  problems  that  were  derived  from  the  video  story.  Before  the  initial  test  phase  we  thought 
that  users  would  favor  the  (unsupervised)  spreadsheet  problems  as  they  were  "more  interactive"  (users  could  enter 
arbitrary  numbers  and  formulas,  visualize  the  data  in  various  ways,  etc.)  while  multiple  choice  questions  are  rather 
"boring"  in  a way  that  interaction  is  reduced  to  clicking  a radiobutton.  After  the  first  test  phase  we  knew  that  it  was 
just  the  other  way  round,  as  one  of  the  mostly  stated  suggestions  for  improvements  was  "There  should  be  more 
multiple  choice  questions". 

Multiple  choice  questions  are  an  example  for  supervised  interaction:  the  user  clicks  a button  or  enters  a number 
and  immediately  gets  a response  whether  he  was  right  or  wrong.  The  problem  with  supervised  interaction  is  that  its 
coding  complexity  increases  exponentially  with  the  possibilities  left  to  the  user.  Multiple  choice  questions  are 
simple:  The  user  can  only  select  between  a fixed  number  of  choices,  some  of  them  are  right  and  some  are  wrong.  If 
checkboxes  or  radiobuttons  of  a multiple  choice  test  are  exchanged  by  edit  fields  where  users  can  enter  their  answer 
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as  natural  language,  things  start  getting  difficult.  They  even  get  harder  if  a problem  is  made  up  of  several  smaller 
subproblems  which  might  be  solved  in  arbitrary  ways  and  in  arbitrary  order. 

In  order  to  provide  supervised  interaction  not  the  user  but  the  application  must  be  in  control.  Each  action  of  the 
user  must  be  tracked  and  matched  with  a list  of  correct  and  wrong  solutions.  Restrictions  must  be  set  up  to  prevent 
the  user  from  doing  something  completely  silly  that  is  not  contained  in  the  right-and-wrong  list.  All  problem-solving 
interaction  that  contains  some  kind  of  mathematics  is  faced  with  the  "magical  number" -problem:  e.g.  The  user 
introduces  a new  variable  named  "x"  with  a value  of  42.  What  does  42  stand  for?  Is  there  a reasonable  meaning  on 
"x"?  We  don't  know  and  a piece  of  software  surely  cannot  know  either.  To  prevent  this  problem,  generation  of 
numbers  may  only  be  allowed  by  using  a well-defined  set  of  functions.  But  would  you  like  to  force  your  users  to 
type  in  a more  or  less  cryptic  command  just  to  sum  up  1 and  I? 

Within  the  last  years  some  progress  has  been  made  in  the  area  of  supervised  interaction  without  restricting  the 
user  too  much  (e.g.  Cognitive  Task  Analysis  (Lovett,  1998)).  But  most  of  this  progress  has  been  made  in  the  domain 
of  mathematics,  statistics,  and  programming  languages,  where  the  alternatives  of  the  user  are  inherently  restricted  to 
either  numbers  or  a well-defined  set  of  symbols.  Future  will  show  if  some  of  these  ideas  can  be  mapped  to  less 
structured  and/or  less  formal  domains  such  as  economics  or  language  learning. 


Structure  and  Navigation 

Computers  are  not  books  and  access  can  not  only  be  provided  by  a table  of  contents.  To  access  the  contents  of  a 
book  the  table  of  content  or  the  index  is  used.  To  access  the  contents  of  most  hypermedia  application  the  table  of 
contents  or  the  index  is  used  as  well.  Still  most  educational  hypermedia  are  build  on  hierarchical  structures:  two 
levels  of  headings  lead  to  sequences  of  screen  pages. 

There  is  nothing  wrong  with  hierarchical  structures  as  in  most  cases  they  reflect  the  structure  of  the  knowledge 
domain  of  interest  very  well.  They  just  get  inappropriate  for  hypermedia  applications  if  there  is  more  than  one 
reasonable  structure  on  the  knowledge  domain  or  if  they  provide  the  only  possibility  to  access. 


Tracks  in  Hyperspace 

One  of  the  experiences  we  made  is  that  users  - especially  if  they  are  unfamiliar  with  either  the  contents  or  the 
presentational  form  of  educational  hypermedia  - like  to  follow  a sequential  track.  This  track  may  as  well  be  a video 
story  as  a guided  tour  or  some  kind  of  "where  to  go  next"  hints. 

As  a consequence  of  this  experience  our  current  application  about  statistics  contains  two  sequential  tracks:  a 
video  story  as  a central  thread,  and  a collection  of  problems  for  explorative  learning.  The  theoretical  part  of  the 
application  is  structured  as  a layered  hierarchy  of  concepts  (e.g.  descriptive  statistics,  univariate  data,  measurements, 
mean  and  median,  calculating  the  mean  from  classified  data).  An  user  may  enter  the  application  either  by  watching 
the  story,  by  solving  problems,  or  by  working  through  the  theoretical  part. 


Video  Story  Theory  Problems 


Figure  1: 


The  logical  structure  of  Statistics  Interactive:  Two  sequential  tracks  - story  and  problems  - and  a 
"classical"  tree-structured  main-part. 
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Switching  between  the  three  parts  of  the  application  is  made  as  easy  as  possible: 

The  video  story  can  be  split  into  a number  of  small  clips.  After  each  clip  users  can  either  continue  watching  the 
story  or  switch  to  the  theory  to  glean  information  about  the  concepts  introduced  by  the  last  clip. 

If  the  user  needs  some  further  hints  while  working  in  the  StatLab,  he/she  can  always  jump  to  the  theoretical  part 
to  learn  about  the  concepts  or  formulas  required. 

After  working  through  a concept  in  the  theoretical  part  of  the  application,  a user  may  either  Jump  to  the  StatLab 
to  practice  and  proof  his/her  new  knowledge  or  go  to  the  story  track  to  see  how  this  knowledge  is  used  to  solve 
a real-life  problem. 

Providing  multiple  ways  to  access  the  content  of  an  application  will  be  attractive  for  both  inexperienced  and 
experienced  users.  Learning  to  understand,  learning  to  pass  an  examination,  learning  about  a whole  domain,  and 
looking  up  a single  keyword  or  formula  is  all  supported  by  providing  multiple  tracks. 


Layering 

Some  knowledge  is  more  important,  fundamental,  or  useful  than  other  knowledge.  Knowing  about  purpose  and 
application  of  a concept  is  more  fundamental  than  knowing  its  definition  or  formula.  Knowing  a formula  is  more 
useful  than  knowing  its  proof 

Currently  many  hypermedia  applications  still  make  this  layering  of  importance  only  explicit  by  using  textual 
explanations.  We  prefer  to  make  it  explicit  by  means  of  structure.  Therefor  we  suggest  a cascaded,  four-layered 
content  structure  with  the  most  fundamental  and  most  important  knowledge  at  the  top  layer: 
contextual  knowledge,  e.g.  what  is  the  purpose  of  measures  of  central  tendency 
conceptual  knowledge,  e.g.  what  is  the  arithmetic  mean 
abstract  knowledge,  e.g,  how  to  calculate  the  arithmetic  mean 
additional  knowledge,  e.g.  proofs  and  less  important  measures 

From  each  screen  page  a user  can  either  go  on  to  a page  about  the  next  concept  on  the  current  layer  or  jump  to 
the  first  concept  of  a deeper  layer.  E.g.  the  page  about  measures  of  central  tendency  on  the  contextual  layer  has  two 
successors:  contextual  knowledge  about  measures  of  dispersion  and  conceptual  knowledge  about  the  arithmetic 
mean.  The  page  about  the  mean  on  the  conceptual  layer  has  also  two  successors:  the  median  and  calculating  the 
mean  from  raw  data. 

By  allowing  both  horizontal  navigation  (staying  on  the  same  layer)  and  vertical  navigation  (jumping  to  a deeper 
layer),  users  will  easily  see  how  important  and  how  fundamental  a certain  concept  or  topic  is  for  understanding  the 
whole  knowledge  domain  of  interest.  By  layering  information  we  further  on  allow  users  first  to  get  an  overview  of 
the  whole  domain  before  getting  into  details. 


Conclusion 

In  this  paper  we  tried  to  point  out  how  educational  hypermedia  currently  is  still  captured  by  many  old  textbook 
paradigms.  Especially  the  overuse  of  text,  missing  interaction,  and  pure  hierarchical  structures  very  often  hide  away 
the  advantages  of  this  new  teaching  paradigm.  To  overcome  these  deficits  we  propose  to  put  very  strong  emphasis 
on  the  subterms  ’’multi”,  "hyper",  and  "media"  in  "educational  multimedia/hypermedia’’: 

Text  is  only  suitable  if  it  is  printed  out,  it  is  not  suitable  if  it  must  be  read  online.  Continuous  media  should  be 
used  instead  whenever  possible.  Low-bandwidth  yet  high-quality  encoding  techniques  like  streaming  video  (e.g. 
RealMedia)  or  vector  animation  (e.g.  Macromedia  Flash)  make  video  and  animation  the  media  of  choice  even 
for  web-based  applications. 

Interaction  is  hypermedia's  sharpest  sword.  It  not  only  supports  the  idea  of  "learning  by  doing"  but  even  allows 
users  to  proof  their  newly  obtained  knowledge.  An  optimal  interaction  design  should  be  unrestricted,  users 
should  be  able  to  click,  switch,  or  enter  whatever  they  want.  This  demand  is  difficult  to  uphold  if  interaction  is 
supervised  as  keeping  track  with  and  interpreting  the  users  actions  usually  is  a task  of  exponential  complexity. 
Hypermedia  applications  should  be  structured  in  a way  that  allows  for  multiple  access.  By  providing  a 
sequential  story  or  case  study  together  with  a hyperlinked  theoretical  part  the  demands  of  both  novice  and 
experienced  users  can  be  satisfied.  Differences  in  importance  and  relevance  of  topics  and  concepts  should  be 
made  explicit  by  structural  means. 
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Navigation  should  be  horizontal  and  vertical,  that  the  user  might  either  navigate  on  the  same  layer  of 
abstraction  or  go  into  detail  about  a certain  topic  by  switching  to  a more  concrete  layer.  Guided  tours  and 
overview  maps  should  always  be  provided  as  alternative  ways  of  navigating  through  the  application. 

A video  story  cannot  only  be  used  for  motivating  and  for  creating  a learning  context.  It  is  also  well  suited  as  a 
navigational  item  as  it  provides  users  with  a central  thread  through  the  application. 

We  are  going  this  way  of  strong  definition  from  conventional  textbooks  for  four  years  now.  When  we  started,  we 
were  less  radical  than  we  are  now.  We  thought  that  users  always  wanted  something  familiar,  but  that  was  wrong: 
users  don't  want  something  familiar  - they  want  something  good! 
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Abstract:  This  experiment  examined  the  effects  of  content  domain  expertise  on  the  ability  to 
identify  goal-related  units  on  a hypertext  content  map.  Novice  learners  based  their  unit 
selections  on  similarities  between  the  wording  of  the  unit  titles  and  learning  goals  (i.e.,  short- 
answer  questions).  Wording  similarities  had  significantly  less  effect  on  the  selections  of  experts. 
Experts’  and  novices’  confidence  ratings  indicated  that  both  groups  were  able  to  predict 
accurately  the  probable  correctness  of  their  selections.  These  results  show  that  novices  studying 
hypertext  may  select  information  on  the  basis  of  superficial  text  features,  while  experts  can  use 
prior  knowledge  to  identify  goal-related  units.  The  fact  that  novices  had  less  confidence  in  their 
incorrect  selections  suggests  that  they  may  readily  correct  these  selections  with  further  use  of  the 
content  map. 


Introduction 

Instructional  hypertexts  can  potentially  facilitate  study  efficiency  by  permitting  students  to  determine  the 
order  and  content  of  their  learning  material  according  to  their  individual  needs.  However,  efficient  learning  from 
hypertext  depends  on  how  easily  readers  can  locate  the  information  required  for  the  study  goal.  To  aid 
information  location,  many  large  instructional  hypertexts  provide  an  overview  of  the  text  through  the  use  of 
content  maps  (Dee-Lucas  & Larkin,  1995).  Content  maps  display  titles  indicating  the  subject  matter  of  the 
hypertext  units  (i.e.,  brief  segments  of  information  in  the  hypertext).  Students  select  units  to  read  by  clicking  on 
the  titles  on  the  map.  This  experiment  investigated  how  learners'  expertise  in  the  hypertext  content  domain 
influences  their  unit  selections  from  a content  map.  It  compared  novices'  and  experts'  ability  to  identify  hypertext 
units  relevant  for  answering  short-answer  questions  when  the  wording  of  the  questions  varied  in  its  similarity  to 
the  unit  titles  on  the  map. 

Subject  matter  expertise  influences  the  cues  readers  use  to  guide  text  processing  when  studying  traditional 
texts.  For  example,  novices  studying  science  texts  consider  the  same  information  to  be  more  important  when  it  is 
expressed  in  the  form  of  an  equation  as  opposed  to  a written  sentence,  while  experts  are  not  influenced  by 
information  form  in  their  importance  judgements  (Dee-Lucas  & Larkin,  1991).  Novice  readers  rely  to  a greater 
degree  than  experts  on  superficial  characteristics  of  the  text  structure  and  content  to  guide  text  study  (Kieras, 
1980,  1985;  Dee-Lucas  & Larkin,  1986,1988a).  Experts  use  their  knowledge  of  the  content  domain  to  develop 
effective  study  strategies  (Alexander,  Kulikowich,  & Jetton,  1994;  Dee-Lucas  & Larkin,  1988b;  Spilich, 
Vesonder,  Chiesi,  & Voss,  1979). 

Traditional  text  research  suggests  that  readers’  subject  matter  expertise  will  influence  how  they  identify  goal- 
relevant units  on  a hypertext  content  map.  The  initial  unit  selections  made  by  novices  may  be  based  on  superficial 
text  features,  such  as  wording  similarities  between  the  learning  goals  and  unit  titles.  If  so,  the  accuracy  of  their 
selections  would  depend  on  how  well  the  unit  titles  matched  the  learning  goals.  Because  experts  can  use  their 
knowledge  of  the  subject  matter  to  guide  unit  selections,  the  accuracy  of  their  selections  may  be  less  dependent  on 
the  degree  of  similarity  between  the  goals  and  units. 

This  experiment  examined  the  effects  of  wording  overlap  between  unit  titles  and  short-answer  questions  on  the 
units  selected  by  experts  and  novices  from  a hypertext  content  map.  Students  were  given  two  sets  of  questions 
and  were  asked  to  identify  on  a content  map  the  units  that  they  thought  contained  the  answers.  One  question  set 
included  all  or  part  of  the  title  of  the  goal-relevant  unit  within  the  questions;  the  other  set  posed  the  same 
questions  without  the  wording  overlap.  It  was  expected  that  novices  would  look  for  wording  cues  to  select  units, 
and  therefore  would  be  better  able  to  identify  the  correct  units  when  the  titles  were  incorporated  into  the 
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questions.  Experts  were  expected  to  use  their  knowledge  of  the  content  domain  in  selecting  units,  and  thus 
perform  similarly  on  both  question  sets. 

Method 

Materials 

The  content  map  was  taken  from  a hypertext  about  the  scientific  measurement  of  length  and  time  (see  Figure 
1).  Because  the  goal  of  the  experiment  was  to  determine  the  effects  of  expertise  on  unit  selection,  only  the 
hypertext  content  map  was  used.  Prior  research  using  the  complete  hypertext  indicated  that  students  were  able  to 
use  this  content  map  effectively  when  studying  for  a variety  of  learning  goals  (Dee-Lucas,  1996b). 


Algebraic  Consistency 
Symbols 

Figure  1.  Hypertext  content  map. 

A set  of  six  short-answer  “target”  questions  was  developed,  with  two  versions  of  each  question.  In  one 
version,  the  questions  contained  all  or  part  of  the  titles  of  the  units  providing  the  answers  (i.e.,  “Overlap”).  The 
other  version  contained  the  same  questions  without  the  wording  overlap  (i.e.,  “No  Overlap”).  Some  of  the 
questions  are  shown  below. 


Unit  Title 

Tareet  Question  without  Overlan 

Tareet  Question  with  Overlap 

Significant  Figures 

How  much  variation  would  be  expected 
in  a measurement  written  as  1.524  m? 

How  much  variation  would  be  expected 
in  a measurement  written  with  4 
significant  figures  (for  example,  1.524  m)? 

Unit  Consistency 

What  makes  this  equation  obviously  What  inconsistency  makes  this  equation 

false:  x m = y ft/s  (where  x and  y obviously  false:  x m = y ft/s  (where  x and 

are  numbers)?  y are  numbers)? 

Standards  & Units 

What  does  "SI"  stand  for? 

In  specifying  standards  and  units,  what  does 
"SI"  stand  for? 

In  addition  to  the  six  target  questions,  three  “distracter”  questions  were  included  that  did  not  have  the  same 
pattern  of  wording  differences  as  the  target  questions.  Both  versions  of  the  distracter  questions  contained  wording 
suggesting  specific  units  as  potentially  relevant.  The  No  Overlap  distracters  signaled  an  incorrect  unit  as  relevant; 
the  corresponding  Overlap  questions  signaled  the  both  the  incorrect  unit  and  the  correct  unit  as  possibly  providing 
the  answer  to  the  question.  Two  versions  of  a distracter  question  are  provided  below. 

Unit  Titles  Distracter  Question  without  Overlap  Distracter  Question  with  Overlap 

Precise  Definition  of  What  is  defined  as  the  distance  traveled  by  What  unit  of  length  is  defined  as  the 
distance 

Length  (incorrect)  light  in  a vacuum  during  a specific  period  traveled  by  light  in  a vacuum  during  a 
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of  time? 


specific  period  of  time? 


Units  of  Length 
(correct) 

In  the  No  Overlap  distracter,  the  phrasing  “defined  as  the  distance  traveled”  suggests  that  the  unit  “Precise 
Definition  of  Length”  might  provide  the  answer.  The  parallel  Overlap  version  directly  signals  “Units  of  Length” 
as  important  by  including  this  title  in  the  question,  but  also  includes  the  wording  indirectly  suggesting  that 
“Precise  Definition  of  Length”  might  be  relevant.  Thus  the  Overlap  versions  of  the  distracter  questions  suggested 
more  than  one  unit  as  the  potential  source  for  the  answer  to  the  question.  The  distracter  questions  were  included 
to  help  ensure  that  readers  did  not  detect  a pattern  between  the  two  question  sets. 


Procedure  & Participants 

The  participants  in  the  experiment  were  16  physics  graduate  students  (experts)  and  16  history  graduate 
students  (novices).  The  study  was  conducted  as  a paper- and-pencil  task  in  order  to  allow  greater  flexibility  in  data 
collection.  Participants  were  given  the  content  map  and  one  of  the  question  sets.  They  were  instructed  to  write 
down  the  title  of  the  unit  that  they  thought  contained  the  answer  to  each  question,  and  to  indicate  their  confidence 
in  their  response  using  a 1 (very  sure)  to  5 (just  guessing)  rating.  They  were  told  that  they  could  use  the  same  title 
for  multiple  questions. 

There  were  two  sessions.  Participants  received  the  No  Overlap  questions  in  the  first  session,  and  the  Overlap 
questions  in  the  second  session.  The  questions  were  given  in  this  sequence  to  examine  the  extent  to  which 
readers'  ability  to  identify  question-related  units  improved  when  portions  of  the  unit  titles  were  incorporated  in  the 
questions.  The  second  session  was  conducted  at  least  5 days  after  the  first  to  minimize  the  effects  of  completing 
the  first  question  set  on  responses  for  the  second.  Readers’  did  not  have  their  responses  to  the  No  Overlap 
questions  available  during  the  second  session. 


Results 

All  data  were  analyzed  with  an  analysis  of  variance  including  expertise  (expert  vs.  novice)  as  a between- 
subjects  variable  and  wording  overlap  (overlap  vs.  no  overlap)  as  a within-subjects  variable. 


Ability  to  Identify  Question-Related  Units 

To  assess  readers’  ability  to  identify  question- related  units,  the  proportion  of  correct  unit  selections  for  the 
target  questions  was  analyzed.  The  results  indicated  significant  main  effects  due  to  expertise  and  wording 
overlap,  and  a significant  interaction  between  these  two  variables  (see  Figure  2). 
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Figure  2.  Mean  proportion  of  correct  unit  selections  by  each  group  for  each  question  set. 

Experts  made  more  correct  selections  than  novices,  and  both  experts  and  novices  performed  better  on  the 
Overlap  than  the  No  Overlap  questions.  However,  the  improvement  in  performance  on  the  Overlap  questions  was 
greater  for  novices  than  experts.  These  results  indicate  that  both  groups  used  wording  overlap  as  a cue  for 
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information  location,  but  experts  did  not  require  this  cue  to  the  same  extent  as  novices.  Additionally,  experts 
performed  significantly  better  than  novices  on  the  Overlap  questions,  indicating  that  subject  matter  expertise 
provided  an  additional  advantage  beyond  the  wording  cues  for  identifying  goal-related  units. 

The  unit  selections  for  the  three  distracter  questions  were  also  analyzed.  The  results  indicated  only  a 
significant  effect  of  wording  overlap,  with  both  groups  making  more  correct  selections  for  the  Overlap  distracters 
(a  mean  of  .48  correct)  than  the  No  Overlap  distracters  (a  mean  of  .18  correct).  This  indicates  that  readers  with 
the  Overlap  questions  selected  on  the  basis  of  title  overlap  as  opposed  to  the  indirect  wording  cues  (signaling  the 
incorrect  unit).  Although  there  were  few  questions  in  this  set,  the  lack  of  effects  due  to  expertise  suggest  that  both 
groups  experienced  a similar  degree  of  difficulty  when  the  questions  contained  misleading  or  multiple  selection 
cues.  These  results  suggest  that  experts  may  be  influenced  by  wording  cues  when  plausible  alternative  units  are 
suggested  by  different  information  sources  (i.e.,  prior  knowledge,  indirect  wording  cues,  wording  overlap). 


Changes  in  Unit  Selection 

To  further  examine  differences  in  the  use  of  wording  cues,  the  proportion  of  unit  selections  that  differed  for 
the  No  Overlap  and  Overlap  versions  of  the  target  questions  was  analyzed.  The  results  indicated  a significant 
effect  of  expertise,  with  novices  changing  a mean  of  .63  of  their  initial  selections  and  experts  changing  a mean  of 
.20.  This  result  reflects  differences  between  groups  in  their  ability  to  identify  question-related  units,  and  further 
indicates  that  novices  used  wording  cues  in  making  their  unit  selections. 

A parallel  analysis  of  the  three  distracter  questions  indicated  a trend  towards  more  changes  by  novices  which 
did  not  reach  significance  (p<.077).  Novices  changed  a mean  of  ,60  of  their  initial  selections,  while  experts 
changed  a mean  of  .44.  The  lack  of  a significant  effect  of  expertise  again  suggests  that  experts  had  more 
difficulty  with  the  distracter  than  target  questions. 


Confldence  in  Unit  Selection 

To  determine  if  experts  and  novices  differed  in  their  ability  to  predict  the  accuracy  of  their  selections,  the 
mean  confidence  ratings  assigned  to  correct  and  incorrect  responses  by  each  reader  were  analyzed.  Because 
readers  made  more  errors  on  the  distracter  questions,  both  target  and  distracter  questions  were  included  in  the 
analysis  to  provide  more  data  for  assessing  the  relationship  between  confidence  and  response  accuracy.  The 
results  indicated  significant  main  effects  of  expertise,  wording  overlap,  and  accuracy,  and  an  interaction  between 
expertise  and  wording  overlap  (see  Figure  3). 
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Figure  3.  Mean  confidence  ratings  given  to  each  question  set  by  each  group. 


Both  groups  of  readers  had  greater  confidence  in  correct  selections,  and  experts  were  more  confident  in  their 
responses  overall  than  novices.  However,  the  interaction  showed  that  while  experts  were  equally  confident  in 
their  responses  to  both  versions  of  the  questions,  novices  were  more  confident  of  their  selections  for  the  Overlap 
questions.  This  interaction  is  consistent  with  differences  between  experts  and  novices  in  their  ability  to  identify 
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the  correct  units.  This  again  suggests  that  experts  relied  on  their  prior  knowledge  in  selecting  units,  in  that  their 
confidence  ratings  were  independent  of  question  wording.  Novices,  on  the  other  hand,  were  more  confident  when 
the  questions  included  wording  overlap,  indicating  that  they  used  wording  cues  in  selecting  units. 


Discussion 

Goal-directed  learning  from  hypertext  depends  on  the  ability  to  select  and  organize  the  relevant  text  content. 
The  results  of  the  current  research  indicate  that  familiarity  with  the  text  subject  matter  influences  how  students 
identify  goal-related  units  on  a content  map.  Novices  used  wording  similarities  between  questions  and  unit  titles 
to  guide  unit  selections,  and  had  difficulty  identifying  goal-related  units  without  such  cues.  Experts  were  also 
influenced  by  question  wording,  but  to  a much  lesser  degree  than  novices.  This  suggests  that  experts  were  able  to 
use  their  prior  knowledge  to  determine  which  units  contained  the  information  needed  for  the  learning  goals. 

The  results  of  the  current  research  indicate  that  expertise  is  more  likely  to  influence  hypertext  study  when  the 
hypertext  lacks  structural  and/or  content-level  support  for  selecting  goal-appropriate  content  (Dee-Lucas  & 
Larkin,  1995).  Such  support  would  probably  not  be  important  for  experts,  but  would  be  helpful  to  novices  in 
locating  the  information  they  needed.  McDonald  and  Stevenson  (1998)  reported  similar  findings  in  their 
comparisons  of  more-  and  less-knowledgeable  readers  studying  differently  structured  hypertext.  They  found  that 
prior  knowledge  facilitated  information  location  when  the  hypertext  lacked  cues  as  to  the  content  structure  of  the 
subject  matter,  indicating  that  the  knowledgeable  readers  were  able  to  use  prior  knowledge  to  structure  the 
content.  Research  by  Dillon  (1991)  also  suggests  that  subject-matter  experts  have  an  advantage  in  being  able  to 
rely  on  familiar  domain-specific  organizations  to  guide  electronic  text  study. 

These  findings  are  also  consistent  with  research  showing  that  novice  learners  use  hypertext  more  effectively 
when  the  access  facility  provides  detailed  information  about  the  text  content.  Dee-Lucas  (1995,  1996a)  compared 
novice  learners’  ability  to  complete  a variety  of  learning  goals  when  the  hypertext  content  was  divided  into  many 
small  units  (with  relatively  specific  unit  titles)  and  fewer  larger  units  (with  more  general  unit  titles).  The  content 
map  displaying  many  small  units  facilitated  information  location  for  a greater  variety  of  goals.  This  finding  may 
have  been  due  to  differences  in  the  specificity  of  the  unit  titles.  The  more  specific  titles  of  the  smaller  units  may 
have  provided  a closer  match  to  the  wording  of  the  learning  goals  than  the  more  general  unit  titles.  The  general 
titles  may  have  not  provided  enough  wording  overlap  to  prompt  selection  of  the  relevant  units  for  some  goals. 
The  current  research  indicates  that  the  Dee-Lucas  findings  would  not  necessarily  generalize  to  experts  because 
they  do  not  rely  as  extensively  on  the  match  between  unit  titles  and  learning  goals  in  selecting  units. 

Although  novices  in  the  current  research  had  more  difficulty  selecting  goal-related  units,  their  confidence 
ratings  indicate  that  they  were  able  to  assess  the  potential  accuracy  of  their  selections.  There  was  no  difference 
due  to  question  wording  in  the  confidence  ratings  of  experts,  but  novices  were  more  confident  in  their  selections 
for  the  Overlap  questions.  Both  novices  and  experts  had  more  confidence  in  correct  than  incorrect  selections. 

The  fact  that  novices  recognized  when  their  selections  were  not  well-founded  suggests  that  they  may  be  able 
to  readily  correct  their  selection  strategy  with  repeated  use  of  a hypertext.  Learning  from  hypertext  requires  the 
student  to  make  decisions  and  predictions  about  the  relationship  between  the  information  in  the  text  and  the 
requirements  of  the  learning  goal  (Mohageg,  1992;  Wenger  & Payne,  1996).  If  novices  recognize  when  their 
selections  are  tentative,  they  may  be  more  likely  to  attend  to  content  cues  within  the  units  which  would  enable 
them  to  subsequently  make  correct  choices,  increasing  their  study  efficiency  over  time. 

The  findings  from  this  research  indicate  that  the  effects  of  expertise  on  hypertext  study  are  mediated  by  the 
relationship  between  the  study  goals  and  the  information  access  facility.  Subject  matter  expertise  can  increase 
study  efficiency  when  the  access  facility  does  not  provide  a clear  indication  of  the  location  of  goal-related  content. 
These  findings  emphasize  the  central  role  of  the  access  facility  in  determining  the  effectiveness  of  hypertext  for 
novice  learners  who  lack  prior  knowledge  to  guide  text  study.  From  a methodological  viewpoint,  the  findings 
indicate  that  research  investigating  variables  influencing  information  location  (e.g.,  hypertext  structure,  learners’ 
background,  content  domain,  etc.)  needs  to  take  into  account  the  relationship  between  the  study  goals  and  the 
access  facility  provided  in  the  experiment  in  interpreting  the  effects  of  other  variables. 
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Abstract:  This  paper  reports  on  the  findings  of  an  evaluation  conducted  on  Web-based  course 
authoring  tools.  A comprehensive  framework  encompassing  pedagogical,  institutional,  and  ethical 
aspects  among  others,  was  used  to  examine  the  instructional  effectiveness  of  the  online  learning 
environment  by  applying  the  framework  to  completed  courses  rather  than  assessing  the  specific 
features  of  the  tool  under  investigation.  A review  of  these  courses  and  the  attributes  of  the  tools 
used  to  deliver  them  revealed  several  conclusions.  In  order  for  Web-based  instruction  to  be 
effective  it  must  be  pedagogical ly  driven,  dynamically  designed,  interaction  oriented,  and  content 
specific.  Focus  should  be  placed  on  designing  a pedagogical  approach  appropriate  for  the  content, 
inclusion  of  organization  and  interaction  strategies  that  enhance  the  student's  processing  of  the 
information,  and  integration  of  the  medium's  attributes  to  support  the  designated  goals  and 
objectives  of  the  course. 

Introduction 

The  move  to  online  technologies  is  an  evolving  and  dynamic  process  of  investigation  particularly  in  the 
area  of  examining  the  ability  of  such  technologies  to  improve  the  teaching  and  learning  process.  In  the  case  of  Web- 
based  course  authoring  tools  (also  known  as  Internet-based  training  tools),  several  evaluation  frameworks  or 
guidelines  have  been  used  to  determine  the  degree  of  success  of  those  tools  in  designing,  delivering,  and  managing 
instruction.  For  example,  Hansen  and  Frick  (1997)  suggest  four  areas  that  should  be  addressed  by  a Web-based 
course  authoring  tool  for  it  to  become  a standard  for  the  development  of  Web-Based  Instruction  (WBI):  presenting 
information,  providing  human  interaction,  assessment  of  learning,  and  course  management.  Another  example  is  the 
framework  used  by  PC  Week  Labs  (August,  97)  in  conjunction  with  its  corporate  partner,  Wisconsin  Technical 
College  System,  to  evaluate  courseware  authoring  systems  for  Web-based  training.  The  products  used  to  create  the 
training  programs  and  the  resulting  programs  themselves  were  judged  on  conformity  to  the  corporate  and  education 
landscape,  exploitation  of  the  Intemet/intranet,  manageability,  flexibility,  and  ease  of  use.  In  another  effort  aimed  at 
helping  educators  and  administrators  select  new  online  delivery  methods,  Landon  (1998)  conducted  an  analysis  of 
online  delivery  software  in  which  he  describes  and  compares  these  applications  by  focusing  on  their  technical 
specifications,  instructional  design  values,  media  capabilities,  features/tools,  ease  of  use,  accessibility  to  persons 
with  disabilities,  and  potential  for  collaboration  and  connectivity.  Although  useful,  these  frameworks  (and  others) 
have  focused  on  the  technological  aspects  of  the  features  that  enable  the  development  of  online  instruction  (Dabbagh 
& Schmitt,  98)  and  have  adopted  a piecemeal  approach  to  the  evaluation  process  which  makes  it  difficult  to  form  an 
authentic  representation  of  the  learning  environment  generated.  As  asserted  by  the  Australian  National  Training 
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Authority  (ANTA),  Web-based  authoring  tools  seem  to  be  adhering  to  a common  pattern  of  development  and 
analysis: 


An  emerging  pattern  for  the  development  of  online  courses  is  that  of  a staged  and 
incremental  process.  In  the  first  round  of  investigations,  the  focus  for  developers  is  more 
on  understanding  the  potential  of  the  technology  and  exploring  its  features  (March,  98). 

The  purpose  of  this  paper  is  to  use  a more  comprehensive  framework  to  evaluate  Web-based  course 
authoring  tools  encompassing  pedagogical,  institutional,  and  ethical  aspects  among  others,  and  to  examine  the 
instructional  effectiveness  of  the  online  learning  environment  by  applying  the  framework  to  completed  courses 
rather  than  assessing  the  specific  features  of  the  tool  under  investigation.  The  term  used  to  describe  such  courses  in 
this  discussion  will  be  Web-Based  Training  (WBT). 

The  Framework 

The  proposed  framework  known  as  The  Framework  for  Web-Based  Learning  (thereafter  referred  to  as 
WBL  Framework),  was  developed  by  Khan  (1998)  and  consists  of  eight  dimensions:  pedagogical,  technological, 
institutional,  ethical,  interface  design,  resource  support,  online  support,  and  course  management.  Each  dimension 
has  various  items  addressing  its  constituents  in  relation  to  Web-based  learning  environments.  For  example,  the 
pedagogical  dimension  addresses  issues  such  as  instructional  goals  and  objectives,  the  overall  design  approach  of 
the  course,  content  sequencing  and  organization,  instructional  methods  and  strategies,  the  instructional  medium  (the 
Web  in  this  case)  and  the  extent  to  which  the  course  uses  its  attributes,  accuracy  of  subject  matter,  and  learner 
assessment.  For  more  details  about  this  framework  and  its  dimensions  see: 
http://www.gwu.edu/~etlkhan/keynote/dimensions/frameworkl.html. 

There  are  three  issues  to  consider  when  attempting  to  answer  this  question.  First  is  the  issue  of  the 
transformation  that  traditional  courses  experience  when  redesigned  for  Web  delivery.  Typically  such  courses 
undergo  a "pedagogical  reengineering"  (Collis,  1997)  because  of  some  of  the  inherent  features  of  the  Web  that  are 
integral  to  the  design  of  WBI  (Dabbagh  & Schmitt,  98).  Second  is  the  purpose  of  the  Web-based  course  in  the 
learning  environment.  Is  the  course  a supplement  to  on-campus/on-site  instruction  or  is  it  intended  as  a distance 
learning  alternative  in  which  the  instruction,  interactions,  and  feedback  are  delivered  via  the  Web?  Third  is  the  type 
of  authoring  tool  used  to  develop  the  course  for  Web-delivery  and  the  features  it  affords  the  developer.  For  example, 
integrated  application  suites  (authoring  tools)  refer  to  applications  that  provide  specific  tools  designed  for  each  of 
three  user  groups:  learners,  instructors,  and  technical  administrators  whereas  component  applications  refer  to  more 
specialized  applications  that  generally  provide  few  instructor  tools  (Center  for  Curriculum  Transfer  & Technology, 
http://www.ctt.bc.ca/landonline).  We  will  address  these  issues  by  applying  the  WBL  Framework  to  six  WBT  sites 
that  have  been  developed  using  six  different  Web-based  course  authoring  tools  (see  table  1).  We  will  then  use  the 
results  of  this  evaluation  to  determine  the  extent  to  which  the  technology  of  delivery,  in  this  case  the  Web-based 
authoring  tools,  can  shape  the  pedagogy  of  courses  and  assist  course  developers  in  focusing  on  instructional 
considerations  when  capitalizing  on  the  attributes  of  the  medium. 

The  Courses 

Selecting  the  courses  was  not  an  easy  task.  We  searched  for  complete  courses  that  demonstrated  a full  and 
authentic  use  of  the  application.  All  of  the  product  Web  sites  contained  at  least  one  demonstration  course  that  could 
be  accessed  by  registering  and  receiving  a password  or  a log-on  ID.  These  product  site  courses,  however,  varied  in 
their  ability  to  demonstrate  all  of  the  components  of  the  authoring  tool.  A few  of  the  courses  were  designed  and 
taught  by  instructors  for  accredited  programs,  but  no  longer  included  the  student  exchanges,  student  projects,  tests, 
and  complete  content.  Other  courses  were  designed  specifically  to  demonstrate  the  product  and  therefore  were  not 
actually  used  by  instructors  or  students  in  a learning  context.  In  the  quest  to  obtain  courses  that  reflected  a more 
realistic  use  of  the  Web-based  course  authoring  tool,  we  were  able  to  obtain  access  to  a few  uncomipted  accredited 
courses  which  we  felt  were  sufficiently  representative  of  the  authoring  tool  and  therefore  befitting  the  purpose  of 
this  chapter. 

The  Evaluation 
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Due  to  the  comprehensiveness  of  Khan's  framework,  which  includes  eight  dimensions  and  more  than  sixty 
sub-components,  a simplified  strategy  was  selected  for  our  informal  evaluation  of  these  courses.  Mapping  the 
framework  to  the  various  courses  revealed  specific  strengths  and  weaknesses  of  each  course  that  could  be  aligned 
with  many  of  the  framework's  components.  To  avoid  unnecessary  complexity,  we  chose  to  focus  on  the  major 
course  attributes  in  relation  to  the  framework.  The  major  attributes  of  each  course  which  emerged,  both  positive  and 
negative  dictated  our  interpretive  evaluative  approach  using  the  WBL  Framework. 

Marketing  Certiflcate  in  Small  Business  Management 

This  comprehensive  demonstration  course  highlights  many  features  of  WebCT.  Most  notable  is  the  student 
guidance  for  on-line  learning  and  navigating  the  course  as  well  as  excellent  content  organization.  Developed  in  an 
on-line  mode  format  (Brenda,  is  this  redundant?),  this  course  provides  the  student  with  extensive  explanation  of  the 
purpose,  objectives  and  activities  involved  in  addition  to  directing  students  to  recommended  approaches  for  learning 
the  material  in  this  delivery  format.  The  instructional  content  is  thoroughly  presented  in  a program-centered  manner 
(WBL  framework)  providing  the  student  with  clear  and  concise  partitioned  segments  of  content.  There  is  evidence 
that  the  instruction  was  originally  interspersed  with  interactive  elements  such  as  forms,  and  assessment  elements 
such  as  on-line  quizzes. 

On  first  glance,  the  course  seems  to  represent  a more  objectivist  pedagogy  (WBL  Framework)  primarily  in 
delivering  content  but  inside  collaboration  is  also  included  with  discussion  email  and  on-line  chat  mechanisms  as 
well  as  conferencing  bulletin  board  sessions  that  automatically  create  a conference  topic  relevant  to  the  current 
content  section  chosen.  Administration  and  tracking  mechanisms  are  also  included  providing  some  metacognitive 
suppport  for  the  student  in  resuming  their  place  in  the  course  and  tracking  progress  through  presenting  the  number 
and  history  of  pages  viewed  and  section  printing  capabilities  (WBL  Framework),  Assessment  of  student  learning 
through  on-line  quizzes  and  case  study  responses  are  evident;  however,  it  is  not  clear  how  the  students  receive 
feedback  on  their  responses.  Efforts  for  formative  evaluation  on  the  course  are  elicited  through  various  methods  of 
response  including  email  and  forms. 

SUNY  TopClass  Virtual  Lounge  & HelpDesk 

WBT  Systems,  the  distributor  of  TopClass,  recommended  a course  designed  and  conducted  by  William 
Graziadei  of  the  State  University  of  New  York,  Plattsburgh,  as  a demonstration  of  their  Web-based  course  delivery 
system.  In  spite  of  its  title,  "SUNY  TopClass  Virtual  Lounge  & HelpDesk  (STCVL),"  this  is  a complete  course. 
STCVL  is  meant  to  provide  a self-paced  guided  training  for  SUNY  students  and  instructors  who  will  participate  in 
online  teaching  and  learning  using  the  TopClass  application.  The  course  overview 

states  that  Professor  Graziadei  hopes  the  discussion  portion  of  the  course  will  become  a forum  for  faculty  and 
students  of  different  disciplines  to  share  their  experiences  with  online  education. 

The  course  overview  encourages  contributions  through  email  (which  is  always  available  on  the  tool  bar  at 
the  bottom  of  each  screen),  and  discussions  can  be  accessed  through  the  course  homepage,  however,  group 
interaction  and  collaboration  (WBL  Framework)  are  not  an  integral  part  of  the  course  design.  Only  one  folder,  the 
"live  interactive  tutorial,"  appears  to  provide  interaction  between  student  and  instructor.  Unfortunately,  access  to  this 
tutorial  is  restricted  to  the  SUNY  community.  Another  weakness  of  the  course  is  the  absence  of  a search  engine, 
index,  or  glossary  to  help  students  who  return  with  specific  questions,  sift  through  the  profusion  of  information 
presented  under  each  topic/unit.  Also,  considering  the  amount  of  information  in  some  folders,  the  topic  of  Online 
Education  could  have  provided  many  more  outside  links  to  resources  on  distance  learning  and  instructional  design 
strategies  relevant  to  this  delivery  medium. 

Effective  Business  Writing  English  1007 

The  purpose  of  this  course  is  to  be  delivered  half  the  time  in  a face  to  face  format  and  the  other  half  fully 
online.  This  is  known  as  the  mixed  mode  compared  to  an  adjunct  or  an  online  mode.  In  an  adjunct  mode,  the  online 
portion  of  the  course  is  intended  as  a supplement  to  enhance  traditional  classroom  instruction.  The  online  mode  is 
when  the  network  serves  as  the  primary  environment  for  course  discussions,  assignments  and  interactions.  These 
modes  are  referred  to  as  'modes  of  teaching'  by  Virtual-U;  the  tool  used  to  develop  this  course,  and  are  offered  as 
guidelines  for  instructors  in  the  Instructor  Tools  and  Support  component  of  the  tool. 
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The  course  begins  with  four  options  to  choose  from;  course  syllabus,  course  overview,  course  calendar,  and 
glossary.  These  options  are  not  standard.  They  can  be  customized  to  reflect  different  choices  based  on  the  content  of 
the  course  thus  providing  structural  flexibility  (WBL  Framework).  The  course  syllabus  is  an  advance  organizer  and 
contains  hypertext  links  to  the  course  topics,  units,  assignments,  tests,  activities,  resources,  and  online  conferences 
and  discussions.  The  links  encompass  a wide  variety  of  electronic  media  utilizing  virtually  every  attribute  of  the 
Internet  and  the  World  Wide  Web.  They  include  text  files,  tutorials,  audio  files,  video  clips,  animations,  graphics, 
online  resources  (with  built-in  search  engines),  and  PowerPoint  files  to  present  course  content  and  learning 
activities,  as  well  as,  synchronous  and  asynchronous  communication  forums  ranging  from  email  to  a virtual  cafe  for 
socializing.  Although  intricately  rich  in  multimodal  representations  and  interaction  levels,  the  variety  is 
overwhelming  and  could  present  a level  of  complexity  for  first  time  users.  However,  it  is  evident  that  online  support, 
resource  support,  and  inside  and  outside  collaboration  is  demonstrated  through  the  use  of  these  features  (WBL 
Framework). 

With  each  of  these  views  there  are  Use  features  that  can  be  accessed  by  the  student  and  the  instructor  to 
perform  certain  activities.  This  Use  feature  promotes  a student-centered  approach  to  learning,  which  is  one  of  the 
underlying  instructional  principles  that  Virtual-U  supports.  The  workspace  interface  is  a good  anchor  point  for  the 
user  and  perhaps  should  serve  as  the  primary  navigational  structure  for  the  course.  It  will  improve  the  user 
friendliness  of  the  interface. 

Health  Care  Management 

Health  Care  Management  first  conducted  in  the  fall  of  1997  provides  an  example  of  a course  delivered  via 
Web  Course  in  a Box  (WCB)  authoring  software.  The  course  is  structured  using  face  to  face  class  sessions 
interspersed  with  at  least  five  electronic  seminar  discussions.  This  demonstration  course  highlights  the  main  features 
and  aspects  of  the  WCB  software  but  is  currently  deliberately  restricted  in  content  and  resources  as  well  as  corrupted 
by  multiple  users  experimenting  with  the  authoring  tool.  However,  in  reviewing  the  structure  and  content  that  is 
available,  many  attributes  of  the  course  and  the  software  are  revealed. 

The  primary  strengths  of  this  course  are  the  instructor's  apparent  attempts  at  management  (WBL 
Framework)  and  organization  of  the  content  within  the  software  constraints  in  addition  to  adequately  addressing  the 
on-line  support  issues  (WBL  Framework)  necessary  for  student  participation.  These  factors  have  pedagogical  as 
well  as  interface  usability  implications.  The  electronic  seminars  focus  on  discussions  about  the  off-line  readings. 
These  discussions  are  well  directed  with  specific  objectives  and  clearly  outlined  expectations  established  prior  to 
providing  a link  directly  to  the  conference  on  that  topic.  This  organization  facilitates  the  discussion  of  a specific 
topic  focusing  student  contributions  and  easily  directs  students  to  the  related  conference  session.  Additional 
management  issues  are  addressed  in  the  course  such  as  providing  a help  section  and  learning  links  dealing  with  Web 
development  skills,  however,  the  relationship  of  these  skills  to  the  required  assignments  could  not  be  clearly 
determined.  Inside  collaboration  (WBL  Framework)  between  students  in  the  course  is  fostered  by  WCB's  providing 
space  for  personal  biographies  as  well  as  individual  and  public  class  email  capabilities. 

Hospitality  Industry  Law  and  Liability 

LeamingSpace  featured  Hospitality  Industry  Law  and  Liability,  a course  from  the  University  of  Wisconsin- 
Stout,  Department  of  Hospitality  and  Tourism.  The  course  was  entirely  Web-based  having  no  classroom  component. 
The  class  databases  were  modified  to  remove  copyright  material,  s^tudent  discussions,  and  student  profiles.  Only  a 
few  units  of  the  course  are  shown  in  their  complete  form,  however,  those  sections  demonstrate  both  the  efficient 
ordering  of  content  to  help  the  learner  achieve  the  objectives  (Kemp,  Morrison,  & Ross,  1994)  and  clear 
expectations  of  what  the  student  is  required  to  do  (WBL  Framework).  The  units  featured  in  the  "Schedule"  section 
provide  reading  assignments,  content  and  process  objectives,  the  purpose  of  the  unit,  as  well  as  student  and 
instructor  evaluation  criteria.  The  WBL  Framework  suggests  that  the  course  promote  inside  collaboration  for 
working  on  joint  projects.  The  instructor  of  this  course  requires  collaboration  with  team  members  on  many 
assignments,  such  as  critiquing  case  studies.  Collaboration  is  facilitated  by  the  LeamingSpace  stmcture.  It  provides 
a shared  workspace  for  team  assignments.  Teams  may  choose  to  restrict  access  to  this  space  to  members  only.  A 
message  appears  on  the  assignment  link  to  indicate  whether  the  work  is  "In  Progress,"  "Ready  for  Grading,"  or  ready 
for  "Instructor  Review."  Collaboration  is  further  promoted  through  discussion,  which  may  be  directed  to  the  entire 
class  or  to  team  members. 

Programming  Your  Avilar  VCR 
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This  course  is  intended  as  standalone  instruction.  It  is  self-contained  and  designed  using  tried  and  true 
instructional  design  principles.  The  pedagogical  philosophy  used  in  the  overall  design  of  the  course  actually  shapes 
the  learning  environment  (WBL  Framework).  The  course  starts  out  with  clear  instructional  objectives  and  user 
directions  on  "how  to  take  the  course".  Using  a frame-based  interface,  the  course  is  conveniently  divided  into  six 
modules:  lesson,  exercises,  questions,  review,  resources,  and  assessment.  Buttons  for  each  of  these  modules  remain 
accessible  throughout  the  entire  lesson,  however  they  are  disabled  and  enabled  based  on  where  the  learner  is  in  the 
lesson.  One  must  complete  a lesson  module  first;  then  complete  related  exercises,  answer  questions,  get  feedback, 
review  the  lesson,  and  take  the  assessment.  The  progress  through  the  lesson  is  therefore  dictated  by  the  organization 
of  the  content  (to  a large  extent)  and  proceeds  from  basic  to  more  complex  learning  outcomes.  Although  there  is  not 
much  structural  flexibility  (WBL  Framework),  it  is  possible  to  jump  between  course  modules  using  embedded 
hypertext  or  the  Course  Contents  menu.  It  is  recommended,  however,  that  first  time  learners  complete  the  course  in 
the  intended  sequence.  The  modules  can  later  be  used  as  a reference  or  a job  aid  if  one  forgets  how  to  perform  a 
certain  function. 

The  assessment  and  questions  modules  demonstrate  a variety  of  multiple-choice  question  formats  that  can 
be  used  to  evaluate  learners'  mastery  of  learning  outcomes.  Equally  evident  is  the  feedback  feature  after  each 
response  and  the  instant  grade  reporting  that  follows  an  assessment.  Web  Mentor  supports  the  use  of  email, 
conferencing,  chat  rooms,  application  sharing,  and  other  synchronous  and  asynchronous  interactions  that  foster 
collaboration  and  interactivity.  The  course  content  however  was  not  appropriately  matched  to  the  capabilities  of  the 
delivery  medium  (WBL  Framework)  and  the  use  of  a familiar  methodology  in  designing  instruction  for  a new 
technology  may  have  once  again  contributed  to  the  under-utilization  of  the  tool. 

Discussion  and  Implications 

As  is  often  evident  in  face  to  face  courses,  the  Web-based  courses  demonstrated  a mix  of  methods, 
strategies,  and  pedagogical  approaches.  Various  strengths  and  weaknesses  were  noted  in  each  case  when  mapped 
against  the  WBL  Framework.  Some  courses  adopted  an  objectivist,  straightforward  approach  of  content 
segmentation  and  delivery  for  mastery  (see  Programming  VCR  - Web  Mentor),  while  others  capitalized  more  fully 
on  the  inherent  collaborative  attributes  of  the  Web  in  providing  students  with  shared  workspace  (Hospitality 
Industry  Law  and  Liability  - Learning  Space)  and  a constructivist  student-centered  approach  (Effective  Business 
Writing  - Virtual  U).  Most  presented  a clear,  logical  and  organized  format  (SUNY  Top  Class  Virtual  Lounge  and 
Help  - TopClass)  and  well-directed  objectives  and  expectations  (Health  Care  Management  - WCB)  while  some 
provided  exceptional  guidance  for  learners  with  recommended  approaches  to  the  material  (Marketing  Certificate  in 
Small  Business  Management  - WebCT). 

In  examining  the  courses  and  hence  the  Web-based  authoring  tools,  we  found  that  the  WBL  Framework  is 
an  effective  tool  for  informal  evaluation  and  a good  guiding  mechanism  or  checklist  for  the  development  of  Web- 
based  courses.  Our  position  is  clear  in  attempting  to  promote  a more  comprehensive  evaluation  of  courses  developed 
with  these  authoring  tools  rather  than  examining  the  tool  itself  Educators  do  not  evaluate  a classroom  course  merely 
by  the  technology  of  delivery,  (e.g.  attributes  of  overheads,  LCD  display,  teacher-directed  discussion)  therefore  we 
should  not  discern  the  effectiveness  of  on-line  learning  environments  merely  by  dissecting  the  delivery  tool. 

Our  review  of  these  courses  and  the  attributes  of  the  tools  used  to  deliver  them  revealed  several 
conclusions.  In  order  for  WBI  to  be  effective  it  must  be  pedagogically  driven,  dynamically  designed,  interaction 
oriented,  and  content  specific.  Focus  should  be  placed  on  designing  a pedagogical  approach  appropriate  for  the 
content,  inclusion  of  organization  and  interaction  strategies  that  enhance  the  student's  processing  of  the  information, 
and  integration  of  the  medium's  attributes  to  support  the  designated  goals  and  objectives  of  the  course.  Developers 
of  Web-based  training  and  educational  materials  need  to  place  emphasis  on  these  tasks  and  view  the  technology  in 
relation  to  its  capacity  to  deliver  the  planned  design.  This  focus  is  perhaps  best  promoted  by  ANTA: 

While  we  are  constantly  reminded  that  learning  must  be  developed  around  learning 
needs,  meeting  educational  objectives  and  producing  viable  graduates,  the  lure  of 
'exploring  the  technology'  is  often  at  the  expense  of  equal  investment  in  the  underpinning 
educational  design. 

The  intersection  between  pedagogical  considerations  and  the  attributes  of  the  Web-based  authoring  tools 
may,  in  fact,  yield  the  most  educational  impact.  Our  reviews  of  these  courses  showed  that  attributes  of  the 
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technological  tools  may  also  influence  the  pedagogical  aspects  of  the  course.  This  has  been  previously  demonstrated 
in  a case  study  by  Dabbagh  and  Schmitt  (1998)  whereby  a course  that  was  redesigned  for  Web-delivery  underwent  a 
pedagogical  reengineering  from  a primarily  instructivist  pedagogy  to  a more  constructivist  one  due  to  the  focusing 
of  the  developer  on  the  inherent  attributes  of  the  Web  through  the  use  of  the  authoring  tool.  Web  authoring  tools 
which  include  components  that  facilitate  the  team  process,  such  as  LeamingSpace,  promote  the  inclusion  of 
collaboration  as  an  instructional  strategy.  Therefore,  if  the  content  and  planned  strategy  dictates  a strong  focus  on 
students  working  together  then  taking  advantage  of  the  attributes  of  this  particular  authoring  tool  may  in  fact 
enhance  the  educational  effectiveness  of  the  course. 

Unfortunately  no  one  tool  can  deliver  every  possible  instructional  approach.  The  software  tools  seem  to 
have  some  common  attributes  as  well  as  individual  strengths  and  weaknesses.  Most  current  web-based  course 
authoring  tools  can  also  be  viewed  as  restrictive  in  nature  due  to  the  visual  metaphors  employed  by  each  tool  that 
place  constraints  on  the  organization  of  content  and  the  potential  learning  strategies  employed.  This  factor  certainly 
can  restrict  the  learning  environment  produced  and  may  prompt  the  need  for  web-based  tools  that  are  more  open- 
ended,  comprehensive  and  customizable.  Hedberg  & Harper  make  the  valuable  suggestion  that  new  metaphors  for 
authoring  should  be  developed  to  match  current  theory.  This  notion  seems  a good  start,  however,  a comprehensive 
advisement  mechanism  included  within  Web-based  authoring  tools  providing  guidance  in  the  areas  of  pedagogical 
approach,  instructional  strategy,  and  on-line  support  and  resources,  may  go  further  to 
support  the  developer  who  is  lacking  instructional  design  skills. 
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Abstract  - This  study  investigates  the  effect  of  hypermedia  presentation,  its  activity 
relevance  and  the  interactions  between  the  two  on  peak  experience.  The  results  suggested 
that  the  activity  content  has  major  influences  on  motivation  but  presentation  is  a double- 
edge sword.  Hypermedia  presentation  adds  appeals  to  instructions  that  motivate  students  if 
they  are  used  appropriately.  When  the  content  relevance  is  complicated,  complex 
presentations  can  be  distracting.  Consequently,  hypermedia  elements  should  be  used 
sparingly  at  the  beginning  of  a lesson  when  challenges  are  high  and  students  are  unfamiliar 
with  the  material.  As  the  lesson  progresses  it  could  be  used  gradually  as  the  content 
challenges  are  reduced. 


Introduction 

Hanaffin  and  Dalton  (1993)  proposed  new  technologies  facilitate  instruction  in  six  areas:  adaptability,  realism, 
hypermedia,  open-endedness,  manipulatibility,  and  flexibility.  Technology  brings  realism  to  instructions  with 
greater  breadth  and  depth.  Norman  (1994)  illustrated  the  situation  with  this  example.  A large  screen  with  high- 
quality  sound  improves  the  ability  to  be  captured  by  the  events.  The  relatively  small  screen  and  sound  system  of  the 
typical  home  television  distance  the  viewer  from  the  event.  In  any  environment,  an  event  best  captures  the  attention 
and  improve  concentration  when  the  sensory  experience  is  maximized  and  distraction  minimized.  However, 
Moore,  Burton  and  Meyer  (1996)  cited  that  while  much  of  the  evidence  from  research  supports  multiple-channel 
design  to  provide  additional  reinforcement,  the  overall  evidence  on  its  effectiveness  is  inconclusive.  They  proposed 
that  human  information  processing  has  its  limited  and  is  subjected  to  overload.  This  proposal  is  supported  by  the 
cognitive  load  theory:  our  working  memory  limits  our  learning  capacity.  The  prescription  of  instruction  requires 
careful  analysis  to  consider  the  memory  load  implication  of  the  different  combination  of  content  and  instruction, 
sequencing  and  pacing  (Sweller  & Chandler,  1994).  Unfortunately,  the  prescription  is  mainly  related  to 
instructional  contents.  The  impact  of  hypermedia  presentation  on  learning  and  its  interaction  with  contents  are 
largely  unexplored. 

The  Internet  signifies  different  things  to  different  people.  While  some  often  find  it  a frustration,  others  view  its  as 
an  end  in  itself  They  report  a great  sense  of  involvement  and  excitement.  Working  with  new  technology  such  as 
the  Internet  appears  to  induce  flow  experience  (Miller,  1996;  Norman,  1994).  Flow  is  an  optimal  state  described  by 
Csikszentmihalyi  (1975,  1982,  1985,  1988,  1990,  1994,1997).  During  flow,  people  become  so  intense  that  nothing 
seems  to  matter.  The  experience  is  so  enjoyable  that  they  will  do  it  for  its  own  sake.  When  people  reflect  on  how  it 
felt  when  they  are  in  flow,  they  mention  these  aspects:  (a)  sensing  that  one's  skills  are  balanced  challenge;  (b) 
engaging  in  a goal-directed  activity;  (c)  receiving  clear  feedback;  (d)  feeling  in  control;  (e)  intensifying 
concentration,  with  a sense  of  (f)  merging  action  and  awareness;  (h)  disappearing  of  self-consciousness;  and  (h) 
distorting  sense  of  time;  and  (i)  experiencing  great  gratification  that  the  activity  is  intrinsically  rewarding.  Flow 
represents  a descriptive  dimension  that  signifies  some  purer  instances  of  intrinsic  motivation.  When  highly 
intrinsically  motivated,  we  become  extremely  aroused  and  experienced  such  emotion  as  flows  (Deci  & Ryan,  1985). 
As  indicated  by  flow  theorists,  flow  experience  is  affected  by  contextual  factors,  including  the  task  structure  and 
quality  of  presented  information.  This  study  was  designed  to  investigate  how  task  relevance  and  quality  of 
presented  information  influent  flow  experience  in  instructional  activities. 

Perelman  (1992)  predicted  demise  of  the  traditional  educational  system.  A new  wave  of  knowledge  technology,  the 
hyperleaming  technology,  is  coming.  It  enables  anyone  to  learn  anything,  anywhere,  anytime,  with  same 
effectiveness  as  classroom  instructions.  While  business  is  relying  less  on  traditional  instruction,  students  from 
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kindergarten  through  university  are  using  technology  as  supplemental,  not  substitutes  for  formal  classroom. 
Although  the  Internet  has  a limited  reach  today,  it  is  growing  exponentially.  Statistics  show  that  by  the  year  2000, 
95%  of  public  school  will  be  connected  to  the  “Information  Superhighway".  Despite  the  great  hype  and  promise, 
and  its  ability  to  connect  us  to  a vast  amount  of  information  in  text,  picture,  animation  and  sound,  this  “great 
technology  hope’’  also  gives  us  a sense  of  deja  vu.  Cuban's  (1986)  examination  into  the  optimism,  enthusiasm, 
hesitancy  and  frustrations  of  integrating  technology  and  education  revealed  the  great  complexity  of  the  issue. 
Unfortunately,  the  educational  system  is  propelled  into  the  "Brave  New  World"  at  great  momentum,  even  if  we  are 
not  quite  ready,  and  its  impacts  are  far  from  certain.  In  this  age  of  expectation  and  uncertainty,  the  effect  of  new 
technology  on  learning  is  a critical  research  issue. 

Methodology 

Four  groups  of  20  students  were  randomly  selected,  then  randomly  assigned  to  a treatment.  Searching  for 
information  is  motivating  because  of  the  perceived  need  for  the  information  (Krikelas,  1 983).  By  seeking  research 
information  or  browsing  on  irrelevant  material,  we  change  the  motivation  level.  Embellishments  increase  novelty 
and  complexity  that  positively  affect  motivation  (Lepper  & Cordova,  1992).  Using  hypermedia  or  traditional 
computer  platforms,  we  also  change  the  motivation  level.  With  a 2x2  design,  the  two  levels  of  content  relevance 
and  two  levels  of  presentation  quality  are  as  the  following: 

1 . Reading  MS  Word  - students  browsed  over  a text  file  about  dinosaurs.  The  tasks  were  imposed  not 
negotiated,  and  they  were  not  told  about  the  purpose  of  the  activity. 

2.  Reading  CD-ROM  - students  browsed  on  a CD-ROM  on  dinosaurs  that  contained  image,  audio,  video 
coupled  with  a hypertext  organization  for  non-linear  navigation. 

3.  Searching  on  ERIC  - students  searched  the  ERIC  via  remoter  connection  with  a monochrome  text- 
based  system,  where  they  searched  for  material  on  a class  project. 

4.  Searching  the  Web  - students  searched  for  material  on  the  Web  using  a Netscape  browser  with  a list 
of  sites  pertinent  to  their  course  project. 

The  Flow  State  Scale  (FSS)  was  developed  to  assess  flow  during  sport  participation  (Jackson  & Marsh,  1996).  It 
has  36-item,  5-point  Likert-type  scale,  with  4-item  for  each  of  the  9 flow  dimensions.  The  instrument  was  adapted 
for  assessing  flow  in  computer-based  activity  (Appendix).  A score  from  definitely-5,  agree-4,  may  be-3,  disagree-2 
and  definitely  disagree- 1 point  was  assigned.  Strong  agreements  suggested  a high  degree  of  flow,  and  flow 
subscales.  The  proposed  FSS  dimensions,  and  its  construct  validity  have  been  theoretically  supported  by  research 
(Csikzentmihalyi,  1988;  1990;  1992).  Psychometric  assessments  of  the  original  instrument  (Jackson  & Marsh, 
1996),  and  the  adapted  instrument  (Chan,  1998)  reported  alpha  from  .71  to  .93. 

Data  was  collected  during  class  activities  in  spring  1998.  At  the  last  40  minutes  of  a regular  class  session,  students 
were  ushered  into  a PC  lab  with  the  necessary  computer  application  loaded  for  the  randomly  selected  treatment. 
After  30  minutes,  students  were  instructed  to  stop.  They  were  asked  to  reflect  upon  their  experiences  during  the 
session  while  completing  the  questionnaire.  The  procedure  was  repeated  for  the  next  class  section  with  the  next 
randomly  selected  treatment  until  all  treatments  were  administrated.  Univariate  Normal  Plot,  Hartley  Fmax, 
reliability  and  correlation  analyses  were  conducted  to  examine  the  data  characteristics.  Data  were  examined  with 
two-way  ANOVA  on  the  total  and  subscale  scores  of  the  Adapted  FSS.  If  the  interaction  effect  was  significant, 
simple  main  effect  for  the  content  and  presentation  will  also  examined. 

Results 

Out  of  the  total  enrollment  of  97  students,  80  participated  in,  and  all  completed  the  study.  The  sample  comprised 
majority  female  (72%),  between  age  18  to  25  (74%),  Anglo  (73%),  with  39%  Junior  and  24%  Sophomore. 
Cronbach’s  alpha  yielded  .93  for  the  flow  total,  and  .69  to  .88  for  the  subscales.  Descriptive  statistics  reported  the 
highest  score  for  the  flow  total  scale:  CD-ROM  (low  content,  high  presentation,  A/=l  19.70,  iSD=15.50),  ERIC 
(high  content,  low  presentation,  M=114.15,  5'D=19.53),  Internet  (high  content,  high  presentation,  A/=111.55, 
57)=! 8.36),  and  MS  Word  (low  content,  low  presentation,  A/=  105.50,  57)=  14.31).  Normal  distribution  and 
homogeneity  of  variance  held  for  all  treatments.  Pearson  r was  reported  from  .58  to  .89  between  the  flow  subscales 
and  total  scale. 
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The  autoletic  subscale  detected  significant  main  effect  of  presentation  (F(i„76)=27.61,  /?<.03).  Significant 
interaction  effects  were  reported  for  the  flow  total  (F(i„38)=9.07,  /?<.005),  and  subscales:  challenge  (F(i,,38)=14.79, 
p<.0004),  goal  (F(ij6)=9.36, /?<.0031),  feedback  (F(i,76)=7.75, /?<.0068),  awareness  (F(i  76)=1 1.41, /?<.001),  control 
(F(i,76)=4.15, /?<.04),  and  consciousness  (F(i,76)=5.23, /?<.025).  Simple  main  effect  analyses  reported  participants 
using  high  presentation  quality  platform  scored  higher  on  the  total  (F(i,3S)=9.07, /?<.005),  challenge  (F(i,38)=14.79, 
p<.0004),  goal  (F(i,38)=1  1.37,  /?<.002),  feedback  (F(i,38)=5.39,  /?<.02),  awareness  (F(i,38)=21.1 1,  /?<.0001)  and 
consciousness  (F(i,3S)=  10.94,  /?<.002)  subscales  at  the  low  content  relevance  activity,  and  but  no  significant 
presentation  quality  effect  on  flow  and  flow  subscales  at  the  high  content  relevance  activity.  It  also  reported 
participants  in  high  content  relevance  activity  scored  higher  on  the  challenge  (F(i,38)=6.56,  /?<.01),  goal 
(F(i,38)=5.78,/?<.02)  and  feedback  (F(i,38)=5.72,/?<.02)  subscales  while  using  the  low  presentation  quality  platform. 
However,  participants  in  low  content  activity  scored  higher  on  the  challenge  (F(i,38)=7.00,  /;<.01),  awareness 
(F(i,38)=21.73,/?<.0001)  and  consciousness  (F(i  38)=1  1.90,/?<.001)  using  high  presentation  quality  platform. 


Discussion 

The  study  revealed  strong  interaction  between  content  and  presentation  on  flow  experience.  Presentation  quality 
enhanced  flow  {d  = .46),  as  measured  by  challenge,  goal,  feedback,  awareness  and  consciousness  in  low  content 
relevance  activities.  However,  it  did  not  have  significant  effect  on  flow  in  high  content  activities.  Content 
relevance,  on  the  other  hand,  enhanced  flow  experience  {d  = .36)  while  using  low  presentation  quality  platforms. 
However,  it  impeded  flow  experience  {d  = -.43)  while  using  high  presentation  quality  platforms. 

Hypermedia  is  a double-edge  sword  for  instruction.  While  not  necessary  increase  learning,  multimedia  adds 
appeals  to  the  instruction,  motivate  students  and  facilitate  flow,  as  long  as  they  are  used  appropriately.  Multimedia 
should  integrate  gradually  into  a lesson.  When  the  content  relevance  is  high  and  adequate  challenge  is  already 
provided,  high  presentation  quality  is  distracting.  As  instructional  designers,  we  must  integrate  multimedia  into  the 
lesson  carefully,  or  it  has  negative  consequences.  Multimedia  elements  should  be  used  sparsely  in  the  beginning 
when  challenges  are  high  and  students  are  unfamiliar  with  the  material.  The  elements  should  be  incorporated 
gradually  as  lesson  progresses  when  challenges  are  reduced,  at  which  multiple-channel  stimuli  will  no  longer 
impede  performances. 

However,  multimedia  can  be  an  effective  mean  in  alleviating  boredom  for  expert  students.  An  intrinsic  problem  in 
instructional  design  is  its  necessity  to  aim  at  an  average  or  normative  level.  As  Csikszentmihalyi  (1982)  noted, 
easy  material  makes  schooling  a bore  for  many  students.  For  others,  the  difficulty  of  the  material  causes  great 
anxiety.  When  the  content  relevance  is  low  and  inadequate  challenge  is  provided  to  students,  high  presentation 
quality  has  a positive  effect  on  motivation.  Findings  in  this  study  suggested  that  instruction  could  be  designed  to 
present  materials  to  expert  students  with  more  hypermedia  elements.  High  quality  presentation  with  advance 
technology  can  be  employed  to  maintain  students’  motivation. 

Autotelic  or  intrinsically  rewarding  experience  is  the  most  important  dimension  of  flow  and  directly  related  to 
intrinsic  motivation.  Significant  main  effect  of  presentation  in  the  autoletic  subscale  indicated  that  students 
preferred  high  presentation  quality  regardless  of  the  content  relevance  of  the  activity.  The  richness  of  hypermedia 
increases  motivation,  despite  its  negative  impact  when  the  students’  capacity  is  already  stretched  during  high 
content  relevance  activities.  Interestingly,  the  result  indicates  that  students  are  not  very  effective  in  self-monitoring. 
The  excessive  stimuli  during  a high  content  relevance  task  impede  flow.  Nevertheless,  students  still  prefer  the 
multi-channel  presentation,  and  are  unable  to  realize  its  negative  impacts. 

This  study  is  only  a beginning  step  to  the  formulation  of  a prescriptive  model  for  the  motivational  aspects  of 
instructional  design  based  upon  the  flow  theory.  There  is  absolutely  no  reason  why  the  classroom  cannot  release  as 
much  excitement,  energy  and  achievement  as  are  seen  in  sport  stadium  and  playing  fields  around  the  world  every 
weekend.  More  researches  and  explorations  in  the  motivational  aspects  of  instructional  design  will  help  transform 
the  classroom  from  a place  where  students  “have  to  be”,  into  a place  where  students  actually  “want  to  be”. 


292 


References 

1 . Chan,  T.S.  (1998).  Factorial  Validity  & Reliability  of  the  Adapted  Flow  State  Scale.  Paper  presented  at  the 
Annual  Meeting  of  the  Southwest  Educational  Research  Association,  Houston,  TX. 

2.  Csikszentmihalyi,  M.  (1975),  Beyond  Boredom  and  Anxiety.  San  Francisco:  Jossey-Bass. 

3.  Csikszentmihalyi,  M.  (1982).  Learning,  flow  and  happiness.  In  R.  Gross  (Eds.),  Invitation  to  Life  Long 
Learning.  Chicago:  Follett, 

4.  Csikszentmihalyi,  M,  (1985),  Emergent  motivation  and  the  evolution  of  the  self  In  M.L.  Maehr  & C.  Ames 
(Eds,),  Advances  in  Motivation  and  Achievement,  4,  93-119.  Greenwich:  JAl  Press. 

5.  Csikszentmihalyi,  M.  (1988).  The  flow  experience  and  its  significance  for  human  psychology.  In  M. 
Csikszentmihalyi  & 1,  Csikszentmihalyi  (Eds.),  Optimal  Experience:  Psychological  Studies  of  Flow  in 
Consciousness  (pp.  15-35),  New  York:  Cambridge  University  Press. 

6.  Csikszentmihalyi,  M.  (1990).  Flow:  The  Psychology  of  Optimal  Experience,  New  York:  Harper  and  Row. 

7.  Csikszentmihalyi,  M.  (1992),  A response  to  the  Kimiecik  & Stein  and  Jackson  papers.  Journal  of  Applied 
Sport  Psycholosv.  4,  181-183, 

8.  Csikszentmihalyi,  M.  (1994).  The  Evolving  Self : A Psychology  for  the  Third  Millennium.  New  York:  Harper 
Collins, 

9.  Csikszentmihalyi,  M.  (1997),  Creativity:  Flow  and  the  Psychology  of  Discovery  and  Invention.  New  York: 
Harper  Collins, 

10.  Cuban,  Larry  (1986).  Teachers  and  Machines:  The  Classroom  Use  of  Technology  since  1920.  New  York: 
Teachers  College  Press, 

1 1 . Deci,  E.  & Ryan,  R,  (1985),  Intrinsic  Motivation  and  Self  determination  in  Human  Behavior.  New  York: 
Plenum. 

12.  Hannafin,  M.F.,  Hannafin,  K.M.  & Dalton,  D.W.  (1993).  Feedback  and  emerging  instructional  technologies. 
In  J.V.  Dempsey  & G.C.  Sales  (Eds.),  Interactive  Instruction  and  Feedback  (pp.  105-32).  Englewood  Cliffs, 
NJ:  Educational  Technology. 

13.  Jackson,  S.A.&  Marsh,  H.W.  (1996).  Development  and  validation  of  a scale  to  measure  optimal  experience: 
The  flow  state  scale.  Journal  of  Sport  and  Exercise  Psychology,  18(1),  17-35. 

14.  Krikelas,  J.  (1983).  Information-seeking  behavior:  patterns  and  concepts.  Drexel  Library  Quarterly,  19,  5-20, 

15.  Lepper,  M.  & Cordova,  D.  (1992).  A desire  to  be  taught:  Instructional  consequences  of  intrinsic  motivation. 
Motivation  and  Emotion,  3,  187-208. 

16.  Miller,  L.  (1996,  April  30).  The  theory  behind  that  on-line  altered  state.  USA  Today,  pp,  D1-D3. 

17.  Moore  D.M.,  Burton,  J.K.,  & Meyer,  R.J,  (1996).  Mutliple-channel  communication:  The  theoretical  and 
research  foundations  of  multimedia.  In  D.H,  Jonassen  (Eds.),  Handbook  of  Research  for  Educational 
Communication  and  Technology,  (pp,85 1-878),  New  York:  Macmillan. 

18.  Norman,  D.A.  (1994).  Things  that  Make  Us  Smart.  Reading,  MA:  Addison-Wesley. 

19.  Perelman,  L.J.  (1992).  Schools  Out:  Hyperlearning,  the  New  Technology  and  the  End  of  Education,  NY: 
Avon  Book. 

20.  Sweller,  J.  & Chandler,  P.  (1994),  Why  some  material  is  difficult  to  learn.  Cognition  and  Instruction,  12(3), 
185-233. 


er|c 


293 


User  Centred  Courseware 


Loma  Uden 

School  of  Computing,  Staffordshire  University 
The  Octagon,  Beaconside,  Stafford  ST  18  OAD,  England 
l.uden@soc.staffs.ac.uk 

Alan  Dix 

School  of  Computing,  Staffordshire  University 
& aQtive  limited,  Birmingham,  UK 
alan@hiraeth.com 


Abstract:  User  Centred  Courseware  Development  (UCCD)  offers  an  alternative  approach 
to  courseware  design.  Instead  of  the  traditional  approach  involving  only  the  designers, 
UCCD  provides  a learner-centred  design  approach.  It  offers  a comprehensive  alternative 
design  process,  involving  different  activities,  practices  and  methodologies  appropriate  for 
the  different  phases  of  system  development.  This  paper  describes  UCCD  in  the  context  of 
a case  study  involving  the  development  of  courseware  for  a high  school  in  Stafford,  UK 
(students  aged  11-18).  It  shows  how  UCCD  can  be  used  to  design  effective  and  usable 
courseware  which  meets  users  requirements  and  expectations. 


Introduction 

In  this  paper  we  will  give  a brief  review  of  User  Centred  Courseware  Development  (UCCD)  a novel 
courseware  development  process  building  on  principles  of  user  centred  design  from  human-computer 
interaction  and  information  systems.  We  will  start  by  giving  a short  overview  of  the  background  and  structure 
of  UCCD.  The  main  part  of  the  paper  will  look  at  UCCD  in  more  detail  as  applied  to  a specific  case  study, 
the  development  of  courseware  about  World  War  1 for  13/14  year  olds  in  a Stafford  high  school.  This  will  be 
followed  by  a short  analysis  of  the  lessons  learnt  from  this  exercise  and  for  user  centred  courseware  design  in 
general. 


User  Centred  Design 

Traditional  courseware  development  is  normally  carried  out  by  designers,  who  take  complete  control. 
It  is  rare  that  the  end  users  (who  include  the  teacher  and  learners)  are  involved.  Although  the  courseware 
produced  may  be  effective,  it  often  lacks  usability.  User-centred  design  is  an  approach  to  integrate  user 
requirements,  user  interface,  validation  and  testing  into  a design  process  to  make  the  software  usable.  Rather 
than  being  technology-driven,  user-centred  design  focuses  on  the  people,  or  users. 

The  benefits  of  adopting  UCD  to  courseware  design  are  that  it  accelerates  software  development  and 
improves  quality  of  life  of  users  by  reducing  stress  and  improving  satisfaction.  This  in  turn  improves  user 
interface  and  system  functionality,  which  improves  the  level  of  productivity  of  users  with  fewer  errors. 
Overall  the  system  is  easier  to  understand  and  use,  thus  reducing  training  costs,  and  finally  software 
maintenance  in  terms  of  support  calls  and  documentation  errors  is  dramatically  decreased. 


Methods  for  User  Centred  Design 


O 
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There  are  several  methods  which  are  applicable  to  the  development  of  user-centred  courseware.  Two 
of  the  well-known  methods  that  have  been  used  successfully  in  software  development  are  Participatory 
Design  (PD)  and  Joint  Application  Development  (JAD)  (Carmel  1993).  There  are  several  similarities 
between  PD  and  JAD.  Both  methodologies  stress  a high  degree  of  user  involvement  as  imperative  to  good 
design  of  software  systems.  Both  represent  new  thinking  on  the  traditional  forms  of  user  involvement.  Both 
involve  the  users  in  workshops  and  to  various  degrees,  encourage  creativity  and  new  thinking.  Practitioners 
in  PD  and  JAD  often  employ  simple,  low-level  documentation  and  visualisation  methods  in  their  workshops. 


User  Centred  Courseware  Development 


We  have  developed  a method  known  as  User  Control  Centred  Development  (UCCD)  which 
incorporates  some  of  the  principles  of  PD  and  JAD  using  prototyping.  At  a high  level,  major  steps  in 
developing  a courseware  application  in  UCCD  include  those  shown  in  Figure  1 . 


Deploy 


Figure  1.  Macro-level  steps  in  UCCD  development  process 

The  process  is  an  iterative  and  incremental  development  process,  in  that  the  software  is  not  released  in 
one  “big  bang”  at  the  end  of  the  project.  An  iterative  life-cycle  is  based  on  successive  enlargement  and 
refinement  of  an  application  through  multiple  development  cycles  of  analysis,  design,  development, 
implementation  and  evaluating.  The  application  grows  by  adding  new  functions  within  each  development 
cycle. 

The  first  phase  of  UCCD  is  the  plan  and  elaborate  phase.  This  phase  includes  the  initial  conception, 
investigation  of  alternatives,  establishing  the  rationale  of  the  project,  planning,  specification  of  requirements 
and  so  on.  During  this  phase  we  also  collect  more  detailed  requirements,  do  high-level  analysis  and  design  to 
establish  the  baseline  architecture  and  create  a plan  for  construction.  The  construction  phase  consists  of  many 
iterations,  in  which  each  iteration  builds  production-quality  courseware,  tested  and  integrated,  that  satisfies  a 
subset  of  the  requirements  of  the  project.  Each  iteration  contains  all  the  usual  life  cycle  of  analysis,  design, 
development  and  evaluation.  Each  iteration  or  development  tackles  a relatively  small  set  of  requirements. 
The  system  grows  incrementally  as  each  cycle  is  completed.  This  is  in  contrast  to  the  traditional  courseware 
cycle  in  which  each  activity  (analysis,  design,  and  so  on)  is  done  for  the  entire  set  of  system  requirements. 
The  advantages  of  using  an  iterative  development  process  include  the  complexity  is  easier  to  manage  because 
early  feedback  is  generated,  because  implementation  occurs  rapidly  for  a small  subset  of  the  system.  The  last 
phase  is  the  deployment  of  the  application  into  actual  use. 


Case  study:  UCCD 

The  UCCD  method  is  described  in  a case  study  concerned  with  the  development  of  a courseware  for 
students  aged  13-14,  learning  history  on  World  War  I (1914  - 1918).  The  users  involved  are  the  history 
teacher  and  the  students  in  her  class.  The  subsequent  sections  of  this  paper  document  the  UCCD  process. 

Plan  and  elaborate 

The  first  session  was  scheduled  with  the  teacher  and  students  at  the  high  school.  As  designers  we 
believe  strongly  that  correct  and  thorough  requirements  specifications  are  essential  to  a successful  project.  The 
primary  goal  of  the  requirements  phase  is  to  identity  and  document  what  is  really  needed  in  a form  that 
clearly  communicates  to  all  participants.  During  our  first  meeting  we  conducted  a conceptualisation  process. 
Conceptualisation  is  the  process  of  coming  up  with  an  idea  for  a system,  along  with  a general  idea  of  its 
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requirements  and  form.  It  finishes  the  statement,  ’The  system  we  want  is ",  The  primary  output  from  this 

process  is  a set  of  core  requirements  for  the  system. 

We  started  the  conceptualisation  process  by  conducting  a use  case  analysis  to  identify  the  actors 
involved  in  the  system.  Use  Case  analysis  was  first  popularised  by  Ivor  Jacobson  who  defines  a use  case  as  a 
typical  interaction  that  a user  has  with  the  system  in  order  to  achieve  some  goals  (Jacobson  1994). 

An  actor  is  a role  that  a user  plays  with  respect  to  the  system.  There  are  five  actors  identified  for  the 
school:  headmaster,  teacher,  students,  secretary  and,  of  course,  the  computer  systems  itself  Actors  carry  out 
use  cases.  A single  actor  can  perform  many  use  cases.  Conversely,  a use  case  may  have  several  actors 
performing  it.  Actors  do  not  need  to  be  human.  An  actor  can  also  be  an  external  system  that  needs  some 
information  from  the  current  system.  Figure  2 shows  a Use  Case  Diagram  for  the  high  school  in  question. 


Figure  2,  Use  Case  Diagram  for  the  School. 

Following  the  identification  of  the  actors  involved  in  the  system,  we,  with  the  teacher,  then  tried  to 
determine  the  requirements  of  the  system.  The  goal  identified  following  our  conceptualisation  process  was 
“To  develop  a courseware  for  the  students  to  learn  the  history  of  World  War  1 using  the  computer."  In  order 
to  improve  our  understanding  of  the  requirements  we  created  high-level  use  cases  to  quickly  obtain  some 
understanding  of  the  overall  major  processes  involved.  Examples  of  the  use  cases  identified  include:  (I)  to 
describe  the  events  leading  to  World  War  I;  (ii)  to  show  the  timeline  of  World  War  I;  (iii)  to  discuss  the  horror 
of  wars;  and  (iv)  to  understand  the  implications  of  the  use  of  machine  guns  in  World  War  I. 

A prototype  was  created,  using  storyboards,  to  verify  the  requirements  based  upon  the  use  cases 
identified.  The  main  purpose  of  the  prototype  at  this  stage  was  to  aid  understanding  of  the  problem  and 
establish  requirements.  The  students  and  the  teacher  evaluated  the  prototype.  The  identification  of  use  cases 
was  probably  the  most  critical  part  of  the  plan  and  elaborate  phase.  The  main  purpose  was  to  determine  the 
tasks  the  learners  must  perform  to  achieve  the  goals.  An  important  result  of  the  elaboration  is  that  we  have  a 
baseline  architecture  for  our  system.  The  architecture  consists  of  a list  of  use  cases  that  tell  us  what  the 
requirements  are  and  the  domain  model  that  captures  our  understanding  of  the  subject  and  serves  as  the 
starting  point  for  our  key  topics. 

The  architecture  is  the  foundation  of  our  development.  It  acts  as  the  blueprint  for  later  stages. 
Included  in  this  phase  was  the  plan  for  the  project.  The  essence  of  the  plan  is  to  set  up  a series  of  iterations 
for  construction  and  to  assign  use  cases  to  iterations.  The  plan  is  fulfilled  when  each  use  case  is  put  into 
iteration  and  each  iteration’s  start  date  has  been  identified.  The  use  cases  obtained  are  then  categorised  and 
the  level  of  priority  assigned  to  each  use  case.  For  each  prioritised  use  case  it  is  also  necessary  for  us  to 
estimate  the  length  of  time  each  use  case  will  require.  Another  important  analysis  that  is  necessary  is  that  of 
the  learners.  The  purposes  of  analysing  the  learners  are  to  determine  the  entry-level  of  the  course  and  the 
relevant  characteristics  of  the  learners.  It  is  also  important  to  identify  their  intellectual  abilities,  language 
level,  and  learning  styles. 
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Construction 


The  construction  phase  of  a project  involves  a repeated  development  cycle  or  series  of  iterations  within 
which  the  system  is  extended.  The  final  objective  is  a working  courseware  application  that  correctly  meets  the 
requirements.  Each  iteration  is  a mini-project.  We  do  analysis,  design,  development,  evaluation  and 
integration  for  the  use  cases  assigned  to  each  iteration.  We  finish  the  iteration  with  a demonstration  to  the 
users  and  perform  evaluation  to  confirm  that  the  use  cases  have  been  built  correctly.  The  iterations  with 
constructions  are  both  incremental  and  iterative.  The  iterations  are  incremental  in  function.  Each  iteration 
builds  on  the  use  case  developed  in  the  previous  iterations. 

The  development  cycle 

Development  cycles  are  organised  around  use  case  requirements.  The  development  cycle  is  assigned  to 
implement  one  use  case.  The  methodology  chosen  for  the  development  cycle  or  iteration  is  the  Uden 
Courseware  Engineering  Methodology  (Uden  1997).  UCEM  is  a courseware  development  methodology  based 
on  the  construction  of  models  from  analysis  to  implementation.  It  provides  a seamless  transition  from 
analysis  to  implementation,  allowing  a clean  traceability  between  models.  Both  the  teacher  and  the  students 
were  involved  throughout  each  development  cycle.  Besides  the  teacher,  about  six  students  were  chosen  to  be 
co-designers.  For  each  iteration  development  at  least  three  meetings  were  conducted.  The  first  meeting 
normally  only  involved  the  teacher  and  ourselves.  During  this  session  we  carried  out  the  analysis  and  design 
phase  of  UCEM. 

Analysis 

Domain  analysis  is  the  most  critical  step  in  the  courseware  design  process.  The  main  purpose  of  the 
domain  analysis  is  to  determine  the  tasks  or  topics  to  be  learned,  i.e.  what  is  worth  teaching.  The  domain  is 
gathered  by  interview  and  brainstorming  sessions  with  the  teacher  on  the  history  of  World  War  I. 

During  domain  analysis  each  of  the  high-level  use  cases  identified  during  conceptualisation  would  be 
expanded  to  show  more  detail.  From  each  of  the  use  cases  we  produced  a domain  model  based  on  the  domain 
analysis  carried  out  for  each  use  case.  The  domain  model  illustrates  meaningful  concepts  in  a subject  to  be 
taught.  The  domain  model  is  a representation  of  concepts  in  a subject  matter.  It  can  be  illustrated  with  an 
object  diagram  showing  the  topics,  and  subtopics,  their  associations  as  well  as  their  structures.  An  example  of 
the  domain  model  produced  using  an  object  diagram  is  shown  in  Figure  3. 


Figure  3.  Domain  model  of  world  war  1. 


Design 


The  design  phase  is  concerned  with 
identification  of  learning  objectives  and  the 


the  construction  of  the  pedagogical  model, 
design  of  assessment  items  and  instructional 


It  involves  the 
strategies.  The 
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pedagogical  design  deals  with  the  modelling  of  the  instructional  aspects  of  the  courseware.  The  main 
objectives  identified  for  the  history  lesson  are  to  describe  the  events  that  led  up  to  World  War  1;  and  to  show 
the  effects  of  these  events. 

To  determine  if  the  students  have  learned  we  must  assess  them.  This  involves  designing  tests.  The 
tests  designed  must  match  the  objectives  stated  earlier.  According  to  Gagne  (1965)  there  are  different  types  of 
learning  and  each  requires  different  learning  strategies.  Depending  on  the  learning  outcomes,  we  need  to 
design  the  instructional  strategies  for  the  instruction.  This  completes  the  pedagogical  model  ready  to  be  input 
for  development. 

Development 

The  development  phase  of  UCEM  was  carried  out  by  us,  the  designers.  The  pedagogical  model 
produced  must  be  transformed  into  software,  which  can  be  input  to  the  computer.  This  is  carried  out  in  the 
development  phase  of  UCEM,  Development  consists  of  three  sub-processes:  conceptual  modelling,  navigation 
modelling  and  interface  modelling.  During  conceptual  modelling  the  pedagogical  model  is  translated  into 
software.  If  hypermedia  is  used,  the  conceptual  model  is  then  transformed  into  the  navigation  model  taking 
into  account  the  design  of  the  navigation  structures.  This  is  followed  by  the  interface  development  for  the 
courseware.  The  courseware  with  the  complete  development  is  then  ready  for  implementation. 

Evaluation 

Evaluation  is  essential  to  make  sure  that  the  iterative  development  satisfied  the  requirements. 
Evaluation  usually  uncovers  problems  that  may  not  have  been  noticed  during  development.  The  evaluation 
was  carried  out  by  students  and  the  teacher.  This  happened  during  the  second  meeting  of  the  development 
cycle.  The  developed  courseware  was  given  to  the  students  to  evaluate.  The  users,  including  the  teacher,  were 
encouraged  to  come  up  with  suggestions  for  improvements.  We  stayed  'at  a distance’  in  order  to  let  the  users 
themselves  explore  the  prototype.  There  was  no  intervention  or  suggestions  for  improvements  by  us  in  the 
process,  except  in  breakdown  situations  in  which  the  users  had  problems.  The  groups  which  consisted  of  a 
dozen  students  worked  enthusiastically  with  the  prototype  and  came  up  with  constructive  suggestions  for  each 
session.  All  of  the  recommendations  and  suggestions  by  each  user  were  recorded  by  us.  Any  unclear  issues 
were  taken  up  with  the  users  to  make  sure  they  understood  what  was  needed.  No  suggestion  was  considered 
too  trivial.  We  wanted  to  make  sure  the  users  were  happy  and  that  felt  they  were  part  of  the  team.  Users  were 
happy  to  co-operate  and  valued  the  opportunity  to  be  co-designers.  The  suggestions  or  recommendations  were 
incorporated  as  we  received  feedback  from  the  users.  The  revised  courseware  was  then  given  to  the  students  to 
be  further  evaluated.  The  development  cycles  continued  until  all  the  use  cases  were  completely  developed  and 
integrated  into  the  courseware. 

Discussion  and  suggestions 

Although  the  outcome  of  the  project  was  a positive  one,  there  were  also  many  problems.  This  was 
particularly  so  when  we  came  to  the  design  of  the  pedagogical  model.  Being  a subject  expert  herself,  the 
teacher  was  only  concerned  about  the  content  of  the  subject.  It  was  very  difficult  for  her  to  see  the  importance 
of  objective  definitions  in  the  design.  We  are  firm  believers  in  having  sound  principles  from  instructional 
design  theories  incorporated  in  the  courseware.  Unfortunately  this  was  rather  alien  to  the  teacher.  Much  time 
was  spent  in  having  to  explain  the  concepts  to  her.  Although  she  might  not  have  had  much  knowledge  of  the 
processes  involved,  she  did  make  considerable  contributions  in  each  of  the  activities  and  was  very  supportive. 
As  World  War  I was  not  a subject  with  which  we  were  particularly  familiar,  we  had  to  learn  fast  in  order  to 
become  competent.  The  teacher  was  very  patient  with  us  and  taught  us  a lot.  With  hindsight,  we  could  have 
better  prepared  ourselves. 

Another  problem,  which  often  surfaced  during  the  development,  was  that  we  often  did  not  see  things 
from  the  same  perspective.  There  were  ideas  that  we  considered  as  essential  and  pedagogically  sound  which 
were  not  necessarily  acceptable  to  the  teacher.  For  example,  for  learning  the  timeline  we  suggested  the 
drawing  of  bar  charts  should  be  used.  This  was  rejected  because  students  are  not  expected  to  do  this  in  a 
history  lesson.  We  found  the  teacher’s  insistence  that  this  activity  was  the  province  of  a Mathematics  lesson 
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quite  disturbing,  although  this  was  not  really  the  teacher’s  fault.  Traditionally  learning  in  school  is  concerned 
with  particular  content  in  discrete  subject  disciplines.  A holistic  approach  is  seldom  encouraged,  resulting  in 
students  passing  examinations  but  not  being  able  to  apply  knowledge  in  a wider  context. 

Finally,  users  who  are  not  very  conversant  with  technology  often  have  higher  expectations  of  what  can 
be  achieved.  We  found  that,  due  mainly  to  media  hype,  they  demanded  more  than  we  were  able  to  provide. 
For  example,  they  wanted  video  footage  of  World  War  I,  which  we  were  unable  to  obtain.  On  the  positive 
side,  the  teacher  found  the  co-designer  process  helpful  and  learned  much  about  the  importance  of  instructional 
design  principles  and  appreciated  their  use.  Because  the  teacher  was  part  of  the  design  team  and  thus  had 
ownership  of  the  problem,  she  was  more  willing  to  adapt  to  an  unfamiliar  teaching  situation.  The  resulting 
synthesis  of  ideas  led  to  a very  usable  product.  For  the  students,  they  also  experienced  the  ownership  of  the 
problem  and  took  upon  themselves  the  fact  that  it  was  their  project  too.  This  lead  them  to  become  more 
motivated  in  their  learning  of  history.  They  found  learning  fun  because  they  were  creating  their  own  learning 
materials  and  deciding  what  was  good  for  them.  The  interactive  nature  of  the  computer  with  its  powerful 
graphics  facilities  also  fascinated  them. 

Lessons  Learned 

We  learned  many  things  trom  UCCD.  Firstly,  users  rarely  know  what  they  want  at  the  start  of  the 
project.  As  designers  we  must  provide  the  scaffolding  to  guide  them  in  identifying  their  learning  objectives. 
Secondly,  working  with  the  prototype  allows  the  users  to  get  a better  'feel'  for  the  proposed  courseware  lessons 
so  that  they  could  be  more  confident  with  the  system  to  be  built  and  be  able  to  contribute  constructively. 
Thirdly,  when  users  saw  what  the  prototype  could  do,  their  expectations  changed  and  they  wanted  more 
features.  It  is  important  to  set  a realistic  threshold  on  their  requests.  Fourthly,  as  users  were  part  of  the  team, 
they  were  very  keen  to  be  involved  and  felt  they  'owned'  the  problem.  Fifthly,  UCCD  shows  us  the  importance 
of  designing  for  the  users  and  their  requirements.  Sixthly,  the  process  enables  us  to  produce  a system  that  is 
more  tailored  to  the  users'  needs.  Finally,  for  user-centred  courseware  design  to  work,  it  is  important  to  have 
a graphical  user  interface  tool  such  as  Visual  Basic,  which  allows  us  to  produce  a quick  product. 

Conclusion 

To  develop  usable  courseware  is  a difficult  task.  Typically,  courseware  development  based  on  the 
expert  approach  does  not  always  produce  systems  that  meet  users'  requirements.  For  courseware  to  be  usable 
it  must  be  accepted  by  the  users  and  meet  their  needs.  This  means  that  users  must  play  an  active  role  in  the 
design  and  development  process  as  co-designers.  Designers  and  users  must  join  forces  together  in  the  design. 
Because  the  users  know  best  the  context  of  their  environments,  they  are  able  to  overcome  many  of  the 
problems  that  designers  are  not  able  to  envisage.  The  user-centred  design  paradigm  enables  designers  to 
produce  courseware  that  will  meet  the  users'  needs.  We  have  learned  from  our  case  study  that  Experts  alone 
cannot  design  systems  that  will  satisfy  users.  Only  the  users  - in  the  context  of  their  work  or  learning  - can 
provide  the  necessary  expertise  for  successful  design. 
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Abstract:  Making  intelligent  learning  supports  available  on  World  Wide  Web  (WWW)  is  a 
potential  way  to  enhance  a student’s  learning  capability  by  integrating  the  flexibility  and 
adaptability  of  intelligent  supports  with  world- wide  availability  of  WWW-based  applications. 
In  this  paper  we  present  the  design  and  implementation  of  a collaborative  intelligent  learning 
support  environment  called  CILSE-GCE  for  global  climate  exploration  on  WWW.  The  task 
domain  is  inherently  a scientific  classification  problem,  in  which  students  are  expected  to 
induce  climate  classification  rules  and  develop  their  own  interpretations  by  making 
observations  of  a couple  of  climatic  features.  Based  on  a collaborative  study- based  learning 
model,  we  develop  a set  of  tutoring  strategies  and  implement  the  corresponding  intelligent 
learning  supports.  As  the  classification  task  involves  fuzzy  climatic  features,  we  develop  an 
intelligent  fuzzy  tutor  to  evaluate  and  keep  the  student’s  exploration  going  toward  target 
knowledge. 


Introduction 

This  is  a prosperous  age  in  terms  of  rich  learning  theories,  instructional  designs  and  computer 
technologies.  Their  integrated  efforts  have  made  pervasive  influences  on  the  ways  we  teach  and  learn,  and 
definitely  have  created  great  possibilities  and  chances  to  build  an  ideal  learning  environment,  where  students 
enjoy,  share  and  appreciate  the  learning  itself.  In  recent  years,  research  and  development  of  intelligent  learning 
environments  (Major  & Reichgelt,  1992;  Chan,  1994;  Bell,  Davis,  & Linn,  1995;  Edelson,  Pea  & Gomez,  1996) 
have  been  growing  rapidly  and  widely.  This  paper  demonstrates  a project  working  toward  this  trend.  In  this 
project,  advanced  computer  technologies,  such  as  artificial  intelligence.  World  Wide  Web  (WWW),  Geographic 
Information  System  (GIS)  and  multimedia  are  integrated  successfully  with  the  collaborative  constructivist 
learning  theory  (Black  & McClintock,  1996;  Honebein,  1996),  resulting  in  a collaborative  intelligent  learning 
support  environment  on  WWW. 

This  two-year  project,  beginning  from  mid  1998,  aims  to  build  a collaborative  learning  environment 
(named  CILSE-GCE,  Collaborative  Intelligent  Learning-Support  Environment  for  Global  Climate  Exploration) 
that  provides  constructivist-inspired  learning  tools  for  open-ended  inquiry  as  well  as  communication  tools  that 
support  multi-learner  collaboration.  In  this  environment,  students  learn  by  exploring  the  global  climate  patterns, 
sharing  their  discoveries  and  producing  the  target  knowledge  collaboratively.  Special  attentions  would  be  paid 
to  the  kinds  of  learning  supports  the  system  could  provide  for  the  student’s  exploration  and  collaboration 
processes.  The  ultimate  goals  of  this  project  include  investigating  the  collaborative  constructivist  learning  model 
and  instruction  design  principles,  the  development  model  of  learning  environments  and  system  functionality,  and 
the  feasibility  of  applying  artificial  intelligence  to  support  constructivist  learning. 

Based  on  the  collaborative  study-based  learning  model  by  Black  & McClintock  (1996),  we  develop  a 
two-level  group  learning  model  suitable  to  be  practiced  in  so-called  virtual  classrooms ^ and  implement  the 
corresponding  teaching  supports  and  communication  utilities.  Besides,  as  the  classification  task  involves  fuzzy 
climatic  features,  an  intelligent  fuzzy  tutor  is  designed  and  implemented  to  evaluate  and  keep  the  student’s 
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exploration  going  toward  target  knowledge.  It  could  also  help  students  in  the  construction  of  climatic 
interpretation  models  as  concept  maps. 

This  learning  environment  is  designed  with  the  intention  not  only  to  teach  students  the  target 
knowledge,  but  also  the  scientific  ways  of  study  and  communication  skills.  We  believe  the  students  will  achieve 
higher  learning  goals  through  the  collaborative  process  of  creating  knowledge  by  themselves.  These  might 
include  the  ownership  of  knowledge,  high-order  thinking  skills,  and  the  potential  of  becoming  a long-life  learner. 
Preliminary  formative  evaluations  have  been  performed  during  the  late  1998  and  early  1999,  while  a more 
complete  evaluation  is  scheduled  to  be  performed  and  investigated  in  the  second  year  of  this  project.  Hence  in 
the  following  we  will  focus  mainly  on  the  design  and  implementation  issues  of  the  learning  environment,  and 
present  some  preliminary  findings  of  the  first  year  research. 


The  Collaborative  Study  Model 

We  adopt  the  perspective  that  the  basis  for  cognition  is  interpretation  based  on  background  knowledge 
and  beliefs  (Winograd  & Flores,  1986).  Thus,  the  key  consideration  in  our  design  is  fostering  the  construction  of 
interpretations  based  on  observations  and  background  contextual  information.  This  learning  process  is  often 
referred  to  study  (Black  & McClintock,  1996),  in  which  students  make  observations,  construct  interpretations 
and  arguments  via  access  to  background  and  contextual  materials.  All  stages  of  the  study  can  be  performed 
collaboratively.  Based  on  the  aforementioned  theory,  we  propose  a two-level  group  study  model  suited  for 
virtual  classroom  practicality,  as  shown  in  Fig.  1 


Figure  1:  The  two-level  group  study  model. 

In  the  two-level  group  study  model,  there  are  two  forms  of  discussion.  One  is  the  class  discussion  and 
the  other  is  the  group  discussion.  Students  are  first  pre-assigned  to  specific  groups  as  they  wish,  except  with  the 
only  constraint  of  group  size  of  5 to  6 (a  variable  that  needs  further  investigation).  The  human  teacher  then 
announces  the  target  problems  to  the  student  groups  and  coaches  some  basic  problem-solving  skills  through  the 
class  discussion.  Then  the  student  groups  commence  the  collaborative  study  process  by  first  negotiating  a 
proper  decomposition  of  tasks.  Each  group  member  is  assigned  a specific  learning  task  and  starts  to  explore  the 
problem  in  the  learning  environment,  trying  to  work  out  the  answers  and  put  down  the  results  in  notebooks  and 
concept  maps.  Each  group  then  negotiates,  defenses  and  integrates  the  member  products  to  achieve  group 
consensus  and  generate  their  final  group  product.  Finally,  the  teacher  and  all  students  start  to  discuss,  defense 
and  evaluate  all  group  products  one  by  one  to  achieve  class  consensus.  As  a result,  students  could  develop  their 
social  communication  skills  and  learn  to  investigate  problems  from  different  perspectives  as  illustrated  by  the 
works  of  their  colleagues. 
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The  Global  Climate  Domain 


The  target  domain  draws  sources  from  the  instructional  material  in  the  geographic  climate  course  of 
senior  high  schools  in  Taiwan.  The  contents  consist  of  (1)  constituting  attributes  of  climates,  (2)  influential 
attributes  of  climates  and  (3)  twelve  climate  patterns  on  the  earth.  The  constituting  attributes  include  the 
atmosphere  temperature,  atmosphere  pressure,  wind  system,  and  so  on.  The  influential  attributes  encompass  the 
degree  of  latitude,  topography,  distance  from  the  sea,  etc.  Finally,  the  climate  patterns  include  the  tropical  wet 
climate,  temperate  wet  climate,  and  so  forth.  The  domain  knowledge  could  be  interpreted  in  a two-level 
abstraction  as  shown  in  Fig.  2.  The  first  knowledge  level  is  about  each  climate  pattern,  which  is  recognized  as  a 
specific  set  of  the  climatic  attributes.  The  second  knowledge  level  is  a deeper,  causal  knowledge  that  involves 
the  influential  attributes  and  their  interactions,  which  comprise  a scientific  interpretation  model  that  explains  a 
terrain’s  climate. 


Figure  2:  The  knowledge  structure  of  the  climate  patterns. 


The  task  domain,  global  climate  exploration,  is  inherently  a scientific  classification  problem.  In  the 
member  study  phase  of  the  group  study  model,  each  student  is  expected  to  induce  classification  rules  by  making 
observations  of  a couple  of  climatic  features.  A visualized  information  system  based  on  the  GIS  technique  is 
developed  to  help  students  make  observations,  data  collections,  comparisons  and  recordings  about  climatic 
features  within  and  between  any  regions  of  the  globe.  The  student  could  send  his/her  findings  to  a fuzzy  tutor 
that  evaluates  the  results  and  gives  proper  suggestions  about  the  student’s  exploration  process.  Students  are  also 
required  to  construct  an  interpretation  model  of  the  climate  patterns,  explaining  why  and  how  the  features 
interact  to  each  other.  The  concept  map  (Jonassen,  etal.,  1997)  is  used  as  a mind  tool  for  recording  such  kind  of 
learning  states.  The  concept  maps  along  with  the  electronic  notes  serve  as  the  basic  contents  for  knowledge 
collaboration  process. 


The  Learning  Environment 

Three  components  of  the  CILSE-GCE  learning  environment  are  presented  in  this  section.  They  are  the 
Virtual  Classroom,  Visualized  Data  Viewer,  and  Intelligent  Fuzzy  Tutor,  respectively.  The  system  architecture 
is  shown  in  Fig.  3.  To  facilitate  our  further  research,  all  learning  activities  of  the  students  in  this  environment 
are  logged  and  could  be  queried  back  later  for  more  analysis.  The  Virtual  Classroom  serves  as  the  origin  where 
teachers  and  students  coordinate  and  collaborate.  Through  the  Virtual  Classroom,  students  could  access  the 
multimedia  course  base,  the  climatic  GIS  database  (via  the  Visualized  Data  Viewer)  and  the  historical  literature 
database.  These  rich  data  sources  allow  students  to  observe,  search  and  collect  related  information  in  different 
aspects  regarding  to  the  problems  at  hand.  Editing  tools  like  the  electronic  note  and  the  concept  map  editor  are 
also  provided  to  help  students  write  down  their  discoveries  and  final  works.  All  these  personally  created  stuff 
are  stored  and  managed  by  the  system  automatically,  and  if  authorized,  can  be  accessed  by  other  students,  thus 
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enriching  the  system  data  resources  in  a multi-perspective  manner. 
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Figure  3:  The  system  architecture  of  CILSE-GCE, 

In  the  Visualized  Data  Viewer,  the  climate  information  could  be  displayed  in  different  layers  of  maps 
covering  the  globe.  Students  could  select,  resize  and  combine  different  information  layers  for  display  to 
investigate  the  climate  attributes  in  different  perspectives,  Hotlinks  to  climatic  data  and  statistical  graphs 
associated  with  the  typical  cities  are  also  provided  to  allow  students  to  do  some  measurements  and  inferences. 

We  design  an  intelligent  fuzzy  tutor  to  help  students  induce  the  classification  Riles,  which  are 
represented  as  a setting  of  the  constituting  attributes  of  the  climate  patterns.  During  the  rule  induction  process,  a 
student  has  to  identify  what  the  settings  of  the  relevant  attributes  are  by  exploring  resources  of  all  kinds.  When 
he/she  determines  a specific  set  of  attribute  values,  the  intelligent  tutor  would  evaluate  the  student’s  answer,  and 
give  suggestions  to  guide  the  student’s  further  exploration.  As  the  classification  task  involves  fuzzy  climatic 
attributes,  we  develop  an  intelligent  fuzzy  evaluator  to  assess  and  keep  the  student’s  exploration  going  toward 
target  knowledge. 

When  the  student  arrives  at  correct  settings  of  the  attributes,  he/she  is  encouraged  to  construct  an 
interpretation  model  that  explains  the  causal  relationship  between  the  influential  attributes  and  the  constituting 
attributes.  Students  might  query  the  material  database  to  investigate  the  explanations  of  the  various  phenomena, 
and  then  deduce  the  causal  model  for  the  climate  patterns.  The  intelligent  tutor  would  check  the  models  that 
students  build,  and  give  remedial  hints  if  anything  goes  wrong. 


Detailed  Design  Decisions 


Learning  Thoughtfully  versus  Learning  by  Memorization 

In  this  section,  we  briefly  summarize  some  design  issues  that  were  tackled  for  the  CILSE-GCE  learning 
environment.  The  first  issue  is  to  cope  with  the  traditional  problem  of  learning  by  memorization,  a strategy 
adopted  ubiquitously  by  Taiwanese  students  in  geographic  climate  classes.  Learning  by  memorizing  facts 
empowers  students  for  solving  routine  problems  efficiently.  However,  it  leads  to  inflexible  use  of  the 
memorized  knowledge  in  novel  situations.  On  the  other  hand,  learning  to  study  things  thoughtfully  helps 
students  to  solve  complex  problems.  Thus  we  provide  a manipulable  visual  environment,  in  which  students 
could  engage  themselves  in  the  whole  study  process  thoughtfully.  In  this  visual  environment  students  could 
observe  real  world  data  in  different  perspectives,  derive  their  own  classification  rules  and  test  the  rules. 
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Whole  Learning  Tasks  versus  Component  Skills 

The  second  issue  is  about  the  argument  of  focusing  on  whole  learning  tasks  or  component  skills. 
Having  students  perform  whole  tasks  has  the  advantages  of  practicing  the  integrative  skills  that  are  necessary, 
and  developing  strategies  suitable  to  the  tasks  as  a whole.  But  it  is  difficult  for  students  to  focus  on  a particular 
weakness  in  the  tasks.  On  the  contrary,  making  students  focus  on  sub-skills  is  sometimes  very  productive,  but 
ideally  this  should  occur  when  a weakness  has  been  diagnosed.  Our  solution  is  to  start  by  involving  students  in 
the  whole  induction  tasks,  and  then  to  proceed  to  component  tasks  when  appropriate.  In  this  environment, 
students  first  visit  the  visual  environment,  considering  the  problem  in  all  its  aspects,  and  then  develop  and  test 
their  own  classification  rules.  These  rules  could  be  assessed  by  an  intelligent  fuzzy  tutor,  which  would  identify 
the  weakness  in  the  student’s  exploration  and  provide  some  suggestions  such  as  the  important  aspects  that  should 
be  focused  or  the  appropriate  ways  to  investigate  the  climatic  features. 


Levels  of  Learning  Supports 

The  third  issue  is  related  to  the  determination  of  levels  of  learning  supports.  Learning  supports  should 
be  effective  but  not  deprive  the  students  of  their  learning  opportunities.  For  example,  the  hints  or  suggestions 
given  by  the  fuzzy  tutor  should  not  be  so  direct  such  that  the  answers  can  be  derived  straightforwardly.  The 

scaffolding  should  also  prevent  students  from  answer-guessing  activities.  For  example,  an  entry  list  for  the 

student's  answer  should  not  be  defined  in  advance,  instead,  it  should  be  a freely-input  entry.  Besides,  the 

evaluation  feedback  of  student's  induction  results  should  not  be  just  wrong  or  correct.  It  would  be  better,  for 

example,  to  give  some  questions  that  require  the  students  to  think  about  another  possibilities.  Accordingly  the 
environment  allows  students  much  more  flexibility  to  explore. 

For  example,  we  design  an  intelligent  agent  to  support  the  induction  of  climate  classification  rules.  The 
student  inputs  the  climate  attribute  values  (i.e.,  a set  of  numerical  ranges)  and  sends  the  answers  to  the  tutor 
agent  via  the  internet,  and  then  the  tutor  receives  and  makes  a fuzzy  evaluation  of  the  student’s  inputs.  Finally, 
the  tutor  sends  back  the  evaluation  results,  including  suggestion  messages,  and  the  fuzzy  degree  of  correctness  of 
each  climate  attribute.  The  system  also  makes  a log  tracing  the  whole  induction  process  of  the  student  so  that  the 
student  could  see  and  backtrack  his/her  induction  process  to  a better  position  towards  the  target  goal. 


The  Collaborative  Study  Process 

The  last  issue  is  about  the  determination  of  the  collaborative  study  processes.  We  develop  a virtual 
classroom  on  WWW  in  which  students  study  and  work  in  groups.  The  human  teacher  first  assigns  problems  to 
the  students  and  coaches  some  basic  problem  solving  skills.  Then  the  student  groups  start  the  collaborative 
study  process,  working  out  the  answers,  and  making  defense  within  and  between  groups.  We  believe  this 
collaborative  process  will  foster  all  the  constructive  design  principles  mentioned  in  (Black  & McClintock,  1996; 
Honebein,  1996),  including  observation,  interpretation  construction,  contextualization,  cognitive  apprenticeship, 
collaboration,  multiple  interpretations  and  multiple  manifestations,  ownership  of  knowledge,  self-awareness  of 
construction  process.  Besides,  our  emphasis  on  classification  induction  problems  is  distinctive. 


Conclusion 

We  adopt  a form  of  iterative  design  to  develop  the  CILSE-GCE  environment.  There  is  a strong 
connection  between  many  of  the  design  decisions  taken  and  the  results  of  formative  evaluation.  During  the 
whole  development  process,  we  found  that  the  most  challenging  task  is  to  identify  the  information  needs  of  the 
students.  What  material  should  be,  and  in  what  way,  stored  in  the  material  database?  Did  we  provide  enough 
information  and  investigation  tools  for  the  students  so  that  they  could  learn  the  greatest  from  this  environment? 
These  questions  worth  further  investigation  in  our  second  year  project.  The  answers  to  these  questions  are  not 
only  essential  to  this  environment  but  also  important  to  the  research  area  of  developing  educational  systems. 
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It  is  also  indicated  that  during  free  exploration  of  a problem  space,  greater  learning  occurred  if  students 
adopted  more  systematic  strategies  for  rule  induction  (Vollmeyer,  Bums  & Holyyoak,  1996).  In  our 
environment,  students  would  experience  the  process  of  looking  for  patterns.  However,  students  are  different  in 
the  degree  of  systematicity  with  which  they  formulate  and  test  their  hypotheses.  It  is  therefore  important  to 
provide  instruction  in  the  use  of  such  strategies  in  order  to  allow  maximum  benefit  from  free  exploration  of  the 
problem  space.  Further  evaluation  tests  will  be  conducted  to  provide  evidences  of  such  kinds  of  learning. 
Another  direction  for  future  work  would  involve  analyzing  students’  study  strategies  and  examining  their 
transfer  performance  in  the  consequence  of  problem  solving. 
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Abstract:  This  paper  proposes  an  open  architecture  of  virtual  experiment  environment  on  WWW 
by  using  intelligent  agent  techniques.  Several  experiment  components  and  a science  experiment 
agent  are  analyzed  and  implemented  to  support  teachers  to  quickly  assemble  a social  learning 
system.  Such  design  is  based  on  distributed  network  architecture  to  attain  high  reliability  and 
efficiency. 

This  architecture  of  agent-based  experiment  environment  is  composed  of  two  major  layers, 
Coordinate  Layer  and  Communication  Layer.  And  KQML  (Knowledge  Query  and  Manipulation 
Language)  is  used  to  realize  communication  protocols  and  information  sharing  language  for  the 
interactions  among  experimental  equipment  and  science  experiment  agents.  A workable 
experiment  system  based  on  our  architecture,  called  SoftLab,  is  implemented  on  WWW  to 
demonstrate  its  feasibility  and  flexibility. 

Keywords:  virtual  experiment,  experimental  equipment,  WWW  (World  Wide  Web),  KQML 
(Knowledge  Query  and  Manipulation  Language),  intelligent  agent 


Introduction 

In  the  last  ten  years,  personal  computers  have  sparked  a revolution  in  education.  Everyone  from  preschool  children 
to  senior  citizens  can  now  develop  science  projects  and  prepare  reports  by  using  computer  technology  at  home. 
Besides  being  passive  learning  tools  as  textbooks,  paper  and  pens,  computer-aided  education  (CAE)  systems  can 
prompt  learners  some  feedback  and  responses  in  appropriate  ways  during  learning  process.  (Diessel  1992; 
Futtersack  1992;  T.-W.  Chan  1995)  Take  interactive  tutorials  for  example,  this  kind  of  CAE  systems  can  not  only 
teach  learners  specific  knowledge,  but  also  evaluate  what  learners  has  been  learned  and  re-teach  based  on  the 
evaluation.  Any  paragraph  in  a course  of  study  can  certainly  be  reviewed  to  meet  the  needs  of  learners.  (J.C. 
Gonzalez  1992;  H.  Lianjing  1992) 

Considering  the  practicum  of  conventional  education,  "learning  by  doing"  design  brings  the  idea  of  situated  learning 
into  computer  environment,  as  the  interactions  of  human  cognition  are  studied  and  improved  in  different  computer- 
based  learning  environment.  (Brown,  Collins  & Duguid,  1989)  Hence,  a simulated  platform  which  provides  learners 
experiment  with  visual  equipment  on  computers,  called  virtual  experiment  environment  (VEE)  had  been  constructed 
on  the  computer  network.  (Y.-W.  Jeng  et  al.  1996  J.C.  Gonzalez  et  al.  1998)  This  paper  proposes  an  open 
architecture  of  agent-based  virtual  experiment  environment  on  WWW. 

Section  2 identifies  these  problems  when  establish  a virtual  experiment  environment  on  WWW.  Besides  several 
important  issues  are  analyzed,  an  open  architecture  then  can  be  constructed.  After  the  open  architecture  is  proposed, 
more  detailed  designing  for  a specific  agent-based  architecture  are  designed  by  Section  3.  An  experiment  system 
constructed  on  WWW,  is  so-called  SoftLab,  providing  a workable  agent-based  platform  of  physics  in  Section  4. 
Section  5 makes  a summary  and  discusses  those  possible  future  works. 
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Architecture  of  Agent-Based  Virtual  Experiment  Environment 

The  great  advantages  of  WWW  are  information  transmission  and  knowledge  sharing.  Unfortunately,  those 
capabilities  to  support  high  quality  interactive  learning,  retaining  information  of  learners  and  even  tracking  the 
learning  progress  are  still  under  disputed.  Since  an  open  architecture  of  agent-based  virtual  experiment  environment 
on  WWW  is  proposed,  the  idea  of  Software  ICs  (Integrated  Circuits)  should  be  taken  into  consideration. 

Based  on  the  precompiled  binary  code,  components  become  independent  of  the  language  in  which  they  were 
created.  Furthermore,  there  are  two  more  characteristics  of  components:  reusability  and  flexibility.  Reusability 
means  that  components  can  be  used  in  many  applications,  continually  enhancing  and  improving  without  having  to 
recompile.  On  the  other  hand,  the  flexibility  of  components  provides  that  suitable  components  can  be  simply  chosen 
to  extend  the  standard  functionality  for  the  specific  needs. 

Although  the  Software  ICs  are  very  useful  for  establishing  such  open  architecture  on  WWW,  there  must  have  a 
container  comprised  several  ICs  and  channels  for  exchanging  data.  It  implies  that  if  some  experimental  equipment 
used  for  experiments,  an  experiment  environment  must  exist  firstly.  And  than  the  experiment  can  be  observed 
through  those  interactions  in  experiment  environment.  According  to  such  viewpoints,  a uniform  interface  among 
components  is  also  necessarily  provided  by  the  open  architecture. 

Cooperative  learning  can  improve  both  professional  and  social  skills  of  learners,  including  face  to  different  opinions 
and  interdependence  of  group  members.  (Jamie  C et  al.  1998;  Chanchai  Singhanayok  1998;  Doug  L 1998) 
Therefore,  the  communication  mechanism  between  participants  of  experiments  has  to  design  is  also  an  important 
issue  in  order  to  achieve  the  goal  of  constructing  synchronized  and  cooperative  experiment  environment. 

As  mentioned  above,  all  issues  are  taken  into  consideration.  The  architecture  of  agent-based  virtual  experiment  is 
then  constructed  as  (Fig.  1)  shown. 


Figure  1.  An  open  architecture  of  agent-based  virtual  experiment 


(Fig.  1)  indicates  the  architecture  is  composed  of  two  major  layers.  Coordinate  Layer  and  Communication  Layer. 
Because  distance  learners  may  use  browsers  to  participate  in  the  collaborative  online  activities,  the  responsibility  of 
Coordinate  Layer  is  to  deal  with  the  cooperative  learning  when  experiment  in  the  virtual  environment  through  the 
computer  networks.  And  learners  can  work  together  for  accomplishing  a common  task  or  experiment  through  the 
agency  of  Coordinate  Layer. 

Besides  these  interactions  among  participants,  interactions  among  components  in  the  virtual  experiment  environment 
have  to  handle.  To  describe  and  maintain  this  sort  of  relations  between  the  equipment,  such  like  ball  and  slope  in 
the  experiment,  a management  manner  as  Communication  Layer  provides  is  necessary. 
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Generally  speaking,  there  are  some  standard  physics  laws  illustrating  the  relationships  between  equipment  (or 
simulated  objects)  in  the  environment  to  regulate  the  interactions.  Two-ball  collision  is  a good  example  for 
explaining.  The  key  point  is  that  balls  never  know  they  have  a collision  with  the  other  one.  They  only  know  that  if 
they  are  forced,  then  they  must  response  an  action  corresponding  to  the  excitation.  So  the  environment  must  provide 
a mechanism  to  coordinate  the  behaviors  of  the  objects  and  transfer  the  responses  appropriately. 

Although  an  open  architecture  with  agent-based  virtual  experiment  on  WWW  is  constructed  in  this  section,  more 
detailed  designs  still  need  be  done  before  this  architecture  can  really  open  for  educational  purposes.  After  the 
Communication  Layer  is  well  designed,  either  learner  to  cooperatively  /competitively  learn  with  learners  or  teacher 
to  teach/evaluate/answer  learners  will  be  replaced  by  intelligent  agents  (lAs). 

Designing  the  Virtual  Experiment  Environment 

Besides  the  utilization  of  component  technology,  more  important  issue  which  makes  the  architecture  constructed  in 
the  previous  section  becomes  open  is  a graph  model  for  representing  the  relationships  between  experimental 
equipment.  The  detailed  explanation  for  the  graph  representation  is  in  the  following. 

Actually,  experimental  equipment  in  VEE  must  be  embedded  in  such  an  experiment  board  in  the  environment  so 
that  these  interactive  relations  between  experimental  equipment  can  be  generated  based  on  the  interaction  principles. 
There  is  three-stage  mechanism  in  the  experiment  board  to  make  the  interactions  between  experimental  equipment 
come  true.  At  first,  the  relations  between  experimental  equipment  are  described,  and  a directed  graph  is  adopted  to 
describe  the  experiment  and  store  it  in  the  form  of  adjacency-list.  Secondly,  the  standard  physics  laws  are  pre-stored 
in  the  experiment  board.  It  means  each  node  in  the  adjacency-list  will  be  searched,  and  a mapping  from 
experimental  equipment  into  the  desired  physics  laws.  The  last  stage  implements  the  experiment  based  on  the 
standard  physics  laws  and  those  data  structure  produced  in  the  second  stage. 

The  relations  between  experimental  equipment  are  described  as  a directed  graph  G{V,  E),  where  Kis  a finite  set  and 
£ is  a binary  relation  on  K The  vertex  set  V of  G contains  all  the  experiment  equipment  vertices  in  this  experiment; 
whereas,  the  edge  set  £ of  G stores  all  the  relation  between  the  equipment.  (Fig.  2a)  represents  an  example  directed 
graph  G of  vertex  set  { 1,  2,  3,  4}.  If  (t/,  v)  is  an  edge  in  graph  G,  then  the  equipment  u will  interact  with  equipment 
V.  An  adjacency-list  of  (Fig.  2b)  stores  the  relations  between  the  equipment  vertices  in  (Fig.  2a). 


Take  electric  resistance  connections  for  example  to  illustrate  the  above  three-stage  mechanism.  The  electric 
experimental  equipment  in  (Fig.  3)  is  added  as  battery,  resistance  1,  2 and  3.  From  (Fig.  2a)  the  representation  of 
experiment  after  transformed  into  graph  is  shown  in  (Fig.  3),  which  means  graph  G has  four  vertices:  K/-Battery,  Kj- 
Resistance  1,  K^-Resistance  2,  K#-Resistance  3,  and  five  edges:  (K/,  K?),  (K?,  Kj),  (K?,  K^),  (Kj,  K/),  (K^,  K/). 

After  the  first  stage,  the  experimental  graph  then  will  be  stored  in  the  form  of  adjacency-list  as  (Fig.  2b)  shown. 
Finally,  a specific  tree  is  generated  from  the  adjacency-list  based  on  the  standard  physics  laws  just  like  (Fig.  2c) 
shown.  Accordingly,  all  the  physics  quantities  in  the  electric  circuit  of  this  experiment  can  then  be  obtained. 
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Figure  3.  Electrics  resistance  connections  example 

Regarding  the  agent-based  issue,  intelligent  agent's  behaviors  for  the  educational  proposes  should  be  analyzed  first. 
Three  behaviors  can  be  simply  told  out  as  follows; 

1)  Online  recording, 

2)  Online  replaying, 

3)  Online  navigating. 

In  most  cooperative  learning  systems,  it  is  necessary  to  design  a communication  protocol  for  information  and 
knowledge  exchange.  In  the  open  architecture  of  agent-based  virtual  experiment  environment,  the  most  useful  agent 
communication  language.  Knowledge  Query  and  Manipulation  Language  (KQML),  is  chosen  as  our  communication 
protocol  in  Communication  Layer. 

KQML  focuses  on  an  extensible  set  of  performatives,  which  define  the  permissible  operations  that  agents  may 
attempt  on  each  other's  knowledge  and  goal  stores.  The  performatives  comprise  a substrate  on  which  to  develop 
higher-level  models  of  inter-agent  interaction  such  as  Contract  Nets  and  Negotiation.  (Tim  Finin  1992) 

Conceptually,  a KQML  message  consists  of  a performative,  its  associated  arguments  which  include  the  real  content 
of  the  message,  and  a set  of  optional  arguments  which  describe  the  content  in  a manner  which  is  independent  of  the 
syntax  of  the  content  language.  For  example,  a login  to  a group  learning  system  might  be  encoded  as: 

{tell  ’.content  (Login  (Ken  (jkl234y,  user ) ) ) 

’.language  list 
’.ontology  Group-Learning ) 

In  this  message,  the  KQML  performative  is  tell,  the  content  is  (Login  (Ken  (jkl234y,  user  ) ) ) and  the  assumed 
ontology  is  identified  by  the  token  : Group-Learning. 

Implementing  the  Virtual  Experiment  Environment  on  WWW 

(Fig.  4)  in  the  following  depicts  the  architecture  for  electricity,  only  with  minor  differences  from  the  original  one. 
An  experiment  system  of  physics  course  then  is  implemented  under  the  open  architecture  proposed,  called  SoftLab. 
Besides  these  synchronous  and  asynchronous  tools  for  Web-based  learning,  an  intelligent  science  experiment  agent, 
called  Lancelot,  is  also  added  in  SoftLab.  The  specific  science  experiment  agent  owns  its  architecture  just  like  (Fig. 
5)  indicates.  Distance  learners  are  allowed  to  participate  in  those  collaborative  on-line  activities  through  browsers. 
And  with  the  characteristics  of  Web  technology,  it  even  can  increase  students'  awareness  of  related  concepts  through 
the  inclusion  of  hypertext  links  to  other  Web-based  information.  Chat-room  is  the  only  component  that  can  be  used 
by  learners  to  communicate  with  others  in  VEE.  Teacher,  moreover,  can  also  use  it  to  give  hints  and  comments. 

Considering  the  benefits  of  component-oriented  analysis,  Microsoft  ActiveX  is  chosen  to  implement  the  experiment 
system  described  above.  It  is  easy  to  divide  the  whole  system  into  components  with  abstract  models  of  abilities  and 
behaviors.  Each  learning  work  is  implemented  as  an  independent  component.  Owing  to  ActiveX  technology, 
Microsoft  Internet  Explorer  is  chosen  as  the  platform  of  VEE.  And  Visual  Basic  Script  is  then  the  bridge  between 
components. 

When  designing  the  experiment  system,  performance  issue  of  the  system  is  the  most  important.  The  traditional  BBS 
(Bulletin  Board  System)  server  is  analyzed  and  modified  in  order  to  solve  this  problem.  The  major  modification 
objectives  include  discussion  board,  flow  control  of  learning,  privilege  management  and  group  management.  There 
are  three  parts  in  the  whole  SoftLab,  including  VEE,  experiment  tools  and  the  science  experiment  agent  Lancelot,  as 
(Fig.  6)  shown. 
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Figure  4.  Architecture  of  SoftLab 


Figure  5.  Architecture  of  science  experiment  agent 


Figure  6.  Virtual  experiment  environment  of  circuit  experiment 
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Conclusion 


In  this  paper,  the  possibility  of  constructing  an  open  architecture  of  agent-based  VEE  on  WWW  is  fully  discussed 
and  analyzed.  Two  major  layers  compose  the  architecture  of  agent-based  experiment  environment  that  are 
Coordinate  Layer  and  Communication  Layer.  And  the  interactions  between  experimental  equipment  are  realized  by 
K.QML.  SoftLab  is  a workable  experiment  system  implemented  on  WWW  based  on  the  architecture  proposed  in 
this  paper  to  demonstrate  the  feasibility  and  flexibility. 

Another  interested  research  direction  is  the  courseware  and  visual  components  design  when  an  interactive  user 
interface  is  used.  As  mentioned  in  the  introduction,  we  believe  that  only  those  learning  workspace  systems  designed 
with  open  architecture,  such  as  VEE,  can  be  extended  easily.  Furthermore,  an  intelligent  agent  system  approach, 
such  as  the  mechanism  of  knowledge  formation  during  learning,  might  be  analyzed  deeply.  In  the  agent-based  VEE, 
learners  can  process  a virtual  experiment  with  agent  cooperatively/competitively.  Hence,  how  to  evaluate  the 
learning  progress  of  learners  is  also  a good  research  topic. 
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Abstract: 

The  as. fit  course  has  been  developed  for  training  office  employees  of 
SMEs  in  the  use  of  advanced  ICTs  for  telecooperation  and  e-commerce. 
It  is  delivered  via  tele-training  to  the  workplaces  of  the  participants, 
using  a combination  of  online  and  offline  media.  The  research  work 
accompanying  this  course  focuses  on  the  question  of  how  to  define  the 
adequate  mixture  of  media  in  tele-learning  and  how  to  integrate  them 
into  a learning  environment  for  effective  training  on  the  job. 


1 The  Course 

Small  and  medium  sized  enterprises  (SMEs)  today  face  the  challenge  of  adapting  to  the  new 
conditions  of  doing  business  in  the  Information  Society  (see  Alpar  1998,  Reichwald  1997).  The 
changes  that  cut  deep  into  the  organisation  of  the  company  can  be  characterized  by  acceleration, 
globahzation  and  virtualization.  The  basic  question  for  SMEs  is:  how  can  they  manage  change 
actively  so  as  to  be  among  the  „Information  Society  winners“? 

One  of  the  preconditions  for  profiting  from  new  opportunities  is  to  master  the  new 
information  and  communication  technologies  (ICT)  and  the  implications  of  their  use.  By 
„mastering“  we  do  not  mean  their  simple  operation,  but  the  skills  and  qualifications  that  are 
necessary  to  integrate  them  into  the  enterprise  and  to  create  management  structures  that  allow 
for  rapid  change  and  flexibility. 

With  the  project  as.fit  diploma  we  offer  building  blocks  of  these  qualifications,  as.fit 
(Advanced  Skills  for  Information  Technology  and  Telecooperation)  is  a tele-training  course  for 
administrative  workers  in  SMEs.  The  principles  of  this  training  are 

• embedding  training  into  the  company:  involvement  of  management,  contents  that  fit 
participants'  needs 

• tele-learning  on  the  job:  use  of  different  technologies  for  interacting  intensely  with  the 
student 

• embedding  learning  into  practice:  problem-oriented  tasks  and  activities;  unity  of  tools  for 
learning  and  working. 

• The  learners  whom  we  address  with  our  training  are  administrative  workers  (primarily 
secretaries  or  office  managers)  of  SMEs.  We  consider  the  administration  of  an  SME  to  be  the 
future  nerve  centre  of  information  management  in  a company.  We  involve  the  company  with 
prehminary  investigation  and  interviews  and  with  accompanying  consultancy  and  support. 

With  its  contents,  the  course  centers  on  today’s  practical  needs  of  companies  on  their  first  steps 
into  telecooperation,  e.g. 

• How  to  communicate  efficiently  via  e-mail 

• The  legal  aspects  of  doing  business  via  the  Internet 

• The  use  of  the  Internet  for  efficient  information  retrieval  and  processing 

• Which  Internet  services  are  appropriate  and  effective  for  the  marketing  of  the  company’s 
products 

• The  role  and  purpose  of  the  company’s  website,  and  consequently  its  design  and  contents. 
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• The  choice  of  adequate  communication  technologies  for  the  company’s  internal,  business-to- 
business  and  external  communication  needs  (e-mail,  video-conferencing,  maihng  lists, 
newsgroups,  chat,  interactive  web  pages  etc.) 

These  issues  are  treated  in  ten  modules,  varying  in  length  from  one  to  three  weeks.  The  duration 
of  the  complete  course  is  20  weeks.  It  contains  only  three  face-to-face  stages  - at  the  beginning,  in 
the  middle,  and  at  the  end,  respectively.  This  shows  that  the  course  relies  heavily  on  tele- 
learning. 


2 Pedagogical  Setting 

2.1  Motivation  for  tele-learning 

The  pedagogical  setting  we  chose  for  this  course  is  tele-learning  in  a combination  of 
communication  media  and  self-paced  study.  By  tele-learning  we  here  mean  distance  learning 
supported  by  electronic  media,  or  - the  other  way  round  - „the  application  of  telematics  for 
learning-related  purposes"  (Colhs  & Fisser  1998). 

The  main  motivation  for  delivering  this  course  via  tele-learning  is  to  take  training  to  where 
it  is  immediately  useful.  This  has  always  been  the  aim  of  „ training  on  the  job",  e.g.  in  models  of 
apprenticeship.  In  the  ideal  case,  learning  and  practising  merge  imperceptibly.  This  ideal  case, 
however,  demands  constant  presence  of  a trainer  and  training  for  individuals  or  very  small 
groups.  Tele-learning  can  recreate  some  elements  of  training-on-the-job  for  larger  and  dispersed 
learning  groups.  This  requires  intensive  communication  with  various  tools  and  individualized 
coaching  of  the  learner  in  the  transfer  from  training  to  practice  (Baumgartner  & Payr  1998). 


2.2  Technologies  used 

The  range  of  technologies  and  media  that  we  use  in  the  course  is  broad: 

• VideO'Conferencing:  students'  and  trainers'  PCs  are  equipped  with  desktop  video  systems 
connected  via  ISDN  hnes  that  are  also  used  for  telephony  and  Internet  access.  The  result  is 
not  high-quahty  image  and  sound,  but  experiences  (Wiendieck  et  al.  1996)  have  shown  that 
the  quality  is  sufficient  to  create  a „presence“.  On  the  other  hand,  the  technology  we  use  is  the 
same  that  can  and  will  be  used  in  the  very  near  future  in  any  office  for  communicating  via 
video.  The  conference  is  switched  via  the  multipoint  control  unit  of  a telecom  operator  (Post  & 
Telekom  Austria  PTA).  Video-conferencing  is  used  for  short  lectures  to  the  whole  group  of 
students  (up  to  20  persons),  but  also  for  individual  communication  between  students  and  the 
trainer,  or  among  students.  The  working  groups  that  students  form  to  work  on  given  tasks  or 
projects  will  also  use  video-conferencing  at  times,  with  and  without  the  trainer. 

• Web  server  and  web-based  groupware.  The  material  for  self-paced  study  itself  is  delivered  in 
printed  form  (textbook)  and  on  CD-ROM  (interactive  courseware).  However,  materials  created 
during  the  course  - by  the  trainer  (resumes,  feedback,  additional  explications,  task  sheets  etc.) 
and  by  the  students  (feedback,  results,  solutions,  questions,  criticism,  etc.)  - are  made 
available  on  the  web.  For  this  purpose,  we  use  a simple  groupware  system  (GMD’s  BSCW  = 
Basic  Support  for  Collaborative  Work)  with  the  typical  basic  functions  of  groupware:  shared 
and  versioned  documents,  closed  user  group,  and  a discussion  forum. 

• E-mail  and  mailing  lists  come  in  as  those  media  that  allow  „pushing''  information  instead  of 
the  „puH''  principle  of  the  web.  With  these  tools,  we  can  make  sure  to  reach  the  students 
immediately,  and  vice  versa. 

• Chats  complement  video-conferencing  as  another  synchronous  communication  tool.  Chats  are 
used  mainly  in  group  work. 
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The  following  table  (table  1)  lists  the  technologies  used  in  as.fit  by  the  characteristics  of 
communication: 


Communication 

synchronous 

asynchronous 

1:1 

phone,  video-telephony,  chat 

e-mail 

l:n 

video-conferencing 

mailing  list 

n;n 

video-conferencing,  chat 

discussion  forum 

Table  1:  Communication  channels  and  media  used  in  as.fit 


2.3  The  Training  Scenario 

The  basic  „unit“  of  the  course  is  the  weekly  lesson  centered  around  a specific  issue.  The  lesson  is 
introduced  in  a short  video-conference  with  all  participants,  where  the  contents  and  tasks  of  the 
week  are  presented.  During  the  following  two  working  days,  students  work  on  the  subject  using 
their  textbook  and,  where  available,  the  interactive  courseware  developed  specifically  for  the 
course  and  distributed  on  CD-ROM.  Task  sheets  and  questionnaires  for  individual  work  can  be 
downloaded  from  the  groupware  server.  Questions  and  requests  for  help  and  clarification  can  be 
sent  in  via  e-mail  (to  the  trainers  only)  or  circulated  via  the  mailing  list  (for  the  attention  of  all 
participants). 

The  second  general  video-conference,  halfivay  through  the  week  (and  the  lesson)  gives 
answers  and  additional  information  on  the  questions  and  issues  raised  by  the  students.  By  that 
time,  the  first  part  of  the  week's  tasks  have  already  been  done  and  sent  in,  so  that  a prehminary 
overview  of  the  students*  progress  is  available.  The  second  video-conference  also  sets  out  the 
second  part  of  the  week's  workload.  The  final  video-conference  coincides  with  the  first  one  of  the 
following  lesson:  the  results  of  the  lesson  are  resumed  and  commented,  before  the  course  goes  on 
to  new  contents. 

Work  in  small  groups  (three  to  five  students)  plays  an  important  role.  It  takes  place  with  and 
without  the  trainers*  participation.  Most  of  the  tasks  contain  elements  of  cooperation  or  at  least 
the  request  to  exchange  and  discuss  results.  Students  use  the  same  technologies  for  their  work  in 
group  as  for  communication  in  the  big  group,  in  particular  e-mail,  chat,  and  video-conferencing. 
Each  student  has  to  take  her  turn  in  leading  and  animating  groupwork,  e.g.  by  setting  up  a video- 
conference,  requesting  contributions,  editing  final  texts  that  have  been  written  collaboratively. 


3 The  Challenge  of  Media  Integration 

3.1  Self-paced  vs.  scheduled  training 

The  use  of  both  self-study  materials  with  asynchronous  communication  (distance  learning  model) 
and  scheduled  video-conferences  and  synchronous  chats  (course  model)  is  at  once  a challenge  and 
a necessity. 

Most  participants  are  not  familiar  with  distance  learning  and  don't  possess  the  required 
„learning  skills".  Together  with  a setting  where  learning  takes  place  at  the  workplace,  during 
working  hours,  the  risk  of  students  who  drop  out  because  they  can’t  manage  their  schedule  is 
particularly  high.  So  we  combine  this  method  of  learning  with  scheduled  sessions  and,  especially 
at  the  beginning  of  the  course,  with  a relatively  rigid  weekly  plan  for  responding,  sending  in  work 
results,  etc. 

On  the  other  hand,  the  rather  short  periods  of  time  dedicated  to  self-paced  activities 
require  an  almost  constant  presence  of  the  trainer  team,  because  the  tight  schedule  does  not 
allow  for  long  delays  between  requests  from  students  and  replies  from  the  trainer.  In  this  respect, 
the  course  has  indeed  aspects  of  „training-on-the-job“. 
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3.2  Tools  for  training  vs.  tools  for  work 


The  intensive  use  of  different  communication  technologies  that  we  require  from  the  students 
serves  two  purposes:  one  is,  obviously,  to  allow  for  intensive  trainer- trainee-interaction.  The  other 
is  to  put  the  contents  of  the  course  to  immediate  use.  The  tools  that  are  used  for  learning  are  the 
same  that  the  participants  learn  to  use  for  doing  their  job  efficiently.  They  are  thus  encouraged  to 
practise  immediately  what  they  have  learned.  At  the  end  of  the  course,  they  are  experienced 
users  of  a wide  range  of  Internet  services  and  of  video-conferencing  and  can  support  the 
implementation  of  new  communication  and  e-commerce  practices  in  their  company. 

The  seamless  transition  from  learning  tools  to  working  tools  is  taken  into  consideration  in 
the  choice  of  software  tools  for  the  delivery  of  the  course.  For  example,  we  refrained  from  using 
dedicated  tele-training  systems  that  combine  e.g.  video-conferencing  classroom  tools  with 
groupware  and  discussion  functions.  We  use  instead  nothing  but  the  software  that  is  installed 
together  with  the  desktop  video  system,  separate  mailing  lists,  Internet  chat,  and  a separate  web- 
based  groupware  system. 


3.3  Classroom  vs.  group  vs.  individual  training 

The  first  generation  of  tele-training  via  video-conferencing  appears  today  like  the  revival  of  the 
„good  old  classroom  teaching“  with  the  addition  of  a new  technology.  The  reason  for  this  was 
partly  the  special  equipment  that  was  needed  - or  considered  to  be  needed  - for  video- 
conferencing,  i.e.  TV-studios  or  studio  equipment  with  several  cameras,  professional  sound 
engineering,  and  the  hke.  The  other  part  of  the  reason  have  been  and  still  are  educational 
traditions,  where  the  first  thing  that  came  to  mind  as  the  use  for  video-conferencing  was  the 
classical  „lecture“  that  could  be  followed  simultaneously  on-site  and  at  one  or  two  remote  sites. 

Only  gradually  it  became  clear  that  the  new  technology  both  requires  and  enables  new 
methods  of  teaching.  Video  and  sound  quahty  in  themselves  are  not  what  tele-training  aims  at, 
but  a new  quality  of  remote  social  learning.  Video-conferencing  is  an  educational  tool  that  has  to 
be  available  to  both  trainers  and  trainees  to  establish  virtual  learning  communities.  Students  in 
the  as.fit  course  are  therefore  encouraged  to  use  video-conferencing  not  only  to  participate  in  the 
„classroom“  sessions,  but  also  and  even  more  to  establish  individual  contacts  among  the  student 
group  and  to  collaborate  on  their  coursework. 


3.4  Online  communication  vs.  o£Qine  courseware 

It  is  clear  that  even  in  a tele-training  course  that  relies  heavily  on  synchronous  and  asynchronous 
communication  technologies  like  this  one,  individual  study  stdl  takes  up  most  of  the  participants^ 
study  time.  The  „bulk“  of  the  subject  matter  to  be  dealt  with  is  therefore  de  five  red  quite 
traditionally  on  paper.  In  our  case,  there  is  no  added  value  in  using  e.g.  the  Web  for  delivering 
text  material.  In  this  reasoning,  we  follow  a simple  principle:  use  each  medium  for  what  it  is  best 
for.  And  paper  is  stdl  the  best  medium  to  read  from,  in  a situation  where  none  of  the  advantages 
of  web-based  texts  supersedes  - e.g.  making  texts  available  to  anonymous  readers  at  unspecified 
locations  and  times. 

We  also  use  courseware  that  we  have  developed  specifically  for  this  training  measure. 
While  a non-negligeable  part  of  the  development  effort  went  into  this  courseware,  its  place  in  the 
range  of  educational  media  is  again  restricted  to  a specific  role  that  cannot  be  fulfilled  by  other 
media:  each  courseware  module  presents  a problem  that  students  are  confronted  with  in  their 
daily  work,  e.g.  how  to  deal  with  an  order  that  comes  in  via  e-mail,  or  how  to  search  the  Internet 
efficiently,  how  to  organise  the  findings  and  make  them  available.  The  courseware  thus  offers  the 
„missing  link"  between  the  factual  knowledge  in  the  textbook  and  the  practice  that  students 
acquire  by  using  the  „real“  software  tools  for  their  coursework  and  their  daily  work. 
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The  method  of  learning  that  the  courseware  focuses  on  is  problem-solving  and  decision- 
making (cf.  Baumgartner  & Payr  1994).  In  each  situation,  the  student  is  offered  several  choices 
for  solving  it.  Different  alternatives  can  be  followed  and  can  lead  to  a solution  - as  in  real  life, 
there  need  not  be  exactly  one  „right“  answer  or  procedure.  The  cognitive  map  of  each  module  is 
therefore  represented  dynamically  by  a complex  graph. 
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Figure  1:  Dynamic  decision  tree  as  clickable  cognitive  map  of  the  Fitzcarraldo  courseware 


Each  node  of  this  graph  consists  of  a graphical  representation  of  the  contents  (currently  a screen, 
document,  or  drawing  — but  also  images  and  video  clips  are  possible)  on  which  the  two  actors,  a 
„trainer“  and  a „trainee“  discuss,  offer  comments,  questions  and  answers. 


4 Questions  for  Research 

From  the  point  of  view  of  research,  we  set  up  a „real-life  experiment"  with  this  tele-learning 
setting.  Most  projects  of  this  kind  have  to  deliver  „real“  and  possibly  efficient  tele-training 
without  a pre-existing  basis  of  acquired  knowledge  and  rules  on  how  to  develop  and  use  the 
different  media  that  are  available  today.  So  we  cannot  speak  of  a classical  experiment  where  the 
various  parameters  can  be  changed  at  will  and  where  failure  is  allowed  as  a valuable  source  of 
knowledge.  We  are  rather  in  the  domain  of  design,  where  modification  and  adaptation  are 
possible  underway,  but  where  the  basic  hjqDotheses  have  been  set  in  advance. 


4.1  Finding  the  right  ,4nedia  mix^^ 

The  main  research  question  is:  what  is  the  most  appropriate  combination  of  media  in  this  setting 
for  tele- learning?  How  much  communication,  scheduUng,  individual  coaching  and  tele-cooperation 
is  necessary,  sufficient  and  desirable  for  creating  a remote  social  learning  environment  that 
supports  learning  efficiently?  What  are  the  preconditions  for  successful  tele-learning?  What  is  the 
adequate  „mix"  of  online  and  offline,  of  synchronous  and  asynchronous,  textual  and  visual  media? 
The  aim  is  to  offer  a model  for  tele-training  that  can  be  applied  to  other  contents,  target  groups 
and  social  settings. 
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4.2  Methods 


Progress  of  the  students  is  continuously  monitored  through  the  course  work  they  have  to  send  in, 
the  requests  for  support  they  submit,  their  communication  activities  with  the  trainer  team  and 
with  other  students.  Texts  (e-mails,  contributions  to  lists  and  chats,  etc.)  as  well  as  video-tapings 
of  the  conferencing  sessions  are  analysed  using  quahtative  methods,  drawn  mainly  from  discourse 
and  conversational  analysis  (Schmitt  1997,  Weinig  1996).  With  these  methods,  we  expect  to  gain 
insights  into 

• the  specific  nature  of  remote  didactic  interaction 

• the  conditions  for  successful  training  via  video-conferencing 

• the  conditions  for  successful  learning  on  the  job 

• the  conditions  for  the  viabifity  of  virtual  learning  communities 

• the  appropriate  combination  of  educational  media  in  different  tele-learning  settings 

At  the  time  of  this  Conference,  a pilot  course  will  have  been  completed  and  analysed.  Two  regular 
courses  are  under  way  at  different  stages,  modified  and  adapted  according  to  results  from  the 
pilot  phase.  The  presentation  therefore  focusses  on  the  latest  experiences  and  empirical  results. 
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Abstract:  Emerging  approaches  to  developing  rich  learning  environments  combine  multimedia,  computer  mediated 
communication,  and  a host  of  interactive  strategies  to  connect  people  in  varied  and  robust  ways.  But  traditional 
understandings  of  learning  environments  and  interaction  usually  stop  short  of  the  kind  of  engagement  that  will  allow 
learning  communities  to  form.  This  paper  examines  theoretical  and  practical  issues  around  promoting  the  growth  of  virtual 
learning  communities.  It  considers  issues  surrounding  virtual  learning  communities  that  can  emerge  using  communication 
technologies  in  formal  and  informal  learning  environments. 


The  metaphor  of  community  has  been  used  to  describe  a wide  range  of  contexts,  from  communities  of 
practice  in  the  corporate  world  (Brown  & Isaacs,  1995;  Wenger,  1998)  to  virtual  community  networks  (Brook 
& Boal,  1995;  Horn,  1997;  Rheingold,  1993;  Schuler,  1996).  Given  the  right  circumstances,  any  of  the  range  of 
communities  can  act  as  learning  communities,  typically  when  they  engage  purposefully  in  the  acquisition, 
transformation  or  creation  of  knowledge.  But  these  entities  do  not  focus  exclusively  on  learning.  For  example,  a 
community  of  practice  may  give  much  of  its  energy  to  identifying  tacit  knowledge  in  a corporate  setting  and 
making  the  tacit  knowledge  explicit  for  the  good  of  the  company  (Wenger,  1998).  Similarly,  virtual  or  online 
communities  may  devote  their  resources  to  connecting  people  in  new  ways,  and  the  substance  of  the 
connections  may  include  anything  from  sharing  information  to  building  interpersonal  relationships  (Cohill, 
1 997),  but  they  may  exclude  learning  entirely. 

Regardless  of  their  focus,  little  is  known  about  how  people  in  virtual  environments  are  influenced  by 
those  environments.  We  glibly  suppose  that  people  who  are  connected  electronically  are  enriching  their 
interpersonal  network  of  relationships,  whereas  some  emerging  research  suggests  that  electronic  saturation  may 
actually  contribute  to  a sense  of  isolation  among  participants  (Kraut,  Paterson,  Lundmark,  Kiesler, 
Mukophadhyay,  & Scherlia,  in  press).  Social  critics  of  technology  voice  concerns  that  the  values  and  strengths 
of  communities  are  undermined  by  the  very  technology  that  promotes  new  ways  of  interacting  with  others 
(Ehrenfeld,  1996;  Selznik,  1996).  With  equal  force,  critics  of  distance  learning  suggest  that  technology-based 
courses  emphasize  transmission  of  information  and  isolate  learners  by  placing  technological  barriers  between 
learners  and  real  people  (Farrow,  1999). 

In  this  paper  I will  argue  that  what  may  be  most  restricting  about  electronic  types  of  learning 
environments  is  that  they  fail  to  promote  a sense  of  community,  that  they  remain  interactive,  yet  fall  short  of 
becoming  communities  of  learners.  When  technology  is  introduced  to  learning  communities,  there  is  a risk  of 
promoting  interaction  without  the  concomitant  elements  required  to  turn  a virtual  learning  environment  into  a 
virtual  learning  community,  and  this  paper  will  explore  what  those  elements  are. 


A Definition  of  Virtual  Learning  Community 

Drawing  on  the  definition  by  Kowch  and  Schwier  (1997),  communities  are  collections  of  individuals 
who  are  bound  together  for  some  reason,  so  a learning  community  emerges  when  people  are  drawn  together  to 
learn.  A learning  community  is  a group  of  individuals  engaged  intentionally  and  collectively  in  the  transaction, 
or  transformation  of  knowledge.  Although  learning  communities  emphasize  outcomes  in  education,  their  power 
resides  in  their  ability  to  take  advantage  of,  and  in  some  cases  invent  a process  for  exchanging  ideas  and 
learning  collectively.  Virtual  learning  communities  happen  when  the  process  of  learning  takes  place  outside  the 
boundaries  of  face  to  face  contact,  typically  electronically.  For  the  purpose  of  this  paper,  I want  to  use  the 
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broadest  possible  notion  of  learning.  Learning  as  discussed  here  may  include  anything  from  formal  education 
to  very  informal  forums  of  exchange — any  context  that  people  can  use  to  learn  something  of  benefit  to  them, 

A virtual  learning  community  usually  depends  on  the  participation  of  relatively  autonomous, 
independent  individuals  who  are  mutually  engaged  in  the  learning  process.  Outside  mandatory  attendance  in 
some  school  settings,  participants  can  not  only  leave  the  community,  they  can  sometimes  participate  in  the 
community  without  revealing  who  they  are  to  the  other  participants.  Autonomy  and  independence  present 
difficult  challenges  for  educators  who  want  to  build  and  maintain  a learning  community. 


Growing  Virtual  Learning  Communities 

How  does  one  begin  a virtual  learning  community?  First,  we  need  to  realize  that  building  a community 
is  not  an  organizational  engineering  problem  — the  problem  is  one  of  motivating  participants  to  create  a 
community  and  giving  them  an  opportunity  to  do  it,  Clark  (1998)  uses  the  apt  term  “growing”  in  place  of 
“building”  or  “constructing”  to  emphasize  the  organic  nature  of  doing  this  kind  of  work.  Ultimately, 
communities  are  built  or  dismantled  by  those  in  the  communities,  not  by  the  people  organizing  or  managing 
them.  It  is  therefore  a matter  of  providing  an  appropriate  structure  and  sufficient  support  — the  conditions  for  a 
community  to  develop.  Some  of  these  conditions  include: 

• A leader  or  leaders.  The  leader  sets  the  agenda  and  the  tone  for  the  virtual  learning  community, 
and  is  the  person  known  to  all  of  the  members  of  the  community  as  the  touchstone  for  protocol 
and  administrative  issues 

• Transparent  support  technologies  (Norman,  1 998),  Transparent  technologies  allow  participants  to 
concentrate  on  the  tasks,  relationships  and  ideas  at  hand.  Extensive  usability  testing  (see  Nielsen, 
1994)  should  be  implemented  at  each  stage  of  development  to  with  the  explicit  goal  of  increasing 
the  transparency  of  the  interface, 

• A safe  and  open  mixture  of  interpersonal  contact.  People  need  to  feel  comfortable  to  participate, 
and  unless  the  invitation  to  participate  is  explicit,  and  the  boundaries  of  acceptable  behavior  are 
shared  and  understood,  people  will  not  be  as  likely  to  take  risks  in  their  communication  with  other 
members  of  the  community.  It  is  reasonable  to  publish  written  codes  of  conduct  to  keep 
communities  on  track  (Bruckman,  1996), 

• Liberal  use  of  narrative.  Clark  (1998)  makes  a persuasive  case  for  the  use  of  narrative  in  virtual 
learning  communities.  He  argues  that  because  virtual  communities  are  worlds  of  words,  then  the 
stories  we  tell  are  a powerful  and  appropriate  vehicle  of  expression. 

Elements  of  Community 

Selznick  (1996)  identified  seven  elements  of  community-features  that  must  be  present  to  give  an  entity  the 
label  of  community.  Selznick's  elements  underscore  the  idea  that  communities  are  complex.  Any  adequate 
description  of  virtual  learning  communities  needs  to  address  these  elements,  and  recognize  that  they  are 
variables  that  interact  multidimensionally.  A successful  community,  actual  or  virtual,  will  exhibit  a balance  of 
these  elements. 

• Historicity,  Communities  are  stronger  when  they  share  history  and  culture.  Conversely,  they  are  weak 
when  they  are  based  on  general  interests  and  abstract  ideas, 

• Identity.  Communities  foster  a sense  of  shared  identity.  Successful  virtual  learning  communities  need  to 
have  boundaries — an  identity  or  recognized  focus, 

o Mutuality.  Communities  spring  from,  and  are  maintained  by  interdependence  and  reciprocity.  Participants 
construct  purposes,  intentions  and  the  protocol  for  interaction, 

• Plurality,  Communities  draw  much  of  their  vitality  from  "intermediate  associations"  such  as  families, 
churches,  and  other  peripheral  groups, 

• Autonomy,  Within  the  emphasis  on  group  identity,  it  is  important  that  communities  respect  and  protect 
individual  identity. 

• Participation.  Social  participation  in  the  community,  especially  participation  that  promotes  self- 
determination  supports  autonomy  and  sustains  the  community. 
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• Integration.  All  of  the  above  elements  depend  on  supportive  norms,  beliefs  and  practices. 

It  is  probably  apparent  from  this  list  that  these  elements  are  not  realized  by  chance.  Communities  do 
not  just  happen.  But  neither  are  they  created.  What  we  are  attempting  to  do  as  educators  is  promote  the 
development  of  virtual  learning  communities  by  nurturing  the  conditions  under  which  they  can  arise.  A 
sampling  of  ideas  for  promoting  virtual  learning  communities  is  presented  in  Table  1 . 


Element 

Implications  for  Virtual  Learning  Communities 

Historicity 

Incorporate  what  members  have  done  in  the  past,  and  make  their  stories  part  of 
the  community  culture.  Explicit  mention  of  the  culture,  value  and  context  of  the 
virtual  community.  Make  public  the  history  of  the  community. 

Identity 

Use  team-building  exercises,  develop  community  logos,  and  publicly 
acknowledge  accomplishments  by  the  group  and  individual  members  within  the 
community.  Articulate  the  focus  or  purpose  of  the  community,  and  outline  the 
requirements  and  rituals  accompanying  membership  in  the  community. 

Mutuality 

Include  group  exercises,  assignments,  activities  that  require  each  member  to 
contribute  to  the  final  product.  Ask  leading  questions  that  encourage  members  of 
the  community  to  invest  in  concerns  held  by  other  members,  and  to  share  ideas 
and  possible  solutions. 

Plurality 

Encourage  membership  and  participation  from  and  association  with  groups 
related  to  the  learning  focus.  These  might  include  businesses,  professional 
associations,  or  groups  in  other  countries  exploring  similar  issues. 

Autonomy 

Foster  individual  expression  and  comment  explicitly  on  its  value.  Set  up 
protocol  for  respectftil  communication  and  reach  consensus  in  the  group.  Create 
strategies  for  settling  disputes  or  inappropriate  behavior. 

Participation 

Allow  members  of  the  group  to  shape  learning  agendas.  Give  guidance  to  new 
community  members,  and  promote  opportunities  for  established  members  to  go 
outside  the  boundaries  of  the  learning  event  or  focus.  Encourage  lurkers  and 
voyeurs. 

Integration 

Articulate  a set  of  belief  statements,  and  identify  group  norms  as  they  emerge 
and  evolve.  Adopt  and  firmly  adhere  to  a learner-centered  philosophy,  and 
employ  pedagogy  that  celebrates  individuals  while  building  a group  identity. 

Table  1,  Representative  implications  of  Selznik’s  elements  for  virtual  learning  communities. 


Lifecycles  of  Virtual  Learning  Communities 

It  is  reasonable  to  think  of  learning  communities  as  having  a life,  one  that  goes  through  fairly 
predictable  stages.  There  are  at  least  three  distinct  stages:  the  formative  stage,  the  mature  stage,  and  the 
metamorphosis/declining  stage. 

The  formative  stage  in  the  life  of  a virtual  learning  community  is  characterized  by  the  attraction  of  new 
members.  The  identity  of  the  community  is  malleable,  and  participants  are  typically  somewhat  tentative  as  they 
try  out  communicating  and  making  connections  with  other  community  members.  This  stage  of  development 
requires  a great  deal  of  leadership.  The  leader  at  this  stage  is  trying  to  set  the  tone  of  the  community,  attract  and 
welcome  new  members  to  the  community,  and  lay  out  the  purpose  and  guidelines  for  participation  with  the 
group  as  it  forms.  At  this  point,  the  virtual  learning  community  will  be  evolving  from  what  its  creators  first 
imagined  into  what  it  will  ultimately  become.  The  purpose  may  change,  expand  or  constrict,  and  it  is  the  first 
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place  in  which  the  members  will  either  successfully  or  unsuccessfully  impose  their  will  on  the  makeup  of  the 
community.  Users  will  test  the  boundaries  of  the  community  and  determine  whether  they  will  remain  as 
members.  If  required  to  be  members,  they  will  be  deciding  how  significant  the  community  will  be  to  them,  or 
how  they  can  shape  it  into  something  they  can  use.  In  all,  it  is  a time  of  testing,  negotiating,  and  shaping,  and 
the  match  between  the  purpose  of  the  community  and  the  importance  of  that  purpose  to  members  will  determine 
the  length  of  its  survival  and  the  strength  of  its  influence. 

The  mature  stage  of  life  in  the  virtual  community  is  ultimately  achieved  once  the  purpose,  shape  and 
operation  of  the  community  are  settled.  At  this  point  the  leader  doesn’t  have  to  play  as  central  a role  in 
negotiating  the  purpose  and  monitoring  the  activities  of  members.  The  purpose  and  codes  of  conduct  are 
known,  and  the  members  of  the  community  exercise  their  control  over  the  community  by  doing  much  of  the 
monitoring.  In  later  stages  of  maturity,  the  community  is  more  institutionalized  and  entrenched.  Codes  of 
conduct  are  more  rigid,  as  are  the  boundaries  around  acceptable  topics  and  modes  of  expression.  A mature 
community  may  start  to  take  on  the  trappings  of  terrestrial  communities,  and  become  much  more  formal  in  its 
operation.  This  may  be  characterized  by  the  introduction  of  some  form  of  governing  body  or  fund  raising 
activities,  for  example. 

Ultimately,  most  virtual  communities  will  be  challenged  to  change,  to  go  through  a metamorphosis  and 
become  a new  entity  with  a focus  that  is  different  from  the  original  conception  of  what  the  virtual  learning 
community  would  be.  It  is  possible  that  one  indication  of  this  change  will  be  resistance  to  change — with  the 
focus  of  those  who  manage  the  virtual  community  on  the  preservation  of  it;  the  focus  on  maintaining  the 
organization  itself  rather  than  on  extending  its  purpose  or  mission.  One  of  the  possibilities  at  this  stage  of  life 
is  that  the  virtual  learning  community  has  made  its  contribution  and  is  now  in  a state  of  natural  decline.  There 
may  be  the  rare  virtual  community  that  becomes  so  entrenched  that  it  will  survive  without  significant  change, 
but  most  virtual  communities  will  face  greater  volatility. 


Research  Issues  Raised  by  Virtual  Learning  Communities 

If  educators  choose  to  support  the  development  of  virtual  learning  communities,  a number  of  issues 
emerge.  Some  issues  are  financial  and  logistic — how  does  one  assemble  the  technological  and  personal  systems 
necessary  to  construct  and  maintain  a communication  system.  But  the  more  important  questions  centre  around 
the  design,  implementation,  pedagogy  and  effects  of  virtual  learning  communities — the  socio-educational 
aspects  of  learning  through  this  means  of  communication.  A few  of  these  issues  which  invite  investigation  are 
listed  below,  although  many  more  seem  to  arise  every  day. 

• How  do  people  select  virtual  learning  communities  and  how  do  they  make  use  of  them  for  learning? 

• How  do  voluntary  members  of  virtual  learning  communities  differ  from  those  who  are  assigned  to  learning 
communities  in  formal  educational  contexts? 

• Are  there  rules  of  engagement  or  particular  protocols  for  insinuating  an  individual  into  the  fabric  of  a 
virtual  learning  community,  and  is  the  process  contextually  or  culturally  bound?  Does  the  process  mirror 
interpersonal  group  learning  contexts? 

• Are  there  power  relationships  in  virtual  learning  communities,  and  how  do  they  interact  with  learning 
variables? 

• Do  Selznik’s  elements  of  community  exhibit  themselves  in  virtual  learning  communities,  and  do  they 
inform  our  understanding  of  how  these  communities  contribute  to  learning  environments? 

• What  is  the  nature  of  learning  in  virtual  contexts,  and  how  do  architects,  active  members  and  spectators 
describe  their  experiences? 

• What  value  do  administrators,  educators  and  learners  place  on  virtual  learning  communities?  How  do 
political,  social,  educational  and  personal  agendas  interact  in  the  development  maintenance  and  alteration 
of  virtual  learning  communities. 

• Do  virtual  learning  communities  exhibit  lifecycles,  and  what  significance  does  this  have  for  their  design? 


321 


What  are  the  pedagogical  issues  involved  in  virtual  learning  communities?  How  can  they  be  used  for 
teaching? 
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Algorithm  Animations  supporting  the  Education  in  Distributed  Systems 


Abstract:  In  order  to  support  the  education  in  distributed  system  fundamentals  and  to 
elucidate  the  function  of  distributed  algorithms  and  protocols  we  apply  animations  of  formal 
models  which  concentrate  on  the  mediation  of  aspects  of  the  dynamic  system  behaviour.  For 
this  purpose  we  adapted  and  extended  approaches  and  tools  supporting  the  animation  of 
sequential  algorithms.  Besides  of  sets  of  special  animations,  the  results  comprise  a set  of 
view  types  providing  for  the  convenient  presentation  of  dynamic  properties.  Furthermore  we 
report  on  experiences  we  gained  from  a series  of  corresponding  educational  experiments. 


Introduction 

With  respect  to  the  general  introduction  into  distributed  systems  the  experience  shows,  that  many  students 
have  problems  to  grasp  abstract  conceptions  such  as  concurrency,  partially  ordered  causality  structures,  and 
nondeterminism.  Moreover,  even  less  abstract  characteristics  of  distributed  systems  - like  allocation  of 
activities  and  resources  to  several  sites,  message  exchange  based  cooperation  mechanisms  between 
autonomous  parties,  distributed  state  components,  loose  synchronisation,  variable  transit  delays  and  faults  in 
unreliable  communication  systems  - cause  problems  since  their  demonstration  mostly  has  to  be  based  on  non- 
trivial example  systems  which  tend  to  be  difficult  to  survey.  In  particular,  subjects  concerning  the  dynamic 
behaviour  of  distributed  systems  tend  to  overtax  the  imagination  of  students.  Therefore,  we  developed 
animations  of  distributed  algorithms  and  communication  protocols  in  order  to  complement  the  traditional 
means  of  education  by  facilities  which  enable  students  to  experience  dynamic  behaviours  of  models.  The 
animations  concentrate  on  the  elucidation  of  distributed  system  characteristics,  on  the  design  of  suitable  views 
supporting  the  understanding  of  distributed  systems,  and  on  the  exemplification  of  protocols  and  algorithms. 

The  animation  of  distributed  systems  is  based  on  results  imported  from  the  research  field  of  sequential 
algorithm  animation  (Brown  1987,  Stasko  1998).  In  particular,  we  use  the  Zeus  algorithm  animation  system 
(Brown  1992)  which  supports  the  animation  of  Modula-3  based  algorithm  implementations  (Harbison  1992). 
Furthermore,  there  are  animations  accessible  via  World  Wide  Web  which  are  implemented  as  Java-applets  in 
HTML-pages  (Arnold  et  al.  1998,  Ragett  1998). 

The  objective  of  supporting  the  education  in  distributed  systems  by  animations  was  pursued  by  a series  of 
experiments.  We  started  with  the  ZADA  project  (Mester  w/o  date).  It  was  performed  in  the  course  of  a so- 
called  group  project  where  about  10  students  cooperate  during  one  year  in  order  to  achieve  a development 
project  goal  and  to  experience  team  work  and  cooperation  means.  The  student  group  extended  and  applied  the 
Zeus  system  and  developed  a set  of  protocol  and  algorithm  animations.  In  particular,  they  designed  suitable 
types  of  views  supporting  the  visual  presentation  of  the  animations.  The  team  presented  its  results  to  the 
audience  of  a fundamental  distributed  system  lecture  and  learned  from  the  feedback.  The  ZADA  project 
supplied  relatively  universal  animation  platforms  to  students  and  suggested  to  them  to  design  animations  and 
to  perform  experiments  for  their  own.  In  comparison  to  this,  the  provision  of  well-prepared  and  well-designed, 
animations  can  structure  and  accelerate  the  learning  process.  Especially,  it  can  help  students  to  gain  a first 
access  to  new  subjects.  Presently  we  therefore  offer  World  Wide  Web  accessible  lecture  note  complements 
which  introduce  into  the  field  of  distributed  systems  and  provide  for  applets  animating  the  execution  of 
distributed  algorithms  and  protocols  (Krumm  1997).  In  the  distributed  system  lecture  references  to  the 
complements  are  made.  The  feedback  from  oral  examinations  is  very  positive. 

Algorithm  Animation 

Algorithm  animation  is  a section  of  software  visualization,  which  Price  et  al.  define  as  "Algorithm 
visualization  or  animation  is  understood  to  be  the  visualization  of  a high-level  description  of  a piece  of 
software,  which  is  in  contrast  to  code  or  data  visualization  (which  are  collectively  a kind  of  program 
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visualization)  where  actual  implemented  code  is  visualized."  (Price  et  al,  1993),  Algorithm  animation  is 
typically  used  as  learning  aid,  as  high-level  debugging  aid,  and  as  design  aid. 

Most  algorithm  animation  systems  (e,g.  Zeus  (Brown  1992),  Tango/Xtango  (Stasko  1992)  with  several 
successors)  provide  for  execution  of  sequential  algorithms,  interactive  control,  e.g,  at  least  different  speeds, 
single  stepping,  breakpoints,  and  algorithm  input  determination,  as  well  as  the  mapping  between  the  state 
and/or  the  actions  of  the  algorithm  onto  the  state  and/or  actions  of  some  or  all  of  its  visualizations. 
Furthermore  it  synchronizes  between  the  algorithm  and  its  visualizations.  Animation  designers  are  supported 
with  an  animation  library  leveraging  the  effort  of  building  sophisticated  views  (including  animated  graphics, 
3D,  sound).  Animations  built  ad-hoc  without  an  animation  system  would  have  to  reimplement  major  features 
of  this  environment. 

The  architecture  of  Zeus  separates  between  the  algorithm  and  the  views  of  the  animation.  One  has  to  annotate 
the  source  code  of  the  algorithm  (Zeus:  Modula-3)  to  be  animated  at  ’interesting’  (i.e.,  animation  relevant) 
locations  with  directives  to  emit  a stream  of  ’interesting  events’  to  the  animation  system  on  execution.  The 
animation  system  then  filters  the  events  according  to  the  users  interests,  distributes  the  events  to  connected 
views  (separate  windows)  and  synchronizes  among  them,  i.e,,  assures  the  concurrent  handling  of  the  same 
event  in  all  views.  Events  normally  cause  display  changes  in  views.  The  algorithm  system  may  also  provide  for 
a channel  back  into  the  algorithm,  i.e.,  manipulations  of  program  data  displayed  in  views  effect  changes  of 
program  data  variables. 

Still  missing  are  systematic  guidelines  for  successful  visualization,  though  some  empirical  studies  about 
specific  aspects  exists  (e.g.,  Stasko  1997),  The  current  trend  in  algorithm  animation  is  its  seamless  integration 
into  Web-based  electronic  textbooks  / courseware,  which  also  allow  for  other  types  of  media.  Another  trend  is 
to  provide  for  collaborations  between  learners  (e,g,,  Brown  et  al,  1997,  Guzdial  et  al.  1997). 

Distributed  System  Animation 

In  comparison  with  the  animation  of  sequential  algorithms,  the  animation  of  distributed  systems  imposes 
additional  requirements.  Animations  have  to  handle  the  concurrency  of  activities.  They  must  model  the 
distribution  of  activities  and  resources  to  sites  as  well  as  the  distances  and  transit  delays  between  sites. 
Furthermore,  the  inherent  non-determinism  resulting  from  varying  transit  delays  and  processing  speeds  has  to 
be  managed.  On  the  one  hand,  these  requirements  demand  special  features  of  the  animation  system.  On  the 
other  hand,  they  ask  for  special  design  and  layout  principles  in  order  to  offer  convenient  presentation 
interfaces.  Special  features  of  a suitable  animation  system  are: 

• Distributed  system  definition  language;  it  has  to  support  the  definition  of  processes  with  private  resources 
and  the  definition  of  communication  networks  supporting  the  exchange  of  messages.  Additionally,  the 
definitions  of  the  animation  events  have  to  be  supplied. 

• Scheduler;  the  execution  of  activities  during  animations  has  to  be  controlled  by  a scheduler  component 
which  achieves  the  selection,  synchronisation,  and  management  of  concurrent  and  nondeterministic 
actions.  Preferably,  the  scheduler  provides  for  different  scheduling  strategies  (e.g.,  arbitrary  action 
selection,  systematically  exhaustive  depth-first  or  breadth-first  exploration,  user-defined  action  priorities, 
user-interaction  controlled  selection)  which  can  be  combined  with  several  processing  speed  controls 
(e.g,,  continuous  processing  with  user-chosen  processing  speed,  single-step  execution,  stop  at 
break-points)  and  suitable  experiment  control  strategies  (e.g.,  forward  processing,  backward  processing, 
reset,  choice  of  alternative  execution  sequences,  repetition  of  sequences). 

• Node  and  communication  system  models;  while  distributed  system  nodes  commonly  are  represented  by 
event-discrete  models  which  perform  actions  from  time  to  time,  the  communication  system  preferably 
reflects  the  continuous  propagation  of  signals  over  links.  Therefore  the  execution  machine  has  to 
support  more  fine-grained  communication  system  steps  as  well  as  more  course-grained  node  actions. 

While  concurrency  and  nondeterminism  mostly  result  in  complex  and  intertwined  activity  threads,  the 
presentation  interface  of  the  animations  has  to  provide  for  clear  impressions  which  support  gradual  learning 
processes.  In  particular,  special  view  types  for  distributed  system  animations  are  of  interest.  The  views  shall 
refer  to  common  paradigms  and  emphasize  the  properties  of  interest. 

The  views  influence  the  convenient  perception  of  animations.  Following  main  view  types  were  identified  and 
defined:  a)  Topology;  the  view  depicts  a graph  representing  the  nodes  and  links  of  a network,  b)  Message 
sequence  charts;  time-distance  diagrams  clarify  the  propagation  of  messages,  c)  State  machines;  state  machine 
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diagrams  elucidate  the  control  flow  of  nodes,  d)  Petri  nets;  nets  and  token  games'  can  support  representations 
of  the  system  and  its  concurrent  threads  of  activity  as  a whole,  e)  Code;  often,  details  of  a system  are  preferably 
demonstrated  with  reference  to  its  definition  text  where  the  constructs  of  momentary  interest  are  emphasized. 


(Fig.  1)  presents  the  screen  shot  of  an  animation  of  a 
parallel  network  exploration  algorithm.  The  figure 
exemplifies  the  topology  view  type  which  comprises 
icons  of  nodes,  links,  and  messages.  Message  types  are 
coded  by  color.  Additionally,  the  major  state 
component  values  of  nodes  are  represented  by  the  color 
of  the  node  icons.  The  values  of  essential  minor  state 
components  influence  the  color  of  links  (i.e.,  the 
exploration  state  of  a node  and  the  state  of  the  spanning 
tree  under  construction).  Moreover,  the  bright  points  in 
the  nodes  act  as  indicators  for  minor  state  components 
of  nodes  (i.e.,  the  management  of  exploration  messages 
and  acknowledgements). 

Figure  1:  Topology  view 

(live  applet:  http://ls4-www.cs.iini-dortmund.de/ 

RVS/MA/hk/OrdnerVertAlgo/EchoAlgoText.htmn 


Thus,  (Fig.  1)  also  exemplifies  the  graphical  representation  of  state  component  values.  We  recommend  to  our 
students,  firstly  to  observe  system  executions  in  this  topology  view,  since  it  provides  for  an  intuitive 
understanding  of  distribution,  message  propagation  and  concurrency. 

(Fig.  2)  gives  an  example  of  the  message  sequence  chart  view.  It  also  presents  an  animation  of  the  parallel 
exploration  algorithm.  On  top  of  the  picture  we  see  the  topology  view  again  which  is  known  from  (Fig.  1). 
Mostly,  we  use  the  topology  view  as  recurring  main  view.  A mapping  was  applied  which  linearizes  the 
topology  area  in  order  to  support  the  direct  correspondence  of  x-coordinates  in  topology  and  message  sequence 
chart  view.  On  bottom  of  (Fig.  2)  the  code  view  of  the  animation  shows  the  definition  of  the  last  action  of  the 
selected  node.  The  selected  node  is  highlighted  in  the  topology  view.  Since  the  selection  feature  is  combined 
with  a break  point.  The  animation  stops  whenever  the  selected  node  performs  an  action  and  the  user  can 
investigate  the  definition  text  of  the  action. 

Finally,  (Fig.  3)  depicts  the  state  machine  view.  The  figure  is  a screen  shot  of  an  animation  of  the  Alternating 
Bit  Protocol.  On  top,  the  topology  view  is  used  again  as  main  view.  Here,  the  minor  state  component  values  of 
the  sender  and  receiver  buffer  contents  complement  the  icons  of  the  nodes.  Below,  the  animated  transition 
diagram  of  the  selected  node  represents  a state  machine.  Old  and  new  states,  as  well  as  the  fired  actions  are 
highlighted  by  colored  borders.  Note,  that  the  state  diagram  concentrates  on  few  major  state  components. 
Additionally,  it  is  important,  that  the  action  names  of  the  diagram  directly  correspond  to  the  action  names  of 
the  code  view. 


ZADA  Project 

Within  the  project  1 1 students  and  two  advisors  from  the  distributed  systems  group  tackled  the  problem  of 
making  the  comprehension  of  the  functionality  (i.e.  dynamic  behaviour)  of  distributed  algorithms  easier.  The 
aim  of  the  project  was  the  development  of  methods  and  tools  for  an  utmost  simple  generation  of  effective 
visualizations  of  protocols  and  distributed  algorithms.  As  only  few  original  work  to  this  problem  existed, 
which  mainly  focused  on  parallel  algorithms  or  on  examplary  animations  of  isolated  examples,  this  project 
was  targeted  more  exploratory.  The  group  explored  existing  animations  and  animation  systems  and  quickly 
turned  to  both  didactical  and  technical  matters:  (1)  elaboration  of  guidelines  for  the  effective  visualization,  (2) 
development  of 

an  approach  to  and  a tool  for  the  modelling,  the  execution  and  the  instrumentation  of  distributed  algorithms, 
(3)  selection  and  implementation  of  typical  representatives  for  a broad  range  of  distributed  algorithms  and 
protocols,  and  - based  on  the  didactical  guidelines  - (4)  exploration  of  visualization  facilities  (and 
implementing  them  as  views)  for  this  class  of  algorithms,  as  well  as  (5)  evaluating  meaningful  views  for  the 
algorithms  selected.  To  validate  the  ideas,  (2),  (3),  and  (5)  have  been  bundled  with  the  Zeus  animation  system 
to  form  a collection  of  'Zeus-based  Animations  of  Distributed  Algorithms  and  communication  protocols 
(ZADA)'  (6). 
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(1)  Not  all  that  is  technically  possible  also  is 
didactically  feasible.  Therefore,  a framework  for  the 
systematic  design  of  animations  was  developed.  It 
profited  from  contributions  in  cognitive  psychology  and 
science  of  education  and  covers  topics  like  how  to 
represent  specific  distributed  algorithm  event  classes 
(e.g.  message  send/receive,  loss,  node  state  change), 
how  to  place  views,  how  much  information  can  be 
presented  in  one  view,  in  how  far  views  have  to  be 
explained  or  augmented,  how  much  parallel  activities 
are  feasible,  how  to  help  the  viewer  to  relate  correlating 
activities  in  different  views,  which  colors  and  color 
combinations  fit  best,  what  can  be  supported  with 
sounds,  etc. 

Figure  2:  Message  sequence  chart  and  code  view  (live 
applet:  http://ls4-www.cs.unl-dortmund.de/ 


(2)  For  modelling,  the  Actor-model  has  been  adopted  and  utilized  in  Modula-3.  The  adaption  allows  for  a 
nearly  direct  take-over  of  pseudocode  descriptions  from  common  textbooks  into  a Modula-3  program  for  later 
integration  into  the  animation  system.  For  execution,  Zeus  has  been  selected  as  underlying  animation  system. 
It  has  been  extended  by  an  animation  scheduler  component  which  models  the  propagation  of  messages  in  the 
communication  system  and  triggers  the  actions  of  the  distributed  system  nodes.  For  instrumentation, 
standardized  events  for  distributed  algorithms  have  been  defined  (e.g.  message  send/receive,  node  status 
change),  which  are  raised  automatically  by  the  animation  scheduler  without  the  necessity  of  further  annotation 
of  program  texts. 

(3)  The  selection  of  algorithms  described  abstractly  in  Modula-3  can  be  regarded  as  broad,  since  all  relevant 
algorithm  subclasses  are  represented:  a)  application  oriented:  consistency  of  distributed  data,  distributed 
transaction,  and  distributed  computation,  b)  control:  consensus,  network  exploration,  adoption,  backward  error 
correction,  election,  net  structure  exploration,  shortest  path,  and  termination,  c)  communication  protocols: 

- - - - alternating  bit  and  sliding  window  protocol. 

; I uickB^^nFox  Jtj  sOveKiheLazy  1 1 examination  of  possibilities  for  the  visualization 

^ of  distributed  algorithms  resulted  in  a collection  of 

I Jum  TheO  [ ^ canonical  and  specific  animation  views.  The  views  stem 

from  the  three  animation  view  classes:  direct 

visualization  on  different  abstraction  levels,  functional 
visualization,  and  views  serving  as  overviews.  The 
following  animation  views  were  build:  topology, 
instance,  time/distance,  petri-net,  finite  automata, 
message  queues,  area  codings  (polygon  view),  waves, 
state  trees  over  time,  checkpoints,  and  timer.  Reusable 
views  are  parametrizable  in  their  visualization 
characteristics  at  object  instance-level  (e.g.  legend, 
colors,  icons,  action  triggering  by  events).  (Fig.  4) 
depicts  an  area/polygon  view,  where  the  major  state  of 
the  local  node  is  represented  by  a colored  area  around 
it.  (Fig.  5)  shows  a wave  view  on  the  same  topology, 
where  growing  circles  represent  spreading  knowledge 
about  e.g.  a value. 

Figure  3:  State  machine  view 

(live  applet:  http://ls4-www.cs.uni-dortmund.de/ 

R V S/M A/hk/OrdnerVertAlgo/A ltBitText2.html  ) 
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(5/6)  The  collection  of  animations,  suitable  views, 
guidelines,  modules,  and  experiment  parameter  files  forms 
ZADA.  In  connection  with  animations  it  recommends 
experiment  parameters  to  support  common  learning 
experiences:  Specific  settings  (topologies,  algorithm  input 
data,  initiator  status,  etc.)  have  been  prepared  and 
documented  in  order  to  facilitate  a common  base  for 
discussions  and  individual  explorations. 

The  ZADA  package  primarily  adresses  students  in  higher 
classes  for  computer  networks  and  distributed  systems.  It 
serves  as  interactive  tool  for  exploratory  learning  of 
selected  algorithms.  Mainly,  the  students  will  use  the 
preplanned  experiments.  Moreover,  advanced  students  can 
design  experiments  for  their  own.  They  can  concentrate  on 
the  modelling  of  the  algorithm,  since  re-usable  canonical 
views  and  the  standardized  instrumentation  provide  for  an 
automated  creation  of  animations. 

Figure  4:  Adoption  Area/Polygon  View 


Animations  in  Complementary  Lecture  Notes 

We  offer  two  courses  which  refer  to  animated  models  in  complementary  lecture  notes.  One  course  introduces 
into  computer  networks  and  distributed  systems  in  general.  The  other  course  concentrates  on  distributed 
algorithms,  their  specification,  and  verification.  The  complementary  lecture  notes  are  accessible  via  World 
Wide  Web  (Krumm  1997)  and  can  be  experienced  by  Java-enabled  HTML-browsers.  Presently,  the  notes 
contain  the  following  animations  (introductions  into  these  algorithms  may  be  found  in  (Raynal  1987):  a) 
Parallel  network  exploration,  b)  Serial  network  exploration,  c)  Message  extinction  principle,  d)  Distributed 
snapshot  scenario,  consistency,  and  snapshot  algorithms,  e)  Alternating  Bit  Protocol. 

Each  topic  is  delt  with  a small  series  of  animations  in  few  different  views.  At  the  one  hand,  there  is  a lot  of 
work  necessary  to  develop  convenient  animations.  On  the  other  hand,  we  do  not  want  to  overload  the 
perceiption  of  the  students.  Therefore,  we  presently  provide  for  a relatively  small  set  of  animations  only.  We 

propose  that  the  students  start  with  the  observation  of  a 
system  in  topology  view  in  order  to  get  impressions  of  the 
global  behaviour.  All  other  animations  refer  to  the 
topology  view  as  main  view.  Stepwise  executions 
combining  the  topology  view  with  the  code  view  are  of 
interest  to  introduce  the  formal  system  definition.  It 
supplies  the  definitions  of  the  process  types  containing  the 
declarations  of  the  state  variables  and  the  definitions  of  the 
process  actions.  A combination  of  topology  and  message 
sequence  chart  view  can  clarify  concurrency  and  message 
propagation  issues.  Finally,  state  machine  views  can  help 
to  structure  and  understand  the  local  behaviour  of  nodes. 
(Fig.  1),  (Fig.  2),  and  (Fig.  3)  are  screen  shots  from  such 
animations  and  exemplify  this  sequence  of  views. 

Figure  5:  Wave  View 


All  animations  provide  for  restricted  interaction  facilities.  Students  may  control  the  animation  speed.  They  can 
start,  stop,  and  continue  an  animation.  Moreover  they  can  select  nodes  in  order  to  choose  the  focus  of  state 
machine  and  code  views.  The  code  view  selection  is  combined  with  an  automated  break  after  each  action  of  the 
selected  node.  Additionally,  a general  single  step  mode  is  available.  There  is  a deterministic  scheduling  which 
is  defined  implicitly  with  the  system  definition.  Some  models  provide  for  nondeterministic  execution  variants 
resulting  from  varying  message  delays.  In  general,  we  do  not  want  to  provide  for  an  universal  animation 
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experiment  platform.  We  only  want  to  offer  a small  and  preplanned  set  of  marked  animations  which  mediate 
the  essentials  in  an  efficient  way.  Animated  topology  and  message  sequence  chart  views  shall  support  a first 
access  to  the  properties  of  distribution,  concurrency,  and  nondeterminism.  Animations  of  the  Alternating  Bit 
Protocol  shall  exemplify  the  cooperation  of  protocol  entities  and  their  use  of  basic  channels.  Finally, 
animations  of  snapshot  scenarios  shall  clarify  the  snapshot  problem,  the  role  of  the  causal  dependency  relation, 
and  ist  influence  to  the  concistency  of  snapshots. 

Results  and  Experiences 

We  used  educational  animations  of  distributed  system  models  in  two  different  ways.  We  reported  on  the  work 
of  the  students  of  a project  group  which  deepened  their  knowledge  of  distributed  systems  by  developing  an 
animation  system,  designing  animations,  and  evaluating  the  feedback  of  other  students  to  the  animations. 
Furthermore,  we  outlined  the  World  Wide  Web  accessible  complementary  lecture  notes  which  support  the 
introduction  of  distributed  system  essentials  and  aim  to  the  time-efficient  mediation  of  subjects. 

As  the  experience  showed  so  far,  there  are  the  following  main  aspects  of  animation  application  and  animation 
design  which  should  be  coordinated  with  each  other:  a)  Team  objectives  and  size,  b)  Available  period  of  time, 
c)  Guidance,  d)  Scope  of  experiment  design,  e)  Interaction  facilities,  and  f)  Animation  development 
automation.  For  instance,  the  animations  of  the  complementary  lecture  notes  (cf  Sect.  5)  are  mainly  used  by 
single  students,  who  belong  to  a larger  course  group  but  perform  the  experiments  alone  in  order  to  support 
their  individual  learning.  While  there  is  no  tuition  which  directly  addresses  animation  experiments,  the 
lectures  preferably  rely  on  common  experiences  of  the  students  which  result  from  the  fact,  that  all  students 
experienced  similar  experiments.  To  support  the  similarity  of  experiments,  the  scope  of  experiment  design  is 
relatively  small  and  the  interaction  facilities  are  restricted.  This  is  of  interest  also  because  our  students  tend  to 
be  very  sparing  with  their  learning  time  and  consider  well-prepared  animations  only.  Therefore,  the 
animations  had  to  be  designed  creatively.  Of  course  we  used  a common  framework  supplying  basic  features 
and  guidelines.  In  particular,  however,  the  design  of  suitable  scenarios,  the  selection  of  meaningful  state 
components,  and  the  design  of  striking  icons  could  not  be  automatized.  They  resulted  from  individual  efforts 
and  dominated  the  animation  development. 

Concluding  Remarks 

In  summary,  our  experiences  plead  for  the  provision  of  animations  for  the  education  in  distributed  systems.  In 
order  to  support  more  fundamental  courses,  however,  the  animations  have  to  be  designed  very  carefully  and 
therefore  bind  many  development  resources.  Hence,  the  exchange  of  animations  between  universities  and 
cooperative  developments  are  of  great  interest.  They  can  be  supported  by  World  Wide  Web  and  Java 
technologies,  which  also  support  access  for  students  learning  at  home. 
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Abstract 

The  degree  to  which  technology  can  serve  the  needs  of  culturally  diverse  students  is  dependent 
upon  the  adoption  of  culturally  appropriate  and  inclusive  instructional  design  models.  We  can  adopt 
Papert's  (1993)  idealistic  belief  that  "the  computer  changes  who  can  do  what,  and  at  what  age"  and 
regard  the  computer  as  a tool  for  liberation,  achievement  and  multicultural  learning.  But  we  must 
do  more  to  realise  the  objective  of  culturally  appropriate  instructional  design.  Technologists  and 
multicultural  educators  need  to  plan  for  cultural  pluralism  when  planning  and  designing  instruction 
and  delivery  options,  if  Web  sites  are  to  cater  for  diverse  learning  needs,  cultural  perspectives  and 
learning  styles.  The  design  of  a World  Wide  Web  learning  environment  for  Indigenous  Australian 
students  is  outlined,  and  multiple  cultural  model  of  instructional  design  model  that  informed  the 
development  is  discussed.  The  cultural  aspects  of  the  design  process  are  outlined,  and  implications 
for  design  of  Web  sites  for  culturally  contextualised  learning  are  drawn. 


Context  of  the  study 

The  use  of  information  technology  by  Indigenous  Australians  is  very  limited.  As  part  of  The 
Commonwealth  Government’s  Access  and  Equity  Policy  in  Australia,  there  has  been  increased  funding  to 
support  greater  Indigenous  use  of  communications  technologies  and  for  information  literacy  skills.  In  1998 
Edith  Cowan  University  in  Western  Australia  successfully  obtained  funding  to  launch  its  pre-university 
bridging  courses  on  the  World  Wide  Web  to  cater  for  the  state’s  Indigenous  population.  The  online  learning 
environment  and  content  were  designed  to  enhance  the  participation  of  Indigenous  students  learning  in 
Web-based  learning,  and  to  increase  their  academic  success  through  increased  proficiency  and  awareness  of 
computer  technology.  In  addition,  the  initiation  and  implementation  of  the  project  was  intended  to  produce 
other  outcomes,  such  as  insights  into  learning  styles  and  how  students  responded  to  the  on-line 
environment. 

This  paper  describes  the  design  and  epistemology  of  a culturally  appropriate  Web  site  for  the  Indigenous 
Australian  students  who  are  novice  users  of  the  technology,  and  who  are  making  the  transition  to  tertiary 
studies.  The  project  is  planned  to  extend  over  three  years,  and  in  this  time  six  units  of  a tertiary  bridging 
course  will  be  redesigned  and  made  available  to  learners  in  an  open  learning  mode.  Until  this  point  in  time 
learners  have  accessed  the  materials  in  the  print  mode  only.  The  project,  entitled  Indigenous  Online 
Network  (ION)  aims  to  provide  appropriate  learning  materials  and  contexts  for  Indigenous  students  to 
access  study  skills  courses  online,  and  provide  a communications  network  for  Indigenous  Australian 
academics  within  Australia. 


Theoretical  aspects  of  WWW  design:  Cross  cultural  dimensions 

The  question  of  cultural  contextuality  in  the  creation  of  a Web  site  for  a specific  cultural  group  can  be 
regarded  as  a central  issue  in  the  design  of  both  interface  and  learning  activities.  Often,  minority  cultures 
wishing  to  incorporate  technology  in  their  learning  are  not  well  served  by  instructional  approaches  designed 
for  mainstream  values  (Roblyer  et  al  1996).  As  learning  is  a cultural  activity,  the  design  of  a Web  site  is 
also  infused  with  cultural  meaning  and  with  cultural  nuances  and  identity  issues,  and  instructional 
designers  and  developers  bring  their  own  viewpoints  and  perspectives  to  the  design  process.  The  World 
Wide  Web  can  attract  global  audiences  to  the  many  sites  that  can  be  accessed.  But  to  what  extent  is  the 
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Web  increasing  cross-cultural  understanding  and  bridging  the  gap  between  cultures?  Can  we  assume  that 
the  content,  interaction  and  learning  experiences  afforded  by  the  WWW  will  have  the  same  relevance  and 
meaning  to  diverse  audiences? 

Collis  & Remmers  (1997)  have  defined  two  categories  of  sites  that  have  cross-cultural  implications: 

(i)  Sites  that  are  made  in  context  and  culture,  but  visited  by  other  cultures 

(ii)  Sites  designed  specifically  for  cross-cultural  participation.  (See  Figure  1 .) 

The  provision  of  Web  sites  for  Indigenous  Australian  students  would  fit  into  category  one,  as  they  are 
intended  to  cater  to  the  needs  of  a particular  cultural  group.  In  designing  a Web  environment  for  this  target 
group,  it  was  envisaged  that  other  Indigenous  groups  and  individuals  from  around  the  world  would  visit  the 
site.  However,  the  primary  objective  was  to  cater  for  the  needs  of  Indigenous  students  seeking  a bridge  into 
university.  In  applying  an  instructional  design  model  for  the  sites,  our  objective  was  to  create  a culturally 
appropriate  learning  environment  for  the  design  of  this  and  future  sites  for  Indigenous  students.  Primary 
considerations  were  influenced  by  the  educational  values,  needs  and  learning  styles  of  Indigenous 
Australian  students,  and  the  integration  of  these  cultural  variables  into  the  design  process.  In  addition,  it 
was  hoped  that  the  initial  experience  of  the  students  would  be  monitored  so  that  feedback  on  learning 
processes  and  on  line  interactions  would  enable  us  to  further  refine  Web  sites  tailored  to  the  specific  needs 
of  these  learners. 


Figure  1:  Categories  of  Websites 
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Indigenous  epistemology  and  values 

Our  first  step  in  designing  the  Web  site  for  the  Indigenous  students  was  to  ensure  cultural  inclusivity  in  all 
aspects  of  pedagogy,  form  of  interaction  and  learning  processes.  This  involved  review  and  analysis  of  a 
broad  range  of  literature  and  extensive  consultation  with  Indigenous  students,  community  members  and 
academic  staff  In  order  to  define  and  plan  instruction  for  the  particular  educational  setting,  the  adoption  of 
an  appropriate  instructional  design  model  was  essential.  Henderson  (1996)  reminds  us  that  instructional 
design  is  an  intangible  aspect  of  culture,  but  once  it  is  transformed  into  a material  object,  it  becomes  an 
artefact  of  the  culture  is  which  it  is  embedded.  In  support  of  this  view,  Flechsig  (1997:  27)  affirms  that  “ 
instructional  ‘designs’  are  specific  ways  of  cultural  transmission  and  of  organising  learning  processes”. 
From  the  outset,  therefore,  the  design  process  was  infused  with  the  awareness  that  the  development  of  the 
Website  should  be  contextualised  within  the  epistemology,  pedagogic  philosophy  and  models  of  knowledge 
construction  that  underpin  Indigenous  values  and  approaches  to  learning. 

The  literature  on  Indigenous  epistemology  provides  several  insights  that  can  inform  the  design  of  learning 
activities  and  pedagogy.  Christie  (1988)  suggests  that  in  genuine  Indigenous  Aboriginal  education  the  basic 
goal  is  to  teach  the  harmony  and  unity  of  Aboriginal  life.  An  important  goal  is  to  preserve  continuity  with 
the  past,  the  land  and  the  people.  From  an  Indigenous  perspective,  cultural  integrity  and  survival  are 
fundamental  concerns.  Triandis  (1990)  describes  the  distinction  between  individualism  and  collectivism  as 
the  most  important  dimension  of  cultural  difference  in  social  behaviour.  If  it  is  accepted  that  Indigenous 
students  differ  significantly  from  peer  groups  of  non-Aboriginal  students  concerning  values  that  serve 
collective/community  as  opposed  to  individual  interests,  this  is  likely  to  have  planning  and  design 
implications.  In  practical  terms,  for  the  design  of  the  learning  environment,  this  involved: 

• community-based  communication  processes  and  task  orientation; 

• consultation  and  decision  making  by  Indigenous  students;  and 
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• participatory  planning  in  the  design  process. 


For  adult  Indigenous  learners,  cognitive  orientations  to  instruction  have  been  found  to  be  unsupportive  of 
traditional  values  and  ways  of  learning  (Ryan,  1992).  Instead,  educational  models  based  on  concepts  of 
community  and  social  participation  have  been  found  most  productive  (Whatman,  1995).  The  successful 
design  of  the  Web  site  therefore  depended  on  the  adoption  of  an  instructional  design  approach  that  was 
congruent  with  Aboriginal  values,  participatory  structures  and  way  of  communicating. 

Throughout  the  planning  stage,  there  was  constant  discussion  within  the  Indigenous  community  about 
content,  graphic  design  and  instructional  approaches.  To  further  support  the  cultural  relevance  of  the  site 
for  Indigenous  learners,  it  was  essential  to  conceive  of  the  development  within  an  appropriate  theoretical 
framework. 


Theoretical  framework 

Through  the  process  of  consultation,  two  interrelated  theoretical  models  of  learning  were  found  to  be 
supportive  of  Indigenous  values  and  pedagogies.  To  support  the  notion  of  community,  the  creation  of  a 
‘community  of  practitioners’  (Lave,  1991)  affirmed  the  processes  of  dialogue,  shared  interest,  participation 
and  collaboration.  This  concept  permeated  the  design  at  various  levels,  from  the  creation  of  the  on-line 
environment  through  to  the  design  of  learning  activities,  support  structures,  and  collaborative  links  with  the 
community.  For  students,  the  concept  of  belonging  to  an  online  community  resonated  with  their  own 
community-based  expectations,  and  allowed  them  to  see  technology  not  only  as  a means  of  widening  their 
own  circle,  but  also  as  a means  of  taking  newly  found  skills  into  their  own  communities.  Thus,  the  social, 
communicative  and  collaborative  aspects  of  the  ‘community  of  practitioners’  approach  were  culturally 
appropriate.  For  Lave  & Wenger  (1991),  social  action  and  communicative  praxis  are  primary,  and  learning 
is  both  social  and  a language  based  practice.  Another  tenet  of  this  theory  is  “the  increased  access  of 
learners  to  participating  roles  in  expert  performance”  (Lave  & Wenger,  1991:  17).  This  means  that  learners 
are  always  participants  in  the  learning  process  and  acquire  greater  expertise  though  observation,  and 
“legitimate  peripheral  participation”  (Lave  & Wenger,  1991:29).  Even  as  novices,  learners  have  access  to 
different  levels  of  activity,  and  learn  initially  by  taking  on  simpler  tasks  but  gradually  moving  towards  full 
participation. 

Supports  for  learning  and  collaboration  were  incorporated  into  the  design  and  were  conceptualised  and 
augmented  by  the  theoretical  framework  of  Vygotskyan  theory  (1978)  in  which  learning  is  viewed  as  being 
a participant  in  a particular  social  world.  For  Vygotsky,  learning  is  development,  and  through  the  processes 
of  social  interaction  and  dialogue  with  more  experienced  others,  learners  are  assisted  in  achieving  higher 
levels  of  competence.  Another  fundament  tenet  of  Vygtoskyan  theory  is  that  learners  continually  re-enter 
new  zones  of  development  recursively  throughout  their  lives.  As  technology  is  increasingly  used  to  deliver 
instruction  and  to  facilitate  communication,  information  literacy  skills  for  Indigenous  students  can  be 
interpreted  as  entering  new  zones  of  development.  For  Indigenous  learners,  it  is  nevertheless  important  to 
retain  their  own  sense  of  cultural  identity,  and  to  integrate  new  skills  with  other  learning  needs,  ie  tertiary 
literacy  skills  and  Indigenous  cultural  knowledge.  (See  Figure  2). 

Figure  2:  Multiple  cultural  model  of  online  learning 


The  “zones  of  development”  for  Indigenous  learners  on-line  would  therefore  have  to  combine  the  skills 
associated  with  the  academic  culture,  of  university  with  Indigenous  cultural  perspectives  and  relevant 
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disciplinary  skills  for  tertiary  success.  By  integrating  the  participatory  pedagogy  of  Lave  & Wenger 
(1991)  based  on  a community  of  practice,  an  emancipatory  learning  experience  was  conceptualised  as 
collaborative  groups  of  learners  working  at  appropriate  levels  while  acquiring  new  skills.  In  this  way,  a 
theoretically  valid  and  culturally  appropriate  pedagogical  framework  was  developed  for  the  Web  site. 


Instructional  design:  Limitations  of  existing  models 

Linked  to  the  theoretical  framework  of  the  community  of  practitioners,  an  instructional  design  model  was 
sought  to  support  the  learning  activities  of  sharing  knowledge  and  learning  through  participation,  while 
recognising  the  unique  cultural  tenor  of  the  Web  site.  There  are  many  current  instructional  design  models 
and  paradigms,  each  of  which  can  be  interpreted  as  culturally  and  socially  determined  (Branch,' 1997). 
Instructional  designers  however,  may  not  always  incorporate  cultural  understandings  of  technology  into  the 
design  of  learning  materials.  Instructional  design  models  include  cognitive,  social  and  pedagogical  issues, 
but  may  not  acknowledge  the  need  for  cultural  contextuality.  Reeves  & Reeves  (1997)  for  instance,  outline 
ten  pedagogical  dimensions  that  can  be  used  to  design  interactive  multimedia  tools  and  learning 
environments.  Cultural  sensitivity  is  described  as  building  in  diversity  and  adapting  instructional  design  to 
cultural  norms.  Henderson  (1996)  argues  that  cultural  issues  are  fundamental,  as  instructional  design  is 
about  the  creation  of  cultural  identity:  “Instructional  design  cannot  and  does  not,  exist  outside  of  a 

consideration  of  culture”  (Henderson,  1996,  p.  86).  In  creating  a learning  space  for  Indigenous  learners,  the 
development  team  had  to  ask:  How  can  the  dimension  of  cultural  contextuality  inform  the  instructional 
design  process? 

An  important  part  of  Henderson’s  (1996)  work  is  the  identification  of  several  design  paradigms,  each  of 
which  reflects  particular  world  views,  values,  pedagogies,  inclusions,  and  exclusions  that  result  from  the 
designer’s  own  societal  context.  As  instructional  designers  are  instrumental  in  creating  and  developing 
interactive  multimedia,  courseware  and  learning  environments,  they  can  also  influence  cultures  of  learning. 
Among  the  paradigms  identified  by  Henderson  (1996)  there  are  three,  all  of  which  are  limited  with  respect 
to  cultural  dimensions  of  learning  and  pedagogy.  For  example,  the  inclusive  paradigm  which  is  motivated 
by  equity  issues,  acknowledges  diversity  and  multiculturalism  in  the  design  of  learning  materials.  However, 
this  type  of  design  tends  towards  soft  multiculturalism  which  ignores  the  complex  issues  involved  and 
includes  only  superficial  aspects  of  the  minority  culture  that  do  not  challenge  the  majority  view.  The 
inverted  curriculum  approach  takes  the  minorities’  perspective,  and  criticises  the  status  quo,  or  dominant 
group  pedagogy.  However,  it  does  not  empower  students  who  must  try  to  survive  within  the  mainstream 
academic  culture. 

This  consideration  of  ID  models  alerted  the  development  team  to  issues  of  tokenism,  soft  multiculturalism 
and  cosmetic  multiculturalism.  To  arrive  at  an  appropriate  design  model,  several  questions  were  posed  to 
all  interested  parties,  including  prospective  students,  community  members  and  teachers,  so  that  the  design 
would  be  informed  by  Indigenous  perspectives. 

• What  kind  of  learning  environment  is  most  familiar  to  the  students? 

• How  does  the  cultural  background  of  the  students  influence  their  use  and  view  of  time? 

• How  do  Indigenous  students  conceive  the  role  of  the  teacher? 

• What  kind  of  relationship  do  students  want  with  a teacher? 

• What  kinds  of  assessment  tasks  will  be  fair  and  unbiased? 

• What  rewards  and  forms  of  feedback  will  be  most  motivating  for  these  students? 

• Is  the  locus  of  control  congruent  with  these  students’  own  sense  of  personal  control? 

What  cognitive  styles  characterise  the  target  group? 

Responses  to  these  questions  contributed  to  the  development  of  the  Web  site  and  to  the  creation  of  learning 
activities,  pedagogical  support  and  collaborative  task  design.  The  adoption  of  an  ‘eclectic  paradigm’  or 
multiple  cultural  model  allowed  variability  and  flexibility  and  that  enabled  students  to  learning  through 
interaction  with  materials  that: 

• reflect  the  multicultural  realities  of  society; 

• include  multiple  ways  of  learning  and  teaching;  and 

• promote  equity  of  learning  outcomes. 
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The  adoption  of  the  multiple  cultural  model  is  reflected  in  different  ‘zones’  of  learning  that  matched  the 
cultural  experience  of  Aboriginal  students  entering  university.  (See  Figure  2.) 


Pedagogical  features  of  the  on-line  community 

A constructivist  approach  aligned  to  student  centred  approaches  to  knowledge  interpretation  was  built  into 
the  design  process.  As  knowledge  extended  beyond  the  classroom  into  the  community,  the  concepts  of 
knowledge  and  teaching  were  redefined  for  the  Indigenous  learners.  The  creation  of  a community  of 
learners  online  was  achieved  through  task  design,  communication  forums,  shared  spaces  for  working  and 
collaboration  and  dialogue  and  support  from  peers  and  tutors.  Each  distributed  group  of  students  had  an 
online  mentor  or  virtual  tutor  who  provided  feedback  and  fostered  further  dialogue. 

A broader  view  than  just  designing  instruction  for  mastery  of  content  was  adopted.  The  Web  site  fostered 
conceptual  transformative  learning  by  creating  a holistic  learning  environment  where  social  processes, 
community  consultation  and  learning  strategies  for  tertiary  study  were  built  in.  These  features  are 
summarised  in  Table  1. 


Table  1:  Features  of  the  online  community  of  inquiry 


Feature  of  community  of  inquiry  learning 

Cultural/cognitive  application 

• Epistemology  and  pedagogy  congruent  with 
community  needs 

Socially  situated  learning  based  on  Love's  (1991) 
community  of  inquiry 

• Communication  forums  and  interaction 

Negotiation  of  shared  meaning;  peer  teaching 
and  conversation 

• Shared  experience  and  knowledge 

Participatory  tasks,  collaborative  learning  and 
group  projects  (Scheel  & Branch,  1993) 

• Supportive  teacher  roles  and  responsibilities 

On-line  tutors  scaffold  learning  through 
mentorships,  dialogue  and  modelling 

• Creation  and  manipulation  of  shared  spaces 

Use  of  bulletin  boards  for  shared  experiences  and 
talk;  Use  of  the  larger  space  of  the  WWW  to  find  & 
share  resources 

• Student  roles  and  responsibilities 

Clear  statements  of  outcomes;  negotiable  tasks; 
posting  of  shared  problems  to  a FAQ  space 

• Situated  discourse  and  collaboration 

Use  of  Lipman's  (1991)  model  of  inquiry:  mutual, 
questioning,  feedback  & deliberation  through 
problem-based  tasks 

• Support  mechanisms  within  the  community 
and  outside  the  community 

Access  to  peer  scaffolding;  outside  experts, 
mentoring  by  online  tutors  through  online  help, 
group  forums 

• Social  interaction  to  support  group  cohesion 

Use  of  multiple  forms  of  communication  and 
dialogue  & tasks  designed  for  collaboration- 

Task  design  incorporated  instructional  tools  to  enable  students  to  formulate  and  develop  responses  to 
problems,  and  develop  culturally  pluralistic  ways  of  perceiving  information  and  skills.  This  approach  was 
based  on  Vosniadu’s  (1994)  notion  of  conceptual  change  as  embedded  in  existing  conceptual  structures  and 
revisions  of  these  through  authentic  learning  experiences. 

By  providing  a community  focus  for  learning  and  task  design  , the  Indigenous  learners  were  able  to  make 
connections  with  the  instructional  environment  itself,  through  the  use  of  culturally  familiar  metaphors, 
icons  and  graphical  features.  Consideration  of  the  socio-cultural  backgrounds  of  the  learners  and  their 
styles  of  learning  ensured  that  the  learning  tasks  and  processes  of  instruction  were  embedded  in  culturally 
sensitive  contexts  that  validated  Indigenous  ways  of  learning. 

Apart  from  celebrating  cultural  diversity,  the  design  of  the  Web  site  for  Indigenous  Australian  learners 
shows  how  technology  can  be  used  to  support  contextualised  design  and  enhance  student  achievement  by 
explicitly  acknowledging  the  learners’  unique  cultural  perspective. 
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Abstract:  After  one  year  (1997-1998)  of  working  extensively  with  our  instructors, 
particularly  all  of  those  responsible  for  our  first-year  courses,  17  courses  required  for  all 
students  and  several  senior  elective  courses  (3)  are  now  using  their  tailored-made  Web-based 
course-support  environments.  In  addition,  during  the  1998-99  academic  year  all  of  the 
second-year  courses  will  be  adapted  to  be  delivered  with  their  new  course-support 
environments.  In  this  paper  we  describe  how  a broad  range  of  instructors,  not  only  those  who 
volunteer  or  have  an  initial  interest  in  using  WWW-based  course  support,  but  also  those  who 
are  required  to  do  so  as  part  of  a faculty-wide  change  process  work  through  a design  process 
based  on  rapid  prototyping  to  evolve  tailor-made  WWW-based  course-support  environments. 
What  are  the  options  instructors  choose  for  these  environments  and  how  popular  are  various 
types  of  options  (for  example,  shared  workspaces,  WWW  boards  for  computer  conferencing, 
etc.)  for  course-support  environments?  Trends  and  implications  in  the  set  of  options  chosen 
by  instructors  in  our  faculty  are  discussed. 


Introduction:  TeleTOP  at  the  University  of  Twente 

The  University  of  Twente  in  The  Netherlands  has  national  and  international  reputation  in  the  field  of 
telematics,  the  combination  of  information  and  communication  technologies.  Not  only  is  there  an  extensive 
amount  of  research  being  done  in  the  area  but  also  the  application  of  telematics  applications  to  the  teaching  and 
learning  process,  what  we  call  "tele-learning"  (Collis,  1 998)  has  a high  priority.  The  most  ambitious  of  the  tele- 
learning  initiatives  at  the  University  of  Twente  is  the  TeleTOP  project  in  the  Faculty  of  Educational  Science  and 
Technology.  The  overall  goal  of  the  project  is  to  stimulate  the  innovative  and  appropriate  use  of  telematics  for 
learning  purposes  within  the  faculty  in  order  to  make  the  educational  delivery  more  efficient,  more  enriched,  and 
more  flexible. 

To  achieve  these  goals  a special  system  has  been  developed  by  a professional  team  of  educational  and 
technical  designers  and  developers.  The  system  is  database-driven  and  Web-based  and  uses  a variety  of 
telematics  applications,  ranging  from  e-mail  and  Web  pages  to  shared  workspaces.  The  system  generates 
WWW-based  environments  are  meant  as  course-support  environments  for  the  instructors  and  students  and  not  as 
replacements  for  face-to-face  sessions  and  textbooks.  Contact  between  instructors  and  students  still  is  one  of  the 
essential  aspects  of  a learning  process  (see  Tielemans  & Collis,  1 998,  for  a further  description  of  the  educational 
and  technical  requirements  for  the  system  and  how  they  have  been  realized). 

However,  no  matter  how  elegant  a system  is,  instructors  must  use  it.  Reasons  that  instructors  in  higher 
education  do  not  make  use  of  computer-based  resources  to  support  the  instructional  delivery  of  their  courses  are 
well  known  (see  for  example,  Northrup  (1997)  and  Collis  & Pals,  1998).  With  respect  to  WWW-based 
environments,  most  current  usage  in  mainstream  programs  in  higher  education  is  voluntary,  and  although  there 
are  an  extensive  number  of  examples  of  such  voluntary  use  there  are  far  many  more  instructors  who  are  not 
making  use  of  any  of  the  instructional  possibilities  of  telematics.  There  are  relatively  few  examples  of  efforts  to 
engage  all  the  instructors  in  a faculty  in  the  use  of  telematics  applications  in  their  courses.  Medic  and  Walker 
(1997)  describe  the  Virtual  Office  Hours  environment  at  the  Department  of  Chemistry  and  Biochemisty  at  the 
University  of  California  as  one  such  example.  Another  is  the  TeleTOP  Project  at  the  University  of  Twente. 
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The  decision  (made  by  the  faculty  administration)  to  involve  all  courses  in  a re-design  process,  both 
pedagogically  and  to  include  use  of  telematics  tools  and  WWW  sites,  involves  the  challenge  of  working  with  a 
wide  variety  of  courses  and  instructors.  To  do  this  we  have  developed  a WWW-based  decision-support  tool 
which  we  use  as  part  of  a rapid-prototyping  process  designed  to  maximize  instructor  involvement  in  the  re- 
design of  his  or  her  course.  Our  situation  has  also  given  us  an  excellent  opportunity  to  study  what  instructors 
actually  want  and  use  when  given  the  possibility  of  a wide  range  of  WWW-based  tools  and  suggestions  for  using 
those  tools.  In  the  next  section  we  describe  the  rapid-prototyping  process. 


Supporting  Instructors  in  the  Decision-Making  Process  when  Designing  a WWW-based 
Course-Support  Environment 

WWW-based  environments  to  support  all  of  the  first-year  courses  in  the  Faculty  of  Educational  Science 
and  Technology  were  ready  in  a definitive  form  at  the  beginning  of  June  1998.  How  did  this  come  about? 
Beginning  in  September  1997,  a weekly  instructors'  session  began  in  the  faculty.  These  sessions  were  voluntary, 
and  well  attended.  During  the  first  two  months  of  the  sessions,  instructors  were  introduced  to  a way  of  thinking 
about  their  courses,  in  terms  of  a matrix  in  which  the  rows  are  standard  components  of  courses  (course- 
organization  aspects,  face-to-face  sessions,  self-study  and  practice  activities,  projects  and  major  assignments, 
testing,  and  general  communication)  and  the  columns  relate  to  three  motivations  for  change  in  each  of  the 
components  (to  make  the  component  more  efficient  for  instructors  and  students,  to  enrich  the  component,  to 
make  the  component  more  flexible  for  different  types  of  students).  Links  to  examples  of  how  telematics 
applications  could  support  each  category  of  change  were  demonstrated  and  discussed. 

In  November  1997  instructors  were  encouraged  to  consider  their  own  courses  and  make  a list  of  re- 
design options  with  could  be  facilitated  by  a WWW-based  course-support  environment  tailored  for  their  own 
particular  courses.  In  December  1997  one-hour  individual  instructor  sessions  with  the  instructors  of  each  of  21 
courses  were  organised  (all  the  first-year  courses  and  a variety  of  other  courses).  The  primary  goal  of  the  session 
was  to  use  the  especially  made  TeleTOP  Decision  Support  Tool  (DST)  (Collis  & De  Boer,  1998)  in  order  to 
interact  intensively  with  the  instructor  whose  course  was  being  re-designed,  trying  to  identify  which  WWW- 
compliant  tools  and  associated  pedagogical  approaches  are  most  likely  to  be  acceptable  and  interesting  to  the 
particular  course  of  the  instructor  and  his/her  way  of  teaching.  For  each  of  the  course  components  a number  of 
questions  were  asked  relating  to  ways  that  a WWW  site  could  be  used  relative  to  that  component.  Links  to 
examples,  primarily  from  courses  already  using  the  WWW  in  our  own  faculty,  were  provided  for  every  question. 
Figure  1 shows  a portion  of  the  DST,  with  the  overlay  screen  showing  the  example  available  from  the  link 
associated  with  a particular  question.  Immediately  after  the  last  of  the  questions  in  the  DST  was  completed,  a 
WWW  page  was  generated  for  the  instructor  summarizing  the  choices  that  had  been  made,  and  providing  the 
example  links  for  those  choices  so  that  the  instructor  could  further  consider  them  via  the  use  of  an  ordinary 
WWW  browser  at  his  or  her  convenience.  This  site  generated  by  the  DST  served  as  the  product  of  the  first  round 
of  rapid  prototyping. 
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Figure  1.  A screen  dump  of  the  WWW-based  decision-support  tool:  questions  and  examples. 

The  second  goal  of  the  DST  session  was  to  respond  to  the  instructors’  reactions  with  ideas  and 
suggestions,  as  well  as  to  skip  suggestions  which  did  not  seem  like  they  would  be  comfortable  for  the  instructor. 
The  tool  made  it  easier  for  instructors  to  make  decisions  with  regard  to  making  (some  of)  the  components  of 
their  own  courses  more  efficient,  enriched  and/or  flexible  in  their  new  WWW-based  learning  environments.  The 
instructor  needed  to  decide  what  he  thought  was  appropriate  for  his  or  her  course.  Twenty-one  sessions,  one  per 
course,  were  organised  in  December  and  January  97/98  according  to  this  approach,  with  one  or  two  instructors 
participating  and  the  authors  carrying  out  the  interview  with  the  support  of  the  DST  at  each  session.  A follow-up 
visit  in  the  instructor's  office  one  week  after  the  DSST  also  occurred,  to  walk  through  the  first  site  from  their 
decisions  with  the  DST,  and  make  a second  round  of  refinement  of  those  decisions.  Following  this,  a first 
prototype  version  of  a course  site  tailored  to  the  instructor's  choices  was  generated,  through  the  use  of  the 
TeleTOP  database  system.  This  process  is  being  repeated  in  the  same  period  in  1 998-99  for  all  the  second-year 
courses. 

The  next  step  for  the  instructors  was  to  come  together  every  week  and  practice  with  these  prototype 
environments.  The  Wednesday  sessions  in  which  this  occurred  became  an  important  part  of  the  implementation 
process.  Instructors  had  the  chance  to  work  together  and  exchange  ideas.  A few  months  later  the  instructors 
again  went  through  the  process  of  using  a second  WWW-based  decision  tool,  choosing  a final  set  of  options  for 
their  course-support  environments.  As  a result  of  this  rapid-prototyping  process,  instructors  not  only  are  closed 
involved  in  the  design  process  of  the  WWW  sites  that  will  support  their  courses,  but  also  develop  competency  in 
handling  those  sites  and  the  associated  telematics  tools  and  applications.  The  results  of  the  process  are  tailored 
course-support  environments,  and  also  the  creation  of  a sense  of  community  among  the  instructors  and  a 
heightened  level  of  awareness  and  literacy  throughout  the  faculty  with  regard  to  the  handling  of  telematics 
applications,  network  issues,  and  the  instructional  integration  of  telematics  applications  into  regular  courses. 

Instructors  thus  had  an  extensive  opportunity  to  try  out  a large  variety  of  telematics  tools  and 
applications  over  the  course  of  the  rapid-prototyping  process.  What  options  have  they  actually  chosen?  In  the 
following  section,  a summary  of  the  options  eventually  selected  by  21  courses  in  the  1997-98  cycle  is  given  (see 
also  Dijkstra  & Collis,  1998,  and  Winnips,  Collis,  & Moonen,  1998,  for  discussions  of  two  of  the  courses  in 
detail). 


Designing  A Course  Environment:  The  Options  Chosen 

During  the  second  round  (April,  May  1998)  instructors  could  make  a final  list  of  options  for  their  course 
environment.  Table  1 gives  an  overview  of  the  options  selected  by  the  instructors  for  their  environments. 
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Table  1.  The  options  chosen  for  the  WWW-based  course-support  environments 


Component 

Options 

Description 

General  course 
information,  self- 
study,  lectures  and 
support 

News 

A place  for  up-to-date  information 

Roster 

This  is  an  important  part  of  the  environment  and  the  ideas  behind  it. 
Here,  instructors  can  put  their  study  materials,  assignments,  sheets, 
notes  and  feedback  related  to  the  lectures. 

Quizserver 

This  option  enables  easy-to-make  (self)  tests. 

Course 

information 

A course  description  can  be  put  here.  Think  of  goals,  organisation, 
assigned  text,  which  books,  etc. 

Communication 

Email 

In  the  mail-centre  addresses  of  individuals  and  groups  can  be  found. 
Mails  can  be  send  from  here. 

Discussion 

The  discussion  area  can  be  used  for  a-synchronous  discussions. 

Question  and 
answer 

Same  as  the  discussion  area,  here  with  the  focus  on  question  towards 
the  instructor. 

Chat 

Synchronous  communication. 

Group-work 

Group  Ware 

An  easy  to  use  file  management  area,  for  collaborative  work. 

BSew 

A more  advantaged  file  and  communication  management  area,  for 
collaborative  work. 

Presentation 

Presentations  and  other  products  can  be  presented  in  this  part. 

Resources 

Glossary 

Area  where  concepts  can  be  explained.  Relations  with  other  areas  can 
be  made  clear  as  well. 

WEB  links 

Resources:  web  pages  on  the  WWW 

Multimedia 

Resources;  to  the  multimedia  database. 

Search 

A search  centre:  within  the  course  environment  or  the  WWW 

The  options  Newsflash ^ Roster,  Course  information  and  Email  centre  were  chosen  by  all  instructors  and 
thus  are  being  strongly  advised  as  the  basic  environment  for  the  new  round  of  courses  being  tailored  during  the 
1 998-99  academic  year.  The  other  options  were  chosen  in  various  combinations  by  the  instructors.  Instructors 
with  specific  needs  could  get  more  tailor-made  options.  Figure  2 gives  an  overview  of  the  percentages  of  the 
instructors  who  choose  each  of  the  above  options.  These  numbers  are  based  on  the  first  21  courses,  of  courses 
being  offered  between  April  1998  through  April  1999. 


Figure  2.  Percentages  of  instructors  choosing  various  options  (N=21). 


Using  the  Course  Environment 

At  the  start  of  the  academic  year  1998-99  three  courses  had  already  used  their  WWW-based 
environments  and  seven  courses  were  ready  to  start  (timetabled  in  the  first  trimester).  Participating  in  the 
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courses  are  not  only  our  regular  students  but  also  a new  group  of  students  participating  mainly  at  a distance 
while  remaining  in  the  workforce.  The  latter  group  of  students  come  one  day  every  two  week  to  the  University 
for  a variety  of  face-to-face  sessions.  All  the  other  courses  that  had  been  through  the  rapid-prototyping  process 
were  busy  organising  their  environments,  while  learning  from  the  experiences  of  their  colleagues.  Figure  3 is  a 
screen  dump  of  one  of  the  course  environments.  At  the  left  side  of  the  screen  the  list  of  chosen  options  can  be 
found.  This  is  the  menu  area.  At  the  right  side  all  information  will  be  displayed.  All  courses  have  a similar 
interface  design,  but  differ  in  their  chosen  functionalities. 
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Figure  3.  The  Home  page  of  the  Instrumentation  Technology  2 course  (see  Winnips,  Collis,  & Moonen,  1998). 

One  part  of  the  environment  support  the  on-going  organisation  of  the  course.  We  call  this  the  roster 
area.  All  instructors  use  the  matrix-like  rosters.  This  area  helps  them  to  place  all  organisational  messages  and 
materials  needed  before,  during  and  after  each  contact  session.  Students  can  find  the  self-study  materials  or 
instructions  in  the  roster.  They  can  find  the  lecture  notes  and  sheets  and  possible  follow-up  assignments,  and  can 
submit  their  assignments  and  comments  directly  via  the  roster.  Instructors  do  not  receive  these  submissions  in 
their  ordinary  e-mail.  All  submitted  materials  are  directly  posted  in  the  WWW  environment,  where  the  instructor 
can  access  them  in  a systematic  way,  whenever  and  wherever  he  or  she  chooses.  The  instructor  needs  no  special 
software  environment  or  tools;  only  a normal  WWW  browser  and  ordinary  Internet  access.  Figure  4 is  an 
example  of  such  a roster,  used  in  the  Instruction  Design  Theories  course,  1998  (see  also  Dijkstra,  Collis,  & 
Esryel,  1998). 
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Figure  4.  The  roster  of  the  Instruction  Design  Theories  course 
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Conclusion 


The  TeleTOP  rapid-prototyping  approach,  supported  by  two  versions  of  a WWW-based  decision- 
support  tool,  is  a managable  and  effective  way  to  involve  instructors  in  the  re-design  of  their  courses,  and  at  the 
same  time  in  skill  development  at  handling  WWW-based  tools  and  environments.  The  options  chosen  by  the 
instructors  build  upon  their  existing  ways  of  teaching,  but  extending  these,  primarily  through  opportunities  for 
the  students  and  the  instructor  to  efficiently  communicate  and  for  students  to  add  new  resources  to  the  course 
sites  as  follow-up  assignments  between  face-to-face  sessions.  We  believe  this  approach,  building  upon  the 
instructional  methods  already  familiar  to  the  instructor,  is  a useful  strategy  particularly  when  instructors  who 
would  normally  not  consider  making  use  of  a WWW  environment  are  involved.  The  approach  also  works  for 
instructors  already  experienced  with  WWW  environments  as  well.  For  more  information,  see 
http://teletop.edte.utwente.nl 
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Abstract  : This  paper  suggests  a mMmedba  edacafjoaa/  gfs/em  wbkb  bas  /be  abb/tty  fo  extract 
hte/f/geat  wstraction  on  tbe  cbbstri bated  eaobron/aent  Tbe  proposed  syste///  /s  designed  for  supporting 
individuai  instruction  and  real  time  user  interaction.  As  tbe  gystem  based  on  CORBA,  it  bas  ability 
for  distributed  computing  facilities.  Using  MHEG  standard,  iw  can  provide  mu/timedia  courseware 
and  real  time  user  interaction.  To  diagnose  students*  reponses  and  generate  evaluations,  ufe  use  semrai 
linguistic  variables  and fu^  theory. 

There  are  two  mayor  advantages  for  using  this  system.  This  system  can  provide  ^r/amic  generation  of 
problems  and  tbe  ability  to  provide  a cp/namic  instruction  strategy.  And  it  can  increase  reusability  f 
courseware  materia/  for  using  standard  of  mu/timedia  representation  and  communication.  IVe  use 
COREA  arrd  JWHEG  to  overcome  tbe  disadvantage  of  tbe  Web,  passive  protoco/  and  poor 
interactivity,  HTTP. 


1.  Introduction 

The  Web  has  been  spread  world  wide  as  a standard  of  distributing  multimedia/hypermedia  information.  There  are 
many  applications,  include  distance  learning,  based  on  the  Web.  The  advantage  of  using  distance  learning  on  the 
Web  is  apparent.  We  can  provide  instmaional  information  using  a hypertext  form,  the  same  as  students’  cognitive 
structure[Qavener  98].  But  there  are  some  disadvantages  using  the  Web:  passive  protocol  and  poor  interactivityfBolk 
and  Britton  98].  These  problems  can  be  overcome  with  two  models,  that  could  be  integrated  in  WWW.  A uniform 
paradigm  to  encapsulate  and  stmcture  web  components  can  be  achieved  thanks  to  the  OMG’s  CORBA[OMG  95]. 
And  real  time  user  interaction  facilities  can  be  achieved  by  international  standard,  MHEGfISO  95]. 

Despite  the  WWW  is  able  to  access  distributed  information  and  services  on  Internet,  it  can  not  provide 
individualized  information  to  students[Bolk  and  Britton  98].  In  addition,  unlike  existing  stand-alone  ITS,  students 
must  have  the  ability  to  study  freely  and  search  related  information  easily  Therefore  the  system  must  overcome 
potential  disorientation  problems  that  could  burden  accessibility. 

In  this  paper,  we  suggest  an  intelligent  multimedia  educational  system  that  can  make  intelligent  instruction  and 
individu^  instruction  on  distributed  environment.  Our  system  can  support  concurrent  learning  and  dynamic  lesson 
planning  for  each  student  by  using  linguistic  variables  and  fuzzy  theory.  In  addition,  our  system  provides 
instructional  units  in  which  students’  response  are  diagnosed,  and  compared,  thereby  allowing  dynamic  instructional 
strategies  to  develop  in  step  with  the  students*  progress. 

2.  Consideration  for  Multimedia  Educational  Systems 

In  this  chapter,  we  consider  major  problems  for  developing  multimedia  educational  system  on  the  Web. 

2.1  Problems  of  the  Web  as  educational  infrastructure 

Students  do  not  require  only  delivering  information,  but  learning  altogether.  The  major  problem  when  developing 
educational  system  on  the  Web,  is  very  weak  interactivity.  For  students  to  learn  more  effectively,  they  need  a high 
degree  of  interaction.  But  most  current  Web  material  is  represented  as  HTML,  so  the  Web  today  provides  only  low 
level  interactionfBolk  and  Britton  98]. 
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The  second  problem  is  poor  ability  for  representir^  materials.  Because  the  Internet  depends  on  HTML,  the  materials 
seem  to  poorly  designed  books  with  hypertext  capabilitiesfBolk  and  Britton  98]. 

To  increase  interactivities  and  multimedia  facilities,  there  have  been  many  studies  such  as  Java,  CXjI,  ActiveX,  and 
some  extensions  of  HTML.  But  these  methods  are  not  fit  for  educational  requirements.  Most  students  need 
individual  learning  only  for  themselves.  When  considerii^  difference  among  students,  this  is  very  important.  But  the 
Web  lacks  capabilities  for  individual  learning. 

2.2  The  MHEG  standard 

The  MHEG  standard  provides  representation  of  multimedia  objects  on  heterogeneous  networked  environmentflSO 
95].  It  can  support  real  time  user  interaction  when  presentation.  Multimedia  information,  which  encoded  by  the 
MHEG  standi,  is  the  MHEG  objea.  As  using  MHEG  objeas  as  inherited  form  in  applications,  we  require  an 
environment  to  present  these  objects.  The  MHEG  standard  defined  such  environment  as  “engine”  [ISO  95]. 

The  MHEG-5  standard  is  the  fif&  subset  of  the  MHEG  standard.  It  defines  some  classes  in  detail.  Those  classes  are 
appropriate  to  some  applications  such  as  video  on  demand,  audio  on  demand,  interactive  TV  and  hypermedia 
navigation  [ISO  96]. 

So,  we  apply  MHEG,  the  standard  of  multimedia  information  representation,  when  developing  multimedia 
education^  system  on  the  Web. 

3.  System  Design 

In  this  chapter,  we  design  multimedia  system  that  has  intell^ent  educational  capabilities.  Figure  1 shows  the  system 
configuration. 


When  learning,  students  execute  web  browser  that  is  able  to  run  Java  applet.  Through  log  in  process,  courseware 
server  sends  materials  to  students  in  forms  of  MHEG  objeas.  Qient  browser,  include  MHEG  ermine,  presents  those 
materials  to  them.  The  system  logs  student  s learning  history  into  student  module,  and  diagnoses  them  in  expert 
module,  and  decides  student  s next  course  in  instmctional  strategy  module. 

When  students  participate  in  synchronous  education  program,  such  as  video  lecturing,  session  management  module 
monitors  students’  activities.  Event  module  processes  happened  user  events,  and  interact  with  educational  modules 
include  student  module. 

3.1  Educational  Modules 

Educational  modules  consist  of  expert  module,  student  module,  and  instmctional  strategy  module. 

3. 1. 1 Expert  module 

The  domain  knovdedge  will  be  used  in  coaching  the  students,  diagnosing  possible  misconceptions  after  a student  has 
made  an  error,  and  providing  appropriate  remedial  materials.  This  domain  model  has  dual  representation  network, 
the  IUN(Instmctiond  Unit  Network)  and  the  CN(Conceptual  Network). 
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Within  the  lUN,  each  node  represents  a learning  unit,  and  it  includes  the  theme  of  units,  learning  aim,  the  link  of  the 
concepts  related  to  the  learning  units,  the  links  to  the  problems,  and  the  links  to  the  subordinate  units. 

There  are  two  types  of  the  instructional  unit.  One  is  ^e  explain  unit,  the  other  is  the  problem  unit.  The  explain  unit 
has  instructional  materials  that  have  various  media  types  such  as  text,  graphic,  still  image,  sound,  and  moving  picture. 
The  problem  unit  has  two  major  concept  to  diagnosing  the  student’s  state,  difficulty  and  importance.  These  concepts 
are  used  when  lesson  planning  and  gradual  instruction  objective  creating. 

The  CN  is  a knowledge  network  that  piles  up  the  knowdedge  of  being  contained  in  subject  domain  to  be  learned  by 
the  student.  The  CN  is  a data  structure,  which  is  a set  of  coUeaed  knowdedge  concerned  with  the  declaring  concept 
of  domain  knowledge,  but  independent  upon  the  instructional  knowdedge.  Each  node  within  the  CN  represents  the 
instructional  concept,  and  the  links  between  the  concepts  indicate  the  relationship  of  them.  Using  these  relations,  it 
is  possible  to  generate  problem  dynamically,  and  to  diagnose  the  student’s  error. 

According  to  environments  of  ^e  CORBA,  expert  module  can  access  CORBA  ORB  to  assist  easy  retrieval  of 
materials,  and  URL  indexing  and  caching  demon  to  improve  performance  of  system. 

Unlike  existing  stand-alone  ITS,  user  verification  have  to  be  supported  so  as  to  provide  the  individual  learning 
according  to  the  rate  of  each  student. 

3. 1.2  Student  module 

The  instructional  strategy  module  uses  the  diagnostic  result  generated  by  student  module  in  order  to  inference  the 
basis  for  determination  of  learning  progress,  supplying  advice,  and  selecting  comments.  In  this  paper,  the  student 
module  is  based  on  the  overlay  model.  Ihe  diagnostic  results  of  the  student,  problem  type,  and  examples  are  saved  at 
nodes  in  the  student  knowledge  network  And  the  form  of  student  knowdedge  network  represents  the  learning  path. 
When  the  domain  knowdedge  expert  constructs  the  lUN  and  the  CN  by  means  of  authoring  interface,  the  two 
networks  contain  slots  represented  by  linguistic  variables  for  holding  importance  and  difficulty  that  are  the  attributes 
of  each  problem  and  instructional  unit. 

The  linguistic  variables  about  the  problem  are  used  for  diagnosing  student's  response.  Usir^  the  linguistic  variables, 
the  domain  expert  can  input  at  more  user- friend  environment  because  of  being  able  to  use  lir^t^tic  variable  and  no 
need  to  decide  special  re^  number  for  attributes  of  all  learning  elements.  The  linguistic  variable  is  the  kernel  of  fuzzy 
theory  that  makes  use  of  the  fuzzy  representation  of  human  being  in  computer.  Usir^  the  concentration  and  dilation 
in  the  fuzzy  theory,  we  can  define  the  function  that  diagnoses  the  student's  current  knowledge 
The  author  can  change  the  membership  functions  upon  “Good”,  “Medium”,  and  “Bad”,  and  the  diagnostic  results 
can  also  vary  according  to  the  membership  function  applied.  Hence,  the  author  can  control  the  diagnostic  results  by 
changing  these  three  membership  functions. 

Normalmtion  is  defined  as  follows:  suppose  that  all  problem  is  correct,  then,  calculate  the  DV  and  divide  into  the 
result  of  calculation  about  correct  answer.  If  the  importance  and  the  difficulty  are  applied  at  the  same  time  to  the 
same  problem,  select  one  that  miriiriiizes  the  membership  degree  and  applies  it. 

To  use  more  various  values  for  representir^  the  importance  or  the  difficulty,  we  must  consider  modifiers  applicable 
to  the  linguistic  variables 

regardless  of  its  number  of  repetition.  Hence,  the  inconsistency  of  di^nostic  result  caused  by  repeated  applications 
of  the  modifier  is  removed.  For  example,  “Eas/’  is  never  less  than  “Very  Eas/’  in  spite  of  the  number  of  “Litde” 
modifier's  applications. 

The  calculation  result  by  DV  function  for  the  di^nosis  of  student's  response  is  a real  number  that  raises  between 
zero  and  one.  To  execute  fuzzy  inference  using  linguistic  variable,  result  obtained  by  the  DV  function  has  to  be 
fuzzified.  Fuzzified  result  is  one  of  the  linguistic  variable  that  consists  of  Good,  Medium,  Bad,  and  the  modifier 
[Very]'*'',  [Less]'^ 

If  we  fuzzify  the  diagnostic  results  into  the  linguistic  variables,  then  they  can  be  used  together  with  the  inference 
rules  using  the  linguistic  variables. 

If  the  author  input  the  inference  rule  and  there  was  conflicting  among  those  rules,  that  is,  different  result  is  input  at 
the  same  condition,  the  author  is  presented  the  condition  and  is  asked  what  result  should  be  used.  The  result  of 
learning,  whether  well  or  poor,  is  determined  according  to  what  deterministic  variable  is  input  by  the  author.  This 
shows  that  the  author's  intention  can  be  reflected  to  inference  rule. 

3. 1.3  Instructional  strategy  module 

The  instructional  strategy  module  mediates  and  develops  the  learning  of  the  students  through  the  planning  of  a 
curriculum  to  order  the  domain  knowledge  that  must  be  supplied  to  students,  and  the  planning  of  teaching  to  decide 
the  presentation  method,  re- planning  to  construct  proper  planning  and  monitor  the  action  of  students. 

Among  the  contents  of  a lesson  which  are  taught  to  the  students,  the  total  contents  of  a lesson  are  composed  of  the 
network  of  the  main  objectives  of  instmctions  that  are  relatively  independent  each  other. 

Lesson  plan  presents  the  evaluation  problem  and  the  instmctional  material  of  instructional  unit,  decided  by  the 
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curriculum  planning  to  students  in  the  proper  level.  The  Instructional  strategy  module  offers  students  the 
instmaional  material  on  the  instmction  unit  and  the  evaluation  problem  by  means  of  the  method  divided  by  the 
curriculum  planning. 

The  gradual  instruction  objective  must  be  created  by  considering  the  student  level  that  has  acquired  the  degree  of 
difficulty  of  instmction  unit  and  students*  records  obtained  previously.  For  an  efficient  instmction,  problem 
presentation  order  must  be  planned  dynamically  by  according  to  the  smdents*  level.  The  instmctional  strategy 
module  can  create  dynamically  the  problem  about  the  concept  learning  and  the  discrirniriation  learning  using  the  CN. 
Also  instmctional  strategy  module  can  dynamically  limit  the  number  of  links  which  the  smdent  can  selea,  according 
to  the  level  of  each  student.  This  module  have  functions  which  provide  the  smdents  for  advice  when  the  smdents 
refer  the  irrelevant  to  the  present  leamir^  content  frequendy  or  when  disorientation  problem  occurs. 

3.2  Activity  Managing  Modules 

In  this  section,  we  design  session  managing  module  for  providing  time  dependent  group  activities  for  synchronous 
learning/ training.  Table  2 defines  operations  for  it. 


Table  2 Synchronous  leaming/training  operations 


▼'"r  7 XllraWri  All  I 

” 1 X I X r : 

Basic  Operations 

Create  Group 

Oestroy  Group 

Group  Manaiement  Operations 

Join  Group 

Leave  Group 

Manage  Applications 

ManageTime  Event 

Control  Token 

Manage  Group  Context 

Oata  Transmission  Operation 

SupportUnicast/Muiticast 

information  Mana|ement  Operation 

Manage  and  Retrieve  User/Gronp  Information 

Figure  3 stands  out  session  managing  module  configuration. 
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Figure  3.  Sesssion  Managing  Module 

Group  manager  creates  groups  as  response  for  user  requests,  and  manages  them  into  group  pool.  Events  caused  at 
each  group  are  sent  to  event  handler,  and  processed  using  group  managing  interface. 


3.2. 1 Event  Handler 

Information  sent  to  event  handler  consists  of  event  data  and  consumer  information.  This  information  is  managed  as 
RFO  stmcture.  Time  event  manager  provides  time  dependent  group  activities.  The  creator  can  set  lifetime  of  group, 
in  this  case,  a group  whose  lifetime  ended  closed  automatically.  This  facility  is  usable  when  lecturing  has  time 
dependency  in  educational  system. 

Time  event  manager  manages  information  per  group  unit.  Time  event  service  process  time  event  defined  by  user. 
When  predefined  event  caused,  it  send  it  to  group  manager. 


3.2.2  Group  Manager 

Group  manager  manages  group  context  that  consists  of  students  information,  operates  for  interaction  among 
students.  Group  context  includes  student  information  who  participate  group  currently,  lecturer  s information.  Group 
manager  maintains  context  recendy  processed  event.  When  new  event,  such  as  participate  and  leave,  caused  context 
switching  to  appropriate  context.  Wlien  context  switching  caused,  k notifies  students  of  that  case  using  event 
handler. 
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User  can  play  a role  of  “Group  Member”  or  “Inspector”  and  lecturer  has  a “Group  Qiairman”  role.  Lecturer  as 
group  creator  has  some  privileges.  Lecturer  can  give  roles  to  students,  close  group  oppressively,  modify  and  manage 
group  information. 

A student  must  own  token  when  transmit  data  to  other  students.  When  a lecturer  requires  token,  group  manager  give 
token  to  lecturer  immediately.  But  when  a student  asks  token,  group  manager  sends  that  request  to  the  lecturer  and 
then  the  lecturer  arranges  that.  During  context  switching,  such  as  changing  token  owner,  participate  or  leave,  group 
manager  sends  that  to  event  manager. 

Group  manager  provides  “SendMessageToAll”  operation  when  a student  sends  message  to  all  participants.  Session 
manager  invokes  it  and  group  manager  sends  real  data  consists  of  student  identifier  and  message  contents  using 
event  handler. 


3.3  MHEG  Engine  on  Client  Side 

In  this  section,  we  des^n  MHEG  engine  that  can  support  real  time  user  interaaion  and  standard  multimedia 
representation.  F^;ure  4 shows  the  MHEG  engine  configuration. 


Figure  4 MHEG  engine  on  client  side 

The  engine  consists  of  objea  mani^ing  module  and  presentation  manager.  The  encoder  and  the  decoder  are  based 
on  class  library.  The  class  library  serves  converting  between  runtime  objects  and  the  MHEG  objeas.  The 
presentation  manager  includes  objea  interpreter  and  the  interaction  handler. 


3.3.1  Object  Managing  Module 

In  our  class  library,  we  define  the  base  class  to  inherit  subclass.  The  MH5 Object  class,  the  base  class,  has  method 
that  can  encode  and  decode  one  BER(Basic  Encoding  Rule)  item.  Each  member  variable  of  the  MH5 Object  class  is 
mapped  into  octets  of  the  BER  The  MH5 Objea  class  has  methods  for  processing  oaets.  It  decodes  the  BER  items, 
and  provides  decoded  information  to  child  classes.  It  also  encodes  information,  which  child  classes  specify,  to  the 
BER 

The  root  class  is  parent  class  of  all  other  MHEG  classes.  The  root  class  has  an  identifier  of  the  application  class.  The 
group  class  includes  the  aaion  object  that  must  be  executed  after  the  MHEG  application  activated.  Users  can 
decode  the  application  object  by  invoking  member  mahod.  During  encoding  of  the  application  object,  we  must  add 
the  BER  lengtlis  of  lower  level  objeas  to  length  octet  of  higher  level  objea.  Thus,  we  need  two  encoding  phases. 

The  scene  class  group  defines  the  ir^redient  classes  activated  simultaneously.  Unlike  the  application  class,  the  scene 
class  includes  member  variables  that  are  required  for  presentation  such  as  position  and  size.  But,  encoding  and 
decoding  of  the  scene  object  is  sarrie  as  them  of  application  class. 

3.3.2 Presentation  Manager 

The  objea  interpreter  analyzes  internal  object  from  decoder,  converts  them  to  objea  lists,  and  then  stores  in  the 
object  pool.  The  internal  object  from  decoder  is  a tree- like  structure  whose  root  is  the  application  class.  The  object 
handler  extracts  behaviors  of  presentable  objects,  and  relocates  objects. 

When  the  object  interpreter  converts  objects,  we  use  temporal  and  spatial  informatioa  Since  objects  are  relocated  in 
accordance  with  spati^  information,  spatial  synchronization  and  simple  temporal  synchronization  in  list  stmcture  are 
accomplished.  The  objea  pool  contains  lists  of  internal  objects. 

The  purpose  of  interaction  classes,  such  as  hypertext,  entry  field,  button,  and  slider,  is  supporting  interaaion  with 
users.  The  interaaion  handler  gathers  and  processes  user  inputs  and  events  from  presentable  objeas.  The  interaction 
handler  analyzes  transmitted  events,  and  seleas  action  objects.  After  that,  k analyzes  those  action  objeas,  and  sends 
appropriate  messages  to  message  handling  routine.  Each  interaaion  requires  an  action  that  selects  certain  object  or 
modifies  one  or  more  objects. 

All  input  has  theirs  own  type  in  accordance  with  the  interaction  classes.  The  interaction  classes  are  divided  into  real- 
time interaction  classes  and  non  real-time  interaction  classes.  Table  3 shows  types  of  user  input  for  each  MHEG- 5 
user  interaction  class. 
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Table  3 Types  of  user  input 


SoataTypSuo 

?i|i 

Hypertext 

Select/not 

Boolean 

real'time 

Entry  Field 

Text  String 

String 

non  real-time 

Push  Button 

Click/  Not 

Boolean 

real-time 

Radio  Button 

Press/Release 

Boolean 

non  real-time 

Hot  Spot 

X,Y  Position 

Integer  pair 

real-time/ 
non  real-time 

Slider 

Value 

Integer 

real-time 

Link  class  consists  of  link  condition  and  link  effect.  Link  condition  is  a condition  that  invokes  a link  operation.  It  can 
be  a combination  of  other  conditions.  Link  effect  is  an  action  list  that  is  executed  when  a link  operates.  Link  effect 
consists  of  event  source  and  event  data.  Event  data  is  a user  interaction  class  that  causes  an  event  for  link  operation. 
The  interaction  handler  compares  event  data  to  user  input.  If  this  comparison  returns  true,  the  interaction  handler 
executes  actions  specified  in  defined  condition.  Link  effect  is  to  be  an  elementary  action  or  action  list.  Action  list  can 
include  an  action  list.  In  case  of  this,  aaion  size,  the  whole  size  of  aaion  list,  is  required.  Action  targets,  which  are 
modified  or  selected  by  each  action,  are  also  required. 

4.  Conclusion  and  Future  Work 

We  design  multimedia  system  that  has  intelligent  education  capabilities  on  the  Web.  Our  system  has  several 
advantages  than  legacy  systems  on  the  Web.  We  provide  individual  learning  facilities  using  linguistic  variables  and 
fuzzy  logic.  And  we  provide  more  interactivities  using  MHEG  standard.  We  can  distribute  educational  system  on  the 
Web  using  OORBA  Our  system  is  flexible  for  distance  learning  and  trairiing  that  requires  more  specific  and 
dedicated  user  interaction. 

Of  course.  We  consider  introduce  XML  to  our  system  for  multimedia  representation  on  the  Web.  It  accomplished, 
we  can  provide  more  generalized  education  features  on  the  Web. 
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Abstract:  A performance  support  system  provides  a mechanism  by  which  human  skill 

levels  within  a particular  task  domain  can  be  improved.  This  paper  discusses  the  growing 
importance  of  distributed  performance  support  systems  and  their  use  within  the  context  of 
creating  a ‘learning  organisation’.  The  design  and  development  of  a distributed  performance 
support  system  called  ‘CLASS’  is  described.  Its  use  as  a performance  tool  is  then  compared 
with  conventional  approaches  to  aid  provision  within  the  context  of  creating  supportive 
environments  to  facilitate  the  use  of  computer-assisted  learning  within  an  organisational 
framework. 


1.  Introduction 

In  their  day  to  day  activities,  human  beings  are  often  required  to  perform  a wide  range  of  different  tasks  and 
jobs.  These  tasks  vary  quite  considerably  in  their  complexity.  The  successful  execution  of  many  of  the  more 
complex  tasks  often  requires  the  use  of  some  sort  of  tool,  aiding  facility  or  performance  support  system  (PSS). 
The  basic  rationale  underlying  the  use  of  a PSS  is  that  individuals  or  groups  of  people  (working  together  on  a 
common  project)  are  provided  with  appropriate  tools  and  techniques  to  support  the  tasks  that  they  have  to 
perform.  According  to  Barker  and  Hudson  (1998),  four  major  objectives  of  a performance  support  system  are: 
(1)  to  achieve  increases  in  productivity;  (2)  to  improve  the  overall  quality  of  task/job  execution;  (3)  to  improve 
the  overall  quality  of  the  environments  in  which  people  work;  and  (4),  from  a human  perspective,  to  reduce  the 
complexity  of  the  processes  involved  in  executing  a task. 

Simple  examples  of  performance  support  tools  include;  a typewriter,  a telephone,  a bicycle  and  an 
automobile.  Increasingly,  computer-based  resources  are  being  used  to  implement  performance  support  systems. 
An  electronic  performance  support  system  (EPSS)  is  therefore  a facility  which  uses  various  types  of  computer 
technology  to  realise  each  of  the  four  previously  listed  objectives  within  a given  working  environment  or 
problem  solving  domain.  Software  packages  such  as  word-processing  systems  and  spreadsheets  provide  a good 
example  of  computer-based  performance  support  systems. 

Naturally,  a fundamental  requirement  of  an  EPSS  is  that  it  should  increase  its  users’  on-the-job 
performance  within  a given  task  domain.  This  usually  involves  improving  skill  levels,  reducing  task 
complexity  and/or  providing  appropriate  training.  These  requirements  can  be  achieved  in  two  basic  ways. 
First,  through  the  provision  of  ‘automation  aids’;  and,  second,  by  providing  various  mechanisms  to  support  ‘on- 
the-job’,  ‘just-in-time’  (JIT)  training  which  will  enable  users  of  any  given  system  to  ‘learn  as  they  do’.  This 
latter  requirement  is  an  important  aspect  of  building  a ‘learning  organisation’  (Lassey,  1998). 
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According  to  Malhotra  (1996),  a learning  organisation  is  one  which  has  'an  ingrained  philosophy  for 
anticipating,  reacting  and  responding  to  change,  complexity  and  uncertainty*.  Because  organisations  must 
adapt  dynamically  to  the  forces  generated  by  change  agents,  ongoing  learning  and  training  activities  are  a 
fundamental  requirement  to  their  success.  As  has  been  discussed  elsewhere  (Gery,  1991;  Beacham,  1998; 
Barker,  van  Schaik  and  Hudson,  1998),  performance  support  systems  are  one  way  of  realising  this  requirement 
within  an  organisational  context. 

Many  of  the  early  developments  in  EPSS  have  been  described  by  Gery  (1991),  McGraw  (1994), 
Vamadoe  and  Barron  (1994),  Raybould  (1995),  Banerji  (1995)  and  Barker  (1995a).  Currently,  there  are  two 
important  directions  of  development  for  EPSS  facilities.  First,  the  creation  of  integrated  toolsets  that  are  able  to 
meet  the  performance  support  needs  of  particular  application  domains  (Barker  and  Hudson,  1998;  Hudson, 
1 998);  and  second,  a growing  requirement  to  facilitate  group  working  at  a distance  through  the  incorporation  of 
network  technology  and  the  principles  of  distributred  computing  environments  (Beacham,  1998).  Indeed,  the 
advent  of  relatively  low-cost  network  technologies  and  the  widespread  availability  of  facilities  such  as  the 
Internet  and  the  World  Wide  Web  has  meant  that  performance  support  systems  can  now  become  highly 
distributed  in  nature  - that  is,  different  components  can  reside  (or  be  replicated)  at,  and  be  accessed  from, 
different  geographical  locations. 

As  far  as  this  paper  is  concerned,  we  therefore  regard  a distributed  performance  support  system 
(DPSS)  as  an  integrated  and  globally  accessible  collection  of  electronic  tools  and  data  that  can  be  used  (as  and 
when  required)  at  particular  points  of  need  (within  a workgroup  or  an  organisation)  in  order  to  improve  human 
performance  within  a given  task  domain.  Because  of  their  future  importance  as  a resource  for  developing 
learning  organisations,  the  remainder  of  this  paper  describes  and  discusses  the  evolution  and  future  potential 
utility  of  distributed  performance  support  systems.  The  design  and  construction  of  a single-site,  multiple-server 
distributed  performance  support  facility  called  CLASS  (‘Computer  Learning  and  Support  System’)  is  then 
described  and  an  outline  is  given  of  some  the  ways  in  which  we  have  been  using  it.  An  attempt  has  been  made 
to  measure  the  potential  utility  of  the  system  from  the  perspective  of  supporting  its  end-users’  use  of  computer- 
assisted  learning  (CAL)  packages.  The  findings  from  this  work  are  briefly  reviewed. 


2.  Case  Study  - The  CLASS  Facility 

This  section  of  the  paper  briefly  discusses  the  background  to  the  DPSS  work  that  we  have  recently  been 
undertaking.  The  implementation  of  the  CLASS  facility  is  then  outlined.  Finally,  a description  is  given  of  the 
ways  in  which  we  have  been  using  CLASS  to  support  computer-assisted  learning  (CAL)  and  computer-based 
training  (CBT)  within  the  context  of  a learning  organisation. 


2.1  Background 

Some  of  our  early  work  in  developing  EPSS  facilities  has  been  described  by  Banerji  (1995)  and  Barker  (1995a; 
1995b).  Banerji  showed  how  the  performance  of  students  using  a file  transfer  program  (kermit)  could  be 
improved  through  the  provision  of  PC-based,  online  help  and  interactive  aiding  and  training  facilities. 
Similarly,  Barker  (1995b)  has  shown  how  the  emerging  principles  of  performance  support  can  be  used  to  apply 
EPSS  techniques  to  the  design  and  development  of  integrated  support  tools  for  use  by  students  and  staff  in  an 
academic  setting  within  a university  environment.  Each  of  these  applications  involved  the  development  of 
stand-alone  EPSS  tools.  Our  first  attempt  at  building  a distributed  performance  support  system  involved  using 
a computer  network  system  (the  Internet)  in  order  to  access  different  forms  of  subject  expertise  that  resided  at 
different  geographical  locations  (Barker,  Richards  and  Banerji,  1994).  This  work  has  recently  been  extended 
and  enhanced  through  the  use  of ‘mobile  computing’  environments  (Barker,  1996) 

Many  of  the  presently  available  distributed  performance  support  systems  that  are  now  in  use  have 
evolved  from  conventional  stand-alone  EPSS  facilities  (such  as  those  described  above)  in  two  basic  ways.  First, 
by  providing  ‘dial-in’  access;  second,  by  implementing  systems  that  can  be  shared  in  an  in-house  fashion 
through  the  use  of  local  area  networks.  Current  research  in  this  area  now  seeks  to  incorporate  activities  of  this 
sort  within  a more  holistic  approach  that  also  involves  the  utilisation  of  global  trans-world  networks  such  as  the 
Internet  and  World  Wide  Web. 
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In  an  attempt  to  rationalise  activity  in  this  area,  we  have  proposed  a DPSS  taxonomy  that  incorporates 
two  basic  classification  dimensions:  first,  the  number  of  information  servers  that  are  involved;  and  second,  the 
number  of  interlinked  sites  that  make  up  the  network  topology.  These  dimensions  permit  a basic,  four-class 
taxonomy  to  be  proposed  (Beacham,  1998).  The  classes  within  this  taxonomy  are:  single-site,  single-server; 
multiple-site,  single-server;  single-site,  multiple-server;  and  multiple-site,  multiple-server.  The  CLASS 
facility,  that  is  described  in  the  following  section,  is  an  example  of  a single-site,  multiple-server  system. 


2.2  Implementing  CLASS 

The  development  of  CLASS  took  place  in  three  stages.  The  first  stage  involved  conducting  a needs  analysis. 
Subsequently,  in  the  second  stage,  a prototype  system  was  built  which  was  used  for  formative  evaluation 
purposes.  In  the  third  stage,  the  prototype  system  was  augmented  and  expanded  into  an  operational  DPSS. 
The  needs  analysis  and  implementation  rationale  are  briefly  described  below. 


The  Needs  Analysis 

The  needs  analysis  was  undertaken  within  the  School  of  Business  and  Management  (SBM)  at  the  University  of 
Teesside  in  the  UK.  Its  main  aims  were:  (1)  to  identify  the  main  limitations  (if  any)  of  current  course  provision 
within  the  SBM;  and  (2)  to  explore  how  CAL  and  PSS  techniques  might  be  used  to  make  any  improvements 
that  were  thought  to  be  necessary.  Both  staff  (teaching,  research  and  administrative)  and  student  (full-time  and 
part-time)  perspectives  were  studied  using  questionnaire-based  techniques.  In  some  situations,  ‘follow  up’ 
interviews  were  conducted  in  order  to  clarify  issues  that  were  not  clear  or  in  order  to  obtain  more  information 
about  particular  aspects  of  the  study. 

The  results  of  the  SBM  needs  analysis  were  compared  with  two  other  related  surveys  (one  internal 
and  one  external)  in  order  to  look  for  similarities  and  trends.  The  internal  (campus-wide)  study  was  conducted 
by  the  University  of  Teesside’s  Library  and  Information  Service  (as  part  of  its  forward  planning  activities).  The 
external  survey  was  obtained  from  another  UK  university.  As  is  discussed  elsewhere  (Beacham,  1998),  the 
findings  from  both  of  these  other  surveys  supported  many  of  the  findings  of  the  SBM  needs  analysis. 


Implementation  Rationale 

The  findings  from  the  needs  analysis  were  used  to  ‘steer’  the  design  of  the  CLASS  facility  and  also  set 
priorities  with  respect  to  the  order  in  which  computer-based  resources  and  online  facilities  were  created  and 
made  available.  As  was  mentioned  above,  the  initial  design  and  development  strategy  involved  building  a 
prototype  system.  This  was  based  on  the  use  of  an  in-house  intranet  facility  and  incorporated  a virtual 
university  metaphor  (Barker,  1999).  This  allowed  staff  and  students  to  use  virtual  offices,  a virtual  library  and 
virtual  classrooms.  HTML  (Version  4)  was  used  as  the  development  language  and  Netscape  Navigator 
(Version  4)  was  used  as  the  access  and  delivery  vehicle.  Extended  interactivity  within  the  system  was  achieved 
through  the  use  of  CGI  scripting  techniques,  JavaScript  and  Java  applets.  Appropriate  use  was  also  made  of 
‘helper’  applications  (primarily  for  Microsoft’s  PowerPoint,  Word  and  Excel  packages)  and  Netscape  ‘plug- 


2.3  Using  CLASS  for  Performance  Support 

This  section  of  the  paper  briefly  describes  the  main  ways  in  which  the  CLASS  facility  has  been  used  to  support 
teaching,  learning  and  administration  activities  within  the  SBM  at  the  University  of  Teesside. 

Once  an  intranet  facility  had  been  created  within  the  SBM  it  became  possible  to  use  it  to  store, 
retrieve,  deliver  and  maintain  virtually  all  course-related  documentation.  By  dispensing  with  ‘paper’  and 
providing  electronic  access  to  these  materials  through  computer  terminals,  the  productivity  of  staff  and  students 
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increased  substantially  - in  terms  of  ease  of  access  to  course-related  information  and  the  ease  with  which  this 
could  be  modified  and  updated. 

The  basic  rationale  underlying  our  use  of  CLASS  for  the  support  of  teaching  and  learning  activities  is 
similar  to  that  outlined  by  Barker  et  al  (1995)  in  their  description  of  the  use  of  a global  performance  support 
system  for  students  and  staff  However,  the  CLASS  system  goes  much  further,  in  that,  it  embeds  a virtual 
university  metaphor  and  allows  various  approaches  to  both  synchronous  and  asynchronous  teaching  and 
learning  activities.  Some  of  the  facilities  that  CLASS  contains,  or  provides  access  to,  include;  online  lecture 
notes;  copies  of  lecture  presentations  (PowerPoint);  problems  and  worked  solutions;  course -related  CAL/CBT 
packages;  details  of  project  work;  tutorial  and  workshop  descriptions;  email  facilities;  class  schedules; 
conferencing  facilities;  electronic  notice-boards;  automatic  marking  of  certain  types  of  submitted  work;  and 
automated  student  self-assessment  of  progress.  Although  CLASS  is  by  no  means  in  its  final  form,  the  feedback 
that  we  have  received  with  respects  to  its  acceptance  and  utility  provides  significant  motivation  for  its  ongoing 
development.  This  feedback  is  discussed  in  the  following  section. 


3.  Evaluating  Class 

An  extensive  evaluation  of  the  CLASS  facility  has  been  undertaken.  Details  of  this  are  presented  elsewhere 
(Beacham,  1998).  This  section  of  the  paper  provides  an  overview  of  the  methods  used  and  the  results  obtained. 


3.1  Methodology 

A four-step  evaluation  strategy  was  used.  This  involved  assessing  both  staffs  and  students’  attitudes  towards 
four  different  approaches  to  the  provision  of  support  to  facilitate  the  teaching  and  learning  of  a particular  topic 
X within  a given  module  T of  a degree  course  Z.  The  four  situations  that  were  considered  were:  (1)  minimal 
support;  (2)  conventional  support  with  ‘add-on’  CAL;  (3)  conventional  support  with  ‘ownership’  of  CAL;  and 
(4)  the  use  of  a DPSS.  In  case  (1),  a CAL  package  was  used  to  teach  the  topic  (X);  no  lectures  were  given  and 
tutorials  were  organised  in  an  ad-hoc  basis,  as  and  when  they  were  needed.  Case  (2)  involved  a conventional 
teaching  situation  (with  scheduled  lectures  and  tutorials  on  topic  X)  with  students  using  the  CAL  package  as  an 
‘add  on’  extra.  Resource-wise,  case  (3)  was  similar  to  case  (2)  but  involved  re-organising  the  way  in  which  the 
CAL  package  was  used;  in  this  approach  the  CAL  program  was  used  (by  the  lecturer)  as  a central  resource  (for 
demonstration  and  illustration  purposes  within  lectures,  tutorials  and  workshops)  and  by  students  (for  the 
support  of  self-study  activities).  Case  (4)  was  the  same  as  case  (3)  but  with  the  addition  of  a purpose-built 
DPSS  facility  (CLASS)  to  augment  the  course  Z (in  general)  and  the  teaching  of  the  module  Y and  topic  X (in 
particular). 

In  each  of  the  evaluation  scenarios  described  above,  questionnaires  were  used  in  order  to  obtain  the 
opinions  of  staff  and  students  with  respect  to  pedagogic,  technical,  logistic,  organisational  and  resource-based 
issues.  Some  of  the  results  that  were  obtained  are  summarised  in  the  following  section. 


3.2  Results 

As  was  mentioned  above,  four  basic  evaluation  scenarios  were  employed.  These  were  used  to  gather  feedback 
about  the  different  types  of  support  environment  (for  using  CAL  packages)  and,  in  particular,  the  potential 
utility  of  the  CLASS  facility  as  a support  mechanism  for  teaching  and  learning  within  an  organisational 
context.  Based  on  the  various  types  of  questionnaire  that  were  employed  in  the  evaluation,  a set  of  six  ‘derived’ 
attributes  were  identified  as  a basis  for  comparing  the  different  modes  of  support.  The  six  attributes  that  were 
used  are  listed  in  (Tab.  1)  along  with  the  attribute  values  that  have  been  derived  from  the  questionnaire  data. 
Overall,  330  student  volunteers  participated  in  the  evaluation  process.  All  students  were  involved  in  studying 
the  topic  of  ^Marginal  Costing"  as  part  of  a degree  module  entitled  "Introduction  to  Financial  Decision 
Making"  (the  total  enrolment  for  this  course  at  the  time  the  evaluations  were  conducted  was  549).  The 
distribution  of  students  over  the  different  evaluation  scenarios  listed  in  (Tab.  1)  was  as  follows:  Case  1 (61); 
Case  2 (183);  Case  3 (43);  and  Case  4 (43).  Some  members  of  staff  also  participated  in  two  of  the  evaluation 
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scenarios  (Cases  3 and  4).  However,  the  data  derived  from  the  staff  questionnaires  is  not  included  in  (Tab.  1); 
this  is  presented  and  discussed  elsewhere  (Beacham,  1998). 


Attribute  Assessed 

Case  1 
(Minimal) 

Case  2 
(Add-on) 

Case  3 
(Ownership) 

Case  4 
(CLASS) 

Knowledge  of  topic  after  using 
CBL  package 

Low 

Moderate 

Very  Low 

High 

Like  using  the  CBL  package 

Very  Low 

Low 

High 

High 

Factors  limiting  use  (difficulty 
with  package) 

Low 

Low 

High 

Very  Low 

CBL  package  easy  to  use 

Low 

Very  Low 

High 

High 

Environment  contains  features 
that  are  helpful 

Very  Low 

Low 

High 

Very  High 

The  range  of  sources  available 
for  practice 

Very  Low 

Low 

Low 

High 

Table  1:  Summary  of  Evaluation  Results 


3.3  Discussion 

As  might  be  expected,  the  results  shown  in  (Tab.  1)  clearly  reflect  the  apparent  superiority  of  Cases  3 and  4 
over  Cases  1 and  2.  Undoubtedly,  this  arises  because  the  ‘ownership’  model  of  CAL  usage  and  the  integrated 
holistic  approach  provided  by  a DPSS  (CLASS)  each  embed  much  richer  teaching  and  learning  strategies. 
Having  said  this,  inspection  of  the  results  in  row  1 of  (Tab.  1)  suggests  that  the  topic  knowledge  transferred  by 
CAL  techniques  using  Case  3 seems  to  be  less  than  that  which  takes  place  in  Cases  1 and  2.  This  is  indeed 
surprising  since  we  would  have  anticipated  that  the  knowledge  transfer  achieved  using  this  approach  to  CAL 
package  utilisation  would  be  comparable  with  that  observed  in  Case  4.  We  believe  one  possible  reason  for  this 
anomaly  is  that  the  high  level  of  knowledge  transfer  that  took  place  in  lectures  and  tutorials  (using  the 
‘ownership’  model)  reduced  the  need  for  students  to  make  additional  use  of  the  CAL  package  themselves  as  a 
self-study  resource. 

As  can  be  seen  from  (Tab.  1),  the  only  approach  which  scores  positively  in  all  six  attributes  is  Case  4. 
These  results  reflect  that  students  felt  that  there  were  particular  benefits  to  be  gained  by  embedding  a CAL 
package  within  the  CLASS  facility;  namely,  it  was  easier  to  use,  it  reduced  the  amount  of  difficulty  experienced 
and  provided  a large  number  of  sources  from  which  to  obtain  information  and  practice  skills.  Bearing  this  in 
mind,  we  believe  that  these  findings  make  a sound  case  for  the  more  extensive  use  of  the  DPSS  approach  to  the 
provision  of  aiding  and  support  facilities. 


4.  Conclusion 

Computers  and  electronic  communications  technologies  are  increasingly  being  used  to  provide  mechanisms  for 
the  support  of  training,  teaching  and  learning.  These  technologies  make  it  possible  to  augment  conventional 
approaches  to  instruction  and,  more  importantly,  provide  mechanisms  to  facilitate  ‘education  on  demand’  in  a 
variety  of  different  situations  and  contexts  - such  as,  in  the  home,  in  the  workplace,  in  libraries  and  in  colleges. 
They  also  provide  facilities  for  developing  new  approaches  to  skill  and  knowledge  acquisition  in  the  workplace 
through  techniques  such  as  ‘on-the-job’  learning  and  ‘just-in-time’  training.  Within  the  context  of  creating 
‘learning  organisations’,  an  integrated,  holistic  approach  to  the  provision  of  access  to  teaching/leaming 
resources  is  now  needed.  We  believe  that  this  requirement  can  be  fulfilled  through  the  provision  of  distributed 
performance  support  systems  similar  to  the  CLASS  facility  that  has  been  described  in  this  paper. 


351 


References 


Banerji,  A.K.,  (1995).  Designing  Electronic  Performance  Support  Systems,  PhD  Thesis,  School  of  Computing  and 
Mathematics,  University  of  Teesside,  Middlesbrough,  UK. 

Barker,  P.G.,  (1995a).  Electronic  Performance  Support  Systems,  Special  Issue  of  Innovations  in  Education  and  Training 
International,  32(  1 ),  1-73. 

Barker,  P.G.,  (1995b).  Emerging  Principles  of  Performance  Support,  Proceedings  of  ONLINE  Information  ‘Pi,  Learned 
Information  (Europe),  Oxford,  UK,  407-416. 

Barker,  P.G.,  (1996).  Towards  Real  Information  on  Demand,  Proceedings  of  ONLINE  Information  '96,  Learned 
Information  (Europe),  Oxford,  UK  261-269. 

Barker,  P.G.,  (1999).  Using  Intranets  to  Support  Teaching  and  Learning,  Innovations  in  Education  and  Training 
International,  36(1),  3-10. 

Barker,  P.G.,  Banerji,  A.,  Richards,  S.  and  Tan,  C.M.,  (1995).  A Global  Performance  Support  System  for  Students  and 
Staff,  Innovations  in  Education  and  Training  International,  32(1),  35-44. 

Barker,  P.G.  and  Hudson,  S.R.G.,  (1998).  An  Evolving  Model  for  Multimedia  Performance  Support  Systems,  60-73  in 
Trends  in  Communication:  Part  4 Interface  Technology  - Enhancing  the  Quality  of  Life,  edited  by  H.  van  Oostendorp  and 
A.G.  Arnold,  Boom  Publishers,  The  Netherlands. 

Barker,  P.G.,  Richards,  S.  and  Banerji,  A.,  (1994).  Intelligent  Approaches  to  Performance  Support,  ALT-J  - Journal  of  the 
Association  of  Learning  Technology,  2(1)  63-69. 

Barker,  P.G.,  van  Schaik,  P.  and  Hudson,  S.R.G.,  (1998).  Mental  Models  and  Lifelong  Learning,  Innovations  in  Education 
and  Training  International,  35(4),  3 1 0-3 1 8. 

Beacham,  N.A.,  (1998).  Distributed  Performance  Support  Systems,  PhD  Thesis,  School  of  Computing  and  Mathematics, 
University  of  Teesside,  Middlesbrough,  UK. 

Gery,  G.,  (1991),  Electronic  Performance  Support  Systems  - How  and  Why  to  Remake  the  Workplace  through  the  Strategic 
Application  of  Technology,  Weingarten  Publications,  Boston,  MA. 

Hudson,  S.R.G.,  (1998).  Multimedia  Performance  Support  Systems,  Draft  PhD  Thesis,  School  of  Computing  and 
Mathematics,  University  of  Teesside,  Middlesbrough,  UK. 

Lassey,  P.,  (1998).  Developing  a Learning  Organisation,  Kogan  Page,  London. 

Malhotra,  Y.,  ( 1 996).  Organisational  Learning  and  Learning  Organisations:  An  Overview,  URL: 
http://www.brint.com/papers/orgling.htm. 

McGraw,  K.L.,  (1994).  Performance  Support  Systems:  Integrating  AI,  Hypermedia  and  CBT  to  Enhance  User  Performance, 
Journal  of  Artificial  Intelligence  in  Education,  5(1),  3-26. 

Raybould,  B.,  (1995).  Development  Tools  for  EPSS,  Journal  of  Interactive  Instructional  Development,  7(3),  7-10. 

Vamadoe,  S.  and  Barron,  A.E.,  (1994).  Designing  Electronic  Performance  Support  Systems,  Journal  of  Interactive 
Instructional  Development,  6(3),  12-17. 


er|c 


352 


Deployment  Scenarios  of  DVEs  in  Education 


C.  Bouras 

Computer  Technology  Institute,  Patras,  Greece,  bouras@cti.gr 
V.  Kapoulas 

Computer  Technology  Institute,  Patras,  Greece,  kapoulas@cti.gr 
A.  Koubek 

Technikum  Joanneum,  Graz,  Austria,  koubek@fh-joanneum.at 
H.  Mayer 

Joanneum  Research,  Graz,  Austria,  harald.mayer@joanneum.ac.at 


Abstract:  DVEs,  along  with  the  Internet  in  general,  offer  a number  of  attractive  features  for 
the  education  sector.  However  deployment  in  traditional  educational  environments,  such  as 
secondary  schools,  has  not  been  achieved  on  a noticeable  scale  so  far.  In  this  contribution,  we 
discuss  technical  and  pedagogical  issues,  and  specifically  define  possible  scenarios  of  how 
this  technology  can  be  introduced  to  the  school  sector  in  a sustainable  way. 


Introduction 

An  accelerating  factor  in  the  use  of  the  Information  and  Communication  Technologies  (ICT)  is  the 
recent  advances  in  the  fields  of  multimedia  and  telematics  applications.  Multimedia  and  telematics  comprise 
the  hard  core  of  ICT.  Multimedia  can  be  considered  as  the  combination  of  any  two  or  more  different  media 
types  (text,  graphics,  images,  audio,  and  video).  Furthermore,  hypertext  and  hypermedia  technology  uses 
hyperlinks  in  order  to  represent  in  a structured  way  information  that  could  be  in  various  media  types.  This 
technology  gives  the  user  the  ability  to  explore  the  information  in  a non-linear  format  that  changes  dynamically 
according  to  the  user§  choices.  Additionally,  virtual  reality  applications  offer  3D  interactive  environments  that 
can  simulate  real-life  events  and  have  great  potential  for  educational  purposes. 

On  the  pedagogical  side,  new  learning  methods  arise  from  the  use  of  multimedia  and  telematics 
applications.  These  methods  focus  on  the  learner  and  allow  an  active,  creative  construction  of  his/her 
knowledge.  New  pedagogical  models  exploiting  the  potential  of  these  technologies  are  emerging  and  new 
information  or  knowledge  sources  are  now  at  the  leamerS  fingertips.  Informal  access  to  knowledge  through 
museums,  science  centres,  libraries  or  browsing  on  the  Internet  is  playing  an  increasing  role  in  the  learning 
process,  but  its  impact  is  far  from  being  understood.  On  the  content  side,  the  subject  matters  as  the  learning 
goals  themselves  are  rapidly  changing.  On  the  software  side,  software  agent  technology  enhances  learning  by 
providing  intelligent  assistance  to  learners  in  situations  where  direct  manipulation  interfaces  are  insufficient. 
Virtual  reality  techniques  reduce  the  cognitive  load  in  the  use  of  some  applications  and  thereby  improve 
learning  efficiency. 

Today,  most  countries  consider  multimedia  and  telematics  technologies  as  an  educational  tool  and  are 
trying  to  incorporate  them  actively  in  their  curricula.  The  main  concept  is  that  the  use  of  these  technologies  can 
significantly  contribute  to  the  educational  process,  supporting  in  parallel  the  role  of  the  teacher. 

Furthermore,  many  pedagogues  have  stated  that  multimedia  and  telematics  applications  could  improve 
the  quality  of  the  offered  education,  because  they  can  simulate  many  basic  models  of  the  learning  process  and 
the  natural  proceeding  in  which  man  assimilates  knowledge.  In  particular,  telematics  technologies  can  simulate 
the  learning  by  exploring  and  the  incidental  learning,  while  they  could  be  used  for  the  immediate  access  of 
learning  materials  and  the  collaboration  among  students. 
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Distributed  Virtual  Environments 


A simple  definition  of  a Virtual  Environment  (VE)  is  a computer-generated  simulation,  and  the  term 
usually  implies  the  use  of  3D  computer  graphics  in  the  interface.  VEs  can  in  addition  be  multi-user,  supporting 
multiple  interacting  users,  and  distributed,  running  on  several  computers  connected  by  a network.  It  has 
become  common  to  refer  to  a VE  with  both  these  additional  properties  as  a DVE  (Distributed  Virtual 
Environment).  A Distributed  Virtual  Environment  (DVE)  allows  a group  of  geographically  separated  users  to 
interact  in  real  time.  The  environment  in  which  a DVE  user  is  immersed  is  three-dimensional  to  the  eye  and 
ear.  Moving  in  the  environment  changes  the  user§  visual  and  auditory  perspective.  Unlike  a video  conferencing 
system  (where  an  attendee^  screen  shows  other  attendees  in  their  own  video-conferencing  rooms),  DVE  users 
assemble  in  a virtual  world  - they  are  all  seen,  for  example,  seated  together  around  a conference  table  in  one 
room,  or  walking  together  in  a virtual  building.  Every  user  of  a DVE  appears  in  the  computer  environment  as 
an  avatar  - either  a customised  graphical  representation,  a video  of  the  user,  or  some  combination  of  both  -, 
which  he  or  she  controls.  The  user,  besides  interacting  with  one  another,  also  deals  with  one  or  more  computer 
simulations.  While  in  use,  a DVE  can  change  continually,  in  every  aspect.  Further,  a DVE  can  grow 
dynamically  by  accepting  contributions  of  objects  and  structures  from  many  sources.  On  one  hand,  interactivity 
is  a basic  requirement  of  all  virtual  environments.  On  the  other  hand,  distributed  virtual  worlds  require  a 
certain  level  of  persistence  to  achieve  the  impression  of  a single  shared  world.  Persistence  is  realised  by 
distributing  and  synchronising  user  input  as  well  as  user  independent  behaviour. 


The  role  of  DVEs  in  education 

Implementations  of  DVEs,  along  with  the  Internet  in  general,  offer  a number  of  attractive  features  for 
the  education  sector: 

• They  may  have  cross-platform  compatibility. 

• Software  for  creating  DVEs  content  can  be  downloaded  for  free. 

• As  some  DVEs  sit  upon  the  World  Wide  Web,  existing  student  knowledge  is  applicable,  easing  use. 

• Pupils  are  used  to  DVEs  from  popular  games,  and  expect  immersion  from  innovative  computer 
applications. 

DVEs  in  education  can  be  used  in  various  ways  but  the  most  promising  ones  are  either  as  mediums  for 
education  and  research  or  as  collaborative  mediums. 


As  a medium  for  education  & research 

DVEs  are  being  proposed  as  a means  to  facilitate  both  teaching  and  research  in  the  educational  sector. 
Indeed  this  is  the  most  obvious  role  for  DVEs.  Three-dimensional  content  can  be  created  and  distributed  to 
one§  own  students,  or  made  more  widely  available  to  learners  across  the  globe.  There  are  plenty  of  obvious 
applications  for  educational  simulators  in  a number  of  fields:  physics,  planetary  exploration,  archaeology, 
biology,  and  chemistry  all  can  benefit  greatly  from  better  visualisation  technologies.  The  point  of  these  systems 
is  not  necessarily  the  amount  of  things  that  can  be  done  with  them  but  the  fact  that  students  can  have  the  power 
to  create  them  on  their  own  and  can  find  it  fun  and  motivating  to  do  so.  Any  textbook  or  course  materials  that 
have  3-D  graphics,  such  as  architecture  and  engineering,  may  benefit  from  DVEs.  If  the  course  materials,  or 
the  research  papers,  require  multiple  drawing  of  a 3D  object  or  space,  then  clearly  DVEs  will  give  benefits. 


As  a collaborative  medium 

One  of  the  longer-term  opportunities,  but  possibly  the  most  exciting,  lies  in  the  educational  use  of  the 
shared  spaces  for  distance  teaching  and  project  collaboration.  The  communicative  worlds,  similar  to  those 
described  above,  would  allow  for  students  and  stuff  to  engage  in  distributed  meetings,  seminars  and  tutorials.  A 
number  of  trials  using  these  existing  technologies  are  being  tried. 

Collaborative  virtual  environments  provide  gathering  grounds  for  new  communities  and  types  of 
interactions,  and  they  give  people  a voice  like  they  have  never  had  before.  We  can  share  experiences  and 
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visions  and  leam  to  understand  the  other  person^  point  of  view.  We  can  attend  concerts,  act  in  plays,  and  attend 
classes  with  an  international  audience.  As  long  as  people  have  something  to  say  to  somebody  else,  they  can  say 
it  online. 


Virtual  European  School 

VES  is  a European  project  in  the  Educational  Multimedia  Sector,  aiming  to  build  up  a comprehensive 
resource  on  teaching  and  learning  material  for  secondary  schools  (pupils  10-19).  The  offer  of  the  final  VES 
system  (ready  in  February  2000)  will  contain  multimedia  material,  CBT  products,  and  also  additional 
background  materials,  such  as  passages  from  schoolbooks,  or  Internet  resources.  A major  strength  of  VES  is 
that  a large  consortium  of  educational  publishers  supports  it.  Over  20  publishers  from  4 countries  participate  in 
VES  providing  content  and  designing  the  system  in  such  a way,  that  VES  fits  into  the  working  process  of 
publishing  houses.  Therefore  a smooth  work-flow  for  delivery  of  material  is  guaranteed,  and  VES  has  the 
potential  to  become  in  short  time  one  of  the  primary  providers  of  on-line  material  in  the  major  participating 
countries,  that  is  Austria,  Italy  and  Greece.  However,  in  order  to  become  a successful  system,  VES  not  only 
needs  to  fit  the  needs  of  publishers,  but  also  the  needs  of  its  end-users,  that  is  pupils  and  teachers  in  their  role 
as  multipliers.  In  order  to  integrate  schools  in  the  design  process,  VES  has  established  a continuous  feedback 
process  in  each  of  the  three  VES  countries.  In  each  country  a regional  school-network  is  collaborating  with 
VES.  Ideas  are  brought  to  the  schools  for  evaluation  and  feedback  is  integrated  into  the  design  process.  VES 
will  be  turned  into  a commercial  system  in  the  beginning  of  2000.  Billing  and  Copyright  issues  need  to  be 
implemented  in  a sustainable  way.  These  topics  will  be  essential  for  the  success  of  the  exploitation  strategy  of 
VES. 


Deployment  scenarios  of  DVEs  within  VES 

Pedagogical  issues 

Using  innovative  technologies,  VES  aims  to  support  new  ways  of  learning.  Providing  students  an  access 
to  the  whole  base  of  learning  material  gives  them  the  opportunity  to  become  more  autonomous  in  their  learning 
process.  Students  will  no  longer  rely  on  only  one  source  of  information,  but  will  be  able  to  compare  different 
sources  and  find  additional  information  on  their  subject. 

A typical  example  could  be  a DVE  discussion  platform  on  the  various  parliamentary  systems  in  Europe. 
The  DVE  environment  could  provide  the  data,  but  the  pupils  can  discuss  on  which  voting  or  representation 
rules  seem  the  most  effective  and  democratic  ones.  For  younger  pupils  a DVE  on  different  climates  throughout 
Europe  would  be  more  appropriate.  A teacher  can  be  present  in  such  an  environment,  monitoring  the  cross- 
national discussions,  and  intervening  only  on  specific  open  questions. 

The  key  feature  in  these  applications  is  immersion.  DVEs  can  give  a three-dimensional  realistic  view  of 
an  environment  or  situation.  Therefore  navigating  within  a DVE,  learning  is  an  active,  explorative  process. 
Views  of  the  environment  are  individual,  since  persons  are  located  in  different  positions  in  the  environment 


Scenarios  of  usage 

Using  DVEs  in  education  encounters  several  obstacles.  First  and  most  important  the  limited  bandwidth 
with  which  most  schools  are  connected  to  the  information  infrastructure.  However  also  more  soft  facts,  as  the 
missing  of  VR  content,  and  the  very  limited  experience  of  teachers  with  this  sort  of  technology,  need  to  be 
taken  in  account,  when  one  designs  a learning  platform  based  on  DVEs. 

Within  VES  this  problem  was  tackled,  by  defining  several  strategies,  how  to  introduce  DVEs  to  the 
target  groups.  More  specifically  there  are  three  main  scenarios  envisaged: 

• DVEs  as  Web  site  entry  and  navigation  tool:  A DVE  used  as  main  VES  user  interface,  which  leads, 
through  objects  allocated  into  that  world,  to  other  VES  functionalities,  such  as  search,  compose  or  retrieve 
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content.  These  functionalities  in  a first  development  step  can  be  traditional  functions  (chat,  newsgroup, 
search,  etc.). 

• DVEs  as  learning  units:  VES  will  contain  many  individual  learning  units  from  different  content  providers. 
Some  of  those  learning  units  could  be  DVEs,  as  fixed  virtual  worlds  where  objects  in  those  worlds  could  be 
linked  to  specific  content  objects  (such  as  texts,  videos,  and  simulations).  Further  of  course  in  such 
environments  VR  objects  are  envisaged.  Participants  can  interact  with  objects,  and  gain  access  to  the 
relevant  learning  units. 

• DVEs  as  general  content  user  interface:  As  a third  development  step,  it  is  planned  to  provide  a DVE  tool 
set  to  teachers,  containing  several  model  worlds  and  a database  of  3D  objects  to  be  allocated  into  these 
worlds  by  the  teachers. 

In  sequel  we  present  these  scenarios  in  more  detail. 


DVEs  as  Web  site  entry  and  navigation  tool 

The  Virtual  Learning  Centre  represents  an  attempt  to  build  a learning  environment  combining  real  life 
situations,  educational  principles,  virtual  reality  features  and  WWW  technology  features.  The  development 
scenario  foresees,  that  in  a first  stage  the  Centre,  will  be  merely  an  entry  point  to  traditional  functionalities, 
such  as  search,  browse  content  etc.  in  a two  dimensional  interface. 

However  the  aim  is  to  evolve  this  entry  point  to  a fully  functional  Virtual  Learning  Centre.  Within  this 
Centre  the  participant  will  have  access  through  familiar  objects  to  typical  functions.  One  example  abstraction 
that  we  use  for  representing  the  stored  material  of  the  VES  system  is  a building  of  an  exhibition  centre,  where 
publishers  have  their  own  place  to  display  their  educational  products.  The  centre  is  organised  in  halls,  or 
thematic  units.  Each  hall  is  organised  around  a learning  subject,  like  natural  sciences,  art  and  culture, 
technology,  foreign  languages  and  others.  Furthermore,  each  hall  that  represents  a specific  subject  is  divided  in 
rooms,  where  each  room  represents  a different  learning  age. 

Every  room  of  the  centre  offers  different  material,  but  the  structure  of  the  room  is  the  same  for  all.  In 
every  room,  there  are  kiosks  that  represent  the  publishers  that  put  material  in  the  VES  system.  Every  publisher 
has  its  own  kiosk  in  every  room,  where  it  displays  the  material  that  belongs  to  him.  In  the  kiosks  there  are 
entities  that  represent  the  content  material  of  that  publisher.  For  example,  a user  can  go  to  a kiosk,  select  a 
book,  open  it,  see  the  metadata  information  about  it  that  is  stored  in  the  database  of  the  VES  system  and  if  he  is 
interested  in  this  item  then  he  can  view  or  download  the  relative  content  unit.  There  is  physical  representation 
of  the  most  recently  added  items  in  the  database,  while  the  older  items  are  inserted  in  a list,  which  the  user  can 
browse  and  select  to  view  any  of  these.  In  this  way  there  is  no  need  to  have  physical  representation  for  every 
content  unit  that  is  stored. 

Having  this  hierarchical  structure,  the  DVE  environment  becomes  very  easy  for  the  user  to  navigate  in  it 
and  find  the  material  that  he  wants.  In  addition,  the  user  can  get  used  to  the  virtual  environment  after  some 
visits  and  not  get  lost  in  it.  Every  room  of  the  centre  is  a separate  small  file  of  a virtual  world,  reducing  thus 
downloads  times,  and  the  user  can  go  from  one  room  to  another  through  appropriate  portals. 

The  Virtual  Learning  Centre  comprises  a number  of  other  spaces  characterised  by  function.  There  is  an 
information  centre  where  the  user  can  get  available  information  about  the  environment,  the  rooms  and  how  to 
get  there,  latest  news  and  others.  Furthermore,  there  exist  a conference  room  where  predefined  lectures, 
presentations,  seminars  and  other  events  take  place.  As  the  VES  system  expands  there  will  be  and  other  rooms 
with  specific  functionality. 


DVEs  as  learning  units 

DVEs  are  not  at  all  common  in  the  educational  sector,  and  especially  educational  publishers  have  hardly 
any  experience  in  this  field.  Building  a comprehensive  resource  on  learning  material,  based  on  VR  techniques, 
is  currently  not  feasible.  ' 

In  the  case  of  content  production,  it  is  possible  to  circumvent  this  problem  by  starting  to  develop  single 
DVE  content  units,  where  a clear  educational  added  value  can  be  seen,  and  where  content  providers  (in  the  case 
of  VES  publishers)  are  sustained  by  VR  specialists  in  the  creation  of  such  worlds. 
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In  the  VES  context  these  worlds  are  planned  to  be  non-populated,  but  rather  targeted  at  immersion 
properties  of  DVEs.  An  example  could  be  the  simulation  of  sewage  treatment  plant,  where  all  the  work  cycles 
are  shown,  and  the  reaction  of  the  system  is  simulated  when  different  mechanism  are  being  used.  Limiting  the 
DVEs  to  this  scope  not  only  facilitates  the  production  and  technical  issues,  but  also  gives  a controllable  way  for 
teachers  not  used  to  this  technology  to  adopt  virtual  environments  in  classroom  teaching. 

Within  VES  this  development  step  is  certainly  not  seen  as  an  end-point,  but  rather  as  a first  step  towards 
deployment  in  school  use.  Once  teachers,  as  multipliers  get  acquainted  with  this  novel  form  of  content,  more 
interactive  and  open  environments  will  be  inserted. 


DVEs  made  from  collections  of  building  blocks 

The  most  advanced  scenario,  in  terms  of  user  integration  is  the  one  of  DVEs  as  general  content  user 
interface.  In  this  case  it  is  envisaged  that  all  content  is  allocated  into  virtual  worlds.  The  working  scheme  would 
foresee  that  publishers  and  VR  experts  create  an  abundant  database  of  basic  DVEs  and  base  objects,  which  can 
populate  the  environments. 

An  example  of  such  a unit  could  be  a VR  timeline.  The  time-line  indicating  the  years  (or  centuries) 
would  present  the  basic  DVE,  while  the  base  objects  would  be  for  example  figures  (e.g.  an  image  of  Napoleon), 
scenes  and  movies  (a  short  video  on  life-style  during  French  revolution  period),  but  also  more  basic  objects  as 
dictionaries,  notice  boards,  etc.  Teachers  would  then  take  the  empty  world  and  populate  it  with  the  objects 
suiting  their  needs,  associating  to  the  base  objects  functions  defined  by  them. 

A user  analysis  performed  within  the  project  VES  has  shown  that  this  deployment  of  DVEs  is  currently 
not  sustainable  in  most  educational  settings.  Therefore  it  is  necessary  to  start  out  with  the  less  advanced 
scenarios,  in  order  to  acquaint  teachers  to  the  new  technology. 


Design  Issues 

The  design  of  a Virtual  Learning  Environment  can  be  based  on  the  idea  of  an  exhibition  centre,  where 
the  educational  material  is  organised  hierarchically  according  to  a universal  or  subject-specific  classification 
scheme  for  educational  content.  The  exhibition  centre  is  organised  in  thematic  hallsj’  each  of  them  organised 
around  a learning  subject,  such  as  mathematics,  science,  geography,  art  and  culture,  foreign  languages,  etc. 
Each  thematic  half’representing  a specific  learning  subject  is  further  divided  into  rooms  representing  the 
subdivisions  of  each  subject  according  to  the  classification  scheme.  Moreover,  at  the  bottom  level  of  the 
hierarchy  there  exist  rooms  representing  educational  content  for  different  learning  ages. 

In  order  to  have  a user-friendly,  integrated,  and  interactive  user  interface,  the  3D  environment  of  the 
Virtual  Learning  Environment  can  be  combined  with  several  HTML  frames,  each  of  them  having  its  own 
functionality.  Thus,  one  frame  is  used  for  embedding  the  Virtual  Learning  Environment,  another  is  used  for 
displaying  status  messages  to  the  user  and  the  rest  display  a console  of  buttons  for  interaction  with  the  Virtual 
World.  For  example,  there  always  exists  a plan  view  representing  the  current  user§  position  at  the  Virtual 
Learning  Environment  and  facilitating  the  navigation  through  different  rooms.  Also,  the  user  can  switch 
viewpoints,  take  predefined  paths,  or  view  certain  simulations  of  specific  tasks  in  the  world.  Finally,  the  content 
units  that  the  user  selects  to  view  are  displayed  in  a separate  window  so  that  the  main  console  is  always  visible, 
avoiding  in  this  way  continuous  back  and  forward  actions. 


Implementation  Issues 

An  approach  is  to  use  VRML  for  the  implementation  of  the  Virtual  Learning  Environment.  VRML  is  an 
acronym  for  the  Virtual  Reality  Modelling  Language.  The  aim  of  VRML  is  to  bring  to  the  Internet  the 
advantages  of  3D  spaces,  known  as  VRML  worlds  whether  they  comprise  environments  or  single  objects.  The 
ISO/IEC  14772-1  international  standard  specifies  the  current  version  VRML  97.  VRML  offers  a number  of 
attractive  features  for  presenting  educational  material  including  ease  of  use  and  cross-platform  compatibility. 
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Another  approach  for  the  implementation  of  the  Virtual  Learning  Environment  is  to  use  SuperscapeS 
VRT,  which  is  an  application  for  creating  3D  environments.  The  file  format  of  VRT  worlds  is  SVR,  which  is  a 
compressed  binary  file  that  contains  all  the  geometry,  textures,  sounds  and  behaviours  needed  for  the 
interactive  3D  world,  in  a highly  compact  and  encoded  form,  giving  short  download  times  and  rapid  set-up 
times.  The  3D  browser  that  displays  VRT  worlds  is  called  Viscape  and  it  is  freely  available.  VRT  has  also  a 
purpose-built  behavioural  control  language,  called  Superscape  Control  Language  (SCL)  which  is  based  on  the 
C language. 

In  order  to  add  the  capability  of,  more  than  one,  users  accessing  the  same  Virtual  Environment  at  the 
same  time  and  to  create  a Distributed  Virtual  Environment  (DVE),  a communication  platform  is  needed  for 
performing  the  interaction  between  the  different  users  across  the  network.  A DVE  provides  the  users  with 
access  to  a shared  3D  environment.  All  participants  are  presented  with  the  illusion  of  being  located  in  the  same 
place,  such  as  in  the  same  room,  building,  or  terrain.  That  shared  environment  represents  a common  location 
within  which  other  interactions  can  take  place.  The  shared  environment  presents  the  same  characteristics  to  all 
the  participants.  For  example,  all  the  participants  could  get  the  same  sense  of  the  temperature  and  weather,  as 
well  as  the  acoustics.  Additionally,  DVEs  provides  some  way  of  communication  between  the  participants  by 
means  of  typed  text,  voice,  etc.  A DVE  system  consists  of  four  basic  components  that  work  together  to  provide 
the  sense  of  immersion  among  users  at  different  sites.  These  are  a graphic  display,  communication  and  control 
devices,  a processing  system,  and  a data  network. 

The  main  commercial  platform  for  implementing  DVEs  from  a VRML  or  a SVR  world  is  the  Blaxxun 
Community  Server.  The  Blaxxun  Community  Server  is  a multi-user  server  that  supports  the  operation, 
administration  and  usage  tracking  of  virtual  worlds  that  many  users  can  inhabit  simultaneously.  It  is  an  open 
system  that  supports  all  the  relevant  standards:  HTML,  VRML,  Java,  vCard,  ActiveX,  OCX,  Direct3D  and 
OpenGL  to  enable  3D  multi-user  interaction  and  it  works  together  with  the  Blaxxun  Community  Clients. 


Conclusions 

The  construction  of  Distributed  Virtual  Environments  should  be  seen  as  a design  problem  in  the 
broadest  sense.  VR  projects  with  an  educational  remit  should  recall  at  all  times  the  criteria  of  educational 
effectiveness.  The  drawbacks  as  well  as  the  advantages  of  various  aspects  should  be  evaluated,  including  the 
effects  of  concretising  information,  of  appropriateness  of  spatial  metaphors  and  of  the  interface.  Regarding 
shared  worlds,  they  should  be  investigated  to  support  distance  tutoring  and  research,  but  with  caution  to  ensure 
that  the  social  grouping  effect  of  actual  meeting  is  not  lost. 

References 

Davis  S.,  Huxor  A.,  Lansdown  J.  (1996)  The  DESIGN  of  Virtual  Environments  with  particular  reference  to 
VRML,  Centre  for  Electronic  arts,  Middlesex  University 

Hand  C.  (1996)  Some  User  Interface  Issues  for  Hypermedia  Virtual  Environments,  Position  Paper  for  the 
Workshop  on  Virtual  Environments  and  the  WWW,  Fifth  International  World-Wide-Web  Conference 

Broil  W.  (1996)  Extending  VRML  to  Support  Collaborative  Virtual  Environments,  German  National  Research 
Center  For  Information  Technology 

Waters  R.,  Anderson  D.,  Barrus  J.  (1996)  Diamond  Park  and  Spline:  A Social  Virtual  Reality  System  with  3D 
Animation,  Spoken  Interaction  and  Runtime  Modifiability,  Mitsubishi  Electric  Research  Laboratory 

D.  Brutzman,  M.  Macedonia,  M.  Zyda  (1995).  Internetwork  Infrastructure  Requirements  for  Virtual 
Environments,  Computer  Science  Department,  Naval  Postgraduate  School. 

W.  Winn,  (1997).  The  Impact  of  Three-Dimensional  Immersive  Virtual  Environments  on  Modern  Pedagogy, 
HIT-LAB,  Seattle  R-97-15. 

VES  - The  Virtual  European  School  (1998).  ANNEX  I - Project  Programme. 


358 


Designing  Collaborative  Distance  Learning  Environments  for  Complex 

Domains 


J.  Michael  Spector 

Educational  Information  Science  & Technology,  University  of  Bergen,  Norway 
Michael.Spector@ifi.uib.no 

Barabara  Wasson 

Pedagogical  Information  Science,  University  of  Bergen,  Norway 
Barabara.Wasson@ifi.uib.no 

Pal  1.  Davidsen 

System  Dynamics  Program,  University  of  Bergen,  Norway 
Pal.Davidsen@ifi  .uib.no 


Abstract:  Our  aim  is  to  describe  and  illustrate  a theoretically  based  approach  to  the  design  of 
collaborative  distance  learning  environments  for  complex  domains.  There  is  an  obvious 
growth  in  the  use  of  distributed,  online  learning  environments.  There  is  evidence  to  suggest 
that  collaborative  learning  environments  can  be  effective,  especially  when  using  advanced 
technology  to  support  learning  in  and  about  complex  domains.  There  is  also  an  extensive 
body  of  research  literature  in  the  areas  of  situated  cognition  and  problem-based  learning  that 
provides  a theoretical  perspective  for  the  design  of  such  learning  environments.  What  is 
lacking  is  a clear  articulation  of  design  principles  to  guide  the  implementation  of  collaborative 
distance  learning  environments.  We  provide  a description  of  such  design  principles,  explicitly 
drawing  on  a socially-situated  view  of  problem-based  learning  in  technology-mediated 
environments.  We  conclude  with  a description  of  how  this  framework  can  be  applied  to  the 
design  of  system  dynamics  based  learning  environments. 


Introduction 

Technology-based  learning  environments  are  growing  in  number  and  prevalence  in  spite  of  continuing 
debates  in  the  academic  literature  with  regard  to  their  learning-effectiveness,  impacts  on  organizations,  and 
overall  utility  to  society  (Carter  1997,  Clark  1994,  Kozma  1994,  Lengel  1997).  There  are  many  investigations 
into  how  people  might  learn  using  new  technologies,  and  these  studies  are,  in  turn,  causing  much  discussion  with 
regard  to  foundational  issues  in  learning  theory  and  instructional  design.  One  challenge  is  to  make  effective  use 
of  new  technologies  to  support  learning  and  instruction.  It  is  our  hope  to  provide  a design  framework  for 
technology-mediated  learning  environments.  This  framework  has  implications  for  designing  learning 
environments  for  complex  systems.  We  proceed  by  briefly  identifying  and  reviewing  relevant  theoretical 
perspectives.  We  then  establish  a unifying  perspective  with  clear  design  implications  in  the  context  of  these 
theories.  This  unifying  perspective  will  then  be  used  to  describe  our  framework.  We  conclude  with  an  illustration 
of  the  framework. 


Theoretical  Perspective 

We  begin  by  identifying  relevant  assumptions:  (1)  learning  is  a natural,  human  activity;  (2)  the  unit  of 
analysis  for  learning  effectiveness  is  an  individual  human  learner;  and,  (3)  learners  are  rational.  The  prevalent 
research  perspective  is  that  instructional  design  is  primarily  a prescriptive  enterprise  forming  a bridge  between 
descriptive  learning  research  and  practical  development  of  learning  environments  (Reigeluth  1983). 

There  is  practical  value  in  adopting  this  research  perspective.  By  varying  the  instructional  methods  used 
in  certain  conditions,  one  can  measure  outcomes  and  study  the  effects  of  those  methods  on  learning  outcomes.  If 
enough  data  is  collected,  one  then  hopes  to  be  able  to  establish  a strong  argument  for  the  desirability  of  a 
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particular  method  given  certain  learning  conditions,  thus  prescribing  how  one  ought  to  design  instruction  to 
achieve  desired  outcomes.  In  this  perspective,  it  is  possible  to  identify  how  various  types  of  learners  engage  in 
this  process  and  then  to  take  those  differences  into  account  in  the  design  of  instruction.  Learners  are  rational  in 
the  sense  that  they  are  goal-driven,  purposeful  agents  with  the  ability  to  identify  and  select  reasonably  efficient 
means  to  achieve  goals.  Typically,  conditions  are  held  constant  and  various  interventions  (instructional  methods 
instantiated  in  particular  learning  environments)  are  investigated.  Learning  outcomes  or  effects  are  then 
measured  on  individuals. 

Such  a research  paradigm  has  in  fact  produced  many  useful  findings,  so  it  should  not  be  discounted.  We 
refer  to  this  as  the  atomistic  perspective  because  it  is  characterized  by  an  atomistic  view  of  learning,  both  in 
terms  of  units  of  learning  (very  specific  and  discrete  conditions,  methods,  and  outcomes)  and  in  terms  of  learners 
(typically  focusing  on  individual  learners).  The  atomistic  perspective  can  be  contrasted  with  what  we  call  the 
integrated  perspective  (Spector  1995).  The  integrated  perspective  views  learning  as  holistic  and  learners  as 
members  of  a society  or  language  community.  The  goals  of  a society  or  language  community  typically  include  a 
strong  survival  element,  although  this  is  quite  often  not  made  explicit.  Living  consists  of  working  and  learning, 
which  are  viewed  as  essentially  collaborative  efforts  to  achieve  commonly  held  goals.  From  this  perspective, 
individuals  might  manage  to  acquire  extremely  high  levels  of  performance  at  particular  tasks  while  the  larger 
social  group  consistently  falters.  This  would  not  count  as  effective  learning  from  an  integrated  perspective. 


Complex  Domains 

Complex  systems  can  be  depicted  as  a collection  of  inter-related  items  (e.g.,  stocks  and  flows  in  system 
dynamics),  characterized  by  internal  feedback  mechanisms,  nonlinearities,  delays,  and  uncertainties  (Sterman 
1994).  These  systems  typically  exhibit  dynamic  behavior,  especially  in  the  sense  that  how  they  behave  has  an 
effect  on  the  structure  of  the  system,  perhaps  strengthening  or  changing  the  feedback  mechanisms.  This  change 
in  internal  structure  in  turn  has  consequences  for  how  the  system  will  behave  in  the  future  (Davidsen  1994, 
1996).  Such  complexity  is  difficult  to  understand,  especially  for  newcomers  to  a complex  domain. 

Complex  systems  can  be  found  in  abundance  at  many  different  levels.  Economics,  ecology, 
epidemiology,  project  management,  and  training  all  typically  involve  complex,  dynamic  systems.  People  have 
difficulty  in  understanding  and  making  good  decisions  about  such  systems  (Domer  1996).  There  are  clearly 
individual  exceptions,  persons  who  somehow  acquire  a deep  understanding  of  such  systems.  Typically,  such 
deep  understanding,  characterized  by  effective  decision  making  across  a wide  variety  of  changing  conditions, 
takes  years  to  acquire,  and  appears  not  to  be  easily  acquired  in  spite  of  concentrated  education  and  training 
efforts  (Dreyfus  & Dreyfus  1986).  Why  have  we  failed  to  improve  our  thinking  skills  in  complex  domains  in 
spite  of  such  persistent  and  serious  efforts?  In  part,  we  have  not  fully  understood  relevant  psychological  and 
sociological  factors.  In  part,  we  have  not  fully  integrated  relevant  principles  about  human  learning  into  design 
praxis. 

People  have  difficulty  in  estimating  the  effects  of  accumulation  over  time,  in  predicting  the  effects  of 
delays,  and  in  calculating  nonlinear  outcomes  (Sterman  1994).  People  tend  to  focus  on  local  problems  as 
opposed  to  whole  systems,  even  when  told  that  a holistic  understanding  is  essential  for  solving  particular 
problems.  People  become  cynical  and  overlook  possible  solutions  when  initial  attempts  fail.  People  do  not 
communicate  effectively  in  crisis  situations.  People  have  difficulty  inferring  underlying  system  structures  from 
externally  viewed  system  behaviors  (Domer  1996). 

We  believe  that  instructional  scientists  have  not  fully  understood  the  socially-situated  learning 
perspective  and  its  implications  for  human  learning  in  and  about  complex  systems.  There  is  a great  deal  of 
discussion  about  situated,  problem-based,  and  collaborative  learning,  but  we  are  missing  critical  pieces  of  a 
design  framework.  Put  differently,  we  believe  that  we  lack  a well-articulated  design  framework  with  sufficient 
detail  to  take  us  from  a socially-situated,  problem-based,  collaborative  learning  perspective  to  the  design  of  a 
particular  learning  environment  for  a particular  subject  domain.  The  closest  such  approach  we  find  is  cognitive 
apprenticeship  (Collins  1991).  We  regard  our  work  as  an  extension  to  that  approach. 

Situated  Learning  and  Vygotsky’s  Cultural-historical  Theory 

The  theoretical  foundations  for  this  effort  come  primarily  from  a socially-situated  learning  perspective 
(Lave  1988,  Piaget  1970,  Vygotsky  1978).  Within  this  perspective,  learning  is  viewed  as  an  active  process  of 
knowledge  construction  in  which  learners  are  typically  involved  with  other  learners  in  authentic,  problem- 
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solving  situations.  The  need  to  leam  created  by  a realistic  problem  provides  motivation,  and  interaction  with 
other  similarly  immersed  learners  provides  facilitation.  We  are  favorably  inclined  to  an  attitude  similar  to  Sfard’s 
(1998)  that  emphasizes  the  need  for  both  an  acquisition  metaphor  (static  knowledge  objects  with  learners 
acquiring  expertise)  and  a participation  metaphor  (dynamic  knowledge  objects  with  learners  as  active 
apprentices).  Much  higher  order  learning  relies  on  knowledge  and  associated  learning  activities  that  might  best 
be  supported  within  the  acquisition  view.  However,  to  progress  beyond  competent  performance  and  become  a 
proficient  expert  (Dreyfus  & Dreyfus  1986),  we  believe  that  the  participation  metaphor  with  its  emphasis  on 
active  learner  participation  in  socially-situated  and  problem-oriented  settings  is  crucial. 

For  Vygotsky  (1978),  human  mental  functions  appear  first  as  inter-individual  and  later  as  intra- 
individual. This  process  involves  the  use  of  socially  developed  tools.  For  Vygotsky,  the  unit  of  analysis  for 
human  activity  and  for  human  learning  was  the  mediated  action  of  an  individual.  This  broadens  the  unit  of 
analysis  identified  earlier  from  just  the  individual  to  include  an  artifact  with  which  an  individual  interacts. 
Davydov  (1988)  developed  a psychological  theory  of  learning  activity  which  focused  on  goal-oriented, 
collaborations  within  a social  context.  From  this  perspective,  the  aim  of  a learning  activity  is  to  teach  study  skills 
that  enable  learners  to  think  on  their  own. 


Collaborative  Learning 

Collaborative  learning  is  based  on  the  notions  of  socially  shared  cognition  (Resnick,  Levine  & Teasley 
1991),  of  distributed  cognition  (Salomon  1993),  and  of  jointly  accomplished  performance  (Pea  1993).  In  the 
latter,  cognitive  development  is  viewed  as  occurring  though  interactions  between  students,  as  well  as  between 
students  and  knowledgeable  environments.  Collaborative  teleleaming  emphasizes  collaborative  interactions 
among  students  and  between  students  and  supporting  actors  in  a distance  learning  environment  (Wasson  & 
Bourdeau  1998).  Introducing  collaboration  into  a teleleaming  situation  raises  new  challenges  related  to  the 
logistics  and  to  the  logistical  support  of  collaboration.  The  majority  of  research  into  collaborative  learning  has 
focused  on  collaboration  between  physically  present  actors  and  has  generally  focused  on  whether  and  under  what 
circumstances  collaborative  learning  was  more  effective  than  individual  learning.  More  recent  efforts,  however, 
have  been  directed  towards  a more  process-oriented  account  of  collaboration  where  the  focus  is  on  the  role  that 
variables  such  as  group  size,  group  composition,  and  communication  paradigm  play  in  mediating  interactions. 
This  is  a clear  indication  that  the  unit  of  analysis  has  been  appropriately  enlarged  beyond  the  individual  learner. 

We  have  now  collected  several  design  principles  which  form  key  features  of  a design  framework,  and  we 
have  shown  their  linkage  to  learning  theory.  These  principles  might  be  summarized  as  follows: 

1.  Provide  support  for  the  joint  construction  of  knowledge  objects  and  for  the  joint  constmction  of 
problem  solutions.  This  can  be  accomplished  by  providing  learner-extensible  databases,  by  providing 
learner-modifiable  simulation  models,  by  supporting  learner-learner  and  learner-tutor  collaboration  in 
making  changes  and  extensions  to  existing  knowledge  objects,  and  by  supporting  learner-learner  and 
learner-tutor  reflection  on  outcomes  of  those  modifications. 

2.  Provide  tools  to  support  joint  negotiation  of  alternatives.  Commenting  tools  are  a beginning.  Allowing 
changes  to  be  tested  when  appropriate  is  also  important.  It  is  especially  important  to  allow  learners  to 
try  out  alternative  simulation  models,  to  observe  results,  and  to  support  learner-learner  and  learner-tutor 
discussion  and  analysis  of  those  results. 

3.  Provide  both  public  and  private  feedback  support  mechanisms  for  learners  while  emphasizing  to  those 
intelligent  agents  providing  feedback  that  it  is  important  for  learners  to  gradually  improve  their  own 
ability  to  monitor  and  assess  progress  toward  desired  goals. 

In  short,  this  framework  aims  to  support  both  individual  work  and  collaborative  activities,  and  it  especially 
aims  to  support  the  development  of  collaborative  learning  communities. 


Computer  Supported  Collaborative  Learning  (CSCL)  & Telelearning 

The  important  implications  for  CSCL  that  emerge  from  these  theoretical  foundations  include  a wide 
range  of  designs  for  computer-based  learning  environments,  as  well  as  the  view  that  the  computer  is  a mediating 
tool  that  needs  to  be  seen  in  the  context  of  the  entire  learning  environment  within  which  it  will  be  used.  That 
context  includes  the  instructional  setting,  the  presence  or  absence  of  a teacher,  the  role  of  teachers  and  tutors,  the 
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role  of  the  learner  and  other  learners,  the  curriculum,  the  organizational  setting,  etc.).  Much  contemporary 
research  that  falls  under  the  umbrella  term  socio-cultural  theory  (e.g.,  Collins  1991)  has  been  inspired  by 
Vygotsky  and  his  followers.  CSCL  requires  thoughtful  and  careful  design  of  learning  scenarios  and  learning 
support.  CSCL  also  requires  technological  design  encompassing  the  teleleaming  environment  configuration  as 
well  as  the  tools  and  services  available. 

The  term  teleleaming  is  used  to  designate  new  forms  of  distance  or  of  computer  mediated  learning, 
where  the  distance  is  not  only  distance  in  space  or  time,  as  in  traditional  distance  learning,  but  distance  in  terms 
of  culture  and  community,  as  well  as  distance  in  the  mediation  of  learning  activities  and  knowledge  objects 
(Wasson  & Bourdeau  1 998).  Distance  learning  evolved  from  an  initial  need  to  ensure  equal  access  to  education 
for  all  students.  The  most  obvious  feature  of  a distance  learning  setting  is  that  students  and  teachers  do  not  all 
meet  at  the  same  place  or  at  the  same  time.  Individual  learning,  individual  tutoring  and  asynchronous 
communication  are  typical  features  of  a distance  learning  situation,  requiring  extensive  macro-  and  micro- 
instructional  design,  and  a strong  student  support  system.  Variations  of  teleleaming  presence  need  to  be  taken 
into  consideration,  including  the  sense  of  telepresence  in  a virtual  meeting,  the  sense  of  telepresence  in 
interactions  with  rich  multimedia  environments,  and  the  sense  of  telepresence  in  extensive  human  collaborations 
with  online  knowledge  objects  and  virtual  worlds  (e.g.,  online  micro  worlds). 


A Design  Framework  for  Collaborative  Learning  Environments 

A relevant  paradigm  for  the  design  of  these  learning  environments  can  be  found  in  the  computer 
supported  collaborative  learning  (CSCL)  literature  (Koschmann  1 996).  This  paradigm  focuses  on  the  use  of 
information  and  communications  technology  as  a mediating  tool  within  a collaborative  framework  (e.g.,  peer 
learning  and  tutoring,  reciprocal  teaching,  project-  or  problem-based  learning,  simulations,  games)  of  learning. 
This  particular  approach  emphasizes  an  understanding  of  language,  culture  and  other  aspects  of  the  social  setting 
and  can  be  traced  in  part  to  a socially-situated  perspective  of  learning,  as  we  have  already  indicated.  While  some 
suggest  that  these  theories  are  not  compatible,  we  suggest  that  they  in  fact  provide  an  excellent  theoretical 
framework  for  the  design  of  online  collaborative  learning  environments,  especially  for  complex  domains.  The 
common  and  unifying  notion  of  situated  or  shared  cognition  emphasizes  the  larger  environment  within  which 
learning  takes  place;  learning  is  viewed  in  part  as  entering  into  a community  of  practice  with  a shared  language 
and  understanding.  The  problem  of  establishing  such  a community  of  practice  in  distributed  and  online  settings 
is  a critical  design  aspect  that  has  been  poorly  understood  and  not  especially  well  implemented  in  practice. 

The  design  methodology  that  we  advocate  is  based  on  cognitive  apprenticeship  (Collins  1991), 
providing  more  learner  support  and  facilitation  for  less  experienced  learners  and  gradually  fading  support  and 
facilitation  for  more  experienced  learners.  The  learning  setting  is  teleleaming,  as  previously  described.  The 
overall  learning  approach  is  collaborative  in  nature,  consistent  with  the  theoretical  foundations  cited  earlier.  The 
specific  view  of  collaboration  is  that  tasks  and  activities  should  be  realistically  arranged  and  mediated  in  ways 
that  naturally  involve  and  recognize  particular  interests  and  skills  of  learners  (Salomon  1993).  That  is  to  say  that 
the  collaborations  should  not  be  artificially  enforced  and  that  the  sense  of  collaboration  from  the  learner's 
perspective  should  be  such  that  collaboration  is  viewed  as  necessary  in  order  to  complete  the  desired  goals.  In 
the  marine  bio-diversity  example,  students  work  in  small  groups  of  two  and  three  and  rely  heavily  on  the 
assistance  of  graduate  assistants  and  teachers  to  check  their  work.  Nevertheless,  they  understand  the  significance 
of  the  activities  in  which  they  are  engaged  and  recognize  their  contributions  as  valued  and  significant. 

We  conclude  with  an  example  to  illustrate  this  theoretical  framework.  As  we  have  already  said,  we  do 
not  believe  that  any  large-scale,  commercial  learning  environment  has  been  designed  with  a systemic  view  of 
learning  according  to  a theoretical  perspective  such  as  that  presented  here.  Figure  1 is  drawn  from  a learning 
environment  designed  according  to  this  framework.  Several  such  learning  environments  have  been  constructed 
and  some  data  pertaining  to  learning  effectiveness  has  been  accumulated.  That  data  suggest  that  this  approach 
does  promote  learning  and  has  been  reported  elsewhere  (Spector  1 998). 


Exemplifying  the  Framework 

We  have  thus  far  identified  the  following  elements  of  an  online  learning  environment  for  complex  domains: 
• Provide  support  for  collaborative  construction  of  components; 
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• Provide  tools  to  support  negotiation  of  alternatives; 

• Provide  both  public  and  private  feedback  support  mechanisms; 

• Provide  mechanisms  to  share  and  exchange  information,  objects,  views,  etc.; 

• Facilitate  a meaningful  division  of  labor; 

• Support  joint,  online  thinking,  commentary,  etc.; 

• Include  meaningful  learning  scenarios; 

• Design  authentic  problems  and  legitimate  cases  as  the  basis  for  learning  activities; 

• Take  into  account  the  entire  learning  environment; 

• Support  mediation  among  all  the  participants;  and, 

• Foster  a sense  of  a collaborative  learning  community. 

Figure  1 shows  a screen  from  one  version  of  a learning  environment  for  resource  management  in  large-scale 
instructional  development  projects.  This  screen  shows  that  learners  have  multiple  views  of  a complex  domain. 
Consistent  with  our  framework,  this  environment  supports  both  learner- learner  and  learner- tutor  interactions,  as 
well  as  learner  manipulation  and  modification  of  the  underlying  simulation,  after  learners  have  passed  beyond 
the  level  of  advanced  beginner  in  this  domain. 


Concluding  Remarks 

It  is  premature  to  argue  that  this  framework  will  produce  significant,  long-lasting,  and  positive  learning 
effects  in  and  about  complex  systems.  Based  on  existing  data  collected  on  our  learning  environments  and  from 
data  reported  on  some  of  those  which  adopt  similar  approaches  (Sioutine,  Davidsen,  & Spector  1998),  we 
believe  that  there  is  great  promise  in  designing  collaborative  teleleaming  environments  from  a socially-situated 
learning  perspective  with  heavy  emphasis  on  collaborative  learner  participation  in  the  creation  and  modification 
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of  knowledge  objects  and  artifacts.  We  have  been  encouraged  to  learn  that  the  system  dynamics  community  is 
coming  around  to  the  view  that  it  is  not  generally  sufficient  to  provide  only  behaviorally  oriented  feedback  about 
complex  systems  if  one  wishes  to  promote  deep  understanding  of  such  systems.  Learners  should  be  encouraged 
to  engage  in  model  alteration,  model  construction,  and  policy  and  strategy  design  in  a collaborative  context  if 
one  expects  lessons  to  transfer  from  the  learning  environment  to  real-world  settings. 
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Abstract : Cyber  university  is  a new  field  that  can  overcome  the  limitation  of  present  education 
criteria.  Student  can  receive  learning  in  anywhere  and  anytime  using  information 
communication  infrastructure.  But,  many  researches  in  field  of  cyber  university  concentrate  on 
development  of  solutions  based  application  level.  These  solutions  lack  a flexibility  and 
adaptability  to  be  adapted  in  various  environment  of  educatioa 

In  our  research,  we  propose  a framework  that  is  not  application  level  in  solution  for  cyber 
university.  This  framework  is  designed  using  CORBA  that  is  standard  in  distributed  object 
middleware.  The  CORBA  based  framework  can  be  integrated  with  other  systems  easily. 

The  framework  is  mainly  composed  by  four  components.  Course  manager  maintains  all  learning 
materials  using  workflow  concepts.  Student  manager  maintains  profile  and  conditions  of 
learning  for  each  student.  Intelligent  agent  determines  leamii^  step  and  coarse  using  student*s 
condition  dynamically.  Session  manager  maintains  group  activity  between  lecture  and  students. 


1.  Introduction 

In  advance  of  computing  power  and  networking  technologies,  Many  applications  are  developed  by  many 
corporations  and  institutes.  Especially,  Cyber  university  is  a new  field  that  can  overcome  the  limitation  of  present 
education  criteria[Ellis  1998,  Dewy  1998,  Lee  1997,  Barajas  1998,  Mittrach  1998].  In  Cyber  University,  students  can 
access  training  and  performance  supports  via  the  Internet  when  and  where  they  need  itfWade  1998].  And,  it 
significantly  reduces  the  cost  and  time  requirements  associated  with  more  traditional  types  of  training.  Now,  many 
research  and  solutions  are  provided  for  constmction  of  cyber  universityfDewey  1998,  Lee  1997]. 

But,  the  stmcture  of  cyber  university  is  regarded  as  very  difficult  process  because  of  diversity  of  educational 
environments.  In  spite  of  diversity,  almost  of  solution  and  researches  are  concentrated  on  application  level. 
Application  level  based  solutions  lack  a flexibility  and  adaptability  to  be  adapted  in  various  environment  of 
educationfBarajas  1998,  Mittrach  1998]. 

We  proposed  the  framework- based  solution  for  cyber  umversity.  Because  of  framework- based  solution,  the 
solution  presents  flexibility  and  adaptability  to  constmction  of  cyber  university  in  diverse  environment. 

2.  Requirment  for  Framework 

Cyber  university  can  maintain  a lot  of  leamir^  contents  effectively,  and  present  contents  to  students  by  fit  methods. 
Also,  It  is  difficult  for  students  to  judge  own  state  of  learning  and  to  determine  subjects  for  achieve  own  goal. 
Because  cyber  university  is  less  sufficient  interaction  between  instmctor  and  student  than  real  university,  functions 
that  must  determine  student’s  coarse  and  monitor  leamir^  state  are  required.  System  can  diagnose  of  the  state  of 
student  and  satisfaction  of  content  presented  to  student,  and  determine  learning  step  and  coarse  using  student’s 
condition  dynamically.  Also,  because  they  would  like  to  act  on  participants  in  group,  group  activity  can  supported 
by  system[Manthel996,  Rezende  1995,  SolyoU  1996]. 

Figure  1 describes  the  essence  of  education  for  cyber  university.  Cyber  university  allows  students  to  access  the 
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topics  they  need,  when  they  need  them.  Students  may  choose  to  play  entire  chapters,  or  utilize  the  search  function 
to  answer  a specific  question.  System  determines  the  best  course  which  student  can  approve  to  own  goal  using  user 
requirement.  After  course  is  determined,  system  composes  user  s profile  and  prepares  learning  contents  for  student 
and  presents  it  them.  Then,  students  would  learn  learning  contents  by  determined  course.  Result  of  learning  is  used 
by  diagnose  process.  Diagnoses  result  is  feedback  to  profiling. 


Figure  1.  Essence  of  education  for  cyber  university 


• Student  Requirements  : It  is  goal  which  student  want  to  learn  through  cyber  university. 

• Course  Determine  : System  determine  student  s learning  course  using  instructor  s knovdedge. 

• Profiling  : It  maintains  student  profile  that  contains  private  information  with  requirement  which  student  want  to 
learn,  and  topic  information  which  students  access. 

• Content  management : Content  management  maintains  learning  contest  and  provides  it  to  student. 

• Education  : Students  can  learn  the  course  and  contents  decided  matching  functionality. 

• Observation  & Diagnosis  : The  process  of  learning  is  monitored  continuously  and  result  of  student *s  learning  and 
satisfaction  are  analyzed  and  returned  feedback  to  students*  profiling. 

3.  Framework  Design 

In  this  paper,  we  design  CORE  A based  framework  for  cyber  university.  The  framework  is  mainly  composed  by 
four  components  and  they  are  des^ned  using  CORE  A objects.  Therefore,  this  framework  is  integrated  with 
external  application  and  database.  And  it  is  used  in  heterogenius  environment. 

3.1  Overview 

[Fig.  2]  shows  overall  configuration  of  cyber  university  using  the  COREA  based  framework. 


Administration  Center 
figure  2.  CORBA  based  framework 
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BEST  COPY  AVAILABLE 


3.1  Session  Manager 

In  server  side,  session  manager  has  group  managir^  module  and  event  handler.  [Fig.  3]  shows  architecture  of 
session  manager. 


Figure  3:  Session  manager 


Group  managing  module  operates  as  gateway.  It  authenticates  using  information  in  student  profile  manager.  And  it 
manages  group  participated  by  students  and  lecturer.  Event  handler  processes  events  that  occur  in  each  group. 
Group  managing  module  creates  appropriate  group  for  user  requirement,  and  manages  those  usir^  group  pool. 
Once  event  occurs,  interaction  handler  catches  that.  If  it  cannot  be  handled,  interaction  handler  sends  it  to  event 
handler.  Event  handler  processed  that  using  group  managing  interfaces.  Events  handled  by  event  handler  require 
group- related  operations  such  as  create,  destroy,  join,  and  leave.  Table  1 describes  operation  of  Session  Manager. 


Table  1 Synchronous  learning/training  operations 


BasicOperations 

Create  Group 

Destroy  Group 

Group  ManagementOpe  rations 

Join  Group 

Leave  Group 

Manage  Applications 

Manage  Time  Event 

Control  Token 

Manage  Group  Context 

Data  Transmission  Operation 

Support  UnicasVMulticast 

information  Management  Operation 

Manage  and  Retrieve  User/Group  Information 

3.2  Content  Manager 

Content  manager  manges  leamir^  contents  using  workflow  concepts.  Workflow  based  data  management  presents 
efficient  routir^  of  work  from  person  to  person  or  to  groups  and  easy  access  to  pertinent  documents.  Task  is 
defined  students  and  authors  who  use  contents.  Also,  Detailed  reports  of  all  activities  associated  with  a process. 
This  manager  has  task  manager  and  flow  manager.  Author  designs  the  flow  and  task  of  learning  conterits.  Task 
manager  and  flow  manager  control  the  learning  content  and  monitor  actions  which  is  performed  to  contents. 
Authors  and  instructors  design  flow.  And  course  manager  maintains  components  composed  in  document  and 
relationship  among  the  components.  The  relationships  are  made  up  with  update  and  delete  relation.  [Fig  4]  shows 
Flow,  Task,  Task  Manager  object. 
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Figure  4.  Objects  of  contet  manager 


For  interface  with  database  management  systems,  it  uses  persistence  service  in  Common  Object  Specification 
Services  (GOSS).  If  we  constmct  cyber  university  using  the  framework,  this  manager  is  connected  to  content 
management  center.  [Fig.  5]  shows  Course  manager. 


Content  management  center  helps  content  professionals- creators,  editors  and  site  administrators-  manage  the  full 
life  cycle  of  content. 

3.3  Student  Profile  Manager 

Student  profile  manager  maintains  student  profile,  includir^  student  private  information,  result  of  learning,  learning 
tendency.  Profile  manager  presents  operations  described  in  Table  2. 


Table  2 Operations  of  Student  Profile  manager 


OjefaHais 

; 7 ; "Fanctiofis  ' Y ^ ' ' 

Basic  Operations 

Create  Users 

Destroy  Users 

Fine  users 

Modify  user  profiles 

Registration  user's  goal 

Maintenance  of  Learning  State 

GetLeamIgHistory 
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SetLeamingHistoiy 
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3.4  Intelligent  Agent 

Intelligent  agent  determines  student  s learning  course  using  requirements.  Kno\\dedge  base  is  constructed  by 
instructors.  Then,  determined  result  is  used  for  designing  process  of  flow  of  contents  in  Content  manager. 
Intelligent  agent  has  extract  module  and  diagnosis  module.  Extract  module  extracts  parameters  for  diagnosis  of 
student  from  result  of  student*s  learning  and  return  to  diagnosis  module.  [Fig.6]  is  IDL(Interface  Definition 
Language)  interface  for  Extraction  module. 


stnjctProblem_Stnjct{ 

string  testNodelD; 
string  type; 
string  difficulty; 
string  conectness; 

); 

typedef  sequence<Problem_Stnjcl>  Problem Jnfo; 
interface  Extraction  { 

stiingSolving_Problem(ln  Userjnfo  user,  in  Problemjnfo  problem); 
string  Leaming_Node(ln  Userjnfo  user,  in  string  Nexfl; 


Figure  6.  IDL  interface  for  Extraction  module 

Diagnosis  module  diagnoses  learning  state  using  parameters  from  extract  module.  Diagnosed  result  is  used  for 
updating  profile  information.  [Fig.7]  is  IDL  interface  for  Diagnosis  module. 


stnjctCondition_Stnjc( 

Userjnfo  user; 
string  condition; 

1; 

typedef  sequence<Condition_Stnjci;>  Condition Jnib; 
interface  Diagnosis  { 

Condition  Jnfb  Examine(  in  string  UserJnput); 

J 

Figure  7.  IDL  interface  for  Diagnosis  module 

4.  Conclusion  & Future  Works 

Designed  framework  has  several  advantages.  It  can  provide  flexible  and  adaptable  solution  for  construction  of 
cyber  university.  Learning  course  is  designed  automatically  using  student *s  requirements  and  result  of  learning  is 
analyzed  and  feedback  for  the  bast  fit  education  can  be  presented  to  students.  Workflow  based  content 
management  supports  step-by-step  education. 

Also,  because  the  framework  is  designed  using  OORBA  objects,  system  expansion  and  integration  can  be  done 
easily.  There  is  an  important  feature  to  enhance  educational  values  of  this  system.  Currently,  We  plan  to  implement 
integrated  distance  learning  system  that  using  this  framework 
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Abstract:  Walden’s  Paths  provides  annotated  guided  paths  over  World-Wide  Web  pages.  Designed 
as  an  approach  for  organizing  classroom  use  of  contextualized  Web-based  material,  Walden’s  Paths 
provides  a general  mechanism  suitable  for  use  in  implementing  a range  of  different  pedagogical 
strategics.  This  paper  reports  on  some  of  the  uses  that  teachers  have  made  of  Walden’s  Paths.  One 
of  the  surprises  for  us  in  this  project  has  been  the  range  of  ways  in  which  Walden’s  Paths  can  be 
used  in  practice. 


Introduction 

Over  the  past  three  years  we  have  been  developing  Walden’s  Paths  (Shipman  et  al.  1997;  Furuta  et  al.  1997; 
Shipman  et  al.  1998),  software  supporting  the  use  of  existing  Web-based  materials  in  elementary  and  secondary 
classrooms.  This  work  has  been  motivated  by  the  belief  that  as  more  information  is  made  available  via  the  Web,  that 
there  is  a greater  potential,  and  indeed  need,  for  educators  to  make  use  of  this  resource. 

The  Internet  is  becoming  an  intrinsic  part  of  society  and  the  most  effective  method  of  distributing  many  types  of 
information.  Because  of  the  relatively  low  publication  cost  of  the  Internet,  economic  pressures  will  favor  increased 
use  of  it  for  information  dissemination.  Schools  will  have  to  find  methods  of  making  use  of  this  resource  or  ignore  a 
growing  amount  of  content  with  educational  potential.  The  question  should  not  be  whether  to  use  the  Internet  in  the 
classroom  but  how. 

There  are  a number  of  problems  with  using  the  Web  in  the  classroom.  The  Web  is  an  unedited  publication  media 
and  so  there  is  information  that  is  inaccurate  and  inappropriate  for  students.  Web  materials,  when  on  topic,  are  most 
often  authored  for  other  purposes  than  education,  leaving  students  confused  due  to  their  lack  of  vocabulary  or 
context.  Finally,  browsing  Web  materials  can  reduce  the  student  to  being  a “button  presser”,  navigating  from  page  to 
page  as  they  might  flip  from  channel  to  channel  on  television. 

Walden’s  Paths  represents  one  attempt  to  ameliorate  these  difficulties  by  providing  teachers  a way  to  focus  student 
exploration,  to  add  explanation  and  context  to  materials,  and  to  integrate  the  Web  resources  in  the  existing 
classroom  environment. 

The  next  section  of  this  paper  provides  an  overview  of  Walden’s  Paths’  functionality  and  interface.  After  this  we 
present  examples  of  how  teachers  have  made  use  of  Walden’s  Paths  within  the  context  of  their  existing  curriculum. 
This  leads  to  a discussion  of  issues  for  the  use  of  Walden’s  Paths  and,  more  generally,  the  use  of  the  Web  in  the 
classroom. 

What  is  Walden’s  Paths? 

Walden’s  Paths,  named  after  the  paths  Thoreau  unselfconsciously  created  through  the  woods  while  living  at  Walden 
Pond,  is  a tool  that  allows  teachers  to  construct  guided  paths  using  the  information  on  the  Internet.  Like  Thoreau’s 
trails,  guided  paths  provide  structure  through  an  otherwise  unstructured  environment. 
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Figure  1:  The  Path  Authoring  Too! 


The  guided  path  as  a concept  goes  back  at  least  as  far  as  Vannevar  Bush’s  Memex  (1945).  Paths  have  been 
developed  in  specialized  software  environments  and  investigated  with  limited  user  communities  in  the  late  1980’s 
(Zellweger  1989;  Trigg  1988).  Paths  in  Walden’s  Paths  are  an  ordering  of  Web  pages  with  associated  annotations. 
The  pages  used  can  be  from  anywhere  on  the  Internet  and  their  ordering  in  the  path  is  independent  of  any  existing 
navigational  structure. 

Walden’s  Paths  consists  of  three  components,  a Path  Authoring  Tool  for  creating  and  editing  paths,  a Path  Database 
for  storing,  retrieving,  and  sharing  paths,  and  a Path  Server  that  provides  access  to  published  paths. 

The  Path  Authoring  Tool,  shown  in  Figure  1,  is  a Java-based  interface  that  allows  keyword  searches  for  Web 
materials,  the  display  of  those  materials  in  an  external  browser  (e.g.  Netscape  Navigator),  and  the  selection, 
ordering,  and  annotation  of  pages  for  the  current  path.  The  authoring  tool  also  includes  a “Work  Space”  for  storing 
pages  that  do  not  fit  into  the  current  path  but  may  be  of  use  in  the  future. 

Authored  paths  are  stored  in  the  Path  Database.  This  database  provides  each  authorized  path  author  with  a working 
area  for  storing  paths.  Paths  stored  here  are  not  visible  to  the  students  and  may  be  recovered  for  editing  or  continued 
work.  When  a path  is  ready  for  access  by  readers  (here  we  are  using  the  tenn  reader  instead  of  student  to  convey  that 
the  person  accessing  the  path  is  not  necessarily  a student)  it  is  “published”  to  the  Path  Server. 

The  Path  Server  is  a Common  Gateway  Interface  (CGI)  program  that  creates  the  list  of  available  paths  and  their 
presentation  for  readers.  When  a reader  requests  a path  page,  the  Path  Server  constructs  control-flow  and  annotation 
frames  to  appear  at  the  top  of  the  browser  and  requests  the  material  from  the  source  site  on  the  Internet  to  appear  in  a 
lower  frame  (as  shown  in  Figure  2).  The  reader  can  use  the  control-flow  controls  to  step  along  the  path.  In  addition, 
the  links  in  the  source  page  remain  active.  Following  any  of  them  takes  the  reader  off  the  path,  as  shown  in  Figure  3. 
In  this  mode,  further  unconstrained  browsing  is  permitted.  The  reader  can  return  to  the  path  departure  point  at  any 
time  by  selecting  the  “return  to  path”  button  in  the  Walden’s  Paths  navigation  area  of  the  display. 

Use  in  the  Classroom 

The  variety  of  ways  in  which  Walden’s  Paths  has  been  used  to  take  advantage  of  Web-based  materials  in  the 
classroom  has  been  one  of  the  surprises  during  the  project.  Our  original  conception  of  educational  paths  was  that 
they  would  be  created  by  teachers  for  use  in  the  classroom  as  a convenient  way  for  collecting  widespread  materials. 
We  expected  that  teachers  would  then  use  the  paths  in  ways  similar  to  the  use  of  traditional  filmstrips — as  an 
enabling  way  to  direct  their  students’  traversal  of  a sequence  of  related  information  items.  In  use,  we  discovered  that 
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this,  indeed,  did  occur.  However  we  also  discovered  that  unexpected  uses  arose  as  well;  uses  that  reflected  a mixture 
of  roles  for  both  the  materials  and  for  the  students.  The  following  subsections  will  describe  five  separate  uses  of  the 
system  by  teachers.  For  privacy  reasons  we  do  not  identify  the  specific  classrooms  involved;  some  are  located  in 
Texas  and  others  on  military  bases  in  Germany  and  Italy. 


Embedded  Assignments 

A middle-school  science  teacher  decided  to  use  the  path  mechanism  to  create  a virtual  tour  of  the  Kitt  Peak 
Observatory.  By  collecting  pages  from  different  Web  sites,  he  created  a tour  that  not  only  described  Kitt  Peak,  but 
also  compared  it  to  other  observatories.  A page  from  this  path  is  shown  as  Figure  4. 

The  teacher  made  much  use  of  the  annotation  feature  of  Walden’s  Paths,  helping  provide  transitions  between  the 
information  on  the  different  pages  and  embedding  questions  for  the  students  to  answer  on  each  page.  The  students, 
while  traversing  the  path,  were  asked  to  answer  (on  paper)  the  various  questions  making  use  of  the  information 
provided  on  the  pages. 

The  features  are  most  interesting  with  this  use  of  Walden’s  Paths  are  the  mixtures  of  media — electronic  and 
paper — used  in  achieving  the  teacher’s  classroom  goals.  Here,  the  teacher  has  blended  Walden’s  Paths  annotation 
feature  with  a paper-based  assignment  to  focus  and  enhance  the  acquisition  of  information  from  the  mixture  of 
electronic  information  resources  presented. 

Modernizing  Existing  Practices 

In  a high  school  art  and  art  history  class,  a teacher  who  had  been  using  slides  of  paintings  and  sculptures  for  reviews 
and  exams  decided  to  use  Walden’s  Paths  to  improve  current  teaching  practices.  Traditionally,  during  an  art  history 
review  or  exam,  the  students  have  to  fill  out  a form  as  the  teachers  presents  images  in  a 35mm  slide  show.  Questions 
on  the  form  might  include  the  artistic  period  or  style  of  the  piece,  the  artist,  defining  characteristics  of  the  piece,  etc. 

One  problem  with  this  approach  is  that  each  slide  is  available  to  the  student  during  a short  period  of  time.  The 
student  must  quickly  write  down  answers  to  the  questions  before  the  next  slide  appears.  The  teacher  decided  to 
replace  the  35mm  slide  show  with  paths  of  images  from  on-line  museums.  In  the  revised  exam  each  student  could 
spend  as  much  they  needed  on  each  image  (within  the  time  constraints  of  the  class  period). 

The  interesting  feature  of  this  use  is  the  way  in  which  a traditional  teaching  practice  could  be  adapted  and  improved 
using  the  new  medium.  The  updated  solution  preserves  the  familiar  characteristics  of  the  original  situation  while 
overcoming  the  technological  limitations  imposed  by  the  equipment  used  previously. 
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Resources  for  Creative  Work 


A high-school  computer  music  class  teacher  used  Walden’s  Paths  as  a way  to  identity  useful  resources  to  students. 
Due  to  the  technical  nature  of  the  computer  music  class  there  were  few  textbook  resources  available  but  much 
information  on  the  Web. 

The  students,  whose  task  it  was  to  compose  a new  piece  of  music  using  MIDI,  needed  both  explanatory  and 
reference  materials  on  the  equipment  and  software  they  were  using.  The  teacher  created  paths  connecting  sites  with 
useful  information  on  MIDI  to  act  as  a shared  bookmark  list  for  the  whole  class.  Thus  the  teacher  created  a resource 
that  enabled  the  use  of  Web-based  materials  in  support  of  existing  creative/constructive  tasks. 

The  use  of  Walden’s  Paths  to  collect  separate  resources  together  to  form  a convenient  package  appears  to  be  a 
common  use  of  the  system  as  we  have  observed  it  in  other  teachers’  applications  as  well.  In  this  use  of  Walden’s 
Paths,  the  contextualization  of  information  is  not  as  important  as  is  the  coalescing  of  it  in  one  place. 

Class  Projects 

Another  high-school  music  class  used  Walden’s  Paths  in  support  of  learning  about  musicians  and  music  history.  The 
unique  aspect  of  this  use  is  that  Walden’s  Paths  became  an  authoring  environment  for  the  students.  The  students  had 
to  form  groups,  pick  topics,  and  author  paths  on  these  topics  much  like  traditional  group  assignments  to  write  reports 
on  a particular  topic. 

Topics  of  paths  ranged  from  specific  composers  and  artists  to  areas  of  music  (e.g.,  jazz).  The  students  had  to  search 
and  browse  the  Web  to  locate  information  on  their  topic  and  compose  a useful  path  on  the  topic.  The  information 
selected  by  students  included  materials  on  music  history,  on  the  time  period  and  context  of  the  music,  biographies  of 
artists,  and  renditions  of  specific  pieces. 

A similar  project  was  carried  out  in  another  high-school  music  class  where  each  student  had  to  suggest  one  or  two 
pages  for  a class  path  on  a particular  topic.  The  students  had  to  provide  a reason  why  they  thought  their  suggestion 
should  be  included  in  the  path  (why  the  reader  would  find  it  useful,  or  a description  of  the  type  of  information  it 
provided.)  In  this  case,  the  teacher  acted  as  a filter  on  what  pages  were  included  in  the  path. 

While  such  an  open-ended  project  could  be  given  to  students  in  this  high-school  classroom,  there  might  be 
difficulties  in  such  an  assignment  for  younger  students  due  to  the  challenges  of  locating  information  on  the  Web. 

Extra-Credit 

Another  case  of  students  authoring  paths  came  in  a high-school  art  class.  In  this  case,  an  individual  student,  who  was 
ahead  of  her  fellow  students,  was  given  assignments  for  authoring  paths  that  could  be  used  by  other  students.  This 
not  only  provided  the  student  something  creative  to  do  that  related  to  the  course  curriculum  but  also  provided  the 
teacher  with  assistance  in  developing  materials  for  other  students. 

The  positive  feelings  associated  with  seeing  ones  own  work  available  to  others  helped  drive  much  of  the  early 
publication  on  the  Web.  The  publication  of  paths  can  be  used  as  an  incentive  for  students. 

Discussion 

The  experience  we  gained  from  applying  Walden’s  Paths  in  schools  suggests  some  additional  observations  to  us, 
which  perhaps  are  less-related  to  the  specific  characteristics  of  our  technology  and  more  outcomes  of  Web-based 
projects  in  general.  In  this  section,  we  discuss  some  of  these  issues. 

Technological  and  social  solutions 

Paralleling  the  observations  that  we  have  developed  earlier  in  this  paper  about  incorporation  of  new  technology  into 
existing  practices  and  association  with  existing  artifacts,  we  were  struck  by  the  effectiveness  of  combined 
technological-social  solutions  in  resolving  technology-introduced  issues.  A frequent  concern  expressed  to  us  during 
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the  development  of  the  Walden’s  Paths  project  have  been  the  issues  surrounding  identification  of  appropriate 
material  for  use  by  schoolchildren,  particularly  those  in  lower  grades.  The  path  allows  the  selection  of  suitable 
materials  and  the  annotation  associated  with  path  material  permits  the  teacher  to  provide  contextualization  of  that 
material.  However,  off-path  browsing  remains  unconstrained,  as  do  general  browser  controls. 

Our  early  assessment  of  the  situation  was  that  the  commercial  sector  was  likely  to  provide  technological  solutions 
that  would  ease  this  situation  through  the  development  of  blocking  software.  To  a good  degree,  this  seems  to  be  the 
case.  However,  we  were  impressed  by  the  simplicity  and  effectiveness  of  the  solution  adopted  by  a middle-school 
teacher  in  one  of  our  test  sites — rearranging  the  classroom  to  place  the  newly-obtained  computers  in  the  center  of 
the  room,  surrounded  by  the  desks  of  the  students. 

Local  and  global  path  visibility 

Addressing  issues  relating  to  intellectual  property  protection  have  been  among  the  most  interesting,  challenging,  and 
persistent  topics  faced  by  the  Walden’s  Paths  project.  As  we  reported  in  a previous  paper  (Shipman  et  al.  1998), 
early  concerns  focused  on  reader  confusion  between  content  and  container — between  original  source  material  and 
navigation/annotation  material  provided  by  Walden’s  Paths.  We  have  carried  out  several  redesigns  of  the  reader’s 
interface  to  make  these  distinctions  stronger;  the  design  shown  in  this  paper’s  figures  is  the  latest  in  this  process. 
However,  we  have  come  to  recognize  that  an  interacting  issue  is  that  of  path  visibility. 

Paths  have  been  created  for  many  purposes — to  serve  as  a resource  for  an  individual  classroom,  to  communicate 
findings  from  a student  to  a teacher,  to  promote  a class’  efforts  to  the  outside  world,  and  to  serve  as  a guide 
providing  an  overview  of  a Web  site  or  organizational  location.  Implicit  in  each  of  these  uses  is  an  expectation  of 
who  the  appropriate  audience  will  be  for  the  path — an  individual,  a localized  group,  or  the  whole  Web.  Our  current 
path  directory  mechanisms  do  not  allow  the  identification  of  the  intended  reader  population,  providing  global  access, 
but  not  localized  access.  This  leads  to  confusion  both  by  readers  but  also  by  content  providers  who  notice 
unexpected  changes  in  access  patterns  as  the  paths  become  more  widely  used. 

One  of  our  current  project  priorities,  therefore,  is  the  development  of  a richer  directory  implementation  that  will 
enable  more  fine-grained  specification  of  the  community  of  interest  for  paths.  However  a more  comprehensive 
solution  may  require  the  development  of  a more  flexible  model  of  ownership  and  location  for  Web  resources. 

The  Path  Server  is  essentially  operating  as  a proxy  server — obtaining  requests  from  clients  and  redirecting  those 
requests  to  servers.  The  TranSend  project  at  U.C.  Berkeley  (http://transcend.cs.berkeley.edu/),  which  provides  a 
generally-available  proxy  service,  noted  an  unexpected  side  effect  of  their  service,  namely  the  general  availability  of 
services  site  licensed  to  the  Berkeley  campus.  This  was  because  their  proxy  service  runs  inside  of  Berkeley’s 
domain;  special  action  was  required  to  restrict  access  to  licensed  services. 

In  our  current  implementation,  material  is  only  redirected  when  on  a path  (off  path  material  is  obtained  directly  by 
the  client  computer  and  is  subject  to  the  restrictions  at  the  client’s  site).  Consequently  we  have  not  yet  encountered  a 
corresponding  situation  although  the  potential  exists  for  a path  author  to  take  advantage  of  our  architecture  to  allow 
outside  access  to  restricted  services.  A comprehensive  solution  here  may  also  involve  reexamination  of  ownership 
for  Web  resources. 

Conclusions 

In  this  paper,  we  have  described  some  of  the  unexpected  uses  of  Walden’s  Paths  that  appeared  during  teacher  use. 
Although  we  had  specific  pedagogical  goals  in  mind  when  we  initiated  the  project  (Shipman  et  al.  1996),  our  current 
evaluation  is  that  the  tool  created  is  relatively  independent  of  specific  pedagogy.  Consequently,  Walden’s  Paths  can 
be  incorporated  into  a variety  of  teaching  practices.  Furthermore,  if  specific  pedagogical  guidelines  are  to  be 
supported,  Walden’s  Paths  provides  an  appropriate  substrate  for  implementation  of  authoring  filters  that  encourage 
following  of  those  guidelines. 

The  Walden’s  Paths  Web  pages  are  located  at  http://www.csdl.tamu.edu/walden/ 
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Abstract:  The  potential  of  the  Internet  to  deliver  instruction  should  not  be  underestimated. 
However,  Internet  use  raises  a number  of  issues,  including  equitable  access.  The  Americans  with 
Disabilities  Act  (ADA)  requires  that  programs  in  the  United  States  be  made  accessible  to 
qualified  participants  with  disabilities  and  the  ADA  accessibility  requirements  apply  to  Internet 
resources.  Distance  learning  course  developers  should,  therefore,  take  steps  to  assure  that  their 
courses  are  accessible  to  students  with  a wide  range  of  disabilities.  But,  what  does  this  mean  for 
a course  offered  over  the  Internet?  This  paper  discusses  access  issues  and  provides  an  example 
of  an  accessible  course  that  is  currently  offered  via  the  Internet. 


Much  has  been  written  about  the  revolutionary  impact  that  networking  technology  is  having  and  will  continue  to 
have  on  all  levels  of  education.  Multimedia,  simulations,  real-time  communications,  interactions  with  artificial 
intelligence  systems  — the  list  of  capabilities  seems  endless.  Distance  learning  classes,  once  conducted  using  postal 
mail,  FAX,  and  the  telephone,  are  rapidly  being  converted  to  Internet-based  delivery  systems  (Driscoll  1998; 
Garrison  1989;  Palloff  & Pratt  1999;  Stewart  1995).  Although  the  electronic  delivery  of  courses  is  unlikely  to 
completely  replace  traditional  classroom  instruction,  this  powerful  option  for  the  delivery  of  information  and  the 
facilitation  of  communication  should  not  be  ignored  or  underestimated  by  institutions  of  higher  education, 
corporations,  and  other  organizations. 

One  issue  that  should  be  raised  when  the  Internet  is  used  to  deliver  instruction  is  that  of  equitable  access.  The 
Americans  with  Disabilities  Act  (ADA)  of  1990  requires  that  organizations  in  the  United  States  that  provide 
programs  and  services  to  the  public  must  provide  the  same  programs  and  services  to  people  with  disabilities  that 
they  provide  to  people  without  disabilities.  According  to  this  law,  no  otherwise  qualified  individual  with  a 
disability  shall,  solely  by  reason  of  his/her  disability,  be  excluded  from  the  participation  in,  be  denied  the  benefits 
of,  or  be  subjected  to  discrimination  in  program  offerings.  When  people  think  of  the  ADA  they  usually  think  of 
accommodations  such  as  elevators  in  buildings  and  lifts  on  busses.  But,  according  to  the  United  States  Department 
of  Justice,  the  ADA  accessibility  requirements  apply  to  the  Internet  world  also.  A 1996  opinion  letter  ("ADA 
Accessibility,"  1997)  states  that  “Covered  entities  that  use  the  Internet  for  communications  regarding  their 
programs,  goods,  or  services  must  be  prepared  to  offer  those  communications  through  accessible  means  as  well.” 
Specifically,  if  you  offer  a distance  learning  course  via  the  Internet,  to  comply  with  the  law,  your  offering  must  be 
made  available  to  students  who  have  disabilities. 

Distance  learning  course  developers  should  take  steps  to  make  their  courses  accessible  to  students  with  a wide 
range  of  disabilities.  They  can  employ  “universal  design”  principles  to  assure  that  their  materials  are  not  only 
accessible  to  the  average  student  in  a class,  but  to  all  students  who  might  enroll,  including  those  with  disabilities. 
But,  what  does  this  mean  for  a specific  course  that  you  may  wish  to  offer  over  the  Internet?  This  paper  discusses 
access  issues  and  provides  an  example  of  an  accessible  course  that  1 currently  offer  via  the  Internet. 


The  Students 

With  the  rapid  development  of  adaptive  technologies  that  allow  people  with  a wide  range  of  disabilities  to  access 
computers,  students  with  the  following  characteristics  may  be  attracted  to  a distance  learning  class  offered  over 
the  Internet. 
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• Trevor  is  blind.  He  uses  a computer  with  voice  output.  Basically,  Trevor’s  computer  reads  with  a 
synthesized  voice  whatever  text  appears  on  the  screen.  He  uses  a text-only  browser  to  navigate  the  World 
Wide  Web.  One  of  the  challenges  he  faces  when  he  accesses  Web  sites  occurs  when  graphics  appear  and 
there  is  no  text  alternative  for  his  voice  output  system  to  read.  He  may  just  hear  ’’graphic"  when  a 
detailed  picture  of  the  anatomy  of  a frog  is  presented  on  the  screen,  “link”  when  no  further  descriptive 
text  is  provided  in  the  link,  and  “image  map”  in  place  of  the  beautiful  image  map  with  linkable  options 
that  sighted  visitors  see. 

• Maggie  has  low  vision.  She  uses  special  software  to  enlarge  all  screen  images.  Even  with  her  large 
monitor,  she  only  sees  a small  portion  of  a Web  page  at  a time.  She  is  challenged  when  pages  are 
cluttered  and  when  the  page  layout  is  inconsistent  from  one  page  to  the  next. 

• Mark  is  deaf  The  Internet  has  been  a great  resource  for  him  since,  for  the  most  part,  the  ability  to  hear  is 
not  required.  Lately,  however,  he  is  running  into  more  and  more  multi-media  sites  with  sound  clips  that 
are  not  captioned  or  transcribed.  He  becomes  totally  lost. 

• Sarah  was  an  innocent  victim  in  a drive-by  shooting.  Her  brain  injury  has  resulted  in  impaired  fine  motor 
skills.  She  uses  one  hand  to  operate  a keyboard  and  track  ball.  She  finds  it  difficult  to  select  buttons  on 
the  screen  when  they  are  very  small. 

• Ryan,  a bright  college  student,  has  a learning  disability  which  impacts  his  ability  to  read  and  organize 
information.  He  has  difficulty  understanding  Web  sites  when  the  information  is  cluttered  and 
unorganized  and  when  the  screen  layout  changes  from  page  to  page. 


How  can  you  assure  that  these  students  will  be  able  to  access  a distance  learning  class  that  you  offer  on  the 
Internet?  1 will  share  what  I have  learned  in  teaching  an  Internet-based  course  for  three  years. 


The  Course 

My  Adaptive  Computer  Technology  distance  learning  class  (Burgstahler,  1 997)  is  designed  for  teachers,  parents, 
service  providers,  and  computer  lab  managers.  It  is  offered  for  three  college  credits  in  both  rehabilitative  medicine 
and  education  through  the  University  of  Washington  in  Seattle.  The  course  surveys  the  field  of  adaptive 
technology  as  it  impacts  the  lives  of  people  with  disabilities,  including  applications  to  employment,  education, 
and  recreation.  Topics  include  interface  devices,  computer  applications,  compensatory  tools,  access  to  information 
technology,  legal  issues,  and  implementation  strategies. 

The  electronic  mode  of  delivery  is  an  excellent  choice  for  this  course.  Because  of  its  specialized  nature,  few 
people  in  one  locality  are  interested  in  taking  it  at  any  specific  time.  Students  have  participated  from  throughout 
the  United  States  as  well  as  from  Canada,  Italy,  Germany,  Japan,  Australia,  and  Hong  Kong.  Being  in  the  same 
place  at  the  same  time  is  not  a prerequisite  for  participation.  Students  who  have  successfully  completed  the  course 
include  those  who  have  visual  and  hearing  impairments,  mobility  impairments,  and  specific  learning  disabilities. 
Design  features  of  the  course  have  made  it  accessible  to  everyone  who  has  enrolled. 


The  Instructors 

The  first  time  this  course  was  offered  I taught  it  with  a co-instructor  who  is  blind  and  lives  in  New  York.  Since  his 
computer  is  equipped  with  a screen  reader  and  speech  synthesizer,  he  required  no  special  accommodation  as  we 
developed  the  course.  We  prepared  materials  and  coordinated  lessons  via  electronic  mail.  We  ’’met"  many  times 
to  discuss  the  progress  of  the  course,  but  never  in  person. 
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The  Instruction 


Students  in  the  Adaptive  Computer  Technology  course  purchase  textbooks  and  videotapes  before  the  class  begins. 
They  are  placed  on  an  electronic  distribution  list  managed  by  standard  ListProcessor  software.  The  distribution  list 
is  used  to  send  the  syllabus  and  other  course  materials  to  the  students.  Lessons  are  distributed  over  a period  of  10- 
12  weeks,  but  discussions  continue  for  a total  of  six  months.  Students  are  required  to  read  and  respond  to 
electronic  mail  at  least  once  per  week  during  the  time  that  lessons  are  distributed.  Class  participation  is  required. 
To  keep  communications  lively  and  prevent  some  students  from  just  "lurking,”  I require  each  student  to  contribute 
at  least  one  comment  to  each  lesson  discussion.  Once  distributed,  the  lessons  are  archived  on  the  course  World 
Wide  Web  site  where  students  and  instructors  can  easily  access  them. 

Textbook  reading  and  videotape  viewing  assignments  are  distributed  via  electronic  mail  with  the  weekly  lessons. 
After  an  assignment  is  given  to  read  material  or  watch  a videotape  presentation,  the  class  discusses  the  content  via 
the  electronic  distribution  list.  Small  group  discussions  break  off  from  full-class  discussions  from  time  to  time  as 
individuals  find  common  interests  with  others.  Participants  also  communicate  individually  with  each  other  and 
with  me  via  electronic  mail.  Guest  speakers  join  in  class  discussions  periodically.  One  regular  guest  speaker  is  the 
author  of  the  course  textbook.  He  lives  far  away  from  me  and  is  an  adaptive  technology  user  himself  He  usually 
joins  the  discussion  list  for  two  weeks. 

In  this  distance  learning  course,  all  assignments  are  turned  in  to  me  via  electronic  mail.  Students  share  summaries 
with  their  classmates  via  the  course  distribution  list.  The  first  assignment  for  students  in  class  is  to  distribute  an 
introductory  biography  to  the  class.  Some  of  the  required  "papers"  involve  writing  on  a topic  related  to  the  course 
content  using  Internet  resources,  visiting  a local  facility  and  evaluating  computer  access  for  individuals  with 
disabilities,  and  making  recommendations  regarding  access  to  a specific  facility  or  program.  When  ready,  each 
student  requests  the  final  exam  via  electronic  mail.  Once  it  is  delivered,  he/she  has  several  days  in  which  to 
complete  this  essay  test  and  return  it  to  me  via  electronic  mail. 


The  Accessibility  of  the  Course 

For  the  Adaptive  Computer  Technology  course  there  is  no  meeting  "place."  This  means  that  finding  an  accessible 
classroom  to  accommodate  students  with  disabilities  is  not  required.  In  this  course,  the  primary  vehicles  for 
learning  are  a textbook,  videotape,  electronic  mail,  an  electronic  distribution  list  and  the  World  Wide  Web.  Care 
is  taken  to  assure  that  each  delivery  mode  is  accessible  to  all  learners. 


Textbook 

The  required  textbook  for  this  class  is  available  in  recorded  form  for  students  who  are  blind  or  who  have  specific 
learning  disabilities.  Recorded  and  electronic  copies  of  books  can  be  purchased  from  Recordings  for  the  Blind  and 
Dyslexic.  On  most  campuses,  the  Disabled  Student  Services  office  can  help  instructors  determine  how  a specific 
text  can  be  provided  in  Braille,  on  disk,  or  on  tape  if  such  a request  is  made. 


Videotape 

Students  in  the  course  also  purchase,  as  part  of  the  required  course  materials,  three  videotape  presentations  which 
demonstrate  key  concepts  covered  in  the  course.  Videotapes  are  open-captioned  to  make  them  accessible  to  deaf 
students.  In  addition,  students  who  are  blind  can  purchase  them  in  audio-described  format.  This  special  format 
provides  audio  descriptions  of  videotape  content  that  is  presented  visually. 


Electronic  Mail  and  an  Electronic  Distribution  List 

Electronic  mail  is  highly  accessible  to  those  with  disabilities.  Media  conversion  and  other  customized 
accommodations  are  minimized  since  all  participants  are  required  to  have  access  to  computers  when  they  enter  the 
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class.  Whatever  adaptive  technologies  they  use  facilitate  access.  For  example,  a blind  student's  computer  output 
method  (usually  a screen  reader  and  voice  synthesizer)  translates  the  course  material  available  via  electronic  mail 
into  an  accessible  format  (e.g.,  speech).  A student  who  cannot  use  her  hands  can  command  her  computer  to  “turn 
pages.”  Similarly,  a deaf  student  does  not  require  interpreters  or  amplification  systems  since  lectures  and 
discussions  occur  on-line. 


World  Wide  Web 

A World  Wide  Web  page  serves  as  the  course  "library."  Links  to  useful  sites  provide  students  with  thousands  of 
pages  of  resources  for  their  papers,  projects,  and  extended  learning.  The  course  Web  pages  are  designed  in  such  a 
way  that  people  using  a wide  range  of  adaptive  technologies  can  access  them  (Burgstahler,  Comden,  & Fraser 
1997;  Dixon  1996;  Resmer  1997).  Universal  design  principles  are  employed  to  maximize  accessibility.  They 
include  simplicity,  consistency  and  redundancy.  A few  key  access  issues  are  discussed  below.  More  detailed  Web 
accessibility  guidelines  can  be  found  on  the  Web  itself.  The  site  maintained  by  the  World  Wide  Web  Consortium, 
at  http://www.w3.org/WAI/,  is  a good  place  to  start. 


Simplicity 

To  make  Web  pages  accessible,  it  is  important  to  use  a simple  page  layout.  Buttons,  navigational  links  and  logos 
should  be  in  logical  places  on  each  page.  A simple  page  is  particularly  helpful  to  people  who  have  visual 
impairments  and  to  those  with  specific  learning  disabilities.  In  addition,  large  buttons  allow  for  easy  access  for 
individuals  with  limited  fine  motor  skills  as  they  use  mice,  track  balls,  and  other  pointing  devices.  A high  level  of 
contrast  between  the  background  and  the  text  on  the  screen  and  simple  background  designs  ease  access  for  people 
who  are  colorblind  or  who  have  low  vision.  Using  universally  recognized  HTML  (Hypertext  Markup  Language) 
tags  allows  all  Web  browsers  to  accurately  display  the  content. 


Consistency 

Maintaining  the  same  page  layout  from  page  to  page  within  a Web  site  makes  it  more  accessible.  Buttons, 
navigational  links,  and  logos  should  be  located  in  the  same  places  on  each  page.  A consistent  page  layout  makes 
navigating  the  site  easier  for  everyone,  but  particularly  for  people  who  have  visual  impairments  and  for  those  with 
specific  learning  disabilities. 


Redundancy 

Some  Web  users  cannot  see  images  or  hear  sounds  because  of  sensory  impairments.  Providing  content  in 
alternative  formats  can  make  information  accessible  to  everyone.  For  example,  blind  Web  users  typically  read  the 
text  at  a Web  site  by  using  voice  output  systems.  They  do  not  have  access  to  information  presented  in  graphical 
form.  Therefore,  it  is  important  to  provide  alternate  text  for  each  graphic  so  that  those  who  cannot  view  the  image 
can  access  the  information  it  presents.  Captions  on  pictures  should  be  written  in  such  a way  that  they  are 
meaningful  to  visitors  who  cannot  see  the  pictures.  If  manuscripts  are  displayed  in  an  image  format  or  video  clips 
are  used,  text  transcription  should  be  provided.  Image  map  choices  should  be  provided  in  a text-based  list  in  order 
to  assure  that  the  links  embedded  in  the  image  map  are  available  to  those  accessing  only  the  text  of  the  page. 
Tables,  frames,  forms,  databases  and  special  features  should  be  used  sparingly.  They  are  difficult  or  impossible  to 
use  by  blind  students  and,  thus,  require  that  an  alternative  format  be  provided. 

Web  pages  for  your  distance  learning  class  should  be  tested  with  a variety  of  Web  browsers  and  monitors.  One  of 
the  browsers  used  should  be  text-only,  such  as  Lynx.  In  addition,  a site  can  be  tested  for  accessibility  using 
"Bobby.”  Bobby,  created  at  the  Center  for  Applied  Special  Technology,  is  an  HTML  validator  program  that  tests 
for  accessibility  and  identifies  non-standard  and  incorrect  HTML  coding.  Bobby  is  located  at 
http://www.cast.org/bobby. 
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“Way  Cool”  Technology 


In  my  distance  learning  course  I made  a conscious  choice  to  use  simple  software  tools  - electronic  mail, 
distribution  lists,  the  World  Wide  Web.  But,  what  about  chat  rooms,  specialized  bulletin  board  and  conferencing 
software,  interactive  video  and  other  emerging  technologies?  When  it  comes  to  people  with  a wide  range  of 
disabilities,  generally  speaking,  the  simpler  the  software  (e.g.,  text-based),  the  easier  it  is  for  them  to  access.  For 
example,  real-time  chat  systems  are  impossible  to  use  by  individuals  for  whom  input  and  output  methods  are  slow, 
whereas  simple  electronic  mail  is  totally  accessible.  Video  and  audio  interactions  require  that  access  barriers  for 
visually  or  hearing  impaired  students  be  overcome.  Whatever  tools  you  choose  to  use,  refer  back  to  the 
descriptions  of  Trevor,  Maggie,  Mark,  Sarah,  and  Ryan  at  the  beginning  of  this  paper.  Make  sure  that  they  can 
fully  participate  in  your  course. 


Conclusion 

The  Internet  is  a powerful,  flexible,  and  efficient  tool  for  the  delivery  of  instruction.  It  provides  new  ways  for  us 
to  teach  and  learn.  Distance  learning  courses  offered  over  the  Internet  attract  people  from  all  over  the  world.  Some 
potential  students  have  disabilities.  A course  using  the  Internet  can  reduce  the  necessity  for  special 
accommodations  for  students  with  disabilities,  but  only  if  it  is  designed  with  access  in  mind.  Employing  universal 
design  principles  will  make  your  electronic  resources  accessible  to  people  with  disabilities  and  will  also  improve 
the  access  for  people  using  older,  slower  technology.  With  a properly  designed  course,  those  with  limited  abilities 
to  speak,  hear,  see,  or  move  will  not  be  limited  in  participation.  In  fact,  your  most  "vocal"  student  may  not  even 
be  able  to  speak  in  the  traditional  way. 
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Abstract:  Based  on  long  experience  with  educational  software,  both  as  developers  and  users,  and 
with  the  use  of  telematics  applications  in  educational  settings,  a team  responsible  for  the 
implementation  of  more-flexible  education  within  the  Faculty  of  Educational  Science  and 
Technology  at  the  University  of  Twente  derived  a set  of  requirements  integrating  technical  and 
pedagogical  principles  for  a WWW-based  course-support  system  for  the  faculty.  These 
requirements  have  been  realized  in  a system  built  by  the  team  and  now  is  use  across  the  faculty. 
The  paper  gives  the  system  requirements,  an  overview  of  the  TeleTOP  system  as  realized  to  meet 
the  requirements,  and  some  indicators  of  the  accuracy  of  the  requirements  based  on  experiences 
with  the  broadscale  use  of  the  system. 


Introduction 

Since  the  introduction  of  computers  into  education,  there  have  been  efforts  to  develop  educational  software  that 
could  replace  the  instructor.  Despite  extensive  efforts  to  adequately  model  the  critical  variables  in  the  learning 
process,  the  results  have  generally  not  led  to  software  that  has  taken  over  from  the  instructor,  with  exceptions  for 
what  Draper  calls  "niche  markets"  (Draper,  1998).  Within  traditional  higher  education,  the  use  of  educational 
software  as  a replacement  for  the  instructor,  the  textbook,  the  laboratory  session,  or  the  lecture,  has  not  much 
occurred,  especially  at  a faculty  level  where  the  scale  would  be  large  enough  to  make  an  impact  on  curriculum  and 
program  delivery.  A number  of  analyses  have  studied  the  implementation  problem  and  share  the  conclusion  that  the 
instructor  is  a critical  variable  in  any  use  of  educational  software  in  school  and  institutional  settings  (see  Collis, 
Knezek,  Lai,  Muyashita,  Pelgrum,  Plomp,  & Sakamoto,  1996,  for  an  international  overview). 

The  availability  of  the  WWW  has  brought  a new  wave  of  interest  into  the  previous  educational-software  domain, 
particularly  in  higher  education.  However,  we  see  in  many  cases  a repeat  of  early  ideas  that  accompanied  computers 
in  education,  in  particular  the  idea  that  a WWW-based  course  could  provide  a replacement  for  the  instructor.  The 
new  variation  of  this  is  reduction  of  the  instructor  to  someone  who  answers  e-mails  when  students  have  a question. 
We  believe  the  same  problems  will  confront  WWW-based  courses  as  confronted  educational  courseware:  problems 
related  to  not  being  able  to  anticipate  all  the  sorts  of  human-human  interaction  and  issues  that  arise  in  the  regular 
teaching-learning  situations  which  the  good  instructor  handles  based  on  his  or  her  experience  and  intuition.  That  is 
why  we  endorse  the  use  of  WWW-based  applications  as  tools  to  empower  and  extend  good  teachers,  not  replace 
them. 


The  Teletop  Project 

Since  March  of  1994  enthusiastic  instructors  in  the  Faculty  of  Educational  Science  and  Technology  of  the 
University  of  Twente  have  been  using  WWW-based  environments  to  support  their  regular  courses  (for  a review  of 
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four  year’s  evolution  of  one  such  course,  see  Collis,  1997).  In  these  courses,  the  instructor  was  not  less  present  or 
less  involved  with  his  or  her  students,  but  was  empowered  by  the  WWW-based  tools  to  do  more  than  before.  Based 
on  the  experiences  of  these  pioneer  instructors,  the  reactions  of  the  students,  and  the  idea  of  extending  and 
empowering,  not  replacing,  the  good  teacher,  the  adminstration  of  the  Faculty  decided  in  May  1997  to  commit 
substantial  resources  to  the  systemic  support  of  change  throughout  the  faculty:  all  courses  to  be  re-designed  to  be 
more  flexible  and  to  embody  the  principle  of  extending  the  good  instructor.  WWW-based  tools  were  to  be 
the  major  technological  instruments,  while  extensive  work  with  instructors  in  a new  approach  to  rapid 
prototyping  of  WWW-based  course-support  documents  was  a parallel  emphasis  (De  Boer  & Collis, 
1998a,b).  This  initiative  is  called  the  TeleTOP  Project.  Along  with  the  authors  (who  are  responsible  for  the  project) 
are  five  full-time  instructional  designers  supported  by  the  MediaLab  of  the  faculty.  Throughout  the  1997-98,  all  of 
the  courses  of  the  first-year  as  well  as  a number  of  other  courses  have  been  redesigned,  and  are  now  in  operation 
with  a double-sized  student  group  (the  normal  intake,  and  a new  intake  of  about  the  same  size  of  professional 
persons  in  the  workplace  wishing  to  complete  a flexible  version  of  our  programme,  primarily  at  a distance  but 
coming  to  the  campus  one  day  every  two  weeks  for  intensive  face-to-face  contact  with  their  instructors).  Now, 
during  the  1998-99  academic  year,  the  majority  of  the  other  courses  in  the  required  program  will  be  similarly 
adapted. 


Requirements 

While  the  original  examples  in  the  faculty  of  use  of  WWW  environments  to  support  courses  were  those  of 
enthusiastic  pioneer  instructors,  the  TeleTOP  Project  has  to  deal  with  a more  difficult  target  group:  all  instructors, 
including  the  less-interested,  less-motivated,  and  sceptical,  are  to  be  supported  in  their  re-design  of  their  courses  and 
their  design  of  WWW  sites  to  support  their  courses.  Given  these  two  basic  aspects— empower,  not  replace  the 
instructor  with  technology;  and  engage  all  instructors,  with  a wide  variation  in  computer  skills  as  well  as  a wide 
variety  of  levels  of  interest  in  the  use  of  technology  in  their  own  teaching— we  developed  the  following  set  of  12 
guiding  principles: 

1.  To  make  the  threshold  of  use  as  low  as  possible  for  the  instructors,  the  WWW-environments  must  be  usable  by 
the  instructors  without  having  to  have  a special  training  course  and  as  easily  as  they  handle  a word  processor.  The 
TeleTOP  team  realized  this  principle  by  developing  a database-generated  system  in  which  a flexible  course-support 
environment  is  available  to  the  instructor.  The  instructor  chooses  the  features  he  or  she  desires  to  support  the  course, 
and  only  has  to  type  into  various  types  of  fill-in  forms  to  organize  the  notes  and  other  material  that  is  to  be  put  into 
the  WWW  site. 

2.  Students  also  must  be  able  to  use  the  system  without  instruction,  and  no  more  than  a small  manual.  Once  they 
are  familiar  with  the  interface  of  one  course-support  environment,  they  have  a consistent  interface  in  other  courses, 
reducing  the  need  to  learn  to  handle  new  environments  with  each  new  course.  The  environments  are  to  be  accessed 
through  familar  browsers;  no  new  packages  to  learn,  and  nothing  that  requires  them  to  come  to  the  computer 
laboratory  to  use. 

3.  Similarly,  the  instructor  must  be  able  to  do  everything  he  or  she  wishes  with  the  course  through  an  ordinary 
WWW  browser;  no  special  authoring  software,  no  special  client.  An  instructor  travelling  out  of  the  country  can 
work  on  his  or  her  course  wherever  there  is  access  to  the  Internet  via  a standard  browser.  A guest  instructor  in 
another  country  can  enter  notes  and  materials  directly  into  the  course  site,  again  without  any  special  equipment  and 
without  needing  to  have  access  to  the  server.  Thus  the  system  should  not  be  a server-client  model,  whereby  the 
instructor  in  addition  to  the  Internet  browser  has  to  install  special  software  at  the  client  side  to  get  access  to  the 
learning  environment. 

4.  By  the  principle  "empowerment  of  the  good  instructor",  the  instructor  must  be  able  to  choose  for  him  or  herself 
the  way  that  a WWW  site  will  be  used  to  support  his  or  her  course.  There  is  no  standard  pedagogical  model  that  all 
are  expected  to  follow.  With  this  in  mind,  the  TeleTOP  Team  developed  a WWW-based  Decision  Support  Tool 
(DST)  (see  De  Boer  & Collis,  1998a,b)  to  help  the  instructor  get  a systematic  overview  of  examples  of  different 
ways  WWW-based  tools  can  be  used  within  the  organizational  aspects  of  their  courses,  the  aspects  relating  to 
lectures  and  face-to-face  sessions,  aspects  related  to  communication  with  and  among  students,  to  the  presentation  of 
course-related  materials,  to  file  distribution  and  the  addition  of  resources  to  the  course  site.  After  an  interview 
session  with  TeleTOP  Team  members  and  the  DST  , the  instructors  have  a more-informed  idea  of  what  is  possible, 
and  more  importantly,  what  they  will  find  appropriate  to  use  in  their  own  courses.  Directly  after  the  use  of  the  DST, 
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a WWW-based  framework  of  each  instructor’s  new  course-support  site  is  available,  to  be  tried  out  by  the  instructor 
via  an  ordinary  browser.  We  believe  the  instructor  must  be  the  decision  maker  about  the  course  site,  but  we  know 
from  our  experience  with  authoring  systems  for  instructors,  that  this  principle  often  does  not  lead  to  any  use  of  the 
authoring  system  or  products  made  from  it.  This  our  approach  involves  continual  contacts  with  the  instructors. 

5.  We  do  not  see  the  course-support  sites  as  replacing  the  textbooks  in  the  courses  or  making  lectures  unnecessaiy. 
Instead  the  course-support  sites  help  the  instructor  add  extra  opportunities  for  student  reflection,  for  communication, 
for  student  contribution  of  additional  learning  resources,  for  peer  interaction  and  peer  evaluation,  and  to  add  a 
’’preparation  for”  and  ’’follow-up  from”  each  face-to-face  session.  Thus  we  conceptualize  our  course-support 
environments  as  information-communication  exchange  environments,  that  are  also  coupled  with  other  information 
systems  of  the  faculty  such  as  the  bureau  responsible  for  student  issues  and  administration. 

6.  The  course-support  environments  must  be  capable  of  supporting  a large  variety  of  different  types  of 
instructional  approaches,  from  courses  focused  on  reading  and  written  assignments  with  classic  final  examinations, 
to  courses  with  complicated  approaches  to  group  work  and  project-based  education.  Tools  to  support  any 
instructional  approach  must  be  available,  including  shared  workspaces,  test  banks,  and  discussion  boards. 

7.  The  system  must  work  will  all  other  WWW  products,  for  example  Java  applets  and  plug-ins. 

8.  The  instructor  must  be  able  to  put  in  and  take  out  whatever  is  necessary  in  the  course  site  without  needing  direct 
technical  support  (although  we  have  student  assistants  who  help  out).  Uploading  and  downloading  attachments  of  a 
variety  of  types  is  particularly  important. 

9.  The  sourse  sites  must  help  instructors  organize  the  information  streams  within  a course;  instead  of  student 
messages  coming  to  the  instructor’s  e-mail  address,  for  example,  they  can  be  posted  directly  into  the  course  site, 
either  as  private  (only  for  the  instructor  to  see)  or  public  (for  all  in  the  course  to  see).  Feedback  from  students  or  the 
instructor  must  follow  the  same  principle.  Also,  there  should  be  easy-to-set-up  ways  for  messages  to  be  sent  to  a 
group  of  students,  or  all  the  students  in  the  course,  or  other  groupings  within  the  course,  all  from  the  same  WWW 
environment. 

10.  Access  to  the  system  must  be  organized  on  the  basis  of  inlog  data  which  are  used  by  the  system  database  to 
tailor  what  can  be  seen  and  not  seen  to  each  individual.  Also  it  must  be  easy  to  leave  the  system  and  go  to  an 
external  site  on  the  WWW  and  then  return  to  the  system,  without  leaving  the  browser.  Privileges  must  therefore  be 
regulated  at  the  resource-document  level,  as  to  who  has  read  and/or  write  rights  to  any  item  in  the  database.  The 
author  of  an  item  should  be  able  to  decide  him-  or  herself  who  has  rights  to  a submitted  item. 

1 1 . The  system  must  be  efficient  to  maintain,  thus  no  labor-intensive  hand-made  HTML  pages,  but  pages  generated 
dynamically  out  of  a database. 

12.  The  system  must  handle  multimedia  resources  the  same  as  text  resources.  Video  and  audio  must  be  streamed 
over  the  bandwidths  available  to  the  students. 


Realizing  the  requirements:  The  TeleTOP  sytem 

On  the  basis  of  our  prior  experiences,  a detailed  market  research  of  available  course-support  systems,  participation 
in  an  international  evaluation  of  several  such  systems,  we  came  to  the  conclusion  that  none  of  the  commercial 
products  we  studied  met  all  of  our  requirements,  even  those  still  in  the  advertisement  stage.  We  needed  to  proceed 
quickly;  we  realized  our  own  system  that  meets  all  the  requirements  listed  above.  Fig.  1 on  the  next  page  shows  the 
basic  architecture  of  the  system. 

On  the  basis  of  a Web  database  that  is  optimized  for  free-format  data  (because  we  did  not  want  to  decide  in  advance 
what  an  instructor  or  student  may  want  to  include  in  a course  environment)  and  that  can  support  a large  number  of 
simultaneous  users  (a  Domino  server,  4.6),  we  have  created  an  environment  that  from  the  instructor  perspective  is  as 
simple  as  a great  number  of  hierarchially  ordered  Web-based  fill-in  forms.  If  anyone  with  the  login  privileges  of  an 
instructor  inlogs,  he  or  she  sees  not  only  the  site  as  the  students  see  it,  but  also  a number  of  extra  edit-icons  whereby 
on  any  page  the  instructor  can  make  changes  in  the  content  and  even  the  fill-in  forms  themselves,  can  make  changes 
in  text  or  HTML,  can  add  links  to  external  WWW  sites  (and  also  to  other  materials  in  the  database),  and  can  add 
files  as  attachments  to  any  page.  Anything  entered  by  the  instructor  is  placed  on  the  server  and  in  the  database.  This 
concept  is  carried  out  on  all  pages,  for  consistency  and  leamability,  even  the  most-complicated  pages. 

As  an  example,  an  instructor  can  choose  for  a matrix,  or  roster  model  to  indicate  the  organization  of  a course.  Fig.  2 
shows  an  example  of  the  instructor’s  view  of  a roster  page  and  of  the  student’s  view. 
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Figure  1.  System  architecture  for  TeleTOP 


Figure  2.  Instructor's  and  students'  view  of  a roster  page. 
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In  the  edit  view,  the  instructor  can  choose  to  add  new  rows  to  the  roster,  and  can  choose  the  labels  for  each  cell  in 
the  roster.  At  the  moment  that  the  instructor  types  in  a label  on  a previously  empty  cell,  for  example,  a date 
indicating  when  a self-study  assignment  is  due,  a new  page  is  associated  with  this  link,  with  a basic  fill-in-form  and 
an  upload  button,  so  that  the  student  can  submit  his  or  her  assignment  directly  through  this  new  page.  Fig.  3 shows  a 
page  ready  for  the  students  to  submit  an  assignment.  The  students  can  also  decide  if  they  wish  their  assignments  to 
be  readable  by  everyone  in  the  class  or  only  by  the  instructor. 

Students  can  move  through  the  course-support  site  pages,  sometimes  to  submit  assignments  or  ideas  through  the  fill- 
in-form  system,  can  add  files  to  the  site,  and  can  use  the  e-mail  tools  for  general  e-mail  communication  (standard 
Internet  addresses,  sendable  within  the  intranet  or  over  the  Internet). 

Coupled  with  the  database-server  system  is  a video  database,  Medi abase,  on  a Silicon  Graphics  server.  Via  this 
system,  video  segments  in  whatever  form  and  length  and  with  whatever  bandwide  requires  the  user  wishes  can  be 
also  delivered  to  the  TeleTOP  WWW  environments. 


Experiences  with  the  system 

As  the  system  is  in  use  we  are  also  carrying  out  an  extensive  evaluation  study,  led  by  persons  external  to  our  team. 
Students  and  instructors,  again  using  fill-in-forms,  are  being  asked  to  record  the  amount  of  time  they  spend  on 
different  aspects  of  a course  and  of  using  the  course-support  environments  as  well  as  many  other  aspects  relating  to 
the  quality  of  the  course  experience.  The  first  formal  results  will  be  published  in  November  1998  and  the  final 
evaluation  results  in  June  1 999  (in  time  for  presentation  at  ED-MEDIA  ’99).  We  can  say  with  our  results  so  far  that: 
Even  for  instructors  who  were  sceptical  or  showed  resistance  initially  about  changing  in  their  teaching,  the  use  of  the 
system  is  proceeding  without  real  problems.  All  first-year  instructors  are  using  the  system  and  many  are  coming  to 
the  Team  to  ask  for  help  to  try  out  even  more  new  ideas.  The  students,  and  particularly  the  part-time  students 
attending  primarily  at  a distance,  are  not  having  any  particular  start-up  problems  (we  did  all  we  could  to  prevent 
this,  by  providing  them  with  fully  installed  computers  and  modems  at  a low  price,  most  of  which  they  do  not  have  to 
pay  back  if  they  maintain  their  grades).  The  instructors  are  particularly  pleased  with  being  able  to  make  changes  and 
last-minutes  entries  to  their  sites  whenever  they  wish,  even  a few  minutes'  before  a face-to-face  session.  They  also 
appreciate  being  able  to  directly  respond  to  what  occurs  in  the  course,  by  entering  the  site  to  make  changes  or  add 
new  materials  at  any  time.  Compared  to  the  previous  evaluation  data  from  earlier  years  of  using  WWW-based 
environments,  the  students  are  finding  the  environments  easier  to  navigate  and  use.  And,  we  receive  many  requests 
for  demonstrations  for  persons  outside  the  university,  who  typically  are  enthusiastic  about  the  system:  "A  simple  to 
use,  transparent  system  but  one  with  which  you  can  do  anything  you  want  in  a WWW  environment". 


Further  developments: 

Even  though  the  current  system  already  consists  of  a rich  WWW-based  collection  of  learning  environments,  we  still 
have  plans  to  continue  extending  the  system,  particularly  in  terms  of  logistic  management  and  supporting  even  more 
variety  in  instructional  activities.  For  example,  even  though  the  roster  approach  gives  the  instructor  considerable 
freedom  and  a great  many  possibilities  for  different  types  of  student  interactions,  there  are  two  further  developments 
which  we  feel  are  important  to  our  Empowering  the  Good  Instructor  principle: 

1.  First,  we  are  beginning  to  research  how  we  can  use  video  to  increase  the  quality  of  contacts  and  communication 
between  instructor  and  student.  We  can  support  videoconferencing,  but  are  particularly  interested  in  capturing  via 
video  episodes  of  good  communication  during  group  activities  or  in  a discussion  session,  and  integrating  these 
directly  into  the  course  site.  This  can  be  to  the  benefit  of  persons  not  at  the  class  session  (part  of  our  flexibility 
principle),  but  more  creatively,  can  be  used  to  add  the  students'  own  ideas  and  comments,  in  spoken  form  as  well  as 
written  form,  to  the  resources  in  the  course  site  (see  Winnips,  Collis,  & Moonen,  1998,  for  an  example).  We  see  this 
as  a way  to  support  the  Master- Apprentice  idea  that  we  believe  to  be  part  of  good  teaching,  whereby  the  instructor 
and  students  together  can  easily  access  captured  video  for  further  reflection. 
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2.  Secondly,  we  are  working  to  further  develop  another  aspect  of  the  Empower  the  Good  Instructor  idea:  how  to 
help  the  instructor  find  and  use  small  units  of  learning  material  such  as  instructional  applets  within  the  course  site, 
not  as  a replacement  for  him  or  herself  or  the  textbook,  but  as  added  value.  The  collection,  classification,  and  use  of 
"learning  molecules"  from  via  the  roster  or  anywhere  else  in  the  course  site,  orchestrated  by  the  instructor  (or 
delegated  by  the  instructor  to  the  students)  is  another  area  we  are  working  on. 

For  more  information,  see  http://teletop.edte.utwente.nl 
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Abstract:  The  U.S.  military’s  emphasis  on  night  operations  has  made  the  development  of 
night  vision  goggle  (NVG)  training  increasingly  important.  The  Warfighter  Training  Research 
Division  of  the  Air  Force  Research  Laboratory  (AFRL/HEA)  has  been  developing  and 
distributing  such  training  since  1990.  This  paper  documents  the  development,  evolution, 
administration,  and  possible  future  of  multimedia-based  NVG  training  courseware. 

A videodisc  version  of  the  current  NVG  Training  Course  contains  numerous  motion 
video  segments  demonstrating  various  NVG  effects.  The  videodisc  provides  high  quality  video 
and  random  access  to  various  course  elements.  However,  the  videodisc  cannot  be  revised 

without  substantial  effort  and  cost.  Established  digital  video  formats  such  as  Video  for 

Windows,  QuickTime,  and  MPEG-1  do  not  provide  the  level  of  resolution  necessary  to 
adequately  depict  intensified  imagery  visual  teaching  points.  Emerging  digital  video 

technologies  such  as  MPEG-2  and  DVD  may  be  of  sufficient  quality  to  allow  for  distribution  of 
the  course  in  an  all-digital  format.  Testing  and  implementation  of  these  newer  technologies  for 
converting  the  course  from  analog  to  digital  media  is  discussed. 

The  NVG  Training  Course  is  taught  at  numerous,  dispersed  facilities. 

Telecommunication  technologies  that  are  or  could  be  used  for  course  administration  such  as 
courseware  distribution,  user  databases,  and  feedback  on  course  content  are  addressed. 


Introduction 

Night  vision  goggles  (NVGs)  have  been  used  by  the  United  States  Armed  Forces  since  the  1940’s. 
The  technology  has  vastly  improved  since  the  first  devices,  using  infrared  illuminators  and  detectors,  were 
mounted  to  sniper’s  rifles  in  World  War  II.  Some  current  third  generation  NVGs  have  such  improved 
resolution  that  one  can  see  20/20  under  optimum  conditions. 

The  increased  use  of  NVGs  by  the  U.S.  Air  Force  (USAF),  U.S.  Navy  (USN),  and  U.S.  Marine  Corps 
(USMC)  aviation  communities  created  an  expanded  need  for  NVG  training.  The  Warfighter  Training 
Research  Division  of  the  Air  Force  Research  Laboratory  (AFRL/HEA)  has  been  developing  and  distributing 
such  training  since  1990.  One  product,  the  Night  Vision  Goggle  Training  Course,  is  the  initial  curriculum  for 
USN,  USMC,  and  USAF  aircrew  who  fly  missions  with  NVGs.  The  NVG  Training  Course  is  not  specific  to 
any  one  aircraft  type  as  it  addresses  training  objectives  common  to  most  tanker,  transport,  helicopter,  fighter, 
and  attack  aircraft. 

There  is  a standardized  configuration  for  the  physical  layout  of  facilities  used  for  NVG  training  in  the 
United  States.  This  configuration  is  known  as  a “NITE  Lab”  and  can  be  housed  in  existing  buildings, 
doublewide  trailers,  or  new  buildings.  A large  portion  of  the  NITE  Lab  consists  of  the  classroom.  The 
classroom  is  normally  configured  to  handle  up  to  15  students.  Multimedia  courseware  used  in  the  training  and 
the  equipment  necessary  to  present  the  courseware  are  located  in  the  classroom.  A terrain  board  is  also  part  of 
the  facility.  These  boards  are  typically  eight  to  ten  feet  square.  The  terrain  board  depicts  a variety  of  terrain 
including  an  airfield,  landing  zones,  cultural  features,  water,  mountains,  farmlands,  desert,  and  forests.  A light 
source  is  used  to  provide  light  levels  ranging  from  overcast  starlight  to  full  moon.  The  light  source  can  also  be 
adjusted  to  emulate  different  moon  angles.  Students,  viewing  the  board  through  NVGs,  stand  around  the 
terrain  board  and  observe  various  phenomena  as  presented  and  described  by  the  instructor.  Another  important 
element  of  the  NITE  Lab  is  the  eye  lane.  The  eye  lane  is  a light-tight  room  equipped  with  resolution  charts  and 
an  illumination  device.  The  eye  lane  is  used  to  teach  aircrew  proper  NVG  adjustment  and  assessment 
procedures. 
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This  paper  documents  the  development,  evolution,  administration,  and  possible  future  of  multimedia- 
based  courseware  used  for  NVG  training  in  NITE  Labs  and  other  facilities. 


Multimedia  Technologies  for  NVG  Training 

Video  examples  of  NVG  effects  as  they  happen  in  the  real  world  are  an  excellent  teaching  tool.  Such 
examples  are  shown  to  the  students  to  make  them  aware  of  various  NVG  phenomena  and  the  conditions  under 
which  they  occur.  Therefore,  intensified  motion  video  segments  are  an  integral  part  of  all  NVG  training 
courseware. 

Six  video  programs  were  initially  produced  to  support  NVG  training  (Joralmon,  1992).  These 
videotapes  were  used  for  both  primary  and  refresher  NVG  training.  Each  program  is  approximately  ten 
minutes  in  length  and  covers  a different  topic.  The  six  video  programs  are  entitled  NVG  Image 
Characteristics,  Luminance  Variations,  Lighting  Issues,  Terrain  Albedos,  Weather  Effects,  and  Navigational 
Issues. 

The  six  videotape  programs  were  replaced  by  the  first  version  of  the  NVG  Training  Course  published 
in  1994.  This  courseware  is  presented  via  an  interactive  videodisc.  The  videodisc  is  designed  to  be  a 
multimedia-based  instructor  aid.  All  USAF,  USN,  and  USMC  NITE  Labs  are  equipped  with  videodisc 
players.  All  of  the  graphic  slides  and  accompanying  motion  video  segments  for  the  entire  course  are  contained 
on  a single  two-sided  disc.  The  videodisc  uses  Level  II  programming  which  allows  the  instructor  to  navigate 
through  the  various  lectures.  A Level  II  interactive  videodisc  contains  all  of  the  navigational  programming  on 
the  second  audio  channel.  Each  time  the  disc  is  loaded  into  a compatible  player,  the  navigation  program  is 
loaded  into  the  player’s  on-board  microprocessor  thus  allowing  the  courseware  to  be  delivered.  The 
presentation  is  presented  to  the  class  either  on  a large  television  monitor  or  on  a screen  via  a video  projector. 
The  videodisc  contains  six  academic  lectures  and  three  NVG  adjustment  and  assessment  lectures.  The 
academic  lectures  are  Introduction  to  the  Course,  The  Human  Visual  System,  Introduction  to  Night  Vision 
Goggles,  The  Night  Environment,  Misperceptions  and  Illusions  (specific  to  fixed  wing  and  rotary  wing 
aircraft),  and  Night  Operations.  The  adjustment  and  assessment  lectures  cover  procedures  for  AN/AVS-6, 
MXU-810U  CATSEYE,  and  AN/AVS-9  NVGs.  A printed  instructor’s  guide  is  packaged  with  the  interactive 
videodisc.  The  instructor’s  guide  contains  pertinent  administrative  information  as  well  as  the  main  course 
content  to  be  delivered  by  the  instructor. 

An  alternate  way  to  deliver  the  initial  version  NVG  Training  Course  was  produced  in  order  to  provide 
the  courseware  to  those  units  that  might  not  have  a compatible  videodisc  player.  The  alternate  method  uses 
35mm  slides  as  a storage  medium  for  the  graphic  slides  and  a VHS  videotape  contains  the  motion  video 
segments.  Graphic  slides  with  an  accompanying  motion  segment  have  a prompt  telling  the  instructor  to  play 
the  appropriate  motion  video  segment.  The  graphic  slides  can  also  be  delivered  via  a number  of  different 
computer  presentation  programs  (e.g.  Microsoft  PowerPoint).  These  alternate  delivery  methods  virtually 
ensure  that  any  unit  can  present  the  courseware  to  students  even  though  there  is  a loss  in  image  quality  when 
presenting  the  motion  video  segments  on  VHS  videotape. 

Version  2 of  the  NVG  Training  Course  was  completed  in  early  1998.  Again,  the  multimedia  portion 
of  the  courseware  is  packaged  on  a Level  II  interactive  videodisc.  Revisions  to  course  content  were  made 
based  on  instructor’s  recommendations.  A printed  instructor’s  guide  continues  to  accompany  the  interactive 
videodisc.  The  alternate  delivery  method  uses  Microsoft  (MS)  PowerPoint  for  presentation  of  the  graphic  still 
frames  and  VHS  videotape  for  the  motion  segments.  35mm  slides  of  the  still  frames  are  no  longer  produced. 
A CD-ROM  has  been  produced  which  contains  the  MS  PowerPoint  files,  the  instructor’s  guide  (in  Adobe 
Acrobat  .pdf  format),  and  other  ancillary  information. 

This  next  section  of  the  paper  discusses  multimedia  technologies  and  how  they  might  or  might  not  be 
suitable  for  NVG  courseware  delivery.  Current  technologies  will  be  discussed  first,  followed  by  an 
examination  of  emerging  technologies.  For  purposes  of  this  discussion,  current  technologies  are  defined  as 
those  that  are  prevalent  in  society  (e.g.  computers,  CD-ROM  drives,  and  videocassette  recorders)  and  emerging 
technologies  are  those  that  have  been  recently  marketed  but  have  not  reached  the  general  public  in  large 
numbers  (e.g.  DVD  drives,  digital  videocassette  recorders,  and  advanced  video  compression  algorithms). 


389 


Current  Multimedia  Technologies 

Laser  videodisc  was  chosen  as  the  primary  distribution  format  due  to  its  superior  video  resolution  and 
random  access  capabilities.  The  resolution  of  a videodisc  is  much  better  than  that  of  a VHS  format  videotape. 
This  is  a critical  issue  in  the  production  of  night  vision  aircrew  training  multimedia.  Video  recorded  with 
image  intensified  cameras  has  significantly  lower  resolution  than  what  can  actually  be  seen  through  an  NVG. 
Therefore,  it  is  important  to  first  record  the  intensified  image  on  a high-quality  videotape  format  and  then 
minimize  the  number  of  generations  each  scene  gathers  when  it  is  being  edited  into  a final  production.  A 
generation  occurs  each  time  an  analog  videotape  is  copied  and  results  in  some  loss  of  quality. 

The  disadvantages  of  laser  videodiscs  are  their  high  cost  to  produce  and  the  inability  to  easily  and 
quickly  update  the  disc  once  it  has  been  mastered  and  replicated.  This  is  a significant  problem  since  NVG 
technology  and  employment  techniques  are  in  a constant  state  of  evolution.  The  need  exists  to  be  able  to 
readily  revise  NVG  Training  Course  materials.  The  challenge  to  date  has  been  how  to  provide  high  quality 
revisions  in  a timely  manner  and  at  a reasonable  cost. 

The  biggest  hurdle  in  transforming  the  NVG  Training  Course  into  an  all-digital  format  is  digitizing 
the  motion  video  segments. 

Digital  video  formats  available  for  use  in  computer-based  training  (CBT)  and  Internet  (WWW) 
applications  can  be  visually  intriguing  and  of  high  quality.  However,  the  demanding  digital  storage  and 
processing  requirements  for  full  screen,  full  motion  video  can  result  in  compromised  image  resolution.  Digital 
motion  video  saved  in  a computer  environment  can  also  require  enormous  amounts  of  storage  space. 
Hardware  and  software  solutions  are  available  to  transform  motion  video  segments  into  manageable  file  sizes. 
However,  this  transformation  process  often  reduces  the  resolution  of  the  motion  video  segment.  How  much 
resolution  can  be  lost  without  negatively  impacting  the  effectiveness  of  the  presentation  depends  on  the 
specific  application.  Computer  playback  of  motion  video  is  also  demanding.  Again,  available  hardware  and 
software  solutions  are  used  to  display  the  video.  The  quality  of  software-based  playback  is  dependent  on  and 
limited  by  computer  processing  power.  Hardware  playback  methods  often  include  enough  additional 
processing  power  such  that  full  motion,  full  screen  playback  is  possible. 

How  good  digitized  video  needs  to  be  is  one  consideration  the  courseware  developer  or  educator 
should  ascertain  prior  to  using  such  presentation  technologies.  What  makes  video  quality  acceptable  is 
dependent  on  the  educational  objective  or  content  the  video  is  supporting.  In  some  applications,  a low  quality 
"talking  head"  scene  playing  back  at  lOfps  may  be  sufficient.  Low  or  medium  quality  digital  video  playing 
back  at  low  frame  rates  may  also  be  sufficient  to  show  such  items  as  control  panels  or  displays.  Other  cases, 
such  as  those  in  which  the  viewer  must  clearly  see  visual  details  within  the  image  (e.g.  night  vision  aircrew 
training),  require  a higher-quality  digital  video  format. 

Current  digital  video  formats  are  not  of  sufficient  quality  to  support  NVG  training.  Apple  QuickTime 
(.mov  files)  and  MS  Video  for  Windows  (.avi  files)  typically  play  motion  segments  back  at  15  frames  per 
second.  The  video  is  displayed  in  a small  window  on  a computer  screen.  Increasing  the  size  of  the  window 
can  result  in  the  image  appearing  quite  pixelated.  Video  streams  using  the  MPEG-1  compression  standard  play 
at  30  frames  per  second  but  again  in  a small,  default-sized  window  of  352  by  240  pixels  within  the  computer 
screen.  MPEG-1  can  be  displayed  full  screen  on  a regular  television  monitor,  but  this  requires  using  two 
screens  for  presenting  course  content.  One  screen  displaying  the  graphic  still  frames  and  another  displaying 
the  MPEG-1  motion  video  segments. 


Emerging  Multimedia  Technologies 

Recently  emerging  digital  video  formats  and  computer  media  show  promise  that  they  are  suitable  for 
use  in  presenting  NVG  training  courseware.  Two  of  these  technologies  being  tested  by  AL/HRA  are  MPEG-2 
video  compression  and  DVD  media. 

One  of  the  most  advanced  digital  video  compression  algorithms  is  the  MPEG-2  standard.  Full  D-1 
MPEG-2  video  files  play  back  at  thirty  frames  per  second  on  a computer  screen  in  a native  size  window 
measuring  720  by  480  pixels.  The  quality  is  comparable  to  (if  not  better  than)  analog  laser  videodiscs. 
Interactivity  and  random  access  capabilities  are  possible  using  a variety  of  CBT  authoring,  WWW  page 
development  and  computer  programming  techniques.  MPEG-2  video  files  can  be  encoded  using  various 
software  and,  ideally,  hardware  solutions.  The  file  sizes  of  MPEG-2  motion  segments  are  large,  but  the  ever 
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increasing  storage  capabilities  of  desktop  and  portable  computers  as  well  as  interchangeable  storage  media  is 
sufficient  to  handle  this  large  amount  of  data. 

The  MPEG-2  standard  is  currently  being  used  in  DVD’  movie  titles  and  telecommunications 
applications  for  the  transmission  of  broadcast  quality  audio  and  video.  DVDs  are  the  same  size  as  compact 
discs  (CD),  but  can  store  much  more  data.  A data  CD  has  the  capacity  of  650  megabytes  (MB)  while  DVDs 
can  hold  anywhere  from  just  below  4 gigabytes  (GB)  to  approximately  17GB.  These  capacities  are  sufficient 
to  hold  the  entire  NVG  Training  Course  with  the  motion  video  segments  encoded  in  the  MPEG-2  video 
standard. 

NITE  Labs  and  any  other  site  desiring  to  use  an  all-digital  version  of  the  NVG  Training  Course  would 
require  additional  hardware  to  support  DVD  and  MPEG-2  technologies.  DVD  requires  a computer  drive  be 
placed  into  the  computer  supporting  multimedia  playback  in  the  classroom.  The  price  for  a DVD  drive  is 
approximately  $300.  A DVD  drive  typically  replaces  a computer’s  CD  drive  since  the  DVD  drive  can  read 
both  DVDs  and  CDs.  Playback  of  MPEG-2  files  is  best  accomplished  with  an  additional  hardware  board 
placed  into  the  multimedia  computer.  The  price  for  an  MPEG-2  hardware  decoder  is  about  $250. 

Testing  of  MPEG-2  and  DVD  technology  is  currently  underway  at  AL/HRA.  Prototype,  all-digital 
versions  of  the  NVG  Training  Course  will  be  produced,  demonstrated^  and  evaluated.  Initial  test  versions  will 
use  MS  PowerPoint  to  present  the  graphic  still  frames  and  the  motion  video  segments.  Those  graphic  still 
frames  that  have  an  accompanying  motion  video  segment  (not  all  do)  will  contain  an  element  that  allows  the 
instructor  to  play  the  MPEG-2  video  file.  An  example  of  such  a still  frame  is  displayed  in  Figure  1 . 


Figure  1.  An  example  of  an  MS  PowerPoint  slide  that  contains  an  MPEG-2  format  video  segment. 

The  presenter  clicks  on  the  gray  caption  bar  at  the  bottom  of  the  photograph  to  begin  playing  the  video 
segment.  The  gray  bar  is  transformed  into  a video  playback  control  bar  and  the  area  that  was  previously 


‘The  term  DVD  is  becoming  the  standard  form  of  referring  to  this  specific  storage  medium.  The  term  was 
chosen  to  minimize  confusion  and  ensure  commonality  between  those  who  thought  DVD  meant  digital  video 
disc  and  those  who  thought  DVD  meant  digital  versatile  disc. 

^A  demonstration  of  the  digital  version  of  the  NVG  Training  Course  (using  MPEG-2  video  files  and  DVD  as 
the  storage  medium)  will  be  made  during  the  presentation  of  this  paper. 
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occupied  by  the  photograph  is  replaced  by  the  motion  video.  The  photograph  and  the  gray  caption  bar  return 
when  the  video  segment  ends.  The  presenter  then  has  the  option  of  playing  the  video  segment  again  or  moving 
to  another  slide  in  the  presentation.  Applications  that  allow  the  course  to  be  presented  using  a WWW  browser 
program  will  also  be  tested.  There  are  no  current  plans  to  serve  the  NVG  Training  Course  over  the  Internet 
since  bandwidth  limitations  would  significantly  impede  downloading  of  the  large  video  segment  files.  Instead, 
if  a WWW  browser  program  were  used  for  presentation,  the  course  would  be  distributed  in  a format  that  would 
allow  playback  through  the  browser  program  via  the  host  computer. 


NVG  Training  Course  Administration 

The  various  NITE  Labs  are  located  throughout  the  United  States  and  abroad.  This  presents,  to  course 
developers,  the  challenge  of  administering  the  course  over  distance.  The  NVG  Training  course  has  been  taught 
since  September  of  1994  and  several  trends  have  emerged  relative  to  its  use.  One  such  trend  is  that  some 
instructors  modify  the  course  to  fit  their  specific  training  needs.  These  modifications  take  the  form  of  either 
deleting  course  content  or  supplementing  the  course  with  additional  material.  The  ability  to  modify  a course  to 
suit  a specific  training  need  is  one  reason  why  having  the  course  in  a digital  format  is  desirable.  A digital 
format  allows  instructors  to  pick  and  choose  or  cut  and  paste  what  best  suits  the  training  requirements. 
However,  course  administrators  should  be  aware  of  how  the  course  is  being  altered  lest  any  essential 
information  be  deleted  or  erroneous  information  be  added.  Both  the  deletion  of  critical  information  and  the 
dissemination  of  false  information  could  result  in  safety  of  flight  issues.  Therefore,  investigating  and  reporting 
on  course  administration  over  distance  issues  could  lead  to  recommendations  suited  to  alleviating  any  potential 
problems  and  lead  to  better  communication  amongst  those  involved  in  NVG  training. 

Those  involved  in  administering  the  NVG  Training  Course,  AL/HRA  for  the  USAF  and  the  NITE 
Lab  Model  Manager  (located  at  MAWTS-1  in  Yuma,  Arizona)  for  the  USN  and  USMC,  handle  a variety  of 
course  administration-type  subjects  including  course  material  distribution,  NITE  Lab  hardware  acquisition  and 
maintenance,  and  instructor  certification.  All  of  these  tasks  are  typically  dealt  with  using  some  form  of 
telecommunications  (e.g.  the  telephone,  e-mail)  since  the  parties  are  geographically  separated.  Currently,  there 
is  no  formal  communication  forum  between  course  administrators  and  course  instructors.  Communication  is 
primarily  one-to-one  with  no  means  for  one-to-many  or  group  interaction.  All  involved  would  seemingly 
benefit  if  some  system  were  in  place  that  would  allow  for  the  easy  exchange  of  information  as  it  pertains  to  the 
NVG  Training  Course.  The  following  are  some  efforts  underway  and  recommendations  that  could  assist  in  the 
administration  aspects  of  the  NVG  Training  Course. 


Recommendations 

An  effort  is  being  made  to  establish  a comprehensive  database  that  lists  all  current  NVG  instructors 
and  their  respective  point-of  contact  information.  This  information  includes  a description  of  their  teaching 
facilities  (including  telecommunications  capabilities),  the  courseware  they  are  using,  and  the  type  of  NVGs  and 
aircraft  used  at  their  location.  Two  software  programs  are  being  used  to  build  the  database.  MS  Access  is  one 
and  FileMaker  Pro  4.0  is  the  other.  It  is  hoped  that  upon  completion  the  database  can  be  distributed  and/or 
shared  electronically.  When  the  database  is  complete,  it  will  be  important  to  instill  responsibility  into  the 
instructors  to  make  sure  they  keep  course  administrators  apprised  of  any  changes,  especially  any  changes  to  the 
course  content  and  any  instructor’s  changes  in  duty  assignments. 

At  a minimum  and  as  another  initial  step,  an  electronic  mail  broadcast  group  of  instructors  and 
administrators  could  be  established.  This  would  allow  asynchronous  communication  within  the  group  in  the 
one-to-one  and  the  one-to-many  formats.  An  electronic  mail  group  would  also  allow  for  attached  files  (e.g. 
courseware  updates)  to  be  transmitted. 

NVG  Training  Course  administration  would  seemingly  benefit  and  improve  if  some  form  of 
computer-mediated  communication  (CMC)  were  implemented.  A dedicated  system,  such  as  the  FirstClass 
CMC  software  by  Soft  Arc  Incorporated  or  Lotus  Notes,  would  provide  a centralized  system  of  communication 
for  both  administrators  and  instructors.  Discussion  threads  could  take  place  on  a number  of  subjects  such  as 
NVG  experiences  (lessons  learned),  listing  of  course  errata,  technical  updates  and  course  materials  distribution. 
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An  Internet  application  could  also  be  used  to  improve  NVG  Training  Course  administration  and  might 
possibly  be  the  best  solution.  Current  information  could  be  posted,  message  boards  and  discussion  groups 
established  (though  the  author  has  found  such  web-based  applications  less  than  efficient),  and  the  ability  to 
download  files  included.  A File  Transfer  Protocol  (FTP)  site  could  also  be  brought  on-line.  Security  issues 
could  be  addressed  by  using  Defense  Information  Security  Agency  (DISA)  networks  like  Unclassified  (but 
sensitive)  Internet  Protocol  Routing  Network  (NIPRNET)  and  Secret  Internet  Protocol  Routing  Network 
(SIPRNET).  NIPRNET  would  be  the  appropriate  network  to  use  for  NVG  training  applications  since  (at  the 
time  of  this  writing)  none  of  the  information  is  classified.  These  networks  provide  certain  degrees  of 
controlled  access  to  one’s  site  and  thus  provide  protection  to  sensitive  and  proprietary  information. 

Sophisticated  messaging  systems  are  being  demonstrated  that  could  be  used  for  NVG  Training  Course 
administration  (Bryant,  1996).  The  U.S.  Navy  X.400  defense  messaging  system  capability  allows  users  “to 
include  all  types  of  media  (imagery,  video,  slide  shows,  etc.)  as  attachments  to  standard  navy  messages  (p.  1). 
The  author  does  not  know  the  suitability,  interservice  availability,  and  full  specifications  of  the  X.400 
messaging  capability  at  this  time  but  warrant  further  investigation. 


Conclusion 

This  paper  has  discussed  multimedia  technologies  that  can  be  used  to  deliver  aircrew  training.  Such 
papers  serve  as  a benchmark  in  the  timeline  of  technology  development  and  implementation.  It  is  interesting 
to  look  back  at  past  conference  proceedings,  see  what  media  technologies  were  being  discussed,  and  now 
reflect  on  the  degree  of  success  those  technologies  achieved.  This  paper  has  also  discussed  course 
administration  issues  that  are  present  when  a course  must  be  managed  over  distance.  Future  testing  and 
implementation  of  various  recommended  technologies  will  hopefully  improve  upon  the  quality  of  continued 
night  vision  goggle  aircrew  training. 


References 

Bryant,  K.  (1996).  Navy  Demonstrates  X.400  Tactical  Messaging  Capability  During  JWID-95.  [On-line], 
Available:  http ://w ww.chips.navv.mil/chips/archives/96Jan/file9.htm 

Joralmon,  D.  Q.,  & Antonio,  J.C.  (1992).  Night  Vision  Goggle  Training:  Development  and  Production  of  Six 
Video  Programs.  (AL-TR- 1992-0 136).  Brooks  Air  Force  Base,  TX. 


393 


Designing  Instructional  Multimedia  Applications: 
Key  Practices  and  Design  Patterns 


Maia  Dimitrova  and  Alistair  Sutcliffe 

Centre  for  HCI  Design,  City  University,  Northampton  Square,  London  EC  IV  OHB,  U.K. 
M.T.Dimitrova@city.ac.uk,  A.G.Sutclifife@city.ac.uk 


Abstract:  This  paper  is  divided  in  two  parts.  The  first  part  presents  a study  of  the  existing 
design  practice  in  instructional  multimedia  organisations,  and  describes  the  stages  of  the 
design  process.  It  reveals  the  major  problems  in  design,  which  include  complexity  of  the 
design  process,  which  necessitates  the  involvement  of  a number  of  specialists  from 
different  professional  areas,  who  need  to  communicate  on  a common  ground,  and 
the  tight  delivery  deadlines  against  which  designers  have  to  compete.  In  part  two,  an 
approach  to  supporting  the  design  process  is  proposed.  In  involves  reusing  previous 
proven  design  expertise  through  the  use  of  design  patterns. 

1.  Introduction 

The  effective  design  of  instructional  multimedia  applications  is  an  intersection  of  different  scientific  fields 
including  pedagogy,  cognitive  psychology,  computer  science,  and  media  design.  The  design  of  such  applications 
is,  therefore,  a complex  process  that  needs  to  take  into  consideration  a number  of  interdependent  factors.  In  a 
learner-centred  approach  to  designing  instructional  multimedia,  the  primary  focus  should  be  on  the  learner  and 
their  existing  knowledge  and  experience,  as  well  as  the  learner’s  pre-determined  learning  style,  and  individual 
approach  to  learning  (Wild  and  Quinn  1998).  The  characteristics  of  the  learning  material,  and  the  tasks  to  be 
performed  during  learning  also  affect  learning  from  multimedia  (Najjar  1997).  Thus,  the  media  resources  that 
satisfy  the  information  requirements  for  each  task  must  be  carefully  chosen  while  designing  the  presentation 
(Sutcliffe  & Faraday  1994).  To  aid  designers  deal  with  the  complexity  of  developing  such  applications,  sound 
methodological  support  and  guidance  is  required.  In  order  to  improve  the  design  process,  an  understanding  of  the 
current  approach  to  design  is  necessary.  In  the  literature,  a number  of  experiences  of  designing  multimedia 
courseware  have  been  reported,  for  instance  the  WinEcon  (Soper  1997).  These  reports  outline  main  design  issues, 
and  how  multiple  media  was  used  to  address  the  educational  needs  of  that  particular  applications.  Liu  et  al.  (1997) 
also  studied  the  multimedia  production  process.  However,  they  emphasised  mainly  on  the  overall  development 
process,  and  do  not  provide  details  about  the  design  process.  There  is  a need  for  more  comprehensive  studies  of 
instructional  multimedia  design  practices  to  inform  researches  of  the  design  techniques  used,  and  the  major 
problems  designers  encounter. 

This  paper  presents  a study  of  the  current  design  practice  in  instructional  multimedia  organisations.  Section  2 
introduces  the  designers  and  the  projects  examined.  The  results  of  the  study  are  revealed  in  Section  3.  A 
prototypical  approach  to  improving  the  presentation  design  process  through  the  reuse  of  previous  design  rationale 
is  suggested  in  Section  4.  Some  example  design  patterns  are  also  presented. 

2.  Study  Background 

2.1  Multimedia  Designers 

Eight  designers  from  four  educational  multimedia  organisations  within  the  U.K.  participated  in  the  study.  The 
instructional  designers  came  from  a variety  of  professional  backgrounds,  the  majority  of  who  had  a mixture  of 
qualifications,  including  fine  arts  and  social  sciences,  electronics,  electrical,  telecommunications,  sound  and  video 
engineering.  Half  of  them  also  had  a qualification  in  education  or  training.  The  designers’  experiences  with 
multimedia  ranged  from  two  to  sixteen  years,  with  a mean  of  five  years. 

The  interview  questions  were  divided  in  five  main  areas  covering  designers’  experience  on  a recent  multimedia 
project,  the  approach  adopted  to  the  design,  the  way  of  selecting  and  integrating  representation  media,  any  style 
guidelines  applied,  and  finally  the  main  problems  encountered  during  the  design  process. 

2.2  Design  Projects 
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The  projects  represented  a variety  of  application  areas,  which  according  to  their  learning  objectives  can  be 
classified  into  two  categories:  training  (including  Captain  Emergency  Training,  Call  Handling,  and  Hotel 
Induction),  and  educational  (including  Innovation  Process  Management,  Oil  Lubrication,  and  Key  Stage  2 Maths). 
In  the  training  applications  there  was  a well-defined  set  of  learning  objectives  and  skills  that  the  learner  had  to 
acquire.  The  aim  was  not  so  much  at  teaching  new  knowledge  in  a subject  matter,  but  at  acquiring  skills  and 
applying  knowledge.  In  contrast,  the  educational  applications  aimed  to  develop  a deeper  understanding  in  a 
subject  matter. 

3.  Study  Results 


3.1  Design  Process 


Although  the  design  methods  varied  across  companies  and  individual  projects,  a ‘waterfall’  approach  was 
adopted,  including  four  main  stages:  Specification,  Conceptual  Design,  Prototyping  and  Presentation  Design, 
which  are  illustrated  on  (Fig.  1). 


Figure  1 : Instructional  Multimedia  Design  Process 


3.1,1  Specification 

Every  project  commenced  with  setting  up  the  aims  and  objectives  of  the  courseware.  This  was  done  in  an 
iterative  process  with  the  active  participation  of  a subject  matter  expert  from  the  client  company,  or  qualified 
teachers.  To  be  effective,  the  training  material  has  to  be  defined  in  the  context  of  the  learner  population.  However, 
only  in  half  of  the  cases  the  target  user  population  was  explicitly  studied  to  identify  their  educational  or 
professional  background  (3),  computer  literacy  (2),  prior  domain  knowledge  (2),  age  (2),  and  job  specification  (2). 
There  was  no  evidence  that  the  individual  learning  styles  of  the  target  population  were  formally  established. 


3.1.2  Conceptual  Design 

During  the  Conceptual  Design  stage  the  courseware  was  specified  in  detail  and  the  overall  presentation  style 
was  determined.  In  five  cases,  every  single  training  point  was  defined,  including  the  material  to  be  taught  to  the 
learner,  and  the  interactions  they  had  to  perform.  The  content  was  divided  into  sections,  which  were  structured  in 
a logical  way.  In  the  other  three  projects,  as  the  content  was  based  on  existing  material,  the  content  was  already 
defined  and  structured.  The  subject  matter  expert  was  highly  involved  in  this  iterative  process. 


The  learning  objectives  appear  to  have  a strong  influence  on  the  choice  of  representation  style  adopted. 
According  to  the  overall  representation  approach,  the  projects  can  be  divided  into  tutorial,  simulation,  and  game, 
as  shown  on  (Tab.  1).  Apart  from  the  training  objectives,  other  factors  reported  to  influence  the  choice  of 
representation  style  were  the  target  learner  population,  and  the  budget  available. 


Table  1:  Representation  Styles  According  to  Learning  Objectives 


In  the  tutorial  applications,  as  the  primary  aim  was  to  provide  knowledge  and  make  the  learner  reflect  on  it, 
the  material  was  structured  into  topics  and  subtopics.  At  the  end  of  each  section  the  learner  was  given  the 
opportunity  to  reflect  on  what  was  learned,  by  answering  test  questions,  or  engaging  in  activities.  Simulation  was 
selected  to  allow  the  learner  to  develop  skills  and  strategies  for  dealing  with  certain  situations.  That  was  achieved 
by  allowing  the  learner  to  interact  with  a realistic  environment,  giving  them  an  opportunity  to  actively  explore  it 
(e.g.  browse  bridge  of  a ship),  and  interact  with  it  in  order  to  achieve  certain  goals  (e.g.  fight  fire  on  board  of  a 
ship).  In  the  Captain  Emergency  Training  application  the  situation  evolved  dynamically  depending  on  the  user’s 
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decisions  and  actions,  who  then  got  to  experience  the  consequences  of  those  actions  (e.g.  decrease  or  increase 
danger  of  fire),  thus  providing  learning  by  discovery.  Assessment  was  based  on  effectively  completing  the  tasks. 
At  the  end  the  learner’s  performance  was  assessed  against  company’s  standards,  and  they  were  provided  with 
feedback  as  to  how  they  could  have  reacted  better.  A game-like  approach  was  chosen  to  encourage  children  to 
learn  mathematics  by  engaging  in  different  problem-solving  activities.  The  system  monitored  student’s  problem- 
solving attempts,  and  provided  them  with  an  immediate  feedback  on  their  performance,  with  an  option  to  look  at 
help  screens  which  provide  deeper  knowledge  of  the  subject  matter. 

3.1.3  Prototyping 

In  all  cases  some  kind  of  prototype  was  developed,  which  was  presented  to  users  for  their  feedback.  The 
content  and  the  level  of  detail  included  in  the  prototype  varied  from  sample  screens,  images  and  artwork  (4), 
through  screens  including  partial  functionality  (1),  to  a pilot  of  a full  module  (3).  The  way  the  prototypes  were 
evaluated  also  varied.  Two  of  the  screen  prototypes  were  presented  to  some  users  for  their  comments  and 
suggestions.  In  the  latter  four  cases,  the  evaluation  included  observing  users’  actions  to  discover  problem  areas,  as 
well  as  getting  their  feedback.  As  the  evaluation  was  not  done  in  a systematic  way,  the  user  feedback  consisted 
predominantly  of  high-level  comments  regarding  the  colour  scheme,  or  the  look  of  the  graphics.  Only  the  pilot 
prototype  resulted  in  clarifying  functionality,  verifying  the  accuracy  of  procedures  to  be  implemented,  as  well  as 
adding  usability  features. 

3.1.4  Presentation  Design 

In  the  Presentation  Design  stage  the  abstractions  of  the  Conceptual  Design  were  mapped  onto  a real 
multimedia  environment.  In  five  projects  scripting  technique  was  used  for  designing  the  applications.  However, 
the  amount  of  details  included  in  the  scripts  varied  across  projects.  In  three  cases  every  single  screen  was  defined, 
including  a schematic  layout  of  the  screen,  detailed  description  of  the  graphics  and  video  sequences,  as  well  as 
precise  scripts  of  speech  track  and  text.  The  relative  position  of  media  resources  and  their  synchronisation  were 
also  specified  wherever  appropriate.  Navigation  controls  were  defined,  as  well  as  branching  from  one  screen  to 
other  screens.  In  the  rest  two  cases  partial  scripts  were  produced,  specifying  wording  on  screen,  screen  layout, 
questions  to  be  presented,  as  well  as  branching  to  other  screens.  In  the  two  full  simulation  cases  a different 
approach  to  presentation  design  was  adopted.  As  the  material  consisted  of  separate  training  exercises,  each  of 
which  had  different  actions  to  be  performed,  first  a transition  state  diagram  was  created  for  every  possible  action. 
After  that  the  screens  for  each  exercise  were  designed. 

Media  Selection 

The  designers  were  asked  to  define  the  reasons  for  selecting  different  media  resources  used  in  the  applications. 
The  answers  showed  that  the  factors  considered  in  selecting  the  most  appropriate  media  were  high-level,  and  not 
based  on  sound  pedagogical  or  cognitive  psychology  principles.  Some  factors  considered  were  the  learners’ 
characteristics  (e.g.  reading  abilities,  computer  literacy),  and  the  type  of  information  required.  Other  factors 
influencing  the  choice  of  media  were  client’s  preferences  or  expectations,  limited  memory  space  on  delivery 
platforms,  and  limited  budget.  Some  media  selection  heuristics  were: 

■ Use  speech  as  a primary  instruction  medium,  because  it  is  motivating  to  users. 

■ If  the  users  are  not  very  computer  literate,  use  speech  for  instructions. 

■ Choose  animation  to  represent  concrete  concepts  and  explain  how  they  work. 

■ Select  video  for  representing  customer  service  and  customer  care  skills,  as  the  learner  can  observe  reactions 
and  body  language. 

Media  Combination 

For  the  user  to  extract  the  most  important  information  from  a presentation,  it  is  vital  to  guide  their  reading 
sequence,  as  well  as  to  aid  them  in  establishing  links  between  the  information  presented  with  different  media. 
From  the  study,  we  discovered  that  very  often  the  decision  as  to  what  media  to  combine  and  how  was  left  to 
designer’s  creativity  and  aesthetic  judgement,  and  no  sound  principles  of  human  attention  to  various  media  were 
considered.  It  appears  that  there  were  four  main  reasons  for  presenting  different  media  concurrently,  which  are: 

■ Support:  When  two  different  media  convey  related  information,  where  one  or  the  other  medium  can  present 
additional  information  to  the  other  resource.  For  example,  display  an  image  on  the  left-hand  side  with 
supporting  text  on  the  right-hand  side. 

■ Summary:  When  one  medium  summaries  and  reinforces  the  chief  points  of  the  information  presented  by 
another  resource.  For  instance,  display  bullet  point  text  summarising  the  main  points  talked  about  in  speech 
narration. 
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■ Explanation:  When  one  medium  explains  the  information  presented  by  another  medium.  An  example  is, 
where  a complex  interaction  is  presented  by  a set  of  graphics,  speech  or  text  can  be  used  to  explain  the  aim  of 
the  activity. 

■ Redundancy:  When  exactly  the  same  information  is  presented  by  two  different  media  resources.  For  instance, 
what-tO“do  instructions  can  be  presented  redundantly  using  speech  and  text. 

Depending  on  the  information  requirement,  a combination  of  the  above  could  be  used.  However,  there  was  no 
evidence  of  establishing  explicit  referential  connections  between  different  media  presented  concurrently. 

3.2  Design  Guidelines 

To  establish  how  much  support  designers  used,  they  were  asked  regarding  the  guidelines  and  principles  they 
applied  in  the  design  process.  Five  designers  reported  that  their  companies  had  developed  presentation  design 
guides.  However,  four  of  them  suggested  that  these  guides  were  used  primarily  to  train  new  staff,  and  did  not  get 
referred  to  actively  during  the  design  process.  It  was  pointed  out  that  the  design  guides  contained  only  high-level 
design  principles  regarding  screen,  interaction,  and  navigation  design,  as  well  as  presentation  of  text,  graphic, 
audio,  and  video.  However,  designers  seemed  to  apply  some  interface  design  heuristics,  examples  of  which  are: 

■ Establish  a uniform  screen  layout. 

■ Avoid  using  many  hypertext  links  in  one  page  as  the  user  can  easily  get  lost,  instead  use  more  rigid  menu 
structure. 

3.3  Design  Problems 

The  major  problems  encountered  by  the  designers  fall  in  two  categories:  high-level  constraints  and  design 
problems. 

High-Level  Constraints: 

■ Client  availability:  Client  participation  played  an  important  role  in  ensuring  user’s  satisfaction  with  the 
system.  However,  most  of  the  designers  reported  that  arranging  meetings  with  Subject  Matter  Experts  and 
obtaining  clarification  of  content  was  quite  often  difficult.  Getting  hold  of  sufficient  number  of 'users  for 
prototype  evaluation,  and  receiving  their  feedback  on  time  was  another  problem. 

■ Available  budget  and  delivery  deadline  pressure:  How  sophisticated  and  interactive  media  can  be  created 
often  depended  on  the  budget  available.  In  two  cases  tight  delivery  deadlines  also  resulted  in  reducing  the 
time  spent  on  usability  testing. 

Design  Problems: 

■ Creating  workable  designs:  In  order  to  produce  workable  designs,  designers  had  to  establish  a trade-off 
between  the  learning  objectives,  how  feasible  the  ideas  were  within  the  time  available,  and  whether  and  how 
easily  they  could  be  implemented  using  the  available  technology.  To  achieve  that  input  from  domain 
specialists,  designers,  graphic  artists,  and  programmers  was  required.  However,  some  of  these  specialists  were 
unable  to  appreciate  the  multidimensionality  of  the  design  space,  and  therefore  lacked  common  perspective  in 
finding  the  best  design  solution. 

■ Designing  for  users  with  different  levels  of  abilities:  When  the  target  population  included  users  with  different 
levels  of  computer  literacy,  finding  the  right  balance  between  providing  such  navigation  support,  which  would 
be  sufficient  for  novices,  and  would  not  irritate  experienced  users,  was  a challenge.  Structuring  the  training 
material  to  accommodate  both,  more  and  less  technicality  knowledgeable  learners,  was  also  not  easy,  as  the 
first  group  required  easy  and  random  access  to  technical  information  of  interest  to  them,  and  the  latter  group 
required  more  rigid  structure  and  less  complex  terminology. 

4.  Presentation  Design  Patterns 

The  sections  above  identified  three  major  problems  with  the  current  approach  to  designing  instructional 
multimedia.  These  problems  are:  complexity,  which  necessitates  the  involvement  of  a number  of  specialists  from 
different  professional  areas,  who  need  to  communicate  on  a common  ground,  and  the  tight  delivery  deadlines 
against  which  designers  have  to  compete.  One  way  of  dealing  with  the  complexity  of  design,  and  reducing  the 
time  and  effort  involved  in  the  design  process,  without  compromising  on  the  quality  of  the  software,  is  to  reuse 
previous  design  knowledge  when  constructing  new  applications.  One  way  of  systematically  reusing  previous 
design  expertise  involves  closely  examining  successful  applications,  and  identifying  recurring  solutions  to 
common  design  problems.  These  can  be  encapsulated  in  design  patterns.  Design  patterns  have  been  successfully 
used  in  architecture  (Alexander  et  al.  1977),  as  well  as  in  software  engineering,  predominantly  in  the  object- 
oriented  software  design  community  (Gamma  et  al.  1 995).  A design  pattern  describes  the  core  of  a solution  to  a 
particular  problem  in  an  abstract  way,  which  when  used  in  a given  situation  can  generate  effective  design. 


397 


In  instructional  multimedia,  presentation  design  patterns  can  explore  the  interactions  between  the  information 
and  task  requirements,  and  their  external  representation  using  different  media,  while  explaining  the  pedagogic 
and  cognitive  implications  of  such  compositions  on  the  learner.  Their  focus  on  how  the  media  representation  can 
support  learner’s  cognitive  processes  can  ensure  effective  learning.  As  patterns  can  work  at  different  levels  of 
design  - dialogue,  content,  screen  layout,  navigation  - they  can  attempt  to  tie  these  levels  together. 

From  the  study,  it  emerged  that  five  designers  reused  design  rationale  from  previous  projects.  For  instance,  in 
one  case,  templates  were  used  for  presenting  graphics  and  bullet  point  text  together.  In  other  two  cases  a uniform 
screen  layout  was  set  across  a range  of  products.  The  designers  found  these  templates  time  saving,  and  a good  way 
of  ensuring  interface  consistency  within  a product  or  across  a range  of  products.  However,  reuse  was  not  done  in  a 
comprehensive  way.  A coherent  library  of  design  patterns  could  potentially  speed  up  the  design  process,  provide 
professionals  with  a common  lingua  franca  for  communication,  ensure  interface  usability,  and  pass  design 
expertise  to  novice  designers,  so  they  can  produce  effective  designs  faster.  Two  prototypical  design  patterns  are 
presented  in  the  next  sub-sections.  Included  are  one  content  (4.1)  and  one  screen  layout  (4.2)  pattern.  The  patterns 
are  described  using  a uniform  format,  comprising  of  the  name  of  the  pattern,  the  context  or  the  situation  which 
gives  rise  to  the  problem,  the  problem  that  needs  to  be  solved,  the  proposed  solution,  and  examples  of  existing 
systems,  where  the  patter  is  implemented. 

4.1  How-It-Works  Mechanism  Pattern 

Context:  Consider  this  pattern  in  representing  the  working  mechanism  of  a physical  object,  where  the  behaviour 
of  the  aggregate  object  is  determined  by  the  collaborative  behaviour  of  its  components. 

Problem:  You  need  to  explain  how  a complex  devise  works,  which  is  composed  of  certain  parts.  There  are  two 
main  forces  that  need  to  be  resolved  in  such  representations.  Firstly,  the  user  should  be  able  to  identify  all 
individual  components,  and  the  roles  they  play  in  the  mechanism,  as  well  as  the  relationships  between  the 
components.  And  secondly,  the  user  should  be  able  to  aggregate  behaviour  of  the  components  into  a semantic 
behaviour  of  the  mechanism. 

Solution:  Structure  the  presentation  into  four  parts.  Firstly,  present  an  exploded  view  diagram  of  the  machine  and 
a textual  description  of  its  overall  function.  Then,  present  a schematic  diagram  of  the  machine  and  its 
components,  together  with  textual  descriptions  of  the  characteristics  of  each  component.  These  two  parts  will 
facilitate  the  learner  in  decomposing  the  machine  into  its  basic  elements.  To  aid  the  learner  in  identifying  the 
spatial  relationships  between  devise  components,  present  different  views  of  the  machine.  Text  and  speech 
explaining  component  connections  should  accompany  each  depiction.  Finally,  present  a realistic  animation 
showing  all  component  behaviours  in  actual  order  of  occurrence.  The  animation  should  be  synchronised  with 
speech  commentary  describing  the  dynamics  of  the  machine’s  operation.  This  will  aid  the  learner  in  building  a 
dynamic  mental  model  of  how  the  devise  operates. 

Examples:  Microsoft  Encarta  - The  Human  Heart,  DK  Multimedia  “The  Way  Things  Work”. 

4.2  Interactivity  Screen  Pattern 

Context:  Consider  this  pattern  in  composing  a screen  allowing  the  user  to  perform  a learning  activity. 

Problem:  You  need  to  allow  the  user  to  perform  a learning  activity,  and  to  provide  them  with  adequate  task 
instructions  at  the  same  time.  As  the  activity  is  central,  and  the  learner  will  need  to  concentrate  on  it.  Task 
instructions  and  learning  support  will  have  to  be  presented  as  well.  However,  although  they  are  necessary  for  the 
successful  completion  of  the  activity,  they  should  not  obstruct  the  learner  in  performing  the  task. 


Figure  2:  A Sample  Interactivity  Screen  With  Lefi-To-Right  Visual  Composition  (Aircom  Education  ©). 

Solution:  Divide  the  display  into  two  areas:  interaction  and  control  (see  Fig.  2).  As  the  interaction  is  central,  place 
it  at  the  left-hand  or  top  side  of  the  screen.  Designate  the  smaller  right-hand  or  bottom  part  of  the  display  for  task 
instructions  and  required  learning  aids.  This  will  focus  the  user  on  the  interaction.  Use  text  for  task  instructions. 
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with  redundant  speech  narration,  which  can  enable  the  learner  to  familiarise  with  the  interaction  while  listening  to 
the  instructions. 

Examples:  Aircom  Education  “Maths:  The  Journey  to  Swallow  Farm”. 

5.  Discussion 

The  study,  although  limited  in  size,  provides  insightful  information  about  the  current  practices  of  designing 
instructional  multimedia  software.  The  findings  show  that  the  client  is  involved  extensively  in  specifying  the 
courseware,  and  reviewing  the  progress  made  at  the  end  of  every  stage,  which  is  important  to  ensure  that  client’s 
requirements  are  met.  However,  this  is  not  a sufficient  guarantee  for  the  usability  of  the  user  interface.  The  study 
revealed  that  although  some  learner  characteristics  were  considered  while  making  design  decisions,  that  was  not 
done  in  a systematic  way.  There  is  a need  for  a way  of  ensuring  that  learner’s  needs,  learning  styles,  and 
individual  approach  to  learning  are  addressed  in  the  design,  also  suggested  in  (Soloway  1998;  Wild  & Quinn 
1 998).  From  the  study,  it  became  apparent  that  the  selection  of  appropriate  media  resources  is  primarily  based  on 
high-level  factors,  such  as  selecting  speech  for  motivating  the  learner.  However,  studies  show  that  the  information 
requirements  for  each  task,  and  the  cognitively  relevant  characteristics  of  the  media  are  vital  for  allocating  the 
media  resources  to  be  used  (Sutcliffe  & Faraday  1994),  which  has  to  be  done  on  a moment  by  moment  basis 
(Faraday  & Sutcliffe  1997).  In  order  to  deliver  the  learning  material  effectively,  designers  should  order  shifts  of 
user’s  attention  according  to  the  content  being  shown,  and  ensure  that  the  elements  of  the  presentation  are 
integrated  into  unified  propositions  (Faraday  & Sutcliffe  1 996).  However,  the  four  reasons  that  designers  stated  for 
presenting  different  media  concurrently  although  valid,  are  not  comprehensive  enough  to  ensure  that  the 
presentation  will  not  overloaded  user’s  attention,  and  cause  difficulties  with  comprehending  the  information 
presented.  Although  the  guidelines  and  heuristics  presented  can  be  of  useful,  they  are  of  too  high  level,  and  need 
to  be  augmented  with  sound  cognitive  science  and  pedagogic  principles.  This  evidence  indicates  that  there  is  a 
need  to  further  support  the  process  of  multimedia  user  interface  design. 

A library  of  design  patterns  can  prove  to  be  a better  way  of  delivering  design  knowledge,  as  the  patterns  could 
provide  the  essence  of  proven  solutions  to  common  interface  design  problems,  augmented  with  sound  theoretical 
principles.  Potentially,  this  could  speed  up  the  design  process,  provide  professionals  from  diverse  backgrounds 
with  a common  language  for  communication,  ensure  interface  usability,  and  pass  design  expertise  to  novice 
designers.  The  proposed  design  patterns  lay  the  foundations  of  a pattern  library  for  multimedia  presentation 
design.  In  our  future  work  we  will  continue  extracting  design  patterns,  which  will  be  integrated  into  a coherent 
pattern  library.  A systematic  method  of  applying  design  patterns  will  also  be  developed.  Finally,  the  validity  of  the 
patterns  will  be  tested  to  ensure  their  benefit  to  the  design  process  and  students’  performance. 
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Abstract:  A hypermedia  application  offers  its  users  a lot  of  freedom  to  navigate  through  a large 
hyperspace.  The  rich  link  structure  of  the  hypermedia  application  can  not  only  cause  users  to  get 
lost  in  the  hyperspace,  but  can  also  lead  to  comprehension  problems  because  users  read  information 
in  an  order  not  foreseen  by  the  author.  Adaptive  hypermedia  systems  (or  AHS  for  short)  aim  at 
overcoming  these  problems  by  providing  adaptive  navigation  support  and  by  providing  adaptive 
content. 

We  have  developed  a reference  model  for  adaptive  hypermedia  applications:  AHAM  (for  /idaptive 
//ypermedia  /ipplication  A/odel),  which  is  an  extension  of  the  Dexter  hypermedia  reference  model 
(Halasz  et  al.,  1990,  1994).  The  goal  of  AHAM  is  to  describe  adaptive  hypermedia  applications, 
especially  from  the  point  of  view  of  authors  designing  such  applications.  AHAM  divides  an  AHS 
into  a Domain  Model  (DM),  User  Model  (UM)  and  Teaching  Model  (TM).  This  paper  describes 
support  tools  that  help  authors  to  create  these  three  parts  of  an  adaptive  hypermedia  application, 
and  to  assure  that  these  parts  together  form  a usable  and  consistent  complete  application. 


1.  Introduction 

The  rich  link  structure  that  is  typical  of  hypermedia  applications  offers  users  a lot  of  freedom  to  navigate 
through  a large  hyperspace.  However,  it  also  causes  users  to  lose  their  way  in  that  hyperspace,  and  it  leads  to  com- 
prehension problems  because  users  may  read  pages  in  an  order  which  the  author  did  not  foresee  and  which  may  not 
even  make  sense.  Adaptive  hypermedia  applications  aim  at  overcoming  these  problems  by  providing  adaptive  navi- 
gation support  and  adaptive  content  presentation. 

An  adaptive  hypermedia  system  (AHS)  (Brusilovsky,  1996)  is  a special  kind  of  hypermedia  system  that 
can  adapt  the  contents  and  links  of  an  application  depending  on  observed  user  features.  For  building  an  AHS,  the 
author  has  to  provide  two  (related)  parts:  first  of  all  the  author  has  to  write  the  actual  hypertext;  second,  the  author 
has  to  specify  some  dynamic  adaptation  control  information.  In  practice  it  turns  out  to  be  necessary  to  provide 
authoring  tools  that  facilitate  the  specification  of  both  aspects.  In  existing  AHS  (Brusilovsky  et  al.,  1996a,  1996b, 
De  Bra  et  al.,  1997,  1998,  Calvi  et  al.,  1997)  authoring  is  often  inconvenient,  due  to  the  mixture  of  (textual)  content 
and  diverse  hints  for  adaptation.  In  Interbook  (Brusilovsky  et  al.,  1996b)  the  author  must  write  structured,  annotated 
MS-Word  files.  In  AHA  (De  Bra  et  al.,  1998)  the  author  must  write  annotated  HTML  files.  This  mixture  of  con- 
tent, links  and  adaptation  makes  it  difficult  for  authors  to  maintain  a clear  picture  of  how  concepts  relate  to  each 
other.  This  implies  that  it  is  hard  to  define  adaptation  on  an  abstract  conceptual  level. 

In  (Wu  et  al.,  1998,  De  Bra  et  al.,  1999)  we  have  presented  a reference  model  for  adaptive  hypermedia  ap- 
plications, called  AHAM  (for  Adaptive  Hypermedia  Application  Model).  In  order  to  separate  different  authoring 
aspects,  AHAM  splits  up  an  adaptive  hypermedia  application  into  a domain  model  (DM),  user  model  (UM)  and 
teaching  model  (TM).  TTiese  three  parts  together  form  what  the  Dexter  hypermedia  reference  model  (Halasz  et  al., 
1990,  1994)  calls  the  Storage  layer.  Apart  from  this  layer  Dexter  and  AHAM  also  mention  (but  do  not  formally 
define)  a Runtime  layer  which  deals  with  sessions  and  the  actual  presentation  and  interaction  on  the  screen,  and  a 
Within-Component  layer  which  deals  with  implementation  dependent  representations  of  content  and  links. 

In  this  paper  we  address  operations  of  DM,  UM  and  TM  that  are  needed  for  creating  (authoring)  adaptive 
hypermedia  applications.  We  describe  these  operations  at  the  conceptual  level,  not  the  implementation  level.  The 
definition  of  these  operations  shows  how  the  tasks  of  1)  creating  content,  2)  combining  smaller  pieces  into  compos- 
ite (abstract)  concepts,  3)  defining  relationships  between  these  concepts,  4)  verifying  the  consistency  of  these  rela- 
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tionships,  and  5)  defining  rules  for  the  adaptation  of  content  and  links  to  each  individual  user,  are  separated.  The 
separation  of  these  tasks  will  make  authoring  of  adaptive  hypermedia  applications  easier  than  it  is  today.  The  sepa- 
rated tasks  together  form  a description  of  the  functionality  of  an  authoring  tool  based  on  the  AHAM  model. 

This  paper  is  organized  as  follows:  In  Section  2 we  describe  the  AHAM  reference  model,  with  its  three 
parts  (domain  model,  user  model,  teaching  model).  In  Section  3 we  show  how  authoring  is  facilitated  by  adding 
support  functions  to  these  three  parts.  Section  4 concludes  and  indicates  future  work. 


2.  AHAM,  a Dexter-based  Reference  Model 

In  hypermedia  applications  the  emphasis  is  always  on  the  information  nodes  and  on  the  link  structure  con- 
necting these  nodes.  The  Dexter  model  captures  this  in  what  it  calls  the  Storage  layer.  It  represents  a domain  model 
DM,  i.e.  the  author’s  view  on  the  application  domain  expressed  in  terms  of  concepts.  In  adaptive  hypermedia  appli- 
cations the  central  role  of  DM  is  shared  with  a user  model  UM.  UM  represents  the  relationship  between  the  user 
and  the  domain  model  by  keeping  track  of  how  much  the  user  knows  about  each  of  the  concepts  in  the  application 
domain. 

In  order  to  perform  adaptation  based  on  DM  and  UM  an  author  (or  AHS  designer)  needs  to  specify  how  the 
user’s  knowledge  influences  the  presentation  of  the  information  from  DM,  In  AHAM  this  is  done  by  means  of  a 
teaching  model  TM  consisting  of  so-called  pedagogical  rules.  An  adaptive  engine  uses  those  rules  to  manipulate 
link  anchors  from  the  Dexter  model’s  anchoring  and  to  generate  what  the  Dexter  model  calls  the  presentation  speci- 
fications. Like  the  Dexter  model,  AHAM  focuses  on  the  Storage  layer,  the  anchoring  and  the  presentation  specifi- 
cations. Figure  1 shows  the  structure  of  adaptive  hypermedia  applications  in  AHAM. 
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Figure  1:  global  structure  of  adaptive  hypermedia  applications. 

In  this  section  we  present  the  elements  of  AHAM  that  we  will  use  in  Section  3 to  illustrate  the  authoring 
functionality. 

A component  is  an  abstract  notion  in  an  AHS.  It  is  a pair  (uid,  cinfo)  where  uid  is  a globally  unique  (ob- 
ject) identifier  for  the  concept,  and  cinfo  represents  the  component's  information  consisting  of: 

• A set  of  attribute-value  pairs; 

• A sequence  of  anchors  (for  attaching  links); 

• A presentation  specification. 

A concept  is  a component  representing  an  abstract  information  item  from  the  application  domain.  It  can  be 
either  an  atomic  concept  or  a composite  concept.  An  atomic  concept  corresponds  to  a fragment  of  information.  Its 
attribute  and  anchor  values  belong  to  the  Within-component  layer  and  are  thus  implementation  dependent  and  not 
described  in  the  model.  A composite  component  has  two  additional  attributes: 
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• A sequence  of  children  (concepts); 

• A constructor  function  (to  denote  how  the  children  belong  together). 

The  children  of  a composite  concept  are  either  all  atomic  concepts  (then  we  call  it  a page  or  in  typical  hypertext 
terms  a node)  or  all  composite  concepts.  The  composite  concept  component  hierarchy  must  be  a DAG  (directed 
acyclic  graph).  Also,  every  atomic  concept  must  be  included  in  some  composite  concept.  Figure  2 illustrates  a part 
of  a concept  hierarchy. 

An  anchor  is  a pair  (aid,  avalue),  indicating  the  endpoint  of  a hypertext  link.  The  avalue  for  an  anchor  in  a 
composite  concept  is  the  identifier  of  a concept  that  belongs  to  that  composite. 

A specifier  is  a tuple  (uid,  aid,  dir,  pres),  where  uid  is  the  identifier  of  a concept,  aid  is  the  identifier  of  an 
anchor,  dir  is  a direction  (FROM,  TO,  BIDIRECT,  or  NONE),  and  pres  is  a presentation  specification. 

A concept  relationship  is  a component,  with  two  additional  attributes: 

• A sequence  of  specifiers; 

• A concept  relationship  type. 

The  most  common  type  of  concept  relationship  is  the  type  link.  This  corresponds  to  the  link  components  in  the 
Dexter  model,  or  links  in  most  hypermedia  systems.  However,  in  AHAM  we  consider  other  types  of  relationships 
as  well,  that  plays  a role  in  the  adaptation.  A common  type  of  concept  relationship  is  prerequisite.  When  a concept 
Cl  is  a prerequisite  for  C2  it  means  that  the  user  should  read  Cl  before  C2.  It  does  not  mean  that  there  must  be  a 
link  from  Cl  to  C2.  It  only  means  that  the  system  somehow  takes  into  account  that  reading  about  C2  is  not  desired 
before  some  (enough)  knowledge  about  Cl  has  been  acquired.  Figure  3 shows  a small  set  of  (only  binary)  relation- 
ships, both  prerequisites  and  links.  We  treat  links  and  other  types  of  relationships  in  the  same  way  in  order  to  keep 
authoring  support  tools  as  generic  and  uniform  as  possible. 
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Figure  2:  Example  concept  hierarchy.  Figure  3:  Example  concept  relationship  structure. 

The  atomic  concepts,  composite  concepts  and  concept  relationships  together  form  the  domain  model  DM  of  an 
adaptive  hypermedia  application. 

An  AHS  associates  number  of  (system  or  author  defined)  user-model  attributes  with  each  concept  compo- 
nent of  DM.  For  each  user  the  AHS  maintains  a table  in  which  for  each  concept  the  attribute  values  for  that  concept 
are  stored.  The  structure  of  this  table  is  the  user  model  scheme.  The  table  for  a specific  user  is  a user  model  in- 
stance. If  there  is  no  confusion  between  scheme  and  instance  we  just  use  the  term  user  model.  One  of  the  attributes 
of  the  user  model  is  the  unique  (concept)  identifier.  Other  attributes  commonly  found  in  AHS  are  the  knowledge 
value  and  some  “log”  entry  whether  the  user  has  read  about  the  concept.  The  table  below  illustrates  the  (concep- 
tual) structure  of  a user  model,  for  a course  on  hypermedia.  (In  the  example  table  the  concepts  Xanadu  and  KMS 
were  at  least  partially  learnt.  The  concept  WWW,  consisting  of  two  sub-parts,  is  partially  learnt  because  WWW- 
html  has  been  read  but  WWW-http  has  not.  One  can  see  that  WWW  must  be  a composite  concept  that  is  not  a page 
because  it  is  already  partially  learnt  while  it  has  not  been  read  at  all.) 


1 concept  name  (uid) 

knowledge  value  \ 

read  j 

1 Xanadu 

well  learned  1 

true  1 ,,, ( 

i KMS 

learned 

true  i ...  1 

WWW-html 

well  learned 

true  1 ... 

1 WWW-http 

not  known 

false 

\ WWW 

learned 

false 

i.:  

1“ ; 
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A generic  pedagogical  rule  is  a tuple  (R,  PH,  PR),  where  R is  a “triggered”  rule,  PH  is  the  “phase”  for  the 
execution  of  the  rule  and  PR  is  a Boolean  “propagate”  field  which  indicates  whether  this  rule  may  trigger  other 
rules.  The  “phase”  of  a rule  can  have  the  value  pre  or  post.  The  phase  pre  is  executed  before  and  during  the  gen- 
eration of  the  page,  while  post  is  executed  afterwards.  These  rules  describe  how  an  AHS  works,  but  the  AHS  is  not 
required  to  be  rule  based.  In  many  existing  AHS  the  behavior  of  the  system  is  completely  predefined.  But  in  some 
newer  systems,  including  AHA  (De  Bra  et  al.,  1998),  authors  can  add  some  kind  of  rules  to  the  system. 

A specific  pedagogical  rule  is  a tuple  (R,  SC,  PH,  PR)  where  (in  addition  to  a generic  rule)  SC  is  a set  of 
concepts  used  in  the  rule.  The  rule  manipulates  user-model  attributes  and  “predicates”  over  specific  concepts  of  SC. 

An  AHS  may  have  predefined  or  implicit  generic  pedagogical  rules.  If  these  rules  suffice  there  is  no  need 
for  a language  in  which  authors  can  write  new  rules.  Author  defined  rules  take  precedence  over  predefined  rules. 
Specific  rules  take  precedence  over  generic  rules,  and  are  thus  used  to  define  exceptions  to  generic  rules.  The  exis- 
tence of  the  two  execution  phases  makes  it  possible  to  do  adaptation  based  on  the  “previous”  state  of  the  user  model, 
and  to  update  the  user  model  to  a “new”  state  afterwards. 

A typical  standard  pedagogical  rule  for  the  pre  phase  is  the  rule  that  says  that  a link  anchor  will  be  hidden 
(or  annotated  as  undesirable)  if  some  prerequisite  knowledge  for  the  destination  of  the  link  is  still  not  known.  A 
typical  standard  rule  for  the  post  phase  is  the  rule  that  says  that  a page-concept  becomes  well  learned  when  the  page 
is  accessed  and  all  prerequisite  knowledge  was  present,  and  it  becomes  only  learned  when  the  page  is  accessed  but 
some  prerequisite  knowledge  was  still  not  acquired. 

The  teaching  model  of  an  AHS  is  the  set  of  (generic  and  specific)  pedagogical  rules.  An  adaptive  engine 
(AE)  is  a software  environment  that  performs  the  following  functions  (based  on  the  rules): 

• It  offers  generic  page  constructors. 

• It  optionally  offers  a (very  simple  programming)  language  for  describing  new  page  constructors. 

• It  performs  adaptation  by  executing  the  page  constructors. 

• It  updates  the  user  model  (instance)  each  time  the  user  visits  a page. 

The  adaptive  engine  thus  provides  the  implementation  dependent  aspects  while  DM,  UM  and  TM  describe  the  in- 
formation and  adaptation  at  the  conceptual  level. 

An  adaptive  hypermedia  application  is  a 4-tuple  (DM,  UM,  TM,  AE),  where  DM  is  a domain  model.  UM 
is  a user  model,  TM  is  a teaching  model,  and  AE  is  an  adaptive  engine. 


3.  Authoring  Support  for  AHAM 

In  this  section  we  extend  the  (data)  definitions  of  AHAM  to  include  the  operations  that  are  needed  for 
authoring.  Because  of  lack  of  space  we  only  give  these  definitions  informally.  We  distinguish  three  main  classes 
DM-Tool,  UM-Tool,  TM-Tool.  These  indicate  which  authoring  functionality  is  needed  for  creating  (and  modifying) 
a domain  model,  a user  model  and  a teaching  model.  When  authoring  (adaptive  or  non-adaptive)  hypermedia  appli- 
cations it  is  difficult  to  verify  global  properties  of  the  resulting  hyperdocument.  By  separating  domain  model,  user 
model  and  teaching  model,  the  verification  of  properties  becomes  feasible  and  sometimes  even  easy. 

DM-Tool  must  obviously  offer  methods  to  create  and  delete  concepts  and  concept  relationships.  These 
methods  may  generate  interesting  side-effects  that  reduce  the  (manual)  work  for  the  author.  E.g.,  when  creating  a 
new  atomic  concept,  DM-Tool  can  show  a dialog  box  to  assign  this  atomic  concept  to  a composite  (which  must  be  a 
page).  When  a page  is  deleted,  DM-Tool  can  show  a dialog  box  to  let  the  author  decide  what  to  do  with  the  frag- 
ments (atomic  concepts)  of  this  page.  When  a concept  is  added  to  a composite,  DM-Tool  can  verify  whether  the 
composite  only  contains  atomic  concepts,  or  only  composite  concepts.  We  won’t  go  into  more  detail  here,  but  only 
describe  support  methods  that  work  on  a global  level,  and  that  can  be  used  to  verify  the  integrity  of  the  domain 
model: 

• “verify-hierarchy-is-DAG”  checks  that  no  (composite)  concept  directly  or  indirectly  contains  itself  (as  a child 
or  a lower  descendant).  This  check  can  be  performed  automatically  each  time  a new  concepts  is  added. 

• “get-orphan-concepts”  is  used  to  find  the  atomic  concepts  that  no  longer  belong  to  a page.  It  is  not  desirable  to 
force  the  deletion  of  fragments  when  a page  is  deleted.  The  author  may  wish  to  keep  some  fragments  and  add 
them  to  other  (new)  pages  later. 

Each  concept  relationship  type  may  have  its  own  constraints.  The  combination  of  concept  relationship  types  may 
introduce  additional  constraints,  vl^ether  DM-Tool  can  verify  these  constraints  depends  on  their  definition.  Some 
methods  for  verifying  common  constraints  are: 
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• “verify-connected-links”  checks  whether  every  composite  concept  (including  pages)  can  be  reached  by  follow- 
ing links  from  a given  starting  page. 

• “verify-acyclic-pre requisites*’  checks  whether  no  concept  is  a prerequisite  for  itself,  either  directly  or  indirectly, 

• “verify-conditional-links”  checks  whether  for  every  composite  concept  (including  pages)  there  exists  a path 
leading  to  that  concept,  from  a given  starting  page,  starting  with  the  “default”  user-model  instance,  and  without 
visiting  concepts  for  which  prerequisite  knowledge  is  missing. 

The  third  method  is  a nice  example  of  how  the  verification  of  constraints  becomes  difficult  when  different  types  of 
concept  relationships  are  combined.  Also,  the  verification  is  based  on  implicit  assumptions  about  how  the  user 
model  evolves.  Indeed,  this  verification  only  works  if  we  make  assumptions  about  how  the  user  model  evolves 
based  on  the  user  visiting  certain  concepts.  Figure  3 for  instance  shows  a sound  structure,  only  under  the  assump- 
tion that  visiting  a concept  C\  implies  that  enough  knowledge  is  gained  about  Cj  to  warrant  that  prerequisites  that 
depend  on  Q are  fulfilled. 

In  an  object-oriented  view  the  UM-Tool  can  be  considered  as  an  object  that  contains  DM-Tool  as  a part. 
When  UM-Tool  contains  DM-Tool  it  becomes  possible  to  model  the  interaction  between  the  user  model  and  the 
domain  model  as  being  part  of  the  functionality  of  UM-Tool, 

The  authoring  support  of  an  AHS  must  include  a method  for  propagating  updates  to  the  domain  model  DM 
to  the  user  model.  In  our  view  this  means  that  UM-Tool  needs  a method  for  importing  all  concepts  from  DM  into 
UM.  However,  an  author  may  wish  to  keep  the  user  model  simpler  by  only  keeping  some  concepts  in  the  user 
model.  Therefore,  we  have  an  auxiliary  method  “verify-all-concepts-in-DM”  to  check  whether  all  the  concepts  from 
UM  are  in  DM,  but  no  method  to  verify  the  opposite  (because  the  opposite  may  not  be  desirable). 

UM-Tool  has  different  kinds  of  methods: 

• “add/delete-concept”  is  normally  available  to  the  author,  to  manipulate  the  user  model  and  eliminate  low  level 
concepts  that  exist  in  DM  but  are  not  of  interest,  (The  author  may  wish  to  define  the  adaptation  based  on 
knowledge  of  high  level  concepts  only.) 

• “add/delete-attribute”  is  a method  to  add  a user-rnodel  attribute  to  every  concept.  It  means  that  the  AHS  is  able 
to  record  values  for  this  attribute  about  each  concept.  While  the  AHAM  model  describes  the  user  model  as  a 
table  structure  in  which  all  attributes  are  available  for  all  concepts,  a particular  AHS  may  have  a more  restrictive 
implementation.  In  the  AHA  system  (De  Bra  et  al,,  1998)  for  instance,  there  are  some  “concepts”  that  represent 
color  preferences  for  links.  These  concepts  only  have  a “colof  ’ attribute,  while  the  other  (real)  concepts  only 

Whether  authors  can  extend  the  user  model  with  new  attributes  depends  on  the  AHS,  The  limiting  factor  is  the  abil- 
ity or  inability  to  actually  use  new  attributes.  The  rules  in  TM  are  the  (only)  means  to  use  new  attributes, 

TM-Tool  provides  methods  for  creating  pedagogical  rules.  In  the  object-oriented  view  TM-Tool  contains 
DM-Tool  and  UM-Tool  as  parts.  This  makes  it  possible  to  describe  the  interaction  between  DM-Tool  and  UM-Tool 
as  being  part  of  the  functionality  of  TM-Tool, 

• “add-spec-pre/post-rule”  are  methods  for  creating  specific  rules.  The  author  can  add  rules  that  work  with  spe- 
cific concepts.  The  syntax  of  such  rules  depends  on  the  AHS,  An  example  of  the  use  of  specific  rules  in  AHA 
(De  Bra  et  al,,  1998)  is: 

<1--  if  readme  and  not  intro  --> 

This  text  only  appears  if  "readme"  is  a known  concept 
and  "intro"  is  not  yet  known. 

<!--  endif  - - > 

• “add-gen-pre/post-rule”  are  methods  for  creating  generic  rules.  Generic  rules  specify  how  the  AHS  should 
handle  events  and  concept  relationships  of  a given  type,  A generic  rule  about  prerequisite  relationships  could 
say: 

• When  a user  visits  concept  C and  all  prerequisites  are  satisfied  the  knowledge  about  C becomes  0,8, 

• When  a user  visits  concept  C and  not  all  prerequisites  are  satisfied  the  knowledge  about  C becomes  0,5, 

This  rule  applies  to  all  prerequisite  relationships,  while  the  above  example  from  AHA  only  works  for  a specific 
instance,  with  two  specific  concepts. 

If  TM-Tool  lets  authors  create  their  own  generic  rules,  possibly  using  their  own  attributes  added  to  UM,  it  may  not 
be  possible  to  automatically  verify  the  consistency  of  the  rules.  In  the  current  version  of  AHA  (De  Bra  et  al.,  1998) 
only  specific  rules  are  permitted,  using  a simple  and  not  very  expressive  language.  While  this  limits  the  power  of 
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AHA  it  makes  it  possible  to  perform  the  verification  that  all  concepts  (pages)  can  be  reached  from  the  given  starting 
page  without  violating  any  of  the  conditions. 

An  advanced  AHS  will  let  authors  define  their  own  (generic  and  specific)  pedagogical  rules.  However,  in 
the  near  future  (and  in  our  redesign  of  AHA)  TM-Tool  will  most  likely  only  let  authors  configure  predefined  rules. 
In  the  example  above,  the  knowledge  values  of  0.5  and  0.8  could  be  author-configurable  for  instance. 

4.  Conclusion  and  Future  Work 

We  have  defined  authoring  support  by  specifying  three  main  classes  UM-Tool,  DM-Tool,  and  TM-Tool. 
In  the  class  definitions  we  have  described  the  data  model  and  the  support  operations  for  authoring.  The  division  of 
adaptive  hypermedia  applications  into  a domain  model,  user  model  and  teaching  model  makes  the  task  of  authoring 
clearer  by  separating  different  concerns.  We  have  also  indicated  the  main  support  functions  that  are  needed  to  en- 
sure that  the  three  generated  parts  are  internally  consistent  and  also  consistent  with  each  other. 

We  are  currently  working  on  a redesign  (and  re-implementation)  of  the  AHA  system  (De  Bra  et  al.,  1998) 
which  is  being  used  at  the  Eindhoven  University  of  Technology  for  a course  on  hypermedia  (URL 
http://wwwis,win,tue.nl/2L690/),  a course  on  graphical  user  interfaces  (URL  http://wwwis.win.tue.nl/2M350/),  and  a 
kiosk  information  system  for  students  who  wish  to  do  an  internship  or  master  thesis  in  information  systems  (URL 
http://wwwis,win,tue,nl/IShype/).  AHA  is  also  being  used  for  the  development  of  a course  in  language  learning 
(Calvi,  1998).  AHA  currently  offers  authors  the  ability  to  specify  specific  rules  for  the  inclusion  of  fragments  or  for 
determining  the  desirability  of  links.  We  hope  to  extend  AHA  with  the  ability  to  specify  generic  rules  as  needed  by 
richer  tools  for  creating  user  models  and  teaching  models. 
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Abstract:  In  the  rapidly  developing  area  of  User  Interface  Design  (UID),  the  existing 

knowledge  and  understanding  of  UID  serves  as  a basis  for  further  research  and  study  of 
factors  affecting  users  as  they  interact  with  interfaces  that  utilize  the  technology  of 
multimedia,  virtual  reality  and  intelligent  agents.  Color  and  navigation  are  two  of  the  factors 
that  are  being  investigated  to  help  designers  develop  user  interfaces  that  are  effective  and 
efficient  for  the  users.  This  paper  describes  a color-coded  navigation  tool  developed  to  support 
research  about  the  effects  of  color  on  navigational  decisions  in  a virtual  reality  environment. 
The  tool,  Color-Coded  Virtual  Reality  Navigation  (CVRN),  allows  the  design  and  exploration 
of  color-coded  three-dimensional  worlds  and  provides  for  navigational  tracking.  When  using 
CVRN,  the  structure  of  the  virtual  world  is  a never-ending  building  composed  of  many  empty 
rooms,  CVRN  enables  researchers  to  customize  experiments  that  allow  for  testing  a subject’s 
navigational  behavior  and  color  preferences  in  a virtual  reality  environment. 


Introduction 

Human  Computer  Interaction  (HCI)  especially  as  it  relates  to  User  Interface  Design  (UID)  is  one  of  the 
most  rapidly  developing  subject  areas  in  computer  science  and  information  systems.  There  has  been  a 
tremendous  growth  in  interest  about  human  computer  interaction  in  the  past  two  decades  since  the  development 
of  the  personal  computers,  and  subsequently,  since  the  wide-spread  use  of  computers  by  those  outside  the 
professional  realm  of  computing.  The  study  of  HCI,  a multidisciplinary  area,  has  contributed  substantially  to 
the  knowledge  of  how  humans  interact  with  computers.  The  results  of  such  study  have  enabled  the  developers 
of  computer  systems  to  deliver  products  that  are  usable  and  that  effectively  support  the  user  community. 

Coupled  with  the  development  and  discovery  in  the  area  of  HCI  are  significant  advances  in  technology 
that  add  to  the  complexity  of  the  User  Interface  Design.  In  the  last  ten  years,  issues  in  UID  have  changed  from 
those  related  to  relatively  simple  text-based  non-color  screens  to  those  related  to  interfaces  which  combine 
video  and  live  images  with  text,  sound,  graphics  and  color.  Furthermore,  the  user  is  now  able  to  interact  with 
the  computer  application  system  through  these  interfaces  and  UID  also  involves  interaction  issues.  Currently, 
virtual  reality  and  intelligent  agents  are  also  being  incorporated  into  the  user  interface.  Even  though,  the 
understanding  of  HCI  has  advanced  rapidly  over  the  same  time  span,  technology  has  advanced  to  a greater 
extent.  Consequently,  HCI  needs  to  consider  the  implications  of  new  technology  on  the  development  of  user 
interfaces.  The  basic  knowledge  and  understanding  that  already  exists  regarding  user  interface  design  can 
provide  a good  foundation  upon  which  to  build  a basis  for  research  and  study  of  factors  affecting  users  as  they 
interact  with  interfaces  that  utilize  this  new  technology. 

Color  is  one  of  the  factors  that  has  been  researched  and  studied  both  as  a physical  property  in  the 
world  of  physics  and  as  a factor  in  the  two-dimensional  world  of  user  interface  design.  In  fact  many  guidelines 
for  using  color  in  user  interface  design  are  available  that  aid  the  designer  in  developing  user  interfaces. 
Consequently,  effective  use  of  color  in  screen  design  has  been  increasing  and  is  much  more  prevalent  currently 
that  it  was  at  the  beginning  of  the  decade.  Color  is  used  to  assist  in  formatting  or  organizing  visual  displays,  to 
relate  single  elements  into  groups  or  to  separate  items  from  each  other.  It  is  used  as  a visual  code  to  help  the 
user  identify  or  give  meaning  to  items.  It  is  used  to  attract  the  user’s  attention,  to  increase  the  display’s  appeal 
to  the  user  and  to  portray  natural  objects.  Experience  with  the  use  of  color  in  many  fields  as  well  as  that  in  user 
interface  design  has  provided  the  user  interface  designer  with  much  knowledge  about  what  works  well  with 
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colors  and  what  doesn’t  and  what  the  potential  problem  areas  of  color  usage  are.  However,  "There  is  a lot  to 
understand  about  color  in  user  interface  design."  (Shubin,  et.  al.,  1996).  This  is  especially  true  since  the 
perception  of  color  is  highly  subjective  and  is  affected  by  both  one’s  physical  capability  to  perceive  color  and 
one’s  expectations  about  color  use  based  on  culture  and  learning  or  experience. 

Navigation  is  another  of  the  factors  that  has  been  researched  in  the  two-dimensional  world  of  user 
interface  design.  Interface  designers  have  gained  some  insight  into  the  navigational  patterns  of  different  groups 
of  users  based  on  user  characteristics  (Beasley  & Vila,  1992;  Beccue  & Vila,  1992,  1994;  Rendon,  1994).  As 
designers  work  with  three-dimensional  worlds,  they  need  to  know  whether  the  findings  related  to  two- 
dimensions  or  to  real  world  behaviors  still  apply.  In  reviewing  literature  related  to  navigation  patterns  in  HCI,  it 
was  found  that  the  research  involving  navigation  in  three-dimensional  environments  is  concerned  with  research 
which  focuses  on  navigational  awareness,  spatial  ability  or  wayfinding  (Satalich,  1995).  The  two-dimensional 
research  on  navigation  patterns  more  closely  aligns  with  wayfinding  than  with  the  other  two  areas. 

Regian  and  Shebilske  (1990)  conducted  a wayfinding  study  in  a virtual  reality  environment.  The 
environment  was  a virtual  maze  consisting  of  three  stories  with  four  rooms  on  each  floor.  Each  of  the  twelve 
rooms  contained  a unique  color-coded  geometric  object.  Subjects  were  taken  on  three  different  verbally 
directed  tours,  then  given  one  hour  of  free  time  to  explore  the  environment.  After  this  learning  time,  the 
subjects  were  given  3 way  finding  tasks  each  of  which  consisted  of  finding  a specified  object.  They  were 
instructed  to  take  the  shortest  path  to  the  destination.  The  authors  found  significant  differences  between  the 
number  of  rooms  visited  by  the  subjects  in  each  task  and  the  number  of  rooms  visited  by  a random  walk 
algorithm  for  each  task.  While  this  experiment  addresses  the  ability  of  subjects  to  learn  navigational  skills  in  a 
virtual  environment,  it  does  not  indicate  how  they  learn  i.e.  by  self-exploring,  by  verbally  guided  tours  or  by  a 
combination.  The  study  did  not  investigate  navigational  patterns  that  might  have  been  followed  by  the  subjects. 

Other  research  has  shown  that  people  do  not  make  random  decisions  on  which  path  to  take  at  each 
decision  point  in  wayfinding  (Peponis,  et.  al.,  1990).  In  fact,  Peponis  observed  that  people  tend  to  avoid 
backtracking  and  to  find  visual  vantage  points  or  places  that  provide  the  best  visual  access  to  other  areas  of  the 
environment. 

One  informal  study  investigated  toolsets  for  wayfinding  in  virtual  environments  (Darken  and  Sibert, 
1 993).  The  following  toolsets  that  were  available  to  study  participants:  landmarks,  flying,  spatial  audio  markers, 
visual  markers  (breadcrumbs),  coordinate  feedback,  grid  navigation  and  either  track-up  map-view  or  North-up 
map-view.  The  use  of  the  tools  by  subjects  was  examined  for  three  different  search  conditions.  One  search 
condition  was  to  explore  and  learn  the  environment.  Another  condition  was  to  locate  a known  object.  The  third 
condition  was  to  locate  an  object  when  information  about  its  location  was  known.  Through  observation  it  was 
noted  that  the  subjects  used  the  different  tools  in  a variety  of  ways  and  not  always  as  expected.  For  example, 
breadcrumbs  were  used  more  often  as  landmarks  than  as  a trail. 

Satalich  (1995)  studied  navigation  and  wayfinding  in  a virtual  reality  environment.  The  VR 
environment  was  a U-shaped  building  containing  39  separate  rooms  and  over  500  objects.  The  building  had 
some  wide  open  spaces  and  other  distinctive  areas.  Paintings  and  objects  were  located  so  that  they  could  be 
used  as  landmarks.  Subjects  explored  the  building  in  one  of  the  following  ways:  self-exploration;  active  guided 
where  subject  followed  a pre-determined  path;  and  passive  guided  where  subject  was  moved  through 
environment  at  a constant  speed  with  no  interaction.  The  control  group  only  studied  a map  of  the  building;  they 
did  not  explore  the  environment.  In  addition  to  the  type  of  exploration,  subjects  either  had  access  to  a map  for 
five  minutes  before  entering  the  building  or  they  did  not.  A third  factor  was  whether  or  not  subjects  had  a map 
during  the  exploration  phase  of  the  experiment. 

The  tasks  assigned  to  the  subjects  in  the  Satalich  study  to  investigate  navigational  awareness  utilized 
the  following  three  measures:  orientation,  route  estimation  and  Euclidean  estimations.  The  results  indicated  that 
having  a map  before  entering  the  virtual  environment  improved  performance  but  exploring  the  virtual 
environment  did  not.  In  addition  subjects  were  given  two  wayfinding  tasks.  In  the  first  wayfinding  task,  finding 
a room  with  a particular  object  in  it,  there  were  no  differences  between  the  control  group  and  those  in  the 
different  VR  exploration  groups.  Those  subjects  who  had  access  to  a map  prior  to  entering  the  VR  environment 
performed  significantly  better  than  those  who  had  had  no  access  to  a map  prior  to  entering  the  VR  environment. 
The  results  suggested  that  having  a map  during  the  wayfinding  task  may  have  interfered  with  wayfinding. 
Results  for  the  second  wayfinding  task,  finding  a specific  laboratory  when  the  two  main  routes  were  blocked 
with  fire  walls,  showed  no  significant  differences  across  experimental  conditions.  However,  the  difference 
between  the  performance  of  the  control  group  and  those  groups  who  had  no  access  to  any  maps  was  significant. 
Satalich  observed  that  the  behavior  of  the  subjects  clearly  indicated  that  decisions  on  which  way  to  go  were  not 
random.  Specifically  she  noted  that  "subjects,  even  when  lost,  would  not  backtrack  without  purpose." 
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The  limited  research  into  navigation  patterns  of  users  in  virtual  reality  or  3-dimensional  worlds  raises 
many  questions.  The  current  research  seems  to  indicate  that  navigation  and  wayfinding  performance  is  not 
clearly  understood  and  that  more  investigation  is  needed  in  order  to  gain  a better  understanding  of  the  issues 
involved.  Additionally,  little  research  has  been  done  on  navigational  patterns  although  what  research  is 
available  seems  to  indicate  that  navigational  patterns  do  exist.  One  factor  which  may  affect  a user’s 
navigational  behavior  is  color.  In  order  to  investigate  how  color  may  impact  the  way  that  users  navigate  in  a 3- 
dimensional  world,  one  needs  to  be  able  to  control  and  change  the  color  of  the  3D  environment  and  to  track  an 
individual  user’s  movements  through  the  color-coded  virtual  reality  space. 

This  paper  describes  a color-coded  navigation  tool  that  facilitates  researchers  efforts  in  studying  the 
effects  that  color  may  have  on  navigational  decisions  in  a virtual  reality  environment.  The  tool  allows  the 
design  and  exploration  of  color-coded  three-dimensional  worlds  and  provides  for  navigational  tracking.  The 
structure  of  the  virtual  world  is  a never-ending  building  composed  of  many  empty  rooms.  The  tool  enables 
researchers  to  customize  experiments  that  allow  for  the  testing  of  a subject’s  navigational  and  spatial  behavior 
in  a virtual  world  as  color  is  changed. 


Tool  Use  for  Experiment  Design  & Methodology 

In  this  phase  of  on-going  research  into  navigational  preferences  of  computer  application  users,  a tool 
was  designed  and  constructed  that  will  be  used  to  conduct  research  into  the  effect  that  color  has  on  navigational 
decisions  in  a virtual  reality  environment.  The  Color-Coded  Virtual  Reality  Navigation  (CVRN)  research  tool 
will  allow  the  user  to  explore  a three-dimensional  environment.  The  researcher  can  specify  colors  and  track  the 
user’s  navigational  decisions  when  the  user  is  performing  an  assigned  task.  Thus  the  study  of  color  and  user’s 
reaction  to  it  will  be  enabled  through  the  use  of  CVRN  to  customize  a VR  environment  and  to  track  the  user’s 
actions. 

The  current  version  of  CVRN  incorporates  the  following  components:  the  VR  Environment  Setup,  the 
Demographic  Survey,  and  the  VR  Experiment  Interface. 

The  Demographic  Survey  enables  the  researcher  to  gather  and  record  information  about  the  subjects 
such  as  age,  gender,  level  of  education  and  national  origin.  The  VR  Environment  Setup  module  permits  the 
researcher  to  specify  color  distribution  among  the  different  rooms  that  compose  the  virtual  world.  The  color 
distribution  is  determined  by  means  of  an  RGB  triangular  model  that  permits  multiple  orientations  and  control 
for  saturation  and  lightness.  The  setup  module  enables  the  researcher  to  place  landmarks  within  the  world  to 
support  wayfinding  tasks.  While  the  subject  is  navigating  the  environment  and  completing  the  tasks,  his 
movements  will  be  tracked  and  the  time  spent  in  each  room  logged.  The  VR  Experiment  Interface  module  is  the 
world  with  which  the  user  will  be  interacting  to  accomplish  a specified  task.  Thus,  CVRN  will  facilitate 
research  efforts  into  the  effect  that  color  has  on  navigational  decisions  by  providing  a flexible  tool  that  can 
easily  be  customized  to  conduct  relevant  experiments. 

CVRN  allows  researchers  to  track  navigational  patterns  of  each  subject  when  performing  a task. 
Experiment  designers  can  use  CVRN  to  create  various  customized  color-coded  environments,  to  set  landmarks 
and  to  prepare  tasks  and  instruments  to  track  navigators  decisions.  Analysis  of  data  collected  via  the  use  of 
CVRN  is  still  the  responsibility  of  the  researcher. 

Researchers  through  their  color-coded  designs  can  evoke  a subject’s  natural  route  decision  making,  if 
any.  For  example,  a subject  navigating  the  world  of  an  achromatic  design  may  choose  a different  navigational 
route  than  when  navigating  in  a chromatic  world.  Within  the  chromatic  world,  a researcher  could  also  use  the 
tool  to  shed  some  light  into  the  effect  that  hue,  saturation  and  lightness  may  have  on  route  selection. 

The  tool  will  allow  the  researcher  to  verify  that  color  is  the  determining  factor  in  route  selection 
patterns  since  the  color  orientation  or  distribution  within  a world  can  be  changed  and  controlled. 

CVRN  depicts  a building  with  an  infinite  number  of  rooms,  each  having  four  doors.  Each  room  can  be 
set  up  with  a distinctive  hue,  saturation  and  lightness.  If  the  subject  is  free  to  explore  the  created  environment, 
what  would  the  navigation  trail  be?  Are  there  any  demographics  that  correlate  to  navigational  decisions  within 
particular  color-coding  (i.e.,  gender,  age,  right  handedness,  etc.)?  Are  there  some  color-coded  designs  that 
affect  the  sense  of  orientation  more  than  others?  In  achromatic  or  chromatic  designs,  how  many  turns  can  a 
subject  recall  before  complete  disorientation  occurs?  The  measures  recorded  by  CVRN  such  as  navigator 
location,  time  stamps,  number  and  type  of  turns  and  landmark  finding  provides  the  researcher  with 
electronically  recorded  information  that  will  make  organization,  .manipulation  and  analysis  of  data  more 
efficient  and  accurate. 
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Technical  Description  of  the  Tool:  Color-Coded  Virtual  Reality  Navigation 

The  Color-coded  Virtual  Reality  Navigation  research  tool  (CVRN)  is  comprised  of  three  modules,  the  KR 
Environment  Setup,  the  Demographic  Survey,  and  the  VR  Experiment  Interface.  All  the  modules  are  accessible 
by  pull-down  menus  from  the  CVRN  Main  Screen.  These  modules  were  implemented  using  Microsoft  Visual 
C++,  Autodesk  3D-Studio,  Microsoft  Direct3D,  Microsoft  Visual  Basic  and  Microsoft  Access. 

The  VR  Environment  Setup  includes  the  capability  to  select  the  color  distribution  based  on  a traditional 
RGB  triangular  color  model  and  to  place  landmarks  within  the  Virtual  World.  The  VR  Environment  Color 
Distribution  Setup  Screen  (Figure  1)  includes  the  color  orientation  and  the  saturation  and  lightness  control 
settings  The  Landmark  Placement  Screen  (Figure  2)  includes  drag  and  drop  operation  for  landmark  placement 
within  the  grid  that  represents  the  virtual  world.  The  subject’s  starting  navigation  point  in  the  world  is 
represented  by  a red  square  in  the  middle  of  the  grid.  Originally,  the  landmark,  which  is  represented  by  a blue 
square  is  located  outside  the  grid.  This  object  can  be  placed  anywhere  within  the  gnd.  In  Figure  2,  it  has  been 
placed’ in  the  upper  quadrant  of  the  grid.  One  can  use  the  arrows  in  the  right  hand  side  to  show  a path  to  the 
object.  Currently,  this  capability  is  available  only  in  the  VR  Environment  Setup. 

The  Demographic  Survey  module  is  invoked  from  the  CVRN  Main  Screen’s  Add  pull-down  menu. 
The  Demographic  Survey  Screen  is  a form  for  the  collection  of  demographic  information  from  each  subject 
who  participates  in  the  study.  After  completing  the  survey,  the  subject  proceeds  to  the  VR  Experiment 

Interface.  , • , ■ c ■ u 

The  VR  Experiment  Interface  depicts  a virtual  world  consisting  of  a building  with  an  infinite  number 

of  color-coded  rooms  in  compliance  with  the  VR  Environment  Setup  parameters.  See  Figure  3. 
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Figure  1:  VR  Environment  Color  Distribution  Setup  Screen 
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Figure  3:  The  VR  Experiment  Interface 
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The  VR  Experiment  Interface  supports  two  window  resolutions:  320x200  and  640x480  pixels.  The 
lower  resolution  of  the  VR  environment  can  be  used  on  less  powerful  machines  to  gain  speed  and  provide  for 
smother  animation.  The  higher  resolution  can  be  used  on  more  powerful  machines  to  provide  a larger  VR  field. 
The  navigator  can  traverse  the  world  using  the  arrow  keys  or  look-around  (rotate  in  the  Y-axes)  using  the 
mouse.  Once  that  the  subject  completes  the  specified  task  the  experiment’s  data  is  saved  in  a database  for  future 
analysis. 


Summary 

CVRN  was  designed  and  constructed  to  support  research  about  the  effects  of  color  on  navigational 
decisions  in  a virtual  reality  environment.  Research  in  this  general  area  is  hampered  by  the  need  to  control  or 
stabilize  many  factors  while  allowing  the  investigator  to  examine  variables  of  interest  and  by  the  need  to  take  or 
track  many  measurements  of  the  activity  generated  during  experiments.  In  order  to  investigate  how  users 
navigate  in  a 3D  world,  one  needs  to  be  able  to  track  an  individual  user’s  movements  through  a virtual  reality 
space  when  that  user  is  attempting  to  accomplish  a specific  task.  CVRN  provides  a template  and  data  collection 
capabilities  which  enable  the  researcher  to  conduct  such  investigations  in  an  effective  and  efficient  manner. 
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Abstract:  The  Ohio  University  College  of  Business  has  been  using  problem-based  learning 
approaches  to  instruction  for  the  last  13  years.  In  March  of  1997,  the  MBA  Without  Boundaries 
program  was  launched  as  a distance  offering  using  a project-based  action  learning  format. 
Participants  progress  through  the  program  as  a cohort  and  work  on  nine  projects  over  the  course 
of  two  years.  Participants  come  to  campus  for  4 to  8 days  between  projects  for  debriefing  on  the 
project  Just  completed  and  to  be  introduced  to  the  next  project. 

Participants  work  in  teams  of  four  or  five  using  Lotus  Domino  as  the  conferencing  system  for 
conducting  their  work.  Faculty  members  monitor  the  participant  work  and  offer  two  types  of 
commentary.  First,  there  are  infrequent  coaching  comments  offered  if  a team  is  seen  to  be 
particularly  bogged  down  — coaching  focuses  on  the  problem-solving  tactics  and  group  processes 
being  employed.  Second,  faculty  members  offer  project  reviews  offered  to  the  entire  class.  This 
is  a "how  is  the  class  doing"  perspective  — commenting  on  similarities  and  differences  in  both 
strategies  and  the  products  that  are  being  produced. 

In  this  paper  we  will  present  an  overview  of  the  program  and  then  focus  on  two  issues:  1) 
instructor  coaching  and  feedback  and  2)  participant  interaction  using  Domino.  Goals,  alternative 
strategies,  and  problems  in  providing  feedback  will  be  discussed,  with  particular  attention  to  the 
difficulty  of  balancing  the  felt  need  to  "teach"  the  participants  when  they  encounter  difficulties 
with  the  goal  of  maintaining  learner  ownership.  Participant  conferencing  strategies  and  the 
protocols  they  develop  will  be  discussed.  Both  samples  of  their  conferencing  and  data  from 
interviews  with  the  participants  will  be  presented.  For  both  participant  conferencing  and  faculty 
coaching  we  will  pay  particular  attention  to  how  the  strategies  evolve  across  projects. 


Program  Overview 

The  Ohio  University  College  of  Business  has  been  using  problem-based  learning  approaches  to  instruction 
for  the  last  13  years.  For  more  detailed  descriptions  of  the  evolution  of  the  program  development  see  Milter  & 
Stinson,  1995  and  Stinson  & Milter,  1996.  In  March  of  1997,  the  MBA  Without  Boundaries  program  was  launched 
as  a distance  offering  using  a project-based  action  learning  format.  This  followed  over  a year  of  development  work 
between  the  core  design  faculty  team  and  an  external  review  board  comprised  of  key  individuals  from  corporations 
and  learning  centers.  Selected  participants  must  have  at  least  2-4  years  of  increasingly  responsible  positions  and 
must  be  sponsored  by  their  company.  The  actual  average  experience  level  in  the  first  class  begun  in  March  1997  is 
about  6 years.  The  average  experience  level  for  the  second  class  begun  in  February  1998  is  about  8 years. 

The  MBA  Without  Boundaries  program  uses  an  action-learning  format  with  a theoretical  base  in  cognitive 
constructivism,  which  places  the  learner  into  exactly  the  type  of  projects  and  work  situations  that  he/she  will  face  as 
a leader  of  the  knowledge-age  organizations  of  the  2 1 st  century.  Participants  learn  basic  business  concepts  in  the 
context  of  their  use,  maximizing  their  ability  to  both  recall  and  apply  those  concepts  as  they  move  back  into  the 
work  world.  Participants  also  develop  the  skills  (communication,  collaboration,  teamwork)  and  the  personal 
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characteristics  (initiative,  creativity,  personal  responsibility)  that  are  becoming  so  necessary  for  success. 
Participants  develop  a high  level  of  comfort  with  information  technology  as  they  regularly  access  information 
through  the  resources  of  the  internet,  collaborate  electronically  over  time  and  space,  and  develop  and  make 
professional-level  computer-driven  presentations. 

The  program  targets  9 major  projects.  The  projects  tend  to  be  large  macro  problems  that  address  business 
holistically.  Within  each  project  are  multiple  smaller  problems  that  participants  must  address  to  manage  the  total 
learning  problem.  Participants  construct  their  knowledge  of  business  practices  by  working  their  way  through  the 
problems.  Participant  learning  is  aided  by  the  ability  to  access  appropriate  content  on  a just-in-time  basis. 
Participants  learn  content  at  a time  when  it  will  be  most  useful  to  them  in  their  management  of  the  learning 
problems.  While  some  of  the  problems  are  designed  to  challenge  individuals  separately,  most  of  them  are  designed 
to  be  approached  by  collaborative  learning  groups. 

While  focusing  on  more  macro  problems  and  working  with  larger  groups,  the  action-learning  process 
employed  is  a derivative  of  Reiterative  Problem-Based  Learning,  which  was  developed  by  Howard  Barrows 
(Barrows,  1985),  and  follows  closely  the  concepts  of  cognitive  constructivism  (Savery  and  Duffy,  1994)  and 
cognitive  apprenticeship  (Collins  et.  al.,  1990).  (For  a more  complete  description  of  our  use  of  the  action  learning 
process,  see  Stinson,  1990,  and  Milter  and  Stinson,  1995.) 

The  program  begins  with  an  intensive  one- week  residency  (Sunday  1pm  through  Sunday  Noon).  The 
participants  then  return  to  their  homes  and  work  assignments  not  to  see  each  other  again  for  3 months.  After  3 
months  of  online  interaction  and  collaboration,  program  participants  meet  for  an  intensive  weekend  (Thursday  1pm 
through  Sunday  Noon).  Each  project  begins  and  ends  in  a residency.  Participants  progress  through  the  program  as  a 
cohort  and  work  on  nine  projects  over  the  course  of  two  years.  Participants  come  together  every  three  months 
during  the  two-year  program;  three  extended  weekends  and  one  full  week  each  year. 

The  faculty  team,  in  regular  consultation  with  the  external  review  board,  designs  seven  of  the  nine  projects. 
Two  of  the  projects  are  individual  projects  where  participants  design  and  implement  projects  of  personal  interest 
(these  projects  often  benefit  their  company).  Each  project  is  developed  to  meet  specific  learning  outcomes.  In  order 
to  provide  a foundation  for  the  total  program  design,  meta  outcomes  are  developed  by  faculty  members  in  discussion 
with  executives  and  futurists.  Content-to-action  outcomes  are  more  specific  learning  goals  that  participants  need  to 
learn  in  order  to  fulfill  project  expectations.  The  meta  outcomes  and  content-to-action  outcomes  for  the  MBA 
program  are  as  follows: 


Meta  Outcomes 

(Approached  through  total  program  design) 

A holistic  understanding  of  business  and  the  environment  in  which  business  functions. 

A degree  of  technical  expertise,  but  not  a sub-optimal  functional  orientation. 

Knowledge  of  current  business  practices  and  factors  influencing  those  factors,  but  not  knowledge  alone.  The 
ability  to  apply  the  knowledge  effectively  is  critical. 

A proactive  orientation  with  the  ability  and  self-confidence  to  take  initiative  and  function  independently. 

The  ability  to  tolerate  ambiguity  and  the  ability  and  self-confidence  to  clarify  own  role  through  normal 
interaction  with  others,  internal  and  external  to  the  organization. 

The  ability  to  manage  self  - time,  resources,  priorities,  stress. 

Communication  skills,  with  particular  emphasis  on  the  ability  to  communicate  ideas  succinctly  in  oral  form  and 
electronically.  While  traditional  report  writing  remains  important,  it  will  be  less  so  in  the  future. 

The  ability  to  work  effectively  using  electronic  information  technology. 

The  ability  to  collaborate  effectively  - the  ability  to  influence  others  and  be  influenced,  to  listen  and  understand, 
to  work  out  differences  so  they  do  not  become  destructive  conflicts,  to  use  diverse  perspectives,  cultures,  and 
expertise  to  maximize  effectiveness. 
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Content-to-Action  Outcomes 

(Approached  through  multiple  projects) 

Analyze  an  industry  and  develop  a reasonable  foresight  for  the  industry. 

Collect  competitive  intelligence,  analyze  emergent  strategy,  and  determine  competencies,  strengths  and  weaknesses. 
Analyze  a company,  identify  important  competencies  and  propose  what  the  company  should  do  now  to  compete 
effectively  in  the  future. 

Identify  a market  (customer  value  added)  for  a new  technological  development 
Develop  a proposal  to  a venture  capitalist  to  obtain  capital  for  a new  venture. 

Develop  a business  plan  to  effectively  implement  a new  venture. 

Select,  from  among  alternatives,  the  most  promising  of  new  ventures. 

Develop  a program  to  develop  and  bring  a new  product  or  service  to  market. 

Identify  an  appropriate  manufacturing  system  and  prepare  a macro  design  of  the  system. 

Analyze  a process  and  propose  a redesign  that  minimizes  waste  and  maximizes  quality. 

Select  from  among  alternatives,  the  most  appropriate  operational  improvement  techniques  or  programs,  and  design 
the  implementation  strategy. 

Develop  global  sourcing  strategies  and  programs. 

Define  a market,  identify  demand  determinants  for  a product  or  service,  and  estimate  demand. 

Determine  and  appropriate  price  for  a product  or  service. 

Develop  and  implement  a promotion  strategy  and  program  for  a product  or  service. 

Create,  interpret,  and  use  financial  statements 

Analyze  cost  structure  and  determine  cost  of  products  and  services,  using  different  cost  accounting  approaches  (full, 
marginal,  ABC). 

Using  ratio  analysis  and  other  appropriate  techniques,  determine  the  financial  strength  of  a company. 

Determine  capital  needed  to  support  business  activities  and  determine  how  to  best  obtain  that  capital. 

Demonstrate  sensitivity  to  the  disparate  cultures,  values,  and  behaviors  of  non  US  business 

Perform  country  analysis,  determine  potential  for  products  and  services  in  non-US  countries,  and  identify 
appropriate  modes  of  entry. 

Operate  effectively  within  the  complexity  of  global,  simultaneous  domestic  and  international,  business  operations. 
Identify  and  consider  significant  foreign  trade  issues,  including  laws,  regulations,  and  currency  exchange. 

Perform  basic  business  research,  primary  and  secondary,  using  appropriate  resources. 

Incorporate  ethical  and  social  issues  in  process  of  making  business  decisions. 


Each  project  has  a set  of  "content-to-action"  learning  outcomes  that  are  specifically  linked  to  actions  required 
during  the  projects.  These  outcomes  are  shared  with  participants  and  jointly  agreed  upon  at  the  outset  of  each 
project  during  the  reiteration  on  the  first  day.  For  the  two  individual  projects  participants  develop  a set  of  personal 
learning  outcomes  they  wish  to  target. 


Online  Collaboration  and  Learning 

During  the  3-month  span  between  the  residencies  participants,  faculty,  and  outside  resources  collaborate 
online.  Lotus  Notes  Domino  is  the  software  used  to  provide  a platform  for  teams  to  work  together  and  faculty  to 
interact  with  the  teams,  individuals,  and  other  faculty.  Most  faculty  members  initially  believe  it  takes  a lot  of  time  to 
do  it  well.  In  actuality  only  30  minutes  per  day  checking  on  the  database  is  usually  sufficient  to  read  entries  for  a 
class  of  25  participants.  It  may  take  a bit  longer  to  respond  to  many  entries.  Faculty  members  find  that  they  must 
keep  up  or  the  entries  can  pile  up. 

Like  other  forms  of  groupware,  Lotus  Domino  offers  key  enablers  to  assist  with  the  collaborative  work  of 
learning.  The  elements  of  groupware  as  described  by  Lotus  (1995)  suggests  that  it  must  combine  three  essential 
elements  to  be  viable: 

• A reliable,  scaleable  and  secure  distributed  document  database.  The  database-centric  model  gives  users  the 
ability  to  assemble,  share,  and  manage  the  rich  variety  of  documents  vital  to  conducting  business  -business 
plans,  contracts,  orders,  product  information,  calendars  and  to-do  lists,  multimedia  presentations,  faxes, 
scanned  images  and  video  clips. 

• An  integrated  messaging  system.  Groupware  users  must  be  able  to  take  advantage  of  a messaging 
infrastructure  to  send  and  receive  electronic  mail  and  move  documents  through  a workflow  process. 
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• A rich  application  development  environment  Developers  must  be  able  to  rapidly  build  portable  and 
scaleable  strategic  applications,  which  span  document-oriented  and  structured  database  information. 
Although  most  of  the  previous  use  of  Domino  has  made  use  of  its  ability  to  serve  as  a document  database,  the 
integrated  messaging  system  and  application  potential  is  being  presently  evaluated  for  future  use.  The  present 
configuration  of  Domino  for  the  MBA  Without  Boundaries  program  contains  two  major  sections:  common  and 
project-specific  databases. 

Common  databases  are  available  on  the  Domino  Intranet  Homepage  include: 

• announcements  (information  about  the  program  in  general), 

• address  book  (with  pictures  and  information  on  participants  and  faculty), 

• general  discussion  (about  non-project  matters  or  tangential  issues), 

• learning  resources  (collection  of  selected  best  of  the  breed  for  knowledge  enhancement), 

• technology  issues  (trouble-shooting  area  and  FAQ  items), 

• internet  resources  (online  knowledge  adding  items  and  skills  enhancement  areas), 

• faculty/staff  discussion  (private  area  for  faculty  to  work  together  in  development  and  evaluation). 

Project  databases  on  the  Domino  Intranet  Homepage  include: 

• project  discussion  (specific  discussion  regarding  the  current  project), 

• learning  activities  (learning  issues  and  questions  of  the  week  assignments  for  individual  assessment), 

• database  for  each  team  (collaboration  on  the  current  project). 

The  Intranet  Homepage  also  contains  links  to  a help  document,  chatrooms,  earlier  project  databases,  and  the  College 
and  University  Homepages. 

During  some  projects  a database  for  "Question  of  the  Week"  is  used.  In  this  database  each  week  a faculty 
member  poses  a relevant  question  for  participants  to  respond  to  during  a two-week  period.  Responses  are  evaluated 
on  the  basis  of  their  ability  to  move  the  discussion  forward  by  adding  new  insights  or  appropriate  synthesis 
reflections.  Participant  evaluation  for  each  project  is  split  between  40-60%  on  team  deliverables  and  individual 
performance.  The  individual  performance  is  comprised  of  performance  on  learning  outcomes,  reports,  questions  of 
the  week,  and  post-project  assessments. 


Instructor  Coaching  and  Feedback 

Part  of  coaching  is  the  ability  to  offer  new  information  as  needed  by  individual  learners.  During  the  3- 
month  distance  period  of  each  project,  faculty  members  provide  learning  module  material  via  "streaming  video"  by 
placing  5-10  minute  clips  onto  the  Domino  database.  Participants  use  RealPlayer  to  play  the  video,  audio,  and/or  the 
PowerPoint  slideshow  to  view  these  "video  connect"  segments.  This  allows  individuals  with  various  learning  styles 
to  stay  in  their  strong  suit  as  they  digest  new  information.  Participants  can  question  any  module  material  online  in 
the  project  discussion  database.  They  can  view  the  video  connect  as  often  as  they  wish  merely  by  clicking  on  the 
icon  for  the  clip. 

Another  element  of  coaching  is  the  provision  of  feedback  about  the  process  or  the  content  being  dealt  with 
by  the  learners.  Faculty  members  react  to  the  entries  made  by  participants  in  the  various  individual  and  team 
databases  and  deliver  feedback  on  a continuous  basis.  Each  project  has  a faculty  member  serving  as  the  project 
director.  The  project  director  is  responsible  for  tracking  individual  entries  required  in  the  learning  issues  and 
"question  of  the  week"  databases.  Each  team  is  assigned  a faculty  mentor  who  is  responsible  for  tracking  the  team's 
deliverables  during  the  project.  These  activities  also  occur  via  the  Domino  databases. 

Another  area  for  coaching  is  in  the  learning  issues  database.  As  learning  issues  are  defined,  they  are  stated 
as  questions.  As  participants  find  information  and  draw  conclusions  related  to  the  learning  issues  they  make  entries 
in  the  database.  These  are  reviewed  and  all  participants  and  faculty  members  can  express  comments.  These  entries 
are  one  source  of  evaluation  of  participant  learning. 


Participant  Interaction  Using  Domino 

Participants  use  the  various  databases  to  submit  information,  ask  questions  of  faculty  and  outside  resources, 
discuss  issues  with  colleagues  and  faculty,  socialize,  and  collaborate  in  their  project  teams.  After  a couple  of  days 
most  individuals  become  adjusted  to  using  the  asynchronous  mode  of  interaction  provided  by  the  Domino  database 
for  these  activities.  There  are,  of  course,  times  when  synchronous  discussion  is  preferred.  When  the  team  members 
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need  to  reach  consensus  about  a future  direction  or  the  value  of  a specific  timely  piece  of  information,  it  is 
advantageous  to  meet  together  at  the  same  time.  The  database  contains  links  to  a chatroom  (Microsoft's 
NetMeeting)  that  allows  a team  to  hold  a synchronous  session.  The  chat  software  records  these  sessions  and  the 
chat  log  is  placed  in  the  team's  database  for  tracking  and  archival  purposes. 

The  Domino  database  allows  participants  to  move  quickly  through  various  discussion,  teamwork,  data 
sources,  chat,  and  other  areas  of  collaboration.  It  allows  participants  to  sort  by  topic,  date,  person,  or  category.  It 
also  allows  for  searching  by  key  words.  Because  it  is  on  the  web,  URLs  that  are  entered  immediately  become 
hotlinks  to  the  source  sites.  This  makes  it  convenient  for  sharing  detailed  information  on  an  "as  needed"  basis.  The 
ability  to  function  on  an  "as  needed"  basis  is  pervasive  in  the  learning  approach  used  by  the  MBA  Without 
Boundaries  program.  The  use  of  this  web  technology  enhances  the  ability  to  deliver  learning  modules,  information, 
tutoring,  coaching,  instruction,  and  collaborative  space  when  the  learner  needs  them.  Recognizing  the  differences  in 
learning  and  living  needs  among  a group  of  learners,  it  would  be  difficult  to  offer  such  a program  without  the 
support  of  a software  tool  like  Lotus  Domino. 


Reflections  from  MBA  Without  Boundaries  Graduates 


One  recent  graduate  expressed  the  following  reflections  ...  The  MBAWB  program  has  provided  an 
opportunity  to  define  and  develop  my  individual  and  business  competencies  necessary  to  compete  in  today’s 
challenging  business  environment.  For  the  past  two  years  a virtual  relationship  among  twenty-one  individuals  has 
been  established. 

Sharing  personal  knowledge  and  experiences  has  provided  a unique  learning  experience.  It  has  changed  my 
perception  of  how  I learn.  This  was  a struggle  initially  because  of  past  industry  and  organizational  cultural 
experiences.  Being  able  to  trust  and  share  ideas  on  team  projects  that  offer  solutions  to  complex  problems  has  been 
truly  enlightening. 

During  the  course  of  the  program  we  have  completed  nine  projects.  Seven  of  the  projects  were  team 
orientated  and  the  other  two  projects  conducted  individually.  These  projects  have  provided  the  framework  for  a 
continuous  learning  process  that  will  remain  with  me  for  a lifetime.  The  combination  of  participating  in  concept 
discussions,  learning  issues  and  assigned  projects  has  enhanced  my  skill  set  and  self-confidence  to  a level  that  will 
allow  me  to  achieve  my  desired  goals. 

This  can  be  best  demonstrated  by  the  project  outcomes  presented  to  third  party  clients  involved  with  the 
Global  Account  Organization  and  Euro  Currency.  These  dynamic  learning  experiences  are  unsurpassed  and 
provided  me  with  a tremendous  amount  of  confidence. 

Furthermore,  the  reflection  process  provides  an  opportunity  to  assess  the  past  two  years  and  quantify  those 
competencies  that  were  learned  and  will  make  me  an  effective  business  professional.  First  and  most  exciting  to  me  is 
the  development  of  my  technical  competencies.  Turning  on  the  computer  was  about  the  extent  of  those  skills  before 
entering  the  program.  To  elaborate,  my  ability  to  utilize  MS  Word,  Excel  and  Power  Point  software  is  definitely  a 
milestone.  In  addition,  I am  intimately  involved  in  the  startup  of  a computer  product  and  service  company  with  the 
purpose  of  positioning  it  to  take  advantage  of  the  electronic  business  revolution  transcending  today’s  business  and 
economic  practices. 

Another  recent  graduate  offered  these  reflections  . . . The  hard  stuff  is  not  the  hard  stuff  Actually,  the  soft 
stuff  is  the  hard  stuff.  The  hard  numbers  are  scorecards  documenting  how  well  you  managed  the  soft  stuff.  People 
are  the  ultimate  challenge  in  business.  How  do  you  motivate  them?  How  can  you  minimize  distractions?  How  can 
you  encourage  them  to  adapt  continuously?  How  can  you  persuade  them  to  be  loyal  when  the  commitment  is  not 
returned? 

Leadership  still  matters.  The  forest  and  the  trees  both  matter.  Even  at  the  tree  level,  leaders  must  instruct, 
encourage  and  persuade.  As  information  proliferates,  someone  must  decide  what  is  more  important  and  which 
priority  is  next.  Effective  organizational  leaders  produce  other  effective  leaders.  Ineffective  leaders  stifle  the  entire 
organization. 

Technology  is  a double-edged  sword.  Technology  both  raises  and  lowers  barriers.  Technology  is  a 
wonderful  place  to  hide.  Technology  is  a kind  of  wind  tunnel.  Whether  one  flies  or  crashes  depends  on  how  well  one 
is  designed. 

Learning  is  the  easy  part.  The  challenge  is  in  the  doing.  Asking  the  right  questions  is  important.  And 
figuring  out  what  they  are  is  no  piece  of  cake. 
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Conclusion 


We  began  working  with  this  innovative  and  evolving  platform  for  learning  13  years  ago.  At  that  time  we 
made  the  commitment  that  if  we  ever  thought  we  had  it  right  it  would  be  time  to  get  out  of  the  business.  We  do  not 
believe  we  have  it  right.  We  have  never  offered  the  same  program  two  years  in  a row.  At  each  residency  we  hold  a 
feedback  session  with  the  participants  with  the  goal  being  to  improve  the  program.  We  keep  an  open  discussion  in 
our  databases  for  the  same  purpose.  We  are  constantly  searching  for  better  ways  to  assist  learning. 

It  is  not  easy  to  find  faculty  members  willing  to  step  out  on  such  an  ambiguous  and  often  tenuous  platform. 
When  we  do  find  them  we  cherish  their  involvement  as  much  as  they  fully  cherish  the  experience.  Participants  in 
the  program,  including  faculty  members,  gain  first-hand  experience  in  what  it  takes  to  develop  a true  learning 
community.  These  experiences  usually  transfer  immediately  to  other  learning  situations.  It  becomes  a true  spiraling 
of  knowledge  and  learning  how  to  learn. 
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Abstract;  Over  the  last  3 years,  the  Open  School,  a service  of  the  Open  Learning  Agency  focussing 
on  K-12  curriculum,  has  undertaken  a comprehensive  project  to  develop  an  end-to-end  course 
development  and  delivery  framework  based  on  SGML/XML.  This  paper  discusses  the  scope  and 
direction  of  the  project  and  outlines  the  processes  involved  in  developing  an  Open  School  course, 
from  Instructional  Design  to  delivery  in  multiple  media.  Benefits  of  the  approach  and 
implementation  issues  are  examined. 


Introduction 

Open  School,  a service  of  the  Open  Learning  Agency  focusing  on  K-12  curriculum,  provides  distance  education 
courses  and  resources  for  students  in  the  province  of  British  Columbia,  Canada.  Traditionally,  Open  School  has 
produced  print-based  correspondence  courses.  However,  over  the  past  five  years,  we  have  experimented  with 
distributed  learning  technologies  including  proprietary  client-server  applications,  web-based  course  delivery,  online 
student  collaboration,  database-driven  competency-based  systems,  and  immersive  virtual  environments. 

We  have  learned  that  to  meet  the  needs  of  an  ever  expanding  and  changing  client  base.  Open  School  must  embrace 
new  models  of  delivery  and  services  beyond  our  core  business  as  a producer  of  correspondence  courses.  Open 
School  courses  and  resources  need  to  be  more  flexible  in  their  assemblage,  customizable  for  individual  clients, 
portable  across  different  media,  and  accessible  to  as  many  students  as  possible. 

The  critical  issues  facing  Open  School  with  regard  to  developing  courses  for  alternate  media,  different  types  of 
learners,  and  varied  instructional  contexts  are  cost  and  time.  It  is  simply  not  practical  to  produce  a unique  course  for 
every  combination  of  learner,  delivery  environment  and  instructional  context. 


Features  of  a New  Framework 

In  1996  Open  School  began  work  on  a new  framework  for  course  development  and  delivery.  The  initial  list  of 
desirable  attributes  included  the  ability  to: 


• create  courses  from  Instructional  Design  to  delivery 

• create  assessment  types  and  mechanisms 

• identify,  associate  and  categorize  resources  within  the  course  architecture 

• create  custom  views  of  materials  for  different  types  of  learners 

• store  and  manage  parts  and  pieces  of  course  materials 

• create  a user-defined  catalog  of  the  parts  and  pieces  of  courses 

• output  to  both  print  and  web,  as  well  as  to  other  media 

In  1996,  there  were  no  “off-the-shelf’  products  that  could  address  this  list  in  its  entirety.  Open  School  proceeded 
with  an  internal  research  and  development  project  to  create  a solution. 

One  of  the  more  important  innovations  in  publishing  and  knowledge  management  today  is  structured  information, 
specifically  SGML  (Standard  Generalized  Markup  Language:  ISO  8879-1986).  In  the  summer  of  1996,  Open 
School’s  Research  and  Development  team  began  work  on  an  SGML-based  course  development  and  delivery 
framework. 

This  project  was  completed  in  1998.  As  a result.  Open  School  decided  to  pursue  a structured  information  approach 
to  course  development  and  delivery.  At  that  time,  few  educational  institutions  were  moving  in  this  direction.  We 
saw  this  as  a competitive  advantage,  an  opportunity  to  innovate  and  inform.  The  emergence  of  the  XML  standard  in 
late  1 997  further  validated  our  approach. 


Structured  Information  Standards 

SGML  is  an  international  standard  for  text  and  document  processing.  It  is  a metalanguage  that  provides  a common 
syntax  and  notation  to  create  specialized  markup  languages  for  specific  domains — in  our  case,  distributed  learning. 
Any  markup  language  that  conforms  to  the  standard  can  be  used  with  a wide  assortment  of  SGML/XML  tools  and 
applications. 

A goal  of  an  SGML-based  markup  language  is  to  explicitly  define  the  structure  of  documents.  This  allows 
computers  to  recognize  structural  components  and  metadata  in  a document,  structures  that  traditionally  have  only 
been  implied  by  presentational  cues.  For,  example,  in  most  documents,  a glossary  term  might  be  identified  by 
presenting  it  in  bold  or  italics.  To  the  human  viewer,  this  convention  is  easily  understood.  But  for  a computer  there 
is  nothing  significant  about  this  string  of  characters  apart  than  the  fact  that  its  is  bolded  or  italicized,  which  may  also 
be  the  style  used  to  delimit  an  author’s  name  or  the  title  of  a work.  In  an  SGML  document,  the  glossary  term  would 
be  delimited  by  an  explicit  tag.  This  tag  informs  the  computer  that  it  has  encountered  a glossary  term,  which  can 
then  trigger  specific  processing  of  this  data.  For  example,  in  print  we  may  wish  to  collect  all  glossary  terms  and 
present  them  at  the  beginning  of  a lesson.  If  the  same  document  is  published  on  the  Web,  specific  processing  for 
glossary  terms  might  enable  hyperlinking  each  occurrence  with  its  corresponding  definition  in  a database. 

The  formal  separation  of  content  from  presentation  allows  content  to  be  presented  in  multiple  media  from  a single 
source.  The  styles  used  in  a specific  medium  are  applied  to  the  structural  or  semantic  tags  only  in  final  presentation. 
The  content  is  not  bound  to  a specific  presentational  format  and  can  be  repurposed  with  greater  ease. 

Other  emerging  standards  influenced  the  design  of  the  Open  School  course  development  and  delivery  framework. 
Many  Open  School  courses  refer  to  or  include  various  learning  resources.  In  order  to  classify  and  categorize 
resources  in  our  materials,  we  needed  robust  structures  for  managing  metadata.  We  looked  to  and  were  inspired  by 
the  Dublin  Core  metadata  set.  We  found,  however,  that  local  extensions  were  required  to  describe  the  wide  variety 
of  resources  in  our  course  material. 

Similarly,  the  IMS  Project’s  metadata  work  was  another  standard  we  looked  to  for  insight.  Although  the  IMS 
metadata  standard  is  well  thought  out,  it  had  yet  to  be  realized  in  SGML  or  XML.  We  have  and  will  continue  to 
incorporate  IMS  metadata  mappings  to  our  own  metadata  structures. 
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Finally,  the  new  XML  standard  itself  promises  those  of  us  with  SGML  assets  the  ability  to  easily  interchange 
content  with  others.  But  XML  has  drawbacks  that  result  from  its  “newness”;  parts  of  the  XML  family  of  standards 
are  yet  to  be  finalized.  While  this  does  not  inhibit  our  use  of  XML  for  basic  needs,  it  does  restrict  its  use  for  the 
more  complex  requirements  of  our  documents  and  overall  system,  which  are  better  addressed  today  by  the  more 
venerable  SGML  and  related  ISO  standards  (like  Hytime  and  Topic  Maps).  This  will  undoubtedly  change  in  the  near 
future  and  we  are  preparing  ourselves  to  move  entirely  to  XML  when  possible. 


The  Open  School  Graduation  Program  Project 

The  Open  School  R&D  team  spent  much  of  1997  and  1998  developing  and  prototyping  the  new  course  development 
and  delivery  framework.  In  1998  Open  School  decided  to  implement  the  new  framework  with  14  grade  1 1 and  12 
courses  as  part  of  a new  initiative  called  the  Open  School  Graduation  Program  (OSGP).  The  program  is  designed  to 
provide  high-school  students  with  enough  credits  to  earn  a high-school  diploma,  working  entirely  online  or  in  print. 

The  development  group  consists  of  30  people:  a management  team,  a collection  of  writers,  production  staff. 
Instructional  Designers  and  a small  technical  team.  We  have  identified  and  deployed  software  tools  and 
environments  for  authoring,  middleware  processing,  and  delivery.  Some  of  these  tools  include  Adobe 
FrameMaker+SGML,  AIS  Balise,  Microsoft  Internet  Information  Server  4.0  and  Microsoft  SQL  Server. 


Process 

The  following  outlines  the  basic  process  of  bringing  a new  course  into  the  SGML/XML  framework,  from 
Instructional  Design  through  to  delivery  in  multiple  media.  The  process  is  composed  of  four  overall  phases,  which 
are  decomposed  into  a number  of  incremental  stages. 


Phase  One:  Requirements  Capture  and  Analysis 

For  each  course,  a small  team  is  gathered  to  address  the  business  case,  collect  requirements,  and  prepare  for  the 
Instructional  Design  phase.  The  most  important  document  we  begin  with  is  the  British  Columbia  Ministry  of 
Education’s  Integrated  Resource  Package  (IRP).  It  is  a document  that  outlines  provincial  standards  for  curriculum, 
assessment  strategies,  instructional  strategies,  prescribed  learning  outcomes,  and  recommended  learning  resources. 
Because  its  primary  audience  is  in-classroom  teachers,  the  IRP  needs  to  be  reviewed  and  localized  for  distance 
education.  At  this  time  we  also  review  existing  course  material,  technology  standards,  and  feedback  from  the  field. 

Completion  of  Requirements  Capture  and  Analysis  Phase 

The  requirements  capture  stage  provides  the  initial  inputs  to  the  next  phase,  the  development  of  a highly  structured 
Instructional  Design  Plan.  Initial  learner  profiles  are  identified  at  this  stage:  baseline  attributes  of  learner  motivation, 
level  of  instructional  support  and  technology  access,  informing  the  ensuing  development  by  establishing 
determinants  for  alternate  assessment  and  instructional  strategies  as  well  as  providing  criteria  on  which  to  determine 
the  appropriateness  of  resources. 


Phase  Two:  Instructional  Design 

An  SGML  Document  Type  Definition  (DTD)  supports  each  individual  stage  in  the  Instructional  Design  phase.  An 
SGML  DTD  is  the  formal  set  of  rules  and  element  definitions  that  govern  what  structures  are  allowed  or  required  in 
a document. 

Stage  One 


420 


The  first  stage  of  Instructional  Design  involves  creating  a robust  outline  of  the  course  according  to  an  SGML 
Document  Type  Definition  (DTD).  The  course  outline  consists  of  a hierarchy  of  major  subjects  or  topics,  abstracts 
for  content  to  be  covered,  and  facets  or  themes  (important  subtopics  that  may  pervade  the  course  but  are  not 
represented  in  the  major  subject/topic  hierarchies).  The  initial  Instructional  Design  DTD  requires  a four  level 
hierarchy  of  nodes:  Course,  Module,  Section,  Lesson.  Learning  Outcomes  are  positioned  at  the  section  and  lesson 
level  to  serve  as  determinants  for  formal  and  informal  assessment. 

The  Stage  One  Instructional  Design  Plan  is  an  idealized  mapping  of  subject/content  and  learning  outcomes.  It  does 
not  mean  that  this  hierarchy  is  the  only  “view”  of  the  course  a student  will  see.  It  is  however,  a solid  foundation  on 
which  to  build  materials  that  can  be  reassembled  during  presentation  and  delivery. 

Stage  Two 

Stage  Two  of  the  Instructional  Design  phase  involves  resource  identification  and  association.  Relevant  learning 
resources  are  identified  and  associated  with  the  overall  course  architecture  created  in  Stage  One.  Where  appropriate, 
resources  are  further  decomposed  by  annotation,  describing  their  scope  and  relevance  to  particular  nodes  in  the 
course  hierarchy.  This  stage  allows  many  different  resources,  textbooks,  articles,  online  links,  video  clips  and  CD- 
ROMS  to  be  appropriately  situated  within  a course. 

Stage  Three 

During  Stage  Three,  assessment  strategies  are  designed  for  both  formal  and  informal  assessment  at  the  section  and 
lesson  level.  Each  assessment  strategy  is  written  for  one  or  more  learner  profiles.  They  allow  Instructional  Designers 
to  indicate  resource  dependencies  and  declare  the  type  of  assessment  to  be  used  in  the  activity  or  assignment. 

Exit  criteria  for  this  stage  requires  all  sections  and  lessons  to  contain  assessment  strategies  for  all  learner  profiles 
designed  for  the  course.  Designers  are  encouraged  to  develop  multiple  alternatives  for  each  profile  if  the  schedule 
allows. 

Stage  Four 

An  optional  fourth  stage  (in  development)  of  the  Instructional  Design  process  is  intended  to  provide  designers  with 
the  opportunity  to  develop  alternate  instructional  strategies  at  each  level  of  the  course  hierarchy.  This  stage 
facilitates: 

• Alternate  subject/topic  hierarchies  at  all  levels 

• Custom  sequencing  of  materials  (requires  existing  structured  content) 

• Custom  assembly  of  materials  for  specialized  learner  profiles  (i.e.  learners  with  special  needs) 

• Experimental  approaches  to  learning 

Completion  of  the  Instructional  Design  Phase 

The  completed  Instructional  Design  Plan,  an  SGML  document,  literally  becomes  the  supporting  framework  for 
course  authoring.  All  of  the  information  in  the  design  plan  is  included  in  the  authoring  environment.  This  ensures 
that  the  authoring  process  is  tied  directly  to  the  original  specifications  laid  out  by  the  Instructional  Design  team. 


Phase  Three:  Course  Authoring 

As  with  the  Instructional  Design  phase,  the  Authoring  phase  is  supported  by  an  SGML  DTD  for  every  stage. 

Stage  One 

Assignments  and  activities  are  developed  in  response  to  assessment  strategies  articulated  in  the  Instructional  Design 
plan.  Authors  create  assignments  and  activities  from  standard  architectures  defined  in  the  Authoring  Document  Type 
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Definition.  Criteria  for  assessment  (answer  keys,  marking  rubrics,  feedback  mechanisms,  etc)  are  required  and 
therefore  incorporated  in  the  architectural  template  for  activities  and  assignments. 

Stage  Two 

In  Stage  Two,  topic  explorations  are  created  based  on  the  Instructional  Design  specifications  for  each  lesson.  Topic 
explorations  are  composed  of  text,  graphics  and  multimedia  elements  appropriate  for  one  or  more  learner  profiles. 
Alternate  topic  explorations  are  encouraged  for  each  lesson. 

The  result  of  Stage  Two  is  a collection  of  peer  topic  explorations  and  activities  written  for  one  or  more  learner 
profiles.  At  this  stage  it  becomes  possible  to  select  the  appropriate  topic  explorations,  activities  and  resources  for  an 
individual  learner  profile  from  the  overall  course  architecture. 

Stage  Three 

Overview  material  and  front  matter  are  created  at  this  stage.  This  may  include  learner  profile-dependent  advance 
organizers,  transitional  components,  and  support  materials. 

Stage  Four 

An  optional  fourth  stage  (in  development)  provides  authors  the  opportunity  to  address  alternate  instructional 
strategies  described  in  Stage  Four  of  the  Instructional  Design  document. 

Completion  of  the  Course  Authoring  Phase 

The  completion  of  the  Authoring  Phase  provides  Open  School  with  a highly  structured  representation  of  a course  in 
its  entirety.  In  fact  it  is  a superset  of  all  possible  course  instances  that  can  be  derived  by  individual  learner  profiles. 


Phase  Four:  Delivery 

The  Fourth  Phase  of  the  course  development  and  deliveiy  framework  focuses  on  producing  course  instances  for 
delivery.  At  this  point  in  the  development  process,  the  advantages  of  SGML/XML  come  to  the  fore.  We  bring  an 
arsenal  of  SGML/XML  tools  and  techniques  to  bear  on  the  structured  representations  of  the  course  materials  to  drive 
rendering  for  presentation.  Each  individual  SGML/XML  element  and  its  attributes  are  leveraged  by  processing  tools 
and  associated  with  presentational  rules.  Using  stylesheets  and  transformation  rules,  we  produce  both  print  and 
online  instances  of  each  course.  It  is  important  to  recognize  that  this  process  is  almost  entirely  driven  by 
SGML/XML  programming  tools  and  veiy  little  hand  tooling  is  required;  a single  set  of  stylesheets  and 
transformation  rules  can  be  applied  to  all  14  courses. 


Implementation  Notes 

Implementation  of  the  OSGP  project  has  not  been  easy.  All  members  of  the  team  have  had  to  revisit  best  practices, 
“golden  rules”  and  established  “ways  of  seeing”.  The  technical  obstacles  encountered  during  development  were 
minor  compared  to  cultural  ones. 

Never  before  has  Open  School  simultaneously  produced  14  courses,  let  alone  in  one  year.  The  pressures  and 
constraints  of  this  rigorous  timeline  have  been  extreme.  However,  without  these  constraints  it  is  unlikely  we  could 
have  reached  the  consensus  required  to  move  forward.  Constraint  and  compromise  were  crucial.  A case  in  point: 
during  the  initial  research  and  development  phase,  which  preceded  the  OSGP  project,  it  took  many  months  to  fully 
agree  on  an  operative  definition  of  learning  outcomes. 

The  desktop  publishing  mindset  has  been  hard  to  overcome.  In  the  past,  Open  School  authors  had  individual  control 
over  the  presentation  of  the  materials  they  developed.  The  new  approach  required  them  to  forfeit  some  of  this 
control  to  the  larger  team.  Standard  ways  of  presenting  course  components  were  developed  through  consensus  and 
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implemented  post-authoring  in  the  production  phase.  In  the  beginning,  team  members  struggled  with  this  separation 
of  content  and  presentation.  Once  they  saw  the  published  results  of  the  first  stages  and  how  their  material  began  to 
take  shape  in  print  and  online,  much  of  their  discomfort  was  alleviated. 


Benefits  of  approach 

As  we  enter  the  final  stages  of  the  development  process,  the  benefits  of  the  approach  are  beginning  to  be  realized. 
Overall,  we  have  gained  a much  better  understanding  of  what  is  actually  in  our  course  materials  and  a more  intimate 
appreciation  of  both  their  outstanding  and  not-so-outstanding  qualities. 

A major  benefit  of  our  approach  is  the  structural  consistency  evident  across  all  OSGP  courses.  This  consistency 
allows  the  production  team  to  leverage  global  stylesheets  and  transformation  rules  for  delivery  in  print  and  online. 

The  increased  efficiency  in  the  production  phase  has  provided  more  time  to  evaluate  the  overall  quality  and  design 
of  our  course  materials.  Because  we  produce  a published  product  at  each  stage  of  Instructional  Design  and 
authoring,  the  opportunities  for  formative  evaluation  and  field  review  are  many;  no  longer  do  these  activities  have  to 
wait  until  the  course  is  fully-formed. 

A common  vocabulary  for  course  components  and  Instructional  Design  elements  has  greatly  improved 
communication  across  teams.  Instructional  Designers,  course  authors,  management,  and  production  staff  now  have  a 
common  framework  for  workflow  management. 

We  have  yet  to  realize  the  full  potential  of  our  collection  of  structured  course  materials.  In  the  future  we  anticipate 
being  able  to  produce  new  products  and  services  from  this  collection.  The  ability  to  repurpose  will  extend  far 
beyond  this  year’s  print  and  online  instances. 


The  Future 

In  the  next  year,  we  will  be  acquiring  an  SGML/XML  object  repository  to  manage  and  store  our  SGML/XML 
courses.  This  will  provide  us  with  all  of  the  benefits  of  a database  as  well  as  an  environment  optimized  for 
SGML/XML  documents. 

The  advent  of  XML-enabled  browser  technologies  like  Internet  Explorer  5.0  will  allow  us  to  deliver  our  structured 
content  directly  to  the  Web.  In  the  coming  year  we  hope  to  leverage  this  technology  to  provide  an  online 
environment  for  our  clients  to  create  customized  instances  of  our  courses  tailored  to  their  individual  needs. 

Work  will  continue  on  OSGP  courses  after  delivery  in  September.  The  superset  of  course  components  will  grow  to 
accommodate  additional  learner  profiles  and  instructional  strategies.  It  is  our  intent  to  move  to  an  evergreen  revision 
cycle  for  the  new  course  development  and  delivery  framework. 

In  the  coming  years,  we  anticipate  that  the  establishment  of  metadata  and  markup  technology  standards  in  education 
will  allow  us  to  integrate  and  exchange  our  structured  course  content  with  other  institutions  and  organizations.  We 
feel  we  are  well  positioned  to  take  advantage  of  these  opportunities. 
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Video  Streaming  Medical  Grand  Rounds:  Convergence  of  Real  Time,  Any 

Time,  WAN  and  Internet  Delivery 

(An  early  implementation  of  an  important  distance-learning  technology) 


Jonathon  D.  Levy,  Amelia  Ellsworth 
Cornell  University  Office  of  Distance  Learning 
DistanceLeaming@comell.edu 


Abstract:  Distance-learning  programs  commonly  characterized  as  being  either  synchronous 
(real-time)  or  asynchronous  (on-demand),  video  or  Web-based,  traditional  or  non-traditional 
are  now  blending  one  into  the  other.  A single  word — ^‘‘convergence” — now  characterizes  the 
cutting  edge  of  distance  learning,  allowing  pedagogy  to  fully  drive  the  selection  of  technology 
and  distance-learning  formats.  In  the  following  pages,  we  will  discuss  the  issues  of 
interactivity,  flexibility,  integration,  and  isolation  in  the  context  of  various  synchronous  and 
asynchronous  technologies.  We  will  describe  the  recent  solution  Cornell  University  has 
discovered  by  using  NetShow  video  streaming  software  in  combination  with  HTML  Web 
frames  and  PowerPoint.  By  merging  these  media,  Cornell  can  deliver  Medical  Grand 
Rounds  over  local  and  wide  area  networks  in  both  real  time  and  any  time  scenarios, 
combining  video  with  Web-based  presentation  and  a traditional  format  with  a highly- 
unconventional  delivery  format.  The  presentation  will  feature  a demonstration  of  Grand 
Rounds.  Two  of  these  demonstrations  may  be  viewed  on  the  Internet  at: 
http://www.dl.cornell.edu  (select  “courses”). 


References 

This  is  a report  of  an  early  implementation  of  an  important  use  of  technology;  however,  no  empirical 
examination  has  been  conducted.  Only  anecdotal  evidence  is  reported  herein. 

This  program  may  be  viewed  on  the  Web  at 

http://www.microsoft.com/windows/windowsmedia/techshowcase/  (Microsoft  site;  select  “Streaming  Media 
and  Synchronized  Events”  and  scroll  down) 

or  at  http://www.odl.comell.edu/pr/cumc/default.htm  where  the  presentations  may  be  viewed  at  three  different 
bandwidths. 

The  Media  Player  may  be  downloaded  at  http://www.dl.comell.edu/download.htm. 

The  Problem  of  Distance:  spatial  and  temporal 

Comelfs  Weill  Medical  College  is  located  on  the  Upper  East  Side  of  Manhattan.  There  are  4,500  doctors 
affiliated  with  twenty  Cornell-networked  hospitals  within  the  five  boroughs  of  New  York  who  need  to  attend 
Grand  Rounds  presentations — updates  in  the  various  fields  of  specialty — at  the  Weill  Medical  College.  Grand 
Rounds  are  traditionally  conducted  as  a lecture  presentation  by  top  medical  doctors.  The  lectures, 
accompanied  by  35mm  slides,  focus  on  the  latest  developments  and  breakthroughs  in  medicine.  It  is  very 
important  that  practicing  physicians  at  these  hospitals  attend  those  Grand  Rounds,  but  the  physicians  are 
scattered  amongst  the  20  hospitals,  and  it  is  difficult  for  them  to  attend  the  Grand  Rounds  in  person. 
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For  this  particular  application,  time  shifting  proved  far  more  important  than  space  shifting.  For  most  doctors, 
travel  time  plus  presentation  time  results  in  a lost  half-day  of  office  and/or  hospital  time  and  very  few  are  able 
to  take  advantage  of  the  Grand  Rounds  opportunity.  So  Cornell  began  using  the  streaming  components  of  the 
Windows  Media  technologies  to  deliver  the  lectures.  The  Microsoft  Media  Player  software  runs  with  any  web 
browser  on  a Pentium  machine  with  at  least  32  Megs  of  Ram.  A Macintosh  solution  is  now  available  in  Beta. 

The  Grand  Rounds  were  initially  offered  live  on  the  Internet  to  participating  physicians  and  were  archived  for 
asynchronous  access,  so  that  physicians  could  watch  them  at  a later  time  at  their  convenience.  Interactive 
technologies  were  developed  to  allow  doctors  to  “converse”  with  the  presenter  during  the  synchronous  version, 
using  a “mailto”  button  on  the  Web  page  for  e-mail  messages.  However,  initial  experience  showed  that  none 
of  the  doctors  was  interested  in  a synchronous  presentation  when  an  asynchronous  alternative  was  available,  so 
synchronous  delivery  was  dropped,  and  the  entire  Grand  Rounds  series  was  offered  only  asynchronously. 

Each  Grand  Round  lasts  about  an  hour.  It  is  extremely  important  for  the  physicians  to  be  able  to  see  the  35mm 
slide  image  and  to  hear  the  presenter’s  voice  very  clearly.  Cornell  synchronizes  PowerPoint  presentation  slides 
with  the  actual  video.  The  slides  appear  in  a separate  window  on  the  same  screen.  As  doctors  watch  a 
presentation  they  not  only  see  and  hear  the  presenter,  but  as  the  presenter’s  image  changes  on  the  screen  in  the 
presentation  room,  the  image  changes  on  the  computer  screen  of  the  person  who  is  watching  at  a later  time  and 
at  a distance.  This  allows  Cornell  to  show  very  high-quality  images  accompanying  the  video,  such  as  medical 
slides,  cells,  x-rays,  photographs  of  patients,  or  other  medical  data  that  the  doctors  need  to  see  as  the 
presentation  continues.  The  Media  Player  is  configured  like  a video  recorder,  allowing  doctors  to  stop  the 
presentation  if  interrupted,  and  later  restart  it  from  the  point  where  it  was  paused.  The  video,  audio,  and 
synchronized  presented  image  (slide)  all  cue  up  and  restart  automatically. 

One  of  the  defining  features  of  a learner  in  a distance  scenario  is  their  separation  from  the  origin  of 
information.  This  separation  can  be  represented  by  a spatial  or  temporal  distance.  Learners  in  these  scenarios 
sometimes  find  it  challenging  to  motivate  themselves  and  continue  with  the  learning  process.  In  this  case  the 
audience  for  this  project  is  a highly  motivated  audience  for  whom  delivery  of  the  Grand  Rounds  using  distance 
technology  was,  at  first,  solely  for  convenience.  The  project  was  developed  with  the  idea  that  real  time  and 
any  time/any  where  presentations  gave  the  audience  the  flexibility  they  desired.  Presentation  was  intentionally 
kept  as  close  as  possible  to  the  experience  this  audience  was  familiar  with  in  a traditional  lecture  hall. 
Interestingly,  many  of  the  viewers  reported  that  the  Internet  distance-learning  programs  not  only  accomplished 
the  temporal  objectives,  but  that  the  quality  of  the  presentation  actually  was  improved  with  the  technology 
implementation. 


Medical  Grand  Rounds:  Background 

The  Cornell  University  Medical  College  is  responsible  for  a lecture  series  referred  to  as  Grand  Rounds.  These 
lectures  provide  continuing  education  for  doctors  in  hospitals  throughout  the  greater  New  York  area  by 
providing  information  about  recent  advances  in  their  field  of  medicine.  Traditionally,  Cornell  University 
delivered  these  grand  rounds  to  medical  staff  in  their  twelve  network  affiliated  hospitals  by  requiring  the 
audience  to  travel  to  the  Medical  College  and  sit  in  their  lecture  halls.  For  reasons  described  below,  this  was 
not  an  optimal  solution,  but  until  recently  there  was  no  viable  alternative.  With  the  recent  merger  of  the  two 
teaching  hospitals.  New  York  and  Presbyterian  hospitals,  the  number  of  hospitals  in  the  network  rose  to 
twenty,  increasing  the  challenge  for  delivery  of  continuing  medical  education.  When  the  Weill  Medical 
College  began  to  look  for  a new  way  to  deliver  Grand  Rounds,  distance  learning  communications  technology 
was  a natural  place  to  turn. 

The  traditional  Grand  Round  is  an  hour-long  lecture  given  by  a teaching  faculty  in  a specific  field  of  medicine. 
These  presenters  normally  speak  from  behind  a podium  in  very  dim  light  with  the  focus  of  the  lecture  directed 
towards  a front  screen  where  35-millimeter  slides  are  projected  as  each  point  is  made.  In  the  normal  course  of 
a Grand  Round,  an  average  of  75  slides  are  used.  These  slides  can  be  critical  to  understanding  the  lecture  and 
are  often  high  resolution  graphics  which  depict  the  pathology  or  histology  under  discussion.  There  is  very 
little  interaction  between  the  audience  and  the  doctor  making  the  presentation.  At  the  end  of  the  hour,  there  is 
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a brief  question  and  answer  session  and  the  audience  files  out  to  complete  the  review  paperwork  necessary  for 
their  continuing  medical  education  credits. 


The  Proposals 

Two  methods  for  distance  Grand  Round  delivery  were  considered: 

• Purchase  video-teleconferencing  technology  for  each  of  the  20  network  hospitals  and  deliver  Grand 
Rounds  in  real  time  at  384kps  to  all  sites.  The  presenter  would  be  on  camera  and  the  slides  would  be 
sent  either  as  an  image  taken  off  the  projection  screen  or  document  camera  (possibly  an  RS232  port)  and 
the  quality  would  be  almost  as  good  as  that  in  the  originating  room.  The  rounds  could  be  recorded  and 
played  back  at  a later  time  for  doctors  who  missed  the  presentation. 

• Stream  the  Grand  Rounds  over  the  Internet  using  Netshow  3.0  software.  This  program  allows  an 
audio/video  file  of  the  presenter  to  fill  a frame  in  a web  page  and  stream  at  speeds  as  low  as  28.8  and  as 
high  as  500kps  while  the  second  frame  flips  the  slide  images  synchronized  with  the  presentation. 
Netshow  provides  the  ability  to  capture,  insert,  and  index  time  references  in  the  video  file  to  bring  up  a 
compressed  digital  image  (most  slides  are  usually  no  bigger  than  400k)  at  the  same  time  the  presenter  is 
changing  slides  in  the  traditional  lecture  hall.  These  images  are  scanned  and  cached  ahead  of  time  and 
are  actually  of  a better  resolution  and  quality  than  the  ones  projected  during  the  traditional  presentation 
because  they  are  not  distorted  by  projection.  The  rounds  can  be  streamed  real-time  at  the  same  time  as 
the  actual  presentation,  re-broadcast  at  a later  time  using  multicast  packets  to  conserve  bandwidth,  or 
accessed  anytime  on-demand  at  both  128kps  and  56kps,  allowing  both  LAN  and  dial-up  access. 


The  solution: 

In  this  case,  the  least-expensive  solution  also  turned  out  to  be  the  most  effective  solution.  The  Media  Player  is 
available  to  anyone  at  no  cost  (free  download  from  Microsoft).  All  that  is  required  to  “attend”  Internet  Grand 
Rounds  is  a computer  and  an  Internet  connection.  Because  doctors  valued  their  time  more  than  higher 
bandwidth  (the  highest  being  attendance  at  the  face-to-face  presentation,  followed  by  ISDN  video),  the  lower 
bandwidth  Internet  solution  was  selected.  Interestingly,  this  solution  was  also  evaluated  as  “better  than  being 
there”  by  some  of  the  participating  physicians.  This  is  due  to  the  combination  of  technologies  that  allowed 
access  to  the  video,  audio,  and  synchronized  presentation  material  at  times  most  convenient  to  the  physicians. 

The  lack  of  interactivity  in  this  solution  is  atypical  of  Cornell’s  distance  learning  applications,  but  for  this 
particular  program  it  was  deemed  acceptable,  since  the  face-to-face  Grand  Rounds  offer  only  a short  period  for 
questions,  and  these  are  included  within  the  Internet  streamed  presentation. 
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Abstract:  This  paper  begins  via  a synthesis  of  theories  targeted  at  innovative 
practices  in  the  field  of  adult  learning.  Following  a review  of  the  current  literature, 
the  author  discusses  different  strategies  and  methods  used  to  enhance  adult  learning. 
Separation  of  teacher-centered  and  learner-centered  methodologies  are  examined. 
Environmental  conditions  and  use  of  technology  are  also  items  factored  into  the 
discussion.  Comparative  assessments  of  methodologies  used  at  educational 
institutions  and  corporate  learning  centers  are  reported.  The  efficacy  of  both  specific 
and  generic  theoretical  constructs  is  examined  in  light  of  professional  adult  learners 
and  the  requisite  skills  demanded  by  their  career  positions.  In  other  words,  what 
theories  are  best  suited  to  carry  learning  practices  into  the  future  to  best  suit  the  needs 
of  active  learning  professionals? 


Review  of  Adult  Learning  Constructs 

To  synthesize  a field  of  theories  is  a fairly  presumptuous  task  today.  There  are  probably  as  many 
theories  as  there  are  stars,  and  we  seem  to  be  finding  more  of  them  every  day.  The  purpose  of  this 
“synthesis”  is  to  recognize  the  key  elements  displayed  in  several  cogent  theories  regarding  adult  learning. 
Drawing  on  the  thinking  of  Rorty  (1991)  as  well  as  vonGlaserfeld  (1989),  Savery  and  Duffy  (1994) 
describe  the  basic  principles  for  learning.  These  include  recognition  that:  1)  understanding  is  in  our 
interactions  with  the  environment,  2)  cognitive  conflict  or  puzzlement  is  the  stimulus  for  learning  and 
determines  the  organization  and  nature  of  what  is  learned,  and  3)  knowledge  evolves  through  social 
negotiation  and  through  the  evaluation  of  the  viability  of  individual  understandings.  Such  principles  are 
rooted  not  only  in  learning  theory  but  also  in  innovation.  Without  innovative  thinking  our  understanding 
would  never  be  able  to  match  the  changing  nature  of  the  environment.  All  innovation  must  begin  with  an 
inquisitiveness  that  is  grounded  in  personal  cognitive  conflict.  And  finally,  innovation  to  be  useful  must 
at  some  point  pass  the  tests  of  social  negotiations. 

Barrows  (1997)  has  suggested  responsibilities  of  the  successful  innovator  beyond  the  creation  and 
use  of  new  methods  for  educating  people.  The  most  important  of  these  responsibilities  is  that  the 
innovation  must  provide  a substantial  contribution  to  the  learning  process  for  all  those  involved.  In  other 
words,  the  innovator  must  understand  the  learning  outcomes  (what  is  expected  of  the  learner)  and  how  the 
innovation  relates  to  these  outcomes.  A second  responsibility  is  that  the  innovator  actively  evaluate 
effectiveness  of  the  innovation.  It  is  not  enough  to  get  satisfaction  scores  from  the  users.  The  evaluation 
must  target  whether  the  learning  outcomes  have  been  positively  impacted  by  the  innovation.  A third 
responsibility  of  the  innovator  is  to  promote  the  innovation  through  publication  channels  so  that  word  gets 
out  to  other  educators  and  learners.  This  activity  matches  the  need  to  respond  to  social  negotiations  as 
mentioned  above. 

A final  responsibility  of  the  innovator  is  to  remain  flexible  and  open  to  change.  By  definition 
this  means  to  continue  to  learn.  It  is  too  easy  for  innovators  to  begin  to  believe  their  innovation  is  the 
final  piece  to  the  puzzle.  They  believe  that  with  their  innovation,  the  light  at  the  end  of  the  tunnel  can 
finally  be  seen  as  the  sun  on  the  horizon  and  not  an  oncoming  train.  Learning  innovators  must  take 
special  precaution  to  remain  learning  innovators.  This  means  they  must  remain  learners  and  not  to  close 
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down  channels  for  questioning.  A colleague  of  mine  involved  in  educational  innovation  and  I made  a 
commitment  to  each  other  about  a dozen  years  ago  that  if  we  ever  thought  we  were  doing  it  right  it  would 
be  time  to  get  out  of  the  business.  When  it  comes  to  learning,  the  business  focus  concerning  continuous 
improvement  makes  all  the  sense  in  the  world. 


Challenges  for  Educators 

Continuous  improvement  suggests  an  unflinching  drive  toward  flexibility  and  growth,  or  in  a 
word  ...  learning.  Educators  might  be  best  advised  to  follow  Charles  Handy’s  (1996)  prescription  for 
learning  as  we  attempt  to  enhance  our  abilities  to  facilitate  growth  in  others.  His  learning  cycle  suggests 
that  all  learning  begins  with  an  inquisitive  mind  (questions)  that  leads  to  the  development  of  concepts 
(ideas)  which  must  then  be  evaluated  (testing)  before  a period  of  contemplation  (reflection).  This 
“questions,  ideas,  testing,  reflection”  process  is  not  unlike  that  suggested  in  a more  recent  work.  The 
Power  of  Mindful  Learning,  by  Ellen  Langer  (1997).  Langer  presents  methods  for  putting  into  practice  a 
true  learning  focus  that  transcends  rote  memorization  techniques  typical  of  traditional  teaching  strategies. 

Rote  memorization,  or  learning  out  of  context,  is  a way  that  many  of  us  were  weaned  on 
education  and  learning.  Some  of  us  have  gotten  very  good  at  it  and  have  graduated  into  leadership 
positions  in  the  education  field.  One  question  that  arises  out  of  this  fact  is  whether  we  should  have 
different  learning  strategies  to  facilitate  learning  in  children  than  those  we  have  for  helping  adults  to 
learn.  Although  much  research  has  been  done  to  differentiate  between  the  two,  the  verdict  is  still  out. 
Although  there  are  some  differences  between  the  learning  processes  of  children  and  adults,  such 
differences  may  pale  when  comparing  the  differences  between  adults’  individual  learning  styles.  It  does 
appear  often  times  that  it  is  easier  for  children  to  learn  new  things  (like  languages  and  computer  use). 
Perhaps  the  reason  it  appears  more  difficult  for  adults  to  learn  new  things  is  in  part  due  to  the  ingrained 
process  of  learning  out  of  context.  Adults  must  attempt  to  unlearn  not  only  substantive  issues  but  also 
process  methods  in  order  to  grasp  new  meanings. 

Much  of  this  can  be  attributed  to  the  designs  of  educational  institutions.  According  to  James 
Botkin,  co-author  of  No  Limits  To  Learning,  most  formal  schooling  systems,  with  some  notable 
exceptions,  resist  change  in  their  objectives  and  style  of  learning  and  education.  Botkin  further  suggests 
that  “school,  on  a worldwide  basis,  is  hopelessly  out  of  date.  In  North  America,  despite  widespread 
recognition  of  the  inadequacy  of  K-12  and  higher  education,  the  response  to  date  has  been  limited  to 
‘more’  for  the  future  of  what  has  not  worked  in  the  recent  past.  More  hours,  more  homework,  more 
science  and  math  have  been  the  guiding  principles  rather  than  new  teamwork,  focus  on  values,  or  holistic 
learning.  No  country  in  the  world  has  been  able  to  figure  out  how  to  divest  itself  of  a system  built  for 
another  age  and  another  time  and  to  start  afresh  in  redesigning  formal  education”  (Botkin,  1996). 


New  Learning  Philosophies  and  Methods 

Many  educational  institutions  are  doing  an  excellent  job  using  an  obsolete  model  for  learning. 
Many  leaders  (including  teachers)  in  these  institutions  “think  of  the  mind  as  a storehouse  to  be  filled” 
when  they  should  be  thinking  of  it  “as  an  instrument  to  be  used”  (Gardner,  1965).  Educational  leaders 
might  do  much  better  to  heed  the  edict  of  Plutarch  that  “the  mind  is  not  a vessel  to  be  filled  but  a fire  to  be 
ignited.”  If  educators  refrained  from  considering  the  mind  as  a storehouse  or  a vessel  to  be  filled,  lifelong 
learning  would  take  on  a whole  new  significance.  With  it,  the  distinctions  between  learning  in  children 
and  learning  in  adults  would  begin  to  blur.  The  major  difference  would  become  one  of  context,  with 
adults  having  more  experiences  from  which  to  draw  ways  for  contextualizing  learning.  Of  course,  if 
educational  institutions  at  the  primary  and  secondary  levels  sought  to  develop  learning  skills  in  children, 
maturity  would  be  measured  by  assessing  a person’s  ability  to  refresh  what  they  have  learned  and  apply 
learning  from  one  context  to  another.  Adults  who  have  experienced  this  approach  to  learning  from  the 
start  might  not  be  bogged  down  trying  to  unlearn  the  process  methods  of  passive  learning  before  joining  in 
as  an  active  participant  in  the  learning  process. 
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Some  educational  institutions  at  the  K-12  and  higher  education  levels  have  already  begun  to 
focus  more  on  strategies  for  involving  active  participants  in  learning  process  in  their  design  and  delivery 
systems  (see  Barrows  & Myers,  1993;  Norman  & Spohrer,  1996;  Schank  & Kass,  1996;  Milter  & Stinson, 
1994;  Schauble  & Glaser,  1996;  Duffy,  1994).  The  similarities  between  innovative  programs  in  medical, 
law,  engineering,  and  business  schools  and  the  innovation  displayed  in  primary  and  secondary  schools  far 
outweigh  the  differences. 

One  of  the  key  similarities  exhibited  in  these  innovative  programs  is  a deliberate  push  for 
learner-centered  methodologies  over  teacher-centered  approaches  (Stinson  & Milter,  1996).  This  is  not  a 
recent  development.  In  the  United  Kingdom  in  the  mid-1980’s  business  leaders  and  educators  developed  a 
plan  to  reform  education  which  resulted  in  a report  titled  “Education  2000.”  Out  of  that  report,  a project 
was  created  to  “shift  the  balance  of  teaching  to  learning;  to  provide  a greater  variety  of  learning 
experiences,  and  to  make  clear  the  responsibility  of  the  learner  for  active  participation  in  the  learning 
process  and  for  achieving  successful  outcomes.” 

With  this  shift  comes  acknowledgment  that  it  is  the  individual  learner  (child  or  adult)  who  must 
claim  responsibility  for  their  own  learning.  No  longer  do  these  innovative  institutions  take  on  the  claim 
for  learning  behavior  in  others.  They  attempt  to  provide  an  environment  where  a learning  community  can 
thrive.  An  environment  where  answers  are  not  as  important  as  questions;  where  getting  to  the  answers  is 
more  important  than  the  answers  themselves.  An  attitude  where  the  concern  for  learning  outweighs  the 
desire  to  imbed  specific  facts  into  the  brains  of  others.  Today,  with  knowledge  doubling  every  5 years  and 
the  half-life  of  most  useful  information  being  about  5 years,  it  seems  silly  to  claim  we  can  provide  the 
requisite  information  for  an  individual’s  future  needs  as  a professional  or  a global  citizen.  Today 
innovative  institutions  push  for  knowledge  discovery,  but  not  without  also  pushing  for  recognition  of  ways 
to  apply  the  knowledge  and  the  importance  of  updating  knowledge. 


The  Processes  of  Learning 

Another  similarity  of  innovation  in  education  across  disciplines  and  age  group  boundaries  is  a 
concern  for  the  design  of  educational  methods.  Whether  you  refer  to  it  as  pedagogy,  andragogy,  or 
learning  methods,  the  question  remains  how  to  best  develop  ways  to  assist  learners  in  their  learning 
processes.  In  many  traditional  situations  the  designing  of  methods  and  processes  is  either  done  by  some 
centralized  mechanism  (school  boards  deciding  which  texts  to  use)  or  “on  the  fly”  while  delivering  the 
course  (many  university  faculty  discover  creative  ways  to  develop  processes  during  the  delivery  process 
itself).  Although  neither  of  these  approaches  is  suspect  on  the  surface,  the  danger  comes  from 
acknowledgment  that  the  design  work  is  unimportant  at  the  delivery  stage. 

That  is,  most  educational  institutions  reward  delivery  of  courses  and  programs  and  assume  the 
design  work  is  merely  a precursor.  Corporate  learning  programs,  on  the  other  hand,  tend  to  place  an 
equal  (or  sometimes  even  greater)  emphasis  on  design  of  learning  materials  and  procedures  as  they  do  on 
the  systems  for  delivery.  Such  corporate  learning  programs  will  pay  substantial  salaries  to  in-house 
developers  of  learning  materials  and  proposed  delivery  systems  then  will  hire  outside  agents  to  actually 
deliver  the  programs.  These  external  agents,  though  important,  are  hired  for  their  abilities  to  facilitate 
learning  environments,  not  for  their  expertise  in  the  specific  substantive  domain.  Attention  to  the 
learning  outcomes  and  substantive  domain  is  under  the  responsibility  of  the  internal  designers. 

Although  the  design  work  tends  to  be  performed  within  these  innovative  corporate  educational 
programs,  these  developers  make  no  presumption  of  total  knowledge  of  substantive  domain  areas.  Such 
programs  regularly  poll  internal  and  external  practitioners  in  the  field  to  take  a read  from  their  expertise 
about  what  are  the  most  significant  learning  elements  for  the  future  of  their  profession.  In  order  to  remain 
current,  this  development  activity  needs  to  be  attended  to  on  a regular  basis.  It  is  not  a “once-and-for-all- 
time”  procedure.  It  is  also  critical  as  designers  seek  to  synthesize  input  from  current  practice  that  they  also 
give  consideration  to  what  the  future  holds  for  the  particular  professional  domain.  Sometimes  this  means 
going  outside  the  view  of  current  practice  which  might  have  a tunnel  vision  perspective. 
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Use  of  Technology  in  Learning 


Use  of  technology  is  an  area  that  tends  to  be  engendered  outside  the  specific  practice  domain. 
People  tend  to  resist  change  until  they  see  the  benefits.  Most  times  the  introduction  of  a new  technology 
causes  short-term  drops  in  performance.  This  tends  to  be  temporary  but  nonetheless  frequently  dissuades 
full  acceptance  of  the  technology  by  practitioners.  And  yet  the  use  of  technology  is  clearly  linked  to  the 
development  and  implementation  of  innovative  learner-centered  educational  programs.  Recent 
publications  give  substantive  examples  and  suggestions  for  inclusion  of  more  technology  in  learning 
environments  in  primary  through  secondary  schools  as  well  as  professional  training  programs 
(Hamalinen,  M.,  Whinston,  A.,  & Vishik,  S.,  1996;  Schank,  R.C.  & Cleary  C.,  1996;  Wilson,  1996).  It  is 
not  a passing  fad  that  there  is  a convergence  of  new  learning  environments  and  new  technologies. 

Although  universities  have  often  led  the  way  to  development  of  new  technologies,  they  have 
frequently  lagged  in  response  to  their  full  utilization.  This  is,  unfortunately,  the  case  as  well  in  use  of  new 
learning  technologies.  According  to  Botkin  (1996),  the  most  promising  action  in  reforming  education  and 
modernizing  learning  is  to  be  found  not  in  universities  but  in  the  international  business  community.  He 
attributes  much  of  the  reason  to  the  fact  that  schools  and  universities  still  do  not  have  the  “financial  or 
innovative  human  resources”  to  carry  out  the  fundamental  changes  required  by  the  challenges  of  the 
future. 


Impact  of  Corporations  on  Learning 

Business  organizations,  on  the  other  hand,  have  begun  to  see  the  value  in  investing  in  education 
for  the  future.  As  reported  by  Botkin  in  1997,  corporate  universities  in  Europe  and  North  America  now 
appropriate  more  funds  to  support  learning  than  all  the  world's  universities  combined.  They  are  growing 
100  times  faster  than  the  university  world,  and  the  lifelong  adult  learning  they  support  is  25  times  larger 
than  the  traditional  university  student  market  (Botkin,  1997). 

Corporate  sponsored  programs  have  begun  to  make  tremendous  impact  in  the  world  of  learning 
at  the  primary  and  secondary  levels.  Motorola  has  for  years  pumped  money  into  the  school  systems  that 
supply  its  plant  in  Chicago  with  employees.  But  as  important  as  the  dollars.  Motorola  has  also  formed  a 
sensitive  and  reciprocal  relationship  with  the  people  who  lead  these  school  systems.  They  are  facing  the 
changing  nature  of  the  world  together  and  have  invested  time  and  energy  as  well  as  money  into  the 
development  and  learning  of  young  people  who  will  be  the  leaders  of  tomorrow;  a few  of  them  leaders 
working  for  Motorola.  And  Motorola/Chicago  story  is  not  the  only  example  of  this  type  of  collaboration. 
According  to  a report  filed  in  1992  by  the  Committee  for  Economic  Development,  there  were  in  the  US 
over  140,000  corporate-academic  partnerships  involving  30,000  elementary  and  secondary  schools 
(Dickson,  1992).  Today  this  figure  is  even  larger  and  clearly  not  limited  to  partnerships  in  the  US. 

Universities,  steeped  in  bureaucracy,  tend  to  be  one  of  the  last  organisms  to  experience  needed 
change  and  often  get  dragged  kicking  and  screaming  into  the  future.  People  in  universities  frequently 
become  defensive  in  relationships  with  business  people,  even  though  it  is  business  that  awaits  the  products 
of  university  programs.  The  fact  is  that  the  business  world  is  experiencing  rapid  and  constant  change. 
Organizations  in  that  world  are  learning  to  cope,  or  are  disappearing.  University  leaders  must  realize  this 
fact  and  take  responsibility  for  the  future  of  either  coping  to  survive  or  helping  to  lead  our  students  into 
the  world  that  is  becoming. 


A Wake-Up  Call  for  University  Leaders 

Leaders  in  university  settings  must  begin  to  appreciate  the  fact  that  they  do  not  have  a comer  on 
the  education  market  of  the  future.  It  comes  as  a surprise  to  traditional  educators  to  learn  that  fully  one- 
third  of  professional  educators  are  at  work  not  in  universities  but  in  corporate  institutes  of  education  or 
learning  centers.  Another  one-third  are  in  church-related  education,  and  the  balance  in  conventional 
public  and  private  educational  institutions  (Botkin,  1996).  It  is  time  (in  fact  it  may  soon  be  too  late)  for 
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university  leaders  to  begin  to  find  new  ways  to  bridge  learning  relationships  with  educators  in  these 
different  segments. 

By  “university  leaders”  is  meant  anyone  associated  with  a university  setting  who  is  involved  in 
adding  value  to  the  delivery  of  learning  programs  in  the  future.  By  definition,  therefor,  this  would  not 
include  administrators  busy  about  the  job  of  keeping  the  university  on  a steady  state  following  outdated 
mechanisms  and  teaching  methods.  It  is  time  that  professional  educators  in  university  settings  act  like 
professionals.  It  is  time  to  take  our  mission  seriously.  If  we  are  to  assist  in  developing  lifelong  learners  in 
the  turbulent  and  uncertain  world  of  tomorrow,  it  is  important  that  they  not  only  know  things  but  that  they 
are  also  able  to  act  using  their  intelligence.  Being  intelligent  no  longer  means  scoring  high  on  some 
quantified  psychometric.  Being  intelligent  connotes  that  an  individual  has  "the  ability  to  learn  and  to 
apply  what  has  been  learned  to  adapt  to  the  environment,  or  to  modify  the  environment,  or  to  seek  out  or 
create  new  environments"  (Sternberg,  1997).  University  leaders  need  to  act  intelligently  as  we  prepare  for 
our  futures  and  help  ready  others  for  their  futures. 

It  is  time  that  we  begin  to  question,  develop  and  test  new  ideas,  and  reflect  on  the  processes  that 
we  use  to  assist  others  to  learn.  It  is  time  to  heed  the  challenge  of  Malcolm  Knowles,  author  of  The 
Making  of  an  Adult  Educator,  in  that  “we  must  become  able  not  only  to  transform  our  institutions  in 
response  to  changing  situations  and  requirements,  we  must  invent  and  develop  institutions  that  are 
‘learning  systems,’  that  is  to  say,  systems  capable  of  bringing  about  their  own  continuing  transformation" 
(1995).  In  this  way  we  must  realize  the  importance  of  the  age-old  adage  to  “practice  what  we  preach.” 
But  first  we  must  reconsider  what  it  is  and  how  it  is  we  are  preaching.  Only  in  this  way  will  we  be  able  to 
truly  assist  others  in  their  search  for  learning. 
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Abstract 

The  quest  for  lifelong  study  opportunities  is  expected  to  become  one  of  the 
main  characteristics  of  the  rapidly  evolving  information  society.  Members 
have  to  be  able  and  equipped  to  acquire  knowledge  and/or  skills  in  ways  that 
fit  their  personal  situations.  To  help  fill  this  need  we  developed  a coherent 
design  approach  of  educational  instrumentation  as  a solution.  This  approach  is 
named  E-study.  The  E-study  concept  refers  to  a socio-technical  infrastructure 
for  life  long  learning.  Herein  we  will  describe  the  'technical'  and  the  'socio' 
parts  of  this  infrastructure.  The  technical  site  implies  the  Internet,  while  the 
socio  site  implies  situated  learning.  Then,  we  describe  the  project  ComMedia 
in  which  we  bring  the  E-study  approach  in  practice.  We  have  embedded  the  E- 
study  environment  in  the  context  of  Communication  Studies,  University 
Twente,  Enschede.  Therefore,  we  feel  that  we  can  speak  about  the  application 
of  the  E-study  concept  in  a university  education  context. 


Introduction 

As  the  World  Wide  Web  grows  into  a serious  educational  medium,  learning  will  no  longer  be 
restricted  to  classrooms,  or  in  the  case  of  companies,  training  institutes.  The  Web  enables  a form  of 
flexible  and  interactive  learning  that  can  be  very  valuable.  This  style  of  learning  is  illustrated  in 
Collis  (1991)  where  it  is  concluded  that  the  Web  and  its  associated  technologies  may  possibly 
become  the  "killer  application"  for  "tele-learning".  In  our  research  we  use  the  concept  E-study  to 
refer  to  learning  and  training  by  means  of  the  Web  (de  Vries,  1997;  de  Vries,  1998-a).  We  are 
especially  interested  in  the  design  of  online  study  environments  based  on  the  cognitive 
apprenticeship  approach  as  described  by  Collins,  Brown  & Newman  (1986)  and  Collins  (1988).  In 
other  words,  we  want  to  train  students  to  solve  cognitive  problems  by  providing  modelling, 
coaching,  and  fading  in  online  study  environments.  In  this  paper  we  describe  first  the  concept  E- 
study,  which  is  the  basis  for  our  design.  Secondly,  we  will  describe  four  key  design  guidelines  and, 
as  an  illustration,  we  have  provided  an  example  of  a developed  online  situated  learning  environment 
based  on  these  guidelines. 


The  concept  E-Study  outlined 

The  E-study  concept  refers  to  a socio-technical  infrastructure  for  life  long  learning.  The  concept  of 
life  long  learning  is  used  in  a Tele-learning  sense.  In  short,  'making  connections,  among  persons  and 
stored  resources,  through  communication  technologies  for  learning-related  purposes'  (Collis,  1996). 
In  this  definition,  an  emphasis  is  laid  on  the  use  of  information  and  communication  technologies.  To 
enable  life  long  learning  the  need  for  a socio-technical  infrastructure,  that  is  a hardware  and 
software  infrastructure  embedded  into  an  institutional  infrastructure,  must  be  filled.  The  E-Study 
concept  will  stand  for  life  long  learning  by  means  of  the  Internet.  On  the  Internet,  teachers  or  study 
guides,  are  expected  to  prepare  study  tasks,  guide  study  processes,  and  evaluate  the  study  processes 
and  results.  Students,  or  lifelong  learners,  will  carry  out  relevant  study  tasks  in  ways  that  fit  their 
task  contexts.  The  context  for  instruction  can  be  a university  setting,  an  in-service  training,  a 
specific  on-task  performance  support,  and  a short  training  at  home,  etcetera.  Educational 


instrumentation  is  needed  for  study  guides  and  lifelong  learners  to  carry  out  their  tasks.  Figure  1 
illustrates  the  E-Study  concept. 


Figure  1 An  illustration  of  the  E-Study  concept 

We  describe  four  leading  principles  for  the  design  of  educational  instrumentation  for  e-study.  These 
principles  are  flexibility,  integration,  instructional  interaction,  and  information-enriched.  We  refer  to 
flexibility  as  the  freedom  for  teachers  and  students  to  determine:  what  to  study,  why  to  study,  where 
to  study,  when  to  study,  how  to  study,  and  who  will  be  involved  in  the  study  process.  The  second 
principle  is  integration.  This  principle  is  related  to  flexibility.  While  flexibility  implies  diversity, 
integration  means  unity.  Integration  refers  to  consistency  and  coherence  in  the  provided  educational 
instrumentation.  By  instructional  interactivity  we  mean  interaction,  as  an  input/output  process 
between  one  or  more  users  and  a software  program,  purposefully  intended  for  knowledge 
acquisition  and  incorporation  in  a broad  sense.  The  fourth  principle  is  information-enriched,  which 
basically  means  that  all  relevant  ’curriculum'  information  is  provided.  The  information  society 
provides  members  with  an  enormous  amount  of  information.  Members  must  be  able  to  find 
applicable  information  and  make  use  of  it  in  a proper  way.  Informative  as  a design  principle  implies 
that  users  are  enabled  to  find  and  use  applicable  information  locally  as  well  as  globally.  We  use  the 
concepts  Interactive  Study  Environments,  Interactive  Study  Systems,  and  On-line  Study  Services  to 
describe  the  E-study  concept  (Vries,  1997,  Vries  & Vogel,  1997).  These  concepts  refer  only  to  a 
technical  infrastructure.  In  the  start  we  referred  to  a socio-technical  infrastructure  for  life  long 
learning.  The  described  technical  infrastructure  has  to  be  embedded  into  an  organisational 
infrastructure.  In  this  paper  we  will  further  discuss  the  'socio’-part  of  the  infrastructure. 


An  educational  organisation  as  a context  for  situated  learning 

Our  main  interest  involves  the  design  of  online  study  environments  which  enable  situated  learning. 
We  see  situated  learning  as  'learning  knowledge  and  skills  in  contexts  that  reflect  the  way  the 
knowledge  and  skills  will  be  useful  in  real  life’  (Collins,  Brown,  & Newman,  1986)  and  as  'learning 
tailored  and  tuned  to  a particular  situation'.  If  we  combine  both  descriptions  we  have  a clear  view  of 
life  long  learning.  The  first  part  refers  to  learning  in  an  educational  organisation,  while  the  second 
refers  to  learning  in  non-educational  organisations.  Such  learning  is  from  the  learner’s  point  of  view 
continuing  education.  The  main  issue  for  both  kinds  of  organisations  is  that  they  consider 
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themselves  as  a learning  organisation,  'an  organisation  that  is  continually  expanding  its  capacity  to 
create  its  future'  (Senge,  1990).  Clearly,  situated  learning  reflects  our  ideas  of  the  'socio'  part  of  the 
E‘Study  concept.  Here,  we  will  discuss  situated  learning  in  educational  organisations.  In  de  Vries 
(1998  b)  we  describe  design  guidelines  for  online  study  environments  for  non-educational 
organisations. 


We  took  the  cognitive  apprenticeship  approach  from  Collins,  Brown,  & Newman  (1986),  or  to  be 
more  specific,  their  framework  for  designing  learning  environments  as  the  starting  point  for  the 
design  of  online  situated  learning  environments.  Their  framework  describes  four  characteristics  of 
such  environments:  content,  sociology,  methods  and  sequence. 


In  the  project  known  as  ComMedia,  we  designed  an  online  'virtual  learning  company'  as  a 'real  life 
context'  for  study  projects.  Figure  2 illustrates  the  design  of  ComMedia.  These  study  projects  were 
for  students  who  desired  to  become  specialists  in  the  design  and  implementation  of  interactive 
media  applications  for  corporate  communication.  ComMedia  is  part  of  Communication  & Media, 
one  of  the  six  specialisations  of  the  study  program  Communication  Studies,  University  Twente, 
Netherlands.  A specialisation  it  is  the  subject  in  which  a student  specialises  in  the  final  period 
before  graduation,  comparable  with  a major.  The  field  of  interactive  media  and  corporate 
communication  is  the  subject  of  Communication  and  Media. 
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Figure  2 The  personal  workspace  of  a student  in  ComMedia 

ComMedia  as  a technical  system,  is  a website  on  a Microsoft  NT  Server.  We  are  using  Cold  Fusion 
from  Allaire  as  the  development  platform.  Cold  Fusion  is  a rapid  application  development  system 
for  professional  developers  who  want  to  create  dynamic  Web  applications  and  interactive  Web  sites. 
It  provides  a fast  way  to  integrate  browser,  server,  and  database  technologies  into  powerful  Web 
applications.  ComMedia  can  be  considered  as  an  infrastructure  for  project  education.  Almost  all  the 
information  is  stored  in  a database.  For  our  database  platform  we  are  using  Microsoft  Access  . We 
use  the  term  infrastructure  because  ComMedia  can  be  easily  adapted  and  configured  for  another 
educational  context,  for  instance  for  the  knowledge  domain  'management'  in  higher  education.  We 
will  now  discuss  examples  of  how  we  applied  the  framework  of  Collins.  Brown,  & Newman  for  the 
design  of  ComMedia. 
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Content 


The  characteristic  content  refers  to  domain  knowledge,  heuristic  strategies,  control  strategies,  and 
learning  strategies.  ComMedia  is  comprised  of  insight,  knowledge,  and  skills  in  the  design  and  use 
of  interactive  media  for  Corporate  Communication.  The  basic  design  guideline  is  that  relevant 
’domain’  content  has  to  be  stored  into  a knowledge  base.  This  knowledge  base  must  enable  the 
internalization,  external isation,  socialisation,  and  combination  of  four  modes  of  knowledge 
conversion  described  by  Nonaka  & Takeuchi  (1995).  In  ComMedia  we  call  this  knowledge  base  the 
Cabinet.  The  Cabinet  is  a repository  or  in  other  words  a mine  of  information.  Basically  we 
distinguish  five  knowledge  objects  and  identify  five  knowledge  object  types.  These  knowledge 
types  are  information  fragments,  persons,  groups,  projects,  and  competencies.  Information 
fragments  are  media  products  containing  distinct  messages.  Examples  include  documents,  links, 
multimedia  files,  and  WebPages.  Persons  are  the  members  of  the  expertise  center.  Examples  of 
member  roles  include  junior  consultants,  senior  consultants,  external  consultants,  editors,  managers, 
and  maintainers.  Projects  are  dedicated  workspaces,  for  example  personal,  design,  research,  and 
consultancy  workspaces.  Groups  are  people  who  share  a certain  interest  without  a distinct  project 
goal,  specific  resources  or  deadlines.  Examples  of  such  groups  are  newsgroups,  discussion  groups, 
and  special  interest  groups.  Competencies  are  sub  knowledge-domains.  Examples  are  divisions, 
competence  centers,  and  courses.  Each  knowledge  object  is  described  by  a profile.  The  profile 
characteristics  we  describe  are  for  instance:  identification,  domain  themes,  domain  keywords, 
applicability,  and  accessibility.  Identification  describes  generic,  ownership,  and  author  information 
of  objects.  A domain  theme  describes  the  main  subject  areas  to  which  object  are  related.  In  general, 
there  is  a hierarchical  order  of  themes.  Domain  keywords  describe  the  central  concepts  in  a 
knowledge  domain  to  which  objects  are  related.  Generally,  there  is  a network  of  central  concepts. 
Applicability  describes  the  usability  of  objects  and  refers  to  the  forms  in  which  objects  may  appear, 
for  instance  in  an  online  magazine,  portfolio,  showcase,  project,  or  internal  report.  However, 
applicability  also  refers  to  peoples’  opinions  about  the  value  of  objects.  They  can  grade  objects  and 
present  their  opinion  in  words.  Accessibility  describes  the  availability  level  of  objects.  We  make  a 
distinction  between  the  individual,  project,  competence,  internal,  and  external  level.  A specific 
object  available  on  the  individual  level  means  that  it  is  part  of  a personal  archive.  However,  that 
person  can  make  the  same  object  part  of  a certain  project  archive,  etcetera.  Based  on  these 
characteristics,  it  is  possible  to  create  a characteristic  profile  for  each  individual  knowledge  object. 
In  the  very  near  future,  it  is  expected  that  these  profiles  will  be  ’builf  dynamically,  based  on  the 
persons  behavior.  That  means  that  for  instance,  persons'  profiles,  and  more  specific  the  description 
of  the  persons'  competencies,  are  updated  continually.  This  update  will  be  based  on  the  study 
activities  a person  performs.  The  strategies  are  translated  into  task  objects  and  a set  of  these  objects 
form  workspaces.  Task  objects  are  for  instance:  archive,  personal  management,  design,  and  study. 
Examples  of  workspaces  are  the  personal  workspace,  the  study  project,  and  the  design  project 
workspace.  Each  workspace  is  formed  dynamically,  based  on  the  personal  profile. 


Sociology 

Sociology  is  characterised  by  situated  learning,  culture  of  expert  practice,  intrinsic  motivation, 
exploiting  cooperation,  and  exploiting  competition.  The  basic  design  guideline  is  that  the  learning  of 
knowledge  and  skills  happens  in  contexts  that  reflect  the  way  the  knowledge  and  skills  will  be 
useful  in  real  life.  This  characteristic  is  made  concrete  by  a ’company’  metaphor.  ComMedia  is 
designed  as  a company  that  enables  members  to  prepare,  guide,  carry  out,  and  evaluate  a wide 
variety  of  projects.  As  a company,  ComMedia  has  the  following  mission:  to  offer  consultation  and 
design  services  to  (non-)profit  companies  facing  corporate  communication  media  problems  or 
companies  interested  in  communication  opportunities  made  possible  by  information  and 
communication  technology  developments.  These  companies  wish  to  offer  a usable  information  and 
communication  platform  to  students  and  teachers  to  set  up,  look  after,  pass  through,  and  evaluate 
study  projects.  ComMedia  is  designed  as  a virtual  learning  ’knowledge  company’.  It  is  a knowledge 
company  whose  primary  tasks  concern  knowledge  about  the  design  and  use  of  interactive  media  for 
corporate  communication  purposes.  It  is  a virtual,  imaginary  profit  company  that  only  exists  online. 
Last  but  not  least,  it  is  a typical  learning  company.  It’s  major  task  is  to  enable  ’junior  consultants'  to 
become  'senior  consultants'.  ComMedia  is  a typical  adhocracy.  There  is  a strong  need  for  innovation 
and  creativity.  There  is  no  middle  management  and  consultants  have  a lot  of  freedom  in  carrying  out 
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their  tasks.  We  identify  four  major  roles:  administrator,  senior  consultants  (teachers),  junior 
consultants  (students),  external  consultants  (former  students)  and  visitors.  Tasks  are  carried  out  in 
(study-)  projects.  Each  project  team  has  a lot  of  freedom  in  completing  the  project,  as  long  as  they 
meet  standard  preconditions,  set  by  teachers. 


Methods 

Methods  refer  to  the  ways  students  are  supported  in  acquiring  and  integrating  cognitive  and 
metacognitive  strategies  for  using,  managing,  and  discovering  knowledge.  The  basic  guideline  is 
that  for  each  study  project,  students  and  teachers  need  to  have  the  opportunities  for  modelling, 
coaching,  scaffolding,  fading,  articulation,  reflection,  and  exploration.  The  applied  ’study  method'  is 
a project-based  education.  In  our  view  we  consider  the  senior  consultants,  the  teachers,  as  mentors 
for  the  junior  consultants,  the  students.  Furthermore  we  have  a balanced  structure  in  student 
responsibility.  For  instance,  students  have  to  meet  stated  deadlines  in  the  first  two  course  projects, 
while  they  have  their  own  freedom  in  setting  deadlines  in  the  last  two  course  projects. 


Sequence 

The  characteristic  of  sequence  refers  to  the  way  integration  and  generalisation  of  knowledge  and 
complex  skills  can  be  achieved.  The  basic  guideline  is  one  in  which  the  study  profile  of  a 
curriculum  has  to  reflect  increasing  complexity,  increasing  diversity,  and  global  before  local  skills. 
The  study  profile  of  Communication  & Media  is:  C&MTheory  (7  sp):  a theoretical  introduction  in 
the  design  and  use  of  interactive  media  for  corporate  communication.  In  this  course  the  main 
assignment  for  students  is  to  put  together  a so-called  'webfolio',  a personal  C&M  information 
website,  and  to  make  use  of  each  others  websites  to  write  a 'scientific'  article;  C&MDesign  (7  sp):  a 
practical  introduction  in  methods,  techniques  and  instruments  for  the  design  of  media  for  corporate 
communication.  Here  the  main  assignment  for  students  is  to  write  a project  proposal  for  solving  a 
communication  design  problem  — for  instance  a website  for  marketing  ends,  and  to  design  a simple 
prototype  with  the  function  of  a scale-model  in  the  building  industry.  C&MResearch  (8  sp):  a media 
product  design  research  assignment  where  students  have  to  carry  out  'state  of  the  art'  research 
concerning  design  guidelines  for,  for  instance,  'the  corporate  image  on  the  web'  or  'marketing 
websites  that  sell'.  C&MProblem-solving  (20  sp):  a final  assignment  where  students  must  solve  a 
communication  design  problem  in  a real  life  context.  Students  must  design  a prototype  as  a 'proof  of 
concept'  and  write  a thesis  about  their  work.  One  sp  stands  for  one-week  study,  so  the  whole 
specialisation  concerns  42-study  weeks  full  time  study. 


Some  final  remarks 

The  design  framework  of  Collins,  Brown,  & Newman  although  described  in  1986  is  still  very  useful 
for  designing  online  study  environments.  We  consider  ComMedia  as  a useful  proof  of  concept.  It  is 
successfully  applied  in  the  study  program  of  Communication  and  Media.  Our  main  experience  is 
that  students  appreciate  our  educational  approach.  The  students  evaluations  are  very  positive,  as 
they  stated  'we  really  work  hard  for  the  first  time  in  our  study'.  ComMedia  as  an  environment  is 
only  a tool.  The  'socio'  approach  or  the  organisational  infrastructure  is  in  our  view  crucial.  Based  on 
experiences,  we  are  further  working  on  the  design  of  ComMedia.  Our  aim  is  the  development  of  a 
generic  online  platform  for  situated  learning,  which  is  applicable  as  a university  expertise  center  for 
interested  persons  from  educational  and  non-educational  organisations.  Our  ultimate  goal  is  a 
knowledge  community  for  life  long  learning.  There  is  much  work  to  be  done. 


Discussion 

We  believe  that  the  cognitive  apprenticeship  approach  is  still  one  of  the  foundations  of  education. 
Trends  in  life  long  learning  and  the  upcoming  information  society  only  strengthens  our  belief  The 
Internet  provides  us  with  the  means  of  designing  attractive  and  powerful  situated  learning 
environments.  We  think  that  our  design  approach  offers  practical  concepts  and  guidelines  for 
designing  such  environments. 
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Abstract:  Navigating  the  Web  has  its  difficulties.  If  some  of  the  user  disorientation 
originates  in  the  practical  operations  for  navigation  and  is  reduced  when  the  browser 
provides  a map  of  the  visited  space,  we  argue  that  the  main  difficulty  arises  with 
characterizing  Web  contents.  We  propose  a new  approach  to  Web  navigation  support 
coined  “constructive  navigation  support”.  We  present  the  main  features  of  NESTOR,  a 
browser  implementing  this  approach  and  we  discuss  the  results  we  obtained  during  a 
first  experiment  in  the  educational  domain. 


1 Introduction. 

In  today’s  information  society,  information  gathering  and  use  constitute  a major  activity.  The  two  main 
information  access  modes,  query  and  navigation,  correspond  more  or  less  to  the  extent  to  which  the 
researched  information  spaces  are  « moderated  » i.e.  « the  extent  to  which  the  space  structures  have  been 
coordinated  and  controlled  » (Jul  & Furnas  97).  Query  is  the  access  mode  appropriate  for  moderated  data 
bases,  having  a known  structure:  query  software  can  incorporate  features  which  are  based  on  this  assumed 
structure.  Navigation  is  more  appropriate  for  accessing  low-moderated  information  spaces  which  don’t 
result  from  a unified  design  process  - such  as  the  Web.  Browsers,  which  implement  the  navigation  mode, 
can  hardly  assume  that  the  diverse  documents  they  link  to,  have  a given  structure  on  which  the  design  of 
navigation  aids  can  be  based  (except  the  node  and  link  structure  which  is  inherent  to  hypermedia). 
However,  some  attempts  have  been  made  (Pirolli  et  al.  96) . On  the  Web,  even  though  it  is  not  a moderated 
space,  query  is  used  jointly  with  navigation  : the  space  is  so  huge  that  information  gathering  cannot  but 
begin  with  querying.  However,  this  cannot  be  as  successful!  as  it  is  -sometimes  - with  databases.  The 
search  is  then  pursued  through  navigation.  It  has  been  assumed  much  too  prematurely  that  the  two  modes 
navigation  and  query,  would  add  up,  and,  with  the  use  of  search  engines  on  the  web,  provide  a convivial 
and  at  the  same  time  efficient  way  of  finding  the  information  sought.  But  information  can  be  veiy  hard  to 
locate  and  sometimes  not  even  found  at  all,  although  it  can  be  there,  in  the  very  documents  that  are  being 
screened.  Almost  all  the  media  that  have  been  used  until  now  have  a content  which  is  strongly  structured 
and  controlled:  think  of  newspapers,  TV,  libraries.  While  there  are  large  subspaces  on  the  Web  which  are  in 
fact  moderated  (electronic  libraries,  on  line  news,  institutions  official  websites),  there  are  also  numerous 
websites  which  are  maintained  by  individuals  outside  of  standard,  official  or  academic  information 
structures.  And  this  is  new:  individuals  have  produced  documents  for  years,  but  never  till  now  have  they  be 
able  to  reach  so  quickly  a large  public.  This  might  account  greatly  for  the  success  of  the  Web,  but  it  clearly 
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put  the  cognitive  load  of  information  filtering  on  the  user  shoulders,  while  this  load  was  previously  mainly 
assumed  by  people  acting  as  moderators. 

At  least  two  aspects  of  electronic  navigation  lead  to  navigation  difficulties  : at  the  practical  level  , 
navigating  is  circulating  with  a vehicle  : a matter  of  locomotion  (Jul  & Furnas  97).  Maglio  and  Matlock 
(Maglio  & Matlock  98)  found  that  “people  naturally  think  of  the  web  as  a kind  of  physical  space  in  which 
. At  a more  conceptual  level,  navigating  is  mainly  a matter  of  decision  making  : selecting  links 
to  follow.  Overall,  it  seems  that  the  model  of  Navigation  In-Real-Life  is  still  valid  ; “orienting  in  the 
environment,  choosing  a route,  monitoring  the  route  and  assessing  that  the  destination  has  been  reached** 
(Hook  et  al.  98)  - except  that  in  information  gathering  tasks,  the  destination  is  not  a location  but  rather  a 
goal.  In  fact,  this  happens  too  In-Real-Life  as  people  move  in  a vehicle  to  achieve  a goal.  And  as  In-Real- 
Life,  the  two  levels  of  activity  - practical  and  conceptual  - lead  to  different  problems  and  call  for 
appropriate  support. 

Users  disorientation  in  closed  hypermedia  corpus  (i.e.  moderated  information  space)  has  been 
known  for  sometime  now.  Several  Webmap-based  new  browsers  are  under  construction  (see  for  example 
http://www.cvbergeographv.org/atlas/web  sites. htmO.  After  all,  graphical  user  interface  (GUI)  and 
Hypertext  User  Interface  are  to  date  the  major  human-computer  interaction  modes  that  prevail  on  the 
Desktop  and  on  the  Internet  and  there  is  no  reason  why  they  should  not  be  combined.  Like  others  (Ashmore 
97),  we  observed  that  providing  a map  of  the  visited  information  subspace  can  “ease  inadequacies  in  Web 
navigation**  (Cockbum  & Jones  97).  Still,  we  observed  that  navigating  with  a map-enhanced  browser  in  the 
non-moderated  Web  brings  about  much  disorientation.  User  difficulties  in  making  navigational  decisions 
seem  to  appear  at  a more  conceptual  level  : identifying  documents,  delineating  pertinent  materials, 
organising  the  information  into  categories,  pro-actively  selecting  links  through  contextual  navigational 
cues  - referred  as  "residue/scent"  by  (Jul  & Furnas  97). 

Just  as  it  happens  in  the  world  of  cinema,  orientation  deals  with  expectations  about  the  “off-screen 
space**  i.e.  that  space  which  is  not  visible  on  the  screen,  but  which  is  still  part  of  the  scene  (Persson  98).  As 
noted  by  Harper  (Harper  98)  : “Interpreting  information  is  not  only  a matter  of  what  happens  during 
reading  : it  is  an  activity  that  occurs  within  a broader  institutional  practice  ...  an  academic  journal  for 
example  does  not  just  convey  information:  its  meaning  is  wrapped  up  in  such  things  as  what  is  known 
about  the  status  of  the  journal**.  Knowledge  of  such  things  provides  the  context  for  reading.  Reading  Web 
documents  requires  that  users  develop  a specific  system  of  interpretation  through  which  they  build  the  off- 
screen context.  Because  the  contexts  of  production  of  the  Web  documents  are  so  diverse,  there  cannot  be 
easy  expectations.  There  exists  many  cues  about  the  off-screen  context  which  can  be  collected  along  the 
navigation  process  in  order  to  ease  the  interpretation  task.  But  it  remains  that  reading  Web  documents  and 
thus  navigating  across  them  relies  - more  than  in  the  case  of  other  media  - on  a personal  construction  of 
what  is  meaningful. 

We  propose  then  that  a software  tool  with  the  aim  to  support  navigation  should  be  a constructivist 
environment  i.e.  an  environment  which  provides  means  for  manipulating  and  working  with  self  experience. 
Constructivism  implies  that  knowledge  stems  from  a process  of  interactive  restructuring  and  transformation 
of  personal  experience.  This  implies  - in  the  domain  of  electronic  navigation  - that  the  tools  provided  allow 
for  gathering,  representing,  structuring  and  creating  navigational  objects  - “externalizing**  in  the  wording 
of  Vygotsky.  In  this  way,  in  constantly  re-structuring  the  dynamic  graphical  representation  of  the  browsing 
activity,  users  construct  mental  knowledge  spaces  (Zeiliger  et  al.  97).  Structuring  self  experience  with 
information  requires  conceptual  skills  as  well  as  external  "cognitive"  tools  (Norman). 

This  point  is  even  more  valid  in  the  educational  domain  : indeed  learners  are  used  to  access  info 
spaces  (such  as  libraries)  which  are  not  only  strongly  moderated  but  whose  categories  are  also  close  to 
school  categories.  It  is  likely  that  when  learners  explore  the  Web  they  have  to  face  a new  problem  : it 
conveys  categories  which  conflict  with  the  school  categories  (because  Web  categories  are  linked  to  the 
diverse  intentions  of  the  information  providers).  These  were  the  types  of  problems  that  we  focused  on 
during  studies  - part  of  which  are  reported  here  - that  were  conducted  the  educational  domain  with  a 
specific  navigator  named  NESTOR.  We  implemented  this  new  Web  browser  to  test  the  constructivist 
approach  to  Web  navigation  support. 


440 


2 Implementing  the  Approach:  the  NESTOR  Browser. 


NESTOR  is  a Web  client  which  runs  on  Microsoft  Windows  95/98/NT  platforms.  It  is  a standalone 
Web  browser,  however  the  basic  browsing  and  HTML  editing  features  are  provided  through  Microsoft’s 
Active  X components  (TwebBrowser  and  THTMLEd)  which  means  that  NESTOR  takes  advantage  of 
IntemetExplorer  4.01  retrieval  motor,  options,  popup  menus,  dynamic  HTML  and  so  on.  NESTOR 
browsing  interface  is  very  similar  to  that  of  IE4  - except  perhaps  that  it  has  been  skimmed  a bit  ! The 
NESTOR  left-window  interface  - which  is  specific  - deals  with  the  construction  of  maps  and  of  a personal 
information  space.  In  that  view,  NESTOR  implementation  can  be  considered  as  constituting  an  add-on  to 
standard  Internet  browsers. 

The  main  screen  of  NESTOR  is  divided  into  two  windows  : a standard  browsing  window  (to  the 
right)  and  a map  window  (to  the  left).  While  both  windows  support  a range  of  navigational  activities,  the 
map  window  has  been  designed  to  facilitate  information-structuring  activities.  Direct  manipulation  prevails 
in  the  map  window  while  hypertext  navigation  is  the  standard  mode  in  the  browsing-window.  Graphical 
user  interface  (GUI)  in  general  and  direct  manipulation  in  particular  are  interaction  modes  which  are  more 
adequate  for  constructive  tasks  : tasks  which  require  re-arranging  collections  of  objects.  The  map  window 
can  be  thought  of  -at  first-  as  a dynamic  map  representing  the  visited  Web  subspace.  Most  navigation 
operations  (such  as  backtracking)  are  available  in  both  windows  and  provide  a crossed  feedback.  NESTOR 
also  records  the  raw  navigation  history  and  allows  for  re-constructing  a map  from  a selection  of  this 
history.  As  “re-visit”  accounts  generally  for  more  than  50%  of  users’  operations,  the  map  and  history 
features  (and  their  combinations)  are  very  useftil.  There  is  a special  mapping  of  search  motors  (representing 
queries  in  order  to  facilitate  the  exploration  of  the  filtered  documents)  and  some  handy  features  which 
facilitate  directed  searching  like  a "breadth-first"  mechanism  (Newfield  et  al.  98).  Further  more,  NESTOR 
has  been  designed  to  promote  the  construction  of  a personalized  Web  space.  For  that  purpose,  the  map- 
window  is  interactive  : it  allows  for  re-arranging  as  well  as  creating  new  objects  : documents,  links, 
annotations,  sub-maps,  tours,  search  keywords  and  conceptual  areas.  It  is  best  viewed  as  a sort  of  note- 
taking tool  which  is  consistent  with  the  Web  structure  : notes  take  the  form  of  a personalized  web  which  is 
inter-weaved  with  the  public  WWW. 


Fig  1.  The  main  screen  of  NESTOR. 


We  now  list  the  main  features  of  NESTOR  with  more  details  and  outline  how  they  implement  the 
constructive  navigation  approach. 

2.1  Representing  Self  Navigational  Experience  : while  the  user  performs  navigational  operations  with  the 
Web  browser,  NESTOR  automatically  draws  a graphical  representation  of  the  Web  subspace  which  has 
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been  visited.  Every  visited  document  is  represented  as  a circle  whose  size  depends  on  the  number  of  out- 
going links.  Visited  links  are  represented  as  arrows.  There  is  an  automatic  layout  which  favours  the  point  of 
view  of  the  user’s  self  experience  (as  opposed  to  an  overview  map  based  on  a network  centred 
representation)  : as  long  as  the  user  traverses  to  a new  document,  objects  and  arrows  are  posited  along  a 
straight  line  figuring  a "user’s  route";  the  object  corresponding  to  the  currently  displayed  document  is 
highlighted  providing  situational  cues  for  the  user's  position  in  the  visited  subspace;  when  the  user 
navigates  using  the  back  or  forward  buttons  the  corresponding  objects  are  highlighted;  when  the  user  starts 
browsing  again,  a new  straight  "route"  is  drawn.  This  graphical  representation  is  dynamic  and  interactive  : 
users  can  re-arrange  the  layout  through  direct  manipulation;  any  visited  document  can  be  accessed  directly 
by  double-clicking  on  the  corresponding  graphical  object.  The  use  of  such  maps  increases  the  visual 
feedback  in  practical  navigation  operations  and  facilitate  orientation. 

2.2  Constructing  a Personal  Web  space;  at  the  beginning  the  map  reflects  the  visited  subspace,  i.e.  public 
documents  and  their  public  links.  But  then,  users  can  create  personal  documents  and  personal  links,  and 
inter- weave  them  with  the  public  network.  Personal  links  are  created  through  direct  manipulation  on  the 
map:  i.e.  simply  drawing  an  arrow  between  to  objects  on  the  screen  - either  public  or  personal  - creates  an 
hypertext  link.  Personal  links  are  represented  as  dotted  arrows  while  public  links  appear  as  solid  arrows. 
Public  documents  need  not  be  aware  they  are  linked.  Personal  links  appear  as  hot  words  in  a separate 
window  jointly  with  the  document  they  are  attached  to  but  their  anchor  is  not  incorporated  to  the  document 
itself  Building  thematic  maps  and  creating  personal  hypertext  are  activities  that  promote  directed  Web 
searching. 

2.3  Note-taking  : The  process  of  inter-weaving  the  public  and  the  personal  network  can  be  thought  of  as  a 
sort  of  note-taking  mode  which  is  consistent  with  the  hypermedia  structure  of  the  Web:  i.e.  notes  include 
not  only  text  but  links  as  well,  so  that  the  Web  network  is  annotated  with  a network  of  notes.  Additional 
features  are  available  in  that  line:  annotations  can  be  attached  to  every  visited  document  be  it  public  or 
personal.  Annotation  content  is  HTML-formatted  so  that  it  can  include  text,  links  or  even  questions  in  a 
"CAL  fashion"  as  well.  Whenever  an  annotated  document  is  re-visited,  the  corresponding  annotation  is 
displayed  jointly  in  a separate  window.  A visible  clipboard  - "the  bag"-  is  also  available  to  allow  selecting 
and  gathering  of  information  along  the  navigation  process;  when  pasting  information  into  the  bag, 
NESTOR  includes  a return  link  to  the  source  document.  The  information  gathering  and  note-taking 
activities  help  user  structure  their  thoughts  and  construct  meaningful  information. 

2.4  Creating  Keywords  Objects  and  Conceptual  Areas:  keywords,  conceptual  areas  and  sub-guides  can 
be  inserted  in  maps.  User  created  keywords  are  automatically  searched  for  in  the  visited  document’s  text 
and  highlighted  when  found  (both  on  the  map  and  in  the  document):  this  is  especially  useful  when 
navigating  across  a set  of  documents  which  have  been  selected  by  a search  engine.  Keywords  object  are 
also  extracted  from  the  search  motors  queries.  Web  maps  and  keywords  appear  in  the  same  window  so  as  to 
allow  different  combined  layouts.  Conceptual  areas  help  structuring  the  maps  with  object  clusters. 

2.5  Creating  and  Saving  Navigational  Objects  : all  objects  created  by  users  (maps,  keywords,  conceptual 
areas,  annotations  and  routes)  can  be  saved  to  a file  and  retrieved.  Those  objects  are  considered  as 
"navigational  objects"  in  the  view  that  they  can  serve  to  initiate  new  navigational  operations:  for  example 
navigating  from  the  links  in  the  annotations.  “Conceptual  orientation”  is  facilitated  through  creating, 
arranging  and  capitalizing  on  such  objects  (using  their  navigational  properties  and  methods).  We  use  the 
term  "map"  to  refer  to  such  collections  of  navigational  objects. 

2.6  Sharing  Maps  : NESTOR  allows  navigational  objects  to  be  published  and  shared  by  a group  of  people. 
Simple  URLs  as  well  as  whole  maps  can  be  published.  NESTOR  users  are  warned  whenever  they  navigates 
to  a URL  which  has  been  previously  published  by  one  of  their  colleagues.  In  that  case,  annotations  attached 
to  the  document  (by  the  publisher)  are  available,  as  well  as  the  map  in  the  context  of  which  the  publisher 
has  accessed  the  document.  Bags  can  be  published  too.  Computer  mediated  communications  and 
collaborative  navigation  can  help  users  construct  meaningful  information  : a process  Harper  describes  as 
“collaboratively  defining  institutional  matters  of  relevance”  or  in  short  a “world-known-in-common” 
(Harper  98). 
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3 Evaluating  the  Tool  and  the  Approach. 

3.1  Educational  context.  We  chose  to  evaluate  the  use  of  NESTOR  in  a real  life  setting  (i.e.  in  the  school 
computer  room,  in  cooperation  with  the  school  teachers)  with  students  knowledgeable  in  the  use  of 
computers  and  already  familiar  with  WEB  navigation.  Groups  of  high  school  students  (aged  15-16  years 
old),  having  prior  experience  with  the  Netscape  navigator,  were  given  a pedagogical  task  which  consisted 
in  finding  information  on  the  Web  in  order  to  answer  questions  relating  to  economy  (Task  1)  and  to 
ecology  (Task  2).  They  had  for  each  task  a maximum  of  two  hours.  The  task  was  completed  when  they  had 
found  the  relevant  information  and  written  or  recorded  it  in  the  “bag”  file.  Task  1 dealt  with  a curriculum 
topic  in  economy:  the  students  were  each  given  a paper  document  presenting  the  task,  with  2 questions  to 
be  answered,  and  a list  of  Web  URLs  where  the  required  information  could  be  found.  The  students  were 
also  each  given  detailed  written  instructions  on  how  to  connect  to  the  main  academic  server  where  the  Web 
addresses  were  already  available.  The  students  worked  in  dyads  with  NESTOR,  one  computer  for  two 
students.  Task  2 had  to  do  with  a topic  loosely  related  to  the  curriculum:  ecology.  The  procedure  was  the 
following:  the  students  worked  individually  (5  min.)  and  then  in  a group  session  (25  min.)  with  the  aim  of 
setting  up  a list  of  their  own  keywords  on  pollution  of  air  and  water.  In  this  way  the  topic  was  addressed 
not  only  with  an  academic  label  but  by  a thematic  envelope  as  well  (the  list  of  chosen  keywords).  The 
students  were  then  shown  and  explained  NESTOR  keyword  features  and  given  three  written  questions  to 
answer.  They  had  2 hours  for  their  work  and  were  finished  when  they  had  found  and  annotated  5 relevant 
URLs. 

Having  already  experimented  that  visualizing  of  self  navigational  experience  facilitates  users' 
practical  orientation  (Zeiliger  et  al.  97)  we  were  here  interested  in  understanding  why  a user  does  so  many 
revisits  (more  than  50%  in  our  observations)  of  the  same  sites,  even  when  a map  is  available.  The  first  task 
was  framed  in  order  to  provide  information  on  this  recurrent  problem.  The  question  content  was  labeled  in 
such  a way  so  as  not  to  reproduce  the  exact  labeling  to  be  found  on  the  sites  indicated.  This  refers  to  the 
hypothesis  that  searching  for  information  in  low-moderated  information  spaces  such  as  the  Web  requires 
greater  categorization  competency  on  the  part  of  the  user.  We  wanted  to  explore  how  students  would 
succeed  in  matching  a curriculum  question  with  Web  information.  The  second  task  was  more  open.  We 
hypothesized  that  students  have  difficulties  because  they  do  not  recognize  the  information  they  are  seeking 
when  the  information  is  displayed  and  contextualised  in  categories  different  from  the  “school”  categories. 
We  therefore  wanted  them  to  use  their  own  categories.  This  was  achieved  by  having  them  prepare  their 
keyword  list. 

3.2  Evaluation  results.  We  discuss  here  information  gathered  on  the  use  of  NESTOR  by  three  groups  of 
students  ( two  groups  of  eight  and  one  group  of  ten).  After  eliminating  repetitive  operations  due  to  retrieval 
delay  (still  a frequent  source  of  disorientation  in  academic  use),  we  ended  up  with  1 06 1 operations  within  a 
period  of  1033  minutes  (17  hours  and  22  min).  We  are  concerned  here  strictly  with  the  navigation 
operations,  which  were  recorded  on  each  computer. 


Task  1 

Task  2 

Total  operations 

1061 

Navigating  by  use  of  hotwords 

HOTW 

49% 

48% 

49% 

Navigating  by  use  of  map  objects 

MAP 

21.5% 

19.5% 

20% 

Using  BACK  button 

BACK 

18% 

5% 

14% 

Using  NEXT  buttton 

NEXT 

1 % 

1 % 

1 % 

Typing  an  URL  and  pressing  OPEN 

OPEN 

1 % 

1 % 

1 % 

Going  back  home 

HOME 

1 % 

7% 

3.5% 

Miscellaneous 

11.5% 

Revisiting 

53% 

55% 

54% 

As  expected,  the  percentage  of  re-visits  is  very  high.  Use  of  the  map  is  high  (about  20%,  to  be 
compared  with  the  low  use  of  BACK  around  14%).  Because  NESTOR  map  represents  only  the  visited 
subspace  and  because  - unlike  BACK  - it  provides  a direct  access  to  any  visited  document,  we  can  assume 
that  students  were  aware  of  re-visiting  URLs  in  at  least  20%  cases  . We  have  here  situations  where  students 
revisited  the  same  documents  several  times,  looking  for  information  that  they  were  not  finding,  although 
they  knew  that  it  was  there.  During  interviews  conducted  after  the  sessions,  students  spoke  of  their 
disorientation  caused  by  the  way  the  information  was  presented  in  the  Web  documents.  When  analysing  the 
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individual  traces,  we  find  that  much  revisiting  also  involves  going  along  already  traveled  paths  (among 
which  we  observed  numerous  to-and-fro  cycles).  We  think  we  can  frame  this  in  terms  of  conceptual 
disorientation f not  to  be  confused  with  the  practical  disorientation  which  has  been  reported  when  users 
navigate  in  closed  hypermedia  corpus  using  map-free  browsers.  More  experiments  are  underway  on  a 
larger  scale  : 72  university  students  at  Ecole  de  Management  de  Lyon  (http://www.em-lvon.fr)  have  been 
given  NESTOR  to  complete  a 3 month  information-gathering  task  on  the  Web  and  collaborative  navigation 
features  are  being  evaluated  within  the  framework  of  the  LEARN-NETT  European  project 
(httD://tecfa.unige.ch/proi/leamett). 


4 Conclusion  and  Future  Work. 

We  found  that  navigating  the  Web  even  with  a map-enhanced  browser  causes  much  disorientation 
problems.  We  hypothesized  that  much  of  this  disorientation  arises  at  a conceptual  level,  because  in  non- 
moderated information  space  the  context  for  reading  is  often  lacking.  In  the  educational  domain,  when  the 
researched  information  is  presented  in  a context  different  from  their  academic  context  or  yields  novel 
perceptive  and  lexical  data,  learners  cannot  attain  its  meaning  through  surface  scanning.  To  grasp  the 
meaning  behind  the  linguistic  units,  they  have  to  go  to  the  level  where  lies  the  information  necessary  for 
semantic  categorizing.  NESTOR  maps  and  their  navigational  objects  provide  basic  means  for  gathering 
cues  along  the  navigation  paths  and  structuring  them  with  the  aim  of  building  a context  for  deep  reading.  In 
this  perspective,  future  Web  browsers  should  not  only  provide  maps,  they  must  tend  toward  full 
environments  engaging  users  looking  for  “information  that  counts”  into  more  constructive  activities.  Quite 
the  contrary  to  surfing. 
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Abstract:  The  World-Wide  Web  has  the  potential  to  significantly  change  the  ways  in  which 
students  interact  with  texts,  making  them  more  active  participants  in  their  reading. 
Unfortunately,  while  many  see  the  Web  as  promoting  interaction,  most  pages  provide  relatively 
passive  modes  of  interaction,  only  permitting  students  to  click  on  one  of  a series  of  links  to 
select  what  to  read  next.  In  this  paper,  we  present  one  step  toward  making  pages  more 
interactive:  a system  that  permits  students  to  annotate  arbitrary  Web  pages  with  notes,  glosses, 
questions  and  comments;  to  share  those  annotations  with  their  colleagues;  and  to  develop 
discussions  through  those  annotations. 


1.  Introduction 

The  World-Wide  Web  (Berners-Lee  et  al.  1994).  is  revolutionizing  the  ways  in  which  we  share  information.  In 
particular,  it  is  affecting  the  ways  in  which  we  teach  and  learn.  Unfortunately,  while  the  Web  is  often  seen  as 
making  students  more  active  learners,  most  students  can  only  interact  with  pages  passively,  by  reading  and  clicking 
links.  Some  educators  suggest  that  this  is  no  more  active  than  flipping  pages  or  looking  through  the  table  of  contents 
or  index  in  a book. 

In  fact,  paper  provides  many  opportunities  for  active  interaction  with  the  material  being  studied.  On  paper,  students 
freely  mark  the  pages  in  their  texts,  underlining  words  or  phrases,  jotting  notes  in  the  margins,  summarizing  key 
sections,  and  even  folding  down  comers  to  remind  themselves  of  particularly  important  points  (see,  e.g.,  (Marshall 
1998)).  While  some  of  these  aspects  are  included  as  core  Web  technologies  (e.g.,  the  lists  of  bookmarks  provided  by 
most  browsers  are  similar  to  the  folded  comers  of  printed  texts,  and  perhaps  more  usable),  others  are  less  available 
(e.g.,  it  is  difficulty  to  jot  notes  electronically  and  keep  them  on  an  existing  page). 

In  this  paper,  we  describe  a system  we  have  developed  to  support  annotations  to  pages  on  the  World-Wide  Web 
(WWW)  Our  system  permits  learners  to  add  comments,  notes,  and  glosses  to  existing  Web  pages,  thereby  helping 
students  become  more  active  learners.  While  our  system  does  not  provide  all  of  the  mechanisms  suggested  in 
(Marshall  1998)  (e.g.,  we  do  not  support  highlighting),  our  system  provides  an  appropriate  starting  point  for 
developing  richer  systems. 

We  have  developed  a system  that  permits  selected  annotation  of  arbitrary  Web  pages.  Our  system  permits  students 
to  annotate  pages  with  public  and  private  notes,  glosses,  questions,  answers,  and  discussions.  We  expect  that  as 
students  have  increased  ability  to  interact  with  Web  pages,  they  will  become  more  active  learners. 

There  are  other  annotation  tools  (or  Web-based  systems  that  include  annotations)  available,  such  as  ComMentor 
(Roscheisen  1995),  CoNote  (Davis  1996),  the  Group  Annotation  Transducer  (GrAnT)  (Schickler  et  al.  1996), 
HyperWave  (Maurer  1996),  Medium  (Lapique  and  Regev  1998),  and  WebCT  (Goldberg  1998).  We  distinguish  our 
system  from  by  supporting  annotations  of  both  local  and  remote  pages;  permitting  annotations  at  author-defined  and 
automatically-generated  positions  on  the  page;  supporting  annotations  on  changing  pages;  and  providing  multiple 
protection  levels  for  annotations. 
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We  developed  this  annotation  system  to  serve  multiple  purposes.  First,  an  annotation  system  would  provide  a way 
for  a student  to  take  personal  notes.  Given  the  increasing  number  of  course  pages  available,  and  the  increased  use  of 
computers  in  the  classroom,  it  may  be  possible  for  students  to  take  online  notes  on  course  pages  during  class.  For 
example,  (Rebelsky  1998)  reports  that  in  a computer-equipped  classroom  in  which  daily  course  outlines  were 
available  online  during  class,  some  students  opened  the  daily  outline  in  an  HTML  editor  to  add  their  own  notes. 
Second,  an  annotation  system  is  also  an  appropriate  way  to  share  notes.  For  example,  one  might  post  questions 
about  an  assignment  or  suggest  followup  readings.  Third,  students  can  use  sequences  of  notes  to  provide  a 
discussion  (similar  to  a typical  threaded  bulletin-board,  but  with  focus  on  a particular  point  on  a Web  page). 

For  an  annotation  system  to  be  successful,  it  must  not  only  accommodate  the  needs,  preferences,  and  habits  of  its 
users,  but  also  predict  and  shape  them.  A student  or  instructor  new  to  hypermedia  authoring  and  usage  may 
understand  the  ability  to  add  notes  to  a page,  but  may  not  think  about  other  issues,  such  as  protection  (who  should 
be  able  to  see  an  annotation),  selection  of  annotation  points  (where  annotations  should  go  in  a document),  or 
appearance  and  positioning  of  annotations.  We  report  on  these  design  issues  and  the  user  testing  that  guided  our 
design  choices  in  the  subsequent  sections. 

2.  Architecture 

Among  our  primary  goals  was  to  design  an  annotation  system  that  could  be  installed  and  used  widely.  This  led  to  a 
number  of  core  requirements,  including 

• a teacher  should  be  able  to  install  and  use  the  annotation  system  without  making  modifications  to  the  Web  page 
server; 

• students  and  teachers  should  be  able  to  use  the  annotation  system  with  current  Web  clients;  and 

• students  should  be  able  to  annotated  a wide  variety  of  pages,  including  pages  on  different  servers  and  pages  that 
neither  teachers  nor  students  had  direct  control  over. 

These  requirements  led  to  an  architecture  based  on  a centralized  annotator  and  an  extended  version  of  the  core  of 
Project  Clio  (Becker  and  McLaughlin  1998)  (Becker  et  al.  1999).  A single  CGI  script  is  used  to  coordinate  all 
annotation  activities.  A page  request  is  routed  through  the  CGI  script.  The  CGI  script 

1 . retrieves  the  page  from  a server  (or  from  the  file  system,  if  the  page  is  stored  locally  to  the  CGI  script), 

2.  adds  annotation  points  (described  in  section  2.2), 

3.  identifies  the  annotator  (including  group  permissions), 

4.  fetches  annotations  from  an  annotation  server, 

5.  inserts  the  annotations  or  links  to  the  annotations  into  the  page, 

6.  updates  the  links  on  the  page  so  that  the  CGI  script  is  used  for  further  page  fetches,  and 

7.  returns  the  modified  page. 

In  a future  version,  we  expect  to  be  able  to  use  a form  of  proxy  server  for  the  same  purpose. 

The  retrieval  of  original  page,  identification,  link  update,  and  return  are  handled  by  Project  Clio.  The  addition  of 
annotation  points,  fetching  of  annotations,  and  insertion  of  annotations  were  added  for  this  project. 

Because  the  annotation  system  is  based  on  a CGI  script,  it  is  relatively  easy  to  install  (some  servers  may  need  a flag 
set  to  perniit  use  of  CGI  scripts).  Since  it  does  not  require  modification  to  the  underlying  pages,  it  permits 
annotation  of  pages  that  neither  teachers  nor  students  control.  Since  the  determination  and  fetching  of  annotations  is 
done  at  the  server  side,  and  not  the  client  side,  it  requires  no  extensions  to  clients.  However,  the  current  mechanism 
for  displaying  annotations  does  require  JavaScript,  which  is  supported  by  both  of  the  most  popular  browsers 
(Netscape  Communicator  4.0  and  Internet  Explorer  4.0).  The  display  mechanism  is  described  in  section  3. 
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A settings  file  can  be  used  to  customize  some  aspects  of  the  CGI  script.  In  particular,  it  can  be  set  to  accept  requests 
from  only  certain  domains  or  certain  accounts.  This  permits  instructors  to  limit  access  to  annotations.  (The 
protection  system  described  in  section  2.1  provides  additional  forms  of  access  control.)  The  settings  file  can  also 
influence  which  pages  can  and  cannot  be  annotated.  For  example,  some  instructors  might  wish  to  limit  annotations 
to  their  own  pages,  or  to  a particular  set  of  pages  they  have  already  identified. 

2.1.  Protection 

There  are  three  trends  for  protection  of  annotations:  annotations  are  either  private  (accessible  only  to  the  user), 
group/semi-private  (restricted  to  a group  of  users),  or  public  (accessible  to  anyone  reading  the  page).  Many  systems 
permit  only  one  type  of  annotation.  For  example,  CoNote  provides  only  group  annotations.  Similarly,  systems  treat 
the  different  types  of  annotations  quite  differently.  For  example,  WebCT  (Goldberg  1998)  separates  annotations, 
which  are  private,  from  discussions,  which  are  semi-private,  using  different  interfaces  for  the  two. 

We  chose  to  provide  all  three  “types’*  of  annotations  with  a consistent  interface.  When  a user  creates  a new 
annotation,  (s)he  may  choose  to  make  it  private,  group,  or  public.  Because  we  could  foresee  different  groups  of 
people  using  the  same  page  (e.g.,  faculty,  teaching  assistants,  and  students  might  all  have  discussions  on  a course 
page),  group  protection  also  includes  a note  as  to  which  groups  may  view  the  annotation.  When  browsing 
annotations,  readers  can  see  the  headers  of  all  the  annotations,  but  may  only  view  the  bodies  of  annotations  for 
which  they  have  appropriate  permissions. 

2.2.  Annotation  Points 

An  annotation  system  for  the  Web  must  accommodate  the  differences  between  the  Web  and  paper.  One  particularly 
important  difference  is  that  while  paper  remains  fixed  (and  any  errata  are  separate  documents),  Web  pages  often 
change.  This  means  that  annotations  cannot  be  conveniently  tied  to  a particular  piece  of  text.  Where  should 
annotations  appear? 

Some  annotation  systems,  such  as  the  original  NCSA  Mosaic  annotations  (NCSA  1997),  place  annotations  at  the 
page  level,  with  one  annotation  per  page.  Such  placement  seems  overly  course  given  the  more  typically  fine-grained 
annotations  students  typically  make.  Discussions  with  faculty  and  students  suggested  that  there  is  need  for  more 
specific  placement  of  annotations.  For  example,  a question  about  a particular  problem  in  a homework  assignment 
best  belongs  near  that  problem.  More  importantly,  an  instructor  interested  in  having  students  gloss  a work  (i.e.,  add 
commentary)  will  not  be  satisfied  with  annotations  that  are  so  coarse-grained. 

A finer-grained  ideal  would  be  to  permit  users  to  select  the  text  to  be  annotated.  Such  a solution  is  used  in  the 
Multivalent  Document  system’s  markup  system  (Phelps  and  Wilensky  1997).  Unfortunately,  standard  HTML 
browsers  will  not  easily  support  this  choice.  One  alternative,  used  in  the  Group  Annotation  Transducer  (Schickler  et 
al.  1996),  is  to  allow  users  to  specify  the  text  that  will  be  tagged.  Unfortunately,  entering  the  appropriate  text 
requires  more  effort  on  the  part  of  the  user  and  also  suggests  difficulties  if  the  page  changes  or  if  the  same  phrase 
appears  multiple  times.  Both  of  these  fine-grained  solutions  may  lead  to  annotations  at  awkward,  confusing,  or 
otherwise  inappropriate  points  on  the  page. 

Another  alternative  is  to  allow  authors  to  specify  the  annotation  points  in  a document,  as  in  CoNote  (Davis  1996). 
The  disadvantages  of  such  an  alternative  is  that  it  places  all  the  burden  on  the  author,  and  provides  little  freedom  for 
the  annotator.  An  advantage  is  that  it  permits  annotations  to  remain  through  changes  of  the  document  (something 
particularly  difficult  in  phrase-based  annotations). 

We  chose  a hybrid  solution,  in  which  annotation  points  can  be  added  by  the  author,  but  are  also  generated  by  the 
system  for  “natural”  points  in  the  document.  At  present,  these  points  are  currently  set  to  be  section  headings,  which 
often  designate  a logical  break  from  one  thought  to  another.  For  typical  documents,  section  headings  are 
fine-grained  enough  to  permit  annotation  at  a reasonable  level  but  coarse-grained  enough  that  the  document  is  not 
over-croweded  with  annotation  points.  We  use  heuristics  based  on  approximate  text  matching  to  determine  the 
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(re-)placement  of  annotation  points  in  changed  documents. 

3.  The  User  Interface 

A similarly  interesting  problem  was  where  to  put  the  annotations  themselves.  A number  of  alternatives  suggested 
themselves  (many  of  which  have  been  used  in  other  systems).  One  might  make  links  to  the  annotations  and  bring 
them  up  in  the  same  window  when  clicked.  Such  a solution  is  easy  to  develop,  but  makes  it  more  difficult  for  the 
novice  Web  user  to  view  both  text  and  annotations.  One  might  make  links  to  the  annotations  and  bring  them  up  in 
the  same  window.  One  might  insert  the  annotation  text  into  the  same  page.  However,  this  can  make  the  original  text 
more  difficult  to  read,  as  the  annotation  interrupts  the  flow  of  the  text.  One  might  use  popup  windows,  in  which  the 
annotation  appears  over  the  annotated  text  when  one  moves  the  mouse  over  that  text  or  clicks  on  it. 

3.1.  Surveying  Potential  Users 

We  decided  that  this  was  not  an  issue  we  could  approach  in  the  abstract,  after  the  system  was  constructed.  Hence, 
we  surveyed  potential  users  about  the  strengths  and  weaknesses  of  a variety  of  designs.  We  developed  eight 
different  prototype  interfaces  for  an  annotation  system  and  surveyed  students  with  a variety  of  computer 
background.  The  examples  varied  in  appearance  and  technical  design  issues.  One  example  displayed  annotations  in 
a separate  pop-up  window.  Another  brought  the  user  to  a new  page  which  displayed  the  annotation  and  the 
corresponding  paragraph  (the  annotated  text).  In  a third  design,  the  annotations  were  visible  on  the  page,  within  the 
document.  For  another  the  annotations  were  in  the  margin.  The  colors  and  icons  changed  from  example  to  example, 
while  the  text  and  annotations  were  constant.  Since  we  worried  about  interrupting  the  flow  of  text,  a few  examples 
permitted  the  users  to  turn  the  annotation  feature  on  or  off  We  did  not  test  the  interface  for  entering  the  system  (in 
which  students  enter  their  account  and  password)  which  is  taken  from  Project  Clio  (Becker  and  McLaughlin  1998) 
(Becker  etal.  1999). 

Most  participants  preferred  a pop-up  window  to  view  and  add  annotations,  since  they  did  not  want  to  disrupt  the 
page  they  were  on.  Annotations  directly  on  the  Web  page  seemed  to  add  clutter  and  make  the  page  less  readable. 
Also,  the  less  obtrusive  the  annotation  buttons  were,  the  better.  At  the  same  time,  many  found  the  ability  to  hide  and 
show  the  annotation  buttons  (or  the  annotations  themselves)  confusing. 

Based  on  this  feedback,  we  developed  a hybrid  of  the  eight  used  in  the  survey.  We  chose  to  use  two  small  buttons  to 
indicate  the  annotation  points.  One  button  is  used  to  add  annotations  and  appears  at  every  annotation  point  in  the 
document.  The  second  button  is  used  to  indicate  the  presence  of  annotations  at  that  particular  point  in  the  document, 
and  is  only  used  if  there  are  annotations  for  that  annotation  point. 

3.2.  Interacting  with  the  system 

If  one  clicks  on  the  View  Annotations  button,  a separate  window  appears  showing  a collapsible  tree  of  annotations 
corresponding  to  that  annotation  point.  There  is  a tree  of  annotations  because  we  permit  readers  to  write  followups 
to  other  annotations,  and  found  a tree  the  most  natural  way  to  represent  the  many  discussions  that  may  come  from 
one  annotation  point. 

If  there  is  only  one  annotation  available  for  a particular  annotation  point,  that  annotation  is  used  as  the  ‘‘alt  text”  for 
the  annotation  button.  In  modem  browsers,  such  as  Netscape  Navigator  4.0,  this  provides  a quick  pop-up  view  of  the 
annotation.  If  multiple  annotations  are  available  for  a particular  annotation  point,  the  number  of  annotations  is  used 
as  the  “alt  text”. 

When  the  user  selects  the  Add  Annotation  button  or  the  Followup  button,  a new  window  appears.  The  annotator 
can  enter  a name,  title,  and  the  annotation.  Annotators  can  also  select  permissions:  self,  selected  groups,  everyone  on 
system,  world.  Finally,  an  annotator  can  choose  to  make  an  annotation  anonymous  (so  that,  for  example,  students 
can  post  comments  without  fear  of  appearing  less  confident  or  successful  they  wish  to  appear). 
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While  the  annotations  do  not  preserve  text  formatting  (e.g.,  whitespace),  they  can  be  written  in  HTML,  the 
HyperText  Markup  Language  (Raggett  et  al.  1998).  This  gives  authors  freedom  to  format  their  comment  or  question 
by  adding  emphasis,  tables,  lists,  and  other  structure. 

3.3.  Followup  Surveys 

After  the  new  interface  was  complete  and  the  underlying  system  implemented  (so  that  subjects  could  add  their  own 
annotations  and  not  Just  view  existing  annotations),  a second  group  of  subjects  was  brought  in  to  test  the  new 
interface.  This  group  of  subjects  included  some  of  the  first  group,  but  also  some  new  subjects. 

While  participants  were  generally  satisfied  with  the  structure  of  the  implemented  annotation  system  (e.g.,  the  pop-up 
windows),  many  identified  some  technical  problems  that  would  not  be  obvious  in  a prototype.  For  example,  many 
pointed  out  that  the  windows  should  auto-reload,  so  they  can  see  the  changes  they  made  immediately.  It  was  also 
noted  that  a close  button  was  needed  in  the  annotation  window.  Finally,  many  subjects  needed  more  ready  access  to 
a help  system. 

4.  Future  Work 

We  have  developed  and  begun  testing  of  a system  that  we  expect  will  enhance  the  effectiveness  of  course  pages  by 
permitting  students  to  add  notes  and  questions  to  Web  pages  in  and  out  of  the  classroom.  Our  initial  user-testing 
suggests  that  this  will  be  a usable  system,  but  we  do  not  yet  know  how  much  or  how  it  will  be  used.  Because  the 
system  depends  on  parts  of  Project  Clio,  we  must  wait  until  Fall  1999  for  our  initial  course  trials.  Based  on  feedback 
from  those  trials,  we  expect  to  make  extensions  and  modifications  to  the  system. 

We  also  expect  to  refine  the  interface  for  the  “annotation  tree”  (the  set  of  annotations  for  a particular  point  in  the 
document).  While  we  conducted  regular  testing  of  the  placing  and  appearance  of  annotations  relative  to  the  page,  we 
were  able  to  do  less  testing  of  the  appearance  of  this  tree. 

Finally,  we  intend  to  provide  alternate  “views”  of  the  annotations  so  that,  for  example,  a student  who  wishes  to  print 
a document  with  annotations  may  conveniently  do  so.  We  expect  to  test  at  least  two  alternates  for  this  “all  in  one” 
view:  one  in  which  the  annotations  are  placed  near  the  text  they  annotate  and  one  in  which  they  are  used  as  footnotes 
(with  footnote  numbers  places  in  the  text). 
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Abstract:  An  expanded  multimedia  taxonomy  is  proposed  to  provide  a framework  for  understanding  and 
conceptualizing  interactive  multimedia  for  the  purposes  of  teaching,  research  and  project  design.  The  taxonomy  is  an 
especially  helpful  tool  for  the  generation  of  evaluation  materials.  Specific  examples  of  how  to  generate  evaluation 
categories  and  concerns  is  included. 


WHAT  IS  MULTIMEDIA? 

Multimedia  is  a polysemous  term  - a term  with  many  definitions  and  in  this  case  many  roots.  In  this  paper, 
multimedia  is  defined  as  the  seamless  integration  of  two  or  more  medium.  A multimedia  taxonomy  is  proposed  to 
help  organize  the  discipline  based  on  a previous  media  taxonomy  (Heller  & Martin,  1995).  The  taxonomy  helps  to 
classify  the  space  called  multimedia  and  helps  to  draw  attention  to  difficult  issues.  While  multimedia  is  an  emerging 
field,  the  fact  that  we  can  provide  a taxonomy  indicates  that  it  has  reached  some  level  of  stability  and  maturity  to  be 
reckoned  with. 


THE  TAXONOMY 

The  multimedia  taxonomy  can  be  visualized  as  a three  dimensional  matrix  (Heller  and  Martin,  1998).  Media  type, 
the  rows  of  the  matrix,  pertains  to  the  various  media  involved:  text,  sound,  images,  motion  and  combinations.  Media 
expression,  the  columns,  refers  to  the  level  of  abstraction  portrayed  using  these  media:  elaboration,  representative  and 
abstraction.  Context,  the  third  dimension,  portrays  the  various  roots  in  multimedia:  disciplines,  interactivity, 
audience,  aesthetics,  quality  and  usefulness.  As  originally  expressed  the  taxonomy  had  90  cells  in  three  dimensions  to 
be  defined  (Heller  and  Martin,  1995).  In  the  beginning  work,  it  was  observed  that  evaluation  or  design  questions 
often  repeated  themselves,  regardless  of  the  medium  form  or  expression  format.  For  example,  questions  about  text 
size  are  appropriate  no  matter  the  format  of  textual  presentation.  Based  on  this  observation,  the  taxonomy  was 
expanded  to  provide  a category  in  the  format  called  general  - this  category  is  used  for  questions  that  are  constant  over 
the  format  space.  Hence  there  are  120  cells  provided  by  the  taxonomy  and  each  can  be  expressed  as  a series  of 
questions  that  can  be  used  as  a development  or  evaluation  guide.  The  newly  expanded  taxonomy  becomes  a floor,  not 
a ceiling,  for  a series  of  guidelines  that  can  be  used  to  generate  a series  of  questions  about  an  application. 

Media  type  is  arranged  as  a series  of  individual  medium  of  increasing  complexities  (i.e.  storage).  The  various  media 
should  be  clear.  Text  is  the  presentation  of  information  using  an  alphabetical  symbol  system.  This  includes  prose  is 
various  languages  as  well  as  presentations  in  such  forms  as  mathematics  or  other  symbol  systems.  Sound  is  the 
inclusion  of  spoken  word  as  well  as  generated  tones  forming  music  or  other  audible  information.  Images  include 
photos  and  hand  drawn  items  while  motion  can  be  motion  pictures  or  animations.  Multimedia,  as  defined  above,  is 
the  inclusion  of  any  two  or  more  of  these. 
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Form 
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Elaboration 

Representation  Abstraction 
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Font  for  bold 

correct 

and  italics  is 

Is  it  clear  which  input 

readable 

device  to  use 
Word  choice  was 
unclear/clear 
Grammar  was  correct 
Reading  characters 
on  screen 

(very  hard  to  very  easy) 
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Speech  is  Sound  effects 

(unclear  to  clear) 

Captured  are  recognizable 

Sound  speed 

without  stutters 

(too  fast/slow.. just  right) 
Sound  volume 
(too  loud/soft,  .just  right) 
GraphicQuality  of  image 

Icons  are  clear 

(low  to  high) 

Iconic  Metaphor 

Size  of  image 

is  sharp 

(too  large/small  to 
just  right) 

Motion  Quality  of  motion 
(jerky  to  smooth) 
Multi  Media  Integration 
media  (Uncoordinated  to 
coordinated) 
Media  technical 
synchronization 


FIGURE  1:  Sample  Questions  from  the  Quality  Dimension 

Contextual  categories  that  are  not  applicable  to  the  specific  product  being  evaluated  can  be  eliminated  from  the 
protocol.  The  questions  (Figure  1)  are  framed  either  as  a Likert  scale  question  with  five  gradations  or  an  open  ended 
questions  requiring  a free  form  user  response.  Those  that  are  intended  to  be  considered  using  a Likert  scale  are 
presented  with  the  categorical  extremes  listed  in  parenthesis.  For  example,  to  evaluate  the  item  listed  as  use  of 
motion  one  can  generate  a question  for  the  form: 

Jerky  Smooth 

Use  of  motion  1 2 3 4 5 

The  Multimedia  Taxonomy  is  best  understood  in  its  application.  The  new  media  space  is  6x4x5,  where  each  cell  can 
be  used  to  address  specific  aspects  of  media  design  or  evaluation.  For  example,  the  aesthetics  category  might 
encompass  the  following  concerns:  In  the  general  expression  of  text,  we  might  ask,  is  the  font  size  appealing,  while 
in  the  elaboration  expression  of  text,  it  is  appropriate  to  ask  the  aesthetic  question  concerning  the  richness  of  the 
literary  expression  on  the  continuum  from  dry  to  poetic.  In  representational  expression  of  text,  we  might  ask  whether 
the  format  of  the  outline  is  logically  well-structured  and  in  considering  the  abstract  expression  of  text  within  the 
aesthetic  categories,  we  could  ask  if  the  textual  image  as  an  icon  is  pleasing. 
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Similarly,  when  considering  questions  of  audience  in  the  general  category  for  text  we  might  ask  if  the 
instructions  are  clear,  while  in  elaborative  or  representational  expressions  of  text  we  might  ask  evaluative  questions 
about  whether  the  feedback  to  the  audience  is  appropriate  or  if  there  is  a compatibility  between  the  reading  level  of 
the  audience  and  the  level  of  expression  of  the  textual  description.  The  usefulness  category  prompts  us  to  assess  the 
value  added  by  a medium  type  or  expression.  The  extended  Multimedia  Taxonomy  elicits  questions  about  sound 
quality  as  well  as  sound  impact. 

Under  the  category  of  Usefulness,  the  evaluation  protocols  contained  questions  about  whether  a medium  like 
sound  was  functional,  by  asking  users  to  rate  the  sound  on  a Likert  scale  that  ranged  from  annoying  to  helpful.  The 
protocols  also  contained  open-ended  questions  about  what  the  user  remembered  about  the  sound  in  the  application 
being  evaluated.  Thus,  the  newly  expanded  Multimedia  Taxonomy  becomes  a floor,  not  a ceiling,  for  a series  of 
guidelines  that  can  be  used  to  generate  evaluation  questions  about  a multimedia  application.  Parts  of  the  taxonomy 
not  relevant  to  the  specific  product  being  evaluated  can  be  ignored  when  developing  the  evaluation  protocol  and  data- 
gathering  instruments. 


AESTHETIC 

The  guidelines  in  this  section  are  intended  to  examine  the  appearance,  artistic  look  or  impression  of  the  IMM. 
Questions  related  to  design,  rather  than  function,  are  the  focus  in  this  section  Figure  2 is  an  example  of  questions  for 
aesthetics. 


AUDIENCE 

Audience  guidelines  are  intended  to  direct  the  evaluator  to  address  issues  of  how  the  IMM  relates  to  the 
audience  and  how  the  audience  members  might  be  able  to  process  the  medium  form  in  a specific  medium  format.  For 
example  in  the  area  of  text  as  elaboration,  it  is  necessary  to  ask  whether  the  text  is  at  the  reading  level  for  the 
intended  audience.  On  the  other  hand,  while  text  as  representation  does  require  the  user  to  read,  there  tend  to  be 
fewer  words  and  grammatical  constructions  in  this  category  but  the  representation  (e.g.  outline)  might  not  be  familiar 
to  the  particular  audience.  This  section  requires  the  evaluator  to  know  to  whom  the  IMM  is  intended.  Before 
preparing  an  evaluation  protocol  for  a particular  IMM,  the  evaluator  should  know  who  the  intended  audience  is. 


DISCIPLINE 

Guidelines  in  this  section  are  intended  to  identify  the  content  specific  material  in  each  of  the  media  forms 
within  a specific  medium  format.  For  example  in  reviewing  text  that  is  presented  in  representational  form  (outlines 
or  lists)  is  necessary  to  ask  here  whether  the  lists  are  complete  and  accurate  within  the  content  area.  This  section  is 
often  best  evaluated  by  experts  in  the  content  area  rather  than  by  IMM  designers  or  members  of  the  intended 
audience. 


o 

ERIC 
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Metaphors  arc  consistent 
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Media  overuse? 
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(not  pleasing  to 
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Background 
images  (too  faint) 

Icons  pleasing? 

Figure  2:  Sample  questions  based  on  aesthetics  issues 


INTERACTIVITY 

In  this  category,  evaluators  should  be  addressing  aspects  of  control,  navigation  and  linking.  Aleem  (1997)  is 
examining  the  relationship  between  the  attribute  of  interactivity  to  the  media  type  and  expression.  He  has  further 
subdivided  the  attribute  of  interactivity  into  four  categories:  Passive,  Reactive,  Proactive  and  Directive.  With  the 
Passive  aspect  of  interactivity,  the  user  has  no  control,  but  instead,  all  control  is  embodied  in  the  application  (i.e.,  a 
PowerPoint  presentation)  Reactive  interactivity  provides  limited  response  for  the  user  within  a scripted  sequence. 
Proactive  interactivity  allows  the  user  to  play  a major  role  in  the  design  and  construction  of  situations,  typically  by 
manipulating  values  for  valuables.  Multimedia  that  has  interactivity  at  the  Directive  level  allows  the  user  both  to 
respond  to  and  to  initiate  actions  within  the  application  as  well  as  to  tailor  aspects  of  the  environment,  such  as 
selection  of  color  choice,  feedback  choice,  and  so  on. 


QUALITY 

This  section  refers  to  the  technical,  reproductive  aspects  of  the  IMM.  Quality  issues  relate  to  clear  images, 
correct  synchronization  and  timely  delivery. 


USEFULNESS 

This  category  refers  to  the  value  of  the  material  presented  in  the  IMM  as  well  as  the  ease  of  use  of  this 
material.  Since  Interactivity  covers  the  navigational  issues  and  Usefulness  covers  the  ease  with  which  a user  can 
operate  the  application  there  seems  to  be  a muddy  ground  between  the  two  categories.  But  there  is  a distinction. 
Usefulness  also  includes  whether  the  user  can  operate  the  equipment  necessary  to  make  the  application  run.  For 
example,  questions  such  as  can  the  user  operate  the  head  tracker  or  the  roller  ball  as  well  as  is  there  a need  for 
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external  devices  to  run  the  IMM  are  appropriate  in  the  ease  of  use  category. 

As  can  be  judged  from  the  foregoing  paragraphs,  not  all  of  the  120  cells  contained  within  the  three-dimensional 
Multimedia  Taxonomy  are  completed.  Some  are  being  examined  in  detail,  while  others  are  still  to  be  developed. 
Students  in  courses  in  the  Seminar  on  Multimedia  Evaluation  in  the  Department  of  Electrical  Engineering  and 
Computer  Science  at  The  George  Washington  University  (Heller,  1994)  have  been  successful  in  designing  evaluation 
protocols  using  the  Multimedia  Taxonomy  as  a guideline.  Figure  3 represents  a sample  of  the  survey  comments 
developed  from  the  formative  evaluation  of  Ada  Mentor,  an  online  web  site  for  the  study  of  the  Ada  programming 
language.  The  Multimedia  Taxonomy  helped  pinpoint  such  problems  as  too  many  forms  within  the  web  site  and  the 
inability  for  a user  to  see  the  entire  page.  They  were  able  to  suggest  on  the  bases  of  the  evaluation,  that  the  Ada 
Mentor  developers  reduce  the  navigational  capabilities  and  clarify  the  functionality  of  navigational  icons. 


CONCLUSIONS 

We  believe  the  multimedia  taxonomy  is  a reasonable  organizing  framework.  Admittedly  there  is  room  for  extension 
of  the  multimedia  taxonomy  if  new  ways  of  thinking  about  multimedia  are  presented.  The  extensibility  of  the 
taxonomy  is  demonstrated  by  the  ability  to  take  a cell  or  series  of  cells  and,  using  them,  form  new  detailed  sub- 
taxonomies. This  is  demonstrated  by  the  work  presented  by  Aleem  (1997). 

The  multimedia  taxonomy  is  a solid  framework  - the  details  of  each  of  the  cells  needs  to  be  completed.  Currently  we 
are  beginning  usability  studies  using  the  taxonomy.  Specifically,  we  are  interested  in  whether  evaluators  can  use  the 
taxonomy  as  a guideline  to  improve  the  coverage  of  their  existing  evaluation  protocols  or  does  the  fragmentation  of 
the  taxonomy  interfere  with,  or  distract  from,  the  evaluation.  In  continuing  efforts  to  establish  quality  evaluation 
products  we  are  also  beginning  to  review  the  use  of  object  oriented  design  techniques  and  metaphors  as  a possible 
format  for  expressing  multimedia  evaluation.  As  a beginning  we  are  considering  each  medium  (text,  sound,  stills 
and  motion)  as  a specific  object  and  each  object  has  its  specific  attributes  (size,  color,  form  declaration, 
representation  and  form)  and  behaviors  (interactions).  Objects  can  form  clusters  and  it  is  these  clusters  that  can  have 
different  relations  and  interdependency.  Investigations  into  the  use  of  cluster  analysis  as  a methodology  for 
multimedia  evaluations  is  just  beginning. 

As  an  organizing  principle  the  Multimedia  Taxonomy  presented  here  can  be  used  to  understand  both  design  and 
content  messages.  The  next  step  is  to  review  the  taxonomy  in  light  of  various  studies  in  the  psychological  and 
cognitive  aspects  of  understanding  of  multimedia  applications  to  determine  whether  this  taxonomy  can  shed  light  on 
these  areas.  Questions  such  as  how  we  come  to  understand  an  image  and  how  that  understanding  is  different  from 
our  understanding  of  text  can  be  answered  in  part  by  using  the  taxonomy.  We  have  presented  a few  examples  of  the 
impact  of  the  Multimedia  Taxonomy  on  the  design  and  implementation  of  evaluation  protocols  for  multimedia 
products.  More  work  in  this  area  remains  to  be  done.  Finally,  however,  the  Multimedia  Taxonomy  represents  an 
attempt  at  the  formalism  that  is  needed  to  provide  the  both  a theoretical  and  practical  framework  for  the  new  and 
rapidly  growing  field  of  interactive  multimedia. 
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AdaMentor  Formative  Evaluation  Comments  for  Open  Questions 
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FIGURE  3:  Selected  Responses  to  Formative  Evaluation  Protocol  Using  the  Taxonomy 
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Abstract:  The  design  and  development  issues  surrounding  a World  Wide  Web-based  site  to  house 
documentation  for  a national-level  accreditation  of  a College  of  Education  is  daunting,  to  say  the 
very  least.  Many  different  facets  introduced  themselves  through  out  the  design  and  development  of 
such  a Web  site.  The  Web  site  in  question  concerns  a national  accrediting  body,  National  Council 
for  the  Accreditation  of  Education  (NCATE),  that  has  led  a “bleeding  edge”  push  towards  the 
design  and  development  of  a Web  site  that  houses  pertinent  documentation  concerning  the 
professional  education  unit  that  is  undergoing  the  accreditation  process.  Although  numerous 
possible  future  questions  and  concerns  within  the  design  and  development  process  were  thoroughly 
discussed  and  responses  were  decided  upon,  there  were  numerous  occurrences  throughout  the 
design  and  development  process  that  were  not  expected.  These  occurrences  are  the  primary  focus 
of  learning  and  should  be  discussed  through  a much  larger  audience  so  that  others  working  through 
such  a process  will  be  familiar  with  any  concerns  that  may  arise  within  their  own  situation. 


Introduction 

Accreditation  has  become  a primary  concern  within  the  professional  education  units  over  the  previous 
fifteen  years.  Although  quality  standards  have  been  in  place  within  the  educational  realm  for  well  over  a fifty-year 
period,  recent  events  and  publicity  have  caused  the  strengthening  of  accreditation  through  out  the  United  States  of 
America.  The  accreditation  process  is  built  upon  clearly  defined  standards  that  each  professional  education  unit  must 
delineate  through  out  the  accreditation  process.  As  stated  by  the  National  Commission  on  Teaching  and  America’s 
Future, 


Setting  Standards  is  like  building  a pyramid:  Each  layer  depends  on  the  strengths  of  the 
others.  Students  will  not  be  able  to  achieve  higher  standards  of  learning  unless  teachers 
are  prepared  to  teach  in  new  ways  and  schools  are  prepared  to  support  high-quality 
teaching.  Higher  standards  for  students  must  ultimately  mean  higher  standards  for 
teachers  and  schools.  Otherwise,  the  end  result  of  the  standards  movement  will  be  more 
clearly  documented  failure  rather  than  higher  levels  of  overall  achievement.  (1996,  p.  27) 

Due  to  this  stated  desire  to  raise  the  standards  of  the  educational  units  within  the  United  States  of  America,  a focus 
has  been  placed  upon  accrediting  bodies.  Interestingly,  accrediting  bodies  at  different  levels  through  out  the 
educational  realm.  From  specialization  areas,  such  as  Spanish  through  instructional  technology,  through  overarching 
accrediting  bodies,  such  as  the  National  Council  for  the  Accreditation  of  Teacher  Education  (NCATE). 

Herein  lies  an  interesting  element.  Due  to  the  strong  feelings  of  necessity  towards  an  overarching, 
professional  education  accreditation,  Roames  (1997)  describes  the  national  conference  of  1951  wherein  the  efforts 
of  at  least  four  representative  council  organization  bodies  came  together.  As  a result  of  these  efforts,  the  National 
Council  for  the  Accreditation  of  Teacher  Education  (NCATE)  was  approved.  Although  NCATE  was  developed 
almost  fifty  years  ago,  the  numerous  reforms  that  NCATE  has  worked  through  have  developed  a stronger,  living 
organization  that  spearheads  accreditation  processes.  Herein  lies  the  focus  of  this  article;  NCATE  developed  a task 
force,  the  NCATE  Task  Force  on  Technology  and  Teacher  Education  (National  Council  for  the  Accreditation  of 
Teacher  Education,  1997)  which  noted  the  desire  to  “establish  pilot  projects  with  a few  institutions  to  implement  and 
evaluate  state-of-the-art  uses  of  technology  in  the  current  accreditation  process  ....  These  pilot  projects  should  be 
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conducted  both  by  institutions  seeking  initial  accreditation  and  those  pursuing  continuing  accreditation”  (National 
Council  for  Accreditation  of  Teacher  Education,  1997,  p.  19).  This  statement  not  only  indicated  the  desire  of  the 
NCATE  organization  to  further  discuss  the  implementation  of  technological  innovations  within  the  accreditation 
process,  but  it  seemed  to  be  an  open  invitation  to  professional  education  units  to  establish  pilot  projects  to  introduce 
technological  innovation  into  both  the  initial  and  continuing  accreditation  process. 

Purpose  of  the  Study  Statement 

The  purpose  of  the  study  was  to  qualitatively  describe  the  design  and  development  of  a professional  education  unit 
Web  site  to  integrate  the  unit’s  applicable  documentation  concerning  the  units  anticipated  NCATE  continuing 
accreditation. 

Design 

Instructional  Design  Model 

The  Four  D Instructional  Development  Model  (Thiagarajan,  Semmel,  & Semmel,  1974)  was  implemented 
as  the  instructional  design  model  due  to  its  clarity  and  linear  nature.  However,  the  interesting  aspect  of  this 
instructional  design  model  choice  was  the  choice  of  a linear  model  to  develop  a nonlinear  learning  environment, 
which  may  also  be  described  as  ill-structured  landscape  environments  (Wittgenstein,  1953).  Questions  were  raised 
concerning  the  positive  as  well  as  detrimental  effect  that  a linear  model  would  have  upon  a nonlinear  learning 
environment  and,  after  numerous  discussions,  a clarity  of  thought  between  a linear  instructional  design  model  and  a 
nonlinear  learning  environment  was  reached.  Although  a linear  instructional  design  model  may  be  implemented  to 
develop  a product,  this  linearity  has  no  impact  upon  the  final  product  due  to  the  developmental  structure  versus  the 
product  structure;  although  a linear  model  is  decided  upon  due  to  it’s  clarity  of  process  and  ease  of  use  on  the  part  of 
the  designer  and  developer,  this  would  not  impact  the  product  of  a nonlinear  learning  environment. 

Nonlinear  Learning  Environment 

The  nonlinear  learning  environment,  clearly  displayed  through  the  possible  use  of  the  World  Wide  Web 
environment,  is  one  which  offered  the  ability  to  link  multiple  areas  of  pertinent  professional  education  unit 
documents  together  within  one  Web  site.  This  linking  ability  offered  a nonlinear  environment  through  which  the 
NCATE  Board  of  Examiners  (BOE)  members  who  would  ultimately  view  the  documents.  This  nonlinear 
environment  would  offer  a clarity  of  though  and  information  to  the  NCATE  BOE  members  and,  theoretically,  ease 
the  strain  placed  upon  NCATE  BOE  members  who  are  required  to  traverse  this  documentation  over  a short  time 
period.  Another  positive  aspect  to  the  implementation  of  a nonlinear  learning  environment  for  the  pertinent 
professional  education  unit  documentation  is  the  ability  of  the  NCATE  BOE  members  to  have  the  option  to  view  the 
pertinent  documentation  before  ever  stepping  foot  on  the  university  campus.  The  NCATE  BOE  members  had  the 
ability  to  view  relevant  documents  before  visiting  the  campus  that  would  offer  more  time  during  the  visit  to 
interview  and  view  areas,  persons,  and  places  of  interest. 

Qualitative  Data  Collection 

During  the  initial  components  of  the  research  process,  as  well  as  through  out  the  design  and  development  of 
the  professional  education  unit’s  NCATE  Web  site,  certain  aspects  were  clearly  articulated  as  having  relevance 
towards  the  importance  of  the  process  and,  therefore,  were  clearly  stated  as  areas  of  data  collection.  The  researcher 
implemented  a qualitative  data  collection  so  as  to  obtain  necessary,  relevant,  and  descriptive  data  pertaining  to  the 
design  and  development  of  the  Web  site.  The  three  areas  through  which  data  was  collected  through  out  the  design 
and  developmental  process  were  the  areas  of  historical  research,  participant  observation,  and  interview. 

Historical  Research 

Historical  research  describes  the  location,  obtainment,  development,  and  digitization  of  the  documentation 
that  was  integrated  into  the  Web  site.  Through  out  each  of  these  processes,  perhaps  the  most  difficult  would  be  the 
location  of  the  historical  documentation.  Although  the  faculty  and  staff  within  the  professional  education  unit  were 
clearly  helpful  and  went  beyond  their  duties,  the  difficulty  in  merely  locating  the  historical  documentation  was  a 
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timely  venture.  The  historical  documentation  was  available  to  the  Web  site  developer  without  question;  however,  the 
location  of  the  documentation  and  the  ability  to  actually  “put  hands  upon”  the  documentation  was  another  matter. 
Yet,  as  previously  stated,  the  faculty  and  staff  within  the  professional  education  unit  were  of  the  utmost  help  through 
out  the  location,  obtainment,  development,  and  digitization  of  the  documentation  process. 

Participant  Observation 

The  ability  of  the  researcher  to  become  an  integrated  member  of  the  professional  education  unit  was  of 
utmost  usefulness  through  out  the  design  and  developmental  period.  As  an  “insider”  within  the  professional 
education  unit,  the  faculty  and  staff  were  comfortable  with  the  researcher  and  were  willing  to  offer  personal 
thoughts  and  beliefs  concerning  the  design  and  development  of  the  Web  site,  with  little  to  no  concern  as  to  the 
trustworthiness  of  the  researcher.  The  integration  within  the  professional  education  unit  was  a helpful  part  of  the 
design  and  developmental  process,  and  offered  numerous  instances  of  thoughtful  reflection  and  discussions  with  the 
faculty  and  staff  of  the  professional  education  unit;  without  such  an  integrated  aspect,  the  researcher  would  have 
most  likely  missed  instances  of  clarity  and  breakthrough  discussions  pertaining  to  the  Web  site. 

Interview 

The  Web  site  was  continuously  appraised  through  out  the  design  and  development  periods.  There  were  two 
groups  of  appraisers;  the  first  group  being  technical  and  interface  usage  experts,  with  the  other  group  being  Texas 
members  of  the  NCATE  BOE  organization.  The  reasoning  behind  the  use  of  the  NCATE  BOE  members  from  Texas 
was  the  desire  to  eliminate  any  chance  that  the  NCATE  BOE  members  chosen  for  the  professional  education  unit’s 
BOE  visiting  team  could  possibly  have  been  a member  of  the  Web  site  design  and  development  research;  in  other 
words,  NCATE  BOE  members  from  Texas  would  be  ineligible  to  act  as  NCATE  BOE  visiting  team  members  for 
the  professional  education  unit  that  was  designing  and  developing  the  Web  site,  because  no  BOE  member  may 
examine  a university  within  their  own  state.  This  structure  is  in  place  to  ensure  any  form  of  in-state  bias  would  be 
extricated.  The  willingness  of  the  Texas  NCATE  BOE  members  to  act  as  appraisers  through  out  the  design  and 
developmental  periods  is  a strong  indication  that  the  collegiality  and  desire  towards  excellence  within  the  education 
profession  ensures  the  quality  of  our  education  profession’s  future. 

Development 

The  development  of  the  Web  site  consisted  of  three  main  elements:  the  developmental  process,  the 
integration  of  documentation,  and  quality  assurance.  Within  each  of  these  areas,  the  desire  for  appraiser  feedback  to 
aid  in  the  developmental  process  was  clear. 

Developmental  Process 

The  process  through  which  the  Web  site  development  progressed  was  steady  and  streamlined.  Through  out 
regenerative  periods  of  development,  appraisal,  reflection,  and  significant  redesign,  the  Web  site  focused  upon 
quality  assurance  and  the  integration  of  documentation.  The  appraisers  offered  reflective,  significant  input  through 
the  interview  process  that  aided  in  the  development  of  the  Web  site. 

Integration  of  Documentation 

The  steady  integration  of  documentation  through  out  the  developmental  process  offered  numerous 
instances  of  nonlinear  integration  of  documentation  as  well  as  the  hyperlinking  of  specific  subject  areas  within  and 
between  the  documentation.  The  hyperlinking  process  between  the  specific  subject  areas  within  the  documentation 
was  perhaps  the  most  time-consuming  aspect  of  the  integration  process,  outside  of  the  location,  obtainment  and 
digitization  of  the  documentation. 

Quality  Assurance 

The  quality  assurance  was  an  element  that  was  felt  through  out  the  design  and  development  of  the  Web  site. 
Without  the  quality  aspect  as  an  integral  part  of  the  Web  site,  the  researcher  felt  that  the  design  and  development 
was  a useless  exercise  in  futility.  Therefore,  the  quality  assurance  was  felt  through  the  design,  development. 
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appraisal,  redesign  and  redevelopment  Quality  assurance  can  not  be  overemphasized  as  a primary  aspect  through 
out  the  design  and  development  of  the  professional  education  unit  NCATE  Web  site  to  house  pertinent 
documentation. 

Results 

To  address  the  previously  stated  purpose  of  the  study,  the  methodology  chosen  was  of  a qualitative  nature, 
specifically  ethnography.  Within  the  ethnographic  areas,  a triangulation  between  historical  research,  participant 
observation,  and  interview  was  implemented.  A further  consideration  within  the  research  was  the  limitations  of  the 
study,  which  are  integral  to  the  complete  understanding  and  significance  of  the  research. 

Historical  Research 

The  historical  research  was  an  integral  aspect  within  this  developmental  research.  The  documentation 
offered  the  ability  to  not  only  integrate  and  delineate  numerous  aspects  within  the  professional  education  unit,  but 
also  to  develop  a nonlinear  hypertext  environment  through  which  the  documentation  would  offer  the  ability  to 
display  further  understanding  of  the  knowledge  available.  Although  the  location,  obtainment,  development, 
integration,  and  digitization  of  the  historical  documentation  was  of  primary  concern  through  out  the  design  and 
developmental  periods,  the  careful  development  and  thoughtful  appraisals  were  essential  to  the  quality  assurance  of 
the  Web  site. 

Participant  Observation 

The  use  of  participant  observation  through  out  the  design  and  development  of  the  Web  site  offered  an 
unstructured  viewpoint  through  which  to  obtain  thoughtful  responses  and  viewpoints  without  the  strict  question- 
answer  protocol  that  would  underlie  the  interview  process.  The  ability  of  the  researcher  to  act  as  an  “insider”  within 
the  professional  education  unit  allowed  both  faculty  and  staff  the  comfort  zone  through  which  to  impart  important 
comments  concerning  the  design  and  development  of  the  Web  site,  as  well  as  to  verbalize  any  concerns  or  questions 
that  arose. 

Interview 

The  interview  process  consisted  of  structured  question-answer  sessions  that  were  made  available  through  a 
Web-based  form  that  would  ensure  the  appraiser  a comfortable  environment  to  offer  input,  reflect,  and  further 
comment  upon  requested  information  as  well  as  outside  areas  that  were  important  to  the  appraiser.  Instances  of 
outside  e-mails  also  occurred  through  out  the  appraisal  process;  although  without  any  form  of  incentive  on  the  part 
of  the  researcher.  The  appraisers  seemed  to  require  an  outside,  personalized  mode  of  connection  with  the  researcher 
that  perhaps  fulfilled  a deep-seated  desire  to  connect  on  a more  personal  note,  as  well  as  offer  further  insightful 
information  to  the  researcher.  Through  such  interviews,  whether  through  the  formalized  question-answer  Web  forms 
or  more  informal  e-mail  communications,  the  quality  assurance  of  the  Web  site  was  ensured. 

Limitations  of  the  Study 

The  limitations  of  the  study  were  few;  however,  each  limitation  was  a significance  consideration  through 
out  the  study.  The  first  limitation  was  the  appraisers  might  have  answered  the  stated  questions  so  as  to  please  the 
researcher.  This  is  a possibility  whenever  human  subjects  are  used  in  research  situations;  however,  a considerable 
effort  was  put  forth  through  the  deliberate  working  of  the  Web  site  feedback  forms  so  as  to  not  influence  the 
appraisers.  A second  limitation  as  that  the  appraisers  would  interpret  the  stated  questions  differently  than  the 
intended  meaning.  Due  to  this  possibility,  the  researcher  gave  each  question  careful  consideration  so  as  to  reduce  the 
interference  of  questionable  grammar  or  verbiage.  A third  limitation  is  the  danger  of  generalizability  of  the  study. 
This  study  can  not  be  generalized  beyond  the  professional  education  unit  for  which  the  Web  site  was  developed. 
However,  certain  aspects  of  the  design  and  developmental  process  may  be  useful  for  future  endeavors  and  are 
offered  as  a modest  component  towards  future  efforts. 
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Conclusions 


The  occurrences  through  out  the  design  and  developmental  periods  of  the  Web  site  offer  numerous 
instances  of  thoughtful  reflection,  learning,  and  growth.  Such  experiences  must  be  conscientiously  considered  so 
that  future  endeavors  will  gain  from  such  experiences.  Although  the  observations  and  experiences  attained  through 
this  research  are  considerable,  future  endeavors  will  offer  significant  possibilities  towards  the  establishment  of 
further  defined  and  delineated  facets  of  learning. 
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Abstract:  The  use  of  the  World  Wide  Web  as  a tool  to  disperse  information  for  national 
accreditation  has  been  severely  underutilized.  The  ability  to  offer  informational  documentation 
concerning  the  health  of  an  institution,  such  as  a university,  in  a non-linear  hypertext  format  is  one 
which  could  offer  a healthy,  severely  needed  element  towards  the  accreditation  process.  The  ability 
to  ease  the  accreditation  process  towards  a more  flowing  sharing  of  information  has  been  a long 
time  coming,  but  one  which  is  on  the  brink  of  exploding.  The  National  Council  for  the 
Accreditation  of  Teacher  Education  (NCATE)  is  one  national  accrediting  agency  that  has  realized 
the  Web’s  vital  use  and  has  begun  the  process  of  implementing  this  underutilized  tool. 


Introduction 

The  use  of  the  World  Wide  Web  (Web)  towards  the  development  and  integration  of  information  for  a 
national  accreditation  structure  is  in  its  infancy.  However,  the  utilization  of  the  Web  as  a tool  to  disperse  information 
has  presented  itself  as  a primary  area  of  interest  to  one  national  accrediting  body,  the  National  Council  for  the 
Accreditation  of  Teacher  Education  (NCATE).  NCATE  is  an  accrediting  institution  that  has  stated  their  desire  to 
pull  technology  into  their  accreditation  process.  NCATE  developed  a Technology  Task  Force  (National  Council  for 
Accreditation  of  Teacher  Education,  1997)  that  stated  the  desire  as  well  as  the  need  for  the  ability  to  integrate 
technology  into  the  accreditation  process.  One  such  request  was  the  desire  of  piloting  the  development  of  a Web 
site  through  which  the  integral  documentation  pertaining  to  the  accreditation  process  that  would  be  available.  This 
would  allow  for  the  accreditation  Board  of  Examiners  (BOE)  to  view  the  documentation  before  stepping  foot  on  the 
campus,  as  well  as  the  ability  to  link  documentation  in  a hypertext  format  within  the  Web  site. 

The  importance  of  human/computer  interface  issues  through  out  the  Web  site  developmental  process  can 
not  be  underemphasized.  Although  the  main  focus  of  the  developmental  process  was  the  placement  of 
documentation  on  the  Web  and  the  interrelation  of  the  documents  within  the  site,  the  appraisers  for  the  Web  site 
focused  upon  the  human/computer  interface  issues  to  a large  extent.  Although  the  appraisers  were  asked  to 
comment  upon  structural  strengths  and  weaknesses,  the  appraisers  consistently  focused  upon  the  interface  issues  as 
being  of  primary  importance.  The  issues,  concerns,  and  comments  concerning  the  human/computer  interface  are 
integral  to  the  development  of  a national  accreditation  Web  site  to  house  pertinent  documentation. 

Human/Computer  Interface 


The  World  Wide  Web  (Web)  is  a medium  with  the  capacity  to  display  nonlinear  structures  in  a hypertext 
environment,  which  in  turn  leads  to  a further  understanding  of  ill-structured  knowledge  domains  on  the  part  of  the 
user  (Wittgenstein,  1953).  These  ill-structured,  nonlinear,  hypertext  environments  offer  a cognitive  flexibility  to  the 
documentation,  therefore  the  knowledge,  that  is  obtained  by  the  user  (Spiro  & Jehng,  1990).  Although  the 
constructivist  theoretical  structure  embraces  such  an  environment,  certain  aspects  must  still  be  thoroughly, 
reflectively  considered  through  out  the  development  of  a learning  environment.  Due  to  the  direct  parallel  between 
the  learning  environment  theories  and  the  NCATE  accreditation  Web  site  through  which  pertinent  professional 
education  unit  documentation  is  made  available  to  the  NCATE  BOE  team,  careful  consideration  of  the 
human/computer  interface  issues  must  be  a focus  within  the  design  and  developmental  process.  Reflections 
pertaining  to  human/computer  interface  issues,  concerns,  and  comments  must  be  thoroughly  considered  so  that  the 
future  endeavors  will  have  a basis  upon  which  to  build. 
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Issues  and  Concerns 


Several  noteworthy  human/computer  interface  issues  and  concerns  were  raised  through  out  the  design  and 
development  of  the  professional  education  unit’s  NCATE  continuing  accreditation  Web  site.  Only  the  thoughtful, 
reflective  consideration  of  each  issue  and  concern  offered  useful,  creative  ideas  that  could  be  implemented  in  real- 
world  environments  that  could,  in  turn,  address  the  discussions  surrounding  each  subject. 

Instructional  Design  Model 

The  first  of  these  issues  was  the  questionable  use  of  the  Four  D Model  (Thiagarajan,  Semmel  & Semmel, 
1974),  a linear  instructional  design  model,  to  develop  an  ill-structured,  nonlinear,  hypertext  Web  environment. 
Concerns  arose  surrounding  the  use  of  this  model  because,  as  the  argument  was  delineated,  the  linear  instructional 
design  model  may  impact  the  ill-structured,  nonlinear,  hypertext  Web  site  that  was  to  be  developed.  However,  after 
further  discussion  and  theoretical  underpinnings  were  discussed,  the  realization  that  a linear  instructional  design 
model  does  not  necessarily  lead  to  a linear  product  was  agreed  upon.  A linear  instructional  design  model,  as  the  Four 
D Model  clearly  is,  can  indeed  offer  a product  the  defined  requisite  step  procedures  outlined  in  the  Four  D Model  to 
attain  a nonlinear  Web  environment.  The  basic  parallel  understanding  between  the  two,  descriptively  the  model  and 
the  product,  is  that  the  model  leads  the  progress  through  out  the  design  and  development  of  the  product;  however, 
the  product  demands  to  be  developed  while  focused  upon  its  own  specific  goals  and  objectives.  Therefore,  the 
importance  of  a tight  parallel  between  the  instructional  design  model  and  the  product  goals  and  objectives  is 
obvious;  yet  the  underlying  theoretical  structure  of  the  instructional  design  model  does  not  drive  the  underlying 
theoretical  structure  of  the  product.  The  theoretical  underpinnings  do  not  necessarily  have  to  be  the  same  to  produce 
a strong  product. 

Intellectual  Property  and  Ownership 

A second  issue  was  the  placement  of  faculty  vitae  and  syllabi  on  the  Web  site.  At  the  beginning  of  the 
developmental  process,  the  desire  to  offer  the  professional  education  unit  documentation  and  information  as  a 
freedom  of  knowledge  endeavor  drove  the  location  and  obtainment  of  historical  information.  However,  as  the 
developmental  process  became  more  engrossing,  the  faculty  raised  questions  surrounding  the  idea  of  placing  course 
syllabi  on  the  Web  for  anyone  to  view.  This  concern  arose  due  to  the  knowledge,  history,  research  and  reflection 
that  was  apparent  through  out  the  course  syllabi;  many  faculty  had  built  their  prestigious  careers  upon  the  subject 
matter  contained  within  the  course  syllabi  and  the  faculty  were  not  inclined  to  place  these  course  syllabi  on  the  Web 
for  anyone  to  view,  borrow,  or  use  towards  someone  else’s  own  personal  end.  Although  the  faculty  were  eager  to 
offer  their  aid  to  anyone  that  may  ask  for  information  and  would  most  likely  offer  their  course  syllabi  upon  request, 
the  perceived  lack  of  control,  when  placing  information  on  the  Web,  was  a concern.  The  control  that  was  apparent 
through  the  personal  contact  between  faculty  member  and  the  person  requesting  the  aid  was  a workable  situation; 
however,  when  the  information  was  placed  on  the  Web,  it  became  a question  of  intellectual  property  and  ownership. 
Anyone  could  take  this  information  and  do  whatever  they  wanted  to  do  with  it.  This  was  the  concern  of  the  faculty. 

After  numerous  discussions  and  the  support  of  the  faculty,  the  decision  was  agreed  upon  to  place  password 
protection  upon  the  NCATE  Web  site  that  would  house  the  course  syllabi,  as  well  as  any  other  documentation  that 
the  faculty  agreed  to  include  for  the  Web  site.  The  question  of  intellectual  property  and  ownership  pertaining  to  the 
placement  of  course  syllabi  on  the  Web  site  offered  faculty  the  opportunity  to  clarify  the  gray  area  of  intellectual 
property  and  ownership  within  the  professional  education  unit  environment  and,  on  a larger  scale,  the  university 
environment. 

Copyright 

Within  the  same  thought  pattern  as  the  intellectual  property  and  ownership,  the  question  was  raised 
concerning  copyright  issues  and  concerns.  The  question  of  copyright,  both  of  the  historical  documentation  placed  on 
the  Web  site  and  the  Web  site  itself,  was  discussed  through  out  the  developmental  process.  The  university 
administration  is  still  working  on  the  final  documents  pertaining  to  the  copyright  legalities;  however,  the  concerns 
over  copyright  issues  pertaining  to  this  Web  site  are  quite  clear.  If  the  documentation  was  developed  for  any  part  of 
the  university  environment  during  the  employment  of  faculty,  usually  described  as  research,  service  and  teaching, 
then  it  is  the  property  of  the  university  and  the  university  will  receive  at  least  a percentage  of  the  proceeds. 
However,  anything  is  worthy  of  discussion  and  copyright  may  be  one  of  these  discussion  topics.  Therefore,  the 
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question  of  copyright  when  discussion  the  historical  documentation  that  was  placed  on  the  Web  site  was  the  sole 
copyright  of  the  university.  Also,  the  Web  site  itself  was  the  sole  property  of  the  university  and  copyright  went  to 
the  university,  although  the  theoretical  underpinnings  and  learned  lessons  were  the  knowledge  obtained  by  the 
researcher. 

Template 

The  template  structure  of  the  Web  pages  within  the  Web  site  was  areas  of  discussion  through  out  the 
developmental  process.  The  strong  feelings  of  the  appraisers  pertaining  to  the  template  structure  that  the  Web  pages 
would  follow  were  consistently  questioned,  discussed,  commented  upon,  and  complimented.  The  Web  page 
templates  consisted  of  a title  bar  along  the  top  of  the  page,  with  higher-level  navigation  buttons  directly  below  the 
title  bar,  listing  left  to  right.  Along  the  left  side  of  the  page  were  same-level  navigation  buttons  that  offered  the 
ability  to  quickly  navigate  through  out  other  relevant  Web  pages  without  the  difficulty  of  moving  within  the  Web 
site,  losing  the  place  within  the  Web  site  navigation  structure,  and  forgetting  the  information  of  importance  during 
the  navigation  process.  However,  one  significant  template  structure  addition  that  was  offered  by  the  appraisers  was 
the  desire  to  have  an  area  along  the  right-hand  side  of  the  screen  that  would  offer  textual  links  to  areas  of  related 
importance  within  the  document  that  the  user  was  viewing.  This  aspect  quickly  led  to  an  innovative  aspect  within 
the  Web  pages  that  offered  easier,  quicker  nonlinear,  hypertext  availability  within  the  Web  site.  Although  this  was 
an  obvious  addition  to  the  Web  page  templates,  the  clarity  of  thought  on  the  part  of  the  appraisers  led  to  this 
innovative  use  of  the  nonlinear,  hypertext  environment. 

Comments 

Several  noteworthy  human/computer  interface  comments  were  raised  through  out  the  design  and 
development  of  the  professional  education  unif  s NCATE  continuing  accreditation  Web  site.  Only  the  thoughtful, 
reflective  consideration  of  each  comment  would  aid  in  the  development  of  the  professional  education  unit’s  NCATE 
continuing  accreditation  Web  site. 

Interface  Format 

The  interface  of  the  Web  site  was  the  first  consideration  within  the  initial  design  and  developmental 
process.  The  desire  of  the  developer  to  integrate  a useful,  smooth  interface  into  the  Web  site  that  would  seamlessly 
aid  the  user  towards  the  documentation  of  importance  was  an  issue  that  was  constantly  in  the  developer’s  thoughts. 
Interviews  were  conducted  with  the  appraisers  requesting  feedback  upon  the  template  color(s),  template  text  size, 
and  overall  template  aesthetic  appeal.  Specific  questions  were  asked  concerning  each  templates: 

■ Attractiveness 

■ Unattractiveness 

■ Obvious  and  Easily  Viewable/Readable  Links 

■ Positive  Aspects 

■ Negative  Aspects 

■ Distracting  Aspects 

■ Annoying  Aspects 

■ Personal  Thoughts  Concerning  the  Template 

■ Which  template(s)  were  preferable  for  the  Web  site 

Through  these  overarching  questions,  numerous  insights  were  gained  from  the  appraiser  responses.  The  appraisers, 
as  an  overall  desire,  found  the  simplistic,  straightforward  templates  to  be  most  attractive;  the  reasoning  behind  this 
was  that  the  appraisers  felt  that  the  “busier”,  less  simplistic  templates  would  become  more  of  an  eyesore  as  the  user 
traversed  through  out  the  Web  site.  Although  the  less  simplistic  templates  may  have  been  attractive  for  a one  page 
template,  the  consistent  use  of  the  template  would  become  difficult  for  the  user;  as  one  appraiser  noted,  the  template 
should  be  an  appealing  backdrop  for  the  documentation,  not  the  other  way  around. 

Web  Browser  Inconsistencies 
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A comment  noted  most  loudly  through  out  the  developmental  and  appraisal  processes  was  the 
inconsistencies  between  the  Web  browsers  Netscape  Navigator  or  Netscape  Communicator  (Netscape,  1997; 
Netscape,  1998)  and  Microsoft  Internet  Explorer  (Microsoft,  1997).  Although  the  Web  site  was  developed  for 
consistency  in  viewing  across  platforms  and  browser  types,  the  difficulties  that  became  apparent  when  developing 
the  Web  site  made  the  simplistic  aspects  of  a Web  site  more  intriguing  and  desirable.  The  Web  site  became  a 
streamlined  structure  with  very  few  outside  inconsistencies  not  taken  into  consideration.  However,  although  all 
inconsistencies  between  Web  browsers  were  thoroughly  discussed  and  reflected  upon,  instances  of  inconsistencies 
will  always  arise  when  developing  a Web  site;  the  structure  must  be  flexible  enough  to  work  within  these 
inconsistencies  and  aid  the  user  in  the  user’s  quest  for  information. 

Conclusions 

The  importance  of  human/computer  interface  issues  through  out  the  Web  site  developmental  process  can 
not  be  underemphasized.  Although  the  main  focus  of  the  developmental  process  was  the  placement  of 
documentation  on  the  Web  and  the  interrelation  of  the  documents  within  the  site,  the  appraisers  for  the  Web  site 
focused  upon  the  human/computer  interface  issues  to  a large  extent.  Although  the  appraisers  were  asked  to 
comment  upon  structural  strengths  and  weaknesses,  the  appraisers  consistently  focused  upon  the  interface  issues  as 
being  of  primary  importance.  The  issues,  concerns,  and  comments  concerning  the  human/computer  interface  are 
integral  to  the  development  of  a national  accreditation  Web  site  to  house  pertinent  documentation  and  must  become 
central  to  the  development  of  the  Web  site.  Through  trial  and  error  through  out  the  design  and  development  process, 
the  thoughtful,  reflective  nature  of  the  appraiser  feedback  offered  numerous  instances  of  insight  and  developmental 
aids. 
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Abstract:  The  use  of  technology  within  the  PreK-12  classroom  has  been  steadily  increasing  within 
the  previous  ten-year  period.  However,  teacher  training  has  not  kept  a similar  pace.  The  training 
available  for  the  teachers,  as  well  as  the  administrative-level  school  district  employees,  has  lagged 
desperately  behind  the  technological  hardware  integration.  For  this  reason,  school  districts  are 
looking  towards  training  the  personnel  in  the  district.  Universities  are  continuously  developing 
such  training  for  their  students  at  the  undergraduate  level  as  well  as  graduate  level,  so  the 
universities  are  the  logical  answer  to  the  school  district  needs.  Within  southeastern  metropolitan, 
semi-rural  and  rural  areas  within  the  State  of  Texas,  several  districts  and  universities  have 
developed  partnerships  through  which  the  training  of  school  district  teachers  and  administrators  can 
occur. 


Introduction 

Training  school  district  Instructional  Technology  Coordinators  in  the  use,  development  and  implementation 
within  instruction  of  multimedia  is  an  area  that  is  only  recently  being  addressed.  The  school  districts  have  received 
extensive  amounts  of  money  for  networking  and  hardware  integration  within  the  PreK-12  classrooms;  however,  the 
training  that  must  be  implemented  has  lagged  behind  the  networking  and  hardware.  The  school  districts  have  begun 
looking  towards  outside  entities  through  which  the  training  of  the  school  district  personnel  can  occur,  whether 
through  business  or  educational  arenas.  Within  the  southeastern  metropolitan,  semi-rural  and  rural  areas  within  the 
State  of  Texas,  several  school  districts  have  turned  to  the  surrounding  universities  to  develop  partnerships.  The 
universities  have  answered  the  school  districts  with  an  energetic  response  and  are  eager  to  offer  instructional 
technology  training  to  the  school  district  personnel. 

The  universities  are  developing  a pyramidal,  top-do\vn  structure  through  which  the  persons  trained  will  in 
turn  train  others,  until  each  of  the  district  personnel  are  trained.  However,  this  training  begins  at  the  university  level. 
The  desperate  need  of  the  school  districts  are  being  met  through  the  thoughtful  integration  of  not  only  technological 
training,  but  also  the  modeling  required  to  use  technology  as  an  integrated  tool  within  the  classroom  environment. 
This  modeling  begins  with  training  the  independent  school  district  Instructional  Technology  Coordinators.  The 
Instructional  Technology  Coordinators  are  persons  that  are  responsible  for  training  the  other  district  personnel  on  the 
use  of  computer  software  and  hardware  within  the  district.  One  of  the  primary  areas  in  which  the  district  personnel 
are  to  be  trained  is  the  development,  instruction,  and  use  of  multimedia. 

Training  Modules 

The  first  consideration  when  training  the  school  district  personnel  is  deciding  upon  the  information  that 
needs  to  be  addressed.  A needs  assessment  clearly  is  indicated  at  this  stage  of  the  process;  only  through  a needs 
assessment  will  the  desired,  needed,  or  required  subject  matter  be  indicated  and  offer  the  ability  to  be  addressed. 
Multimedia  is  one  area  that  was  indicated  as  being  not  only  desired,  but  also  a subject  area  that  would  be  necessary 
for  future  endeavors.  Therefore,  multimedia  was  an  area  that  would  be  necessary  to  develop  a training  module. 

Training  modules  are  merely  clearly  articulated  curriculum  that  offers  training,  information  dispersal, 
hands-on  activities,  and  packets  of  information  for  the  trainer  and  the  learner.  Such  information  is  necessary  to 
develop,  unless  a quality  training  module  is  available.  For  the  needs  of  this  situation,  a training  module  was 
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developed  by  the  university  faculty  and  students  to  aid  the  school  district  personnel  in  the  attainment  of  the 
necessary  information. 

Training  the  Trainers 

Once  the  needs  assessment  has  occurred,  with  results  leading  to  the  development  of  a multimedia  training 
module,  the  next  area  of  concern  would  be  the  question  of  who  would  train  the  learners.  The  university  faculty 
would  logistically  be  unable  to  facilitate  each  training  session,  so  the  question  follows  as  to  exactly  who  would 
facilitate  the  trainings.  The  logical  answer  actually  consists  of  two  responses:  school  district  instructional  technology 
coordinators  and  university  graduate  students. 

School  District  Instructional  Technology  Coordinators 

The  school  district  Instructional  Technology  Coordinators  have  perfect  logistical  positions  through  which 
to  train  the  school  district  personnel.  Therefore,  the  Instructional  Technology  Coordinators  would  be  the  primary 
focus  of  the  university  faculty.  Once  the  Instructional  Technology  Coordinators  are  trained  upon  the  integration  of 
multimedia,  the  information  dispersal  would  again  focus  upon  multimedia,  but  upon  the  training  towards  the 
facilitation  and  integration  of  multimedia;  in  this  way  the  trainer  has  the  experience  of  being  on  the  receiving  end  of 
the  instruction  as  well  as  on  the  facilitating  end.  This  two-sided  experience  allows  the  facilitator  to  not  only 
empathize  with  the  learner,  but  to  also  have  the  experience  necessary  to  foresee  possible  problems  or  questions  that 
may  arise  during  the  training  sessions.  However,  the  school  district  Instructional  Technology  Coordinators  have 
positions  which  pull  them  in  numerous  different  directions  all  at  the  same  time;  therefore,  it  may  be  impossible  to 
solely  train  the  Instructional  Technology  Coordinators  as  trainers  due  to  the  possible  inability  of  the  Instructional 
Technology  Coordinators  to  act  as  sole  trainers  for  the  school  district  personnel.  With  the  majority  of  Instruchonal 
Technology  Coordinators  having  numerous  duties,  as  previously  stated,  one  simplistic  answer  would  be  to  train  the 
university  graduate  students  as  trainers. 

University  Graduate  Students 

Training  university  graduate  students  as  trainers  for  school  district  personnel  offers  a multitude  of 
experiences  for  the  university  graduate  student.  Not  only  will  the  graduate  students  receive  real-world  experience 
training  adults  in  educational  settings  and  the  possible  monetary  compensation  that  would  be  available;  but,  the 
graduate  students  would  have  the  ability  to  further  develop  one  or  more  training  modules  that  would  be  beneficial, 
further  their  knowledge  of  learning  environments  and  curricular  events,  and  develop  mentor/mentee  relationships 
with  university  faculty  as  well  as  school  district  Instructional  Technolo^  Coordinators.  The  graduate  students  could 
work  parallel  to  the  school  district  Instructional  Technology  Coordinators,  contract  their  services  through  the 
university  to  the  school  districts,  or  numerous  other  combinations  which  would  offer  the  experience  that  graduate 
students  may  desire  to  obtain  real-world  experience  and  develop  surrounding  school  district  contacts  that  may  aid 
them  in  future  ventures.  The  possibilities  for  the  university  graduate  students  to  work  with  university  faculty  and 
district  personnel  towards  the  development  of  multimedia  offer  a well-rounded  experience  to  university  graduate 
students. 

Technology  Integration 

The  development,  instruction  and  use  of  multimedia  in  a classroom  environment  open  the  world  to 
numerous  instances  of  technological  integration  into  the  educational  curriculum.  Through  this  integration  of 
multimedia,  the  curriculum  can  be  largely  enhanced  and  the  learners  will  have  the  ability  to  develop  further 
knowledge  and  experiences  that  would  have  previously  been  unimaginable.  For  the  purposes  of  the  training  of 
school  district  Instructional  Technology  Coordinators  in  multimedia  development,  instruction  and  use,  three  focuses 
of  technology  integration  will  be  considered:  teacher-centered  technology,  student-centered  technology,  portfolio 
assessment. 

Teacher-Centered  Technology 


467 


The  development  of  training  modules  to  focus  upon  the  teacher-centered  aspect  of  technology  integration 
into  the  curriculum  and,  therefore,  into  the  educational  environment,  offers  numerous  opportunities  for  multimedia 
to  be  integrated.  Multimedia  can  be  used  as  a linear  or  nonlinear  lecture  device  for  the  facilitator,  which  would 
enhance  the  lecture  through  the  integration  of  pictures,  videos,  sound  files,  animation,  and  other  such  facets  ot 
multimedia.  The  ability  to  integrate  such  enhancements  into  a lecture-style  situation  offers  the  ability  to  maintain  the 
motivation  of  the  learners,  emphasize  numerous  integral  aspects  of  subject  matter,  or  merely  display  an  innovative 

curricular  entity.  ^ ^ ^ i 

However  the  teacher-centered  multimedia  technology  must  not  merely  be  used  for  lecture-style  puqioses. 

The  ability  of  the  facilitator  to  develop  multimedia  products  that  would  enhance  prior  knowledge,  develop 
conceptual  frameworks  for  the  learners,  act  as  knowledge-checking  experiences,  or  an  even  more  advanced 
multimedia  package  can  offer  the  ability  to  develop  teacher-centered  multimedia  drills  that  the  learners  would  work 
through  as  a class.  The  primary  aspect  of  the  teacher-centered  technology  is  the  innovative  uses  of  multimedia 
within  the  experience.  Numerous  facilitators  may  already  understand  the  fundamentals  of  a multimedia  package; 
however,  the  true  skill  is  displayed  when  the  facilitators  display  their  knowledge  towards  developing  a teacher- 
centered  or  a student-centered  multimedia  package. 

Student-Centered  Technology 

Student-centered  technology  can  be  further  defined  as  instructor-created  or  student-centered  technology. 
Within  the  area  of  multimedia,  the  ability  of  both  the  facilitator  and  the  learner  to  equally  develop  multimedia 
programs  for  distinct  puqioses  is  an  integral  part  of  the  curricular  and  instructional  situation. 

Instructor-Created  Multimedia 

The  development  of  multimedia  for  the  learner  is  an  experience  that  opens  a multitude  of  environments  to 
the  learner  The  facilitator  may  create  mere  drill  and  practice  environments  through  the  higher-leaming  situations 
that  would  request  critical  analysis  on  the  part  of  the  learner:  Although  the  facilitator  must  carefully  decide  upon  the- 
curricular  integration  and  introductory  environment  through  which  to  engage  the  instructor-created  multimedia,  the 

possibilities  are  endless.  .u  i 

The  facilitator  must  have  the  capability  to  develop  multimedia  products  that  will  facilitate  the  learning 

process  while  engaging  the  learner  until  the  requisite  goals  and  objectives  are  obtained.  This  is  no  easy  feat,  and  one 
that  must  be  considerably  touched  upon  through  out  the  training  of  the  school  district  personnel. 


Student-Created  Multimedia 

The  student-created  multimedia  is  another  aspect,  altogether.  The  ability  to  create  educational 
environments  through  which  learners  can  create  multimedia  products  that  have  educational  relevance  is  a carefully 
learned  ability.  The  training  of  district  personnel  to  have  the  foresight,  comprehend  the  capabilities  of  the  student- 
created  multimedia  products,  and  open  the  curricular  and  learning  environment  to  this  option  is  one  which  must  be 
facilitated  in  a forthright  manner.  The  ability  of  learners  to  create  their  own  multimedia  package  of  their  learning 
opens  the  learning  environment  to  one  of  control  on  the  part  of  the  student,  as  well  as  creates  a collegial  attitude  on 
the  part  of  the  student.  This  type  of  environment  is  one  that  closely  emulates  the  business,  professional  environment 
of  today  and  one  that  must  be  carefully  integrated  into  the  learning  environment  for  our  students  to  emulate.  Of 
interest  is  the  close  parallel  between  student-created  multimedia  and  portfolio  assessment  within  the  educational 
environment. 

Portfolio  Assessment 

The  development  of  student-created  multimedia  products,  and  the  integration  of  these  products  within  a 
larger  structure,  emulates  a digital  portfolio.  The  creative,  developmental  work  that  is  brought  to  a portfolio  entails 
numerous  hours  of  difficult  thought  patterns  and  critical  analysis  of  numerous  high-level  situations.  The  difficulty 
within  a digital  portfolio,  which  is  what  is  being  addressed,  is  the  assessment  factor.  Once  the  learning  occurs  and 
the  multimedia  product  is  developed  and  integrated  into  a portfolio  structure,  how  will  the  portfolio  be  assessed? 

As  with  numerous  paper  portfolio  assessment  structures,  A reflective  piece  is  included  by  the  learner  for 
each  product  that  is  included  within  the  portfolio.  Through  this  reflective  piece,  the  portfolio  assessor  has  the  ability 
to  attain  a working  understanding  of  the  learner’s  thought  process  through  out  the  developmental  penod  and  to 
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assess  the  learning  which  took  place.  On  a larger  level,  a rubric  or  a matrix  is  viable  options  through  which  to  assess 
a portfolio.  Through  a matrix  or  a rubric,  clear  delineations  are  presented  to  the  portfolio  developer  and  assessor  as 
to  the  criteria  through  which  to  obtain  the  desired  grade  within  an  organized  grade  structure. 

Although  there  is  a lot  of  information  pertaining  to  portfolios  and  portfolio  assessments  available,  the 
integration  of  technology  into  the  portfolio  environment  should  be  openly  addressed  within  the  multimedia 
development,  instruction  and  use  training  sessions  for  school  district  Instructional  Technology  Coordinators. 


Conclusions 

Training  school  district  Instructional  Technology  Coordinators  in  the  use,  development  and  implementation 
within  instruction  of  multimedia  is  an  area  that  is  only  recently  being  addressed.  The  desperate  need  of  the  school 
districts  are  being  met  through  the  thoughtful  integration  of  not  only  technological  training,  but  also  the  modeling 
required  to  use  technology  as  an  integrated  tool  within  the  classroom  environment.  This  modeling  begins  with 
training  the  independent  school  district  Instructional  Technology  Coordinators.  One  of  the  primary  areas  in  which 
the  district  personnel  are  to  be  trained  is  the  development,  instruction,  and  use  of  multimedia.  Through  the 
development  of  training  modules,  training  the  trainers,  and  technology  integration,  the  introduction  of  multimedia 
into  the  educational  environment  is  becoming  a reality. 
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Abstract:  In  recent  years,  much  has  been  written  on  the  philosophy  and  theory  of 

constructivism.  Only  recently,  however,  has  it  become  feasible  to  consider  constructivist 
principles  within  the  context  of  adult  distance  education.  The  participant  interaction  that 
is  central  to  most  constructivist  theory  has  become  possible  primarily  because  of 
advances  in  communications  technologies  that  have  become  an  integral  component  in  an 
increasing  number  of  distance  education  environments.  There  remain  a number  of 
barriers  to  full  implementation  of  constructivist  principles  within  the  context  of  higher 
education.  This  paper  will  address  some  of  these  barriers  and  describe  how  instructors  in 
a technology-mediated  distance-delivered  undergraduate  program  designed  a learning 
environment  based  on  constructivist  principles. 


Introduction 

Academic  discourse  relating  to  the  value  of  collaborative  learning  appears  to  be  based  on  the  distinction 
between  the  individual  and  social  nature  of  human  cognition.  The  former  position  reflects  the  work  of  Piaget 
(Wadsworth,  1971)  where  the  focus  is  on  individual  cognition.  In  this  view,  the  function  of  the  group  is  to  provide 
social  pressure  to  solve  problems.  This  pressure  stimulates  the  individual  to  resolve  internal  conflicts  which,  in  turn, 
leads  to  cognitive  growth.  The  latter  position  is  based  on  the  Vygotskian  model,  wherein  the  group  plays  an  integral 
rather  than  peripheral  role  in  cognitive  growth.  Knowledge  is  created  through  social  interaction,  and  then,  primarily 
through  language  (Frawley,  1997).  Young  (1997)  illustrates  the  full  continuum  of  epistemological  stances  relating  to 
how  humans  create  knowledge,  the  relationship  between  the  individual  and  social  context,  and  their  relative  impact 
on  the  process  of  knowledge  creation.  He  identifies  six  schools  of  thought,  each  of  which  can  be  placed  in  one  of 
three  categories:  Category  1 in  which  social  context  and  interaction  are  irrelevant  (i.  Radical,  Material 
constructivism,  ii.  Critical,  Final  constructivism);  Category  2 in  which  the  social  environment  may  or  may  not  play  a 
role  in  the  construction  of  knowledge  (iii.  Existential  constructivism,  iv.  Efficient,  Objective  constructivism);  and 
Category  3 in  which  the  social  nature  of  knowledge  construction  is  central  (v.  Social,  Formal  constructivism,  vi.  Co- 
constructivism). Although  Young's  categories  are  useful  for  describing  the  different  epistemological  positions, 
learners  themselves  are  not  restricted  to  employing  any  one  position  in  their  construction  of  knowledge.  From  both 
a developmental  and  contextual  perspective,  learners  will  naturally  move  fluidly  from  one  knowledge  construction 
mode  to  another,  unless  something  impedes  this  movement.  In  traditional  distance  education,  the  learning 
environment  has  acted  as  such  an  impediment  because  it  could  not  support  the  movement  from  cognitively  oriented 
to  socially  oriented  knowledge  construction. 

Another  central  tenet  of  constructivism  is  an  emphasis  on  process.  Learners  should  participate  in  authentic, 
active  learning  experiences.  This  presupposes  a complex  learning  context,  which  reflects  the  ill-structured  nature  of 
real-world  problems.  In  this  learner-centred  environment,  the  teacher  functions  as  facilitator  of  learning,  rather  than 
dispenser  of  content.  Reflection  is  part  of  the  process  of  building  knowledge,  as  is  articulation,  both  internal  and 
social.  Jonassen  et  al.  (1995)  refers  to  these  attributes  as:  context,  construction,  collaboration,  and  conversation. 
Zahorik  (1995)  categorises  them  as  activating  knowledge,  acquiring  knowledge,  understanding  knowledge,  using 
knowledge,  and  reflecting  on  knowledge.  The  emphasis  accorded  to  each  of  these  facets  by  proponents  of  the 
different  constructivist  schools  of  thought  varies.  There  exist  a number  of  theories,  perspectives  and  approaches  to 
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constructivist  learning  including:  situated  cognition  (Brown,  Collins,  & Duguid,  1989;  Lave  & Wenger,  1991), 
cognitive  apprenticeship  (Duffy  and  Bednar,  1991),  and  cognitive  flexibility  (Spiro,  Feltovich,  Jacobson  & Coulson, 
1991).  Increasingly,  theorists  are  addressing  the  ways  in  which  technology  can  be  used  to  support  constructivist 
learning  environments  (Duffy  & Jonassen,  1992) 


Barriers  to  Incorporating  Constructivist  Principles 

With  so  much  importance  placed  upon  interaction  between  and  among  learners  and  instructor,  and  the 
difficulty  of  achieving  that  interaction  within  the  traditional  distance  education  environment,  it  is  not  surprising  that 
distance  educators  found  it  onerous,  if  not  impossible  to  reconcile  constructivist  principles  with  the  reality  of  their 
working  environments.  The  adoption  of  communication  technologies  by  the  distance  education  community  has 
substantively  removed  one  barrier  to  participant  interaction,  but  there  are  a number  of  other  factors  that  limit  the 
extent  to  which  instructors  apply  constructivist  principles.  While  these  are  not  exclusive  to  the  distance  education 
environment  (Carr,  Jonassen,  Litzinger  & Marra,  1998),  their  impact  may  be  felt  more  strongly  in  that  context. 

Instructional  Design 

The  prevalent  instructional  design  models  tend  to  impede  the  application  of  constructivist  principles 
(Wilson,  Teslow,  & Osman-Jouchoux,  1995).  The  linearity  of  most  instructional  design  models,  coupled  with  the 
objective-driven  development  process,  acts  as  a major  constraint,  one  that  is  more  evident  in  the  context  of  distance 
education  than  it  is  in  the  traditional  face-to-face  higher  education  context.  Most  post-secondary  instructors  develop 
their  courses  with  complete  or  near  complete  autonomy,  but  this  is  seldom  the  case  when  courses  are  developed  for 
distance  delivery  where  planning  and  development  are  frequently  a team  effort.  This  supports  instructors  whose 
experience  with  distance  delivery  is  limited,  or  provides  technical  resources,  thus  freeing  the  instructor  to  focus  on 
content.  The  number  of  proprietary  software  programs  that  have  been  developed  to  replicate  this  environment  by 
providing  ‘course  development  toolboxes’  for  use  by  instructors  who  do  not  have  access  to  instructional  design 
teams  indicates  the  prevalence  of  this  development  model.  While  taking  a template  approach  to  instructional  design 
may  ensure  efficiency  and  consistency,  it  is  seldom  conducive  to  supporting  constructivist  principles. 


Institutional  Constraints 

The  institutional  barriers  to  implementing  constructivist  learning  principles  identified  by  Carr  et  al.  (1998) 
are  essentially  the  same  for  both  face-to-face  and  distance  delivered  courses  and  programs.  Any  differences  tend  to 
be  of  degree,  with  the  distance  context  subject  to  somewhat  greater  constraints.  The  three  institutional  barriers  we 
will  discuss  are  those  of  textbook  focus,  time  constraints. 

The  textbook  focus  evident  in  the  face-to-face  classroom  is  reproduced  in  the  virtual  classroom,  but  with 
the  inclusion  of  an  additional  element — the  World  Wide  Web.  Now,  instead  of,  or  in  addition  to,  providing  printed 
packages  of  readings  and  study  guides,  these  materials  are  often  posted  on  course  web  pages.  This  reliance  on  text 
material  may  be  one  of  the  most  difficult  barriers  to  overcome,  because  is  it  is  based  on  not  only  an  institutional 
bias,  but  also  a cultural  one.  There  is  also  a possibility  that  because  interaction  within  the  context  of  computer- 
conferencing is  mediated  through  text,  i.e.,  the  written  rather  than  the  spoken  word,  such  interactions  are  regarded 
more  highly  by  the  learners  than  are  discussions  in  the  face-to-face  classroom.  Learners  routinely  print  the 
comments  of  fellow  students  for  future  reference  (Technology  in  Higher  Education  (T.H.E.)  Study  Group,  1998). 
Essentially,  they  treat  what  would  be  a conversation,  had  it  occurred  in  a face-to-face  environment,  as  if  it  were  a 
textbook.  ‘ 

Time  constraints  place  a serious  limitation  on  constructivist  learning.  It  takes  longer  to  design  an  authentic 
learning  environment.  Complex,  ill -structured  activities  where  learners  must  experiment  and  negotiate  naturally 
take  longer  to  complete  than  do  activities  designed  to  transmit  information.  The  time  factor  becomes  even  more  of 
an  issue  when  working  in  a text-based,  asynchronous  environment.  It  takes  longer  to  compose  a message  that  will 
be  subject  to  scrutiny  than  it  does  to  make  a similar  comment  in  class  where  a large  portion  of  the  message  is 
transmitted  visually  using  paralinguistic  clues  such  as  body  language  or  intonation  in  ways  that  are  denied  to  the 
writer.  In  addition,  when  conducted  asynchronously,  discussions  and  negotiations  may  be  attenuated  over  a 
considerable  length  of  time. 
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Learner  Resistance 


Adult  learners  are  very  often  self-directed  and  independent  learners.  These  characteristics  are  particularly 
evident  within  the  context  of  distance  education.  Even  within  technology-mediated  environments,  many  learners 
express  a preference  for  content-delivery  facilitated  by  instructional  design  models  (Technology  in  Higher 
Education  (T.H.E.)  Study  Group,  1998).  Communication  technologies  can  provide  an  opportunity  for  interaction 
between  and  among  learners  and  instructors,  but  while  many  participants  enjoy  the  interaction,  fewer  view  it  as  an 
essential  component  of  learning.  Wells  (1993)  notes  that  “Students  who  are  ‘socialized’  to  the  isolation  of  self- 
study  may  even  need  to  be  ‘resocialized’  to  the  educational  benefits  of  interaction!”  (p.  83).  Ensuring  that  on-line 
activities  are  meaningful  may  help  overcome  the  learner  resistance  that  results  from  exposure  to  not  only  new  ways 
of  learning,  but  also  new  technologies.  Interaction  should  not  be  merely  an  add-on  to  course  content,  but  rather,  an 
integral  component  of  the  learning  experience. 


Lack  of  Role  Models 

Instructors  as  well  as  learners  have  been  socialised  into  a culture  of  information  transmission  as  opposed  to 
one  of  knowledge  creation.  The  industrial  model  of  traditional  distance  education  has  provided  even  fewer  models 
of  constructivist  principles  than  has  traditional  face-to-face  education.  Communication  technologies,  however,  can 
provide  the  means  for  distance  educators  to  move  beyond  the  industrial  model  by  supporting  constructivist  learning 
principles  in  a post-industrial  distance  education  context  (Garrison,  1997). 

That  potential  notwithstanding,  distance  educators  have  few  role  models  for  designing  and  facilitating 
constructivist  learning  environments.  Teaching  is  an  activity  often  conducted  in  isolation  from  other  practitioners, 
and  this  is  even  more  of  a factor  for  distance  educators.  Pre-service  teacher  education  rarely  addresses  the  context 
of  distance  education  and  many,  if  not  most,  distance  educators  have  little  or  no  context  specific  instruction. 
Courses  and  programs  that  are  available  generally  focus  either  on  features  of  the  technology  or  on  instructional 
design  models.  There  is  a need  for  professional  development  activities  predicated  upon  constructivist  principles 
where  distance  educators  can  both  observe  and  practise  collaborative  interaction. 


Incorporating  Constructivist  Principles 

The  question  remains— how  can  distance  educators  incorporate  constructivist  principles  into  their  on-line 
classrooms?  Over  the  past  two  years,  we  have  instructed  several  sections  of  a technology-mediated  distance 
education  course  for  adult  educators  who  were  either  beginning  to  work  in  distance  education  environments  or 
expected  to.  Faced  with  the  constraints  previously  identified,  we  have  tried  to  create  complex  learning 
environments,  facilitate  learning,  foster  collaboration  between  and  among  the  course  participants,  and  encourage 
reflection.  Using  the  six  learning  principles  of  mulitiplicity,  activeness,  accommodation,  authenticity,  articulation, 
and  termlessness  identified  by  Koschmann,  Myers,  Feltovich  & Barrow  (1994)  as  a framework,  we  will  describe  some 
of  our  activities  and  their  outcomes. 


Mulitiplicity 

The  principle  of  multiplicity  refers  to  the  multiple  perspectives  inherent  in  any  authentic,  ill-structured 
learning  environment.  Knowledge  construction  is  a complex  activity  and  the  learning  context  should  reflect  this 
complexity.  When  planning  for  learning,  the  instructor  needs  to  focus  on  the  complex  whole  rather  than  on  the 
discrete  parts  of  tasks  and  problems.  This  is  especially  true  in  distance  education  where  breaking  a problem  into 
subsections,  or  a series  of  discrete  steps  is  a common  instructional  strategy.  At  the  same  time,  it  is  essential  to 
distinguish  between  a learning  activity  that  reflects  the  complex  nature  of  knowledge  and  a needlessly  complicated 
learning  activity.  If  an  activity  does  not  lead  to  the  desired  type  of  interaction,  it  is  possible  that  the  activity  itself  is 
overdesigned  and  interfering  with  or  restricting  interaction.  The  simplest  of  activities  can  provide  an  opportunity  for 
a rich  learning  experience  (Carr,  et  al.  (1998). 

The  concept  of  multiplicity  was  integrated  into  all  of  the  learning  activities  used  in  this  course.  Since  one  of 
our  goals  was  to  have  students  challenge  their  assumptions  about  what  it  means  to  teach  and  learn  in  a technology- 
mediated  environment,  students  were  expected  participate  in  and  critique  course  activities  from  the  perspectives  of 
both  instructor  and  learner.  Initially,  even  though  the  participants  are  themselves  instructors,  many  were  reluctant  to 
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relinquish  their  role  of  student,  and  we  found  that  it  was  frequently  necessary  to  actively  encourage  participants  to 
experience  the  activities  from  alternative  perspectives. 

In  order  to  provide  students  with  an  opportunity  to  assume  multiple  roles  and  perspectives,  we  attempted  to 
create  as  authentic  an  environment  as  possible.  It  has  been  our  observation  that  many  experienced  instructors  new  to 
technology-mediated  distance  education  attempt  to  transfer  teaching  and  learning  strategies  directly  from  their 
previous  context,  whether  that  be  face-to-face  or  traditional  modes  of  distance  education.  Since  this  is  seldom 
effective,  we  wanted  to  provide  course  participants  with  an  opportunity  to  experience,  within  a risk-free 
environment,  the  disequilibrium  or  frustration  that  comes  with  moving  from  the  familiar  to  the  unfamiliar,  so  that 
they  would  be  compelled  to  question  both  their  attitudes  and  practice.  To  accomplish  this,  we  first  analysed  the  way 
certain  activities  were  typically  implemented  in  face-to-face  adult  education  courses  and  then  moved  them  in  their 
entirety  into  the  virtual  classroom.  By  dividing  these  activities  into  a series  of  discrete  steps,  we  could  have  ensured 
that  learners  experienced  greater  'success'  in  the  class,  but  that  would  have  inadequately  prepared  them  for  the 
process  of  adapting  and  transferring  their  teaching  methods  to  the  virtual  classroom. 


Active  ness 

This  principle  refers  to  the  process  whereby  learners  assume  ownership  of  the  topic  or  problem  and  explore 
it  in  ways  that  are  most  relevant  to  their  needs.  This  precludes  an  emphasis  on  content  (the  traditional  objective, 
information,  measurement  model  that  guides  and  limits  discussion)  in  favour  of  the  creation  of  authentic  tasks.  Since 
adult  educators  have  more  than  a passing  familiarity  with  the  use  of  icebreakers,  students  were  asked  to  work  in 
groups  to  develop  an  icebreaker  suitable  for  use  on-line.  We  did  not  want  to  force  participants  to  curtail  their 
explorations  of  the  topic  in  order  to  submit  a product  or  to  direct  their  focus  toward  receiving  a grade,  so  no  marks 
were  assigned  to  the  product.  We  also  had,  and  still  have,  reservations  about  the  tendency  to  assign  grades  to  on- 
line activities  in  order  to  stimulate  participation,  when  similar  ‘in-class’  activities  would  not  be  graded  in  a face-to- 
face  environment. 

No  two  groups  approached  the  problem  in  the  same  way  since  the  members  of  each  group  were  influenced 
by  different  factors,  ranging  from  their  experience  with  distance  education  to  their  attitudes  toward  icebreakers  and 
the  characteristics  of  their  students.  Those  groups  where  some  members  had  prior  experience  in  distance  education 
spent  much  more  time  discussing  the  rationale  and  associated  problems  and  processes  in  advance  of  designing  an 
icebreaker  activity.  Those  groups  whose  members  had  no  first-hand  experience  with  distance  education  jumped 
right  into  the  development  stage,  but  as  they  experimented  with  different  icebreakers  and  encountered  an  assortment 
of  unforeseen  problems,  they  began  to  question  their  initial  assumptions.  It  would  have  been  impossible  to  predict 
the  variety  of  factors  and  resources  that  influenced  each  individual’s  contribution.  Nor  would  it  be  possible  to 
condense  their  negotiations  and  conclusions  into  a series  of  steps  that  could  be  used  to  ‘instruct’  another  group  of 
learners. 


Accommodation 

The  principle  of  accommodation  is  based  on  encouraging  learners  to  become  aware  of  and  challenge  their 
assumptions  at  the  most  basic  levels.  One  of  the  ways  we  try  to  help  participants  to  accomplish  this  is  to  expose 
them  to  difficulties  experienced  by  many  distance  education  instructors.  This  enables  them  not  only  to  enhance  their 
knowledge  and  skills,  but  also  to  question  their  attitudes  and  beliefs  about  distance  education  and  what  it  means  to 
be  a distance  educator. 

At  the  outset  of  the  course,  we  explain  the  learning  context  to  the  participants  and  try  to  prepare  them  for  a 
certain  amount  of  frustration  and  discomfort.  It  should  be  noted  here  that  this  course  is  part  of  a program  of  study 
and  that  the  students  have  already  completed  several  courses  using  the  technologies  before  beginning  the  one 
described  here.  In  spite  of  our  warning  and  their  earlier  experiences,  or  perhaps  partly  because  of  the  guided,  highly 
structured  nature  of  those  earlier  experiences,  the  ill-structured  nature  of  this  course  came  as  quite  a shock  to  some 
participants.  Since  the  course  has  a strong  affective  component,  participants  were  encouraged  to  discuss  their 
feelings  of  disequilibrium. 

One  activity  that  encourages  participants  to  challenge  basic  assumptions  is  introduced  at  the  beginning  of 
the  course.  Again,  students  are  asked  to  replicate  a routine  in-class  activity.  In  this  case  they  are  asked  to  "get 
themselves  into  groups"  and  "brainstorm"  a particular  topic.  They  have  one  week  to  accomplish  this  before 
reporting  back  to  the  class.  This  activity  brings  to  the  forefront  assumptions  about  group  dynamics.  Students 
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compared  the  on-line  process  with  what  takes  place  in  the  face-to-face  setting,  debated  the  rights  of  adult  learners  to 
autonomy,  discussed  factors  that  influence  the  success  of  self-selection,  and  suggested  possible  alternatives  for 
group  formation.  Throughout  the  remainder  of  the  course,  we  applied  these  suggestions,  and  they  were  then 
subjected  to  the  same  critique. 

It  must  be  noted  that  students  are  not  the  only  ones  to  experience  a certain  amount  of  turmoil.  Even 
instructors  who  are  proponents  of  experiential  learning  generally  want  those  experiences  to  be  positive  ones.  It  can 
be  difficult  to  let  students  make  'mistakes'.  Instead,  we  have  attempted  to  foster  a culture  of  adventure  rather  than 
one  of  successes  or  failures.  In  support  of  this,  we  facilitate  frequent  debriefings  allowing  learners  to  discuss  the 
process.  Applying  the  concept  of  accommodation  requires  the  instructor(s)  to  be  prepared  to  adapt  to  the  unique 
characteristics  of  each  group  of  learners.  Just  as  each  class  makes  different  suggestions  for  group  formation,  so  they 
suggest  creative,  innovative  variations  on  other  aspects  of  the  course.  And,  as  the  learners  confront  their 
assumptions,  we  too  must  continually  identify  and  challenge  ours. 


Authenticity 

An  essential  component  of  constructivist  learning  is  the  emphasis  placed  on  the  learning  context.  The 
rationale  for  offering  this  course  hinges  on  the  learning  context,  that  is,  making  the  transition  from  the  face-to-face 
classroom  to  the  technology-mediated  environment.  To  this  end  we  encouraged  participants  to  be  as  descriptive  as 
possible  about  their  working  environments,  both  face-to-face  and  virtual.  What  were  the  similarities  in  content, 
educational  goals,  learner  characteristics,  and  institutional  characteristics?  What  were  the  differences?  This  not 
only  helped  the  group  members  understand  the  unique  circumstances  of  the  others,  but  also  highlighted  conflicting 
expectations  resulting  from  the  change  in  context.  The  process  of  painting  these  detailed  pictures  provided  the  class 
with  a far  richer  context  than  could  be  provided  by  any  individual. 

At  another  level,  we  created  a context  for  learning  that  encouraged  risk-taking  and  experimentation.  We 
wanted  learners  to  be  innovative  within  the  virtual  classroom.  We  also  wanted  them  to  gain  enough  experience  that 
they  could  make  sound  instructional  choices.  Through  the  process  of  active  experimentation  and  the  discussions 
that  ensued  from  these  experiences,  we  hoped  to  provide  a model  learners  would  continue  to  use  to  analyse  and 
evaluate  their  practice. 


Articulation 

Learning  within  a constructivist  environment  should  be  a collaborative  rather  than  a competitive 
experience.  To  this  end,  students  are  encouraged  to  work  together  to  solve  problems  and  negotiate  solutions. 
Participating  in  this  way  not  only  helps  them  to  integrate  their  learning  into  their  prior  experience,  but  also  helps 
them  to  develop  communication  and  interaction  skills  that  they  can  apply  to  their  own  instructional  contexts.  For 
learners  who  have  been  socialised  to  be  independent  learners,  it  can  be  difficult  to  make  the  adjustment  to 
collaborative  activity.  And  since  this  type  of  interaction  goes  well  beyond  product-based  group  work,  it  is  not  only 
the  independent  learners  who  must  learn  new  skills.  The  instructor  should  be  prepared  to  model  collaborative 
dialogue.  This  can  include,  but  is  not  limited  to  questioning,  interpreting,  elaborating  upon  earlier  comments  and 
synthesising  the  discussion.  Comstock  and  Fox  (1997)  refer  to  this  as  the  social  process  of  making  meaning.  We 
should  note  here  that  using  a team-teaching  approach  can  be  advantageous  when  attempting  to  model  collaborative 
interaction. 


Termlessness 

At  the  same  time  that  students  are  participating  in  any  given  activity,  they  are  encouraged  to  engage  in  a 
parallel  discussion  about  the  process.  In  a face-to-face  class  it  would  be  necessary  to  halt  the  learning  activity  in 
order  to  discuss  the  learning  process  or  to  explore  one  or  more  of  the  assumptions  contributing  to  the  development 
of  the  activity,  but  computer  conferencing  allows  us  to  transcend  temporal  barriers.  This  parallel  processing 
illustrates  one  of  the  strengths  of  computer-conferencing  where  the  technology  acts  as  a cognitive  tool  that  supports 
metacognitive  activities.  Parallel  discussions  also  take  place  in  order  to  help  participants  process  the  affective 
component  of  the  course. 


o 

ERIC 


474 


Conclusion 


In  this  paper,  we  have  described  how  we  apply  constructivist  learning  principles  in  a technology-mediated 
distance  delivered  course  for  adult  educators.  We  identified  and  addressed  the  barriers  faced  by  adult  educators  who 
would  like  to  apply  constructivist  learning  theory  to  their  distance  education  courses.  Using  a model  developed  by 
Koschmann  et  al.,  1994)  as  a framework,  we  illustrated  some  of  the  approaches  used  to  create  a constructivist 
learning  environment  within  a very  specific  context.  Distance  educators  can  use  communication  technologies  to 
facilitate  the  application  of  constructivist  principles  to  the  on-line  environment  by  increasing  the  opportunity  for 
interaction  between  and  among  students,  overcoming  the  temporal  barrier  that  impedes  parallel  processing,  and 
encouraging  the  development  of  authentic  learning  activities.  If  the  technologies  are  to  meet  their  full  potential, 
however,  researchers  and  practitioners  must  direct  their  attention  to  evaluating  and  implementing  the  principles  of 
constructivist  learning  theory  in  order  to  bridge  the  gap  between  theory  and  practice. 

References: 

Brown,  J.,  Collins,  A.,  & Duguid,  P.  (1989).  Situated  cognition  and  the  culture  of  learning.  Educational  Researcher,  75  (1),  32- 
42. 

Carr,  A.,  Jonassen,  D.,  Litzinger,  M.,  & Marra,  R.  (1998).  Good  ideas  to  foment  educational  revolution:  The  role  of  systemic 
change  in  advanced  situated  learning,  constructivism,  and  feminist  pedagogy.  Educational  Technology,  35(1),  5-15. 

Comstock,  D.,  & Fox,  S.  (1997).  Computer  conferencing  in  a learning  community.  [Online].  Available: 
http://www.seattleantioch.edu/VirtualAntioch/compcon  1 .5.htm 

Duffy,  T.,  & Bednar,  A.  (1991).  Attempting  to  come  to  grips  with  alternative  perspectives.  Educational  Technology,  31  (10),  12- 
15. 

Duffy,  T.  & Jonassen,  D.  (Eds.).  (1992).  Constructivism  and  the  technology  of  instruction:  A conversation.  Hillsdale,  NJ: 
Lawrence  Erlbaum  Associates. 

Garrison,  D.  (1997).  Computer  conferencing:  The  post-industrial  age  of  distance  education.  Open  Learning,  12  (2),  3-11. 

Jonassen,  D.,  Davidson,  M.,  Collins,  M.,  Campbell,  J.,  & Bannan  Haag,  B.  (1995).  Constructivism  and  computer-mediated 
communication  in  distance  education.  The  American  Journal  of  Distance  Education,  9 (2),  7-26. 

Koschmann,  T.,  Myers,  A.,  Feltovich,  P.,  & Barrow,  S.  (1994).  Using  technology  to  assist  in  realizing  effective  learning  and 
instruction:  A principled  approach  to  the  use  of  computers  in  collaborative  learning.  Journal  of  the  Learning  Sciences,  3 (3), 
227-264. 

Lave,  J.,  & Wenger,  E.  (1991).  Situated  learning:  Legitimate  peripheral  participation.  Cambridge:  Cambridge  University  Press. 

Spiro,  R.,  Feltovich,  M,  Jacobson,  M.,  & Coulson,  R.  (1991).  Cognitive  flexibility,  constructivism,  and  hypertext:  Random  access 
instruction  for  advanced  knowledge  acquisition  in  ill-structured  domains.  Educational  Technology,  31  (5),  24-33. 

Technology  in  Higher  Education  (T.H.E.)  Study  Group.  (1998).  [Outreach  student  technology  questionnaire].  Unpublished  raw 
data. 

Wadsworth,  B.  (1971).  Piaget's  theory  of  cognitive  development  from  childhood  to  adolescence:  A constructivist  perspective. 
New  York:  Holt,  Rinehart,  and  Winston. 

Young,  G.  (1997).  Adult  development,  therapy,  and  culture:  A postmodern  synthesis.  New  York,  NY:  Plenum  Press. 

Zahorik,  J.  (1995).  Constructivist  Teaching.  (Fastback  390).  Bloomington,  Indiana:  Phi  Delta  Kappa  Educational  Foundation. 


Acknowledgements 

This  work  is  part  of  an  ongoing  research  program  sponsored  by  the  Office  of  Learning  Technologies  whose  stated  aim  is  to 
expand  innovative  learning  opportunities  through  technologies.  The  Office  of  Learning  Technologies  falls  under  the  auspices  of 
Human  Resources  Development  Canada. 


475 


Supporting  Graduate  Students  Training  District  Coordinators 
in  Multimedia  Development 


Caroline  M.  Crawford 
University  of  Houston  - Clear  Lake 
2700  Bay  Area  Boulevard,  Box  50 
Houston,  Texas,  USA,  77058-1098 
crawford@cl.uh.edu 


Abstract:  The  use  of  graduate  students  to  train  surrounding  independent  school  district 
Instructional  Technology  Coordinators  is  an  important  aspect  to  each  of  the  entities  involved  in  the 
situation:  the  graduate  students  obtain  opportunities  to  display  their  strengths  and  to  introduce 
themselves  to  the  surrounding  professionals  within  their  area;  the  university  obtains  positive 
feedback  concerning  the  strengths  of  their  students,  as  well  as  the  positive  public  relations 
displayed  at  such  times;  the  independent  school  district  Instructional  Technology  Coordinators  are 
offered  opportunities  to  leam  cutting-edge  technology  from  the  university,  the  use  of  university 
resources,  as  well  as  the  opportunity  to  view  possible  future  recruits  in  an  “on  the  job”  real- world 
situation.  Each  of  the  entities  has  positive  situations  through  which  to  explore  future  possibilities. 


Introduction 

Graduate  students  not  only  desire  but  also  need  the  help  of  their  university  faculty  to  present  learning 
situations  to  them.  These  learning  situations  are  not  only  to  aid  the  university  in  its  pursuits  but  also,  more 
importantly,  to  expand  the  students’  opportunities  to  leam  new  situations  and  environments.  However,  the  graduate 
students  must  be  mentored  so  as  to  produce  a positive  experience  within  the  three  primary  groups  involved.  This  is 
where  the  faculty  must  step  in  and  facilitate  the  development  of  the  course  curriculum  and  instructional  situations. 
The  previous  experience  of  the  faculty  member  can  help  guide  the  graduate  students  towards  a positive  experience 
for  all  members  involved. 

Hardware  and  Software  Issues 

Although  the  graduate  student  may  have  everything  ready  to  walk  into  a training  situation  and  successfully 
lead  a training  of  district  coordinators  in  multimedia  development,  last-minute  issues  consistently  arise.  Two  last- 
minute  issues  have  consistently  been  questions  pertaining  to  software  versions  and  computer  security  issues. 

Software  Versions 

The  graduate  student  has  diligently  worked  to  produce  an  awe-inspiring  training  module  and  can’t  wait  to 
jump  into  training  the  District  Coordinators.  However,  the  software  versions  of  the  tools  they  will  be  using  for  the 
training  that  can  be  located  on  the  computer  hardware  lag  behind  the  version  for  which  the  graduate  student  has 
developed  the  training  module.  Although  the  graduate  student  has  considered  absolutely  all  conceivable  situations 
that  may  arise  and  are  secure  in  the  fact  that  they  are  unquestionably  ready  for  anything,  this  problem  has  arisen  time 
and  again.  Although  the  graduate  student  has  the  latest  versions  of  every  single  software  title  available  on  the  market 
today,  the  university  may  lag  behind.  Unfortunately,  this  consideration  may  not  cross  the  faculty  member  or  the 
graduate  student’s  mind  until  faced  with  such  an  issue.  For  the  sake  of  all  concerned  with  the  training,  especially  to 
offer  a stress  reduction  to  the  graduate  student  that  may  be  faced  with  such  an  issue,  it  is  a hard-learned  lesson  to 
always  check  with  the  institution,  whether  university  or  field-based  organization,  that  will  house  the  training.  Not 
only  will  this  offer  a positive  environment  through  which  the  graduate  student  can  begin  the  scheduled  training,  but 
also  the  graduate  student  will  feel  more  secure  when  developing  the  training  module. 


Computer  Security  Issues  (Windows  NT) 

A second  consideration  when  dealing  with  such  basic  issues  as  hardware  and  software  is  the  question  of 
levels  of  computer  security.  This  is  not  usually  a problem  with  the  Macintosh  operating  system  (Apple,  1998), 
unless  another  level  of  software  is  added  to  the  operating  system  that  adds  a computer  security  issue.  Yet  the 
administrator  of  a Windows  operating  system,  whether  Windows  95/97  or  Windows  NT  (Microsoft,  1995; 
Microsoft,  1997a;  Microsoft,  1997b),  can  delineate  the  level  of  computer  security  on  each  computer  in  a training 
classroom.  This  must  be  discussed  before  the  training  is  developed  and  ready  to  facilitate;  without  such  discussions, 
surprises  will  occur.  The  computer  administrator  can  offer  numerous  suggestions  to  the  facilitator,  but  the 
suggestions  are  only  available  if  the  computer  administrator  knows  there  are  concerns  or  issues  that  must  be 
resolved.  Therefore,  the  forthright  inquiries  and  discussions  between  the  faculty  member  aiding  the  graduate  student, 
the  graduate  student,  and  the  computer  administrator  where  the  training  will  occur  is  a strong  suggestion  to  waylay 
any  training-morning  tragedies. 

Insecurity  and/or  Nervousness  of  the  Graduate  Student 

Whether  the  graduate  student  has  made  it  a career  to  facilitate  training  sessions  or  is  a novice  in  this  realm, 
the  insecurity  and/or  nervousness  that  any  facilitator  feels  before  the  beginning  of  a training  sessions  is  a natural 
experience.  However,  the  extent  to  which  a graduate  student  can  experience  this  insecurity  and/or  nervousness  may 
be  multiplied  due  to  the  added  pressure  of  the  faculty  member’s  expectations  and  the  district  coordinator’s 
expectations.  For  these  clear  reasons,  the  faculty  member  may  be  inclined  to  be  extra  sensitive  to  the  graduate 
student’s  needs  throughout  the  design,  development,  and  facilitation  of  the  training  module.  For  the  majority, 
graduate  students  have  a familiarity  with  trainings  and  professional  presentations;  however,  there  may  be  certain 
aspects  of  the  facilitation  of  the  training  modules  that  may  require  extra  sensitivity  and  discussion  between  the 
faculty  member  and  the  graduate  student. 

Interface  Between  the  Graduate  Student  and  the  Learners 

The  interaction  between  the  graduate  student  and  the  learners  in  the  training  session  is  of  the  utmost 
importance.  Although  the  graduate  student  may  be  nervous,  it  is  important  for  the  faculty  member  to  mentor  the 
graduate  student  to  present  a professional  demeanor.  This  is  a “safe”  situation  through  which  the  graduate  student 
can  learn  their  own  personal  training  style  and  have  the  ability  to  try  different  aspects  with  the  security  of  the 
situation  readily  apparent. 

Become  Easily  Flustered 

Although  all  presenters  become  flustered  at  one  time  or  another  in  their  careers,  the  impact  of  early 
presentations  on  the  psyche  of  presenters  can  have  long-lasting  effects.  Therefore,  the  importance  of  the  faculty 
member  acting  as  a mentor  and  guiding  the  graduate  student  is  of  the  utmost  importance.  The  graduate  may  become 
easily  flustered  due  to  the  intense  pressure  they  are  placing  upon  themselves  to  do  a good  job.  The  ease  with  which  a 
novice  presenter  may  become  flustered  is  quite  higher  in  probability  than  a more  experienced  presenter;  therefore, 
the  graduate  student  must  be  carefully  guided  and  aided  through  the  first,  most  important,  facilitation  of  training 
experiences.  As  the  old  saying  goes,  ‘what  can  happen  will  happen’,  and  this  is  doubly  true  when  a novice  presenter 
has  taken  the  helm. 

In  spite  of  the  fact  that  it  is  never  an  easy  situation,  when  a graduate  student  is  indeed  flustered  and  needs 
assistance,  the  faculty  member  should  be  available  to  step  in  and  offer  assistance  without  breaking  the  flow  of  the 
training  session.  The  careful  assistance  that  the  faculty  member  offers  during  the  first  few  training  sessions  that  the 
graduate  student  facilitates  will  develop  the  graduate  student’s  ease  and  confidence  for  future  presentations.  For  this 
reason,  it  is  of  major  importance  for  the  faculty  member  to  carefully  consider  the  point  at  which  to  offer  assistance, 
as  well  as  the  subtle  way  in  which  the  faculty  member  flows  in  and  out  of  the  training  environment;  if  too  obvious, 
the  faculty  member  will  develop  a sense  of  insecurity  within  the  graduate  student  and  may  impact  the  future 
presentation  ease  of  the  graduate  student.  Through  thoughtful  consideration  on  the  part  of  the  faculty  mentor,  the 
ability  of  a graduate  student  to  become  flustered  may  lessen. 

Pace  of  Presentation 
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A difficulty  that  presents  itself  for  novice  presenters  is  the  ability  to  check  the  rate  at  which  the  presentation 
is  flowing  to  ensure  the  learners  are  gaining  knowledge.  Many  novice  presenters  speed  the  training  to  such  a point 
that  the  learners  obtain  very  little  from  the  presentation;  yet  some  presenters,  already  mentored  to  take  a longer 
period  of  time  to  ensure  the  knowledge  is  obtained  by  the  learners,  moves  at  too  slow  of  a pace.  Each  presenter, 
whether  novice  or  expert,  must  gauge  the  learner’s  needs  and  pace  the  presentation  towards  the  learners  in  the 
educational  environment.  Although  this  may  be  an  obvious  presentation  rule,  novice  presenters  may  need  guidance 
to  ensure  the  learners  will  be  able  to  attain  the  knowledge  they  desire. 

Vocal  Range  and  Intensity 

Many  people  become  easily  excited  by  a subject  that  they  are  discussing  or  become  so  wrapped  up  in  their 
own  nervousness  that  it  is  difficult  to  gauge  the  vocal  range  and  intensity  that  is  being  presented.  Much  of  this 
comes  with  experience;  but  much  of  this  can  be  mentored  and  discussed  with  novice  presenters.  The  vocal  range  of 
a presenter  may  have  the  tendency  to  reach  higher  octaves  during  presentations,  especially  at  the  beginning  of  a 
presentation.  The  voice  range  should  be  practiced  and  become  a conscious  part  of  the  presenter’s  internal  checks 
during  professional  situations.  Lowering  the  voice  merely  one  or  two  octaves  offers  the  presenter  the  ability  to 
appear  calm,  centered  and  knowledgeable  even  if  the  presenter  is  about  to  lock  their  knees  and  faint  in  front  of  the 
crowd. 

The  intensity  of  the  voice  is  also  a consideration  worth  noting.  Some  presenters  have  a naturally  “soft” 
voice  that  rarely  can  be  heard  above  a conversational  tone  and  some  presenters  have  “booming”  voices  that  can  be 
heard  across  a grassy  knoll.  Although  both  of  these  situations  have  their  place  in  civilized  company,  the  difficulty  is 
gauging  the  desired  effect  for  the  situation  one  finds  them  in.  The  graduate  student  may  particularly  affected  by  the 
question  of  vocal  intensity  due  to  the  amount  of  professional  presentations  that  have  been  presented.  The  faculty 
member  mentor  can  offer  particularly  helpful  advice  in  this  situation;  although  this  must  be  carefully  done.  If  the 
graduate  student  is  presenting  a training  in  a computer  environment,  the  noise  from  the  machines  will  add  to  the 
intensity  of  the  student’s  voice;  however,  if  the  graduate  student  is  presenting  at  a round  table  discussion,  the 
intensity  of  the  graduate  student’s  voice  will  significantly  drop.  Gauging  one’s  vocal  intensity  to  fit  the  desired 
situation  is  an  area  in  which  a faculty  member  can  mentor  a graduate  student. 

Presentation  Style 

Each  presenter  has  their  own  distinct  presentation  style;  whether  energetic  or  demure,  loud  or  soft, 
structured  or  less  structured,  handouts  by  the  ream  of  paper  or  no  handouts.  There  is  a certain  format  through  which 
the  presenter  finds  a comfort  zone.  Once  this  comfort  zone  is  found,  the  presenter  engages  the  learners  in  a 
consistent  manner;  however,  this  comfort  zone  is  not  usually  yet  discovered  by  the  novice  presenter.  Herein  the 
faculty  member  mentor  has  the  ability  to  aid  the  graduate  student. 

Respect  the  Learners 

Although  this  is  an  obvious  area,  sometimes  people  become  so  involved  in  their  own  insecurities  that  they 
feel  the  need  to  present  themselves  as  the  most  knowledgeable  for  the  subject  they  are  teaching.  Only  they  could 
possibly  have  the  answers  that,  the  learners  are  seeking.  This  develops  beyond  the  training  areas  of  multimedia  and 
sweeps  across  any  subject  area  that  is  available;  it  seems  to  be  a natural  human  reaction  to  insecurity.  However,  the 
learners  have  their  own  prior  knowledge  that  they  bring  to  any  endeavor  and  this  prior  knowledge  and  expertise 
must  be  respected.  An  area  of  considerable  concern  is  the  way  a presenter  may  talk  to  a learner;  sometimes  the  lack 
of  experience  or  security  within  themselves  may  develop  into  a way  of  talking  “down”  to  a learner  in  order  to  feel 
that  the  presenter  is  the  sole  person  in  the  room  with  the  ability  to  offer  any  kind  of  knowledge  attainment  to  the 
learner.  This  must  be  carefully  watched  and  the  faculty  member  mentor  should  work  towards  developing  a sense  of 
respect  towards  the  learners  within  the  graduate  student. 

Speak  to  the  Whole  Group 

When  nervous,  it  can  be  a natural  reaction  to  only  focus  upon  a specific  section  of  the  room  or  a certain 
select  few  members  of  an  audience.  This  can  sometimes  make  the  presentation  of  information  much  less  daunting  to 
a novice  presenter.  But  this  must  be  discussed  and  the  graduate  student  must  be  made  aware  of  these  faults;  only 
through  the  careful  integration  of  all  learners  in  all  areas  of  an  educational  environment  will  the  successful 
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attainment  of  knowledge  be  accomplished.  However,  the  difficulty  lies  within  the  timing  of  mentoring  the  graduate 
student:  if  discussed  before  the  presentation,  the  graduate  student  may  focus  too  heavily  upon  this  area;  if  discussed 
during  a presentation,  the  graduate  student  may  become  more  nervous  and  insecure;  if  discussed  after  the 
presentation,  the  graduate  student  may  harbor  greater  insecurities  concerning  their  presentation  abilities.  This  is  an 
area  that  the  faculty  member  mentor  must  address  on  a student-by-student  basis. 

Handouts 

Learners  enjoy  having  something  that  they  can  physically  hold  onto,  whether  it  is  paper  handouts  or  a CD- 
ROM  that  includes  further  information  or  merely  a Web  page  that  the  learner  can  download  at  a later  time.  Handouts 
tend  to  give  a sense  of  security  to  the  learner  that  is  felt  beyond  the  mere  presentation  by  the  graduate  student.  This 
is  an  area  in  which  the  graduate  students  should  be  mentored;  the  format  of  the  handouts,  the  inclusion  of 
information  within  the  handouts,  and  the  inclusion  of  graphics  within  the  handouts  are  all  important  lessons.  The 
faculty  member  mentor  can  help  guide  the  graduate  student  while  learning  such  issues,  as  well  as  aid  the  graduate 
student  when  difficulties  arise  during  the  design  and  development  process. 

Write  Difficult  Points  on  a Large  Board 

During  the  presentation,  questions  and  issues  may  be  discussed  that  are  easier  to  note  or  present  through  the 
use  of  a blackboard  or  similar  type  of  writing  situation.  Sometimes,  graduate  students  tend  to  shy  away  from  the  use 
of  a large  board  on  which  to  write  important  aspects  of  the  discussion,  yet  this  type  of  visual  display  can  aid  a 
discussion.  Therefore,  the  graduate  student  should  be  encouraged  to  use  a writing  board  when  it  will  aid  the 
discussion  or  attainment  of  knowledge. 

Positive  Feedback 

Humans  enjoy  positive  feedback  if  in  a complimentary,  sincere  manner.  This  is  an  important  aspect  of  any 
presentation,  whether  complimenting  a learner  on  a question  raised,  complimenting  an  innovative  attempt  at 
developing  multimedia,  or  requesting  aid  when  a difficult  part  of  the  learning  process  is  reached,  positive  feedback 
is  important  in  the  motivation  and  success  of  the  learner.  However,  the  majority  of  graduate  students  has  been  on  the 
receiving  end  of  the  positive  feedback  for  an  extended  period  of  time  and  may  not  have  made  the  connection 
between  positive  feedback  and  the  success  of  the  presentation.  The  faculty  member  mentor  can  aid  this  situation  by 
mentoring  the  graduate  students  in  the  use  of  positive  feedback. 

Move  Through  Out  Class  to  Check  for  Understanding 

The  movement  of  a presenter  through  out  an  educational  setting  offers  the  presenter,  as  well  as  the  learners, 
numerous  advantages.  For  example,  the  presenter  can  check  for  understanding,  can  keep  themselves  from  becoming 
a part  of  the  “talking  head”  syndrome,  can  aid  learners  that  may  not  be  exactly  where  the  presenter  hopes  the 
learners  will  be  in  the  educational  situation,  as  well  as  numerous  other  positive  outcomes.  The  learners  receive 
support  from  the  presenter  moving  through  out  the  educational  setting  through  the  ability  to  ask  specific  questions  of 
the  presenter  in  a more  informal  setting,  become  psychologically  more  on  task  with  the  goals  and  objectives  of  the 
educational  setting,  and  make  the  educational  setting  one  of  a more  collegial  attitude.  Through  such  simplistic 
actions  as  the  presenter  weaving  his  or  her  way  through  out  the  educational  setting,  the  feeling  of  the  environment 
can  drastically  shift  to  a more  positive  structure.  This  is  an  area  that  a novice  presenter  may  not  necessarily  be 
comfortable  with  or  be  aware  of,  yet  the  faculty  member  mentor  would  strongly  be  urged  to  make  mention  of  this 
simplistic  addition  to  a presentation. 

Closure  to  the  Learning  Situation 

Unless  previously  educated  as  a presenter,  many  presenters  may  not  be  aware  of  the  necessary  closure  to  a 
learning  situation.  Smiling,  throwing  their  hands  up  in  the  air  and  loudly  stating  “we’re  done!”  is  not  an  appropriate 
closure  to  a presentation;  however,  after  hours  upon  hours  of  developing  presentations  or  training  modules  the 
novice  presenter  may  be  so  overwhelmed  by  being  at  the  end  of  the  presentation  that  they  may  forget  their  closure. 
However,  this  is  an  area  of  the  presentation  of  which  the  faculty  member  mentor  should  be  way  and  guide  the 
graduate  student.  A closure,  as  in  all  presentations,  is  of  the  utmost  importance  and  must  be  carefully  developed. 
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This  is  the  point  at  which  most  learners  will  remember  the  presenter;  therefore,  the  closure  must  not  only  be 
included,  but  must  have  the  strength  to  endure. 

Conclusions 

Graduate  students  have  a multitude  of  abilities  and  strengths  to  offer  within  many  situations;  however,  the 
knowledge  that  the  graduate  students  bring  to  an  educational  situation  may  not  be  couched  within  presentation 
experience  and  training  situations.  The  thoughtful  mentoring  on  the  part  of  a faculty  member  mentor  can  aid  the 
graduate  students  in  the  development  of  presentation  experience  and  training  situations  through  the  thoughtful, 
reflective  guidance  that  is  necessary  to  develop  into  strong  presenters.  Through  the  experience  of  training  district 
coordinators  in  multimedia  development,  the  graduate  students  have  the  ability  to  learn  such  important  aspects  of 
their  graduate-level  work  as  presentation  and  training  skills. 
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Abstract:  Evolutions  in  education  lead  to  an  increased  use  of  problem -based  learning  and  a 
variety  of  technologies.  This  contribution  discusses  the  issue  of  support  in  technologically 
rich,  problem-based  and  collaborative  learning  environments.  Five  dimensions  of  support  are 
identified:  amount  of  support,  topical  object,  formal  object,  delivery  system  and  timing. 

In  the  ParlEuNet-project,  an  European  project,  four  different  support  models  will  be 
investigated  in  a rich  technological,  problem-based  learning  environment.  Two  models  relate 
to  access  to  a dedicated  database,  the  two  others  relate  to  the  role  of  the  teacher. 


Introduction 

New  information  delivery  and  communication  technologies  open  a whole  new  world  of  opportunities 
and  possibilities  to  access  information  and  to  communicate  on  an  international  level.  These  technologies  are 
entering  schools,  which  generates  questions  on  how  to  use  these  effectively  in  regular  class  situations. 

At  the  same  time,  new  insights  in  learning  and  instruction  have  resulted  in  a shift  from  teacher-oriented 
towards  student-centered  education  and  from  encyclopedic  towards  problem-based  instruction  by  presenting  ill- 
structured  tasks  and  supporting  learners.  Teachers  become  facilitators  of  learning  in  the  process  of  gaining 
knowledge. 

The  question  on  the  amount  and  kind  of  support  to  be  provided  is  becoming  of  paramount  importance 
to  integrate  new  technologies  as  valuable  tools  in  powerful  learning  environments.  How  the  question  on  support 
is  answered  depends  on  the  theoretical  stance  that  one  takes  with  regard  to  learning.  In  this  contribution  a 
moderate  constructivist  view  on  learning  is  adopted. 

In  this  contribution  first  the  issue  of  support  is  analyzed  theoretically.  Next,  it  is  discussed  how  support 
is  dealt  with  in  the  ParlEuNet-project.  In  this  implementation  and  research  project  the  issue  of  support  in  a 
technologically  rich,  problem-based  and  collaborative  learning  environment  is  addressed  directly. 

In  the  conclusions  it  is  specified  how  the  ParlEuNet-project  studies  the  support  issue. 


The  Issue  of  Learning  Support 

Given  the  acknowledgement  that  not  all  learners  are  willing  to  execute  all  necessary  learning  activities 
(Elen,  1992;  Jonassen,  Mayes  & McAleese,  1991),  it  is  accepted  even  by  constructivist  that  instruction  has 
primarily  a supportive  role  (Elen,  1992;  Lowyck  & Elen,  1991).  Support  aims  at  enabling  the  learner  to  engage 
in  appropriate  learning  activities. 


Part  of  this  research  is  sponsored  by  the  European  Commission  ‘Targeted  socio-economic  research’.  Directorate  General 
XII,  Science  research  and  development.  Directorate  G,  as  part  of  the  Joint  Call  for  Educational  Multimedia  under  contract 
number  MM  1022. 
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Quality  of  instruction  is  dependent  on  the  choice  of  the  tasks  and  the  adaptation  of  support  to  what  is 
actually  needed  by  individual  learners  and/or  group  of  learners.  In  view  of  deciding  about  support,  an  analysis 
of  different  dimensions  of  support  is  required. 


Dimensions  of  Support 

Decisions  about  support  can  be  situated  on  different  dimensions  (Elen,  1995).  Each  of  them  will  be 
discussed. 

1.  Amount  of  support:  the  learning  process  is  seen  as  an  active,  constructive,  cumulative  and  situated 
process.  The  learner  should  determine  his/her  learning  goals,  select  appropriate  activities  and  find  a way  to 
fulfil  them.  The  necessary  amount  of  support  will  depend  on  the  extent  that  the  learner  is  able  and  willing  to 
construct  his/her  own  knowledge.  Consequently,  the  amount  of  support  has  to  decrease  with  increasing 
knowledge,  ability  and  motivation.  Indeed,  research  demonstrated  that  the  relationship  between  amount  of 
support  and  learning  is  curvilinear,  both  insufficient  and  excessive  support  is  detrimental  (Clark,  1989). 

Support  has  to  be  adapted  to  the  specific  needs  of  individual  students.  A gradual  withdrawal  of  support  is 
generally  advocated.  After  a period  in  which  cognitive  activities  are  modeled  by  the  instructional  agent,  the 
learner  is  gradually  more  and  more  activated  to  execute  cognitive  and  metacognitive  activities.  Finally,  all 
explicitly-designed  support  is  removed  and  support  is  to  be  actively  requested  by  the  learner  (Dillemans, 
Lowyck,  Van  der  Perre,  Claeys  & Elen,  1998). 

2.  Topicafobject  of  support:  the  second  dimension  relates  to  the  topical  object  of  support.  This 
topical  object  especially  relates  in  rich  technological,  problem-based  and  collaborative  learning  environments  to 
different  levels.  With  respect  to  problem  solving  students  will  need  support  on  both  the  process  of  solving  a 
problem  and  the  content  of  the  task. 

Collaborative  learning  increases  the  need  for  support  on  the  group  and  the  individual  level.  Students 
working  in  group  will  benefit  from  guidance  on  the  process  of  working  in  group,  role  division,  work 
distribution  and  balancing  individual  and  group  needs.  A technological  environment  adds  a fourth  level. 
Students  may  require  support  to  use  the  technological  tools.  Students  need  to  know  how  the  tools  work  and 
what  can  be  done  with  them  for  solving  the  task. 

3.  Formal  object  of  support:  Support  has  also  a formal  object  that  relates  to  the  student  characteristics 
targeted  by  support.  Support  may  be  directed  towards  students’  prior  knowledge,  (meta-)cognitive  skills  or 
motivation.  Each  of  these  learner  variables,  sometimes  combined  in  a ‘learning  style’,  may  be  the  formal  object 
of  support  (Dillemans,  et  al.,  1998).  If  lack  of  motivation  is  the  major  problem  or  prior  knowledge  is,  support 
will  differ. 

In  the  research,  attention  has  shifted  from  prior  knowledge  as  the  formal  object  of  support  to 
metacognition  and  motivation.  This  parallels  the  shift  from  encyclopedic  to  problem-based  learning. 

4.  Delivery  system  of  support:  A fourth  dimension  pertains  to  the  system  through  means  of  which  the 
support  is  delivered.  Decisions  relate  to  the  extent  to  which  support  is  embedded  in  technology,  made  available 
through  means  of  textbooks,  or  delivered  by  peers  or  a teacher.  In  rich  technological,  problem-based, 
collaborative  learning  environments  support  can  be  delivered  through  a variety  of  means.  With  increased 
technological  possibilities,  support  can  be  easily  embedded  in  the  technology  or  delivered  through  the 
technology  rather  than  by  a teacher. 

In  a problem-based  environment,  support  is  partly  embedded  in  the  task.  The  task  must  connect  to  the 
prior  knowledge  of  students  as  to  trigger  interest  and  motivation.  In  a collaborative  environment,  other 
members  of  the  group  may  also  deliver  support  (explaining,  solving  problems). 

The  question  of  what  delivery  system  to  use  for  what  kind  of  support  is  basically  a question  of 
efficiency.  Some  support  may  be  more  easily  provided  by  technology,  other  by  teachers  or  peers. 

5.  Timing  of  support:  The  last  dimension,  timing,  has  several  aspects.  First,  given  the  curvilinear 
relationship  between  learning  and  support,  one  may  expect  the  need  for  support  to  decrease  with  growing  prior 
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knowledge,  cognitive  ability  and  motivation  (Vermunt,  1992).  Secondly,  in  the  4C/1D  model,  van  Merrienboer 
(1997)  has  suggested  that  in  order  to  be  effective  some  support  (e.g.  explanatory  information)  has  to  be 
delivered  up-front  and  some  support  just-in-time. 

In  a rich  technological,  problem-based  and  collaborative  learning  environment,  support  on  the 
different  topical  levels  must  be  delivered  at  different  times.  Technology  enables  to  build  just-in-time  support 
systems.  However,  technology  itself  also  requires  just-in-time  support.  Students  need  support  immediately  if 
technology  fails.  For  solving  a problem  and  working  in  group,  support  can  be  given  in  advance  by  for  instance 
explaining  the  principles  of  group  work  and  the  different  steps  in  problem  solving.  Similarly,  moments  of 
orientation  and  reflection  during  problem  solving  can  be  well-suited  for  up-front  feedback,  by  anticipating 
problems  based  on  passed  experiences. 


The  Role  of  the  Teacher  in  these  Different  Dimensions  of  Support 

Teachers  in  rich  technological,  problem-based  learning  environments  are  no  longer  the  ‘experts’. 
Rather,  the  role  becomes  one  of  providing  resources  and  managing  mediating  interventions  in  order  to  enable 
learners  to  achieve  learning  goals  (Devine,  1994).  The  teacher  becomes  a coach  or  learning  facilitator  (Elen, 
1995;  Roblyer,  Edwards  & Havriluk,  1997)  and  delivers  support  in  accordance  to  this  role.  In  a problem-based 
student-centered  learning  environment  support  is  aimed  at  helping  students  to  perform  optimally.  The  teacher 
can  temporarily  provide  adequate  support  in  order  to  enable  students  to  function  within  their  ‘zone  of  proximal 
development’.  Through  this  support  students  may  perform  on  a level  just  above  their  current  level  of 
performance. 

The  topical  object  of  teacher  support  in  rich  technological,  problem-based  and  collaborative  learning 
environments  is  directed  to  all  levels.  Teachers  support  use  of  the  technology,  problem  solving,  working  both  in 
group  and  individually.  The  formal  object  of  this  support  may  differ;  and  is  to  be  adapted  to  the  specific  needs 
of  the  students. 


The  ParlEuNet-Project 

The  ParlEuNet  project  aims  at  creating  a problem-based,  student-centered  rich  technological  learning 
environment  in  which  students  from  12  secondary  schools  in  7 countries  may  learn  about  the  European 
Parliament.  With  the  following  three  main  pillars,  the  educational  use  of  technology  is  focussed  upon: 

(1)  A problem-based  learning  environment  is  offered  in  which  students  work  on  a multi-dimensional  problem. 
In  view  of  acquiring  problem-solving  and  information-processing  skills,  the  task  constitutes  a real  and  ill- 
structured  problem.  Moreover,  motivating  to  students  and  allowing  them  to  apply  a variety  of  information- 
processing and  problem-solving  skills. 

(2)  In  the  ParlEuNet-project  students  work  in  groups  with  peers  from  their  own  classgroup  and  from  classes  in 
different  countries.  The  task  has  to  allow  for  meaningful  and  international  collaboration,  sufficiently 
complex  and  encourage  co-operation  in  view  of  cognitive  load  distribution  (Hmelo,  Guzdial,  & Turns, 
1997).  In  the  ParlEuNet-project  students  receive  a problem  related  to  the  world  of  work  from  a member  of 
the  European  Parliament.  Students  are  requested,  together  with  students  from  internationally  spread 
schools  in  other  countries,  a functional  policy  document  on  employment  by  following  the  regulations  of  the 
European  Parliament.  A minimum  of  structure  must  enable  students  to  finish  this  ill-structured  problem 
within  32  hours. 

(3)  In  order  to  solve  the  above  described  task,  students  get  access  to  a variety  of  information  and 
communication  tools:  a dedicated  database  with  information  on  the  European  Parliament  and  the  task, 
word  processing,  spreadsheet  and  presentation  tools,  access  to  the  Internet;  free  access  to  telephone  and 
fax,  an  e-mail  based  communication  tool  with  white-board  facilities  and  finally  access  to  video- 
conferencing  once  a week  for  communicating  with  their  peers  in  the  other  schools.  Students  themselves 
decide  on  the  use  of  these  tools. 
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Support  in  the  ParlEuNet-Project 


An  important  question  in  the  ParlEuNet-project  addresses  the  issue  of  support.  Some  of  this  support 
will  be  equal  for  all  students  and  some  will  be  variable.  All  students  will  get  support  on  the  level  of  solving  the 
problem;  they  will  get  the  same  task  and  sub  tasks;  collaboration  and  working  with  technology.  For  solving  the 
task  students  may  rely  on  a project  booklet,  containing  a description  of  different  problem  solving  steps.  The 
booklet  also  contains  help  for  working  in  group  (roles  and  work  division)  and  some  explanations  on  the  use  of 
the  tools  are  given  as  well. 

Four  support  delivery  systems  can  be  identified:  booklet,  teacher,  technology  and  peers.  While  the 
booklet  is  identical  to  all  students,  support  delivered  by  teachers  and  database  is  not.  Both  delivery  systems  have 
two  modes  (Tab.  1).  First,  access  to  the  dedicated  database  is  ‘open’  or  ‘scaffolded’.  In  the  ‘open’  mode, 
students  have  access  to  all  the  information  in  the  database  at  any  time.  In  the  ‘scaffolded’  mode,  students  can 
get  hold  of  all  the  information,  but  access  to  the  information  is  structured  in  line  with  the  specifications  of  the 
4C/ID-model  (Van  Merrienboer,  1997)  and  accommodates  the  specific  role  of  the  student  (for  example,  a 
student  with  an  ‘expert’  role  will  be  oriented  towards  using  the  information  on  the  specific  topic  whereas  a 
student  with  a ‘liaison  officer’-role  will  be  oriented  to  the  information  on  publication  regulations  of  the 
European  Parliament). 

With  respect  to  the  teacher  as  a support  delivery  system  two  models  will  be  studied:  a pro-active  and 
re-active  model.  The  teacher  models  in  the  ParlEuNet-project  do  primarily  differ  as  to  who  takes  the  initiative 
to  deliver  support.  In  the  case  of  pro-active  support,  the  teacher  is  part  of  the  group  that  works  on  the  task  and 
may  -as  any  group  member-  take  the  initiative  to  assist  the  students  in  their  work  on  the  task.  In  the  re-active 
support  group,  however,  the  teacher  is  not  part  of  the  group  but  functions  as  an  independent  process-oriented 
consultant.  In  the  re-active  support  mode,  teachers  will  assist  students  only  upon  their  request. 


Database  access 

Open 

Scaffolded 

By  teacher 

Pro-active 

6 schools  / 72  students 

6 schools  / 72  students 

Re-active 

6 schools  / 72  students 

6 schools  / 72  students 

Table  1:  Four  types  of  support  in  the  ParlEuNet-project. 


The  Pro-Active  Teacher 

Support  in  the  pro-active  mode  will  be  primarily  cognitive  apprenticeship-like  (Collins,  Brown,  & 
Newman,  1989).  Apprenticeship  originally  pertained  to  the  process  where  novices  are  acculturated  in  an 
industrial  setting  by  means  of  externalizing  or  verbalizing  the  internal  processes  or  activities  involved  in  expert 
problem  solving  (modeling),  coaching,  and  scaffolding  (Mandl  & Prenzler,  1991).  In  a cognitive 
apprenticeship-environment,  the  acquisition  of  cognitive  skills  is  focussed  upon.  Students  are  encouraged  to 
engage  in  complex  problem  solving,  collaboration  and  reflection  in  view  of  the  development  of  cognitive  skills. 
A teacher  may  model  a cognitive  skill,  coach  learners  in  their  attempts  to  execute  the  skill  and  execute  sub 
tasks  learners  are  not  yet  able  to  execute  successfully.  In  the  ParlEuNet-project  the  students  are  acculturated  in 
the  process  of  international  collaborative  problem-solving.  The  pro-active  teacher  will  be  part  of  the  working 
group  and  will  help  students  while  identifying  problems,  organizing  the  work,  communicating  with  students  in 
other  countries,  retrieving  information,  drafting  a report,  ...  by  modeling,  coaching  and  scaffolding.  This 
support  by  the  teacher  is  not  product-oriented  (finishing  the  task)  but  primarily  directed  towards  the 
development  of  cognitive  skills  and,  hence,  more  autonomy  of  the  students.  The  students  themselves  remain 
responsible  for  executing  the  task. 

The  pro-active  teacher  is  a full  member  of  the  problem-solving  group.  Together  with  the  students, 
teachers  work  towards  realizing  the  task.  As  any  pro-active  group  member  would  do,  the  pro-active  teacher 
may  anticipate  the  needs  of  the  students  and  make  suggestions  on  how  to  solve  the  problem.  For  instance,  the 
pro-active  teacher  may,  even  before  the  student  asked  for  it,  suggest  places  where  relevant  information  can  be 
found  to  a student  who  is  looking  for  information. 
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The  Re-Active  Teacher 


Support  in  the  re-active  mode  is  based  primarily  on  discovery  learning  ideas  and  insights,  with  an 
emphasis  on  self-direction  and  intentionality  in  learning  (Bruner,  1966).  Discoveiy  learning  starts  from  the 
idea  that  cognitive  skills,  like  problem  solving,  emerge  gradually  as  a consequence  of  successful  coping  with 
the  environment.  These  cognitive  skills  can  be  brought  on  their  optimal  level  by  sensitization  and  activation 
through  repeated  experiences  of  their  successful  use  (Crutchfield,  1966).  This  implies  that,  in  order  to 
experience  success,  students  will  explore  and  experiment  with  the  environment,  without  real  guidance  of  the 
teacher.  The  impulse  for  exploring  and  experimenting  in  the  environment  lies  in  the  task.  Two  sources  will 
make  students  intrinsically  motivated  to  work  on  a task  (Bruner,  1966),  curiosity  and  the  drive  to  achieve 
competence.  An  ‘authentic’  and  challenging  task  that  can  be  achieved  and  appeals  to  the  curiosity  of  students 
will  be  motivating  to  students  and  reduces  the  need  for  extensive  teacher  support. 

In  a discovery  learning  approach,  the  teacher  extends  the  student’s  range  of  experiences,  by  providing 
the  students  with  opportunities  to  explore  a situation  and  by  encouraging  students  to  generate  and  test  multiple 
hypotheses.  The  teacher  is  part  of  that  environment,  as  an  information  resource  (Bruner,  1960).  But,  the 
students  will  be  enabled  to  ‘discover’  everything  themselves.  Students  take  the  initiatives.  The  teacher  only 
provides  an  environment  and  challenges  the  students  in  order  to  promote  reflection  (Jonassen,  1996). 

In  line  with  these  principles,  the  re-active  teacher  will  wait  until  the  students  ask  something,  his 
coaching  is  based  on  past  experiences  of  the  group.  The  re-active  teacher  acts  like  an  independent,  process- 
oriented  consultant  to  the  collaborative  learning  group.  On  technical  and  organizational  questions,  direct 
answers  may  be  provided,  in  all  other  cases  answers  of  the  teacher  will  aim  at  encouraging  students  to  reflect 
upon  their  activities  and  explore  alternatives. 


Conclusions  and  perspectives 

As  previously  indicated,  the  ParlEuNet-project  is  both  an  implementation  and  a research  project.  In  the 
above  the  support  and  its  implementation  in  the  ParlEuNet-project  has  been  presented.  The  project  enables  to 
study  the  issue  of  support  in  depth.  The  effects  of  the  four  ‘support  models’  are  investigated.  As  such,  the 
ParlEuNet-project  constitutes  a real  example  of  a design  experiment.  At  least  72  students  will  receive  support 
in  line  with  each  of  the  four  support  models  (Tab.  1).  The  project  aims  at  identifying  the  effects  of  these  four 
support  models  on: 

quality  of  the  policy  document  produced  by  the  students; 

knowledge  of  and  attitudes  towards  the  European  Parliament  and  its  operations; 

attitude  towards  the  educational  use  of  technology;  collaborative  learning,  and  problem-based  learning; 

students’  use  of  technology  during  the  project  sessions,  and 

students’  use  of  cognitive,  meta-cognitive  and  collaborative  strategies  during  the  project  sessions, 
change  in  epistemological  beliefs  of  the  students 

Student  characteristics  (prior  knowledge,  interest  in  the  task,  cognitive  ability,  gender  and  cultural 
background)  will  be  taken  into  account  as  co-variates.  With  respect  to  teacher  support  the  two  main  research 
questions  are: 

1.  Can  differences  in  epistemological  beliefs,  attitude  towards  technology,  problem-based  and  collaborative 
learning  after  the  project,  be  explained  by  (interaction  between)  the  amount  and  kind  of  support  on  the  one 
hand  an  students  characteristics  (prior  knowledge,  (meta)cognitive  skills,  attitudes)  on  the  other  hand? 

2.  Can  the  patterns  of  cognitive  information  processing  and  communication  activities  be  explained  by 
(interaction  between)  the  amount  of  support  on  the  one  hand  and  student  characteristics  on  the  other  hand? 

In  order  to  answer  these  research  questions  a classical  pre-test,  post-test  research  design  has  been 
opted  for.  During  the  actual  work  on  the  task  students’  activities  are  monitored  in  order  to  identify  differences 
between  groups  in  the  use  of  cognitive,  meta-cognitive  and  collaborative  strategies.  Additional  data  will  be 
gathered  through  means  of  the  log-files  that  will  keep  track  of  students’  use  of  the  database  and  of  the 
communication  tools.  As  much  as  possible,  readily  available  research  instruments  are  used.  In  some  cases, 
however,  (e.g.  measuring  prior  knowledge  of  and  attitude  towards  the  European  Parliament)  specific 
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instruments  have  to  be  developed.  For  measuring  the  prior  knowledge  and  attitude  towards  the  European 
Parliament  a questionnaire  has  been  developed,  which  will  be  administered  three  times:  (1)  at  the  start  of  the 
project,  (2)  at  the  end  of  the  project  and  (3)  10  weeks  after  the  project.  This  will  give  us  a better  view  on  the 
evolution  that  students  went  through  and  whether  there  are  differences  between  the  different  conditions. 
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Abstract:  The  use  of  Computer  Mediated  Communications  (CMC)  has  brought  change  in 
our  ways  of  communicating  with  others.  This  is  self-evident  in  the  world  of  commerce  and  it 
seems  intuitive  that  this  should  also  hold  true  within  any  community  whose  use  of  CMC  has 
become  widespread.  This  paper  considers  the  nature  of  change,  if  any,  that  CMC  has  brought 
about  within  academic  research  communities.  A pilot  study  with  groups  of  academic 
researchers  is  used  as  the  focus  of  discussion.  Some  ‘changes’  brought  about  by  the  use  of 
CMC  in  academic  research  were  obvious  and  unsurprising.  Other  changes,  though  less 
obvious,  may  be  more  significant  in  bringing  about  change  in  approaches  to  academic 
research.  The  paper  concludes  by  touching  on  what  was  seen  by  participants  as  not  only  the 
enabling  aspects  of  the  use  of  CMC  but  also  ways  in  which  CMC  must  develop  in  order  to  be 
used  effectively  in  academic  research. 


Introduction 

The  development  of  Arpanet  in  the  USA  and  subsequently  of  university  academic  networks  in  Europe  such  as 
JANET  and  EARN  has  contributed  to  Computer  Mediated  Communications  (CMC)  being  available  in  British 
universities  for  the  last  two  decades.  It  is  in  recent  years,  however,  that  easy  access  to  CMC  has  become  more 
generally  available  to  academics  of  all  disciplines  in  British  universities.  Patterns  underlying  the  increase  in 
CMC  use  within  UK  research  groups  during  this  time  span  are  discussed  in  (Fung  & Weerasinghe,  1998)  and 
can  be  situated  within  the  literature  of  innovation  adoption  (Rogers,  1995).  While  research  into  the  speed  and 
manner  of  adoption  of  CMC  within  communities  (Kerr  & Hilz,  1982,  Lea  et  al,  1995)  is  in  itself  an  area  of 
research  interest,  the  focus  of  this  paper  is  on  the  implications  and  potential  effects  of  the  widespread  and 
increased  adoption  of  CMC  in  academic  research.  Discussion  of  these  is  based  on  the  outcomes  of  a pilot  study 
undertaken  in  the  spring  and  early  summer  of  1998  as  a first  investigation  of  how  UK  academics  perceive  the 
use  of  CMC  in  their  research  work  (Fung,  1998). 

The  term  CMC  covers  the  transmission  of  computer  mediated  data  over  a wide  range  of  communication 
channels,  from  computer  text,  through  audio  to  visual  and  possible  combinations  of  each.  In  this  pilot  study 
the  term  was  used  with  reference  to  electronic  mail.  This  particular  means  of  CMC  was  chosen  for 
investigation  because  it  is  the  medium  which  has  been  available  to  academics  over  the  longest  period  to  date 
and  is  arguably  still  the  medium  most  widely  available  to  academics  in  the  UK.  This  being  so,  looking  at  the 
use  of  email  in  academic  research  seemed  a promising  way  of  beginning  to  track  any  changes  in  approaches  to 
research  due  to  the  influence  of  CMC. 

In  the  following  sections  of  this  paper,  a brief  description  is  given  of  the  study  that  was  undertaken.  The 
outcomes  of  this  study  are  then  used  as  the  basis  for  a discussion  of  email  as  a research  tool  and  areas  of 
change  in  academic  research  which  the  increased  use  of  email  may  influence.  The  paper  concludes  by 
summarising  what  participants  in  the  study  saw  as  the  most  effective  use  of  this  type  of  CMC  and  aspects  of  its 
use  which  merit  further  research. 
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Overview  of  a Pilot  Study  and  Outcomes 


The  aims  of  the  pilot  study  briefly  overviewed  in  this  section  were  threefold.  They  were  to  gather  the 
perceptions  of  academics  concerning  the  use  of  CMC  in  research;  to  identify  any  areas  of  change  in 
approaches  to  academic  research  brought  about  by  its  use  and  to  make  a first  assessment  of  its  perceived 
effectiveness  as  a research  tool.  Six  academic  research  groups  were  approached  and  invited  to  participate.  Four 
of  these  groups  are  based  at  the  Open  University,  Milton  Keynes,  UK,  while  one  is  based  at  Leeds  University, 
UK  and  one  at  the  University  of  Surrey,  UK. 

Participation  in  the  study  was  voluntary  and  in  each  of  the  six  research  groups  taking  part  in  the  pilot  study, 
email  facilities  were  well  established  and  available  to  group  members.  A self-completion  survey  was 
distributed  to  members  of  each  participating  group  and  seven  participants  took  part  in  individual  face  to  face 
interviews,  during  which  aspects  of  their  use  of  CMC  were  explored  in  more  depth. 


A Broad  Picture 

The  purpose  of  the  self  completion  questionnaire  was  to  establish  a picture  of  email  usage  among  the 
researchers  surveyed,  their  perceptions  of  its  use  in  research,  the  advantages  which  they  saw  this  technology 
brought  to  their  research  and  what  they  saw  as  problematic  in  its  use.  In  total  84  researchers  were  surveyed 
and  completed  questionnaires  were  returned  by  54,  representing  a response  rate  of  approximately  64%. 

In  establishing  email  usage,  responses  showed  that,  as  expected,  most  participants  had  easy  access  to  email  at 
work,  though  there  were  two  exceptions.  Most  used  it  on  a daily  basis,  though  again,  relating  to  access,  there 
were  two  exceptions,  both  of  whom  used  it  less  frequently.  In  all  groups  surveyed  responses  indicated  that  the 
bulk  of  administrative  messages  in  their  institutions  came  via  email. 

In  investigating  the  role  that  email  played  in  participants’  communications  with  colleagues,  over  80% 
indicated  that  the  statement  T usually  use  email  rather  than  the  phone  to  contact  colleagues’  was  true  for 
them,  while  only  7%  agreed  with  the  statement  ‘normally  the  phone  is  my  first  choice  for  contacting 

Participants  were  asked  to  rate  their  perceptions  of  the  use  of  email  in  helping  to  establish  research  contacts 
‘that  they  might  not  otherwise  have  made’.  Broadly  speaking,  the  role  of  email  in  making  research  contacts 
that  would  not  otherwise  have  been  made,  was  seen  as  ‘definitely’  helpful.  At  an  international  level,  by  33 
(61%  of  respondents),  at  a national  level  by  31(57%  of  respondents)  and  within  their  own  institutions  by  19 
(35%),  i.e.  it  was  seen  to  be  more  instrumental  in  establishing  research  contacts  in  proportion  to  the 
geographical  distance  of  potential  contacts. 

Perceptions  of  the  usefulness  of  contacts  which  had  been  made  in  this  way  followed  the  same  pattern  in  that 
those  made  at  an  international  level  were  seen  as  ‘extremely’  useful  by  21  (39%  of  respondents),  rating  higher 
than  the  usefulness  of  email-instigated  research  contacts  at  national  and  institutional  levels. 

A common  feature  of  many  email  systems  is  the  facility  to  subscribe  to  groups  or  lists  of  users  with  specialist 
interests  and  to  participate  in  discussions  within  those  groups.  The  majority  of  respondents  subscribed  to  lists 
or  conferences  while  there  were  relatively  few  who  did  not  subscribe  to  any.  Reasons  given  for  not  subscribing 
to  lists  or  conferences  were  lack  of  time  to  ‘get  round  to  it’,  given  by  one  respondent  and  a feeling  that  the 
contents  rarely  proved  to  be  useful,  given  by  four  respondents. 

The  majority  of  those  surveyed  only  skimmed  messages  from  lists  or  conferences  (61%)  and  rarely  participated 
directly  in  discussions  or  information  exchange.  For  many,  however,  59%  of  respondents,  an  important  use  of 
list  or  conference  subscription  seemed  to  be  one  of  information  dissemination,  redirecting  any  messages 
considered  as  being  of  interest  to  other  colleagues. 

Of  the  46  respondents  who  did  subscribe  to  email  lists  or  conferences,  9 people,  almost  20%,  indicated  that 
they  did  not  find  them  useful.  An  explanation  given  by  three  of  those  respondents  subscribing  to  email  lists  or 
conferences  but  reporting  that  they  did  not  find  them  very  useful,  was  ‘a  lack  of  time  to  read  the  contents’. 
This  lack  of  time  was  seen  as  current  but  not  meriting  unsubscribing.  These  respondents  shared  the  view  of 
another  two  respondents  who  did  not  find  ‘lists’  very  useful  but  still  subscribed  to  them  in  the  hope  ‘ that  one 
day  a really  useful  message  that  was  highly  relevant  would  arrive’  and  that  this  made  continued  subscription 
worth  while. 
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Participants  were  asked  what  importance  they  attached  to  email  as  a research  tool  at  three  stages  of  their 
research,  i.e.,  planning,  information  gathering  and  dissemination.  Overall  its  role  was  seen  by  respondents  as 
most  important  at  the  information  gathering  stage  (39  respondents  - 72%),  less  so  at  the  stage  of  dissemination 
(33  respondents  - 61%)  and  least  important  at  the  stage  of  planning  research  (30  respondents  - 55%). 
Nevertheless  39  respondents,  i.e.  over  70%,  felt  that  email  had  changed  how  they  planned  and  carried  out  their 
research.  In  addition  to  that  percentage,  six  respondents  (13%)  indicated  that  email  had  not  changed  how  they 
carried  out  their  research,  since  email  had  always  been  available  to  them  and  had  always  been  used  by  them  as 
a research  tool. 

Responses  given  to  the  question  ‘What  sort  of  differences  do  you  feel  email  has  made  to  the  way  you  carry  out 
your  research?’  showed  that  the  principal  ways  in  which  respondents  felt  that  email  appeared  to  have  made  a 
difference.  These  were  that;  it  had  helped  them  widen  and  increase  the  number  of  their  research  contacts; 
assisted  academic  collaboration  and  was  a practical  help  in  planning  and  implementing  their  research.  The 
category  of ‘other’  ways  in  which  email  had  made  a difference  included  two  threads.  In  one,  three  respondents 
noted  the  way  in  which  email  had  changed  the  pattern  of  relationships  between  researchers  as  supervisors  and 
their  research  students.  In  the  second,  two  respondents  commented  that  requests  for  information  and  replies  to 
these  requests  could  be  sent  off  with  too  little  thought  so  possibly  encouraging  a ‘lazy’  attitude  to  research. 
Participants  were  asked  ‘What  do  you  see  as  the  biggest  disadvantage  of  email  as  a research  tool?’  and  ‘What 

Of  the  50  participants  who  replied  to  the  question  ‘What  disadvantages  do  you  see?’,  10  respondents  (20%), 
saw  no  disadvantages,  while  from  the  other  40  responses,  it  is  clear  that  the  sheer  volume  of  information  and 
communication  is  seen  as  the  biggest  disadvantage. 

Replies  showed  that  the  speed  and  immediacy  of  email  and  the  scope  it  allows  for  widening  research  contacts 
is  clearly  viewed  by  most  (27  participants  - 50%)  as  its  biggest  advantage,  followed  its  use  as  an  information 
resource.  Its  use  in  establishing  collaborative  research  and  in  facilitating  document  exchange  as  a part  of 
collaboration  and  the  benefits  of  its  asynchronicity  were  noted  by  9 respondents  (17%). 


In  Summary 

This  broad  picture  of  the  54  researchers  responding  to  the  questionnaire  shows  that  while  all  but  two  have  easy 
access  to  email  on  campus,  60%  see  it  as  useful  for  making  and  establishing  research  contacts  at  an 
international  level.  70%  feel  it  has  changed  the  way  they  carry  out  their  research  and  85%  of  those  replying 
subscribe  to  email  lists  or  conferences  (though  the  principal  activity  in  relation  to  these  is  to  forward  items  to 
colleagues).  The  biggest  advantage  of  using  email  as  a research  tool  is  seen  to  be  its  speed  and  immediacy,  the 
biggest  disadvantage  is  felt  to  be  the  volume  of  communications  that  it  generates.  Looking  in  a little  more 
detail  at  the  data  which  contributed  to  this  broad  picture,  however,  showed  some  responses  which  were  not 
quite  so  in  line  with  what  might  have  been  expected  and  which  indicated  areas  that  would  be  interesting  to 
probe  further.  To  some  extent  this  was  done  within  the  framework  of  the  personal  interviewing  which  took 
place.  In  other  areas  it  was  change  which  is  not  taking  place  which  was  of  note  and  this  too  will  be  discussed 
in  the  following  section. 

Discussion 

As  one  participant  did,  it  is  perfectly  legitimate  to  ask  the  question  ‘what  does  it  mean,  to  use  email  as  a 
research  tool?’  Within  the  context  of  this  study,  the  answer  to  that  question  is  ‘to  use  email  to  facilitate  the 
processes  involved  in  carrying  out  academic  research’.  All  six  groups  who  participated  in  the  pilot  have  good 
email  facilities  and  all  are  active  research  groups.  As  results  showed,  using  email  in  academic  research  work, 
as  a research  tool,  is  well  embedded  practice  within  these  communities.  The  majority  of  participants  (82%) 
recorded  that  they  usually  use  email  rather  than  phone  to  contact  their  colleagues.  Over  75%  find  it  useful  for 
making  research  contacts,  approximately  70%  feel  that  email  has  to  some  extent  changed  how  they  undertake 
and  carry  out  their  research,  while  another  13%  have  always  used  email  in  their  research  and  as  one 
researcher  noted,  ‘couldn’t  imagine  doing  research  without  it’. 
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In  most  areas  of  questioning  the  results  given  in  the  previous  section  were  similar  across  all  groups.  Since  the 
groups  participating  in  this  pilot  were  relatively  small  in  number,  comparisons  among  groups  could,  in  any 
event,  only  be  made  on  a tentative  basis.  In  practice,  there  were  only  two  areas  in  which  different  ‘group’ 
responses  might  be  interesting  to  consider  further. 

The  first  related  to  the  preference  for  email  contact  with  colleagues.  Here,  within  a humanities  research  group, 
there  appeared  to  be  a much  lower  preference  for  email  as  first  choice  of  contact  with  colleagues.  Only  just 
over  half  of  respondents  in  this  group  (5  out  of  9)  agreed  that  they  usually  used  email  rather  than  phone  to 
contact  colleagues,  the  lowest  proportion  in  all  groups.  It  is  possible  that  this  may  be  an  indication  of  a 
discipline  difference  in  adoption  of  CMC,  as  suggested  by  (Alsop  et  al  1 997),  but  would  obviously  need  to  be 
investigated  with  larger  samples. 

The  second  apparent  difference  between  groups  occurred  in  importance  given  to  the  role  of  email  in  research 
dissemination.  Here,  in  one  group,  7 out  of  11  (over  60%)  saw  email  as  of  major  importance  at  the 
dissemination  stage  of  their  research,  in  comparison  with  a much  lower  proportion  in  all  other  groups.  This, 
too,  may  relate  to  a ‘discipline’  difference,  since  many  of  this  particular  group’s  activities  focus  on 
development  of  communication  technologies.  However,  though  they  may  appear  interesting,  differences  within 
groups  of  such  small  numbers  have  to  be  treated  with  caution  and  can  only  serve  to  highlight  areas  of  further 
investigation. 

These  possible  group  differences  apart,  overall  the  outcomes  indicate  that  these  researchers  see  email  as  an 
essential  and  integral  part  of  research  work.  The  importance  of  email  in  making  research  contacts  is  seen  as 
considerable  and  outcomes  indicate  that  it  does  facilitate  research  processes  and  is  seen  as  having  effect  on  the 
processes  involved  in  research. 

Some  of  those  effects  are  unsurprising.  In  many  ways  the  role  that  email  plays  in  facilitating  contacts  is  simply 
to  do  what  the  phone  and  the  letter  previously  did,  but  more  efficiently  and  faster.  To  recall  those  advantages, 
asynchronicity  removes  an  element  of  uncertainty  that  exists  with  phone  calls.  Messages  may  be  composed, 
revised  and  sent,  regardless  of  the  recipient’s  presence  at  the  expected  location  and  can  contain  a detail  and 
amount  of  information  that  would  be  impractical  to  leave  on  an  answering  machine.  Messages  that  have  the 
equivalent  status  of  hard  mail  can  be  sent  over  distances  and  at  speeds  that  could  not  be  matched  by  traditional 
postal  services.  The  effect  on  the  research  process  appears  to  be  one  of  speeding  up  and  widening  the  scope  of 
research  contacts. 

Similarly,  some  effects  on  research  methodologies  employed  in  the  research  process  can  be  viewed  as  allowing 
the  same  processes  to  be  carried  out  with  more  efficiency  and  to  reach  a wider  audience.  Respondents 
mentioned  the  fact  that,  as  noted  by  (Foster,  1994),  research  surveys  can  be  carried  out  with  one  electronic 
posting  and  qualitative  data  obtained  via  email  is  by  nature  of  the  medium  ‘on-line’,  thus  eliminating  the  need 
for  transcription.  Only  one  participant  in  the  study,  however,  reported  having  used  email  as  a means  of 
carrying  out  research  surveys.  This  may  relate  to  the  relatively  small  number  of  participants  whose  research 
would  encompass  that  research  method.  It  may  also  relate  to  the  fact  that  within  the  UK,  as  yet,  for  other  than 
very  specifically  targeted  populations,  survey  sampling  using  this  medium  would  be  affected  by  level  of  email 
access  and  usage  (Selwyn  & Robson,  1998). 

Since  inevitably  the  ease  and  breadth  of  transmission  result  in  more  research  surveys  reaching  more  people,  no 
less  surprising  is  the  disadvantage,  noted  by  two  respondents,  of  more  people  receiving  more  unwanted  surveys 
via  email. 

While  the  effects  mentioned  above  could  be  classed  as  unsurprising,  there  were  two  areas  in  which  results 
from  the  pilot  study  indicated  effects  on  research  processes  that  seemed  more  surprising. 

One  of  these  was  the  effect  of  email  on  the  process  of  postgraduate  research  training.  Research  into  the  use  of 
email  in  teaching  and  learning  at  undergraduate  level  is  not  new  (Wild  & Winniford,  1993;  Pitt,  1996)  and  is 
ongoing,  particularly  in  institutions  where  distance  education  is  the  principal  teaching  mode,  as  in  the  Open 
University,  UK.  Less  researched  is  the  effect  of  using  email  in  postgraduate  research  training.  An  essential  and 
integral  part  of  practising  research  is  sharing  and  passing  on  research  knowledge  and  knowledge  of  research 
methods  from  skilled  to  apprentice  researchers.  This  is  principally  achieved  in  the  tutor/supervisor  relationship 
and  by  research  training  courses  offered  as  part  of  the  ‘research  training’  package  to  postgraduate  student 
researchers. 

Communication  between  supervisor  and  student  researcher,  i.e.  mature  and  apprentice  researcher,  plays  an 
important  part  in  this  process  and  it  is  here  that  email  may  be  bringing  about  a subtle  change.  Responses  from 
participants  indicated  that  one  of  the  advantages  of  email  was  improving  research  student/supervisor 
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communications.  This  was  investigated  further  during  interviews  with  seven  of  the  participants.  Discussion 
with  both  mature  and  apprentice  researchers  suggested  that  the  use  of  email  could  be  instrumental  in  bringing 
about  change  in  the  nature  of  the  student/supervisor  relationship. 

At  base  this  seems  to  relate  to  the  asynchronicity  of  email  and  to  the  perceived  greater  informality  of  the 
medium.  At  one  level,  email  means  that  student  and  supervisor  are  able  to  be  in  more  frequent  contact  than 
would  traditionally  be  practical.  The  asynchronicity  of  email  allows  messages  to  be  sent,  replied  to,  without 
imposing  or  requiring  a ‘physical’  presence  as  would  happen  if  the  question/information  was  delivered  in 
person  or  conveyed  via  a phone  call.  At  another  level,  interviewees  referred  to  the  ease  with  which  they  felt 
they  could  communicate  by  email  within  this  student/supervisor  relationship.  This  suggests  that,  perhaps,  the 
perceived  informality  of  the  medium  may  encourage  a more  spontaneous  communication  between  supervisor 
and  student  than  would  traditionally  exist.  To  quote  one  interviewee  ‘the  relationship  is  more  like  one  of 

This  perceived  informality  is  also  shown  in  the  way  in  which  respondents  felt  that  it  was  an  ‘easy’  way  of 
making  contacts  with  other  researchers.  No  relevant  interviewees  spoke  explicitly  of  ‘training’  students  to  use 
email  to  approach  and  make  contact  with  leading  experts  in  their  research  field.  In  two  cases,  however,  it  was 
clear  that  it  was  expected  of  students  that  they  would  do  so.  Students  were  encouraged  to  read,  as  part  of 
research  training,  material  which  summarises  and  explains  to  apprentice  researchers  how  best  to  make  use  of 
email’s  potential  in  making  research  contacts  (Agre,  1997). 

The  other  area  in  which,  from  the  survey  responses,  email  appears  to  be  effecting  a change  of  approach  is  in 
academic  research  writing.  Changes  email  had  brought  about  in  collaborative  academic  writing  were 
mentioned  both  in  replies  to  the  questionnaire  and  in  discussion  in  interviews.  Changes  were  principally  seen 
as  positive,  though  some  misgivings  were  expressed. 

Email  was  seen  not  only  as  widening  the  scope  for  collaborative  writing,  but  also  as  a way  of  allowing  a 
greater  freedom  in  collaboration,  since  documents  could  be  so  easily  exchanged,  amended  and  added  to  among 
collaborators.  One  interviewee  expressed  it  as  the  opportunity  to  incorporate  ideas  and  revise  documents  in  a 
way  that  would  have  been  impractical  before  electronic  communications  were  available. 

It  was  also  noted  that  using  email  to  transfer  documents  and  to  discuss  current  writing  meant  that  it  was  easier 
to  obtain  peer  feedback  more  easily  at  an  earlier  stage  and  on  a wider  basis,  thus  making  a positive 
contribution  to  the  quality  of  papers. 

An  aspect  of  this  greater  freedom  afforded  by  the  speed  and  ease  of  document  exchange  and  editing,  however, 
that  aroused  some  misgivings  from  another  interviewee,  was  the  fact  that  this  could  and  did  mean  that 
sometimes  collaborative  research  writing  could  be  more  difficult  to  bring  to  closure. 

It  appears  from  the  pilot  survey  that  it  is  in  these  two  areas,  research  apprenticeship  and  in  academic  writing, 
that  email  seems  to  be  bringing  about  or  have  brought  about,  changes  in  research  practice. 

In  contrast,  there  appears  to  be  one  change,  which  email  might  have  been  expected  to  bring  about,  but  which, 
according  to  data  from  the  pilot,  it  has  not.  List  and  conference  membership,  while  very  common,  does  not 
seem,  for  the  majority  of  respondents,  to  have  instigated  its  frequent  use  for  scholarly  discussion.  Work  by 
(Berge  & Collins,  1 995)  indicates  that  CMC  has  great  potential  for  promoting  scholarly  discussion,  but,  with 
one  exception,  this  was  not  apparent  from  the  pilot  data.  Rather  than  an  opportunity  for  increased  scholarly 
discussion  among  specialist  groups,  participants  in  the  pilot  appear  principally  to  use  these  groups  as 
‘information  sources’.  Responses  indicate  that  the  majority  of  participants  see  access  to  lists  and  to  conferences 
as  an  indirect  means  of  disseminating  research  information.  Certainly,  in  one  sense  this  is  contributing  to 
scholarly  discussion,  but  not  in  the  direct  way  that  one  might  have  supposed. 


Conclusion 

Considering  the  outcomes  discussed  in  the  previous  sections,  there  are  aspects  of  email  use  that  have 
‘changed’  the  way  in  which  research  is  carried  out.  In  some  areas  this  change  is  simply  one  of  moving  to  a 
medium  that  facilitates  faster  communications  and  allows  more  widespread  and  varied  research  contacts  than 
was  previously  possible  using  paper-based  and  telephone  communications.  This  type  of  change  will  continue, 
as  the  technology  develops  and  is  seen  as  positive  and  enabling.  A number  of  comments  were  made  about  the 
problems  of  ‘incompatibility’  of  mail  systems,  the  difficulties  of  attaching  documents,  of  downloading 
documents  and  of  the  ‘overload’  of  information.  To  increase  the  effectiveness  of  email  as  a research  tool  these 
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issues  must  be  addressed  and  given  time  these  technical  ‘challenges’  will  no  doubt  be  resolved.  Academics  too, 
will  presumably  in  time  become  more  adept  and  at  home  with  the  technology,  employing  information 
‘management’  techniques  and  making  more  use  of  information  ‘filters’  as  they  become  available,  to  maximise 
the  efficient  use  of  CMC  in  their  research. 

In  other  areas,  changes  being  brought  about  by  CMC  could  be  more  significant  since  data  indicates  that  the 
use  of  email  may  well  be  influencing  the  ways  in  which  research  is  planned  and  carried  out.  In  the  study 
reported  here,  email  has  been  taken  as  the  medium  to  investigate  the  use  of  CMC  as  a research  tool.  There  are 
possible  indications  (and  given  the  scope  of  a pilot  investigation  these  can  only  be  interpreted  as  possible 
indications),  that  its  use  is  influencing  the  researcher  / student  researcher  relationship,  the  process  of  research 
apprenticeship  and  that  it  could  be  contributing  to  change  in  the  process  of  academic  research  writing.  If  this 
is  so,  then  the  increased  use  of  CMC,  widening  access  to  the  Internet  and  greater  availability  of  new  computer 
based  technologies  is  likely  to  hasten  and  highlight  those  changes.  The  pilot  study  undertaken  indicates  that 
these  are  areas  that  merit  investigation  in  more  detail  and  on  a wider  basis.  Doing  so  would  contribute  to 
incorporating  the  most  positive  aspects  of  change  that  CMC  is  bringing  about  or  likely  to  bring  about  in 
academic  research. 
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Abstract 


The  demand  of  developing  educational  multimedia  systems  has  dramatically  increased. 

Especially  web-based  hypermedia  systems  can  be  valuable  for  the  public  to  use.  However 
it  is  very  complicated  to  develop  large  hypermedia  web-based  systems  with  big  capacity. 

This  paper  will  present  math^ed^ology”^^  system,  an  interactive,  hypermedia,  web-based 
professional  development  program,  I will  also  explain  instructional,  technical  and 
interface  design  as  well  as  the  implementation  of  math*ed*ology^^  system  from  a 
developing  and  programming  point  of  view, 

1 propose  the  ejfective  method  of  programming  for  large  hypermedia  systems  to 
reproduce  using  basic  templates  with  naming  conventions  and  alias  (dummy)  media  files 
which  contains  file  pointers  only.  And  also  evaluation  of  the  math^ed^ology”^^  system  is 
given  by  using  Gagne's  Nine  Events  of  Instruction, 

1.  Introduction 

The  demand  of  developing  educational  multimedia  systems  has  increased  enormously  recently.  Especially  web- 
based  hypermedia  systems  can  be  valuable  programs  for  the  public  to  use.  However  it  is  very  complicated  to 
develop  large  hypermedia  systems  including  all  the  different  types  of  media  files  such  as  text,  image,  graphics, 
audio  and  video.  Many  program  developers  have  asked  themselves  "Which  developmental  tool  is  the  best  if  the  size 
of  the  multimedia  systems  is  large?"  and  "What  kind  of  developing  method  of  huge  size  of  hypermedia  systems  is 
the  most  effective?"  and  "How  can  we  evaluate  the  systems  once  they  are  developed?" 

We  had  the  same  types  of  questions  at  the  beginning  of  math*ed*ology’^'^,  which  is  an  interactive,  multimedia,  web- 
based  professional  development  program,  at  the  developmental  stage,  TBLR  (Technology  Based  Learning  & 
Research  at  the  Arizona  State  University,  Educational  Media  8l  Computer  Department)  team  made  an  effort  for 
more  than  two  years  to  develop  math*ed*ology'^^.  This  paper  will  highlight  the  following  points  from  the  vivid 
developing  experience.  Section  2 will  present  overall  constitutions  of  math*ed*ology^^  system,  I explain 
instructional  design,  technical  design  and  interface  design.  In  section  3,  I present  implementation  of 
math^ed^ology”^^  system  from  development  and  programming  point  of  view.  Section  4 shows  evaluation  of  the 
math^ed^ology”^^  system  is  according  to  Gagne's  Nine  Events  of  Instruction,  Nine  events  are  the  follows:  Informing 
the  learner  of  the  objective,  presenting  the  stimulus  material,  providing  learning  guidance,  eliciting  performance, 
providing  feedback,  assessing  performance,  enhancing  retention  and  transfer,  stimulating  the  recall  of  pre  required 
sites  capabilities,  and  gaining  the  user  attention[l].  Section  5,  presents  a conclusion  and  future  work.  The  full 
twenty-five  lessons  of  math*ed*ology™  are  now  available  on  the  web  site  http://mathedology.ed.asu.edu. 

2.  Design  of  math*ed*ology^M  system 

math^ed^ology”^^  is  an  interactive,  multimedia,  web-based  professional  development  program.  It  is  designed  to 
increase  student  achievement  by  increasing  educators'  knowledge  of  the  NCTM(National  Council  of  Teachers  of 
Mathematics  ) discourse  and  curriculum  standards,  as  well  as,  bilingual  and  ESL  strategies.  The  user  is  identified  as 
in-service  and  pre-service  mathematics  teachers,  staff  development  personnel,  and  professors  of  mathematics 
methodology  for  pre-service  teachers.  The  program  is  a comprehensive  resource  that  includes  a database  of  twenty- 
five  elementary  school  mathematics  lessons  on  a variety  of  topics,  'Solving  an  array  problems',  'Symmetry', 
'Multiples  with  Calculators',  and  'Perimeter  Search'  are  the  examples  of  the  lesson  titles.  These  lessons  are  taught  in 
English,  Spanish  and  bilingual  formats. 

The  lesson  provides  a rich  environment  within  which  educators  can  see  live  lesson  video  of  actual  classroom 
instruction,  and  are  accompanied  by  analytic  commentary  from:  Mathematicians,  Mathematics  Educators,  Language 
Specialists  and  Classroom  Teachers,  Each  lesson  is  preceded  by  a set  of  focus  questions,  a list  of  user  outcomes,  a 
description  of  the  setting  and  context,  a lesson  plan,  language  and  teaching  strategies,  and  applicable  discourse  and 
curriculum  standards,  A section  on  mathematical  concepts  accompanies  the  lessons  and  details  the  concepts 
addressed  in  the  lesson.  And  also  illustrative  animations  are  given  to  help  understand  concepts.  Language 
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Environment  explains  how  the  classroom  students  constitute  in  terms  of  the  language  they  used.  A pre-interview  and 
post-interview  with  the  classroom  teacher  provides  insight  into  the  perspective  of  the  actual  classroom  instructor. 
The  teacher  describes  expectations  and  objectives  for  the  students  and  reflects  on  what  has  happened  and  whether 
the  expectations  and  objectives  were  met. 

The  sitemap  section  is  a graphic  overview  of  the  program’s  overall  organizations.  Users  can  click  on  any  item  to 
directly  navigate  to  a particular  content  area  they  want  to  jump.  The  Help  section  provides  users  with  a brief 
overview  of  the  interface  of  the  math’ed’ology”^^  project  and  offers  trouble-shooting  advice.  A chat  section,  which 
is  online  discussion  forum,  allows  user  to  consult  with  experts  and  fellow  teachers.  A notes  section  provides  users 
with  an  opportunity  to  record  thoughts  and  ideas.  It  is  recommended  that  users  carefully  consider  each  question  and 
then  open  their  favorite  text  editor  (such  as  Microsoft  Word”^^,  WordPerfect^,  Notepad,  WinWord™,  or 
SimpleText,  etc.)  and  write  down  and  save  their  comments  there.  A complete  listing  of  NCTM  Curriculum 
Standards  and  Professional  Teaching  Standards,  and  Bilingual  resources  are  included  for  reference.  An  Assessment 
section  allows  the  user  to  practice  identifying  curriculum  strategies  and  identified  language  strategies,  reflect  and 
extend  section  on  the  lesson  taught,  and  consider  additional  methodologies  for  teaching  similar  lessons.  <Figure  1> 
shows  the  sitemap  of  math*ed*ology™. 
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<Figure  1>  Sitemap  of  math*ed*ology™ 

3.  Implementation  of  math*ed*oIogy™ 

The  web  browser  has  become  a uniform  interface  for  users  to  archive  multimedia  information  over  the  internet. 
The  convenient  uniform  user  interfaces  is  a major  characteristic  of  the  web.  To  enhance  the  functionality  of  the  web 
browser,  it  is  definitely  worthwhile  to  incorporate  the  visual  programming  into  the  browser.  The  visual 
programming  tool  and  Frontpage  Explorer  & Editor  are  useful  for  this  purpose  in  the  project.  Hence  the  system  is 
developed  as  an  embedded  module  to  reproduce  programs.  Some  multimedia  data  such  as  graphic,  animation,  video 
and  music  have  inherent  hierarchical  stnicture[3].  Multi-structure  information  is  useful  for  handling  hypermedia 
documents[4].  Basic  idea  of  dealing  with  thousands  of  audio  and  video  icons  for  huge  hypermedia  system 
efficiently  is  that  separating  generic  and  specific  audio  and  video  files  as  a first  step.  The  next  step  is  making  a 
basic  template  including  all  different  types  of  generic  text  and  image  icons  that  specify  image  source,  default 
hyperlink  location  for  audio  and  video  icons. 

To  make  basic  template  program,  we  use  Frontpage  Explorer  and  JavaScript.  On  the  top  header  frame  file,  we 
include  main  11  section  icons.  Home,  Sitemap,  Introduction,  Bilingual/ESL(English  as  a Second  Language), 
Lessons,  Help,  Chat  and  Notes.  Each  icon  has  hyperlink  so  users  can  navigate  any  time  they  want  to  make  jump.  All 
different  teacher’s  lesson  contents  are  in  the  Lesson  Section.  From  the  Lesson,  user  can  watch  all  twenty-five 
lessons’  database.  Each  lesson  has  overview,  part  1 through  part  3,  (some  of  the  lessons  have  2,4,5  or  6 lesson  parts), 
and  assessment.  Make  separate  teacher’s  image  folder  to  put  all  kinds  of  images  which  are  graphics,  classroom  pan 
image,  student  work  image  and  icon  images.  And  also,  make  separate  teacher’s  media  folder  to  put  all  audio  and 
video  files. 

The  followings  are  the  steps  to  prepare  audio  and  video  files.  Take  video  camera  for  the  real  classroom  teacher’s 
lesson.  From  this  analog  tape,  convert  it  to  digital  master  tape  using  AVID  and  MC  Express  system  software.  And 
export  video  files  to  avi  file  format.  Record  all  voice  over  of  the  teachers,  mathematicians,  mathematics  educators, 
and  language  specialist’s  commentaries  for  the  lessons.  Digitize  the  audio  and  export  to  wav  file  format.  Next  step  is 
using  SoundForge  to  edit  video  and  audio  files.  And  save  files  as  real  audio  and  video(rm  file  type).  To  make 
animation  files,  which  are  in  sub  menu  mathematical  concepts  on  overview  screen,  we  use  3D  Studio  Max. 

At  this  point  we  need  to  think  about  reproduce  the  application  with  different  contents  and  same  types  of  basic 
template  program.  It  is  necessary  for  developers  to  operate  multiple  programs  with  same  format  on  the  network.  An 
ideal  reproduction  system  must  allow  the  developers  to  manage  huge  size  of  programs  as  simple  as  possible.  It  is 
important  not  to  change  every  file  and  every  multimedia  icon  to  reproduce  the  application.  It  is  natural  to  build 
applications  on  the  top  of  the  web  client  to  maintain  the  same  interface  platform.  In  order  to  use  efficient  way  to 
reproduce  same  types  of  hypermedia  system,  it  is  recommended  to  take  naming  convention,  and  to  make 
dummy(alias)  media  files  which  have  only  file  pointers  pointing  the  location  where  real  audio  and  video  files 
exist[10]. 

<Figure  2>  shows  the  inherent  hierarchical  file  structures  to  reduce  efforts  when  we  reproduce  the  same  types  of 
applications. 
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FIGURE  NOT  AVAILABLE 


<Figure  2>  File  Structure  of  matlred-ology’*'^ 

The  summary  of  developing  stage  in  order  to  reproduce  application  program  is  as  follows: 

1)  Separate  files  into  general  (generic)  and  specific  audio.  Media  folder  has  plenty  of  audio  and  video  files, 
including  generic  and  specific  files.  Dummy  media  files  have  file  pointers  only.  To  reproduce  new 
lesson,  it  needs  to  change  the  real  audio  and  video  files  instead  of  changing  whole  html  files,  file 
properties  and  default  locations  of  every  audio  and  video  icons.  All  audio  and  video  icons  have  specific 
image  properties  from  generic  image  folder  in  the  template  program. 

2)  Use  the  same  naming  convention  to  make  file  names  for  all  audio  and  video  data. 

3)  Make  dummy  audio  and  video  file,  which  has  corresponding  file  pointers  using  same  naming  convention. 

4)  Change  only  real  audio  and  video  files  in  real  media  folder. 

Developers  do  not  have  to  change  all  image  properties,  audio  properties,  image  source,  or  image  location  at  all. 
Once  change  real  audio  and  video  files  in  media  folder,  that  is  all  developers  have  to  do  reproduce  multimedia 
application  which  has  the  same  types  of  format. 

4.  Evaluation  of  math»ed»ology'*^^ 

This  self-evaluation  of  math»ed»ology’*’^  is  completed  using  Gagne's  Nine  Events  of  Instruction.  It  is  based  upon  the 
survey  result  of  graduate  students  and  educational  professional  who  reviewed  math-ed-ology’*'^  several  times. 
math»ed*ology^^  was  reviewed  from  the  teacher  standpoint,  and  viewed  as  a teacher  learning  strategies  to  teach 
mathematics. 

1)  Informing  the  learner  of  the  objective:  The  objectives  can  be  found  throughout  the  program.  The  focus 
questions  help  to  bridge  the  user  from  learning  to  actually  doing.  The  user  can  create  their  own  lesson  plan  by 
following  the  focus  questions.  And  also  user-outcome  section  shows  what  the  user  will  learn  from  this  program. 

2)  Presenting  the  stimulus  material:  Through  the  use  of  video,  it  is  possible  to  learn  from  some  of  the  best  teachers, 
and  listen  commentary  from  the  experts  from  various  fields.  For  the  novice  teacher  can  watch  how  the  real 
classroom  will  it  be  and  what  will  the  student  reaction  be. 

3)  Providing  learning  guidance:  math*ed*ology'*'^  provides  guidance  through  the  use  of  real  classroom  video, 
transcripts,  and  all  different  kinds  of  commentaries  for  teachers  that  are  learning  to  teaching  methods  and 
strategies.  For  the  programming  point  of  view,  the  sitemap  section  shows  overall  view  of  this  program  and 
every  spot  has  hyperlink  to  corresponding  pages,  so  users  can  navigate  easily. 

4)  Eliciting  performance:  The  focus  questions  are  a way  to  elicit  performance.  The  teachers  are  asked  to  start 
thinking  about  the  materials  and  applying  it  to  their  work. 

5)  Providing  feedback:  There  is  no  feedback.  The  program  is  interactive  in  the  sense  that  user  can  push  buttons  to 
navigate  as  user  chooses,  but  there  is  no  real  feedback  given.  Using  chat  section  can  solve  this  problem.  The 
chat  section  is  online  discussion  forum  allows  user  to  consult  with  experts  and  fellow  teachers. 

6)  Assessing  performance:  The  assessment  is  a good  tool  to  see  what  the  learner  has  learned.  It  is  important  to  see 
what  the  user  is  able  to  identify  from  the  lesson,  reflect  on,  as  well  as  extend  the  knowledge,  and  applying  it. 
math^ed^ology”*”^  provides  the  assessment  section  includes  overview,  identify,  reflect,  extend  and  apply [8]. 

7)  Enhancing  retention  and  transfer:  The  assessment  provides  a forum  for  students  to  plan  different  activities  based 
on  the  information  that  they  have  learned.  For  the  technical  point  of  view,  one  lesson  is  completely 
programmed,  then  all  twenty-five  lessons  programmed  with  same  basic  templates.  It  gives  benefits  and  cost 
reduction  to  develop  effort  reduction  to  use  the  programs. 

8)  Stimulating  the  recall  of  pre  required  sites  capabilities:  Identified  curriculum  standards  and  discourse  standards 
are  given  with  complete  lists  of  those.  So  user  can  refer  to  write  new  lessons.  Users  are  asked  to  recall  the 
section  required  in  the  assessment  section. 

9)  Gaining  the  user  attention:  Overall,  graphic  icons,  live  classroom  video,  and  animation  are  enough  material 
gaining  user  attention[9]. 

Overall  math^ed^ology”*”^  gives  most  of  the  items  needed  except  feedback  section.  The  program  does  not  evaluate 
the  learner  performance  at  the  end,  so  the  user  does  not  know  if  the  objectives  were  achieved  and  to  what  degree. 
There  needs  to  be  a component  that  will  provide  feedback  to  the  user  on  how  well  s/he  did. 

5.  Conclusion 

The  demand  for  development  educational  hypermedia  system  is  remarkably  increasing  along  with  the  developing 
time  and  cost  because  the  size  of  multimedia  resource  files  is  huge.  In  this  paper,  I introduce  math^ed^ology”*”^ 
system  from  my  efforts  and  experience  when  I worked  on  the  project  for  developing  the  system.  I propose  a method, 
which  supports  not  only  dealing  with  a large  number  of  multimedia  files  efficiently,  but  also  reducing  developing 
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time  in  the  same  types  of  multimedia  application  programs.  The  main  contribution  of  this  paper  is  to  propose  the 
efficient  method  of  programming  for  large  educational  hypermedia  systems  to  reproduce  using  basic  templates  with 
naming  conventions  and  alias  (dummy)  media  files,  which  contain  file  pointers  only.  I present  self-evaluation  of 
math*ed*ology'^^  system,  so  other  developers  can  refer  to  the  view  on  the  designing  stage  on  their  own. 

The  math*ed*ology'^^  system  can  be  extended  with  new  contents  for  middle  school  and  high  school  mathematics 
education  materials.  A feedback  section  can  also  be  added  for  future  research  to  orient  the  user  on  the  system’s 
development  stage. 
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Abstract:  This  article  deals  with  the  use  of  computer-aided  tools  for  teaching  Operations 
Research/Management  Science.  We  describe  two  of  them,  ORWelt  and  ClipMOPS,  and  show 
how  we  use  them  for  teaching  purposes.  ORWelt  is  a hypermedia  learning  environment 
covering  the  subject  area  of  Operations  Research  on  an  introductory  level.  ClipMOPS  is  a 
dedicated  tool  for  modeling  and  solving  optimization  problems.  Students  can  use  these  tools 
in  addition  to  the  lecture.  The  long-term  goal  is  to  change  the  way  of  teaching  towards 
application-oriented  learning  while  basic  content  can  be  learned  with  ORWelt  and 
ClipMOPS. 


Introduction 

In  this  paper,  we  discuss  approaches  of  learning  the  interdisciplinary  subject  area  of  Management 
Science/Operations  Research.  Management  Science  (MS)  is  closely  related  to  Operations  Research  (OR)  and 
summarizes  the  application  of  the  modeling  approach  for  managerial  decision  support.  The  objective  of  OR  is  to 
find  practical  solutions  to  operational  problems  of  all  kinds,  often  within  tight  time  constraints.  The  models  are 
usually  built  using  quantitative  methods,  such  as  linear  programming,  mixed-integer  programming,  network 
models,  decision  analysis,  discrete  and  continuous  simulation,  queuing  models,  and  so  on.  OR  methods  often 
facilitate  an  in-depth  understanding  of  the  problem. 

Operations  Research  (OR)  is  a very  complex  subject  matter  to  teach.  As  indicated  above,  a wide  range  of 
quantitative  methods  is  applied  to  practical  problems.  Typically,  realistic  problems  are  often  unstructured,  it  is 
not  obvious  at  first  glance  which  method  is  appropriate.  Students  learning  the  subject  matter  are  not  perfectly 
prepared  to  handle  the  complicated,  often  messy  problem  situations.  The  usual  teaching  methods  such  as  lectures 
are  not  suited  to  transfer  the  problem  context  beside  teaching  basic  algorithms.  Moreover,  the  traditional,  method 
focused  way  of  teaching  management  science  does  not  leave  much  time  for  case  studies,  discussions,  and  student 
presentations  in  classroom. 

The  ability  to  apply  mathematical  modeling  techniques  can  be  trained  with  case  studies.  Then  methods  are  bound 
to  a situated  context  and  students  have  to  analyze  the  problem  and  find  out  which  method  should  be  applied  in 
which  way.  This  requires  far  more  elaboration  than  merely  applying  a given  algorithm  on  a clearly  defined 
problem.  In  our  point  of  view,  learning  emerges  from  an  active  construction  process  and  knowledge  must  be 
integrated  in  the  learner’s  mental  models  to  make  it  accessible  in  the  long  run.  This  process  can  be  supported  by 
situated  learning,  e.g.  by  discussing  case  studies.  Moreover,  situated  learning  is  more  fun,  thus  increasing 
students’  motivation  to  learn  because  they  can  feel  an  immediate  success  after  constructing  a solution 
themselves.  In  most  cases  it  is  not  possible  to  present  rich  case  studies  in  a lecture  due  to  time  limits. 
Furthermore,  because  there  are  often  more  than  one  hundred  students  with  varying  backgrounds  and  interests 
(majoring  in  business  studies,  business  computing,  industrial  engineering,  computer  science,  or  business 
education)  in  a university  class,  it  is  not  possible  to  adjust  the  pace  so  that  each  student  can  easily  follow  a 
teacher-centered  lecture. 

These  were  the  reasons  why  we  decided  to  provide  students  with  a tool  which  helps  them  to  learn  quantitative 
methods  also  outside  the  classroom  at  their  own  pace,  thus  leaving  more  time  for  discussions,  presentations  and 
teamwork  during  the  classes.  We  believe  that  an  interactive,  computer  based  tool  is  better  suited  to  achieve  this 
goal  than  a conventional  textbook. 
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Our  thesis  is  that  traditional  lectures  can  be  enhanced  but  not  substituted  by  computer  aided  tools.  In  the 
subsequent  sections  of  this  paper  we  present  two  approaches  on  how  this  can  be  done.  First  we  introduce  the 
hypermedia  learning  environment  ORWelt,  second  the  specialized  tool  ClipMOPS  which  serves  as  a modeling 
tool  for  optimization  models. 


ORWelt 

The  ORWelt  project  started  in  February  1996  with  the  intent  of  providing  a hypermedia  learning  system  that 
would  cover  all  standard  introductory  OR  material,  supplemented  by  tests.  Furthermore,  case  studies  were 
provided  in  order  to  support  different  approaches  to  the  same  content.  The  purpose  of  ORWelt  is  not  to  replace 
the  lecture  but  to  give  the  lecturer  more  time  to  discuss  real-world  case  studies  with  the  students.  We  do  not 
expect  that  computer  based  learning  approaches  will  replace  face-to-face  communication  in  real-time  between 
students  and  lecturer.  The  personal  contact  between  teacher  and  student  is  still  important  because  goals  such  as 
motivating  students  for  the  subject  matter,  i.e.,  providing  input  which  affects  the  student  in  a positive  manner, 
can  hardly  be  achieved  by  a computer-based  system  alone. 

Currently,  ORWelt  comprises  19  subject  and  15  test  modules  covering  the  main  subject  areas  of  MS/OR  on  an 
introductory  level.  Content  can  be  directly  accessed  through  a graphical  browser  or  the  learner  can  visit  a 
predefined  guided  tour.  Besides  standard  navigational  tools  such  as  footprints,  bookmarks  and  history,  guided 
tours  can  be  defined  in  order  to  avoid  the  Most  in  hyperspace’  effect.  The  contents  covered  by  ORWelt  are  shown 
in  (Fig.  1).  Shadowed  nodes  labeled  in  italics  denote  subject  modules  supplemented  with  test  modules.  In 
addition  to  the  subject  modules,  ORWelt  includes  test  components  to  allow  students  a self-assessment  of  their 
comprehension.  The  tests  are  usually  not  multiple  choice,  but  a wide  range  of  interactive  methods  is  used  in 
order  to  allow  the  exploration  of  the  test.  Our  goal  is  that  students  get  engaged  in  learning  instead  of  merely 
receiving  information. 


Figure  1:  Content  covered  by  ORWelt 


A key  feature  of  ORWelt  is  the  integration  of  the  professional  optimization  code  MOPS  ® (Mathematical 
optimization  System,  see  (Suhl  1994)).  MOPS  is  an  all  purpose  optimization  code  library  which  can  be  accessed 
from  many  programming  environments,  on  condition  that  Windows  Dynamic  Link  Libraries  (DLL)  are 
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supported.  The  utilization  of  MOPS  allows  the  user  to  solve  predefined  optimization  models  while  varying 
parameters  of  the  model.  The  solver  itself  resides  in  the  background,  and  the  output  has  not  necessarily  to  be 
formatted  as  a tabular  report.  It  is  left  up  to  the  developer  how  to  visualize  the  results.  Most  optimization  models 
in  ORWelt  visualize  the  resulting  solution  graphically,  for  example  using  pie  charts,  indicating  a way  to  interpret 
the  solution.  It  is  important  to  direct  the  learner’s  attention  to  the  critical  point  of  the  solution  shown.  Empirical 
studies  in  the  field  of  illustrations  indicate  that  general  instructions  to  pay  attention  to  a specific  illustration  are 
ineffective,  the  learner  needs  calculated  hints  pointing  out  the  important  aspects  (see  (Weidenmann  1997)). 
Transferred  to  the  field  of  OR,  graphical  visualization  offers  much  more  possibilities  to  direct  the  learner’s 
attention  into  a certain  direction  than  a simple  tabular  report.  This  way  OR  concepts  such  as  sensitivity  analysis 
can  be  explored  in  a very  vivid  way.  ‘Dynamic’  optimization  models  which  can  be  solved  on  the  fly  by  MOPS 
add  a new  quality  of  learning  and  teaching  that  cannot  be  achieved  by  means  of  a normal  textbook.  The  dynamic 
representation  allows  the  learner  to  interact  with  the  model:  he  or  she  can  alter  parameters,  compare  solutions 
and  explore  the  structure  of  the  problem  formulation. 

The  project  ORWelt  started  with  an  experimental  phase  followed  by  iterative  refinements  of  the  development 
process.  We  use  teams  of  typically  2-3  students  attending  a project  seminar  with  a total  of  2-6  groups  per 
semester.  The  seminar  is  supported  by  a group  of  two  knowledge  domain  specialists  and  one  or  two 
programming  specialists.  The  authoring  environment  we  use  is  Asymetrix  Toolbook®,  currently  version  6.1 
(Instructor  II),  Writing  hypertext  is  not  intuitive  and  requires  certain  skills  in  structuring  information.  On  the  one 
hand,  it  is  quite  demanding  for  students  to  develop  educational  material  while,  on  the  other  hand,  they  get  a veiy 
valuable  experience  out  of  it.  Intensive  coaching  of  students  is  necessaiy  but  it  is  possible  to  achieve  high  quality 
software.  See  (Blumstengel,  Kassanke  1998)  for  a detailed  description  of  the  development  model. 

At  the  moment,  ORWelt  is  a supplemental  offer  for  students,  they  can  use  it  but  they  do  not  have  to.  The  long- 
term goal  of  the  ORWelt  project  is  to  change  the  way  of  teaching  towards  application-oriented  learning 
(Blumstengel,  Kassanke,  Suhl  1997),  Lecture  time  should  be  used  more  efficiently  to  discuss  real-world 
problems  while  teams  of  students  learn  parts  of  the  material,  especially  basic  algorithms,  supported  by  the 
learning  environment.  ORWelt  can  also  be  used  as  a reference  and  for  exam  preparation.  For  a more  detailed 
description  of  ORWelt  see  (Blumstengel  1 998).  ORWelt  can  be  downloaded  at  (ORWelt  1 999). 

The  optimization  code  MOPS  is  also  used  in  the  next  example  ‘ClipMOPS’.  In  ORWelt,  students  have  the 
possibility  to  solve  optimization  models  in  the  so  called  MPS  format.  Students  can  use  ORWelt  as  a tool  to  solve 
self  formulated  models  without  having  to  worry  about  formulation  syntax  of  other  solvers  because  the  MPS 
format  is  an  industry  standard  for  optimization  models.  Although  they  can  solve  these  models  using  MOPS,  the 
model  representation  is  not  interactive  in  the  sense  of  direct  manipulation.  Modifying  a single  coefficient 
requires  altering  the  MPS  file  and  reloading  it.  In  ORWelt  it  was  not  possible  to  provide  an  interactive  model 
representation  due  to  technical  restrictions  of  Toolbook.  The  second  tool  discussed  here  is  dedicated  to  modeling 
purposes. 


ClipMOPS 

ClipMOPS  was  developed  in  close  cooperation  with  the  Free  University  of  Berlin  in  1998.  ClipMOPS  is 
intended  as  a students’  tool  for  modeling  optimization  models  and  was  introduced  to  students  in  Berlin  and 
Paderbom  in  the  winter  term  1998/99  for  the  first  time. 

Traditional  optimization  software  is  typically  run  by  a cryptic  command  language.  Students  cannot  fully 
concentrate  on  the  real  task  (modeling  an  optimization  problem)  but  have  to  cope  with  learning  a specific 
command  language.  Graphical  user  interfaces  are  not  very  common  in  this  field  of  software  and  an  easy-to-use 
software  was  not  available.  Currently  we  use  two  software  packages  - AMPL®  and  LINDO®  - in  teaching  OR. 
Both  are  based  on  the  DOS/Windows  operating  system  and  have  few  graphical  visualization  abilities. 
Furthermore,  AMPL  is  a powerful  but  not  easily  mastered  modeling  language.  AMPL  is  not  a beginner’s  tool, 
the  learner  has  to  have  a clear  understanding  of  the  model’s  structure.  The  problem  is  that  we  teach  a broad 
variety  of  students  majoring  in  diverse  studies.  They  usually  have  problems  to  use  the  optimization  software 
currently  used  due  to  their  different  mathematical  background  and  algorithmic  training.  Although,  the  Microsoft 
Excel®  solver  add-in  solves  optimization  models,  it  has  some  drawbacks.  Namely  defining  models  with  the 
solver  add-in  is  not  very  intuitive  and,  more  important,  the  resulting  solutions  provided  by  Excel’s  solver  are  not 
necessarily  reliable  (in  the  case  of  mixed  integer  programs  and  degenerate  solutions,  see  (Thiriez  1 998)). 
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We  wanted  a tool  which  encapsulates  the  formal  description  of  an  optimization  model  in  a graphical,  intuitive 
way  and  additionally  offers  reliable  solutions.  Due  to  these  reasons  we  considered  a spreadsheet  platform  to  be 
ideal  for  our  purposes.  At  this  point,  we  started  to  develop  a solution  ourselves  with  Excel  in  combination  with 
MOPS.  Excel  is  very  common,  most  of  our  students  use  the  Microsoft  Office  suite  for  word  processing  etc.,  thus 
students  have  a basic  understanding  of  how  to  operate  Excel  in  general  (e.g.  cell  editing).  Furthermore  Excel 
meets  the  technical  requirements  for  using  MOPS  (linking  Windows  DLLs). 

Optimization  models  can  be  represented  in  a two-dimensional  matrix  of  decision  variables  and  restrictions.  This 
matrix  can  be  directly  transformed  to  a spreadsheet  representation  of  columns  and  rows  where  columns  represent 
decision  variables  and  rows  are  shown  as  constraints.  Although  this  way  of  defining  a model  is  only  appropriate 
for  small  scale  models  and  does  not  reveal  the  model’s  structure,  students  seem  to  prefer  this  representation  (see 
(Fig.  2)).  Especially  beginners  favor  a matrix  representation  to  the  very  compact  mathematical  formulation  or  a 
model  definition  in  the  MPS  format,  respectively. 
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Figure  2;  ClipMOPS  model  representation  of  the  diet  problem  in  Microsoft  Excel 


ClipMOPS  has  an  easy-to-use  interface  and  allows  an  iterative  refinement  of  optimization  models.  This  way 
students  can  concentrate  on  modeling  the  optimization  problem  and  waste  no  time  to  figure  out  how  to  tell  the 
system  what  they  want  to  express.  The  model  size  is  restricted  to  a maximum  of  100  decision  variables  and  200 
restrictions  but  this  model  size  is  sufficient  for  teaching  purposes.  Existing  models  in  the  standardized  MPS 
format  can  be  imported  and  exported.  Additionally  the  seamless  integration  into  the  Excel  environment  allows  to 
make  use  of  Excel’s  built-in  graphical  abilities.  A student  version  of  ClipMOPS  can  be  downloaded  at 
(ClipMOPS  1998). 


Using  ORWelt  and  ClipMOPS  for  Teaching 

ORWelt  and  ClipMOPS  may  be  used  by  students  and  teachers,  as  well.  As  a teaching  resource  ORWelt  may  be 
used  by  the  lecturer  as  a high-level  visualization  tool,  e.g.,  to  visualize  algorithms  stepwise  and  to  give  further 
explanations  on  each  step.  This  way  he  or  she  can  attract  the  students’  attention  to  crucial  points  in  the 
animation.  Algorithms  are  dynamic  while  the  traditional  methods  used  to  teach  them  are  not.  An  animation 
accompanied  by  a lecturer’s  comments  fits  the  dynamic  nature  of  an  algorithm  much  better  and  adds  a 
presentation  quality  which  normally  cannot  be  achieved  by  traditional  means.  Although  developing  multimedia 
animations  is  expensive,  the  development  effort  for  additional  animations  decreases,  since  ORWelt  contains  a set 
of  all  purpose  animation  routines  which  is  permanently  enhanced  and  can  be  used  repeatedly.  The  presentation 
of  case  studies  can  be  enriched  by  sounds,  photos,  videos,  etc.,  thus  giving  a more  vivid  impression  on  what  the 
case  study  is  dealing  with.  Students  are  not  just  passively  receiving  but  actively  constructing  a solution 
themselves.  As  a hypermedia  environment  ORWelt  integrates  all  types  of  media  in  one  environment,  a laptop 
and  a projection  device  such  as  a video  beamer  or  LCD  panel  are  sufficient.  Additionally,  the  hypermedia 
structure  fits  the  manifold  relations  between  different  concerned  subject  areas  better  than  a linear  one. 

Students  profit  from  the  benefits  of  self-directed  learning  as  well.  They  can  choose  the  location  (where),  time 
(when),  and  duration  (how  long)  they  spend  with  the  learning  environment,  that  means  the  availability  of  the 
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learning  environment  is  enhanced,  there  are  no  fixed  times  or  locations  for  learning.  However,  to  accomplish  the 
engagement  of  students  in  the  learning  environment,  a simple  transformation  of  a book  to  a linear,  electronic 
form  is  not  sufficient.  To  achieve  this  goal,  one  has  to  use  interactive  elements,  e,g,,  simulations  or  dynamic 
optimization  models,  in  the  learning  environment.  The  degree  of  interactivity  is  a crucial  factor  for  the  success  of 
a hypermedia  learning  environment  (see  (Haack  1997,  p,  152,  Schulmeister  1996,  p,  388ff)),  Preliminaiy 
questionnaire  results  indicate  that  students  consider  interactivity,  animations  and  tests  to  be  the  most  important 
advantages  of  OR  Welt  compared  to  traditional  education  material. 

There  is  no  comfortable  way  to  define  optimization  models  within  ORWelt  (except  MPS  files),  this  gap  is  filled 
by  ClipMOPS,  We  consider  ClipMOPS  as  a high-level  tool  which  should  be  used  in  a way  similar  to  a 
calculator.  Our  students  are  required  to  gain  modeling  skills,  the  final  calculations  can  be  done  by  optimization 
software.  Although  it  is  important  to  know  how  optimization  models  can  be  solved  without  the  help  of 
computers,  no  one  will  try  to  solve  a problem  of  realistic  size  without  optimization  software  due  to  the  amount 
and  complexity  of  the  necessary  calculations.  The  computer  is  an  authentic  tool  and  should  already  be  used 
during  students’  education, 

ORWelt  and  ClipMOPS  will  be  extensively  employed  for  teaching  purposes  in  the  future.  Both  tools  have 
reached  a reliable  degree  of  stability  and  will  be  permanently  improved.  We  intend  to  add  a world  wide  web 
(WWW)  based  component  to  ORWelt,  thus  enabling  learners  to  access  it  over  the  internet.  The  long  term  goal  is 
to  change  the  way  of  teaching  towards  application-oriented  learning  in  the  lecture  while  basic  content  can  be 
learned  with  support  from  ORWelt,  Our  Students  need  OR  modeling  skills  to  tackle  the  problems  of  the  future, 
ORWelt  and  ClipMOPS  are  two  examples  how  we  try  to  improve  the  quality  of  teaching  OR, 
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Abstract:  In  the  process  of  learning  how  to  create  effective  computer-based  training  mod- 
ules for  accelerator  physics,  we  have  come  to  believe  that  simple  two-  and  three- 
dimensional  animations  are  often  the  most  valuable  multimedia  tool  for  presenting  techni- 
cal material.  This  talk  demonstrates  a number  of  the  animations  we  have  developed  for 
our  tutorial  series  Accelerators  and  Beams  and  discusses  why  we  think  they  are  success- 
ful. Examples  range  from  illustration  of  the  right-hand  rule  for  vector  cross  products  to 
demonstration  of  the  so-called  “vertical  betatron  oscillation”  that  is  used  for  ensuring  sta- 
ble beams  in  accelerators. 


Introduction 

Four  years  ago  WhistleSoft,  Inc.,  began  developing  a set  of  computer-based  multimedia  tutorials  for  accel- 
erator physics  under  a Small  Business  Innovation  Research  grant  from  the  U.S.  Department  of  Energy.  At  the 
time  we  began  this  project  our  expertise  was  not  in  multimedia  but  in  science.  Company  personnel  consists 
mainly  of  Ph.D.  scientists  who  were  at  one  time  associated  with  the  Los  Alamos  National  Laboratory.  We 
considered  ourselves  as  domain  experts  in  the  subjects  for  which  we  were  about  to  develop  tutorials.  On  the 
other  hand,  we  had  a lot  to  learn  about  user  interface  issues  and  how  to  create  multimedia.  Even  more  impor- 
tant, we  also  had  much  to  learn  about  instructional  design  and  pedagogy. 

We  originally  designed  the  Accelerators  and  Beams  tutorials  for  the  academic  market.'  However,  we 
have  always  had  the  idea  in  mind  that  we  could  later  customize  its  separate  modules  for  laboratory  and/or  in- 
dustrial usage.  This  would  extend  the  use  of  these  modules  to  a non-traditional  student  audience,  such  as  the 
technicians  and  operators  employed  at  large  accelerator  facilities.  We  are  therefore  targeting  a broad  audience 
- from  lower  undergraduates  and  technicians  up  to  graduate  students  and  professionals  in  science  and  engi- 
neering. Most  of  our  work  so  far,  however,  has  been  at  an  elementary  level,  for  end  users  who  have  taken,  say, 
an  algebra-  or  calculus-  based  introductory  physics  course. 

The  modules  comprising  Accelerators  and  Beams  use  all  the  usual  multimedia  techniques:  interactive  On- 
Screen  Laboratories,™  hypertext,  line  drawings,  photographs,  two-  and  three-dimensional  animations,  pro- 
gressive disclosure  (at  the  student’s  choice  of  pace),  and  video  and  sound.  The  multimedia  enhances  the  stu- 
dent's rate  of  learning  and  length  of  retention  of  the  material. 

This  paper  discusses  what  we  have  found  to  be  arguably  the  most  effective  multimedia  technique  for  learn- 
ing physical  concepts:  two-  and  three-dimensional  animations. 


* Our  publisher.  Physics  Academic  Software  (see  http://www.aip.org/pas)  released  Vectors,  late  in  1997.  The  Forces 
module  became  available  in  August  of  1998.  Motion  in  Electromagnetic  Fields  recently  began  shipping,  and  PAS  has  also 
accepted  Dipole  Magnets  for  publication.  We  expect  to  publish  two  other  modules  in  this  series. 
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The  Textbook  Approach 

A typical  modem  physics  textbook  contains  many  figures  and  pictures.  The  diagrams  are  almost  always 
two-dimensional  and  static. 

There  are  a few  exceptions.  Some  textbooks,  such  as  the  classic  text  on  mathematical  methods  by  Morse 
and  Feshbach,  have  experimented  with  doubled,  offset  pictures  which,  when  viewed  with  special  glasses,  give 
a three-dimensional  “depth”  to  the  drawing.  Others,  such  as  Max  Bom’s  Restless  Universe,  give  crude  an- 
imations by  utilizing  an  old  “flip  book”  technique  predating  even  Walt  Disney’s  cartoons. 

These  exceptions,  however,  are  quite  rare,  despite  the  fact  that  many  concepts  in  physics  involve  both  mo- 
tion and  all  three  spatial  dimensions.  We  suspect  that  one  reason  why  there  are  fewer  physics  majors  than, 
say,  psychology  majors  is  this:  there  is  a particular  aptitude  that  only  a few  people  have  for  being  able  to  visu- 
alize motions  and  depth  from  the  two-dimensional  figures  the  texts  and  lecturers  can  present. 

We  also  suspect  that  the  non-traditional  student,  such  as  the  accelerator  technicians  we  are  also  targeting, 
have  a much  more  “hands  on”  approach  to  learning.  Two-dimensional  diagrams,  while  a staple  of  their  trade, 
are  probably  secondary  to  their  understanding  of  a circuit  or  a device. 


The  Computer  Advantage 

A computer-based  tutorial  is  not  a substitute  for  a good  textbook.  It  is  hard  to  read  extended  text  on  a 
computer  screen.  However,  the  personal  computer  has  many  advantages  over  a textbook,  one  of  which  is  that 
it  can  easily  present  a moving  picture  of  a three-dimensional  concept. 

In  what  follows,  we  will  discuss  some  of  the  example  animations  we  have  incorporated  in  our  tutorials.  The 
reader  should  bear  in  mind  that  this  abstract  is  just  like  a textbook,  namely  static  and  two-dimensional.  We 
will  try  to  impart  the  flavor  of  the  animations,  but  there  is  no  substitute  for  seeing  them  “live”  on  a screen. 
The  presentation  of  this  paper  will  demonstrate  these  animations,  projected  from  the  computer. 

Most  of  the  examples  we  will  show  below  are  three-dimensional  animations  dealing  with  concepts  in- 
volving electromagnetic  forces.  That  is  because  of  the  nature  of  the  main  theme  of  our  modules,  Accelerators 
and  Beams,  where  the  electromagnetic  force  is  the  important  one.  We  believe  that  similarly  effective  anima- 
tions would  be  helpful  in  many  other  core  physics  courses,  such  as  classical  and  quantum  mechanics. 


Animation  Examples 

Right-Hand  Rule 

An  important  (and  often  difficult)  concept  for  many  students  is  to  understand  the  direction  of  the  force  felt 
by  a charged  particle  moving  in  a magnetic  field.  This  Lorentz  Force  is  expressed  as  a vector  cross  product  of 
the  particle’s  velocity  with  the  magnetic  field  vector,  as  in  the  following: 
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This  figure  is  a snapshot  of  a portion  of  a page  from  our  Vectors  module.  The  equation  and  the  photo  of 
the  hand  are  static.  The  colored  arrows  at  the  left,  however,  are  shown  as  they  appear  in  the  last  frame  of  an 
animation  which  shows  how  one  “swings”  the  velocity  vector  v into  the  magnetic  field  vector  B to  get  the 
force  direction.  This  is  then  followed  by  a moving  camera  view  of  the  three  vectors  that  shows  them  from  a 
number  of  points  of  view,  ending  up  in  the  position  shown.  The  student  can  repeat  the  animation  as  often  as 
he  or  she  needs  by  clicking  on  a button  labeled  “Animation.” 

Another,  related  animation  in  the  Vectors  module  shows  the  motion  of  a right-handed  screw,  a concept 
difficult  to  describe  in  words  or  in  static  pictures. 


The  page  on  which  this  animation  occurs  first  shows  the  (static)  picture  of  the  wood  screw  shown  on  the  left. 
When  the  student  clicks  on  the  animation  button,  the  screw  is  replaced  by  the  corkscrew-like  spring  that  then 
rotates  and  winds  its  way  up  from  the  lower  right  to  the  final  position  shown. 


Bending  Magnet 

The  concept  of  motion  of  a charged  particle  moving  through  a bending  magnet  is  a direct  consequence  of 
the  Lorentz  Force  Law  illustrated  above.  An  animation  showing  this  is  represented  by  the  following: 


Here  we  see  the  animation  in  its  initial  frame.  When  the  student  clicks  the  button,  the  red  ball  (representing  a 
charged  particle)  moves  into  the  magnetic  field  of  the  dipole  magnet  and  bends  to  the  left.  Again  the  student 
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can  repeat  the  animation  as  often  as  necessary.  One  reason  why  that  might  be  necessary  is  that,  in  this  case, 
our  tutorial  poses  a question  to  the  student:  is  the  red  ball  negatively  or  positively  charged?  (You  can  figure  it 
out  from  the  right-hand  rule.) 

Magnetic  Field  Around  a Wire 

The  Forces  module  also  illustrates  how  a current-carrying  wire  builds  up  a surrounding  magnetic  field. 
The  following  shows  the  animation  associated  with  this  a few  frames  after  its  start. 


Here  the  “red  charges”  start  moving  through  the  wire  to  the  right  (as  shown  on  the  left),  building  up  the  blue 
magnetic  field  lines  looping  around  the  current.  When  the  current  reaches  a steady  state,  the  magnetic  field 
stops  growing,  i.e.,  reaches  its  steady-state  condition  shown  on  the  right.  As  usual,  the  animation  is  repeat- 
able,  as  necessary. 

Cyclotron 

Our  Motion  in  Electromagnetic  Fields  module  contains  a section  on  cyclotrons.  The  “How  It  Works” 
page  has  a useful  two-dimensional  animation,  shown  on  the  next  page.  Here  the  screen  snapshot  shows  the 
situation  partway  through  the  animation.  You  can  “see”  the  acceleration,  which  occurs,  across  the  gap  be- 
tween the  Dees  from  the  spacing  of  the  dots  (and  the  increasing  speed  of  the  red  charged  dot).  The  display 
continuously  updates  the  values  of  the  radius  of  the  arc  and  its  (increasing)  velocity  at  the  lower  left  as  the 
particle  advances  outward. 


Eventually,  the  particle  radius  grows  to  the  point  where  it  leaves  the  upper  cyclotron  Dee  through  the  ex- 
tractor port.  For  the  physicists  among  us,  the  resulting  particle  beam  then  can  be  used  for  a scattering  experi- 
ment or  production  of  a radiopharmaceutical  isotope. 


o 
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This  two-dimensional  animation,  by  the  way,  was  done  entirely  within  the  authoring  framework  we  use 
for  our  tutorials.  We  use  a built-in  scripting  language  to  move  the  red  charge  dot  and  to  leave  its  trace. 

Vertical  Betatron  Oscillation 

Moving  on  to  a somewhat  more  advanced  module,  the  Dipole  Magnets  tutorial  discusses  how  these  mag- 
nets bend  and  focus  charged-particle  beams.  One  of  the  topics  considered  is  betatron  oscillation,  the  use  of 
restoring  forces  to  keep  the  beam  within  the  accelerator  while  it  is  being  accelerated  to  its  final  energy.  The 
animation  for  the  vertical  case  looks  like  this: 


The  student  is  most  likely  to  see  this  perspective-view  animation  after  having  first  viewed  a two-dimensional 
animation  and  another  three-dimensional  animation  looking  at  the  motion  from  the  side.  The  green  circle 
represents  the  “reference  trajectory”  and  the  red  circle  that  of  the  red  particle  which  is  slightly  “off’  that  prin- 
cipal path.  The  restoring  force  that  keeps  the  red  particle  moving  along  the  canted  red  circular  path  comes 
from  the  non-uniform-magnetic  field.  (The  pole  piece  shaping,  which  makes  the  non-uniformity,  is  better 
seen  in  the  side-view  animation.)  The  moving  red  charge  (ball)  also  simultaneously  traces  out  the  sine  wave 
graph  shown  on  the  right  side  of  the  figure. 


Lessons  Learned 

We  began  the  Accelerators  and  Beams  project  without  much  appreciation  of  the  power  of  a good  three- 
dimension  animation.  Clearly,  the  more  we  learned  about  how  to  make  them,  the  more  we  came  to  appreciate 
their  pedagogical  value.  As  we  stated  in  the  introduction,  we  believe  that,  for  many  purposes,  this  is  the  most 
valuable  multimedia  tool  for  presenting  technical  material. 
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Abstract:  A common  problem  of  CSCL  (Computer- Supported  Cooperative  Learning)  is 
addressed:  How  can  we  coordinate  goal-directed,  effective  interaction  in  a group  of  learners?  We 
introduce  the  concept  of  learning  protocols  which  originates  in  script  theory.  We  then  discuss 
which  dimensions  of  cooperative  learning  can  be  supported  by  learning  protocols  and  propose 
how  learning  protocols  can  be  integrated  in  a CSCL  environment. 


1.  Introduction 

Consider  the  following  two  examples: 

• A trainer  asks  his/her  students  a question.  After  some  seconds  without  response  he  thinks  that  nobody 
understood  or  could  answer  the  question  and  re-phrases  his  question.  Increasing  the  time  between  the  question 
and  the  intervention  by  a few  seconds  has  several  positive  quantitative  and  qualitative  effects  on  students’ 
answers  [Rowe  1974]. 

• A group  of  learners  is  discussing  a concept  introduced  in  a presentation.  The  most  eloquent  and  extrovert 
students  dominate  the  discussion.  Ideas  and  arguments  of  others  remain  unheard.  Providing  a set  of  rules  which 
enable  everybody  to  participate,  and  structuring  the  discussion  as  a cooperative  process  can  guide  the 
interaction  in  discussions  and  improve  the  learning  process  [Hall  & Mancini  1997]. 

These  examples  indicate  that  learning  can  be  improved  by  defining  certain  constraints  and  by  structuring 
the  learning  process.  Our  work  deals  with  the  question:  How  can  such  constraints,  rules,  or  process  structuring 
methods  be  realized  in  a computer-supported  cooperative  learning  environment?  We  call  this  kind  of  integrated 
learning  support  „leaming  protocols**. 

We  see  several  developments  which  lead  to  the  need  for  learning  protocols.  During  the  last  decades 
learning  is  regarded  more  and  more  important  due  to  the  increasing  amount  of  knowledge  available  world- wide, 
the  increasing  complexity  of  knowledge,  and  the  increasing  speed  of  these  changes.  This  raises  demands  to 
establish  a new  learning  culture:  (1)  Learning  has  to  be  a life-long  process,  we  have  to  better  support  adult  learners 
whose  learning  processes  differ  in  many  ways  from  those  in  schools  or  universities.  (2)  Individual  learning 
methods  are  not  sufficient  for  mastering  the  complexity  of  knowledge,  we  have  to  provide  methods  and  support  for 
groups  of  learners  to  cooperatively  acquire  knowledge  and  transfer  it  to  daily  practise.  (3)  In  order  to  address 
changes  in  work  organization  as  well  as  in  society  in  general,  learning  cannot  be  limited  to  scheduled  activities  in 
a common  classroom.  Learning  has  to  be  flexible  over  various  dimensions  such  as  time  and  space.  Therefore  we 
need  learning  environments  which  support  different  learning  scenarios,  e.g.  asynchronous  learning  or  learning  by 
geographically  distributed  learning  teams.  Our  focus  is  on  distributed  cooperative  settings,  in  which  learning  teams 
of  adults  want  to  learn  in  a flexible  way. 

Focussing  on  codocated  and  synchronous  settings,  a number  of  field  studies  have  indicated  that  students 
can  learn  effectively  in  pairs  and  in  small  groups,  if  they  are  supported  by  appropriate  learning  methods  [Slavin 
1995].  Cooperative  learning  can  improve  several  cognitive  and  social  aspects  of  the  learning  process  [Mancini  et 
al.  1998].  In  a distributed  cooperative  setting,  learning  teams  need  means  to  communicate  and  to  coordinate  their 
learning  activities.  Given  an  infrastructure  with  networked  computers,  communication  and  cooperation 
environments  which  address  these  needs,  e.g.  by  providing  various  communication  channels  and  shared  data 
repositories,  are  available  (e.g.,  Microsoft  NetMeeting).  Can  cooperative  learning  also  be  used  in  a distributed 
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cooperative  setting  in  order  to  achieve  similiar  positive  results  as  in  a traditional  setting?  Can  we  transfer  the 
cooperative  learning  methods? 

These  topics  are  tackled  by  a field  of  research  called  computer-supported  cooperative  learning  (CSCL) 
[Koschmann  1996].  Some  positive  experiences  have  been  made  using  computers  to  support  cooperative  learning  in 
traditional  settings.  For  example,  in  the  CSILE  system  a certain  cooperative  learning  method  is  modelled,  and 
learners  are  guided  through  the  process  by  specific  rules  [Scardamalia  & Bereiter  1996].  In  [Hron  et  al.  1997]  a 
system  was  successfully  tested  that  controlled  the  dialog  in  cooperative  dyads  with  a specific  communication 
interface. 

There  are  some  additional  constraints  in  a distributed  cooperative  setting:  The  quantity  and  quality  of 
transmitted  information  is  different  because  many  non-verbal  signals  are  not  transmitted.  E.g.,  chat  and  e-mail 
deal  with  written  communication  and  provide  only  limited  support  for  non-verbal  communication  by  using 
emoticons  or  attaching  sound  or  image  files  to  a message.  Audio/video  conferences  provide  audio-visual 
information  in  a lower  quantity  and  quality  than  face-to-face  settings.  A major  problem  is  that  some  cues  are 
missing  which  structure  cooperative  learning  in  the  co-located,  synchronous  setting.  As  a result,  e.g.  simple  turn 
taking  mechanisms  in  discussions  fail  in  such  settings. 

We  see  two  ways  to  address  this  issue: 

• Integrate  nonverbal  signals  in  a CSCL  environment  by  integrating  additional  communication  channels,  e.g. 
sensors  detecting  whether  the  learner  is  currently  present  in  front  of  the  computer  or  cameras  analyzing  facial 
expressions. 

• A second  way  is  to  develop  cooperative  learning  methods  or  adapt  existing  ones  to  the  distributed  cooperative 
setting,  and  thereby  taking  into  account  the  constraints  of  this  setting  such  as  changed  number  and  quality  of 
communication  channels  or  less  ambient  information  flow. 

The  first  solution  leads  to  systems  which  track  and  check  user  activities  with  various  kinds  of  sensors. 
That  can  be  in  the  way  of  virtual  reality  as  immersive  3-dimensional  virtual  worlds  or  by  enhancing  the 
intelligence  of  rooms  and  furniture  in  the  real  world.  The  advantage  of  these  approaches  is  to  represent  reality  and 
thereby  providing  a familiar  environment  with  well-known  or  easy-to-leam  means  of  communication  and 
cooperation.  But  this  advantage  holds  only  as  long  as  the  system  more  or  less  clings  to  reproducing  reality; 
additional  support  provided  by  the  system  has  to  be  learned  by  the  users.  Our  approach,  however,  is  related  to  the 
second  solution.  We  call  cooperative  learning  methods  in  a distributed  setting  learning  protocols. 

The  remainder  of  this  paper  is  organized  as  follows:  In  the  next  section  the  concept  of  learning  protocols 
is  described  in  more  detail.  Section  three  elaborates  on  the  various  ways  in  which  learning  protocols  can  help 
teams  of  learners  to  improve  their  learning  process  and  results.  The  integration  of  learning  protocols  in  a CSCL 
environment  is  presented  in  the  fourth  section.  We  close  with  a description  of  our  next  steps. 


2.  Learning  Protocols  as  Implemented  Scripts 

In  cooperative  learning  we  distinguish  two  kinds  of  supporting  methods  [Hron  et  al.  1997,  Mancini  et  al.  1998]: 

• Global  methods  which  structure  the  cooperation  on  a general  level,  e.g.  by  providing  support  for  group 
organization,  monitoring  constraints  such  as  a maximum  floor  holding  time. 

• Structured  or  scripted  cooperative  learning  methods  which  provide  protocols  for  cooperative  learning  by 
structuring  the  dialog  and  actions  of  the  learners. 

Learning  protocols  perceived  as  scripted  cooperative  learning  methods  are  theoretically  based  on 
psychological  script  theory.  In  cognitive  and  social  psychology,  generalised  knowledge  about  a routine  sequence  of 
related  events  and  activities  is  commonly  called  a script  [Schank  1982],  [Schank  & Abelson  1977].  A script  is  a 
knowledge  structure  that,  once  activated,  yields  information  on  what  events  to  expect  and  how  to  act  in  a specific 
type  of  situation.  For  example,  the  well  known  restaurant  script  describes  the  sequence  of  actions  when  having  a 
meal  in  a restaurant  (e.g.,  ordering,  eating,  paying),  and  it  specifies  what  to  expect  from  others,  e.g.,  from  the 
waiter,  depending  on  one's  own  actions.  A script  can  be  seen  as  a type  of  schema,  which,  in  addition  to  a 
prototypical  chain  of  actions,  contains  slots  to  specify  the  temporary  circumstances,  and  default  values  to  guide 
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expectations  and  inferences  if  no  further  information  is  available.  Hence,  scripts  not  only  control  many  stereotyped 
encounters  of  daily  life,  they  also  reduce  cognitive  effort  by  providing  guidelines  that  decrease  the  need  to  think 
about  one's  actions  in  well  known  situations. 

Scripts  can  be  triggered  by  aspects  of  the  situation  (situation-controlled  script;  e.g.  initiating  a vote  in  a 
controversial  discussion)  or  by  a specific  role  (role-controlled  script;  e.g.  a trainer  is  expected  to  answer  a question 
and  to  determine  the  learning  process).  To  act  according  to  a script  one  has  to  know  the  script  itself  and  one's  part 
in  that  script.  The  activation  of  a script  is  considered  to  be  mostly  automatic. 

Cooperative  learning  methods  can  be  perceived  as  externalized  scripts  attached  to  learning  situations  or 
roles  in  a learning  team.  In  natural  face-to-face  situations,  the  actions  of  persons  applying  the  same  script  are 
aligned  by  a lot  of  explicit  and  implicit  cues  (e.g.,  non-verbal  cues  such  as  pauses  or  changes  in  posture  indicate 
turn-taking  in  a discussion  script).  In  computer-mediated  learning  environments,  however,  these  cues  are  largely 
missing,  and  the  coordinated  enactment  of  a script  can  be  seriously  endangered.  Especially,  cooperating  learners 
may  not  use  the  same  script  at  all,  or  may  be  uncertain  about  which  script  the  others  apply,  or  may  define  their 
roles  in  the  script  differently,  or  may  loose  orientation  where  the  group  currently  is  in  the  script.  These  difficulties 
in  CSCL  environments  suggest  that  one  should  not  rely  on  an  automatic  script  enactment,  but  that  one  should 
provide  scripts  as  implemented  procedures  that  can  be  executed  on  demand.  Learning  protocols,  as  defined  here, 
are  just  a set  of  useful  scripts  for  learning,  externalized  as  executable  methods,  with  roles,  events,  and  actions  made 
explicit. 


3.  How  Learning  Protocols  Can  Structure  Cooperative  Learning 
3.1  Explaining:  An  Example  of  a Learning  Protocol 

Imagine  that  in  a pairwise  learning  situation  person  A needs  an  explanation  about  some  problem.  Hence,  he  asks 
for  explanation  from  person  B,  who  tries  to  provide  the  explanation.  In  natural  face-to-face  situations,  all 
interactions  during  the  explanation  process  are  usually  carried  out  without  much  friction,  since  many  verbal  and 
non-verbal  cues  guide  the  dialog.  However,  in  a distributed  computer  mediated  situation,  person  A might  want  to 
activate  an  "explanation  protocol",  which  then  controls  the  communication  process.  The  explanation  protocol  is 
very  simple,  basically  controlling  the  right  to  speak  and  indicating  which  type  of  communication  is  currently  being 
performed  (if,  e.g.,  a question  is  asked,  or  if  an  explanation  is  delivered).  The  protocol  will,  for  example,  strictly 
switch  between  explainer  and  explainee.  The  explainee  will  respond  to  an  explanation  either  by  asking  for  more 
information,  or  by  deciding  that  he  is  satisfied  with  the  explanation,  which  will  terminate  the  protocol.  The 
explainer  will  either  deliver  a further  explanation,  or  he  will  declare  himself  unable  to  provide  the  explanation, 
which  will  also  terminate  the  protocol.  Learning  protocols  can  be  represented  as  a kind  of  state-transition-diagram; 
Figure  1 gives  an  example.  The  following  sections  show  that  this  very  simple  learning  protocol  can  be  enhanced  by 
integrating  control  over  other  ressources  such  as  learning  artifacts  and  shared  workspaces. 


Figure  1:  State-transition-diagram  of  an  explanation  protocol 


3.2  Levels  of  Learning  Protocols 
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Learning  protocols  are  useflil  on  different  levels  of  the  cooperative  learning  process: 

• On  the  most  basic  level,  they  guide  elementary  communication  processes  among  learners,  e.g.,  discussions  (be 
it  by  audio  or  chat),  presentations,  or  dyadic  dialog. 

• On  the  next  level,  protocols  support  group  activities  which  need  some  coordination  control,  e.g.,  common 
navigation  through  hypermedia  documents  or  preparation  of  agendas  and  time  schedules. 

• The  third  level  refers  to  protocols  which  are  designed  to  support  cooperation,  e.g.  jointly  editing,  domain 
specific  role  plays  or  simulations. 

On  each  level  protocols  range  from  more  generic  collaboration  protocols,  such  as  a discussion  protocol,  to 
very  learning  specific  protocols,  such  as  giving  an  explanation  in  a certain  domain.  In  principle,  the  list  of 
potential  candidates  of  cooperative  learning  processes  that  could  be  translated  into  learning  protocols  is  almost 
infinite.  Special  techniques  such  as  "Jigsaw  11"  [Slavin  1995],  or  different  cooperation  modes  such  as  symmetrical 
(e.g.,  a discussion  among  students)  and  asymmetrical  (e.g.  a presentation)  modes,  depending  on  role  and  group 
structure,  could  be  devised. 

We  think  of  learning  protocols  as  a set  of  potential  support  tools  from  which  learners  and  trainers  may 
select  what  they  consider  to  be  useful.  At  any  time,  participants  may  choose  from  a menu  a specific  protocol,  which 
is  then  activated.  When  finished,  the  group  is  free  to  select  any  other  protocol,  or  to  proceed  without  systematic 
support.  Hence,  the  overall  learning  process  is  not  itself  controlled  by  a protocol,  but  can  be  augmented  at  any  time 
when  support  seems  appropriate. 


3.3  Components  of  a Learning  Protocol 

Irrespective  of  the  level  of  a protocol,  a protocol  consists  of  a set  of  components,  which  can  be  derived  from  script 
theory.  First,  a protocol  has  a name;  the  name  signifies  the  situation  type  to  which  the  protocol  can  be  applied.  For 
example,  a (cooperative)  explanation  protocol  applies  to  situations  in  which  an  explanation  is  needed  by  some 
person  and  can  be  delivered  by  some  other  person.  Unlike  in  natural  situations,  learning  protocols  are  selected 
solely  by  name,  i.e.,  a person  subsumes  the  situation  under  the  category  referred  to  by  the  protocol's  name,  and 
intentionally  activates  the  protocol. 

Secondly,  a protocol  consists  of  a set  of  states  and  transitions.  In  each  state  the  users  can  perform  actions 
such  as  communicate  or  manipulate  artifacts.  A transition  to  another  state  is  triggered  by  an  action  or  a specific 
condition,  e.g.,  the  time-out  of  the  preceeding  state.  Of  course,  various  iterations  and  conditional  branchings  may 
occur.  The  interface  of  the  learning  protocol  has  to  take  care  that  different  types  of  actions  are  appropriately 
perceived  and  classified. 

Thirdly,  a protocol  includes  different  roles  pertaining  to  the  persons  involved  in  the  enactment  of  the 
protocol.  A learning  process  commonly  consists  of  one  or  more  students,  and  one  or  more  trainers  or  teachers. 
Other  roles  specific  to  learning  are  tutors,  experts,  and  moderators.  Each  participant  of  a running  learning  protocol 
needs  to  obtain  a definite  and  unique  role,  and  needs  to  know  which  roles  other  participants  have.  The  role  one  has 
is  either  arbitrarily  defined  by  oneself,  or  is  derived  from  the  overall  group  structure.  Roles  should  also  be  indicated 
on  the  computer  screen,  i.e.,  everybody  should  be  informed  about  his  and  the  others'  roles  by  some  sort  of  visual 
indicator,  in  order  to  prevent  role  confusion. 

Finally,  a protocol  may  contain  various  types  of  artifacts,  i.e.,  text  documents,  graphical  objects,  test 
forms,  etc.  In  many  simple  communication  protocols,  however,  the  definition  of  special  artifacts  is  unnecessary,  or 
only  optional.  For  example,  in  the  course  of  an  explanation,  the  explainer  and  the  explainee  may  use  a shared 
whiteboard  to  outline  some  ideas,  but  this  is  not  essential  for  the  explanation  process. 


3.4  The  Construction  of  Shared  Knowledge 

A special  problem  in  cooperative  learning  is  to  gain  a common  understanding  of  the  shared  knowledge  of  the 
group.  Efficient  communication  requires  that  each  participant  knows,  at  least  approximately,  what  the  other 
participants  know.  Without  this  common  background,  it  would  require  a lot  of  time  and  effort  to  achieve  effective 
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cooperation.  In  cooperative  learning,  due  to  the  continuous  acquisition  of  new  knowledge  of  each  participant,  the 
shared  knowledge  is  ever  changing.  This  makes  it  even  more  important  to  provide  systematic  support  to  construct 
and  represent  the  common  knowledge  of  the  learning  group  in  some  way. 

We  propose  that  learners  externalize  their  knowledge  in  the  form  of  a diagram  with  nodes  and  links: 
nodes  denote  specific  concepts  (from  the  domain  to  be  learned),  and  links  denote  specific  relations  among 
concepts,  such  as  ”is-example-of’,  or  "is-subtopic-of*.  Without  going  into  detail  (see  [Pfister  et  al.  1999]),  it  seems 
clear  that  this  diagram  of  shared  knowledge  needs  to  be  modified  or  updated  regularly  and  systematically.  A 
special  type  of  learning  protocol  is  needed  to  support  the  maintenance  of  the  shared  knowledge,  providing 
protocols  for  expanding,  for  reducing,  for  restructuring  the  knowledge  corpus,  and  for  establishing  consensus 
among  learners  on  when  and  how  to  introduce  changes  to  the  knowledge. 


4.  Implementation 

In  the  CLear  (computer-supported  cooperative  learning)  project  at  GMD-IPSI,  we  develop  computer-supported 
cooperative  learning  environments  and  cooperative  learning  methods  for  such  systems. 

In  a first  step  we  designed  the  VITAL  (^virtual  teaching  and  learning)  prototype  using  the  COAST 
[Schuckmann  et  al.  1996]  framework  for  building  groupware.  As  described  in  section  2,  we  distinguish  two  kinds 
of  supporting  methods  for  cooperative  learning:  global  methods  and  scripted  cooperative  learning  methods.  In 
VITAL  we  focussed  mainly  on  supporting  global  methods.  The  prototype  offers  global  group  organization  methods 
and  a limited  cooperation  support,  e.g.  by  providing  a structured  communication  board  and  different  cooperation 
modes.  Evaluations  of  VITAL  in  university  settings  showed  that  the  system  is  well  accepted  but  offers  only  limited 
support  for  structuring  the  cooperative  learning  process  [Beck-Wilson  in  press].  Therefore  we  extended  the 
approach  and  integrated  learning  protocols  [Pfister  et  al.  1998]. 

Our  approach  can  be  briefly  characterized  by  (1)  providing  a virtual  learning  world  consisting  of  different 
types  of  virtual  rooms,  in  which  learners  can  communicate  and  cooperate  synchronously  and  asynchronously,  (2) 
supplying  each  virtual  learning  room  with  a shared  hypermedia  workspace,  (3)  using  hypermedia  to  represent  the 
learning  material,  and  (4)  using  hypermedia  to  define  learning  protocols  and  visualize  their  execution. 

For  implementing  learning  protocols  in  a CSCL  environment  we  use  several  techniques: 

• Some  global  methods  can  rely  on  access  rights,  rules  and  conditions,  or  mechanisms  for  conflict  resolution. 
Therefore,  we  need  functionality  for  defining  and  checking  these  methods. 

• Scripted  cooperative  learning  methods  are  modelled  using  collaboration  protocols  similar  to  state  transition 
diagrams  (see  Figure  1).  Each  state  is  defined  by  specific  functionalities  and  rights  of  the  involved 
participants,  i.e.,  a set  of  behavior  rules,  which  define  which  role  is  allowed  to  perform  which  operation  on 
which  resource.  The  learning  protocols  are  implemented  using  the  modelling  method  of  SCOPE,  a system 
which  allows  the  definition,  execution,  modification  and  monitoring  of  general  collaboration  protocols  [Miao 
&Haake  1998]. 

The  implementation  provides  guided  interactions  between  the  system  and  the  users.  Depending  on  the 
kind  of  learning  protocol  and  his/her  role  a user  may  initiate  a protocol,  e.g.,  by  simply  pushing  a button.  If  all 
preconditions  are  met,  the  roles  in  the  protocol  are  matched  with  the  actual  team  members.  During  the  execution  of 
the  protocol,  the  user  interface  provides  information,  such  as  the  current  state  and  possible  actions  in  this  state  of 
the  protocol.  Special  templates,  e.g.  for  requests,  explanations,  and  responses  can  be  provided,  and  control  is 
passed  depending  upon  team  members’  actions. 

An  advanced  prototype,  called  CROCODILE  (==creative  open  cooperative  distributed  learning 
environment),  is  currently  being  implemented  based  on  the  experiences  made  with  VITAL  and  SCOPE. 


5.  Next  Steps 

We  want  to  investigate  whether  learning  protocols  really  inherit  the  advantages  of  their  predecessors:  cooperative 
learning  methods  and  scripts.  In  addition  to  structuring  the  cooperative  learning  process  we  also  expect  them  to 
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serve  as  a means  for  meta-leaming,  i.e.  learning  to  learn  cooperatively.  Field  studies  will  show  whether  users  can 
create  (internal!)  scripts  by  multiple  usage  of  learning  protocols  and  thereby  transfer  their  behaviour  to  other 
collaborative  situations. 

We  still  have  a lot  of  open  issues  which  require  further  research,  two  important  ones  being  the  usability  of 
learning  protocols  in  medium/large  groups  and  in  asynchronous  settings. 

• Group  size 

Most  research  on  scripted  cooperative  learning  (with  and  without  using  computers)  deals  with  dyads  [Mancini  et 
al.  1998],  [Hron  et  al.  1997]  or  relatively  low  structured  scripts,  which  structures  the  learning  process  in  coarse 
steps  [Hall  & Mancini  1997].  Only  limited  knowledge  is  available  on  which  methods  are  suited  for  which  group 
size.  This  has  to  be  evaluated  by  performing  extensive  testing  with  user  groups  of  various  size. 

• Asynchronous  learning 

A problem  for  learning  teams  at  the  workplace  is  the  need  to  learn  asynchronously,  e.g.  at  times  dictated  by 
business  requirements.  Asynchronous  cooperation  is  also  very  important  or  in  many  cases  the  only  possible 
solution  for  dispersed  learning  teams  spread  over  different  time  zones.  Many  of  the  traditional  cooperative  learning 
methods  assume  synchronous  settings  and  have  to  be  adapted  for  asynchronous  usage. 

Further  information  about  the  CLear  project  and  the  learning  environments  VITAL  and  CROCODILE  is 
available  at  our  website  http://www.darmstadt.gmd.de/concert . 
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Abstract:  A range  of  pedagogic,  administrative,  socio-economic,  structural  and 
resource  issues  surround  the  establishment  of  a global  campus.  This  paper 
identifies  and  discusses  such  issues,  and  suggests  that  pedagogic  aspects  are 
paramount.  The  limitations  of  conventional  subject-based  approaches  are 
considered,  and  local  experience  in  curriculum  design  and  module  integration  is 
discussed  in  the  context  of  flexible  learning,  wider  access  and  participation.  The 
need  for  a range  of  learner  support  mechanisms  is  indicated,  and  our 
implementation  (together  with  others)  is  described.  The  concept  of  a global 
campus  is  underpiimed  by  appropriate  technological  developments.  Internet-  and 
Web-based  experience  is  discussed,  as  is  the  need  for  more  radical  approaches 
such  as  Virtual  Reality  techniques. 


Introduction 

The  nature  of  higher  education  (H.E.)  is  changing  globally:  the  convergence  of  wider  access 
demands,  greater  emphasis  on  vocational  preparation,  new  teaching  and  learning  delivery 
mechanisms,  and  the  emergence  of  globalised  economies  is  propelling  forward-looking  H.E. 
institutions  into  careful  self-analysis.  At  the  same  time,  many  countries  (for  example,  U.K., 
Australia,  U.S.)  are  witnessing  a withdrawal  from  state-funding  for  both  H.E.  institutions  and  for 
student  maintenance  costs.  With  other  but  equally  significant  structural  and  resource  difficulties, 
developing  nations  also  share  these  problems  (Koul,  1995) 

As  well  as  financial  challenges,  certain  global  trends  are  beginning  to  emerge.  Valcke  and  Vuist 
(1995)  describe  experiences  in  the  Netherlands  that  matches  our  local  experience  in  the  UK. 
Demographic  changes  mean  that  the  profile  of  the  traditional  student  is  becoming  older,  often 
working  or  balancing  the  requirements  of  other  family/career  commitments:  this  implies  a growing 
demand  for  increased  part-time  and  more  flexible  learning  provision.  An  increasing  number  of  UK 
institutions  have  adopted  a modular  approach  to  course  design  and  delivery  - see,  for  example, 
George,  Murfin,  and  White  (1992).  At  the  same  time,  there  are  many  pressures  for  more  effective 
teaching  and  learning  delivery  mechanisms.  Active  learning,  student-centred  and  case-led  methods 
are  now  accepted  parts  of  the  educator's  canon  (Rogers  [1991]),  and  such  changes  have  been 
growing  in  influence  since  the  late  1980s.  What  is  now  emerging  as  a radically  different  and 
(potentially)  revolutionary  change  is  the  use  of  Internet-  and  Web-based  delivery  chaimels. 
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Open  Learning  and  emancipation 


Before  discussing  such  issues,  it  is  worth  recalling  that  one  of  the  strengths  of  an  open  learning 
approach  is  what  Reid  (1995)  refers  to  as  an  ‘emancipatory  tradition’  - the  barrier-dissolving 
aspects  of  such  learning.  Such  emancipatory  aspects  are  principally  associated  with  learners 
previously  denied  access  to  education  and  assume  a much  wider  relevance  within  the  context  of  a 
global  campus.  Reid  (1995)  directs  our  attention  to  ‘structural  constraints’  relating  to  institutional, 
social,  cultural,  and  political/economic  factors.  While  the  cost  of  commissioning  and  building  new 
higher  education  institutions  may  stretch  the  financial  resources  of  the  developing  countries,  the 
ability  to  ‘tap’  in  to  such  programmes  of  study  offered  via  the  Web  may  be  of  great  emancipatory, 
as  well  as  economic,  potential. 

This  implies  fundamental  challenges  for  the  future  of  higher  education.  Dolence  (1995)  suggests 
that  these  challenges  stem  from  the  use  of  an  educational  model  developed  from  and  for  an 
industrial  model  of  society.  Is  this  appropriate  in  what  Dolence  calls  an  ‘information  model’  of 
society?  Is  the  cost  model  of  such  a paradigm  still  valid  given  the  nature  of  such  an  “Information 
age”?  Can  the  higher  education  sector  ignore  the  potential  of  technologies  such  as  Web-based 
‘push’  technology,  ‘click-thru’  mechanisms,  video-conferencing,  and  virtual  reality  models? 

In  a wide-ranging  survey  of  current  thinking  about  pedagogical  issues  facing  H.E.  Diana  Laurillard 
articulates  the  necessity  of  ‘situation-learning’  (Laurillard,  1993).  Such  learning  implies  that  “the 
acquisition  of  inert  concepts  (e.g.  algorithms,  routines,  decontextualised  definitions  - i.e.  the  stuff 
of  many  university  courses)  is  of  no  use  if  the  student  cannot  apply  them.. .[We]  have  to  use  our 
knowledge  in  authentic  activity,  i.e.  genuine  application  of  the  knowledge;  this  allows  us  to  build 
an  increasingly  rich  understanding  of  the  tool  itself  and  how  it  operates”  (page  17).  Here,  the 
application  of  Virtual  Reality  models  and  techniques  offers  potential  for  implementation  of 
‘authentic  activity’  and  ‘situated  learning’. 


Globalised  education 

The  idea  of  a globalised  higher  education  sector  - a ‘global  campus’  - implies  administrative, 
pedagogic,  operational,  and  technical  issues,  and  the  Virtual  Online  University 
(www . tcet . unt . edu)  offers  an  interesting  metaphor  for  the  learning  ‘cyberspace’.  It  is  structured 
around  a virtual  campus,  designated  as  a ‘MOO’  ( an  Object-Oriented  Multi-User  Dimension), 
around  which  students  can  wander  as  they  would  around  a physical  campus.  At  the  same  time,  the 
range  of  courses  offered  remains  very  traditionally  ‘subject-based’.  Is  this  appropriate  to  such  a 
new  learning  environment? 

Such  an  environment  may  well  present  a lack  of  unified  focus  for  learners.  Laurillard  (1993)  draws 
attention  to  the  decentralising  effect  of  new  technology.  She  argues  that  this  can  push  forward  a 
very  fragmented  view  of  knowledge.  This  is  opposed  to,  what  she  typifies  as,  ‘academic 
knowledge’,  which  has  an  integrative  function  different  from  simply  ‘knowledge’.  It  is  this 
reflective  aspect  which  is  likely  to  prove  difficult  to  deliver  in  the  information  model  described  by 
Dolence  (1995).  Learners  in  a virtual  campus  may  well  require  a much  stronger  structural  identity. 
It  is,  therefore,  important  that  there  is  a strong  conceptual  platform  that  integrates  and  preserves  a 
sense  of  intellectual  and  discipline  consistency. 
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Local  implementation 


At  a local  level,  we  have  attempted  to  incorporate  such  issues  and  challenges  within  a conceptual 
and  pedagogic  framework  for  the  education  of  information  engineers.  A set  of  modules  has  been 
implemented  within  a conceptual  framework  entitled  Systems,  Techniques,  Implementation,  and 
Integration  (STIMI).  An  outline  of  this  is  shown  in  figure  1 (The  STIMI  Progression).  In  stage  1, 
the  concept  of  systems  and  systems  thinking  is  introduced  to  students.  This  theme  is  elaborated  in 

separate  semesters  in  stage  2.  The  first 
emphasises  the  usual  systems  analysis 
techniques  of  process,  data,  and  event 
modelling.  Implementation  within  the  context 
of  Computer-Aided  Production  Management 
(CAPM)  is  combined  in  the  second  semester. 
By  stage  3,  management  and  management- 
information  related  issues  are  addressed  by 
examination  of  strategic  and  operational 
planning  aspects.  The  integration  of  ‘islands  of 
automation’  within  the  context  of 
manufacturing  businesses  is  the  prime  concern 
of  the  postgraduate  programme.  This  module  is 
designed  to  enhance  the  learning  experience  of 
those  continuing  students  who  wish  to  pursue  their  studies  at  this  level,  together  with  providing  a 
firm  base  for  external  students  entering  the  programme  from  a wide  range  of  industrial  and 
educational  backgrounds. 

The  basis  of  delivery  for  the  modules  is  illustrated  in 
figure  2 (Support  for  Module  Teaching  programmes).  A 
body  of  structured  teaching  material  (linked  and  sustained 
by  four  pillars  that  rest  on  the  STIMI  conceptual 
foundation)  has  been  developed  to  minimise  duplication 
of  content  and  to  provide  a common  platform  for  delivery. 
Following  Race  (1989),  this  incorporates  a range  of 
techniques  developed  from  Open  and  Distance  Learning 
models,  such  as  self-assessment  questions  and  other 
developmental  questions  and  activities  consolidated 
within  the  text. 


Figure  2:  Support  for  module  teaching 
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The  particular  problems  of  part-time  students,  those  students  begirming  their  studies  at  differing 
entry  points,  and  students  who  require  extra  support  are  recognised,  and  are  of  especial  relevance 
within  the  context  of  this  paper.  Such  students  have  significant  demands  placed  upon  them  by  work 
and  family  pressures  so  such  students  can  also  access  this  material  via  the  University  web  site. 
Open  and  distance  learning  material  may  appeal  to  university  management  because  of  its  apparent 
cheapness,  but  such  students  are  likely  to  require  significant  support  mechanisms.  They  may  be 
unfamiliar  with  the  nature  of  independent  learning  implied  by  a higher  education  programme  of 
study.  How  are  such  mechanisms  to  be  implemented  and  managed  given  an  electronic  local 
campus,  let  alone  a ‘global  campus’?  Learner  support  in  the  global  campus  will  rest  upon  two  pre- 
requisites: learner  support  and  technological  delivery  mechanisms.  We  will  address  the  latter  aspect 
later  in  this  paper.  For  now,  we  wish  to  describe  briefly  some  of  the  issues  involved  in  supporting 
widely-dispersed  learners.  Such  learners  may  well  be  dispersed  temporally  as  well  as 
geographically.  The  use  of  packaged  material  (CD-Rom,  text-based  and  other  resources)  is  the 
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conventional  means  of  handling  knowledge  content.  But  how  can  learners  benefit  from  a shared 
learning  experience?  Romiszowski  (1995)  describes  how  a co-operative  programme  can  be 
established,  and  can  be  extended  by  various  Internet  and  Web-based  technologies  - bulletin-board 
conference  facilities,  email,  links  to  resource  materials  (virtual  museums,  virtual  libraries).  All  of 
these  offer  significant  potential  as  an  integrated  collaborative  platform. 


Distributed  learning 

Such  initiatives  still,  however,  rest  upon  a traditional  view  of  teaching  and  learning.  The  support  of 
Distributed  Learning  paradigms  should  now  be  considered.  Distributed  learning  is  founded  on  the 
belief  that  the  personal  needs  of  the  student  are  paramount.  Academic  institutions  can  evolve 
courses  based  upon  modules  (STIMI),  that  can  expand  choices  for  students  whilst  maintaining 
educational  quality  across  location  and  time-bound  constraints,  as  well  as  providing  the  student 
with  a more  self-paced,  self  directed,  career-biased  learning  experience.  A distributed  learning 
environment  can  essentially  be  stand-alone  with  a single  subject/operator  or  can  be  linked  via  with 
a common  database  and/or  other  systems  into  a distributed  system/servers.  The  common  database 
may  either  be  held  centrally  (and  modified  if  required  by  each  system/server)  or  local  copies  can  be 
used  with  changes  highlighted  to  the  other  components  of  the  system/network.  For  example,  for  a 
design  engineer  a distributed  learning  environment  is  desirable,  but  design  engineers  can  be  trained 
individually.  For  other  learning  contexts,  it  is  mandatory,  especially  where  teams  of  operators 
interact  in  command  and  control  of  resources.  An  example  where  a distributed  environment  would 
be  mandatory  is  in  Air  Traffic  Control  where  visualisation  of  air  space  needs  to  be  accessed  and 
controlled  by  many  operatives. 


Technological  aspects 

One  of  the  potential  problems  with  distributed  learning  is  data  transmission  rates  over  networks. 
Optical  Fibre  technology  has  developed  such  that  within  the  near  future  transmission  rates  of  10 
Gigabits  sec''  will  be  possible  over  broad  band  optical  networks  (Chan,  1995).  A current  or  near 
term  technology  is  the  so-called  "DVD"  CD-ROM  (Bell,  1996)  where  data  is  written  to  a multiple 
layer  CD.  This  technology  gives  the  CD-ROM  the  capacity  to  store  up  to  17  Gigabytes  on  a single 
disc  with  a data  rate  capacity  of  1 1 Mbits  sec''. 

Such  technologies  (given  development)  are  of  direct  relevance  to  Distributed  Learning  and  its 
application,  by  affording  a platform  for  the  development  of  such  teaching  aids  as  Virtual  Reality 
(VR).  Given  the  inherent  ability  of  VR  to  display  three  or  multi-dimensional  properties  of  objects 
to  a operator  its  uses  in  teaching  and  training  are  virtually  endless.  It  must,  however,  be 
remembered  that  VR  is  essentially  only  a display  tool  and  therefore  cannot  fully  substitute  for  other 
forms  of  learning.  It  is  also  important,  given  the  growing  importance  and  development  of  VR 
technology,  that  VR  itself  should  be  taught  as  a module  at  core  level  (STIMI  ).  Being  a multi- 
disciplinary subject  it  can  only  be  taught  at  a relatively  high  level.  It  should  however  become  a 
mandatory  subject  in  all  information  technology  courses  even  if  minimum  time  is  devoted  to  it 
(Elson,  Simms  [1997]). 

Table  1 below  lists  some  potential  applications  for  VR  in  teaching.  This  list  is  not  meant  to  be 
complete  and  many  other  applications  may  occur  to  the  reader. 
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Table  1:  Some  Uses  of  VR  in  education 


Subject 

Use 

Physics 

Motion,  Orbital  Mechanics,  Gas  Physics,  Multi- 
dimensional analysis 

Chemistry 

Structure  of  compounds,  molecules 

Biology 

Structure  of  organisms,  behaviour  of  organisms 

Earth  Sciences 

Earth  observation,  landscape 

Geography 

3-D  structure 

Geology 

Structure  of  fossils 

Engineering 

CAD/CAM 

General 

Display  and  analysis  of  multi-dimensional  data 

For  basic  tutorial  type  work  the  basic  technology  already  exists  with  PC's  and  CD-ROM's.  For 
more  extensive  use  lightweight  cheap  helmet-type  displays  need  to  be  available  along  with  the 
appropriate  computer  technology  (fast  networks  delivering  M-Bytes  sec''^ 

In  conclusion  then,  the  Global  Campus  is  a rapidly  developing  field  with  many  potential 
applications.  It  has  direct  application  to  teaching  and  training  at  many  levels.  Educators  should  be 
made  aware  of  the  capabilities  and  limitations  of  the  technology.  Distributed  systems  are  required 
for  its  effective  use.  The  concept  of  a Global  Campus  offers  a useful  metaphor  for  the  delivery  of  a 
new  mode  of  education,  supported  by  fast  and  flexible  delivery  mechanisms. 
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Abstract:  This  article  is  purposed  to  present  an  intelligent  agent-based  instructional  design 
model  on  the  subject  of  cognitive  mapping  strategy.  It  is  a further  conceptualisation  of  the 
rational  behind  the  existing  SMILE-Maker  tool.  SMILE-Maker  is  a prototype  of  computer 
programme  aimed  at  supporting  the  user  to  build  some  knowledge  and  skills  in  cognitive 
mapping  technique  when  problem  solving  occurs.  A new  instructional  design  scenario  based 
upon  this  model  has  been  added  to  the  programme. 


Introduction 

The  ambition  behind  the  agent-based  instructional  design  model  is  to  construct,  in  more  effective  and  efficient 
way,  the  interaction  between  the  learner  as  a user  and  the  facilitator  as  a system  instructional  designer.  Cognitive 
mapping  is  the  content  this  interaction  deals  with.  Instructional  events  are  supposed  to  play  an  intermediate  role. 
The  interaction  is  formalised  in  the  concept  mapping  format  as  the  gaps  in  overlapping  the  behaviour  patterns  of 
the  user  and  the  system  designer  are  identified  and  then  some  advises  are  provided.  It  is  hypothesised  that  this 
computer-based  instructional  design  should  solve  some  of  the  fundamental  all-the-time  issues  of  learning  and 
instruction.  What  seemed  impossible  for  the  domain  of  instructional  design  is  very  likely  to  happen  now  because 
of  the  new  hardware  and  software  development  and  the  more  appropriate  formalism  for  data  retrieval  and 
knowledge  representation.  The  model  tries  to  balance  constructivism  and  objectivism  as  instructional  design 
paradigms,  content-treatment  and  aptitude-treatment  interaction  as  didactic  approaches,  system-  and  user  locus 
of  control  in  human-computer  interaction,  and  preferential  versus  remediation  matches  to  individual  differences. 
The  article  starts  with  a short  description  of  the  SMILE-Maker  tool,  providing  the  context  that  calls  for  a new 
instructional  model  and  than  an  agent-based  instructional  design  model  (4  AID)  is  presented. 

SMILE-Maker  Tool 

SMILE  stands  for  Solution  Mapping  Intelligent  Learning  Environment.  The  SMILE-Maker  tool  consists  of  the 
components  such  as  ‘ideas’,  ‘profile’,  ‘map’,  ‘method’,  ‘template’,  ‘partner’  and  ‘draw’.  'Ideas'  means  creative 
problem  solving  techniques.  'Profile'  is  associated  with  individual  differences.  'Method'  proposes  a cognitive 
mapping  strategy  based  upon  synergy  of  problem  solving  techniques  and  mapping  approach.  'Map'  includes 
types  of  maps  such  as  concept  map,  mind  map,  process  map,  and  flowscape.  'Template'  provides  with  some 
examples  produced  in  a wide  range  of  particular  subject  domains.  'Draw'  leads  to  a graphical  editor.  ‘Partner' 
proposes  a strategy  and  a convention  for  group  problem  solving  using  cognitive  mapping  method. 

Three  different  scenarios  can  be  recognised  in  SMILE-Maker  tool  - instruction  scenario,  adaptation  scenario  and 
laissez-faire  scenario.  With  instruction  scenario  a user  follows  a general  predefined  path.  Each  information  unit 
(map  information  collection,  map  idea  generation,  map  idea  selection  and  map  idea  implementation)  is  studied  in 
a fixed  order  of  instructional  events  - explanation,  example,  procedure,  practice.  Adaptation  scenario  create 
opportunities  for  user  to  identify  her-  or  himself,  for  example,  as  a type  of  learning  style,  and  than  to  go  to  the 
particular  direction  determined  by  user  preferences.  It  is  expected  that  different  users  have  tendency  to  one  of  the 
instructional  events  - explanation,  example,  procedure,  and  practice.  Laissez-faire  scenario  implicitly  enables  a 
user  to  construct  own  learning  environment  selecting  different  components  and  available  instructional  events  in 
unique  order.  It  is  supposed  that  user  will  create  own  cognitive  mapping  method  combining  in  an  idiosyncratic 
manner  the  different  SMILE-Maker  components.  The  three  scenarios  might  be  distinguished  according  to  a 
disjunctive  theoretical  framework  consisting  of  two  dimensions: 
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• Objectivism  versus  Constructivism  educational  paradigms 

• Content-Treatment  Interaction  versus  Aptitude-Treatment  Interaction  didactic  approaches. 

Within  objectivism/constructivism  paradigm  an  additional  dimension  of  system  control  versus  user  control  in 
human  computer  interaction  can  be  recognised  and  preferential  match  versus  remediation  match  to  individual 
differences  makes  a part  of  content-treatment/aptitude  treatment  interaction. 

The  three  scenarios  have  their  either  advantages  or  disadvantages.  They  dispose  themselves  on  the  continuum  at 
the  one  extreme  of  which  is  laissez-faire  scenario  and  at  the  another  pole  is  instruction  scenario.  Adaptation 
scenario  takes  a middle  position.  Laissez-faire  scenario  is  with  predominantly  constructivistic  and  aptitude- 
treatment  interaction  orientation  with  user  locus  of  control,  while  instruction  scenario  has  more  objectivistic  and 
content-treatment  interaction  direction,  with  system  locus  of  control.  Adaptation  scenario  is  more  objectivistic 
than  constructivistic  and  it  is  more  close  to  aptitude-treatment  interaction  than  to  content- treatment  interaction. 
The  control  is  located  mostly  on  the  system. 

A need  for  designing  a new  versatile  scenario  within  SMILE-Maker  has  appeared.  The  versatile  scenario  is 
based  upon  the  4-AID  instructional  design  model  as  an  intelligent  agent  is  assigned  to  take  the  very  important 
role  of  facilitator  when  trying  to  accomplish  a balance  between  objectivism  and  constructivism,  content- 
treatment  interaction  and  aptitude-treatment  interaction,  system  control  and  user  control,  preferential  match  and 
remediation  match. 

4- AID  Model 

Generic  Agent-based  instructional  design  model  consists  of  four  sub-models:  user  (U),  facilitator  (F),  content 
(C),  and  instructional  events  (IE)  (Fig.l). 

Learner  sub-model  is  defined  by  four  learning  style  components:  activist  (A),  reflector  (R),  theorist  (T)  and 
pragmatist  Learning  style  is  a construct  to  refer  the  people  preferences  to  the  way  of  approaching  to  a 

learning  task  and  how  they  behave  in  a learning  environment.  Honey  and  Mumford  (Honey,  Mumford  92) 
defined  the  learning  styles  of  activist,  reflector,  theorist,  and  pragmatist  in  relation  to  the  four  stages  of  learning 
cycle  - experiencing,  reviewing,  concluding  and  planning.  Learner  sub-model  is  a set  of  learning  style 
components  (LS  = {A,  R,  T,  P})  reflecting  the  behaviour  of  the  user. 

Instructional  events  are  presented  as  explanation  (Exp),  example  (Ex),  procedure  (Pro),  and  practice  (Pra).  The 
explanation  provides  information  what  is  the  cognitive  mapping  approach  and  the  rational  behind  it  in  the  terms 
of  facts,  concepts,  principles  etc.  Examples  require  some  templates  of  different  cognitive  maps  to  be  available. 
Procedure  explains  in  the  step-by-step  format  how  to  construct  a sequence  of  maps  - information  collection  map, 
idea  generation  map,  idea  selection  map  and  idea  implementation  map.  Practice  is  associated  with  opportunities 
to  build  up  some  skills  in  cognitive  mapping  technique.  The  large-scale  complexity  of  the  instructional 
approaches  might  be  reduced  to  those  four  events:  IE={Exp,  Ex,  Pro,  Pra}={IEi  I ie[l,4]}.  They  represent  all 
variety  of  the  treatments.  Different  combinations  of  events,  according  to  the  stylistic  preferences,  enhance  the 
probability  to  design  an  effective  and  efficient  instruction. 

Content  C={MinP,  Mwg,  M^s,  Midi}  is  a set  of four  information  units  concerning  a cognitive  mapping  strategy: 
map  information  collection  (MinpX  map  idea  generation  (M^g),  map  idea  selection  (Mids),  and  map  idea 
implementation  (M^i).  Map  information  collection  is  purposed  to  assemble  all  available  information  in  problem 
space.  Map  idea  generation  is  aimed  at  generating  as  many  problem  solutions  as  possible  . Map  idea  selection 
finds  the  best  candidate  among  the  alternatives.  Map  idea  implementation  operationalises  a selected  solution  in 
the  terms  of  sequence  of  events  and  activities. 


There  is  not  enough  room  to  go  in  details  with  discussion  what  is  the  relationship  between  learning  styles,  cognitive  styles, 
cognitive  controls,  abilities,  personal  traits  and  why  learning  styles  have  been  chosen  as  representative  of  individual 
differences  of  learner.  Learning  style  is  a psychological  construct  including  itself  abilities,  cognitive  controls,  cognitive 
styles,  personal  traits  and  prior  knowledge. 
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These  sub-models  are  closely  related  to  each  other  by  the  behaviour  of  the  user  who  influences  the  behaviour  of 
the  system  facilitator.  Every  user’s  action  concerns  the  sequence  of  instructional  events  and  content  units 
depending  on  the  user  learning  style.  When  the  user  takes  an  action  Uactions  ^ IE={IEj  | ieN,  i€[l,4]}  =>  Faction  ^ 
{identifies  LSy,  ij  € N and  ij  € [1,4]}  U (lE/IEi)  it  is  in  respect  to  the  selecting  of  an  instructional  event  that 
stimulated  facilitator  F to  come  in  action  and  identify  user’s  learning  style  and  make  suggestions  to  him/her  for 
further  IE  sequence.  User’s  action  could  be  also  in  respect  to  content  units,  Uactions  ^ C=(IUi  | ieN,  Ie[l,4]}  => 
Faction  € (identifies  LSy,  i,j  e N and  iJ  e [1,4]}  U {lU/IUi}  that  also  stimulates  facilitator  F to  come  in  action  in 
a corresponding  way. 

Each  learning  style  manifests  the  subject’s  preferences  to  one  of  the  instructional  events.  The  theorist  is  very 
likely  to  attached  him-  or  herself  to  the  explanation.  Reflector  should  feel  comfortable  with  example.  Pragmatic 
should  choose  procedure,  and  activist  should  go  directly  to  the  practice.  The  four  preferences  could  be 
reformulated  as  explanation  type,  example  type,  procedure  type  and  practice  type.  This  might  provide  more 
room  for  flexibility  in  the  attempt  to  deal  with  the  great  diversity  and  complexity  of  the  situations.  Learning 
style  is  a valid  psychological  construct,  but  it  is  not  fixed  to  a person,  a situation  or  a task.  One  can  be  theorist  in 
one  situation,  or  reflector  in  another. 

It  is  also  assumed  that  the  user  will  demonstrate  some  preferences  to  the  different  content  units.  The  theorist 
likes  information  collection  map,  activist  has  strong  bias  to  idea  generation  map,  reflector  is  in  his  or  her  own 
with  idea  selection  map,  and  pragmatist  has  a talent  to  idea  implementation  map.  We  could  reformulate  those 
tendencies  as  information  collection,  idea  generation,  idea  selection  and  idea  implementation  preferences  that 
opens  the  door  for  more  flexibility  as  well  as.  The  instructional  events  and  the  content  are  stable  factors  that  can 
accumulate  the  learning  styles.  Or  put  it  in  another  way,  individual  differences  are  defined  in  the  terms  of 
instructional  events  and  the  content. 


Figure  1:  Agent-based  Instructional  Design  Model  (4-AID  Model) 

The  behaviour  of  the  facilitator,  from  the  methodological  point  of  view,  is  premised  upon  the  assumption  that 
everybody  has  a potential  to  be  complex  and  flexible  in  style  manifestations,  and  if  it  is  so,  the  issue  is  just  to 
provoke  ("wake  up”)  the  personal  capabilities  potential.  Facilitator  adds  flexibility  as  creating  a new  versatile 
style.  It  is  done  as  it  capitalises  on  the  strong  points  and  eliminating  the  pathologies  of  any  particular  style 
instead  to  adapt  the  instruction  to  the  user's  stylistic  preferences  (Stoyanov  97).  The  agent-based  facilitator  (F)  is 
patterned  by  a Master  Performer  Concept  Map  (MPCM)  which  combines  knowledge  of  content,  instruction 
events  and  user’s  model.  Facilitator  monitors  user  recognising  behaviour  patterns  before  reacting  on  them.  This 
is  not  only  pattern  retrieval,  but  also  a pattern  assembly.  The  facilitator  identifies  the  gaps  between  Master 
Performer  Concept  Map  (‘what  should  be’)  and  particular  user’s  model  (‘what  is’)  presented  also  in  concept 
mapping  format.  What  happens  is  that  user  behaviour  is  a stimulus  for  facilitator  to  trigger  pattern  retrieval  on 
the  master  concept  mapping.  If  the  user  demonstrates  learning  style  preferences  to  explanation  type,  than  the 
facilitator  completes  the  processes  spreading  activation  to  example,  procedure  and  practice.  Or  if  the  user 
demonstrates  a pragmatic  learning  style  and  starts  with  procedure  keep  staying  on  that  strategy,  than  the 
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facilitator  provides  opportunities  for  practice,  example  and  explanation.  Thus,  the  user  is  free  to  select  the  first 
instructional  event  (constructivism,  aptitude-treatment  interaction,  learner  locus  of  control,  preferential  match). 
At  the  same  time  he  or  she  is  recommended  to  take  benefits  from  the  v^hole  format  containing  the  other 
instructional  events  as  the  order  is  not  a substantial  factor  (objectivism,  content-treatment  interaction,  system 
locus  of  control,  remediation  match). 

The  facilitator  imposes  the  control  on  the  meta-level  as  well.  If  the  user  is  mostly  affected  by  instruction 
scenario,  than  the  facilitator  will  provide  with  some  suggestions  how  to  become  more  open  to  the  opportunities 
for  constructing  own  learning  environment  and  own  cognitive  mapping  method.  If  the  user  is  determined  only  to 
play  freely  with  the  SMILE-Maker  components,  than  the  facilitator  will  pop  up  advises  for  more  structured 
approach.  The  facilitator  also  might  identify  the  user’s  preferences  to  the  learning  after  an  inventory  or  a 
questionnaire  has  been  filled.  It  prescribes  a particular  learning  path  accompanied  with  some  suggestions  how  to 
compensate  the  style's  weaknesses. 

The  working  cycle  behind  4-AID  model  involves  user  action  in  respect  to  the  presented  content  and  instructional 
events.  Facilitator  recognises  and  analyses  user’s  behaviour  and  reacts  to  it  with  a specific  proposal.  This  cycle 
could  be  repeated  on  every  step  in  the  user-system  interaction  (Fig. 2). 


Figure  2:  Concept  map  based  diagnostic-treatment  cycle  of  the  agent-based  facilitator 

Facilitator  possesses  characteristics  of  a Monitor  agent  and  Helper  agent.  As  a monitor  it  is  operating 
autonomously  watching  for  a specific  event  to  happen  (gaps  between  Master  Performer  Concept  Map  and 
particular  user  model).  As  a helper  it  performs  autonomously  instructional  or  content  actions  in  correspondence 
with  the  occurred  events.  After  proposing  the  ‘treatment’  for  the  detected  problem  it  performs  automatically 
analysis  of  the  situation  and  monitors  for  other  evens.  In  other  words,  agent  F is  monitoring  the  behaviour  of  a 
particular  user  Ui  and  based  on  the  concept  map  knowledge  tries  to  identity  user’s  Ui  learning  style  Lsj  = {lEy  | 
i,jGN[l,4]}.  The  activation  of  facilitator  F is  calculated  by:  Faction  = Dmin  {Ci , lEy},  where  Dmin  is  the  minimum 
distance  between  the  Master  Performer  Concept  Map  (‘what  should  be’  stage)  and  particular  user  model  (‘what 


Figure  3:  System  Architecture 

We  refer  to  intelligent  agents  technology  as  a proper  one  to  fulfil  the  tasks  of  the  facilitator  in  this  system.  Agent 
technology  enables  instruction,  user  and  content  concepts  to  be  directly  represented  in  software.  Proposed  agent- 
based  facilitator  meets  the  dynamic  and  sophisticated  requirement  in  respect  to  user  modelling  in  the  process  of 
providing  instructional  material.  With  intelligent  agents,  instructional  processes  and  user  behaviour  are 
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modelled,  so  that  if  the  processes  change  this  can  easily  be  reflected  in  the  software  by  changing  the  interactions, 
or  at  most  the  combination  of  objects.  The  actual  objects  remain  unchanged  (Aroyo,  De  Diana,  Dicheva  98). 
Agent  technology  is  our  best  candidate  because  of  the  properties  of  agents  that  make  them  different  from  the 
other  type  of  software  entities.  We  present  some  of  the  relevant  for  the  facilitator  case  internal  agent  properties 
(from  the  ‘internal  being’  of  agent)  that  determine  their  actions  (Brenner,  Zamekow,  Wittig  98).  Although  we 
describe  a single  agent  system  in  the  moment,  we  would  like  to  leave  the  space  open  for  employing  new  agents  if 
necessary.  In  this  respect,  we  are  focusing  on  some  of  the  external  properties  of  agents  that  affect  the  interaction 
with  other  agents  and  human-agent  interaction.  Autonomy  is  one  of  the  basic  agents  properties  and  presents  the 
ability  of  the  agent  to  follow  its  goals  autonomously  without  interactions  or  commands  from  the  environment. 
The  intelligence  of  the  agent  is  formed  from  three  main  components:  internal  knowledge  base,  reasoning 
capabilities  based  on  the  content  of  the  KB  (semantic  networks,  neural  networks)  and  ability  to  learn/adapt  to 
changes  to  the  environment.  Agent  is  software  capable  of  reacting  appropriately  to  influences  or  information 
from  its  environment.  It  reacts  either  by  having  internal  sensor  or  by  possessing  internal  model  of  the 
environment.  Agent  is  also  proactive,  that  means  that  it  not  only  reacts  to  the  environment,  but  also  takes  the 
initiative  under  specific  circumstances.  The  fact  that  agent  can  be  proactive  and  take  the  initiative  requires  a well 
defined  set  of  sub-goals  within  the  limits  of  the  main  goal.  In  respect  to  multi-agent  systems  agents  also  can  co- 
ordinate,  co-operate  and  communicate  among  each  other  and  with  other  entities  of  the  system.  In  this  case  we 
are  interested  in  providing  agent  means  for  a single  agent  system  and  we  are  not  concerned  with  their  social 
abilities.  Based  on  all  this  agent  properties  we  believe  that  agent  technology  is  the  most  appropriate  one  for 
implementing  facilitator  behaviour. 

From  an  implementation  point  of  view  the  semantic  net  has  been  considered  as  the  most  direct  and  natural 
representation  of  knowledge  in  SMILE-Maker.  A semantic  net  is  a knowledge  representation  scheme  consisting 
of  a directed  graph  in  which  concepts  are  represented  as  nodes,  and  relations  between  concepts  are  represented 
as  links  (Lehmann  92).  The  analogy  of  a semantic  net  to  concept  map  is  straightforward  and  has  long  been 
recognised.  Semantic  net  can  be  used  as  a unifying  representation  for  a single  user  model,  content  units  and 
instructional  events  or  for  a group  of  users.  In  order  to  maintain  the  consistency  of  the  semantic  net,  rules 
governing  its  description  are  needed.  Two  classes  of  rules  can  be  defined;  (1)  graph-based  rules  for  enforcing 
structural  constrains,  and  (2)  semantics-based  rules  for  enforcing  semantic  constrains. 

The  operators  on  the  semantic  nets  can  be  generally  classified  into  two  categories:  semantic  net  manipulation 
operators  and  semantic  net  presentation  operators.  Semantic  net  manipulation  operators  include  creation, 
copying,  modification  and  deletion  of  nodes,  contents,  links  and  documents.  Semantic  net  representation 
operators  include  the  generation  and  display  of  hierarchical  views  and  semantic  net  views,  and  filtered  views 
(Wang,  Ghaoui,  Rada  97). 

At  that  stage  of  the  SMILE-Maker  development,  the  ’intelligence'  of  the  facilitator  is  restricted  to  the  master 
performer  model  (expert  model).  The  ambition  to  add  'more  intelligence'  in  facilitator  behaviour  should  be  done 
soon  when  the  'Partner'  component  is  going  to  be  designed.  The  idea  behind  'Partner'  is  to  build  a learning 
environment  for  creating  and  exchanging  cognitive  maps  in  group  problem  solving.  The  facilitator  will  take  a 
responsibility  on  the  behalf  of  the  user  for  initiating  a group  session,  for  searching  the  best  team  composition, 
controlling  group  dynamic  process  in  respect  to  rules  of  brainmapping  methods  (brainmapping  pool,  pin  maps, 
mapping  gallery  etc.),  brainmapping  data-  base  storage  and  retrieval.  It  is  expected  as  well  as  that  the  facilitator 
will  'learn'  during  this  process  and  will  adapt  its  behaviour  according  to  that. 
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Abstract:  The  arm  of  the  ECIC  (European  Continuous  Improvement  Circles)  project 
is  the  development  and  promotion  of  methods  of  group  learning  in  innovative 
organisations.  In  the  course  of  the  project,  project  members  realized  that  the  paper 
versions  of  the  manual  did  not  convey  one  of  the  basic  principles  of  the  ECIC 
methods,  that  is,  a high  degree  of  participation  and  user  activity,  in  an  adequate 
way.  Therefore,  an  electronic  version  of  the  manual  was  developed  which  is  more 
interactive  than  the  paper  version  and  which  describes  the  situated  character  of  the 
ECIC  methodologies  in  a better  way. 


1.  Introduction:  The  ECIC  project 

The  aim  of  the  ECIC  project  is  to  offer  public  and  private  organisations  and  structures  a set  of 
flexible  organisational  instruments,  which  contribute  to  enable  them,  not  only  to  react  upon 
approaching  changes,  but  also  to  use  the  new  options  in  a work  for  continuous  improvements.  The 
ECIC  project  has  thus  created  a "toolkit"  of  methodologies,  designed  to  support  SMEs  (small  and 
medium  enterprises)  and  SMOs  (local  and  regional  authorities,  communities  and  other  organisations) 
in  their  internal  capacity  to  deal  actively  with  development  and  innovation  within  a continuously 
changing  context.  A particular  attention  is  given  to  the  organisation's  capacity  to  profit  from 
developments  in  science  and  technology  and  specially  the  challenges  and  chances  of  IT  as  part  of 
innovative  solutions. 

The  central  element  in  the  ECIC  toolkit  is  the  Cl-Circle  (CIC)  - a model  for  groupbased 
participatory  learning,  problem  solving  and  change-management.  It  builds  upon  the  rich  experiences 
from  study  and  research  circles  (Eriksson  & Holmer  1991,  Karlsson  et  al  1991),  in  combination  with 
best  practices  from  continuous  improvements  (Cl)  work  in  both  the  private  and  the  pubHc  sectors. 
The  Cl  circle  relies  on  the  dynamics  of  the  group.  The  group  motivates  the  members  to  put  forward 
and  examine  developmental  issues  important  for  themselves  as  weU  as  for  the  organisation.  The 
search  for  relevant  and  valid  information  and  the  structuring  of  this  information  and  the  forming  of  a 
new  understanding  of  the  issues  mentioned  above  are  core  activities.  Traditional  ways  of  acquiring 
information  and  for  documentation  do  not  always  support  these  activities. 
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The  circle  methodology  has  so  far  with  few  exceptions  mainly  been  established  in  the 
Scandinavian  context,  where  it  forms  part  of  the  general  societal  culture  (Gustavsen  1980).  The  ECIC 
project  tries  to  disseminate  this  imminently  participatory  methodology  also  in  other  European 
countries  and  contexts  where  other  organisational  traditions  dominate. 

A general  dilemma  when  introducing  participatory  formats  into  more  vertical  organisational 
cultures,  is  how  to  do  this  in  a non-vertical  way.  One  of  the  consequences  of  using  linear  text  can  be 
that  authors  "dictate”  the  participation  through  very  detailed  instructions.  This  might  lead  to  more 
activity  but  hardly  more  genuine  participation.  Added  to  the  more  general  new  and  increased 
opportunities  that  ICT  offers  the  Cl-circles  in  their  daily  work,  the  use  of  inherently  activating 
hypertexts  in  manuals  etc.  with  a built-in  participatory  approach,  might  be  an  adequate  answer  to  the 
above  mentioned  dilemma. 


2.  The  interactive  approach  of  the  ECIC  methodology  and  the  electronic  manual 

The  electronic  version  of  the  ECIC  manual  tries  to  inform  about  the  basic  ideas  of  the  project  in 
a way  which  is  inspired  by  the  ideas  of  participation  and  openness.  The  electronic  manual  is 
interactive  and  highly  structured  to  support  active  forms  of  information  processing  with  the  users. 
These  properties  do  not  only  reflect  the  general  orientation  of  the  ECIC  project,  they  also  reflect  the 
results  of  recent  research  in  the  area  of  computer  supported  learning  (Kafai  & Resnick  1996,  Jonas  sen 
1996).  There  is  evidence,  that  active  involvement  of  users  and  intensive  processing  of  the  material  is 
superior  to  the  more  traditional  approach  of  merely  presenting  the  material  to  the  users.  People  who 
are  interested  in  the  ECIC  methods  can  browse  the  system  freely  at  their  own  speed  and  according  to 
their  own  needs.  They  can  explore  the  examples  given  in  the  program  and  try  to  find  out  the 
relationship  of  these  example  to  the  more  abstract  and  theoretical  discussions. 

In  this  context,  it  is  also  important  to  make  the  structure  and  the  interrelatedness  of  the 
material  transparent  (see  Fig.l).  Users  should  be  able  to  look  at  the  contents  of  the  subject  matter 
from  different  points  of  view.  In  the  electronic  ECIC  manual  there  is  another  page  compareable  to  Fig. 
1 which  shows  the  relationship  between  the  study  circle  concept  and  the  other  methodologies  from 
the  point  of  view  of  the  other  methodologies  - how  they  use  the  ideas  of  the  study  circle  method  in 
their  concrete  context. 

Another  important  aspect  of  modern  computer  supported  systems  of  instruction  is  the 
practical  relevance  and  the  situatedness  of  the  systems.  Real  insights  can  only  be  gained  by  discussing 
theoretical  results  in  the  context  of  real  and  relevant  situations.  This  is  in  accordance  with  the  ECIC 
approach  which  assumes  that  the  ECIC  methodologies  can  easily  be  learned  by  using  them  in 
concrete  situations.  To  support  this,  many  practical  examples  were  added  to  the  electronic  ECIC 
manual.  In  most  cases,  the  methods  from  the  ECIC  toolkit  are  open  for  creative  forms  of  application. 
The  ultimate  goal  of  the  ECIC  methodology  is  to  enable  the  users  to  find  their  own  solutions  for  the 
problems  rather  than  strictly  follow  any  guidelines.  The  ECIC  electronic  manual  can  give  a first 
introduction  into  this  kind  of  attitude  because  electronic  documents  convey  a sense  of  flexibility  and 
malleability  much  more  than  printed  text  which  is  always  authoritative  and  fixed  for  eternity. 


3.  Description  of  the  ECIC  electronic  manual 

In  the  ECIC  electronic  manual  we  tried  to  implement  the  desirable  properties  of  computer 
based  educational  systems  described  in  the  last  section:  showing  the  interrelatedness  of  the  material, 
supporting  the  active  involvement  of  the  users,  use  of  practical  examples.  In  the  following,  we  want 
to  describe  the  details  of  this  process.  It  is  important  to  mention  in  this  context,  that  the  project 
members  are  active  participants  in  the  development  of  this  system. 
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3.1.  Interrelatedness  of  the  material 


The  structure  of  the  ECIC  iriformation  system  is  a rather  complex  one  because  of  the  many 
interrelationships  between  the  methodologies.  In  this  sense,  the  topic  of  the  ECIC  project  is  probably 
an  ill-structured  domain  (Nix  & Spiro,  1990).  Ill-structured  domains  consist  at  least  partly  of  complex 
concepts  with  complex  relationships  between  them.  These  concepts  and  their  relationships  cannot  be 
defined  easily  because  of  their  dynamic  character.  In  this  case,  it  sometimes  helps  to  use  diagrams  to 
make  difficult  ideas  more  transparent  and  easier  to  understand.  In  the  course  of  the  project,  the 
diagram  which  is  shown  by  Fig.  1 was  developed  cooperatively,  and  it  is  still  not  "finished".  Project 
members  still  work  on  it  and  change  it.  Because  of  its  electronic  form,  this  is  easily  possible.  This 
diagram  is  a form  of  a "concept  map".  Concept  mapping  is  a method  to  support  learning  processes. 
The  presentation  of  concept  maps  is  a more  efficient  way  to  communicate  ideas  in  instructional 
situations  (Kajstura  et  al  1998,  Novak  & Go  win  1984,  Reader  & Hammond  1994). 


The  EClC-Toolkit:  Connections  between  methodologies 
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Fig.  1.  One  of  the  views  of  the  structure  and  interrelations  of  the  ECIC  methodologies 


3.2.  Active  involvement  of  the  users 

The  teamwork  game  (Fig.  2),  one  of  the  methods  of  the  ECIC  toolkit,  shows,  that  working  and 
playing  at  the  same  time  can  develop  a very  interesting  symbiosis  for  reaching  a company's  aim  in  a 
teamoriented  way.  The  philosophy  of  the  Teamwork  Game  is  that  ideally,  working  as  a team 
everyone  can  profit  from  each  other's  knowledge  and  skills.  Working  in  a team  is  a step  towards 
becoming  a learning  organization  and  achieving  a continuous  stage  of  development.  There  are  some 


basics  for  teamwork,  e.g.  "getting  to  know  each  other",  "developing  co-operation,  interaction  and 
commxmication  skills",  "becoming  aquainted  with  group  phenomena",  "practicing  anticipation, 
identifying  and  solving  problems". 

In  the  game  different  questions  are  put  forward  which  concern  the  above  mentioned  teamwork 
disciplines.  The  players  can  choose,  if  they  want  to  score  points  and  someone  wins  with  the  highest 
score  or  if  group  dynamical  processes  are  prefered.  The  multimedia  documentation  of  ECIC  shows,  by 
presenting  some  examples,  how  easy  playing  this  game  is.  Moreover  it  should  motivate  people  who 
are  a part  of  an  organization  to  play  it.  In  order  to  play  the  full  version  you  have  to  contact  the 
authors  responsible  for  the  Teamwork  Game  at  the  Helsinki  University  of  Technology.  During  the 
phase  of  implementation  the  question  arose,  if  it  is  possible  to  learn  teamwork-skids  by  using  a 
computer  program,  sitting  alone  in  front  of  the  screen  of  a machine.  But  in  our  opinion  teamwork- 
skids  are  not  developed  by  the  program  itself. 

The  object  of  the  computer  program  is  predominatly  to  encourage  people  to  reflect  on  their 
own  situation  in  an  organization.  This  accomplishes  the  goal  of  acquiring  these  skids  because  the 
game  enables  them  to  feel  responsible  for  solving  the  problems  or  at  least  wish  to  participate  and  thus 
knowing  that  they  are  solvable.  Using  this  multimedia  documentation  should  shift  the  players  focus 
on  real  situations  and  problems  in  an  organization,  which  have  to  be  dealt  with  in  real  Hfe  rather  than 
by  a computer  game. 
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Fig.  2.  The  gameboard  of  the  Teamwork  Game 


3.3.  Use  of  practical  examples 
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The  study  circle  method  is  described  at  great  detail  in  the  electronic  ECIC  manual  because  it 
forms  the  basis  for  all  the  other  methods  in  the  ECIC  toolkit.  The  basic  idea  of  study  circles  is  very 
simple.  A group  of  people  meets  regularly  to  learn  cooperatively.  Nevertheless,  it  is  very  difficult  to 
apply  this  meiliod  because  much  experience  is  necessary  to  make  study  circles  work  smoothly. 
Reality  is  often  richer  and  more  varied  than  any  manual  can  anticipate.  Therefore,  it  is  often  useful  to 
show  the  application  of  theoretical  principles  in  the  context  of  a concrete  example  (Kommers  et  al 
1996). 

Early  on  in  the  project,  a group  of  Spanish  female  entrepreneurs,  who  formed  a study  circle, 
complained  that  the  manuals  contained  no  practical  examples  showing  ways  how  to  implement  this 
method  successfully.  Therefore,  we  included  two  concrete  examples  of  study  circles  in  the  electronic 
ECIC  manual.  It  is  possible  to  look  at  the  more  theoretical  guidelines  for  running  study  circles  and  at 
the  concrete  examples  in  parallel.  There  is  a complex  web  of  links  between  the  guidelines  and  the 
examples.  In  this  way,  it  is  possible  to  jump  from  a specific  guideline  to  a description  of  the  concrete 
realization  of  this  guideline  and  back  again.  This  very  rigorous  use  of  the  concrete  examples  in  the 
electronic  ECIC  manual  also  showed  deficiencies  in  the  paper  version  of  the  manuals  and  led  to  a 
reformulation  of  these  manuals. 


4.  Conclusion 

The  ECIC  electronic  manual  differs  fundamentally  from  the  paper  version.  It  can  therefore 
provide  specific  support  for  people  who  are  interested  in  the  methods  of  the  ECIC  toolkit  (e.g.  study 
circles,  teamwork  game).  It  allows  users  to  browse  the  material  at  their  own  convenience  and 
motivates  people  to  develop  their  own  views  about  the  ECIC  methodologies.  It  supports  a more 
active  and  critical  approach  of  the  users  towards  these  methodologies.  Furthermore,  the  interrelated 
character  of  the  methods  can  be  represented  more  detailed  and  from  different  points  of  view  because 
interactivity  is  a main  focus  of  the  ECIC  electronic  manual.  The  electronic  manual  includes  several 
practical  examples  which  show  the  situated  character  of  the  various  methods.  In  this  way,  the 
electronic  version  of  the  ECIC  manual  can  motivate  users  to  a more  active,  critical  and  participative 
behavior  than  the  paper  version.  This  is  especially  important  because  it  corresponds  to  the  main  goals 
of  the  ECIC  methodologies. 


Acknowledgements 

Tlie  ECIC  project  is  financed  by  the  European  Conamission,  DG  XIII/ D 2. 


References 

Eriksson,  Kjell,  Holmer,  Jan  (1991).  Studiecirklar  sorn  stod for forandring  av  arbetslivet  Rapport  nr. 
1991:09.  Goteborg 

Gustavsen,  Bjorn  (1980).  From  satisfaction  to  collective  action:  Trends  in  the  development  of  reseach 
and  reform  in  working  Ufe.  Economic  and  Industrial  Democracy.  Volumel,  Number  2,  May  1980,  Sage 
Publications,  London  Beverly  Hills  and  New  Delhi 

Jonassen,  D.H.  (ed.)  (1996).  Handbook  of  Research  for  Educational  Comynunication  and  Technology.  New 
York:  Simon  and  Schuster  - Macmillan 

Kafai,  Y.,  Resnick,  M.  (eds.)  (1996).  Constructionism  in  Practice  - Designing,  Thinking,  and  Learning  in  a 
Digital  World.  Mahwah,  New  Jersey:  Lawrence  Erlbaum 


528 


Kajstura,  A.,  Romance,  N.,  Vitale,  M.,  O'Karma,  L.  (1998).  An  evolution  of  conceptmaps:  Computer- 
based  interactive  qualitative  and  quantitative  knowledge  maps  for  application  in  college  level  science 
courses.  In:  Th.  Ottmann,  I.  Tomek  (eds.):  ED-MEDIA  & ED-TELECOM  '98. 10th  World  Conference  on 
Educational  Multimedia  and  Hypermedia  & World  Conference  on  Educational  Telecommunications 
(Conference  Proceedings).  Freiburg,  Germany  June  20-25,  658-662 

Karlsson,  I.,  Karlsson,  L.,  Lundgren,  U.,  Neidel,  K.  (1991).  Studienzirkel.  Hippopotamos 

Kommers,  P.A.M.,  Grabinger,  S.,  Dunlop,  J.  (eds.)  (1996).  Hypermedia  Learning  Environments. 
Instructional  Design  and  Integration.  Mahwah,  New  Jersey:  Lawrence  Erlbaum 

Nix,  D.,  Spiro,  R.  (eds.)  (1990).  Cognition,  Education,  Multimedia.  Exploring  Ideas  in  High  Technology. 
Hillsdale,  New  Jersey,  Lawrence  Erlbaum 

Novak,  J.D.,  Gowin,  D.B.  (1984).  Learning  how  to  learn.  Cambridge,  London,  New  York:  Cambridge 
University  Press 

Reader,  W.  Hammond,  N.  (1994).  Computer-Based  Tools  to  Support  Learning  from  Hypertext: 
Concept  Mapping  Tools  and  Beyond.  In:  Computers  Educ.,  22,  99-106 


529 


Virtual  Reality  Model  Access  Project  (VR-MAP):  Helping  Individuals 
With  Disabilities  Develop  Social  Self-Sufficiency 


Dr.  Jane  Kaufman  Broida 

Professor,  Human  Performance,  Sport  & Leisure  Studies 
(303)  556-2978 
broida@mscd.edu 

Metropolitan  State  College  of  Denver 
Technical  Communications  Department 
Campus  Box  35,  PO  Box  173362 
Denver  CO  80217-3362  USA 


The  Virtual  Reality  Model  Access  Project  (VR-MAP)  plans  to  design  and  develop 
a prototype  for  the  application  of  virtual  reality  technology  to  the  enhancement  of 
the  social  self-sufficiency  of  individuals  with  physical  disabilities.  VR-MAP  will 
utilize  Quick  Time  Virtual  Reality  software  to  create  virtual  experiences  that  can 
be  computer  accessed  through  a web  page  on  the  Internet  or  through  provision  of 
free  CD-ROMs.  The  prototype  created  for  this  project  will  consist  of  20sites 
placed  into  virtual  environments.  Using  the  computer  mouse  or  an  adaptive 
device,  individuals  will  be  able  to  “navigate”  through  the  various  environments  or 
access  “hot  spots”  indicating  each  photographic  scene.  Annotations  about  access 
issues  will  be  provided  for  each  site.  Head-mounted  displays  (HMD),  hardware 
similar  to  goggles  with  small  computer  screens,  will  be  available  to  provide 
individuals  with  an  immersiye  experience.  A “head  tracking”  feature  on  the  HMD 
allows  the  user  to  look  at  a scene  by  simply  moving  the  head,  and  when  realistic 
audio  is  added,  the  user  can  be  provided  with  an  experience  that  simulates  reality. 

In  order  to  use  this  prototype  to  assist  individuals  with  physical  disabilities 
integrate  into  society,  certified  therapeutic  recreation  specialists  will  be  trained  to 
access  the  technology.  As  hospital  lengths  of  stay  become  shorter,  helping  clients 
ease  back  into  society  “virtually”  might  be  a viable  addition  to  traditional 
therapeutic  approaches  which  seek  to  maximize  self-sufficiency.  By  enhancing 
social  self-sufficiency  of  individuals  with  physical  disabilities,  it  is  anticipated 
that  carry-over  into  employment,  independent  living,  family  support,  and 
economic  independence  will  occur. 

The  promise  of  virtual  reality  (VR)  technologies  to  provide  powerful  and  unique  educational  and 
informational  experiences  is  well  known  (Heim,  1 998).  Unfortunately,  the  public  has  often  turned  away 
from  these  technologies  when  expectations  went  unfulfilled  due  to  hyperbole  on  the  part  of  the  media. 
Despite  setbacks,  however,  use  of  VR  is  growing  and  will  soon  become  commonplace,  according  to  some 
researchers  (Von  Schweber,  1 998).  Currently,  researchers  at  Metropolitan  State  College  of  Denver  are 
using  VR  to  assist  individuals  with  physical  disabilities  become  more  self-sufficient.  Specifically,  VR 
software  is  being  used  to  model  various  public  buildings  and  other  sites  that  individuals  with  disabilities 
may  want  to  visit.  Therapeutic  recreation  specialists  then  use  the  software  with  patients  to  allow  them  to 
virtually  preview  sites  that  they  may  later  want  to  tour  in  person.  This  approach  may  build  confidence  in 
the  clients  and  may  lead  to  greater  independence  once  they  reenter  society.  By  enhancing  this  social  self- 
sufficiency,  it  is  anticipated  that  carry-over  will  occur  into  employment,  independent  living,  family  support, 
and  economic  independence. 
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Abstract 


Need  and  Target  Population 

One  in  seven  Americans,  or  35  million  individuals,  have  a disability  severe  enough  to  interfere  with 
life’s  day-to-day  activities  (Scherer,  1996,  p.  4).  Of  those  individuals,  305,000  reside  in  the  City  of 
Denver,  Colorado  (United  States  Census  Report,  1990).  Providing  avenues  for  these  individuals  to 
maximize  their  full  inclusion  and  integration  into  varied  aspects  of  community  life  is  essential. 

Statistics  support  the  need  for  inclusive,  transitional  services.  Figures  indicate  that  over  1 ,500  youth 
with  disabilities  are  preparing  for  transition  into  the  community  from  the  public  schools  (Denver  Public 
Schools,  1994).  Furthermore,  clinical  agencies  (e.g.,  rehabilitation  hospitals)  in  the  Denver  Metropolitan 
area  have  over  1,600  new  patients  each  year  (Denver  Parks  and  Recreation  Special  Needs,  1996),  with  over 
200  referred  to  community-based  services  annually  (Denver  Parks  and  Recreation  Transition  to 
Recreational  Activities  in  the  Community,  1994,  1995). 

A number  of  factors  have  influenced  the  growing  need  for  integration  of  individuals  into  society.  With 
advanced  medical  care  available,  individuals  with  multiple  and  severe  injuries  are  surviving  traumatic 
accidents.  Additionally,  due  to  medical  advances,  individuals  with  chronic  illness  are  living  longer  and 
demanding  the  services  and  resources  to  enhance  their  quality  of  life. 

The  national  trend  of  hospital-based  services  toward  managed  care  has  also  impacted  the  need  for 
inclusive  services.  In  rehabilitation,  this  managed  care  includes  regulation  and  monitoring  of  patient 
lengths  of  stay  and  generally,  shorter  lengths  of  stay.  As  people  transition  back  to  the  community  in 
shorter  amounts  of  time,  providing  the  resources  for  their  full  inclusion  and  integration  into  society 
becomes  paramount. 

To  assist  individuals  with  disabilities  transition  back  to  their  home  environment  from  hospital  settings, 
healthcare  professionals  (e.g.,  physician,  physical  therapist,  recreation  therapist,  occupational  therapist, 
speech  pathologist,  social  worker,  and  case  manager)  work  cooperatively  to  plan  patient  treatment. 
However,  due  to  shorter  lengths  of  hospital  stays  and  the  need  to  manage  fiscal  resources,  patients  are 
discharged  quickly  after  achieving  medical  goals.  While  the  individual  with  the  disability  has  been 
medically  stabilized,  moving  that  person  to  successful  independence  in  the  community  might  not  have  been 
realized.  This  is  especially  true  for  individuals  with  severe  disabilities  that  utilize  adaptive  equipment  (e.g., 
manual  and  electric  wheelchairs,  and  walkers).  Not  providing  the  opportunity  to  experience  use  of  this 
equipment  outside  of  the  hospital  facility  prior  to  discharge  is  unfortunate  for  the  consumer.  In  fact, 
people  with  disabilities  have  expressed  their  fear  of  injury  in  new  environments  (Scherer,  1996,  p.  77),  and 
research  has  suggested  that  36%  of  the  women  with  physical  disabilities  that  were  surveyed  indicated 
unfamiliar  surroundings  as  a barrier  to  social  involvement  (Boyd,  Coyle  & Shank,  1998). 

To  mitigate  these  negative  influences  requires  new  approaches.  It  is  proposed  that  virtual  reality 
environments  be  created  to  help  transition  individuals  back  into  the  community.  Virtual  reality  integrates 
computer  graphics,  body  tracking  devices,  visual  displays,  and  other  sensory  input  devices  to  immerse  a 
participant  in  a computer-generated  environment. 

Thus,  the  Virtual  Reality  Model  Accee  Project  (VR-MAP)  will  attempt  to  answer  the  following 
question:  Can  virtual  reality  technology  be  used  effectively  to  assist  persons  with  physical  disabilities 
transition  from  clinical  therapy  into  the  community? 

Review  of  Literature 

Definitions 

The  concept  of  virtual  reality  as  we  know  it  today  was  developed  during  the  late  1 960s  by 
researcher  Myron  Krueger  (Heim,  1996).  Since  that  time,  VR  has  been  so  widely  used  that  that  it  is  nearly 
impossible  to  reach  a consensus  as  to  what  the  term  really  means.  Here  are  a few  examples: 

"(VR)  can  be  defined  as  a class  of  computer-controlled,  multisensory  communication 
technologies”— (Biocca,  1992,  p.  5). 

"Virtual  reality  is  an  alternate  world  filled  with  computer-generated  images  that  respond  to  human 
movements.  These  simulated  environments  are  usually  visited  with  the  aid  of  an  expensive  data  suit  which 
features  stereoscopic  video  goggles  and  fiber  optic  data  gloves"  (Greenbaum,  1992,  p.  58) 

"Virtual  reality  is  an  event  of  entity  that  is  real  in  effect  but  not  in  fact"  (Heim,  1996,  p.  16).  This 
definition  is  expanded  to  list  seven  concepts  that  currently  are  guiding  VR  research.  (1)  Simulation— use  of 
computer  graphics  and  digital  acoustic  space  sound  to  realistically  represent  an  object,  site,  or  other 
"reality";  (2)  Interaction-ability  for  the  user  to  interact  with  objects  and  other  representations  within  the 
artificial  world;  (3)  Artificiality— Extension  of  the  term  VR  to  include  everything  artificial;  (4)  Immersion— 
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ability  to  achieve  the  illusion  of  being  submerged  in  the  artificial  reality.  (5)  Telepresence-ability  to 
achieve  robotic  presence  at  another  location;  (6)  Full-body  immersion— use  of  data  suits,  gloves,  goggles, 
and  other  items  produce  a wide  range  of  sensory  representations,  including  visual,  aural,  and  tactile;  and  (7) 
Networked  communication— ability  to  achieve  all  or  some  of  the  above  concepts  over  a computer  network. 

Little  of  what  today  is  called  "virtual  reality"  includes  use  of  all  seven  of  these  concepts.  However, 
the  technology  is  being  used  successfully  today  in  numerous  applications  with  various  combinations  of  the 
seven  elements. 

Major  applications  of  VR 

In  addition  to  specific  health  care  applications,  which  this  paper  deals  with  in  depth,  VR  is  being 
used  in  a wide  variety  of  other  disciplines  and  industries.  Although  most  use  centers  around  information 
and  education,  the  telepresence  function  is  being  used  in  areas  where  hazardous  environments  and  remote 
locations  prohibit  use  of  humans  to  perform  the  tasks.  Examples  of  widely  disparate  uses  of  VR  today 
include: 

—Use  by  NASA  to  train  astronauts  in  International  Space  Station  operations  (Covault,  1998). 

—Use  by  companies  such  as  Motorola  to  train  assembly  line  workers  months  before  the  lines  are 
actually  completed  and  operating  (Minehan,  1996). 

—Use  by  NASA  to  realistically  portray  the  surface  of  Mars  as  a result  of  exploration  by  the 
Pathfinder  mission  (Jet  Propulsion  Laboratory,  1997). 

—Use  in  theatrical  productions  to  produce  more  complex,  elaborate,  and  realistic  sets  (Reaney, 

1996) 

-Use  in  the  courtroom  as  forms  of  evidence  (Dunn,  1997) 

Use  of  VR  in  Health  Care 

Virtual  reality  literature  also  discusses  many  benefits  for  patients.  Rothbaum,  Hodges  and  Kooper 
(1997)  reduced  the  fear  of  heights  in  patients  after  virtual  reality  exposure  therapy.  Wilson,  Foremen  and 
Stanton  (1997)  demonstrated  that  spatial  information  acquired  by  physically  disabled  children  from 
exploration  of  a virtual  environment  will  transfer  to  a real-world  equivalent  environment;  Ring  (1998)  has 
suggested  that  patients  with  disabilities  can  be  trained  with  virtual  reality  to  judge  architectural  barriers  and 
tackle  environmental  obstacles.  The  researcher  advocates  the  use  of  virtual  reality  as  a significant  assistive 
technology  in  the  future  (Ring,  1998).  This  clear  support  of  technology  deserves  the  opportunity  for 
development,  not  just  to  provide  individuals  with  simulated  experiences  but  to  use  the  technology  for  the 
integration  of  people  with  disabilities  into  society. 

Methods 

The  Virtual  Reality  Model  Access  Project  (VR-MAP)  is  designing  and  implementing  a prototype  for  the 
application  of  virtual  reality  technology  to  enhance  the  social  self-sufficiency  of  individuals  with  physical 
disabilities.  As  hospital  lengths  of  stay  become  shorter  for  patients  with  physical  disabilities,  helping  them 
ease  back  into  society  "virtually"  may  be  a viable  addition  to  traditional  therapeutic  approaches.  It  is  hoped 
that  this  enhanced  social  self-sufficiency  will  carry-over  into  employment,  independent  living,  family 
support,  and  economic  independence. 

The  prototype  created  for  this  project  will  consist  of  VR  models  of  20  public  sites  in  the  Denver, 
Colorado  area  that  individuals  with  disabilities  may  want  to  visit.  Examples  include  libraries,  airports, 
recreation  centers,  sports  stadiums  and  arenas.  In  addition,  other  facilities  frequently  used,  such  as  public 
transportation,  will  be  represented  virtually  as  well. 

Most  Effective  and  Appropriate  Technology 

Two  technologies,  Virtual  Reality  Modeling  Language  and  Quick  Time  Virtual  Reality,  were 
considered  to  accomplish  project  objectives.  Virtual  Reality  Modeling  Language,  or  VRML,  allows  three 
dimensional  structures  to  be  produced  from  blueprints  or  from  “scratch”  by  a graphic  artist.  Complex 
buildings,  objects,  and  other  elements  will  be  produced  using  this  technology. 

The  second  technology  considered  was  Quick  Time  Virtual  Reality  (QTVR),  a product  of  Apple 
Computing.  Instead  of  producing  models  from  blueprints  or  photographs,  this  technology  uses  actual 
digitized  photographs  of  existing  structures.  Multiple  photographs  are  “stitched”  together  to  produce  a 
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larger  panorama.  This  panorama  can  then  be  traversed  by  the  user  in  much  the  same  way  as  VRML. 

QTVR  also  allows  for  stereoscopic  head-mounted  display,  head  tracking,  and  sound. 

Both  of  these  technologies  were  pilot  tested  by  Dr.  Clark  Germann  and  Dr.  Jane  Kaufman  Broida, 
and  it  was  decided  that  QTVR  was  the  desired  and  most  effective  technology  to  use  in  developing  the  VR- 
MAP  prototype.  QTVR  can  be  viewed  by  users  with  Internet  access  on  the  World  Wide  Web  or  on  a 
stand-alone  computer  using  a CD-ROM.  Use  of  the  Web  allows  for  broader  and  more  convenient 
distribution  of  the  prototype. 

Two  means  are  available  for  the  user  to  "navigate”  through  the  various  scenes  of  buildings  and 
other  objects.  First,  various  scenes  are  linked  together  so  that  the  user  can  tour  a building  by  using  a series 
of  mouse  clicks.  Second,  a map  of  the  area  is  available  with  "hot  spots"  indicating  each  photographic 
scene.  By  clicking  on  a particular  spot  on  the  map,  the  user  is  quickly  transported  to  that  location. 
Additional  information  concerning  each  scene  is  "annotated"  into  the  margins.  These  annotations  provide 
additional  information  useful  to  persons  with  disabilities,  including  location  of  handicapped  parking, 
availability  of  electronic  doors,  and  location  of  elevators,  and  accessible  drinking  fountains. 

With  additional  hardware,  users  can  become  more  "immersed"  in  the  scene.  The  hardware 
consists  of  a head-mounted  display  (HMD)  that  is  similar  to  goggles  with  small  computer  screens  for  each 
eye.  The  HMD  allows  for  stereoscopic  viewing  of  the  scenes  that  add  to  the  realistic  effect.  In  addition, 
the  HMD  contains  a "head  tracker"  that  allows  the  user  to  look  around  a scene  by  simply  moving  the  head. 
A third  element  that  adds  to  the  immersive  experience  is  that  of  sound.  Realistic  audio,  such  as  that  of  a 
door  opening  or  a wheelchair  lift  rising,  completes  the  effect. 

This  technology  is  the  most  effective  and  appropriate  currently  available  considering  the  likely 
computer  equipment  and  Internet  connections  available  to  the  user  with  a disability.  QTVR  is  also  "cross- 
platform", meaning  it  can  be  used  on  PCs,  Macintoshes  and  other  types  of  computers.  Virtual  reality 
technologies  change  and  improve  rapidly  and  these  would  be  closely  monitored  during  the  course  of  the 
project.  If  better  technologies  emerge,  the  the  project  would  consider  taking  advantage  of  them. 

Developed  on  Sound  Conceptual  Model  & State-of— the-Art  Technology 

An  extensive  literature  review  was  conducted  into  both  the  available  virtual  reality  technologies  (i.e., 
VRML  and  QTVR)  and  use  of  virtual  reality  for  individuals  with  disabilities.  This  research  resulted  in  the 
formation  and  implementation  of  a pilot  project,  funded  through  the  Provost’s  Incentive  Grants  at 
Metropolitan  State  College  of  Denver  (MSCD).  The  pilot  project  tested  application  of  VRML  and  QTVR 
and  began  to  use  the  technology  for  the  integration  of  individuals  with  disabilities  into  the  community. 

From  the  pilot  project,  it  was  determined  that  QTVR  was  the  preferred  technology  and  that  using  virtual 
reality  did,  indeed,  positively  impact  the  transition  of  individuals  with  disabilities  into  the  community 
(Germann  & Broida,  1 998). 

QTVR  sites  were  researched  and  located  on  the  Internet.  No  specific  applications  were  found  that 
were  being  used  for  previewing  access  to  sites  and  locations  for  people  with  disabilities.  However,  many 
sites  were  found  that  provided  a preview  to  physical  environments  for  other  purposes.  For  example,  the 
Dallas  Cowboys  professional  football  team  (http://www.cowboys.com)  uses  QTVR  to  allow  prospective 
season  ticket  holders  to  view  the  playing  field  from  a variety  of  locations  with  the  stadium  prior  to  making 
a decision.  A number  of  home  builders  use  QTVR  to  allow  potential  customers  to  walk  through  new 
houses  before  they  are  constructed.  Volkswagen  uses  the  technology  to  allow  prospective  buyers  to  inspect 
new  models  on  the  Internet  prior  to  going  to  a dealership,  and  Purdue  University  provides  a virtual  tour  of 
their  campus  (http://www.tech.purdue.edu/resources/map).  It  is  interesting  to  note  that  providing  three- 
dimensional  experiences  are  becoming  more  common  place  in  society.  This  is  best  evidenced  by  the 
provision  of  3-D  glasses  in  the  August  issue  of  National  Geographic. 

Development  and  Testing  in  Appropriate  Environment 

The  virtual  environment  will  attempt  to  simulate  electronically  the  actual  environment  to  the  greatest 
degree  possible.  Technology  today  allows  for  the  use  of  photo-realistic  images  within  which  the  user  can 
navigate  to  some  degree.  By  using  the  mouse,  the  user  can  move  left  and  right,  up  and  down,  and  zoom  in 
and  out.  In  addition,  these  images  may  then  be  linked  to  other  images  located  nearby.  Development  will 
also  include  a map  of  the  facility  or  object  and  annotations  containing  information  useful  to  persons  with 
disabilities. 

The  technologies  and  techniques  for  allowing  the  individual  with  the  disability  to  preview  public 
buildings,  recreation  centers,  and  other  locations  prior  to  an  actual  visit  will  be  developed  and  tested  in 
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therapeutic  environments  including  rehabilitation  centers  and  community-based  settings.  Agencies  that 
have  provided  assurances  of  participation  include  Craig  Hospital,  University  Hospital,  M.S.  Adult  Day 
Enrichment  Program,  and  the  City  of  Denver  Parks  and  Recreation  Department:  Special  Needs  Program. 

Dr.  Dennis  J.  Matthews,  medical  director  of  the  Rehabilitation  Center  of  The  Children's  Hospital,  has 
provided  a letter  of  support  for  the  project. 

Herrick  and  Associates  will  be  contracted  to  test  the  prototype  with  the  target  markets.  They  will 
conduct  the  alpha  and  beta  testing  with  each  user  group,  varying  the  applications.  Further  qualitative  data 
will  be  gathered  from  a Consumer  Advisory  Council  and  the  therapeutic  recreation  specialists  using  the 
prototype  with  their  clients. 

Assessment 

The  research  question  for  this  project  is  as  follows:  Can  virtual  reality  technology  be  used  effectively  to 
assist  persons  with  physical  disabilities  transition  from  clinical  therapy  into  the  community?  To  assist  in 
answering  this  question,  patients  who  have  used  VR-MAP  will  complete  questionnaires  and  respond  to 
interview  questions.  The  questionnaire  consists  of  78  items  in  the  following  categories:  background 
information  and  VR  technology  preference;  importance  of  VR  features;  assessment  of  leisure  participation. 

Interview  questions  will  be  individual  and  site  specific  and  will  follow-up  on  questionnaire  items. 


Results 


Since  this  is  a work  in  progress,  questionnaires  and  interview  summaries  had  been  neither 
completed  or  analyzed  at  the  time  of  this  writing.  However,  anecdotal  evidence  gathered  in  testing  phases 
of  the  project  has  been  positive. 

1.  The  VR  software  was  initially  tested  with  one  of  the  project  coordinators  Will  Spence.  Spence  is  a 21- 
year-old,  student  at  Metropolitan  State  College  of  Denver  who  uses  a wheel  chair  and  is  afflicted  with 
multiple  sclerosis.  He  indicated  that  the  ability  to  virtually  tour  a public  facility  before  going  there  in 
person  could  help  reduce  his  anxiety  of  making  such  a trip. 

2.  The  program  was  also  shown  to  a group  of  30  persons  at  the  Rocky  Mountain  Multiple  Sclerosis  Center 
Adult  Day  Enrichment  Program.  Users  responded  very  positively  to  the  ability  of  VR-MAP  to  assist  them 
in  visiting  sites  throughout  the  Denver  area.  Specifically,  users  mentioned  the  usefulness  of  preparing  them 
to  navigate  older  sports  stadiums  that  were  not  constructed  with  access  features. 

3.  The  program  was  demonstrated  to  70  professional  recreation  therapists  at  the  American  Therapeutic 
Recreation  Association  1998  Annual  Conference.  Reaction  to  the  project  was  overwhelmingly  enthusiastic 
and  positive. 

Discussion 

The  idea  that  patients  can  be  assisted  through  the  use  of  artificially  created  worlds  is  indeed 
seductive.  Literature  reviewed  as  a part  of  this  project  indicates  that  the  technology  and  processes  are 
advanced  enough  to  meet  the  goals  set  forth.  However,  several  factors  need  to  be  considered: 

1 . No  literature  could  be  found  indicating  that  VR  has  been  used  before  for  the  specific  application  of 
assisting  individuals  with  disabilities  successfully  transition  back  into  society.  Thus,  no  predictions  of 
success  can  be  made  at  this  point  in  the  project. 

2.  Although  it  has  been  around  in  some  form  for  many  years,  VR  technology  as  defined  by  Heim  is  still  in 
its  infancy.  As  with  much  advanced  technology,  hardware,  software,  and  techniques  change  rapidly.  The 
technology  chosen  for  this  project— QTVR  and  VRML— could  quickly  become  obsolete. 

3.  Thus,  it  is  obvious  that  significantly  more  research  is  required  before  projects  such  as  VR-MAP  can 
become  truly  useful  and  helpful. 


534 


References 


Biocca,  F.  (1992)  Communication  within  virtual  reality:  Creating  a space  for  research.  Journal  of 
Communication,  (42)4,  5. 

Boyd,  R.,  Coyle,  C.,  & Shank,  J.  (1998,  September).  The  leisure  behaviors,  barriers  and  needs  of  women 
with  physical  disabilities.  Paper  presented  at  the  meeting  of  the  American  Therapeutic  Recreation 
Association,  Boston,  MA. 

Covault,  C.  (1998,  September)  Virtual  reality  utilized  in  station,  shuttle  ops.  Aviation  Week,  74-76. 

Denver  Parks  and  Recreation  Special  Needs  Program.  (December,  1994).  Year-end  report  of  the 
Transition  to  Recreational  Activities  in  the  Community.  Denver,  CO:  Author. 

Denver  Parks  and  Recreation  Special  Needs  Program.  (January,  1996).  Survey  of  Denver  agencies  which 
serve  people  with  disabilities.  Denver,  CO:  Author. 

Dunn,  J.  (1997).  Virtual  reality  evidence.  Available  http://venable.com/litlab/fulldunn.htp,  July  28,  1997. 

Germann  C.,  and  Broida,  J.  (1998,  September).  Using  technology  to  enhance  leisure  education  outcomes. 
Paper  presented  at  the  meeting  of  the  American  Therapeutic  Recreation  Association,  Boston,  MA 

Greenbaum,  P.  (1992,  March)  The  Lawnmower  Man.  Film  and  Video,  9(3),  58-62. 

Heim,  M.  (1996).  The  essence  of  VR.  In  V.  Vitaniza  CyberReader.  Boston:  Allyn  and  Bacon,  16-30. 

Heim,  M.  (1998).  Virtual  Realism.  New  York:  Oxford  University  Press. 

Jet  Propulsion  Laboratory  On-line  (1997).  Available  http://mpfwww.jpl.nasa.gov.  July  29. 

Minehan,  M.  (1996,  August)  Virtual  reality:  The  next  step  in  training.  HRMagazine,  144. 

Reaney,  M.  (1996,  Winter).  Virtual  sceneography.  TD&.T,  32-36. 

Ring,  H.  (1998).  Is  neurological  rehabilitation  ready  for  ‘immersion’  in  the  world  of  virtual  reality? 
Disability  and  Rehabilitation,  20(3),  98-101. 

Rothbaum,  B.O.,  Hodges,  L.,  & Kooper,  R.  (1997).  Virtual  reality  exposure  therapy.  Journal  of 
Psychotherapy  Practice  and  Research,  6,  219-226. 

Rothbaum,  B.O.,  Hodges,  L.F.,  Kooper,  R.,  Opdyke,  D.,  Williford,  J.S.,  & North,  M.  (1995). 
Effectiveness  of  computer-generated  (virtual  reality)  graded  exposure  in  the  treatment  of  acrophobia. 
American  Journal  of  Psychiatry,  152(4),  626-628. 

Von  Schweber,  L.  & E.  (1998,  June).  Virtual  reality.  PC  Magazine,  186. 

Scherer,  M.J.  (1996).  Living  in  the  state  of  stuck:  How  technology  impacts  the  lives  of people  with 
disabilities  (2nd  ed.).  Brookline  Books:  Cambridge,  MA. 

U.S.  Department  of  Health  and  Human  Services  (1991).  Healthy  people  2000:  National  health 
promotion  and  disease  prevention  objectives.  Washington,  D.C.:  U.S.  Government  Printing  Office, 
DHHS  Publication  No.  (PHS)  01-50213. 

Wilson,  P.N.,  Foremen,  N.,  & Stanton,  D.  (1997).  Virtual  reality,  disability  and 
rehabilitation.  Disability  and  Rehabilitation,  19(6),  213-220. 


535 


Case  Tools  for  Organizational  Performance  in  Instructional 
Technology  Evaluation:  CREDIT  & IDEA. 


Mike  Dobson,  Bill  Hunter,  Janet  McCracken,  Larry  Wenger  & Tim  Buell. 
THE  LEARNING  COMMONS 
University  of  Calgary 
2500  University  Drive  N.W. 

Calgary,  Alberta,  CANADA,  T2N  1N4 
Telephone:  (403)  220-6498  Fax:  (403)282-3005 
Email:  mdobson@ucalgary.ca 


Abstract:  The  “lessons  learned”  project  uses  empirical  methods  to  support  quality 
improvement  for  several  technology  integration  projects  at  the  University  of  Calgary.  Meeting 
this  goal  led  to  several  challenges  and  discoveries  that  may  be  of  interest  to  similar  projects. 
The  project  began  with  the  expectation  that  some  of  these  challenges  may  be  met  with  case- 
based  performance  support  tools.  Two  case-based  tools  formed  the  technical  backbone  of  the 
project.  An  instructional  design  evaluation  agent  (IDEA)  guides  users  to  select  evaluation 
strategies.  The  IDEA  tool  is  embedded  in  a case  repository  of  evaluations  and  designs  for 
instructional  technologies  (CREDIT).  These  two  performance  tools  have  supported  around 
twenty  development  projects  over  an  eighteen  month  period.  Two  major  design  hypotheses 
drove  assessment  of  the  tools.  The  first  conjecture  has  implications  for  improved  efficiency  in 
organizational  learning  systems.  Developing  case  studies  for  case-based  learning  systems 
represents  an  overhead  in  the  development  process  and  can  be  very  time  consuming.  The 
CREDIT  case-library  is  constructed  from  documents  created  during  the  normal  development 
schedule.  Our  first  conjecture  was  that  storing  these  artifacts  would  present  less  time  outlay 
for  projects  than  building  post-hoc  case  narratives.  The  second  conjecture  was  that  the 
information  contained  would  be  more  representative  of  events  and  decisions  than  post-hoc 
case  stories.  Experimental  use  investigated  the  accessibility  of  “lessons  learned”  from 
repository  artifacts.  We  expected  that  retrieving  “lessons  learned”  from  these  artifacts  would 
be  difficult,  but  believed  experimental  use  would  clarify  any  special  requirements  for  retrieval 
mechanisms  and  perhaps  particular  information  designs  that  would  improve  users  access  to 
useful  case  elements. 


Evaluation  practice  and  the  case  model 

Case-based  repositories  for  learning  and  performance  support  represent  a promising  approach  to  some 
of  the  difficulties  of  organizational  and  workplace  learning  (Guzdial  et  al.,  1997;  Leake,  1996;  McCalla  et  al., 
1997).  Case-based  learning  deals  with  some  perceived  limitations  of  conservative  instructional  design  by, 
increasing  engagement  through  collaborative  group  work,  increasing  authenticity  through  focusing  on  real 
experiences,  supporting  knowledge  construction  through  reflective  practices,  and  improving  transfer  through 
encouraging  comparison  between  problems  and  cases  (Leake,  1996).  Case  repositories  have  been  successfully 
used  in  problem-based  learning  and  some  studies  even  show  learners  improve  their  argumentation  skills 
particularly  when  the  repository  is  linked  to  specific  support  for  reflection  (Kolodner  et  al.,  1997).  The  case- 
based  approach  may  be  superior  to  less  structured  resource-based  approaches.  Learning  in  a problem-based 
situation  from  disordered  materials,  for  example  from  the  internet,  requires  considerable  support  (e.g.,  Dobson 
& McCracken,  1 997).  Structuring  cases  so  that  learners  can  predict  length  and  outline  headings,  may  supports 
learners’  use  of  resource  based  information.  Providing  cases  in  a similar  template  is  the  simplest  use  of  case- 
based  learner  support. 

Workplace  and  organizational  learning  tasks  are  often  relatively  formalized  and  supported  by  pre- 
constructed forms  and  quality  assurance  manuals.  In  these  situations,  case-based  approaches  can  benefit  from 
more  sophisticated  fine-grained  case-based  reasoning  techniques  that  automate  some  of  the  case-related 
processing  (e.g.,  McCalla  et  al.,  1997).  These  systems  provide  a kind  of  just-in-time  support  for  learners  who 
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reach  an  impasse  in  their  work.  Accurate  case  comparison  allows  presentation  of  appropriately  similar  cases  to 
help  the  learner  overcome  their  immediate  problems. 

Learners  using  case-based  systems  without  these  support  features  have  sometimes  been  limited  by  their 
problem  solving  skills.  Even  when  cases  are  well  designed  and  relevant  to  the  target  problem,  learners  may  still 
need  support  to  help  them  manually  compare,  match  and  synthesize  cases  (Guzdial  et  al.,  1997).  Although 
forms  and  quality  manuals  mirror  a significant  proportion  of  organizational  effort,  a lot  of  work  in  organizations 
cannot  be  easily  systematized  in  this  way.  Computer-based  courseware  development  and  evaluation  is  one  such 
areas.  Although  several  projects  have  aimed  for  reliable  computer  based  courseware  production  models 
(Dobson,  Rada,  Chen,  Michailidis,  & Ulloa,  1993;  Kaye,  1993),  the  scope  for  improvement  and  variation  seems 
not  to  have  diminished.  The  processes  of  courseware  design,  implementation  and  evaluation  can  only  partly  be 
systematized.  The  current  development  practices  in  computer-based  learning  systems  at  the  University  of 
Calgary,  suggested  the  need  for  attention  to  evaluation  processes.  Enhancing  the  implementation  and 
understanding  of  evaluation  practices  for  technology  based  learning  systems  is  a high  priority  of  the  “lessons 
learned”  project.  Empirical  methods  of  evaluation,  particularly  those  intended  to  inform  design,  evidently  form 
an  important  part  of  an  organizational  learning  strategy. 

Evaluation  of  technology-based  learning  is  a good  example  of  a domain  undergoing  constant 
improvement.  This  alone  makes  provision  of  case-based  support  difficult  and  makes  fine-grained  case-based 
reasoning  support  (e.g.,  McCalla  et  al.,  1997)  extremely  difficult.  Universities  may  provide  the  additional 
challenge  that  researchers  from  different  disciplines  have  a variety  of  conceptions  about  evaluation. 
Conceptions  of  evaluation  vary  on  the  questions  it  can  address,  the  stage  of  development  when  it  is  best 
implemented,  the  data  to  collect,  and  which  methods  of  analysis  are  useful.  Established  research  methods  in 
each  discipline  loosely  reflect  assumptions  about  evaluation  methods.  Assumptions  are  also  sometimes  a 
reflection  of  dominant  methods  of  assessment  in  non-technology  based  learning.  In  fact,  several  schools  of 
research  have  developed  methods  that  can  be  used,  sometimes  with  modification,  to  support  improvement  of 
computer-based  learning  systems.  Market  researchers  pioneered  the  focus  group,  economists  created  the  cost- 
benefit  and  impact  analyses,  learning  sciences  created  the  phenomenographic  study,  program  evaluators  make 
extensive  use  of  surveys,  experimental  psychologists  have  championed  the  controlled  condition  study,  and 
computer  scientists  have  developed  several  methods  for  assessing  usability.  The  IDEA  system  is  designed  to 
support  users  in  their  selections  from  alternative  evaluation  approaches. 

The  IDEA  and  CREDIT  systems  are  one  part  of  the  “lessons  learned”  project.  Overall,  the  project  has 
roles  in  communication,  professional  development  and  training.  An  important  element  of  all  parts  of  the  project 
includes  the  improvement  of  learning  system  design  through  the  use  of  empirical  methods. 


The  instructional  design  evaluation  agent  (IDEA) 

Evaluating  technology  integration  systems  can  be  thought  of  as  a set  of  communication  and 
documentation  processes.  Activities  in  group  organizations  often  require  a sequence  or  collection  of  documents 
to  be  written  by  individuals  in  special  roles.  The  individuals  who  have  those  roles  must  also  have  required  skills 
to  adequately  complete  their  tasks.  This  approach  to  modeling  many  collaborative  work  processes  has  been 
developed  in  several  object  oriented  languages  including  AMIGO  (Advanced  Messaging  in  Group 
Organizations)  and  AME  (Activity  Modeling  Environment).  A review  of  these  and  other  object-oriented  group 
modeling  languages  can  be  found  in  (Hennessy  & Benford,  1989).  The  AME  and  AMIGO  languages  were  used 
to  model  several  aspects  of  collaborative  authoring  of  courseware  in  order  to  design  a distributed  support 
system  for  designers  of  computer-based  learning  (Dobson  et  al.,  1993).  The  IDEA  system  uses  a variation  of 
these  object  oriented  models  as  a simple  visual  language  to  demonstrate  evaluation  approaches  supported  by  the 
agent.  The  diagrams  are  also  used  as  an  interface  to  descriptive  materials,  document  templates  and  fully 
implemented  cases  stored  in  a repository. 

The  agent  currently  includes  descriptions  of  twelve  evaluation  protocols.  Each  is  a condensation  of 
research  from  several  sources  that  emphasizes  the  practical  steps  and  skills  needed  to  implement  a study  with 
the  protocol.  The  methods  come  from  many  different  disciplines  of  research  and  evaluation.  Some,  for  example, 
are  motivated  by  conversation  theory  (Pask,  1984),  including  a conversational  breakdown  protocol  (Laurillard, 
1987)  that  has  been  successfully  used  in  evaluating  distance  learning  environments  (Dobson  & McCracken, 
1 996).  Other  protocols  are  distinctly  qualitative,  including  a “connoisseur  method”  that  draws  on  “Arts-based” 
approaches  to  evaluation  (Eisner,  1991)  and  a version  of  the  “phenomenographic”  approach  that  draws  on 
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Marton’s  work  (e.g.,  Lybeck,  Marton,  Stromdahl,  & Tullberg,  1988).  Methods  developed  for  marketing,  such  as 
the  “focus  group”  protocol  (Flores  & Alonso,  1995;  Krueger,  1994)  can  be  useful  for  establishing  and 
elaborating  the  views  of  people  involved  in  technology  integration.  The  focus  group  protocol  is  included  in  the 
IDEA  tool. 

The  IDEA  tool  provides  users  with  templates  for  many  of  the  documents  typically  used  in  each  of  the 
protocols.  All  empirical  projects  that  use  human  subjects,  for  example,  have  certain  ethical  obligations  to  the 
university  . These  obligations  are  dealt  with  in  a proposal  to  an  ethics  committee.  Projects  must  describe  the 
proposed  experiment  and  provide  a copy  of  any  instruments  used.  Provisions  for  getting  the  informed  consent 
of  all  subjects  and  for  describing  the  nature  of  the  study  are  included  in  a template  proposal  to  the  ethics 
committee.  Since  most  of  the  study  approaches  use  human  subjects,  the  template  forms  an  important  component 
of  the  IDEA  tool.  As  completed  documents,  such  as  the  ethics  proposals  are  generated,  they  are  intended  to  be 
stored  in  the  case-repository. 
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Figure  1:  Left,  the  CREDIT  tool,  showing  tabular  view  of  project  progress,  and  right,  a page  from  IDEA 
showing  description  of  objects,  roles  & documents  in  the  focus  group  evaluation  protocol. 


The  initial  design  for  IDEA  included  a plan  to  elaborate  several  heuristic-conditional  rules  that  when 
implemented,  would  guide  project  directors  to  select  evaluation  strategies  best  suited  to  their  project  needs. 
Several  guiding  principles  were  considered.  For  example,  many  of  the  qualitative  approaches,  including  the 
“phenomenographic  interview”  can  only  produce  useful  data  once  the  development  has  reached  a suitable  stage. 
Other  evaluation  strategies  such  as  the  “connoisseur”  approach  can  determine  that  the  courseware  is  ready  for 
these  other  protocols.  Before  these  qualitative  studies  are  really  useful,  the  evaluators  need  to  know  that  users 
will  have  no  special  navigation  or  other  interface  related  problems.  These  qualitative  studies  can  capture 
learners’  subtle  misconceptions  about  a topic,  but  not  if  subjects  are  consumed  by  difficulties  operating  the 
software. 

Access  to  the  development  team  may  also  affect  selection  of  appropriate  methods.  Many  integration 
projects  choose  to  buy  ready-developed  courseware,  rather  than  develop  their  own.  Evaluation  methods 
designed  to  provide  information  for  improving  product  design,  may  be  less  valuable  in  these  cases  than 
approaches  designed  to  compare  competitive  products.  These  and  several  other  heuristic  rules  have  been 
tracked  during  face-to-face  interactions  with  the  development  projects  in  the  university  program. 
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A case-based  repository  for  evaluation  & design  of  instructional  technologies 
(CREDIT). 


Most  users  of  the  CREDIT  and  IDEA  systems  are  engaged  in  a technology  development  project  and 
are  beginning  to  plan  an  evaluation  study.  Users  can  benefit  from  reading  about  similar  projects  that  have 
gained  by  failure-driven  or  success-driven  learning.  Users  benefit,  more  directly,  from  seeing  worked  examples 
of  documents  that  they  are  responsible  for  writing.  Projects  that  have  implemented  the  same  kind  of  study,  but 
perhaps  in  a different  topic  discipline,  are  likely  to  contain  useful  information  that  can  be  quite  easily 
transferred  to  the  user's  particular  needs. 

At  least  two  factors  make  it  difficult  to  implement  all  desirable  case-based  functionality.  There  are  no 
pre-designed  forms  for  evaluation  of  technology-based  learning,  and  the  practice  of  evaluation  in  this  field  is 
constantly  under  revision.  Lack  of  a consistent  methods  of  technology  evaluation  leads  to  difficulties  in 
indexing  cases.  Only  loosely  structured  cases  would  normally  be  possible  for  similar  domains,  and  this  limits 
the  sophistication  of  possible  case-based  reasoning  approaches. 

The  CREDIT  tool  uses  a practical  and  potentially  valuable  indexing  system  that  depends  almost  solely 
on  normal  project  activities.  Project  development  requires  several  stages  of  effort  that  are  often  represented 
directly  by  a document.  Examples  include  the  project  proposal,  a description  of  course  objectives,  the  ethics 
committee  evaluation  proposal,  meeting  minutes,  instruments  used  in  evaluation,  interim  and  year-end  reports, 
external  audits  and  self  assessments.  The  CREDIT  architecture  makes  names  of  these  documents  stand  for  the 
case  outline  in  the  case  repository.  When  a case  is  retrieved  in  the  simplest  of  retrieval  strategies,  all  the 
documents  relating  to  a project  are  presented. 

A case-library  populated  with  this  historical  record  of  decisions,  hypotheses,  discoveries,  timing, 
researching  and  iterative  outcomes  holds  many,  if  not  most,  of  the  “lessons  learned”  during  a project's  life  time. 
It  is  possible  that  these  results  will  be  more  difficult  to  retrieve  from  such  a repository  than  from  a case  library 
built  from  hand  crafted  cases.  One  of  our  design  conjectures,  however,  is  that  this  approach  may  support  two 
major  advantages  over  other  case-based  approaches,  (1)  the  cost  and  consequent  likelihood  of  cases  being 
created,  and  (2)  the  accuracy  of  the  record. 

The  combined  CREDIT  and  IDEA  tools  are  designed  to  be  used  by  two  major  groups  of  users.  The  more 
obvious  group  are  project  developers  actively  working  on  evaluations  of  their  computer-based  learning.  These 
users  can  browse  any  of  the  pages  without  guidance,  retrieving  the  materials  as  simple  resource-based  support. 
Beyond  browsing,  the  evaluation  agent  provides  direction  to  appropriate  evaluation  models,  templates  and 
cases.  The  IDEA  tool  is  designed  to  prioritize  information  for  the  user  based  on  knowledge  about  the  relevance 
of  the  case.  Active  project  workers  may  also  see  which  documents  have  been  submitted  by  other  projects  in  the 
same  group,  although  they  are  not  able  to  read  those  documents  unless  the  owners  have  made  the  pages  public. 

An  evaluation  support  team  are  less  obvious  users  of  CREDIT  and  IDEA,  however  the  advantages  for 
management  and  organization  for  this  group  are  quite  considerable.  A tabular  view  of  project  progress  is  based 
on  our  process-oriented  approach  to  evaluation  (Dobson,  McCracken,  Hunter,  Gaines,  & Wenger,  1996). 
Projects  generally  pass  through  several  stages  of  development  before  completion.  The  exact  sequence  of 
milestones  may  differ  between  projects  and  may  or  may  not  be  accompanied  by  a formal  document.  A tabular 
view  of  documents  that  have  been  completed  gives  quick  progress  information  on  each  project.  Projects  faced 
with  immediate  document  needs  can  be  quickly  referred  to  the  author  of  a similar  document  in  another  project. 


Conclusions 

During  eighteen  months  of  preliminary  research,  development  and  implementation  the  project  has 
produced  several  valuable  insights  that  might  be  of  interest  to  others  considering  similar  projects.  These  insights 
will  partly  guide  the  next  stages  of  our  work  in  supporting  technology-based  learning  initiatives  in  the 
University  of  Calgary. 

Tracking  the  value  of  heuristic-conditional  rules  during  our  interaction  with  projects  revealed  an 
overarching  factor  important  in  methodology  selection  that  had  not  been  previously  considered.  Different 
departments  and  faculties  often  have  experience  with  their  own  preferred  research  methods.  Unless  the 
particular  discipline’s  research  methods  are  difficult  to  apply  to  technology  based  learning  (say  Historians),  this 
factor  influenced  preference  for  study  types  more  than  any  other.  Early  indications  suggest  that  researchers 
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without  applicable  research  methods  tend  to  think  first  of  survey  methods  like  those  found  in  “sociological”  and 
“program  evaluation”  disciplines. 

The  current  program  of  around  twenty  technology  development  projects  resulted  from  competitive  bids 
to  a central  funding  agency.  The  period  and  scale  of  funding,  was  in  the  majority  of  cases,  inadequate  to  take  a 
project  from  design  to  completion.  This  dependency  on  future  funding  appears  to  have  exaggerated  certain  roles 
for  our  “lessons  learned”  project.  More  than  half  of  the  twenty  projects  requested  support  and  cooperation  at  the 
proposal  development  stage.  Many  of  the  projects  have  been  slow  to  make  their  evaluation  plans  available  in 
the  semi-public  CREDIT  environment,  although  a handful  of  projects  have  regularly  met  to  discuss  their  project 
and  have  kept  and  submitted  meeting  notes  each  week.  Instruments  and  decisions  have  been  carefully  noted  and 
the  documents  produced  by  this  small  group  of  projects  will  be  used  to  determine  the  difficulty  of  retrieving 
“lessons”  from  these  files. 

The  IDEA  tool  has  been  used  as  a resource  centre  for  evaluation  planning.  Again,  more  than  half  of  the 
twenty  projects  have  made  extended  use  of  the  evaluation  planning  guides  and  templates.  General  reflections  on 
the  value  of  these  materials  suggest  that  they  are  not  yet  in  a form  suitable  for  projects  to  use  without  additional 
face-to-face  support.  Early  indications  suggest  these  evaluation  guidelines  represent  a profound  shift  of  thinking 
from  what  instructors  and  developers  understand  about  evaluation  methods.  Producing  the  on-line  support 
helped  condense  the  descriptions  of  protocols,  but  if  the  tools  are  to  be  used  as  independent  support,  they  will 
need  further  development. 

The  “lessons  learned”  project  will  continue  to  work  with  the  twenty  or  so  projects  in  the  program  and 
will  continue  to  develop  the  IDEA  and  CREDIT  tools.  After  the  experience  of  the  initial  eighteen  months,  we 
will  likely  spend  more  time  in  face-to-face  interaction  that  we  had  originally  planned. 
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Abstract:  Facing  rapidly  changing  market  conditions  new  requirements  for  information  and 
communication  technologies  have  arisen  in  the  last  years.  This  paper  presents  a new  approach  to  integrate 
computer  supported  learning  environments  and  workflow  management  systems  to  optimize  workflows.  The 
first  step  is  the  integration  of  learning  processes  into  workflow  models.  This  enables  an  easy  development 
of  explorative  learning  environments,  particularly  simulation  games  which  are  used  to  train  users  on  basic 
and  advanced  levels.  Solutions  developed  by  the  users  are  evaluated  and  transformed  into  process  models 
which  might  improve  current  processes.  The  presented  integration  approach  offers  important  synergy 
effects:  a substantial  improvement  of  the  individual  educational  level,  a new  concept  of  employee 
participation,  and  an  essential  increase  of  continuous  process  improvement. 


1 Introduction 

The  increasing  relevance  of  flexibility  and  adaptability  to  rapidly  changing  market  conditions  leads  to  distinctive  requirements  for 
information  and  communication  technology  as  well  as  for  organizational  concepts.  In  this  paper  we  focus  on  the  integration  of  Computer 
Supported  Learning  Environments  (CSLE)  and  Workflow  Management  Systems  (WfMS).  Both  are  influenced  by  the  requirements  of 
continuous  changes. 

On  the  one  hand,  WfMS  are  affected  because  these  systems  manage  and  control  the  processes  of  value  creation.  Obviously,  these 
processes  have  to  be  immediately  adjusted  according  to  required  changes.  On  the  other  hand,  it  is  essential  to  work  with  well-educated 
employees  who  are  able  to  carry  out  both  the  current  and  modified  processes.  For  this  reason  CSLE  are  so  important.  In  previous  work  we 
have  demonstrated  the  need  of  an  integration  of  WfMS  and  CSLE  (Adelsberger,  Komer,  Pawlowski,  98a,  Adelsberger,  Komer,  Pawlowski, 
98b).  In  this  paper  we  introduce  the  idea  of  an  integrative  approach  using  simulation  games  for  the  optimization  of  adaptive  workflows.  We 
demonstrate  the  particular  suitability  of  an  explorative  learning  environment. 

During  the  normal  use  of  WfMS  improvements  are  detected  and  realized  during  explicit  (internal  or  external)  projects.  This  procedure 
corresponds  to  adaptable  processes  or  workflows  in  contrast  to  a continuous  improvement.  Integrating  an  explorative  CSLE  into  the  WfMS 
enables  to  continuously  realize  hidden  optimization  potentials  exposed  during  the  learning  process  using  the  real  information  system  and  real 
data.  Thus,  the  integration  approach  offers  two  important  advantages:  an  substantial  improvement  of  the  individual  educational  level  and  an 
essential  increase  of  continuous  process  improvement. 

After  introducing  the  basics  of  WfMS  and  CSLE  we  present  our  approach  of  an  integrated  design  of  business  and  learning  processes. 
Finally  we  describe  the  constmction  of  an  explorative  CSLE  within  a WfMS  and  the  potentials  concerning  the  continuous  optimization  of 
workflow  models. 

2 Workflow  Management  Systems 

The  use  of  Workflow  Management  Systems  (WfMS)  improves  the  effectiveness  of  internal  and  external  process  management  by  realizing 
the  process  orientation,  supported  by  modem  information  and  communication  technologies.  The  Workflow  Management  Coalition  (WfMC) 
(Workflow  Management  Coalition,  96)  defines  a WfMS  as  “a  system  that  defines,  creates  and  manages  the  execution  of  workflows  through 
the  use  of  software,  running  on  one  or  more  workflow  engines,  which  is  able  to  interpret  the  process  definition,  interact  with  workflow 
participants  and,  where  required,  invoke  the  use  of  IT  tools  and  applications.” 

The  presentation  of  tasks,  the  respective  data,  and  applications  to  the  users  avoids  competence  conflicts  and  wrong  document  routing,  and 
reduces  the  time  to  get  into  the  new  task.  It  is  one  of  the  important  advantages  of  a WfMS  that  its  Run-time  behavior  exactly  matches  the 
defined  (and  ‘optimized’)  process  models.  Furthermore  existing  applications  and  information  systems  might  be  integrated  into,  a WfMS. 
Among  other  things,  this  results  in  a standardized  user  interface  and  the  avoidance  of  redundancies  (Becker,  Vogler,  Osterle,  98). 

According  to  the  Basic  Model  (Workflow  Management  Coalition,  96)  the  use  of  WfMS  covers  two  main  phases  - the  Build-time  and  the 
Run-time.  Within  the  Build-time,  i.e.  the  creation  of  the  respective  workflow  models,  different  workflow  types  are  created,  checked,  and 
administrated.  During  the  Run-time  control  functions  execute  the  workflows  by  interpreting  these  Build-time  definitions  and  models. 
Accordingly,  the  models  cover  the  following  fundamental  perspectives  (Bussler,  Jablonski,  96,  Jablonski,  95): 

Workflow  execution  means  to  carry  out  specific  tasks  according  to  the  modeled  manner.  In  this  sense  Workflow  Definitions  answer  the 
question  “What  has  to  be  done?”  by  defining  (Build-time)  re-usable  workflow  types.  For  the  execution  (Run-time)  workflow  instances  are 
created.  The  definition  of  these  types  is  hierarchical;  elementary  workflows  (the  lowest  level)  refer  to  Workflow  Applications  (see  below). 

To  provide  the  right  data  at  the  right  time.  Data  and  Data  Flow  Definitions  represent  the  data  exchange  between  the  workflows  on  all 
hierarchic  levels.  Control  data  and  workflow  relevant  production  data  influence  the  workflow  control  and  the  selection  of  roles.  In  contrast, 
production  data  do  not  have  any  influence,  as  these  are  not  managed  by  the  WfMS. 

Control  Flow  Definitions  determine  the  order  of  workflows  in  a range  between  absolutely  fixed  to  undetermined  or  unknown. 
(Khoshafian,  Buckiewicz,  95)  describe  a corresponding  workflow  classification.  Traditional  programming  constructs  are  used  and  combined 
to  specify  the  control  flows. 

The  Workflow  Application  Definitions  specify  interfaces  and  parameters  to  handle  automated  and  manual  applications.  The  functions  and 
tasks  themselves  are  not  implemented  in  the  WfMS  but  invoked  by  the  WfMS. 

Workflow  instances  are  assigned  to  actors  (human  beings,  machines,  or  information  systems).  Actors  are  described  by  resources  with 
certain  skills  and  competencies,  influencing  their  participation  in  a role.  The  Organization  Definition  refers  to  the  question  “Who  executes 
the  workflow?”  by  evaluating  the  members  of  the  required  role. 
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All  terminated,  interrupted,  and  active  workflow  instances  are  recorded  in  iht  Audit  Trail  Those  data  might  be  used  to  identify  the  actor  of 
a specific  operation,  for  recovery,  and  for  the  revision,  analysis,  and  re-engineering. 

To  meet  the  increasing  need  for  flexibility  (Allweyer,  Scheer,  95)  formulated  specific  requirements  for  the  creation  of  adaptive  business 
processes  to  enable  continuous  process  improvements  without  realizing  a long-term  and  expensive  Business  Process  Re-engineering  project. 
They  formulated  requirements  for  adaptive  business  processes  - requirements  which  are  fundamentally  fulfilled  by  WfMS. 

The  Build-time  definitions  (see  above)  and  the  recorded  Run-time  data  (Audit  Trail)  offer  a large  potential  to  continuously  adapt  the 
workflows  to  changing  needs.  Nevertheless  these  potentials  are  seldom  used  for  a continuous  improvement.  On  the  one  hand,  this  might 
result  from  missing  manpower  or  missing  skills,  because  workflow  models  normally  are  created  by  external  consultants.  On  the  other  hand, 
the  users,  apart  from  the  traditional  suggestion  scheme,  are  not  included  in  the  improvement  process. 

By  integrating  explorative  learning  environments  into  WfMS  we  obtain  a better  participation  of  the  users.  Thus  it  is  possible  to  expose  and 
use  hidden  optimization  potentials  for  a continuous  workflow  improvement. 

3 Learning  Environments 

Due  to  the  enormous  amount  of  publications  from  a variety  of  research  fields  and  perspectives  the  basic  terminology  of  Computer 
Supported  Learning  Environments  (CSLE)  differs  considerably. 

Computer  Assisted  Learning  (CAL)  denotes  the  conceptual  framework  and  methodologies  as  a base  for  learning,  assisted  by  computers. 
Although  the  exact  meaning  might  be  interpreted  slightly  different  several  terms  are  used  synonymously,  e.g.,  Computer  Supported  Learning, 
Computer  Assisted  Instruction,  Computer  Based  Education.  Therefore,  this  term  is  the  generalization  of  any  concept  and  method  of  learning 
using  information  technologies. 

A Computer  Supported  Learning  Environment  (CSLE)  denotes  the  realization  of  CAL  and  its  concepts  and  methods  through  information 
systems.  The  term  CSLE  therefore  is  a generalization  for  a wide  range  of  instances  and  classes  of  instances  of  educational  information 
systems  (Adelsberger,  Komer,  Pawlowski,  98b). 

According  to  (Kalkbrenner,  96)  Computer  Supported  Learning  Environments  might  be  classified  into  five  classes: 

Drill  & Practice  Systems  are  based  on  the  behavioristic  learning  approach.  Learning  contents  are  gradually  presented,  followed  by 
repeated  training  assignments.  Still  being  used  on  a regular  base  in  practice  this  approach  mainly  serves  for  repeating  or  enhancing  familiar 
learning  contents. 

Intelligent  Tutoring  Systems  denote  systems  transferring  knowledge  specifically  adapted  to  the  level  of  an  individual  user.  The  human 
teacher  is  substituted  by  a computerized  tutorial  function  guiding  the  learning  process,  answering  questions,  or  evaluating  the  user. 

Hypermedia  Systems  are  used  for  the  presentation  of  learning  contents.  Historically  developed  as  a means  for  structuring  text  based 
documents  by  links  and  relations,  recent  developments  integrate  complex  media  formats,  e.g.,  audio,  video,  and  graphical  elements. 

Cognitive  tools  are  “both  mental  and  cognitive  devices  that  support  guide,  and  extend  the  thinking  processes  of  learners.”  (Derry,  90) 
Therefore  those  systems  support  the  learner’s  cognitive  processes. 

Simulation  games  provide  a specific  task  or  problem  to  the  learner  and  a micro  world  for  independently  exploring  and  solving  this  task. 
The  goal  of  the  learning  process  is  to  acquire  complex  and  interconnected  learning  contents.  The  teacher  and/or  the  systems  supports  the  user 
by  coaching  the  learning  process. 

This  paper  deals  with  complex  learning  contents:  business  processes.  Owing  to  its  suitability  for  acquiring  complex  learning  contents  and 
complex  thinking  we  concentrate  on  simulation  games  (Graf,  92,  Schulmeister,  97).  Furthermore  our  approach  using  the  integrative 
development  of  WfMS  and  CSLE  is  particularly  suitable  for  a cost  effective  and  straightforward  development  of  simulation  games. 

4 Integration  of  Business  and  Learning  Processes 

The  following  section  is  divided  into  two  parts.  First  of  all  we  describe  the  utilization  of  WfMS  for  the  development  of  CSLE.  Especially 
for  the  development  of  a simulation  game  the  access  to  existing  process  models  is  essential.  Therefore  it  is  a prerequisite  for  the  integration 
to  add  learning  processes  as  an  equivalent  of  business  processes  to  the  workflow  models. 

The  second  part  deals  with  the  generation  of  a simulation  game  as  an  example  of  an  explorative  learning  environment.  As  a result  of  using 
the  simulation  game  new  and/or  alternative  solutions  to  the  given  task  might  be  determined.  These  solutions  contribute  to  the  optimization  of 
existing  processes.  The  results  are  transformed  into  a workflow  model  which  serves  as  a proposition  for  an  improved  process  model.  After, 
that  the  proposition  is  evaluated  and,  in  case  of  a positive  evaluation,  the  process  model  will  be  updated. 

4.1  Integration  of  Learning  Processes  into  Workflow  Models 

Traditional  WfMS  concentrate  on  the  co-ordination  of  workflows  directly  referring  to  the  production  of  goods  or  services.  They  are 
mainly  limited  to  the  core  business  of  the  enterprise.  Although  learning  processes,  i.e.  the  acquisition  of  a specific  skill,  are  a prerequisite  for 
the  successful  execution  and  realization  of  workflows,  process  models  rarely  cover  this  important  aspect.  As  a consequence  the  planning  of 
basic  as  well  as  advanced  training  is  mostly  independent  of  the  current  demand  of  adequately  skilled  employees.  Our  integrated  approach 
will  eliminate  this  weakness,  connecting  learning  and  business  processes  in  a common  model. 

Furthermore  teaching  and  learning  processes  have  to  be  integrated  into  existing  models  as  a prerequisite  for  generating  a learning 
environment  to  utilize  existing  applications,  models,  and  data.  A simplified  example  of  those  processes  is  shown  in  (Fig.  1)  and  (Fig.  2). 
Those  models  clarify  the  general  steps  of  leaming/teaching  processes  and  the  interfaces  between  components  of  CLSE  and  WfMS. 

As  a result  of  this  integration  the  learner  as  well  as  the  developer  are  enabled  to  work  in  their  usual  working  environment  (WfMS, 
applications).  They  can  easily  access  the  existing  applications,  models,  and  components. 
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Fig.  2:  Learning  Process 


4.2  Optimization  of  Workflow  Models  using  an  Explorative  Learning  Environment 

This  section  deals  with  the  generating  of  an  explorative  learning  environment  based  on  WfMS  components.  Furthermore  we  present  ways 
how  learning  processes  may  contribute  to  the  optimization  of  workflow  models. 

The  learning  content  is  the  mastering  of  a specified  task,  available  in  the  form  of  process  models.  Corporate  processes  are  rather  complex 
because  of  the  multitude  of  participants,  the  problem  structures,  and  the  adequate  use  of  several  applications.  Thus,  a simulation  game  as  an 
explorative  learning  method,  is  appropriate. 

On  the  basis  of  the  integrated  design  the  development  of  simulation  games  - based  on  the  WfMS  - is  possible  without  large  expenditure. 
The  learning  environment  consists  of  all  applications,  components,  and  the  functionality  of  the  real  system.  Obviously  the  learning 
environment  cannot  use  sensitive  data  to  ensure  that  the  daily  business  is  not  influenced.  The  real  history  data  recorded  the  Audit  Trail  are 
used  as  real  but  insensitive  data  for  the  learning  environment. 
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The  learner  learns  within  a well-defined  micro  world.  It  consists  of  one  or  more  sub-processes  extracted  from  the  process  model.  The 
interfaces  to  the  related  processes  outside  the  micro  world  have  to  be  defined.  The  corresponding  data  are  exchanged  with  the  related 
processes,  either  extracted  from  history  data,  or  automatically  calculated.  Alternatively,  an  administrator  or  coach  has  to  make  available  the 
interface  data  during  Run-time. 

Furthermore,  several  constraints  have  to  be  cancelled  to  enlarge  the  learner’s  scope.  These  constraints  are  defined/modeled  in  the  real 
process  model  as  control  flows,  data  flows,  or  workflow  applications.  If  the  learner  violates  unavoidable  rules,  e.g.,  legal  regulations,  the 
learning  environment  must  point  to  the  consequences  of  this  acting.  The  cancellation  of  constraints  enlarges  the  scope  of  the  learner,  because 
his  actions  are  not  restricted  by  existing  process  definitions  and  the  prescribed  applications.  The  learner  himself  decides  about  the  tools, 
applications,  and  manner  to  solve  the  problem. 

During  the  learning  process  the  learner  produces  results,  i.e.  data.  These  have  to  be  evaluated  making  use  of  historic  data  to  determine  the 
learning  success.  Certain  deviations  in  the  results  should  be  allowed  within  certain  total  tolerances.  The  learner  can  access  the  WfMS  as 
specified  in  the  model  of  the  learning  process.  Thus,  the  actions  and  the  use  of  available  applications  within  the  micro  world  are 
approximately  unrestricted.  The  business  processes  and  the  learning  process  are  both  recorded  in  the  audit  trail. 

The  learning  results  have  to  be  evaluated  regarding  several  aspects;  At  first  it  has  to  be  discovered  whether  the  right  data  types  have  been 
modified.  Furthermore  the  modified  and  generated  data  values  must  be  within  the  total  tolerances.  If  the  realized  results  vary  from  the 
historic  data  because  of  significant  process  modifications  the  evaluation  has  to  be  checked  by  a human  teacher.  In  addition  individual 
evaluation  criteria,  e.g.,  the  duration  of  the  learning  process  compared  to  the  real  duration  of  the  task  have  to  be  taken  into  account. 

Besides,  apart  from  the  learner’s  evaluation,  the  recorded  actions  from  the  beginning  to  the  problem  solving  are  analyzed.  This  analysis 
serves  as  a basis  for  the  transformation  of  the  existing  model  to  a modified  one.  During  the  transformation  filtering  (Fig.  4)  redundant  and 
non-productive  learning  steps  are  eliminated.  The  remaining  steps  are  used  to  create  the  modified  process  model.  The  realized  steps  are 
modeled  as  workflows,  the  order  of  the  steps  influence  the  new  control  flows,  and  the  actors  involved  are  included  in  the  organization 
definition.  Afterwards  the  modified  model  is  evaluated  by  use  of  process  controlling  ratios.  In  case  the  modified  process  should  be  better 
than  the  existing  process  an  optimized  model  is  suggested.  The  suggestion  will  be  checked  and,  if  positive,  serves  as  a basis  for  the 
optimization  process  to  update  the  actual  process  model. 

In  the  following  figures  (Fig.  3)  and  (Fig.  4)  the  traditional  isolated  life-cycles  of  WfMS  and  CSLE  are  compared  to  the  life-cycle  of  the 
integrated  approach. 


Fig.  3:  Isolated  life-cycles  of  WfMS  and  CSLE  Fig.  4:  Integrated  life-cycle  of  WfMS  and  CSLE 


The  integrated  approach  offers  substantial  advantages  compared  to  the  isolated  approach;  The  conception  and  implementation  especially 
of  explorative  learning  environments  is  not  very  complex.  In  addition  the  planning  of  the  basic  and  advanced  training  is  considerably 
simplified.  Furthermore,  the  integrated  approach  may  increase  the  motivation  of  the  learners,  because  those  learning  processes  that  have  been 
successfully  completed  may  cause  respective  modifications  and  improvements  in  the  daily  business  processes.  Thus,  there  is  the  possibility 
that  the  employees  actively  and  creatively  take  part  in  the  design  of  their  working  environment  and  business  processes. 

Compared  with  the  traditional  suggestion  scheme  the  integrated  approach  does  not  demand  any  additional  work  on  behalf  of  the 
employees,  because  the  creation  of  suggestions  is  integrated  in  the  learning  process.  Besides  the  procedures  of  unprejudiced  and  neutral 
employees  might  be  more  creative  than  the  procedures  of  employees  having  performed  the  processes  for  a long  time. 


5 Summary 

In  this  contribution  we  described  the  integration  of  CSLE  and  WfMS  using  a simulation  game  for  the  optimization  of  workflow  models. 
As  a first  step  we  presented  an  integrated  modeling  of  business  and  learning  processes.  This  integrated  approach  significantly  decreases  the 
effort  to  develop  learning  environments.  As  an  additional  benefit  the  planning  of  basic  and  advanced  training  is  . 

We  presented  the  application  of  this  approach,  using  a simulation  game  which  can  be  easily  developed  basing  on  existing  processes, 
applications,  and  training  data  of  a WfMS.  On  the  one  hand,  this  method  is  particularly  suitable  for  learning  complex  problems,  e.g.,  the 
acquisition  of  knowledge  concerning  business  processes.  On  the  other  hand,  we  achieved  synergy  effects  for  the  optimization  of  business 
process  and  for  motivational  aspects.  Even  in  the  learning  phase  employees  are  involved  in  planning  and  improving  business  processes  as 
well  as  their  working  environment.  Compared  to  the  traditional  suggestion  scheme  this  method  of  active  participation  will  improve  both  the 
employee  motivation  and  effectiveness. 

The  presented  approach  is  implemented  at  the  University  of  Essen  as  a prototype.  During  the  next  phase  of  our  research  activities  we  will 
evaluate  the  results  by  an  empirical  study.  Furthermore  we  will  investigate  this  approach  from  different  perspectives,  using  varying  learning 
methods  and  learning  contents.  This  investigation  will  provide  in-depths  results  concerning  the  suitability  of  the  integrative  development  for 
specific  requirements  and  fields  of  applications. 
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Abstract:  University  and  college  administrators  intensely  push  for  increased  Web  integration  into 
lecture-based  and  other  types  of  academic  courses.  As  students’  competencies  with  Internet  usage 
increase  they,  too,  demand  educators  to  revise  their  educational  methods.  Some  faculty  members  do 
move  forward  pedagogical ly  in  the  age  of  high  technology.  However,  some  do  not.  Hence 
pedagogical  and  technological  gaps  between  traditional  and  multi-media  teaching  methods  appear. 
The  general  lauding  of  the  Internet’s  educational  benefits  comes  quickly,  perhaps  too  quickly  for 
the  comfort  of  some  in  academe. 

What  motivates  some  faculty  to  use  the  Web  to  deliver  and  to  display  course  materials  as  well  as  to 
provide  lectures?  Or  conversely,  what  inhibits  other  faculty  from  doing  so?  By  obtaining  analyzed 
quantitative  and  qualitative  data  from  faculty  members  from  all  disciplines  this  paper  starts  a cross- 
disciplinary  study  intended  to  answer  these  questions. 


Introduction 

Educators  first  utilized  the  Internet  to  communicate  with  fellow  scholars  and  to  gain  access  to  remote 
research  centers.  Recently  interest  in  the  Internet  and  the  World  Wide  Web  (Web)  becomes  intensely  more  popular 
as  use  increases  among  individuals  living  in  households  and  operating  businesses.  Riding  on  the  coat  tails  of  a non- 
academic  wave  of  the  Internet,  educational  applications  can  capitalize  on  the  tools  afforded  by  the  Internet. 

The  Internet  facilitates  learning  while  increasing  the  number  of  educational  opportunities  for  students. 
Introduced  to  the  Internet  in  grades  K-12,  collegiate  learners  naturally  utilize  the  Web  and  other  Internet  tools.  This 
interest  and  proficiency  coupled  with  the  increase  in  computer  ownership  and  access  by  students,  means  that  the 
Internet  can  become  a powerful  educational  tool  in  higher  education.  While  the  Internet  is  a significant  tool  that 
combines  delivery  medium  with  instructional  content  and  course  management,  effective  use  requires  careful 
consideration  (McManus  1996).  Universities  should  hesitate  giving  a Web  mandate  without  this  consideration. 


Literature  Review:  Diffusion  Patterns  for  Instructional  Technology 

Various  diffusion  models  explain  the  pattern  of  user  acceptance  of  a new  technology  over  time  in  hopes  of 
increasing  the  adoption  rate  (Brancheau  & Wetherbe  1989)  (Rogers  1995).  A simplified  "S"  curve  of  Web 
integration  (Fig.  1)  based  upon  Roger’s  (1995)  diffusion  model  illustrates  that  initially  a small  number  of 
enthusiastic  users  pave  the  way  for  the  next,  but  larger,  generation  of  faculty  adopters. 
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Proficiency  for  the  entry-level  Internet  integration  occurs  quickly  within  one  semester  instructors  can 
include  basic  Internet  options  and  establish  student  e-mail  communication  lines.  Typically,  the  next  step  involves  the 
creation  of  a Web  site  as  a means  of  pulling  the  techniques  together  and  managing  them  at  a central  location.  The 
early  pioneers  had  both  the  HTML  skills  and  desire  to  experiment  independently  with  Web  integration,  usually 
without  administration  incentives. 

Some  of  the  past  primary  drawbacks  to  extensive  use  of  the  Web  in  the  curricula  have  been  the  time,  effort, 
and  HTML  skills  required  of  instructors  (LeCroy  1997).  Today,  many  universities  employ  instructional  technology 
specialists  who  provide  training  and  purchase  Web  course  development  software  to  speed  the  process  of  Web  site 
development.  These  types  of  management  actions  overcome  major  hurdles  and  enable  the  large  second  wave  of 
adoption — the  vertical  portion  of  the  "S"  curve.  Shortly  after  this  point  in  the  diffusion  process,  the  "reluctance  wall" 
appears  and  the  number  of  users  remains  fairly  static,  suggesting  further  management  intervention  is  needed  if 
additional  adoption  is  to  occur.  References  to  these  issues  and  software  reviews  are  appearing  in  non-technical 
academic  journals  such  as  the  Chronicle  (McCollum  1997). 


Current  Study 

At  a selected  regional  liberal  arts  university  of  the  researchers,  this  current  study  surveys  faculty  to  identify: 
(1)  this  chosen  university’s  pioneers  and  second  wave  Web  adopters,  the  significant  motivational  factors  that 
influence  their  Web  use,  and  the  hurdles  they  face(d)  in  using  the  Web;  (2)  the  reasons  other  faculty  do  not  use  the 
Web  (reluctance  will  not  be  treated  derogatorily)  and  possible  motivators  for  nonuse;  (3)  patterns  in  faculty 
demographics  leading  to  adoption  or  non-adoption; 


Findings 

Of  179  faculty  responses,  three  groups  of  self-reported  Internet  usage  and  one  group  of  non-Net  users 
emerged.  The  user  groups  are:  (1)  Current — use  the  Internet  within  existing  courses;  (2)  Expected — plan  to  use  the 
Internet  in  courses  within  the  next  academic;  (3)  year;  (4)  Non-User — do  not  plan  to  use  the  Internet  within  courses 
within  the  next  academic  year. 

The  demographics  of  the  reported  Net  users  indicate  that  69  percent  of  the  current  users  are  tenured,  and  25 
percent  are  not.  However  when  one  compares  what  the  "non-tenured"  individuals  are  doing  with  the  Net  and  Web  to 
what  the  other  faculty  expect  to  do  and  what  they  do  not  expect  to  do,  the  percentage  of  non-tenured  faculty  actively 
using  the  Net  and  Web  is  slightly  higher  than  the  percentages  of  other  two  groups.  This  may  dispel  the  belief  that 
non-tenured  faculty  are  not  motivated  to  work  on  innovative  teaching  methods. 

99  percent  of  the  faculty  responding  to  the  surveyed  maintain  that  they  use  the  Net  and  Web  for 
professional  reasons  daily.  This  may  suggest  that  the  all  the  survey  participants  are  competent  users.  They  are  just 
not  necessarily  innovators  and  developers.  43  percent  of  the  survey  responders  maintain  that  they  do  not  use  the 
Net  and  Web  simply  because  they  do  not  add  anything  to  their  currently  effective  styles  of  teaching.  39  percent  of 
those  interested  feel  that  Internet  specialists  need  to  be  hired  to  assist  interested  faculty.  36  percent  are  not  educated 
on  how  the  Net  and  Web  can  be  used  effectively  in  their  classroom  environments.  Finally,  35  percent  of  the  faculty 
maintain  that  they  faced  more  formal  incentives  for  spending  time  bettering  their  classes  in  traditional  ways  than  for 
integrating  the  Net  and  Web  into  their  classes. 

A regression  analysis  was  tested  against  these  percentages  to  ensure  they  were  not  due  to  random  chance. 
With  the  exception  of  "lack  of  university  incentive",  the  t-statistics  of  all  else  is  significant  at  a 90%  confidence 
level.  Holding  all  else  constant,  this  suggests  that  NMU  faculty  decrease  the  number  of  Internet  features  used  by 
roughly  1 if  they  do  not  find  Internet  and  Web  assessment  tools  useful  [don't  like  e-assess]  or  do  not  know  how  to 
use  these  cybertools  in  their  educational  environments  [applications].  They  will  decrease  the  number  of  Internet 
features  used  by  roughly  2 if  they  find  that  their  teaching  styles  are  already  effective  [not  teach  style]  or  that  they 
spend  too  much  time  in  front  of  the  computer  [computer  overuse].  NMU  faculty  would  increase  the  number  of 
Internet  features  used  by  roughly  2 if  specialists  were  hired  to  assist  or  do  the  work  for  them  [need  specialist].  Lack 
of  university  incentive  [no  university  incentive],  though  shown  as  31-50%  in  the  graph,  proved  not  to  be  a valid 
statistical  factor  in  determining  Internet  use  within  a course. 

As  might  be  predicted,  current  users  have  overcome  many  of  the  hurdles.  The  experienced  group  identifies 
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three  problem  areas  and  appears  more  concerned  about  addressing  these  problems  than  the  less  experienced  groups. 
These  problem  areas  are:  Experienced  users  now  feel  they  spend  too  much  time  in  front  of  a computer,  that  an 
Internet  specialist  is  needed,  and  that  there  is  a lack  of  technical  support.  Non-users  need  more  information  on 
Internet  integration  before  they  are  apt  to  adopt  this  technology. 

Motivating  factors,  shown  in  Fig.  2 for  responding  faculty  currently  using  the  Net  and  Web  appear  more 
proactive  than  reactive.  The  current  users  appear  to  be  motivated  by  intrinsic  pressures  ranging  from  professionally 
driven  motivation  to  desires  to  improve  the  quality  of  their  courses,  enhance  communication  with  students,  and 
disseminate  course  information  and  materials  efficiently.  Newer  users  appear  reactive  to  pressures  from 
administration,  peers,  and  technological  advancements.  External  motivators  are  all  pressure  points:  from  students, 
from  marketplace,  from  peers,  from  technology,  from  administration.  NOTE:  it  is  possible  the  results  would  be 
different  if  the  survey  instrument  had  listed  motivational  factors  rather  than  requested  a written  comment. 


improve  quality  of  course 
delivery  & content 

improve  communication 

disseminate  course  info 
and  materials 

professional  growth  and 
interest 

encourage  self-learning 
by  students 


students  need  to  know 
this  tool 

pressured  by  peers  and 
administration 

access  and  distance 
education 

pressured  by  students 

pressured  by  technology 
advancements 


Figure  2.  Faculty  Motivators 

Adoption  Patterns 

Based  upon  studies  of  how  of  innovations  spread  in  their  use,  categories  of  adopters  have  been  identified.  The 
adoption,  or  diffusion,  of  an  innovation  typically  takes  the  form  of  an  S-shaped  curve  (Rogers  1995)  (Cottrell  1997). 
The  sample  university's  distribution  of  adoption  follows  this  pattern;  thus  sample  faculty  can  be  considered  "normar* 
in  their  approach  to  new  ideas.  The  position  and  numbers  of  current  adopters,  and  the  expected  77  adopters  within 
the  next  year  indicate  that  a critical  mass  has  been  reached.  A critical  mass  is  the  point  at  which  a new  idea  has  taken 
hold  and,  barring  unforeseen  circumstances,  will  eventually  spread  to  almost  all  members  of  a group. 
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S»Shaped  Adoption  Curve  for  NMU 


Figure  3:  Adoption  pattern  at  studied  institution 


Future  Research 

This  study  will  be  expanded  to  include  numerous  other  universities.  Internet  and  Web  policy 
recommendations  will  be  offered  to  university  participants.  Significant  national  trends  may  be  identified  as  the 
national  database  expands  and  survey  results  are  analyzed.  Institutions  interested  in  participating  should  contact  the 
authors. 
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Abstract:  This  session  will  demonstrate  the  results  of  two  years  of  summer 
workshops  designed  to  teach  K-12,  community  college,  and  CalState  instructors  how 
to  use,  and  how  to  motivate  students  to  use,  multimedia  and  the  WWW  in  their 
classrooms.  The  presenters  will  describe  the  training  paradigm  that  was  used  with 
both  groups  of  faculty:  K-12  instructors  and  university  professors.  In  both  1997  and 
1998  there  were  disparities  between  participant  and  instructor  expectations.  The 
biggest  problems  lay  in  the  initial  lack  of  student  understanding  of  the  complex 
nature  of  multimedia  design  and  execution,  the  amount  of  time  needed  to  devote  to 
the  construction  of  even  simple  multimedia  projects  and  the  steepness  of  the  learning 
curve  for  some  multimedia  tools.  These  training  workshops  were  organized  by  the 
Mathematics  And  Science  Teacher  Education  Program  (MASTEP),  a five  year  NSF 
sponsered  consortium  to  improve  mathematics  and  science  in  the  greater  San 
Francisco  Bay  Area. 


Theoretical  Framework 

According  to  Fryatt  (1995),  the  challenges  of  training  teachers  to  use  multimedia  and  to  integrate 
multimedia  and  the  WWW  in  their  classrooms  are  not  easy  ones.  In  her  review  of  the  literature,  she  found  that 
traditional  in-service  instruction  in  the  use  of  technology  is  not  effective  because  there  is  little  active 
involvement,  inadequate  practice,  and  little  or  no  follow  up  and  support.  Ferrier  (1997),  based  upon  his  review 
of  the  current  literature  on  effective  adult  learning  strategies,  believes  that  changes  in  teaching  practice  will 
happen  only  if  professional  development  workshops:  emphasize  a commitment  to  try  out  new  ideas  in  the 
classroom  the  next  day,  provide  a supportive  environment  to  discuss  problems  and  solutions  and  report 
successes  and  failures,  encourage  participation  in  a wide  variety  of  teaching  and  learning  approaches,  and 
reflect  the  needs  of  the  school  communities.  Szabo  and  Schwarz  (1997),  in  a study  of  three  randomly  drawn 
samples  of  1000  full-time  teachers  in  Alberta  Canada,  identified  opinions  of  teachers  that  might  influence  them 
to  make  greater  use  of  computers  in  their  classes.  When  the  teachers  were  asked  who  should  use  computers 
with  students,  71  percent  of  the  teachers  responded  " only  those  teachers  interested  in  doing  so."  In  addition,  99 
percent  of  the  teachers  said  that  the  ideal  number  of  minutes  per  week  of  computer  use  by  students  should  fall 
between  30  minutes  and  eight  hours. 

Models  of  Education 

The  focus  of  these  summer  MASTEP  workshops  was  on  training  teachers  to  incorporate  technology 
into  their  classrooms.  Our  focus  (and  mantra)  was  to  encourage  authentic  uses  of  technology  rather  than 
traditional  uses  of  technology.  According  to  an  Office  of  Educational  Research  and  Improvement  case  study, 
reported  by  Barbara  Means  and  Kerry  Olson  (1994),  traditional  uses  of  technology  include:  technology  as  a 
teachers  presentation  tool,  technology  for  remedial  instruction,  and  teaching  students  about  technology.  In 
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contrast,  authentic  learning  focuses  on  the  student  practicing  advanced  computer  skills;  in  addition,  these  types 
of  technology  uses  in  education  are  generally  collaborative  with  teachers  as  coaches  and  feature  extended  time 
for  completing  assignments.  According  to  Singh  and  Means  (1998),  authentic  uses  of  technology  can  have  a 
significant  impact  as  a catalyst  for  changing  schools  in  ways  that  better  support  the  acquisition  of  higher  order 
skills.  According  to  Brown,  Collins,  Duguid  (1989),  authentic  or  contextual  learning  increases  the  likelihood 
that  students  can  apply  what  they  learn  in  school  to  similar  contexts. 

The  learning  approaches  emphasized  by  the  summer  MASTEP  workshops  were  resource-based 
learning,  collaborative  learning,  and  project-related  learning.  As  another  study  (Bos  1997)  found,  having  K - 12 
students  create  their  own  projects  can  serve  several  purposes  within  a science  curriculum.  Through  WWW 
projects,  student  work  can  be  enhanced  and  the  authenticity  of  the  work  increased.  According  to  Harrington 
and  Oliver  (1995,  1997),  there  are  nine  critical  characteristics  related  to  situated  learning  environments.  The 
learning  environment  needs  to:  "provide  an  authentic  context  that  reflects  the  way  the  knowledge  will  be  used  in 
real-life;  provide  authentic  activities;  provide  access  to  expert  performances  and  the  modeling  of  processes; 
provide  multiple  roles  and  perspectives;  support  collaborative  construction  of  knowledge;  provide  coaching  and 
scaffolding  at  critical  times;  promote  reflection  to  enable  abstractions  to  be  formed;  promote  articulation  to 
enable  tacit  knowledge  to  be  made  explicit;  and  provide  for  integrated  assessment  of  learning."  (Harrington  & 
Oliver  1995,  1997) 

Results 

Placement  of  the  workshops  in  the  institutional  environment 

The  most  successful  attempts  to  equip  faculty  with  the  skills  and  knowledge,  as  well  as  equipment  to 
create  multimedia  based  learning  materials,  have  occurred  where  there  is  a institution-wide  effort;  as 
demonstrated  at  Pennsylvania  State  University  with  Project  Empower  (Noel  & Brannon  1997)  and  outlined  in 
the  TESSI  Project  adopted  by  the  Ministry  of  Education  in  British  Columbia  (Woodrow  et  al.  1997). 
Unfortunately,  not  all  higher  education  institutions  or  educational  agencies  have  arrived  at  this  juncture. 

The  MASTEP  summer  workshops  permitted  the  initiation  of  the  type  of  efforts  carried  out  by  Project 
Empower  within  a limited  framework.  Since  these  summer  workshops  could  not  provide  the  total  framework  of 
institutional  support,  as  provided  in  Project  Empower  for  faculty  trying  new  technology,  the  challenges  for  both 
the  workshop  participants  and  instructors  were  greater.  The  difficulties  for  both  participants  and  instructors  as 
well  as  some  of  the  compromises  which  had  to  made,  can  be  instructive  and  helpful  to  other  educators  in 
educational  institutions  where  a total  institutional  commitment  to  new  technologies  is  not  likely  to  made  in  the 
near  future. 

Summary  of  First  year  workshop 

A summary  of  feedback  from  the  initial  workshops  which  included  two  one-week  ‘"basic”  workshops 
and  two  three- week  “advanced”  workshops  indicated  that  the  participants  felt  that  they  received  valuable 
information  and  that  they  were  able  to  produce  multimedia  products  during  the  workshop  period.  The 
participants  who  entered  the  workshop  with  a clear  project  idea  were  most  able  to  take  advantage  of  the 
instructional  materials  presented  during  the  workshops  and  were  also  closest  in  their  performance  and  outcome 
expectations  to  those  of  the  workshop  instructors. 

The  workshop  instructors  identified  several  areas  of  concern  in  their  assessment  of  the  workshop. 

These  included  a lack  of  appreciation  by  a significant  number  of  the  participants  in  the  advanced  workshops  for 
the  complex  nature  of  multimedia  software  and  hardware.  In  effect,  these  participants  came  into  the  advanced 
workshops  with  the  expectation  that  they  could  create  “plug  and  play”  products  with  little  effort  on  their  part. 
These  same  participants  were  overwhelmed  by  the  expectation  during  the  workshop  that  they  would  actually 
have  to  create  a multimedia  product.  They  had  trouble  making  the  transition  from  passive  learning  ("chalk  and 
talk")  to  active  learning  (participation  and  creation).  Many  faculty  came  into  the  workshop  with  the  assumption 
that  it  would  be  a traditional  "chalk  and  talk"  even  though  they  were  informed  before  hand  that  they  would  be 
expected  to  design  projects.  For  many  students,  the  preconceived  notion  of  a "chalk  and  talk"  experience 
negatively  impacted  upon  their  ability  to  complete  a project  during  the  period  of  the  workshop.  In  this  finding. 
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our  results  agree  with  previous  research  done  by  Oliver  and  Lake  (1997)  who  found  that  students  in  an  authentic 
learning  environments  found  it  took  longer  to  learn  by  doing  then  to  learn  by  lecture  mode. 

Many  participants  in  the  basic  workshops  did  not  have  an  awareness  of  the  state  of  multimedia 
technology  prior  to  the  workshops.  They  would  have  benefited  by  having  more  time  to  be  “brought  into  the 
technology”  using  an  approach  similar  to  that  developed  at  Pennsylvania  State  University  through  the  Project 
Empower  program  (Noel  & Brannon  1997),  In  our  course,  as  in  the  one  by  Oliver  and  Lake  (1997),  using 
technology  seemed  to  hinder  learning  when  technical  problems  arose. 

Many  of  the  participants  in  the  advanced  workshops  also  had  trouble  devoting  enough  time  and  energy 
to  the  workshops  since  they  were  not  compensated  in  monetary  terms  nor  were  they  assigned  college  or 
continuing  education  credit  for  attending  the  workshops  except  for  the  K- 12  teachers.  The  higher  education 
instructors  from  the  state  and  community  colleges  were  taking  time  from  personal  projects  and  other  work 
projects  in  order  to  attend  the  workshops  at  cost  to  themselves.  Because  of  these  factors,  attendance  and 
attrition  during  the  advanced  workshops  was  a problem  during  the  latter  part  of  the  workshop  where  participants 
did  not  receive  formal  instruction  but  were  expected  to  work  on  individual  projects.  Some  higher  education 
instructors  looked  forward  to  release  time  during  the  next  academic  year  to  continue  the  multimedia  projects 
developed  during  the  summer  workshops  and  these  participants  were  more  motivated  to  complete  their  projects. 

Summary  of  second  year  workshop 

June  Higher  Education  Workshop 

In  all,  fourteen  faculty  participated  in  the  June  1998  workshop.  The  workshop  was  divided  into  four 
one  week  sessions.  The  first  week  focused  on  a pedagogical  and  philosophical  introduction  to  the  use  of 
computers  and  education.  Topics  in  the  first  week  included  an  overview  of  various  areas  related  to  the  use  of 
multimedia  and  education.  The  specific  content  for  this  first  week  included  the  philosophy  of  using  multimedia, 
teaching  styles  and  their  impact  upon  educational  technology  used  in  the  classroom,  using  the  WWW  in  your 
classroom,  an  overview  of  basic  instructional  models,  story  boarding  ideas  for  multimedia,  and  copyright 
guidelines  and  issues.  The  first  week  ended  with  demonstrations  and  self  paced  tutorials  of  three  authoring 
programs:  PowerPoint,  HyperStudio,  and  Authorware. 

The  second  week  focused  on  the  use  of  multimedia  in  the  classroom  and  the  third  week  on  the  use  of 
the  WWW  in  the  classroom.  These  two  topics  were  covered  in  separate  weeks  in  order  to  provide  more  time  to 
develop  educational  modules  that  maximized  these  different  types  of  multimedia.  The  last  week  in  the  June 
session  was  spent  on  special  projects. 

Since  these  workshops  were  sponsored  by  M A STEP,  the  participants  were  surveyed  by  external 
evaluator.  According  to  the  evaluation  summary,  the  participants  indicated  that  the  best  aspects  of  this 
workshop  were  the  emphasis  and  time  for  hands-on  experience,  the  tutorials,  the  excellence  of  resources,  and 
the  staff  technical  knowledge.  Additionally,  most  participants  wrote  that  they  could  use  what  they  learn  in  all 
courses  they  teach  for  creating  presentations  for  classroom  use. 

The  attendees  of  this  workshop  differed  in  many  respects  from  the  workshops  of  the  previous  year. 
Most  of  the  faculty  entered  this  session  with  highly  defined,  preconceived  ideas  of  what  they  wanted  to  do. 
Sometimes,  this  led  to  a conflict  between  teacher  and  student  expectations;  incidentally,  this  situation  was  not 
not  reflected  in  the  teaching  evaluations  that  were  excellent.  In  many  ways,  students  were  resistant  to  any 
suggestion  or  teaching  that  did  not  fit  into  their  paradigm  regardless  of  the  need  of  their  design.  For  example,  to 
engineering  faculty  members  attended  the  June  workshop  with  the  goal  of  designing  course  Web  pages.  These 
faculty  members  refused  to  participate  in  any  tutorials  or  class  discussions,  or  learn  any  software  , that  was  not 
directly  related  to  their  task  at  hand.  Although  this  goal  seems  acceptable,  this  resistance  precipitated  the  re- 
teaching of  many  concepts  and  software  throughout  the  four  weeks. 

July  K-12  Workshop 

The  July  workshop  session  was  structured  differently  than  the  June  workshop  in  some  respects.  This 
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session  was  again  divided  into  four  weeks.  However,  the  fourth  week  had  a different  focus  (the  first  three 
weeks  of  this  workshop  paralleled  the  content  of  the  same  weeks  in  the  June  workshop).  The  attendees  of  this 
workshop  were  all  K - 12  teachers  who  had  been  selected  to  receive  equipment  and  software  through  a proposal 
competition  run  by  MASTER.  The  teachers,  by  their  own  admission,  expected  that  the  workshops  would  follow 
a "chalk  and  talk"  format.  At  first,  the  teachers  were  dismayed  that  they  had  to  actively  learn  and  produce 
tangible  products.  However,  compliance  to  teacher  expectations  and  motivation  was  high.  Also,  the  students  in 
this  section  were  graded  on  the  quality  and  appropriateness  of  their  outcomes. 

Comparison  of  1997  and  1998 

Length  and  focus  of  workshop 

In  1997,  the  one- week  “basic”  workshops  were  well  attended  with  little  attrition  during  the  week. 
These  workshops  were  highly  instructor-directed  with  a mixture  of  lectures  and  structured  hands-on  activities. 
There  were  simple  step-by-step  tutorials  designed  to  give  a brief  introduction  to  various  software  packages. 

The  “advanced”  workshops  in  1997  also  initially  had  good  attendance  while  the  instructors  were  directing  all 
activities  during  the  first  week  in  a similar  format  to  that  used  for  the  “basic”  workshop.  However  the  number 
of  individuals  who  enrolled  for  the  “advanced”  workshop  was  lower  than  for  the  “basic”  workshop  due  to  the 
three-week  time  commitment.  During  the  second  and  third  weeks  of  the  “advanced”  workshops,  attendance 
was  sporadic  for  many  participants  who  were  not  fully  committed  to  working  the  full  workshop  time  on  open- 
ended  projects  in  spite  of  the  available  equipment  and  instructor  expertise. 

Based  on  the  1997  issues  about  attendance  for  the  “advanced”  workshops,  it  was  decided  in  1998  to 
shorten  the  workshops  to  one  week  modules.  Two  distinct  groups  of  instructors  were  also  identified  and 
presented  with  separate  workshop  schedules  and  activities  designed  to  address  particular  group  needs.  The  two 
groups  of  instructors  were  1)  higher  education  instructors  from  community  and  state  colleges  and  2)  K-12 
teachers  who  had  committed  to  being  club  advisors  to  science  student  clubs  at  their  respective  school  sites. 

For  the  higher  education  instructors,  the  workshops  were  divided  into  four  consecutive  one  week 
sessions  (discussed  previously).  The  focus  for  each  week  was  very  specific  with  extensive  resources  and  step 
by  step  tutorials  provided  to  the  participants  and  a project  expected  to  be  completed  by  the  end  of  the  four  week 
period.  All  participants  were  expected  to  complete  the  first  week  but  could  choose  which  other  weeks  they 
wished  to  attend.  These  participants  were  not  compensated  monetarily  but  some  community  college  instructors 
received  college  credit  hours  for  the  workshops. 

For  the  K-12  club  advisors,  the  focus  of  the  workshops  were  also  specific  but  the  participants  were 
expected  to  attend  all  four  weeks  of  the  workshops.  These  participants  received  college  credit,  monetary 
compensation  and  the  promise  of  a multimedia  station  at  their  school  sites  upon  successful  completion  of  the 
workshops.  The  focus  of  these  workshops  addressed  basic  multimedia  concepts  and  philosophy,  use  of 
multimedia  and  web  tools  and  the  design  of  multimedia  student  directed  activities  for  the  school  sites.  A key 
difference  was  that  these  K-12  teachers  had  to  produce  specific  multimedia  project  outcomes  in  Weeks  2 and  3 
and  group-designed  activities  in  week  4. 

Compensation  issues 

It  is  not  surprising  to  find  that  faculty,  that  are  more  vested  in  learning  multimedia,  produce  more 
tangible  outcomes.  A continuing  problem  in  providing  multimedia  workshops  for  faculty  is  the  subsequent 
ability  of  the  faculty  to  pursue  multimedia  on  their  own  after  the  workshop  ends.  Since  the  K - 12  faculty  in  the 
1998  section  were  given  equipment  as  well  as  financial  remuneration  for  their  attendance  and  participation, 
there  was  more  motivation  for  the  K - 12  faculty  to  produce  outcomes  in  the  workshop.  These  unsurprising 
results  was  also  found  by  Barnes  (1997).  In  this  study,  project  FIRST,  activities  and  projects  were  developed 
by  district  teams  consisting  of  a regular  educator,  a special  education  preservice  teacher  paired  with  a regular 
teacher  for  a year,  and  a university  faculty  member  in  special  education.  In  addition,  each  team  was  loaned 
technology,  both  hardware  and  software,  for  classroom  use  for  one  year.  Barnes  found  that,  after  six  months, 
participants  continued  to  integrate  technology  in  their  classrooms.  The  need  for  external  reward  also  has  been 
documented  by  Reiss  (1998),  in  his  analysis  of  project  T.E.A.C.H.  in  the  Camden  school  district.  New  Jersey. 
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In  project  T.E.A.C.H.,  48  of  the  original  50  teachers  finished  the  training  program  and  received  a horizontal 
step  raise.  In  subsequent  years,  the  external  motivation  was  that  every  teacher  would  receive  a Macintosh 
computer  for  the  classroom  about  halfway  through  the  program.  These  computers  would  remain  with  the 
teachers,  even  if  the  teachers  moved  to  other  rooms  or  other  schools  in  the  district.  Overall,  as  a result  of  this 
training,  the  Camden  school  district  now  has  more  than  250  T.E.A.C.H.  graduates  out  of  about  1750  teachers. 
More  importantly,  for  most  T.E.A.C.H,  graduates,  computers  are  used  in  the  classroom  on  a daily  basis. 

Instructional  pedagogy/philosophy 

In  the  first  year,  the  philosophy  was  integrated  into  the  workshops  in  concert  with  the  instruction  about 
the  multimedia  tools.  Although  the  first  week  was  advertised  as  a beginning  multimedia  workshop,  the 
instructors  discovered  that  few  faculty  considered  themselves  to  be  beginners  in  technology.  Therefore,  about 
half  of  the  faculty  attending  the  "advanced"  workshop  did  not  attend  the  preceding  "basic"  workshop.  This 
made  it  difficult  to  build  upon  skills  taught  in  the  first  week.  It  also  necessitated  re-teaching  of  various 
techniques,  philosophy,  and  multimedia  tools  in  the  second  to  third  weeks  of  each  session. 

In  both  1997  and  1998  there  were  disparities  between  participant  and  instructor  expectations.  The 
biggest  problems  lay  in  the  initial  lack  of  student  understanding  of  the  complex  nature  of  multimedia  design  and 
execution,  the  amount  of  time  needed  to  devote  to  the  construction  of  even  simple  multimedia  projects  and  the 
steepness  of  the  learning  curve  for  some  multimedia  tools.  Many  participants  came  in  thinking  that  all  they 
needed  to  do  was  to  assemble  ready-made  pieces  into  a whole.  There  was  resistance  to  the  realization  that  each 
piece  of  a multimedia  project  has  to  be  created. 

Another  expectation  among  the  higher  education  instructors  was  that  since  they  were  not  being 
compensated  that  they  could  pick  and  choose  what  they  wanted  to  learn  and  to  do  regardless  of  instructor 
opinion  and  expectations.  As  discussed  above,  two  participants  in  1998  only  wanted  to  create  web  pages  and 
only  worked  on  web  pages  with  virtually  no  attention  to  the  other  multimedia  possibilities  or  design 
considerations.  The  K-12  instructors  were  committed  to  (and  extremely  motivated  by  the  level  of  renumeration) 
attending  the  four  week  workshop  but  did  not  initially  understand  the  expectation  that  they  would  be  expected 
to  produce  multimedia  projects  and  that  they  would  be  graded  on  the  results. 

Use  of  multimedia  tools 

In  the  1 997  workshops,  the  participants  were  given  brief  tutorials  covering  various  authoring  software 
products  as  well  as  tutorials  on  how  to  incorporate  audio,  images  and  video  segments.  There  was  a great  deal  of 
latitude  and  one-on-one  instruction  with  respect  to  other  various  software  applications  used  to  manipulate 
images,  sound,  animation,  and  video  clips.  Participants  varied  considerably  in  their  relative  technological 
sophistication  and  experienced  some  fhistration  with  this  approach. 

For  the  1998  workshops,  the  participants  were  provided  with  step-by-step  tutorials  for  all  software 
applications  used  in  the  workshops.  There  were  specific  applications  (donated  to  the  M A STEP  project  by 
Adobe  Systems,  Inc)  used  for  image  manipulation  and  creation,  video  capture,  and  editing.  Three  authoring 
application  choices  were  offered  for  general  multimedia  projects  and  two  web  based  authoring  tools  were 
offered  for  creating  web  pages.  The  participants,  who  had  little  familiarity  with  computers  or  multimedia 
applications,  were  satisfied  when  offered  very  specific  instructions  with  limited  choices  in  software 
applications. 

Project  outcomes 

The  seven  multimedia  projects  produced  for  the  1997  "advanced"  workshops  were  most  complete  and 
complex  for  the  participants  who  took  the  workshops  for  college  credit  or  who  had  well  defined  projects  in 
mind  before  attending  the  workshops.  In  1998,  the  higher  education  instructors  completed  two  multimedia  and 
two  web  projects  during  the  workshop  time  but  these  projects  varied  considerably  in  the  degree  to  which  they 
incorporated  multimedia  design  principles  and  features  based  on  participant  attitude  and  expectations. 

For  the  K-12  instructors,  the  projects  were  consistent  in  level  of  use  of  multimedia  design  and  use  of 
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features  based  on  highly  defined  instructor  instructions  and  expectations.  These  projects  were  generally  more 
complex  and  complete  than  those  created  by  the  higher  education  instructors  who  were  not  compensated  for 
their  workshop  participation.  In  week  two,  each  K-12  instructor  was  required  to  design  a interactive,  non- 
tutorial, multimedia  mini-lesson  in  math  or  science  using  Authorware  or  PowerPoint.  In  week  three,  the  basic 
requirements  for  the  mini-lesson  were  the  same  but  the  multimedia  projects  were  designed  using  web  tools 
(Adobe  Pagemill  was  the  only  web  authoring  tool  used  in  this  session  because  this  software  was  included  in  the 
equipment  package  they  were  to  receive).  In  the  fourth  week,  the  participants  were  divided  into  collaborative 
pairs  to  devise  eight  student-centered  learning  activities  for  use  in  the  math  & science  clubs. 

Conclusions 

Based  on  the  experiences  from  these  workshops  conducted  during  the  last  two  years  there  are  both 
positive  and  negative  implications  for  training  instructors  to  use  multimedia  in  their  classrooms.  From  a positive 
standpoint,  the  level  of  motivation  among  faculty  in  general  for  improvement  of  their  teaching  was  very  high 
based  upon  workshop  attendance  by  individuals  who  received  no  compensation.  This  fact  is  heartening  based 
on  the  often  expressed  observation  that  faculty  are  rigid  and  disinterested  in  any  improvement  in  their 
instructional  methods.  Another  positive  outcome  of  the  workshop  experience  was  the  ability  of  technologically 
naive  (and  fearful)  individuals  to  produce  fairly  complex  multimedia  teaching  mini-lessons  in  short  period  of 
time.  From  a negative  standpoint,  it  is  difficult  to  assure  consistent  instructor  use  of  new  technologies  in  the 
learning  environment  when  there  is  not  complete  institutional  support.  These  workshops  were  funded  by  NSF 
grant  monies  which  are  not  an  assured  source  of  funding  on  an  ongoing  basis  for  regular  instructional  support 
within  the  institution. 
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Abstract:  The  following  report  illustrates  how  educational  researchers  and  software  developers 
can  work  in  conjunction  with  teachers  to  optimize  learning  and  provide  an  empirical  basis  for 
software  revision.  Using  the  Design  Experiment  Technique,  teachers  and  researchers  together 
determined,  through  successive  classroom  implementations,  how  to  optimize  the  use  of  new 
software.  Objective  assessments  made  along  the  way  indicated  whether  the  curricular  adjustments 
led  to  improved  learning.  The  current  research  involved  four  implementations  of  software  that 
teaches  science  content  and  inquiry  skills.  After  each  implementation,  the  data  were  analyzed,  and 
classroom  activities  were  revised.  Revisions  included  the  addition  of  time  tables,  the  jigsaw 
method  for  doing  background  research,  daily  questions,  and  activity  summaries.  These  revisions 
will  also  be  incorporated  into  future  versions  of  the  software. 


Introduction 

User  feedback  is  a central  component  of  a successful  software  development  project,  whether  the  project  is  targeted 
for  entertainment,  self-help,  or  other  commercial  purpose.  A primary  consideration  is  ease  of  use,  and  developers 
customarily  solicit  impressions  from  users  or  observe  users  as  they  attempt  to  run  the  software.  Developers  of 
educational  software,  on  the  other  hand,  are  interested  not  only  in  ease  of  use,  but  also  in  the  impact  the  software 
will  have  on  learning.  Unfortunately,  the  difficulty  of  obtaining  accurate  user  feedback  on  learning  is  complicated 
by  the  diversity  of  implementations  teachers  may  use  to  integrate  the  software  into  their  curricula.  The  research 
reported  here  illustrates  how  educational  researchers  and  software  developers  can  work  in  conjunction  with 
teachers  to  optimize  learning  and  provide  an  empirical  basis  for  software  revision. 

The  Design  Experiment  Technique  is  a recent  advancement  in  educational  research  (McGee  & Howard,  1998). 
Using  this  technique,  teachers  and  researchers  together  determine  how  to  optimize  the  use  of  new  software  through 
successive  classroom  implementations.  For  each  implementation,  there  are  objective  measures  that  assess  whether 
the  curriculum  adjustments  have  led  to  improved  learning  outcomes  relative  to  the  previous  implementation.  This 
technique  was  used  by  researchers  at  the  NASA  Classroom  of  the  Future  to  evaluate  the  Astronomy  Village^: 
Investigating  the  Universe™  multimedia  package.  This  paper  will  describe  the  design  experiment  process  for  four 
classroom  implementations  and  how  that  process  led  to  refinements. 

Astronomy  Village:  Investigating  the  Universe 

The  NASA  Classroom  of  the  Future™  Program  (COTF)  is  a NASA-funded  research  and  development  center  that 
specializes  in  the  development  and  testing  of  educational  multimedia  for  math,  science,  and  technology  education. 
In  March  1996,  the  COTF  produced  a CD-ROM  called  Astronomy  Village:  Investigating  the  Universe  for  use  as  a 
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curriculum  supplement  in  high  school  science  classrooms.  It  was  distributed  to  over  1 1 ,000  teachers,  educators, 
and  resource  centers,  and  it  won  Technology  and  Learning  magazine’s  Science  Software  of  the  Year  Award  for 
1996.  Astronomy  Village  uses  the  metaphor  of  living  and  working  at  a mountain-top  observatory  (the  village)  as 
the  primary  interface  from  which  students  investigate  contemporary  problems  in  astronomy  (see  Pompea  and 
Blurton,  1995).  Academic  activities  are  designed  to  promote  learning  of  astronomical  concepts  and  processes 
related  to  scientific  inquiry.  Students  join  a research  team  and  choose  one  of  ten  investigations  to  complete.  In  the 
Stellar  Nursery  investigation,  for  example,  students  investigate  how  stars  are  bom.  For  each  investigation,  students 
progress  through  five  phases:  background  research,  data  collection,  data  analysis,  data  interpretation,  and 
presentation  of  results.  For  any  given  phase,  there  are  from  three  to  seven  content-related  activities  to  be  completed 
before  proceeding  to  another  phase.  The  primary  means  of  tracking  progress  through  an  investigation  is  the 
Research  Path  Diagram — a chart  that  displays  icons  representing  activities  within  each  phase.  Each  time  a student 
clicks  on  one  of  the  icons  in  the  Research  Path  Diagram,  a virtual  mentor  appears  and  describes  activities  relevant 
to  that  particular  investigation.  A second  means  of  tracking  progress  is  the  electronic  LogBook,  in  which  students 
record  their  scientific  notes  and  observations. 

Method  and  Data  Sources 

The  design  experiment  consisted  of  four  one-semester  studies.  In  each  study  students  from  one  of  two  area  schools 
visited  the  COTF  facilities  to  use  the  Astronomy  Village  software. 

Design  Experiment  Populations 

The  participating  schools  were  from  a rural  community  with  a population  of  approximately  35,000.  The 
demographics  for  each  study  varied  (see  Table  1).  In  the  first  study  thirteen  students  from  the  ninth-grade  class  of  a 
girls’  academy  (college  preparatory)  participated.  In  the  second  study  nine  students  from  the  eighth-grade  class  of 
the  same  academy  participated.  In  the  third  study  twelve  students  from  the  tenth  and  eleventh  grade  of  a large 
public  high  school  participated.  In  the  fourth  study  nine  students  from  the  eigth-grade  class  of  the  same  girls’ 
academy  participated.  Students  from  the  third  study  were  from  an  at-risk  population.  In  each  case  students  attended 
class  daily  for  approximately  four  weeks  at  the  COTF  facility  in  lieu  of  their  science  class.  Sessions  using 
Astronomy  Village  were  co-taught  by  the  students'  classroom  teacher  and  the  first  author. 

It  should  be  noted  that  the  students  in  Studies  2 and  4 were  younger  than  students  in  Studies  1 and  3,  and  in  Study 
3 the  population  consisted  of  at-risk  youth  in  a public  school  setting.  Research  predicts  that  the  ninth-grade 
students  from  the  college  preparatory  academy  should  have  outperformed  the  other  groups  of  students  (those  in 
lower  grades  and  at-risk)  (Blank  and  Gruebel,  1995). 

Data  Sources 

There  were  three  sources  of  data  for  this  investigation.  The  first  source  was  a compilation  of  documents  in  the 
students’  electronic  notebooks.  In  the  first  two  studies  students  used  the  electronic  notebook  embedded  within 
Astronomy  Village.  This  notebook  was  called  the  LogBook.  In  the  third  and  fourth  studies,  students  used  an 
electronic  notebook  called  the  Collaboratory  Notebook  as  their  LogBook.  The  Collaboratory  Notebook  was 


Study  1 

Study  2 

Study  3 

Study  4 

Time  Frame 

Apr-May,  1996 

Oct-Nov,  1996 

Mar-Apr,  1997 

Oct-Nov,  1998 

Num.  of  Students 

13 

9 

12 

9 

Gender 

All  Female 

All  Female 

F=4;M  = 8 

All  Female 

Grade  Level 

9th 

8th 

lOth-llth 

8th 

Type  of  Students 

college  prep 

college  prep 

at-risk 

college  prep 

Table  1:  Demographics  of  Design  Experiment  Populations 
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developed  as  part  of  the  Learning  Through  Collaborative  Visualization  Project  (CoVis)  Project  at  Northwestern 
University  (Edelson  and  O’Neill,  1994).  For  each  activity  in  Astronomy  Village,  students  were  expected  to  produce 
activity  summaries  in  their  LogBooks.  These  summaries  consisted  of  a brief  description  of  the  activity,  a statement 
of  the  importance  of  the  activity  in  the  context  of  the  main  research  question,  and  any  new  questions  that  arose 
from  the  activity.  The  second  source  of  data  was  videotapes  of  student  interactions  while  using  the  software.  The 
third  source  of  data  was  field  notes  and  classroom  observations  by  the  teachers  and  first  author. 

Dependent  Measures 

There  were  two  aspects  of  the  activity  summaries  that  were  considered  for  each  design  experiment.  First,  how 
many  of  the  activities  in  Astronomy  Village  did  students  complete?  Within  each  study  the  primary  dependent 
measure  for  this  question  was  an  indicator  of  activity  completion  called  the  activity  completion  rate.  This  was  an 
objective  measure  of  the  extent  to  which  students  were  able  to  complete  the  assigned  activities.  Activity  completion 
rate  was  defined  as  the  number  of  activity  summaries  students  completed,  divided  by  the  number  of  activities  the 
mentor  suggested.  Second,  were  students  able  to  relate  the  individual  activities  to  the  basic  question  for  each 
investigation?  In  each  study  the  primary  dependent  measure  for  this  question  was  a measure  of  how  closely  the 
statement  of  importance  within  the  activity  summaries  was  linked  to  the  overall  investigation. 

Design  Experiment  Results 

In  brief,  the  design  experiment  began  by  using  the  instructional  procedures  prescribed  in  the  Astronomy  Village 
package,  and  examining  the  effects  as  the  program  was  implemented.  Each  succeeding  study  refined  the 
procedures  in  order  to  achieve  more  effective  outcomes.  The  next  section  describes  each  study  and  the  impact  of 
changes  to  the  activity  on  the  activity  completion  rate.  Along  with  a description  of  each  successive  study,  there  is  a 
discussion  of  how  the  instructional  procedures  were  modified  for  the  next  implementation. 

Activity  Completion  Rate 

Study  1 

In  the  first  study  the  overall  goal  for  the  students  was  to  complete  all  of  the  activities  as  suggested  by  the 
Astronomy  Village  virtual  mentor.  Our  purpose  was  to  implement  the  program  as  closely  as  possible  to  the 
intentions  of  the  software  designers.  Students  completed  activities  related  to  background  research,  data  collection, 
data  analysis,  data  interpretation,  and  presentation  of  results.  As  intended  by  the  developers,  students  began  by 
watching  a videotape  introduction  to  the  software.  Next,  they  worked  within  their  project  teams  to  complete  a 
tutorial  on  using  the  software.  After  completing  the  tutorial,  students  proceeded  to  the  virtual  Conference  Center  in 
Astronomy  Village,  where  the  virtual  mentors  described  the  different  investigations.  The  students  selected  different 
mentors  and  listened  to  a description  of  the  research  that  each  mentor  was  conducting.  The  students  chose  an 
investigation  that  interested  them  and  joined  an  investigation  team.  Next,  they  accessed  the  Research  Path 
Diagram,  which  introduced  them  to  the  resources  on  the  Astronomy  Village  CD-ROM  that  related  specifically  to 
their  investigation.  By  following  the  Research  Path  Diagram,  students  should  have  been  able  to  complete  the  steps 
necessary  to  learn  the  appropriate  concepts  and  conduct  the  level  of  problem  solving  needed  for  their  investigation. 

Completion  Rate.  Summary  worksheets  made  activity  completion  easier,  although  completion  rates  were  still 
below  teacher  expectations.  The  average  activity  completion  rate  for  this  study  was  42%.  This  value  indicated  that 
over  the  four-week  period,  students  completed  less  than  half  of  the  activities  the  mentor  suggested.  Further  analysis 
of  the  activity  completion  rate  within  each  phase  of  research  (background  research  - 55%,  data  collection  - 75%, 
data  analysis  - 35%,  data  interpretation  - 20%,  and  presentation  - 27%)  revealed  that  the  activity  completion  rate 
was  much  lower  during  later  phases  of  research.  This  analysis  of  the  activity  completion  indicated  that  the 
Research  Path  Diagram  and  virtual  mentors  were  not  sufficient  to  guide  students  through  the  complex  problem 
solving  required  by  the  activities  in  the  investigation.  If  students  were  not  able  to  complete  the  activities  in  the 
investigation,  it  was  believed  they  would  not  achieve  the  desired  learning  outcomes. 
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Study  2 


In  considering  the  disappointing  performance  of  the  students  in  Study  1,  the  teachers  for  Study  2 modified  the 
implementation  slightly  from  that  intended  by  the  designers  of  the  Astronomy  Village.  The  modification  took  three 
forms:  target  dates  for  phase  completion,  more  time  for  activity  completion,  and  more  contextualized  training. 

Modifications.  First,  students  were  given  target  dates  for  each  of  the  phases  of  research  so  they  would  have 
sufficient  time  to  complete  later  phases  of  research.  Second,  teachers  eliminated  activities  that  did  not  seem  to 
benefit  the  students.  That  is,  the  tutorial  was  eliminated,  and  students  selected  a investigation  from  a list  of 
abstracts  instead.  These  two  changes  resulted  in  approximately  17%  more  time  for  completion  of  activities  (3.5 
instructional  days  out  of  20).  Third,  training  was  now  given  on  an  as-needed  basis. 

Completion  Rate.  The  average  activity  completion  rate  for  this  study  was  85%.  This  value  indicated  that  over  the 
four-week  period,  students  completed  twice  as  many  activities  as  the  students  in  the  first  study,  even  though  these 
students  were  a year  younger.  The  activity  completion  rate  within  each  phase  of  research  was  as  follows: 
background  research  - 78%,  data  collection  - 100%,  data  analysis  - 83%,  data  interpretation  - 62%,  and 
presentation  - 100%.  The  students  in  this  study  had  more  opportunity  to  achieve  the  desired  learning  outcomes 
than  students  in  the  previous  study.  Thus,  the  use  of  an  objective  measure  such  as  the  activity  completion  rate  made 
it  possible  to  see  what  factors  were  preventing  students  from  completing  the  activities. 

Study  3 & 4 

Although  the  modifications  in  Study  2 improved  the  completion  rate,  the  teachers  noticed  that  students  were 
having  difficulty  seeing  the  connection  between  the  individual  activities  and  the  investigation  as  whole.  In  Study  3, 
the  researchers  and  teachers  initiated  several  major  modifications  to  help  students  see  the  connection.  These 
modifications  took  three  forms:  all  students  working  on  the  same  investigation,  the  additition  of  a motivating 
question  phase,  and  a shortened  investigation  structure. 

Modifications.  To  help  students  complete  their  investigations  more  quickly,  the  research  phases  were  revised  and 
truncated.  The  new  phases  were  the  motivating-question  phase,  the  background  research  phase,  the  background 
review  phase,  the  data  analysis  phase,  and  the  reflection  phase.  In  the  motivating-question  phase  the  teacher  posed 
the  main  investigation  question,  and  the  students  individually  typed  responses  in  their  LogBooks.  Next,  the  teacher 
showed  the  students  the  data  they  would  be  analyzing  and  asked  them  to  record  observations.  These  two  activities 
were  meant  to  activate  students’  prior  knowledge  and  connect  it  to  the  activities  of  the  investigation.  This  approach 
was  similar  to  Minstrell’s  benchmark  lessons  (Bruer,  1993).  In  the  background  research  phase  the  teacher  selected 
the  three  most  relevant  articles  from  the  investigation,  and  each  student  read  one  of  the  three  and  developed  an 
activity  summary.  In  the  background  review  phase  students  used  their  activity  summaries  to  answer  questions  as  a 
team,  prompting  themselves  to  integrate  across  the  individual  readings.  Since  each  student  was  an  expert  on  only 
one  of  the  articles,  students  would  be  forced  to  discuss  the  readings  with  one  another  in  order  to  answer  the 
questions.  In  the  data  analysis  phase  students  completed  analysis  worksheets  as  a team.  And  finally,  in  the 
reflection  phase  students  responded  to  integrative,  teacher-posed  questions  in  their  LogBooks.  Using  these 
redesigned  research  phases,  it  was  possible  to  fit  three  investigations,  rather  than  one,  into  the  four- week  period. 

Completion  Rate.  The  average  activity  completion  rate  for  Study  3 was  74%.  This  rate  was  comparable  to  the 
activity  completion  rate  from  Study  2,  even  though  the  third  group  consisted  of  students  who  were  at  risk.  The 
activity  completion  rate  for  each  phase  was  as  follows:  motivating  question  - 82%,  background  research  - 85%, 
background  review  - 62%,  reflection  - 73%.  The  activities  for  this  study  involved  m‘ore  explicit  prompts  for 
students  to  reflect  on  their  background  knowledge  and  relate  that  knowledge  to  the  activities  in  the  research 
investigation.  These  more  explicit  prompts  increased  the  likelihood  that  students  would  assimilate  the  new 
information  from  Astronomy  Village  into  existing  knowledge  structures. 

Study  4 used  the  same  structure  as  in  Study  3,  with  one  primary  exception.  The  students  only  completed  one 
investigation  during  an  eight-day  period.  Also,  researchers  once  again  worked  with  eighth-grade  students. 
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□ Connected  to  the  larger 
context 

□ Connect  ed  only  t o 
Astronomy  Village 

□ No  Connection 


Figure  1 , Percent  of  Activity  Summaries  that  Relate  to  the  Larger  Context  of  the  Basic  Science  Question 

LogBook  Quality 

In  addition  to  activity  completion  rates,  researchers  wanted  to  know  how  well  students  were  connecting  their 
activities  over  the  four-week  period  to  the  larger  context  of  the  basic  science  question.  (A  basic  science  question 
addressed  a fundamental  issue  in  astronomy  such  as,  Is  there  other  life  in  the  universe?  In  contrast,  a research 
investigation  question  addressed  a specific  issue  in  astronomy  such  as,  How  do  you  determine  what  areas  of  the 
universe  to  examine  for  evidence  of  planets  orbiting  a distant  star?  ) To  evaluate  these  connections,  the  researchers 
examined  students’  LogBook  responses  to  a particular  question  in  the  activity  summary  worksheets  that  queried  the 
importance  of  the  activity,  A rating  scale  was  developed  to  judge  the  responses.  Responses  that  related  the  activity 
to  a basic  science  question  were  rated  more  highly  than  those  that  merely  related  the  activity  to  the  research 
investigation  question,  while  those  that  discussed  only  the  activity  itself  (zero  connection)  were  rated  lowest. 
Statements  of  importance  were  extracted  from  the  LogBook  activity  summaries  for  each  article  that  students  read. 
Raters  were  blind  as  to  which  study  the  statement  of  importance  came  from.  Raters  made  judgments  by  comparing 
the  statements  of  importance  to  the  article  content,  the  investigation  research  question,  and  the  basic  science 
question. 

Researchers  investigated  the  impact  that  the  various  implementations  had  on  the  quality  of  the  LogBook  entries.  In 
order  to  simplify  the  analysis,  the  studies  were  clustered  into  2 phases,  based  on  the  similarities  of  the 
implementations.  Phase  1 encompassed  Studies  1 & 2,  in  which  students  worked  on  one  investigation  at  their  own 
pace  over  four  weeks.  Phase  2 encompassed  Studies  3 & 4,  in  which  students  worked  on  several  investigations—  as 
directed  by  the  teacher—  and  used  a truncated  set  of  activities  for  each  investigation.  In  addition,  Phase  2 studies 
were  more  explicit  in  providing  students  with  assistance  in  making  connections  between  activities. 

Results  indicated  that  students  from  Phase  2 studies  were  able  to  connect  almost  one-third  of  the  activity 
summaries  to  the  basic  science  question  for  each  investigation  (see  Figure  1 ).  In  contrast,  students  from  Phase  1 
studies  were  able  to  connect  roughly  one-third  of  their  activity  summaries  only  to  the  research  investigation 
question.  The  two  phases  were  comparable  in  the  percent  of  activities  that  contained  no  connection  between  the 
articles  and  the  research  investigation  or  basic  science  question.  A Chi-square  analysis  indicated  that  the  higher 
percentage  of  students  in  Phase  2 who  were  capable  of  connecting  the  activity  summaries  to  the  larger  science 
context  was  statistically  significant  (x  - 12.19,  p < .01). 

These  results  indicated  that  the  addition  of  a motivating-question  phase  and  a reflection  phase  helped  the  students 
to  draw  connections  between  the  various  activities  within  the  extended  project.  By  making  explicit  references  to  the 
basic  science  question  and  prompting  students  to  develop  hypotheses  about  the  basic  science  question,  the  teachers 
in  Phase  2 helped  students  connect  individual  activities  to  the  larger  scientific  context. 

Conclusions 
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The  specific  modifications  in  this  design  experiment  were  unique  to  the  Astronomy  Village  software  and  to  the 
particular  classroom  setting  used  at  COTF.  However,  the  general  issues  that  were  revealed  are  germane  to  many 
software  programs  and  classroom  contexts.  If  teachers  understand  the  issues  they  will  face  while  implementing 
technology  and  are  given  guidance  on  how  to  address  these  issues,  there  will  be  a greater  chance  that  educational 
reform  will  be  sustained  beyond  the  initial  pilot  projects  (McGee  & Howard,  1 998). 

The  results  of  the  design  experiment  helped  the  developers  at  the  COTF  understand  the  kinds  of  supports  that 
would  be  needed  to  optimize  learning.  These  supports  are  now  being  incorporated  into  a new  version  of  Astronomy 
Village  that  will  focus  on  the  solar  system.  In  this  new  version  the  investigations  will  be  shortened  to  one  week  to 
help  students  better  manage  their  time.  With  more  time,  students  will  have  four  opportunities  to  conduct 
investigations  in  a four-week  period.  To  help  students  connect  individual  activities  to  the  investigation  question,  an 
exploration  phase  will  also  be  added,  to  take  place  prior  to  the  background  research  phase.  The  exploration  phase 
will  serve  the  purpose  of  the  motivating  question  phase  in  Study  3.  In  addition,  the  LogBook  will  provide 
structured  prompts  (similar  to  the  activity  summaries)  to  help  students  reflect  on  the  learning  process. 

Reform  will  only  take  place  when  teachers  are  able  to  explore  the  implications  of  a new  curriculum  within  the 
context  of  their  own  classrooms.  That  an  educational  multimedia  program  was  used  successfully  in  a one  school 
district  does  not  necessarily  mean  that  it  will  be  effective  in  another.  Design  experiments  can  be  especially 
powerful  when  results  lead  to  teacher-generated  design  principles.  Creating  partnerships  between  teachers  and 
researchers  will  enhance  reform  by  giving  teachers  tools  for  evaluating  their  own  implementations  and  providing 
researchers  a means  of  comparing  implementations  across  different  contexts. 
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Abstract:  Common  paradigms  for  human-computer  interaction  used  in  the  design  of  general  computer  interfaces 
are  often  not  suited  for  the  telepresentation  scenario.  This  is  true  for  hardware  input  devices,  generally  keyboard  and 
mouse,  as  well  as  for  software  user  interfaces.  The  use  of  alternative  input  devices  and  the  adaption  of  the  according 
software  is  necessary  to  gain  full  profit  of  the  augmented  possibilities  computer  supported  presentations  can  offer. 

In  this  paper  we  will  present  different  hardware  devices  and  discuss  their  usability  for  telepresentations.  Various 
experiments  carried  out  in  miscellaneous  lectures  at  our  university  indicated  numerous  problems  and  requirements 
for  the  hardware  and  software.  We  will  present  our  experiences,  discuss  the  occurring  problems  as  well  as  possible 
solutions,  and  address  current  and  future  research  areas. 

Introduction 

Telepresentations,  i.e.  presentations  given  with  the  use  of  computers,  offer  great  advantages  and  augmented  pos- 
sibilities compared  to  regular  talks.  The  additional  functionalities  can  be  useful  to  improve  the  presentation  and 
to  support  the  person  giving  a talk.  Multimedia  supplements  facilitate  the  understanding  of  complex  facts  and 
offer  new  insights  into  difficult  problems.  Online  transmission  and  recording  for  offline  use  can  be  done  with  a 
high  quality  at  low  costs.  Time  and  space  independence  can  be  achieved  through  these  recording  and  transmission 
facilities.  However,  the  tools  and  devices  which  are  generally  utilized  for  such  a telepresentation  require  additional 
training  and  customization  before  they  can  be  used  in  a gainful  way.  Common  computer  interfaces  are  not  designed 
to  give  presentations  but  to  operate  regular  computer  programs,  such  as  wordprocessing  systems,  databases,  web 
browsers,  and  so  on.  In  the  telepresentation  scenario  these  interfaces  normally  lack  required  functionalities  and 
force  the  lecturer  to  adapt  to  an  interface  which  is  not  suitable  for  the  respective  needs.  In  this  paper  we  will 
address  the  user  interface  issues  for  the  teleteaching  scenario.  We  will  have  a closer  look  on  what  the  needs  are, 
we  discuss  alternative  input  devices,  how  they  can  be  applied  in  telelectures,  and  the  consequences  for  the  design 
of  user  interface  software. 


Telepresentations  and  Ubiquitous  Computing 

In  the  presentation  and  teleteaching  scenario  a computer  in  combination  with  a data  projector  is  generally  used  as 
a replacement  for  traditional  presentation  facilities,  i.e.  chalk  and  blackboards  or  slides  and  overhead  projections. 
The  screen  of  the  computer  running  a so-called  electronic  whiteboard  program  is  projected  on  the  wall  with  a 
data  projector  and  transmitted  over  the  network  to  the  stations  receiving  the  talk.  The  presentation  is  given  at  the 
computer  interface  with  the  use  of  common  input  devices,  i.e.  keyboard  and  mouse.  Many  whiteboards  contain 
connectability  to  the  internet  and  transmission  facilities.  Some  of  them,  such  as  the  AOFwb  [4]  developed  in  our 
group,  also  have  integrated  recording  features. 

One  problem  with  this  approaches  that  the  whole  presentation  has  to  be  given  on  a computer  interface.  This  is 
not  the  common  environment  for  giving  a talk  and  is  especially  a problem  if  the  lecturer  is  not  used  to  work  with 
computers.  Common  electronic  whiteboards  have  interfaces  which  can  be  handled  easily  by  people  who  work  with 
computers  in  their  everyday  life  but  are  not  suited  if  the  lecturer  has  limited  experience  with  common  user  interface 
paradigms.  The  lecturer  also  has  to  sit  in  front  of  a computer.  This  might  result  in  a lack  of  social  contact  with  the 
local  participants  of  the  presentation.  Ideally  the  user  interface  in  the  teleteaching  scenario  should  be  intuitive  and 

*This  research  is  supported  by  the  German  Research  Foundation  (Deutsche  Forschungsgcmeinschaft  DFG),  project  Generation  of  Multi- 
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no  special  training  should  be  required  for  new  users.  The  presenter  should  be  able  to  concentrate  completely  on  the 
content  of  the  talk  and  not  on  the  operation  of  the  teleteaching  interfaces.  No  restrictions  should  apply  to  a lecturer 
about  which  presentation  techniques  to  use.  The  new  tools  and  interfaces  should  be  able  to  be  operated  in  the  same 
manner  as  the  traditional  ones  which  would  be  used  if  the  presentation  had  been  given  without  computers. 

An  interesting  aspect  in  this  context  is  the  concept  of  ubiquitous  computing  introduced  by  M.  Weiser  [9].  Ubiq- 
uitous computing  is  “the  method  of  enhancing  computer  use  by  making  many  computers  available  throughout  the 
physical  environment,  but  making  them  effectively  invisible  to  the  user”  [10].  Computers  are  integrated  into  our 
everyday  environment  in  a way  that  we  can  interact  with  them  without  realizing  them.  Therefore  we  are  able  to 
gain  from  the  augmented  possibilities  offered  by  the  computers  without  even  having  to  think  about  their  existance. 
In  this  way  the  computer  becomes  invisible  to  the  user.  To  apply  the  ideas  of  ubiquitous  computing  to  the  teleteach- 
ing  scenario  we  need  tools  similar  to  a blackboard  or  an  overhead  projector  that  are  also  able  to  perform  additional 
tasks,  such  as  easy  transmission  and  recording  or  the  use  of  multimedia  supplements.  Electronic  whiteboards  could 
be  used  for  this  purpose  but  require  adaption  of  the  hardware  interface  used  for  human-computer  interaction  on 
one  hand  and  of  the  software  interface  on  the  other  hand.  In  the  next  section  we  will  discuss  the  resulting  hardware 
issues. 

Hardware  User  Interface  Issues 

Experiments  with  Different  Hardware  Input  Devices 

As  discussed  in  the  previous  section,  there  is  a need  for  hardware  devices  which  replace  blackboards  and  overhead 
projectors  in  regular  classrooms.  Therefore  we  require  a wall  mounted  interactive  board  and  a touch  sensitive 
device  on  a high  desk  in  combination  with  a data  projector.  They  should  be  similar  to  handle  by  a user  like 
the  respective  facilities  they  are  replacing.  Additionally  they  should  offer  full  access  to  the  software  interfaces 
providing  the  supplementary  features  which  make  telepresentations  such  a useful  and  exciting  area. 

A relevant  overview  on  user  interface  hardware  currently  available  on  the  market  can  be  found  at  the  Web  page 
provided  by  B.  Buxton  at  [2].  Several  of  these  hardware  devices  can  be  used  as  an  interactive  board  to  replace 
a wall  mounted  blackboard.  In  our  tests  we  used  two  large  digitizing  boards  (see  Fig.  1,  left  and  middle).  They 
are  touch  sensitive  and  send  the  coordinates  of  every  position  that  a person  contacts,  either  by  hand  or  by  the 
use  of  another  tool,  e.g.,  a pen,  to  a connected  computer.  If  the  content  of  the  computer  screen  is  projected  on 
the  board,  then  the  mouse  pointer  can  be  moved  and  mouse  clicks  can  be  executed  just  by  touching  the  board. 
This  facility  can  therefore  be  used  to  operate  an  electronic  whiteboard.  The  behavior  of  the  board  is  similar  to  a 
touchscreen  mounted  on  a regular  desktop  computer.  In  the  board  shown  at  the  left  side  of  Fig.  1 the  computer 
content  is  projected  on  the  board  from  a data  projector  standing  in  front  of  the  board.  The  one  shown  in  the  middle 
of  Fig.  1 has  a rear  projection  system  where  the  data  projector  is  placed  behind  a transparent  input  area.  This 
circumvents  the  problem  of  covering  parts  of  the  board  with  your  own  shadow  while  standing  in  front  of  it.  It 
turned  out  that  in  combination  with  an  electronic  whiteboard,  e.g.,  the  AOFwb  [4],  such  a board  is  very  suitable 
as  a blackboard  replacement.  Both  the  lecturer  and  the  students  attending  the  talks  locally  found  it  a pleasant  and 
convenient  tool.  However,  several  problems  occurred  which  will  be  discussed  in  the  next  paragraph.  One  problem 
was  the  limited  size  of  the  board.  The  boards  used  in  our  experiments  had  a diagonal  screen  size  of  58  1/4”  (148 
cm)  and  of  72”  (1 82.9  cm).  In  large  rooms  this  might  be  to  small  to  provide  reasonable  input  and  output  facilities 
for  a regular  talk  or  presentation.  Even  the  largest  board  available  on  the  market  today  seems  to  be  to  small  for 
our  purpose.  One  possibility  to  circumvent  this  problem  is  to  take  two  or  three  of  these  boards  and  put  them  next 
to  each  other  to  get  a wide  interactive  blackboard,  such  as  it  is  done  by  the  Integrated  Publication  and  Information 
Systems  Institute,  IPSI,  at  the  German  National  Research  Center  for  Information  Technology,  GMD.  They  connect 
three  boards,  each  having  a diagonal  screen  size  of  72”  (182.9  cm),  to  form  one  large  interactive  wall,  called  the 
DynaWall  [8].  The  connection  of  more  boards  next  to  each  other  poses  new  requirements  on  the  interface  design 
and  the  implementation  of  the  underlying  programs  which  we  will  address  later. 

There  are  also  some  input  devices  on  the  market  useful  (in  combination  with  a data  projector)  as  a replacement 
for  overhead  projection  systems.  We  tested  a regular  computer  screen  equipped  with  a touchsensitive  display.  An 
electronic  whiteboard  runs  on  the  connected  computer  and  can  be  operated  by  the  user  with  the  fingertip  or  a 
pen  like  facility.  The  display  should  be  mounted  horizontally  to  resemble  writing  on  a piece  of  paper  or  a slide 
lying  on  an  overhead  projector.  The  content  of  the  interactive  computer  screen  is  projected  on  the  wall  with  a data 
projector.  We  tested  regular  touchscreens  and  found  out  that  they  are  not  reasonable  for  our  purpose  because  they 


565 


Figure  1 : Hardware  used  in  our  experiments:  interactive  boards  (left  and  middle)  and  graphic  tablet  (right). 


usually  react  to  every  kind  of  input  performed  on  the  display.  Therefore  it  is,  for  example,  not  possible  to  rest  your 
wrist  on  the  surface  when  writing  with  a pen.  Freehand-writing  on  a horizontally  mounted  display  equipped  with 
such  a touchscreen  is  only  possible  by  radically  changing  your  regular  writing  style.  This  is  not  acceptable  in  our 
scenario.  However,  there  are  a few  hardware  devices  commercially  available  which  use  special  techniques  only 
recognizing  input  from  a particular  tool,  such  as  a special  pen,  but  not  from  other  facilities,  such  as  the  users  hand. 
However,  most  of  them  lack  a size  appropriate  for  our  purpose.  This  is  the  reason  why  we  used  a regular  graphic 
tablet  for  our  experiments,  such  as  the  one  shown  in  Fig.  1 (right).  Here  the  input  is  done  on  the  writing  surface 
of  the  graphic  tablet  with  the  use  of  a special  pen  recognizable  by  the  tablets  hardware.  Every  other  kind  of  input, 
e.g.,  touching  the  tablet  with  the  hand,  is  rejected  from  the  hardware.  The  output  is  indicated  only  on  the  computer 
screen,  but  not  on  the  graphic  tablet.  With  this  technique  our  electronic  whiteboard  can  be  operated  remotely  using 
a pen  instead  of  a mouse.  The  principle  of  this  kind  of  operation  is  the  same  as  using  a regular  computer  mouse 
but  allows  easier  writing  and  freehand  drawing  for  the  lecturer.  Due  to  the  separation  of  input  and  output  area, 
this  kind  of  interface  requires  some  time  to  get  used  to  it.  Our  experiments  indicated  that  some  people  found  it 
a very  useful  tool  for  telepresentations  while  others  did  not  feel  comfortable  with  this  kind  of  “remote  writing”. 
Therefore,  with  some  training  and  in  combination  with  a data  projector,  such  a graphic  tablet  can  be  used  as  an 
alternative  input  device  and  a replacement  for  classical  overhead  projection  systems.  However,  it  is  not  a perfect 
solution.  Another  problem  resulting  from  this  setup  is  that  the  lecturer  is  still  sitting  or  standing  in  front  of  a large 
computer  screen  which  might  block  the  view  to  the  local  attendants.  The  students  felt  this  loss  of  eye  contact  as  a 
disturbing  factor  during  our  experiments. 

Problems,  Consequences,  and  Requirements 

The  used  input  devices  for  our  teleteaching  scenario  should  act  as  reasonable  replacements  for  traditional  repre- 
sentation tools,  such  as  blackboards  and  overhead  projectors.  The  various  experiments  and  real-world  tests  that 
we  did  during  miscellaneous  lectures  and  classes  with  the  devices  introduced  in  the  previous  section  pointed  out 
several  consequences  and  requirements  the  new  hardware  interfaces  should  fulfill: 

• An  input  device  similar  to  a pen  or  a chalk  should  be  supported.  Additional  input  facilities,  such  as  a 
keyboard,  might  be  desirable  in  special  situations. 

• The  input  area  should  be  identical  to  the  output  area.  Remote  input  and  control  of  the  electronic  whiteboard 
is  not  appropriate  because  it  requires  special  training  and  an  adaption  to  the  interface.  In  our  scenario  this  is 
not  an  intuitive  way  to  interact  with  the  application  program(s). 

• “Input  by  accident”  should  be  avoided.  For  example,  resting  your  hand  on  the  input  area  while  writing  with 
a pen  should  not  be  misinterpreted  as  any  kind  of  input. 

• The  input  and  the  output  area  should  provide  an  appropriate  size,  i.e.,  they  have  to  have  about  the  size  of  a 
regular  blackboard  or  overhead  projection.  The  presentations  have  to  be  large  enough  to  be  readable  by  the 
local  attendants  and  large  enough  that  an  appropriate  input  by  the  lecturer  can  be  guaranteed. 

• The  resolution  and  the  sensitivity  of  the  screens  replacing  the  blackboard  or  the  projector  have  to  be  adequate. 
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• The  surface  has  to  provide  good  reading  and  writing.  For  example,  no  reflections  should  disturb  the  audience 
or  the  lecturer  and  no  concavity  should  complicate  the  input. 

• There  should  be  no  need  to  darken  the  room,  because  dark  rooms  are  not  acceptable  for  the  local  attendants. 


• The  use  of  special  hardware  should  not  block  the  attendant' s view  of  the  lecturer  or  to  the  board.  Eye  contact 
between  the  person  giving  the  talk  and  the  audience  is  important. 

These  basic  requirements  resulted  from  the  experiments  we  did  with  special  hardware  facilities.  However,  the  list 
given  here  generally  applies  to  every  kind  of  input  interface.  Further  problems  and  requirements  we  identified  are 
due  to  the  design  of  the  software  interface  and  will  be  discussed  in  the  next  section. 


Software  User  Interface  Issues 

Experiments  with  Standard  Graphical  User  Interfaces 

The  software  design  of  an  appropriate  user  interface  has  to  meet  the  requirements  resulting  from  the  different  input 
devices  and  the  ubiquitous  computing  scenario.  In  stressful  situations,  such  as  giving  a talk,  people  tend  to  have 
wrong  input  behavior  or  to  get  confused  by  the  interface  more  often.  They  are  concentrated  on  their  presentation 
and  not  on  the  interaction  with  the  computer  hardware.  Therefore  the  interface  should  be  easy  and  intuitive  to 
handle. 

For  the  experiments  with  the  alternative  input  devices  discussed  in  the  last  section  we  used  our  electronic  white- 
board, the  AOFwb  [4].  The  interface  of  this  whiteboard  is  comparable  to  regular  computer  interfaces  found  in 
programs  with  similar  tasks.  No  special  adaption  of  the  software  to  the  new  hardware  facilities  has  been  done.  A 
screenshot  of  the  AOFwb  interface  can  be  found  in  Fig.  2. 
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Figure  2:  The  interface  of  the  electronic  whiteboard  AOFwb. 

This  is  a typical  Graphical  User  Interface  (GUI)  found  in  many  similar  computer  programs.  These  interfaces, 
sometimes  also  called  WIMP  (ITindows,  /cons,  Menus,  Pointing)  interfaces,  follow  the  desktop  paradigm.  They 
have  been  designed  for  the  use  with  desktop  computers  for  tasks  such  as  word  processing,  database  management, 
and  so  on,  but  not  to  interact  with  large  interactive  walls.  Our  experiments  showed  that  even  users  experienced 
with  such  common  human-computer  interfaces  have  problems  in  operating  them  during  a telepresentation.  We 
discuss  these  issues  in  the  next  paragraph. 

Problems,  Consequences,  and  Requirements 

Typical  Graphical  User  Interfaces,  such  as  the  one  used  in  the  AOFwb,  have  been  designed  to  be  used  on  desktop 
computers.  They  are  not  suited  for  the  operation  of  large  interactive  walls.  Various  modifications  have  to  be 
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applied.  For  example,  with  the  remote  operation  using  a regular  computer  mouse,  navigation  through  nested  menus 
seems  to  be  a reasonable  way  to  interact  with  a desktop  computer  and  to  execute  commands.  When  standing  in 
front  of  an  interactive  board  using  a pen  as  input  device,  pressing  just  a single  button  seems  to  be  more  appropriate. 
The  desktop  paradigm  has  to  be  replaced  by  a “blackboard  paradigm*’.  The  different  size  of  the  interaction  devices 
also  has  to  be  considered  in  the  interface  design.  Regular  computer  programs  usually  place  things  like  a toolbar  on 
one  side  of  the  interface.  This  is  an  appropriate  solution  for  the  screen  of  a desktop  computer,  but  causes  problems 
on  a large  scaled  interactive  board  because  the,  user  always  has  to  walk  to  the  specific  position  where  the  toolbar  is 
placed.  Ideally  the  toolbar  should  be  accessible  from  anywhere  the  lecturer  is  standing.  “Popup  toolbars”  might  be 
a solution  for  this  problem.  Another  approach  can  be  found  in  [7].  Here  an  additional  palm  top  computer  is  used  to 
hold  the  toolbar.  It  can  be  carried  by  the  lecturer  in  his/her  none-dominant  hand  and  is  connected  to  an  interactive 
board  via  a wireless  connection. 

Another  problem  is  that  it  is  hard  to  remember  which  tool  is  activated  at  a particular  time.  Similar  to  the  use  of 
a computer  mouse,  the  finger  or  pen  of  a user  has  a different  functionality  depending  on  which  tool  is  selected. 
Regular  whiteboards  or  graphic  programs  usually  indicate  the  currently  selected  tool  by  highlighting  the  according 
icon  on  the  toolbar  and  by  displaying  the  mouse  pointer  as  a symbol  indicating  the  functionality  of  the  selected  tool. 
These  software  solutions  do  not  work  well  with  interactive  boards.  Due  to  the  direct  interaction  with  the  board,  no 
remotely  controlled  pointer  is  required.  Indication  of  an  icon  representing  the  current  tool  is  usually  only  possible 
at  the  last  position  of  the  board  that  was  touched  by  a user  because  the  hardware  normally  only  recognizes  if  a 
board  is  touched,  not  if  the  pen  is  moved  without  touching  it.  The  indication  of  the  selected  tool  at  a particular  place 
on  the  board  still  requires  that  one  looks  at  this  position  before  whiteboard  actions  are  performed.  The  question 
rises  whether  it  is  reasonable  to  assign  more  than  one  functionality  to  the  same  input  device  if  the  operation  is  done 
directly  instead  of  remotely.  Some  boards  offer  hardware  based  solutions  to  this  problem.  For  example,  the  board 
that  we  used  comes  with  an  integrated  pen  tray  holding  several  pens  in  different  colors  and  a sponge.  Sensors 
capture  the  tool  which  was  picked  up  at  last.  Each  contact  with  the  board  is  now  interpreted  as  the  board  is  touched 
with  this  tool.  This  method  can  also  cause  problems,  for  example,  if  the  user  picks  up  more  than  one  tool  at  a time 
or  forgets  to  put  a tool  back  to  its  place.  In  our  experiments  we  discovered  that  such  mistakes  happen  especially 
often  if  the  board  is  used  under  stressful  situations  such  as  giving  a talk.  Furthermore,  such  a hardware  oriented 
solution  is  not  particularly  flexible  and  thus  not  open  to  further  modifications  and  extensions  as  a software  based 
one.  An  interesting  approach  in  this  context  is  work  done  by  the  Integrated  Publication  and  Information  Systems 
Institute,  IPSI,  at  the  German  National  Research  Center  for  Information  Technology,  GMD,  and  a group  at  the 
Xerox  Palo  Alto  Research  Center.  Contrary  to  the  traditional  approach  of  first  selecting  a tool  and  then  using  it, 
they  recognize  gestures  that  represent  different  functions.  Therefore  operations  such  as  moving  of  objects  or  the 
reorganization  of  tables  can  be  done  without  the  need  to  activate  a special  tool.  Additional  information  on  their 
work  can  be  found  in  [3]  and  [5,  6]. 

The  question  of  what  functions  should  be  offered  to  a user  must  also  be  addressed.  The  toolbar  on  our  whiteboard 
has  a variety  of  different  functions,  while  the  ZenPad{\^  used  in  the  classroom  2000  project  at  Georgia  Tech  offers 
only  a few  ones.  Their  reason  for  offering  only  some  essential  tools  to  the  user  is  that  they  determined  that  most 
of  the  lecturers  which  utilized  their  whiteboard  only  used  few  of  them.  We  experienced  the  same  results  when 
external  lecturers  used  our  whiteboard.  An  increasing  number  of  options  also  reduces  the  manageability.  However, 
we  also  discovered  that  people  who  are  familiar  with  our  whiteboard  tend  to  utilize  more  of  the  additional  tools 
we  offer.  Based  on  this  fact,  we  believe  that  there  is  a demand  for  these  functionalities.  So  on  one  hand  high 
functionality  discourages  and  confuses  potential  users.  On  the  other  hand  reducing  the  functionality  experienced 
users  might  want  or  need  does  not  seem  to  be  an  appropriate  solution.  One  good  compromise  we  are  working  on 
right  now  is  to  make  the  toolbar  customizable  by  the  user.  A “low  level”  user  interface  should  only  offer  the  basic 
functionality  needed  to  give  a presentation.  Based  on  his/her  own  needs  and  experience,  the  lecturer  can  decide 
what  features  are  necessary  and  still  can  be  handled  during  a special  presentation  and  add  them  to  the  configurable 
toolbar. 

In  our  experiments  we  used  either  the  interactive  board  or  the  graphic  tablet,  not  both  in  common.  In  an  ideal 
scenario  one  would  like  to  have  a wall  mounted  blackboard  as  well  as  an  interactive  high  desk  functioning  simul- 
taneously. The  lecturer  could  then  walk  around  and  switch  between  the  two  input  devices  during  the  presentation 
depending  on  his/her  personal  aims.  However,  due  to  the  use  of  more  than  one  interface  at  a time  the  software 
has  to  handle  the  communications  between  both.  No  inconsistency  should  result  from  the  use  of  different  input 
devices.  Even  if  both  hardware  devices,  the  interactive  board  and  the  high  desk,  are  controlled  by  the  same  com- 
puter several  problems  have  to  be  solved.  Most  of  the  common  operation  systems  permit  connections  with  more 
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than  one  screen  to  a computer.  However,  it  might  be  irregular  for  one  program  to  run  on  both  screens.  Copying 
of  an  object  from  one  screen  to  another  is  generally  also  not  possible.  This  task  is  especially  necessary  if  you  put 
more  boards  next  to  each  other  to  form  a large  interactive  wall.  Therefore  appropriate  network  connections  and 
protocols  have  to  be  established  that  allow  such  operations  without  the  loss  of  consistency.  Work  in  this  area  has 
been  done  by  [8]  and  [7]. 

Finally  the  question  has  to  be  answered  of  how  the  content  presented  on  these  interfaces  during  a talk  should  be 
indicated  on  a receiver  station  during  transmission  or  at  a local  desktop  computer  during  offline  viewing.  The 
relationship  between  the  hight  and  the  width  of  the  input  devices  we  used  were  about  the  same  as  with  regular 
computer  displays.  As  we  already  mentioned,  a larger  size,  especially  concerning  the  width  of  the  interactive 
blackboard,  is  desirable.  Therefore  intelligent  representation  techniques  must  be  introduced  that  handle  the  con- 
flicts resulting  from  the  different  size  of  the  input  devices  used  by  the  lecturer  and  the  output  devices  used  by  the 
remote  attendants  or  the  viewers  of  recorded  talks.  As  well,  the  integration  of  back  channels  for  feedback  from  the 
audience  placed  at  the  remote  channels  has  to  be  done  in  an  appropriate  way. 

Summary 

In  the  preceding  sections  we  discussed  some  hardware  and  software  issues  on  new  user  interfaces  within  the 
scope  of  telepresentations  and  teleteaching.  We  introduced  discoveries  which  we  gained  with  two  alternative  input 
devices:  interactive  wall  mounted  blackboards  and  a graphic  tablet  as  replacements  for  regular  computer  input  via 
mouse  or  keyboard.  Our  experiments  and  analysis  showed  that  input  devices  currently  available  on  the  market 
can  be  used  very  well  for  teleteaching  but  somehow  still  are  a compromise  and  lack  especially  in  the  provided 
size.  We  also  illustrated  that  the  use  of  these  input  devices  raises  new  problems  and  offers  additional  challenges 
in  the  software  design  of  the  user  interface.  Our  current  work  is  to  implement  a new  user  interface  that  addresses 
the  problems  we  discussed  and  to  test  and  evaluate  it  with  other  hardware  devices  we  are  about  to  install  in  the 
teleteaching  environment  at  our  university. 
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Abstract:  Project  DoCTA  focuses  on  the  design  and  use  of  artefacts  in  a set  of  collaborative  teleleaming  scenarios. 
From  a research  perspective,  the  exploratory  studies  being  carried  out  within  DoCTA  will  provide  us  with  insight  into 
the  processes  of  collaboration  enabling  us  to  identify  collaboration  patterns  and  further  our  understanding  of  how 
instructors,  students  and  other  learning  facilitators  organise  their  learning  and  work.  This  paper  describes  the 
theoretical  foundations,  the  four  collaborative  teleleaming  scenarios  and  the  evaluation  approach  of  project  DoCTA. 


1 Introduction 

DoCTA  (Design  and  use  of  Collaborative  Teleleaming  Artefacts)  is  a Norwegian  research  project  with  partners  at 
four  different  institutions  and  funded  by  the  Norwegian  Ministry  of  Education,  Research  and  Church  Affairs  under 
their  program  on  information  technology  in  education  (ITU).  The  project  partners  include  is  the  University  of 
Bergen  (leader),  Nord-Trondelag  College  (HiNT),  Stord/Haugesund  College  (Stord)  and  Telenor  Research  and 
Development,  Kjeller  (Telenor  FOU).  The  16  researchers  and  graduate  students  involved  have  various  backgrounds 
including  computer  science,  psychology,  sociology  and  education. 

Within  project  DoCTA  rhttp://www.if1.uib.no/docta)  we  focus  on  the  design  and  use  of  artefacts  in  collaborative 
teleleaming  scenarios  aimed  at  teacher  training.  Various  scenarios  utilising  the  Internet  will  be  used  to  engage  the 
students  in  collaborative  learning  activities.  Through  participation,  teachers  gain  experience  with  not  only 
collaborative  learning,  but  with  collaborative  teleleaming,  and  the  design  of  textual  or  visual  artefacts.  Each 
collaborative  learning  activity  has  been  designed  to  place  a strong  emphasis  on  active  engagement  through  both 
“hands-on”  practical  experience  and  explicit  reflection  on  the  collaboration  process.  Experiences  can  be  transferred 
back  to  their  own  schools,  and  ideas  about  collaboration  can  be  integrated  into  their  everyday  teaching. 

Four  different  collaborative  teleleaming  are  studied.  The  first  scenario,  Pedinfo,  is  a pilot  study  for  analysing  the  use 
of  TeamWave  Workplace  for  collaborative  activities  in  a graduate  university  course  at  the  University  of  Bergen 
(UiB).  The  next  two  scenarios,  IDEELS  and  Demeter,  involve  European  inter-cultural  simulations  where  the  goal  is 
to  design  a textual  artefact  (such  as  a treaty  or  policy  statement).  A fourth  scenario,  VisArt,  that  has  been  designed 
and  developed  explicitly  for  use  between  the  three  educational  partners  (UiB,  HiNT  and  Stord),  has  the  goal  of 
designing  a visual  artefact  to  be  used  in  teaching  a subject  of  choice. 

From  a research  perspective,  the  exploratory  study  being  carried  out  within  DOCTA  will  provide  us  with  insight 
into  the  processes  of  collaboration  enabling  us  to  identify  collaboration  patterns  and  further  our  understanding  of 
how  instmctors,  students  and  other  learning  facilitators  organise  their  learning  and  work.  The  community  of  study 
includes  teachers,  learners  and  facilitators  participating  in  the  various  collaborative  teleleaming  scenarios.  The  main 
research  question  has  been  formulated  to  ask  how  these  students,  teachers  and  facilitators  organise  their  work  given 
the  different  scenarios.  The  four  collaborative  teleleaming  scenarios  vary  with  respect  to: 

• actor  characteristics  (e.g.,  within  a common  community  vs.  disparate  and  divergent  cultural  backgrounds; 
similar  knowledge  and  preparation  vs.  different  knowledge  and  preparation;  etc.), 

• aspects  of  the  learning  activity  (e.g.,  text  based  vs.  visually  based;  well-defined  learning  tasks  and  goals  vs.  ill- 
stmctured  tasks  and  goals;  etc.), 

• the  kinds  of  artefacts  they  have  access  to  (e.g.,  the  artefacts  provided  in  the  various  internet  environments),  and 

• the  kinds  of  artefacts  they  are  to  design  (e.g.,  textual  or  visual) 

This  paper  provides  an  overview  of  the  theoretical  foundations  that  influence  our  research  and  describes  each  of  the 
four  collaborative  teleleaming  scenarios.  The  paper  concludes  with  a brief  description  of  the  evaluation  activities 
being  carried  out. 
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2 Theoretical  Foundations 


The  theoretical  perspectives  that  provide  inspiration  and  guidance  for  this  research  come  from;  computer  supported 
collaborative  learning  (CSCL),  in  particular  Salomon’s  (1992)  work  on  genuine  interdependence;  coordination 
science  (Malone  & Crowston,  1994);  sociocultural  perspectives  (Wertsch,  del  Rio  & Alvarez,  1995)  on  learning  and 
thinking;  and,  the  emerging  notion  of  distributed  learning  communities. 

Computer  supported  collaborative  learning  (CSCL)  is  an  emerging  paradigm  (Koschmann,  1996)  for  research  in 
educational  technology  that  focuses  on  the  use  of  information  and  communications  technology  (ICT)  as  a 
mediational  tool  within  collaborative  methods  (e.g.,  peer  learning  and  tutoring,  reciprocal  teaching,  project-  or 
problem-based  learning,  simulations,  games)  of  learning.  It  is  an  approach  to  ICT  in  education  that  emphasises  an 
understanding  of  language,  culture  and  other  aspects  of  the  social  setting  (Scott,  Cole  & Engel,  1992).  Its  intellectual 
heritage  can  be  found  in  social  constructivism  (Doise,  1990),  the  Soviet  cultural-historical  psychology  (e.g., 
Vygotsky  (1978),  Leontiev  (1978),  Davydov  (1988))  and  situated  cognition  (Suchman,  1987;  Lave,  1988). 

CSCL  research  involves  naturalistic  observations  being  used  in  an  exploratory  fashion  to  permit  a more  complete 
understanding  of  this  instructional  mode.  The  emphasis  is  on  the  process  and  not  so  much  focused  on  outcome. 
Evaluations  often  result  in  descriptive  studies  which  focus  on  artefacts  that  support  or  are  produced  by  teams  of 
learners  and  usually  contain  participant  accounts  of  their  own  work.  The  influence  of  CSCL  research  on  DoCTA  is 
evident  in  the  type  of  research  questions  we  ask  and  in  the  choice  of  conceptual  framework  for  organising  our 
evaluations. 

Guribye  & Wasson  (1999)  describe  the  underlying  conceptual  framework  adopted  in  DoCTA  as  an  integration  of 
three  different,  although  closely  interrelated  approaches:  activity  theory  (Leontev,  1978,  Engestrom  1987), 
distributed  cognition  (Hutchins,  1995),  and  situated  action  (Suchman,  1987,  Lave,  1988,  Mantovani,  1996).  All 
three  approaches  underscore  the  need  to  look  at  real  activities  in  real  situations  (Nardi,  1996,  our  italics),  and 
always,  in  some  way,  include  the  context  in  studies  of  human  activity.  One  of  the  goals  of  this  research  is  to  argue 
that,  together,  these  approaches  make  up  a rich  framework  for  describing,  evaluating  and  analysing  collaborative 
teleleaming  scenarios. 

Salomon  (1992)  is  concerned  with  effective  collaboration  and  he  argues  that  collaboration  will  only  be  effective  if 
there  is  genuine  interdependence  between  the  collaborating  students.  Genuine  interdependence  is  described  as  1)  the 
necessity  to  share  information,  meanings  conceptions  and  conclusion,  2)  the  necessity  for  division  of  labour  into 
complementary  roles,  and  3)  the  need  for  joint  thinking  in  explicit  terms.  In  project  DoCTA,  Salomon's  ideas 
influence  the  design  of  the  collaborative  tasks  given  to  the  students  - effort  is  placed  in  designing  tasks  which  create 
genuine  interdependence  between  the  students. 

Coordination  theory  (Malone  & Crowston,  1994)  provides  a means  for  specifying  (inter)dependencies  between,  and 
among,  actors,  goals,  activities,  and  resources  by  identifying  a dependency  type  (e.g.,  shared  resource)  and  a 
coordination  process  (e.g.,  group  decision-making)  for  managing  the  dependency.  In  their  work,  coordination  is 
defined  as  managing  dependencies  between  activities,  hence  they  have  focused  on  dependence  between  activities. 
Wasson  & Bourdeau  (1997)  report  that  viewing  collaborative  teleleaming  from  a coordination  theory  perspective 
offers  a means  of  understanding  the  inter-relationships  between  actors  and  entities  and  how  these  relationships  can 
and  should  be  supported.  Adopting  Salomon’s  ideas  about  genuine  interdependence  and  a coordination  science 
approach,  they  modelled  (inter)dependencies  between  actors  in  collaborative  teleleaming  scenarios  and  they  have 
extended  the  definition  of  coordination  to  be  managing  dependencies  between  activities  (Malone  & Crowston,  1994) 
and  supporting  (inter) dependencies  between  actors.  Wasson  (1997,  1998)  proposes  a set  of  actor 
(inter)dependencies  and  related  coordination  processes  for  collaborative  teleleaming.  The  importance  of  this  work 
for  DoCTA  lies  in  making  sure  the  technological  environment  within  which  the  students  carry  out  their  tasks 
provides  mechanisms  that  make  coordination  as  effortless  as  possible.  Otherwise  coordination  issues  can  become  a 
bottleneck  that  hinders  students  from  carrying  out  their  individual  and  collaborative  work. 
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The  final  area  providing  inspiration  is  summarised  as  follows  “...a  distributed  collaborative  learning  community  is  a 
"place'  that  is  created  by  the  individual  students  through  their  individual  and  collective  actions,  ...The  designers’ 
role  is  to  support  the  students'  work  of  creating  that  community^  and  in  such  a way  that  the  computer  systems 
become  integrated  parts  of  the  students’  activity  (Fjuk,  1998,  p,  70)”.  Furthermore,  Fjuk  (1995)  concludes  that 
collaborative  teleleaming  applications  need  to  have  both  a mediating  role  between  the  individual  learner  and  the 
peer-students  and  between  the  individual  learner  and  her  learning  tasks.  Thus,  collaborative  teleleaming  can  be 
understood  as  a medium  for  inter-human  interactions  and  articulation  of  individual  work.  As  designers  of  DoCTA 
learning  tasks  and  the  technological  support  environments,  we  must  keep  in  mind  that  our  role  is  to  provide  a 
supporting  environment  that  makes  coordination,  communication  and  collaboration  as  transparent  as  possible 
(Bourdeau  & Wasson,  1997)  enabling  students  to  create  their  own  learning  community.  The  environment  must  also 
support  both  individual  and  collaborative  work. 

3 The  Scenarios 

The  design  of  collaborative  teleleaming  requires  the  instructional  design  of  collaborative  learning  tasks  and  the 
specification  of  the  technological  design  comprising  the  learning  environment  configuration  including  the  tools  and 
services  available  (Wasson  & Bourdeau,  1998).  Four  scenarios  Pedinfo,  IDEELS,  DEMETER  and  VisArt  have 
been  studied. 

3.1  Pedinfo 

In  the  Pedinfo  scenario,  7 of  18  students  in  a graduate  class  on  research  issues  in  pedagogical  information  science  at 
the  University  of  Bergen,  are  geographically  dispersed  over  Norway  from  Alta  and  Mo  i Rana  in  the  far  North,  to 
Holmestrand  outside  Oslo,  to  Stord  and  Haugesund  on  islands  south  of  Bergen.  The  class  meets  physically  in 
Bergen  one  day  every  third  week.  The  challenge  is  to  support  the  students  in  their  creation  of  a distributed 
collaborative  learning  community.  Collaborative  learning  tasks  and  the  technological  environment  have  been 
carefully  designed.  The  technological  environment  is  comprised  of  their  own  email  system,  a word  processor  of 
their  choice,  and  groupware  TeamWave  Workplace  (TW,  http://www.teamwave.com).  In  TW  (which  uses  a room 
metaphor),  14  tools  are  provided  enabling  teams  to  easily  plan,  set  up  a database,  design,  share  documents,  link  to 
the  web,  chat  with  one  another  etc.  In  order  to  become  better  acquainted  with  one  and  another  and  to  become 
familiar  with  TW,  a collaborative  team  building  activity  was  designed  (see  team  building  room  in  figure  1). 


DISTRIBUTED  TEAM  BUIIDING  ACTMTY 
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Figure  1 Team  building  room  in  TeamWave  Workplace 
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The  team  building  activity  consists  of  first  interviewing  one  another  (virtually)  and  making  a presentation  of  the 
interviewee  to  the  others  (these  were  placed  in  a Participant  Profile  room),  and  a group  activity  (the  class  was 
divided  into  6 groups)  which  was  a survival  game  where  they  had  to  negotiate  a team  strategy. 

In  the  top  left  hand  comer  you  see  a doorway  tool  allows  easy  movement  between  rooms  - here  you  see  the  link  to 
the  Class  Room.  Beside  this  there  is  a voting  roster  tool  which  has  been  used  to  enable  teams  to  indicate  when  they 
have  completed  the  activity.  Next  to  this,  a file  viewer  tool  holds  the  directions  for  the  team  building  activities.  In 
the  bottom  left  and  right  comers,  file  holder  tools  contain  files  (actually  the  files  sit  on  the  TW  server)  for  the 
students  to  download.  To  access  the  files  the  students  just  press  the  Get  button  and  the  file  is  saved  onto  their  local 
machine  (to  place  a file  on  the  server,  one  just  uses  the  Put  button).  On  the  shared  whiteboard  (the  background)  text 
and  graphics  can  be  placed,  and  synchronous  communication  can  be  carried  out. 

TW  was  chosen  for  a number  of  reasons  including  that  it  grew  out  of  several  research  systems  developed  in  the 
Grouplab  Project  at  the  University  of  Calgary,  it  is  tailorable  by  the  learning  environment  designer  (it  is  extensible 
through  a easy  to  use  toolkit),  and  the  instmctor  and  students  themselves  (by  creating  rooms  and  though  access  to  a 
large  number  of  tools),  it  support  both  asynchronous  and  synchronous  collaboration,  and  not  the  least  it  is  both  easy 
to  learn  to  use  and  is  reasonable  priced.  Team  Wave  Workspace  will  also  be  used  in  the  IDEELS  and  VisArt 
scenarios. 

3.2  IDEELS 

Project  IDEELS  (Intercultural  Dynamics  in  European  Education  though  on-line  Simulation)  brings  together  a 
diverse  group  of  educators  and  researchers  from  five  tertiary  institutions  in  four  European  countries  who  share  a 
common  interest  in  simulation  and  games.  IDEELS  is  an  EU  Socrates  curriculum  development  project  with  partners 
at  the  University  of  Bremen,  Germany  (co-ordinators),  the  Polytechnic  University  of  Valencia,  Spain,  the  University 
of  Nice,  France,  and  Nord-Trondelag  College  and  University  of  Bergen,  Norway.  The  goals  of  IDEELS  include 
adding  impetus  to  the  curriculum  development  trend  towards  content  & process-based  learning  and  to  enhance 
European  competitiveness  by  providing  students  with  opportunities  to  learn  essential  cross-cultural,  linguistic  and 
negotiating  skills.  A generic  simulation  game  is  used  to  complement  existing  curricula  in  a wide  range  of  areas 
including  language  learning,  negotiation,  policy  studies,  political  science,  environmental  issues,  cross-culture 
communication,  law,  education,  and  computer  science.  In  IDEELS  simulations,  students  act  as  high-level 
negotiators,  consultants,  and  journalists  in  a fictional  world,  working  to  resolve  real  problems  of  importance  to  the 
European  Community  - problems  that  can  only  be  solved  through  co-operation  at  the  international  level. 

Teams  from  the  various  partner  institutions  take  on  different  roles  in  a simulation  conducted  on  two  levels: 
deliberations  within  a team  and  negotiations  between  or  among  teams.  Thus,  it  can  be  said  that  the  simulation 
scenarios  are  designed  to  require  (inter)dependence  among  both  team  members  and  between  teams.  Teams  are  given 
a common  mission  where  the  goal  is  to  produce  and  ratify  a jointly-written  document  (e.g.,  come  to  a consensus  and 
sign  either  a policy  or  treaty,  or  write  a set  of  recommendations).  For  example,  the  scenario  that  is  running  in  the 
fall  of  1998  (http://www.zait.uni-bremen.de/wwwgast/fzhb/ideels/public_html/)  requires  that  the  students  draw  up 
and  agree  on  a design  plan  for  the  educational  system  for  Eutropolis,  the  New  Eutropian  Capital.  Using  Salomon’s 
(1992)  definition  of  genuine  interdependence  it  can  be  argued  that  in  order  to  participate  in  the  game  there  is  a 
necessity  to  share  information,  meanings  and  concepts  about  the  topic  of  the  simulation,  a necessity  for  division  of 
labour  within  a team  in  order  to  build  up  the  teams  position  before  negotiation  with  the  other  teams,  and  a need  for 
joint  thinking  to  come  to  a consensus  so  a policy  or  treaty  can  be  signed  by  all  partners. 

The  Norwegian  participation  in  project  IDEELS  (UiB  and  HiNT)  is  organised  through  project  DoCTA.  The 
technological  environment  that  has  been  designed  for  the  participating  Norwegian  students  includes  their  own  word 
processor,  TeamWave  Workplace  and  OPUSi,  a computer-based  communications  system  that  is  accessible  through 
the  world  wide  web  and  was  developed  at  the  Department  of  Computer  Science,  University  of  Bremen.  OPUSi 
enables  the  students  to  send  written  messages  within  and  between  teams  as  well  as  providing  for  real-time,  online 
teleconferences.  As  in  the  Pedinfo  scenario,  the  Norwegian  students  participating  as  teams  in  IDEELS  are 
geographically  spread  over  Norway.  It  was  decided  that  OPUSi  does  not  provide  enough  support  for  these  students 
to  collaborate,  so  TeamWave  Workplace  has  been  added  to  support  the  collaboration  activities. 
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3.3  Demeter 


Demeter  (http://hugin.hsh.no/prosickt/demeter/index.htm)  is  a European  distance  education  project  run  by  ITE 
(Organisation  for  Internationalization  of  and  Innovation  in  Teacher  Education)  involving  15  European  Educational 
Institutions  in  Norway  (Stord),  Sweden,  Ireland,  Netherlands,  Germany,  Greece,  England,  Scotland,  Austria, 
Lithuania,  Portugal.  Project  DoCTA  will  be  evaluating  the  Norwegian  students’  (Stord)  participation  in  Demeter. 
Like  project  IDEELS,  Demeter  involves  role-playing  and  provides  for  cross-cultural  education  and  communication 
in  Europe  and  includes  inter-cultural  networks  for  democracy  education,  and  education  in  European  citizenship 
within  the  framework  of  teacher  education.  The  education  is  open  to  students  all  over  Europe  to  study,  compare 
and  produce  materials  about  issues  that  are  of  interest  for  most  parts  of  Europe.  Students  contribute  solutions  to 
local  problems  in  Europe  from  perspectives  true  to  both  their  own  cultural  background,  but  also  as  European 
citizens. 

Demeter  is  based  on  a parliament  metaphor  that  supports  the  vision  of  how  a true  cross-cultural  exchange  of  ideas, 
knowledge,  understandings  and  meanings  can  be  arranged.  Participation  in  the  parliament  requires  debate,  research, 
submitting  proposals  and  propositions  and  voting  on  the  decisions  that  flow  from  them.  When  a consensus  cannot  be 
reached,  respect  for  the  ideas  of  others  taking  part  in  the  democratic  process  must  be  shown.  Topics  discussed  in  the 
Demeter  Parliament  are  authentic  problems  that  occur  in  the  real  world  are  considered.  For  example  Children  Rights 
were  discussed  in  the  spring  of  1998  and  in  1999  the  topic  to  be  considered  will  be  the  preservation  of  heritage  sites. 
The  collaboration  tasks  are  designed  around  committee  work.  Students  participate  as  Members  of  Parliament  and 
join  committees  where  they  are  to  elect  a chairperson  and  secretary,  and  solve  a number  of  tasks  from  a problem- 
oriented  and  problem-solving  perspective.  The  committee  members  are  to  collaborate  to  write  a report  to  be 
delivered  in  a parliamentary  session  that  is  working  towards  a Children’s  Bill  of  Rights.  At  the  end  of  the 
Parliament  session,  members  of  parliament  write  a farewell  speech  which  summarises  the  work  they  have  done,  the 
decisions  this  has  brought  about,  what  has  been  learned,  and  how  the  parliament  session  has  influenced  their  view 
on  the  issue  that  has  been  discussed.  The  farewell  speech  also  describes  how  the  work  has  been  conducted  including 
both  a description  of  socialising  and  the  problems  encountered  while  having  to  cope  with  parliamentary  working 
methods.  Again,  it  can  be  argued  that  this  fulfils  Salomon’s  criteria  for  genuine  interdependence. 

The  technological  environment  is  the  Demeter  Parliament  accessible  through  the  web.  The  Parliament  is  a virtual 
world  that  includes  a Cafe,  a Library,  a Post  Office,  a Lobby  where  participants  can  post  messages  and  join 
discussion  groups,  and  it  provides  participants  with  a number  of  services  including  an  agenda,  access  to  experts,  a 
means  of  join  committees,  an  area  for  voting,  a means  for  chatting,  a forum  for  debate  and  help. 

3.4  VisArt 

While  IDEELS  and  Demeter  dealt  with  the  design  of  textual  based  artefact,  VisArt  has  been  designed  to  explore  the 
collaborative  design  of  a visual  artefact.  The  collaborative  task  will  involve  intra  team  collaboration  to  design  a TW 
room  for  use  in  teaching  some  subject  of  their  choice.  Teams  will  comprise  a student  from  each  of  the  Norwegian 
partner  educational  institutions  (UiB,  HiNT  and  Stord).  This  gives  a rich  mixture  of  students  with  various 
backgrounds.  The  UiB  students  are  graduate  students  studying  pedagogical  information  science  and  thus  have  a 
either  a background  in  both  pedagogy  and  information  science  or  just  information  science.  The  HiNT  students  are 
teachers  taking  an  introductory  evening  course  on  information  technology  and  education  thus  they  have  little  if  no 
backgound  in  the  use  of  information  and  communication  technologies.  The  Stord  students  are  third  year 
undergraduate  students  taking  a pedagogical  information  science  course  so  they  are  primarily  educated  as  teachers, 
but  have  been  taking  a number  of  information  science  courses  as  well.  We  feel  that  this  will  make  an  interesting 
blend  and  will  create  (inter)dependencies  between  the  students.  We  are  curious  to  see  how  they  divide  their  work 
and  reach  a consensus  first  on  what  they  will  design  (we  expect  the  students  with  more  pedagogical  background  to 
take  a lead  here)  and  second  how  they  will  carry  out  the  development  of  the  web  artefact  (we  expect  the  students 
with  more  technical  backgrounds  to  take  a lead  here).  The  technological  environment  being  designed  for  these 
students  includes  email,  TW,  their  own  word  processor  and  the  web.  Prior  to  the  design  task,  students  are  trained  in 
both  use  of  TW  and  in  collaboration. 
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4 Evaluations 


The  evaluation  aims  at  a naturalistic  study  of  how  participants  in  collaborative  tele  learning  organise  their  work  and 
learning  activities.  In  order  to  collect  data  about  the  activities  in  which  the  student  are  engaged,  different  methods 
and  techniques  are  being  used.  The  most  important  sources  of  information  will  be  derived  from  observing  the 
students  as  they  collaborate  and  interviewing  them,  and  also  from  electronic  logging  of  artefacts  used  for 
collaborating  (e.g.,  email,  shared  whiteboards,  chats,  to-do-lists)  and  artefacts  designed  (e.g.,  a TW  room)  during  the 
collaboration.  Our  theoretical  framework  influences  the  evaluation  question  we  have  asked  and  the  analyses  we  can 
make.  Guribye  & Wasson  (1999)  provides  a more  in  depth  rationale  to  the  choice  of  our  evaluation  approach.  For 
example,  three  graduate  theses  focus  on  using  an  Activity  Theory  perspective  to  examine  collaboration  patterns  in 
the  IDEELS  and  VisArt  scenarios.  We  are  supplementing  this  overall  focus  on  collaboration  patterns  and  how  the 
participants  organise  their  work,  with  several  graduate  theses  that  will  focus  on  evaluation  of  the  technological 
environment.  For  example,  one  thesis  is  focusing  on  the  usability  of  TW  by  monitoring  how  students’  opinions 
about  the  utility  of  the  tools  changes  from  their  initial  impression  to  a final  judgement  after  a semester  of  use. 
Another  thesis  is  focused  on  the  analysis  of  TW  data  logs  to  look  at  how  the  different  artefacts  were  used  for 
collaboration  activities.  A third  thesis  is  concerned  with  coordination. 
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Abstract:  Whereas  for  most  people  the  only  application  of  multimedia  technologies  in 
education  is  hypertext  arrangements  of  topics  plus  “page  turning,”  we  view  multimedia 
technologies  as  an  effective  way  of  visualizing  the  abstract  concepts  that  are  generally  hard  to 
grasp.  As  such,  we  attempt  to  create  virtual  representations  of  difficult  concepts  as 
teaching/leaming  aids.  The  important  issue  is  that  the  student  can  then  work  with  the  virtual 
representation  by  making  meaningful  changes  to  the  object  and  seeing  the  results. 

In  this  paper,  we  visualize  the  notion  known  as  the  Pythagorean  numbers,  i.e.  the 
equation  a^  + b - c^,  in  a way  that  it  can  be  manipulated  by  the  student.  As  a part  of  this,  we 
animate  a proof  for  this  equation.  In  addition,  we  implement  the  following  variation  of  the 
Pythagorean  theorem:  if  the  congruent  shapes  are  drawn  extended  from  each  side  of  a right 
triangle,  then  the  area  of  the  shape  which  extends  from  the  hypotenuse  is  equal  to  the  sum  of 
the  areas  of  the  shapes  which  extend  from  the  base  and  height. 


1.  Introduction 

A number  of  experiments  for  improving  the  quality  of  education  by  using  the  modem  technologies  are 
currently  underway.  Especially,  mathematics  educators  show  a great  deal  of  interest  in  the  impact  of  computers 
on  teaching  and  learning  mathematics.  A typical  example  of  new  ideas  in  this  area  is  providing  additional  visual 
and/or  auditory  effects  to  students.  This  is  commonly  achieved  by  simultaneous  use  of  data  in  different  forms 
such  as  text,  audio,  images,  graphics,  video,  animation,  and  so  on.  Generally  these  representations  involve  the 
usage  of  multimedia  tools  and  techniques.  We  will  explore  applications  of  multimedia  technologies  used  to 
develop  the  educational  tools  for  mathematics. 

Many  students  experience  difficulties  in  understanding  certain  concepts  presented  by  the  traditional 
teaching  methods.  For  such  students,  a computer  can  be  a useful  tool  to  enhance  the  students’  understanding  by 
providing  the  visual  and/or  auditory  effects.  Most  of  the  currently  existing  learning  and  teaching  tools  are 
nothing  more  than  mere  “page-turning”  programs  on  the  computer.  That  is,  these  kinds  of  tools  do  not  make 
the  best  use  of  the  strength  of  the  computers  as  a supplementary  method  to  typical  schooling  because  they  are 
just  a copy  of  the  textbook  on  computer.  We  will  examine  these  motivations  for  the  project  closely. 

Another  angle  of  this  research  is  investigation  of  supporting  theories  for  the  development  of  teaching 
tools  for  mathematics.  Many  existing  tools  adopted  behaviorism  as  their  basic  idea.  Since  this  philosophy  has 
been  the  cause  of  a number  of  fundamental  problems,  we  added  constructivism  and  cognitive  theory  to 
construct  a new  background  theory  as  an  alternative  to  behaviorism.  The  basic  concept  of  the  newly 
synthesized  theory  is  that  students  can  learn  the  new  knowledge  by  logically  connecting  it  to  already  existing 
knowledge.  A more  detailed  discussion  of  the  background  theory  will  be  presented  in  the  following  sections. 

To  demonstrate  how  teaching  tools  may  be  developed,  we  have  developed  several  modules  for  a 
variety  of  concepts  in  mathematics,  physics  and  chemistry.  As  an  example  of  selected  mathematical  concepts, 
we  will  present  a tool  for  understanding  the  Pythagorean  theorem  which  seems  very  easy  at  first  glance  but 
requires  much  more  work  to  achieve  mastery  level.  The  new  educational  theory  is  applied  at  each  step  of 
implementation  so  that  the  resulting  program  will  not  be  a mere  explanation  of  the  theorem. 


2.  Multimedia  Technologies  and  Education 
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In  this  section  we  discuss  the  steps  involved  in  the  development  of  multimedia  tools  for  educational 
purposes.  While  this  is  not  intended  to  be  a complete  list,  we  highlight  the  major  steps. 

One  of  the  most  important  tasks  in  developing  an  educational  tool  is  identifying  the  appropriate  topics. 
Computer  technologies  may  or  may  not  be  well  suited  for  all  subject  areas.  In  other  words,  in  some  cases  there 
are  no  apparent  advantages  in  using  multimedia  technologies  for  certain  topics.  Therefore,  to  use  multimedia 
technologies  for  an  educational  program,  the  proper  criteria  for  selecting  topics  has  to  be  established.  Although 
a detailed  discussion  of  the  criteria  is  beyond  the  scope  of  this  paper,  the  following  is  a sample  list:  1)  topics 
that  the  instructors  find  hard  to  explain  through  the  traditional  teaching  methods,  2)  topics  that  students  find 
hard  to  understand  from  the  typical  schooling  method,  3)  topics  that  are  explained  or  learned  better  if 
visual/auditory  effects  are  provided,  4)  topics  concerning  3-D  shapes,  5)  when  computers  can  provide  a 
simulated  environment  that  the  school  cannot  afford,  6)topics  where  students  need  to  solve  many  exercise 
problems  for  better  understanding,  and  7)  topics  that  involve  many  continuous  and  chaining  steps. 

Once  an  appropriate  topic  has  been  selected,  an  overall  environment  must  be  designed.  At  this  stage, 
the  task  must  be  broken  down  into  sub-problems  first.  Breaking  down  the  task  involves  both  analyzing  the 
content  of  topics  and  selecting  the  proper  presentation  method  for  the  topic.  In  the  content  analysis,  the 
following  three  questions  must  be  considered:  a)  What  goals  can  students  or  users  achieve  by  using  the  tool?  b) 
How  deeply  are  the  topics  dealt  with  in  the  program?  and  c)  How  much  related  information  or  knowledge  is 
provided  for  improving  students’  understanding? 

Choosing  the  appropriate  metaphor  and  media  types  for  each  parameter  that  is  relevant  to  the  topic  is 
another  important  task  in  this  stage.  Among  the  various  media  types  such  as  text,  pictures,  images,  sounds, 
animation,  and  videos,  the  most  applicable  types  for  the  topic  have  to  be  selected  in  order  to  achieve  the 
maximum  effect.  Furthermore,  the  organization  of  the  selected  media  should  be  considered  so  that  the  users 
can  get  information  efficiently. 


The  design  stage  follows  task  analysis.  Setting  the  appropriate  scenes  and  choosing  the  method  of  user 
interface  must  be  done  in  this  stage.  Each  scene  must  be  sketched  carefully  according  to  the  results  of  the  tasks 
break  down.  A point  to  be  considered  when  sketching  the  scenes  is  that  each  scene  has  to  contain  a proper 
amount  of  data  or  facts.  If  there  is  too  much  information  in  the  scene,  students  will  be  stunned  by  the  amount 
of  information  they  have  to  digest  and  may  lose  interest.  On  the  other  hand,  if  there  is  inadequate  information, 
the  users  will  be  busy  moving  from  one  scene  to  the  other  to  find  the  desired  information.  Another  point  to  be 
taken  into  account  is  the  flow  of  the  scenes.  The  strong  coherence  between  consecutive  scenes  is  necessary,  of 
course  unless  the  theme  has  changed.  The  smooth  flow  of  scenes  can  be  achieved  by  a well-designed  user 
interface.  An  important  requirement  is  that  users  must  have  no  difficulty  in  navigating  the  program.  To  offer  a 
better  navigational  method  to  users,  many  techniques  have  been  invented,  including  task  bars,  hot  words, 
dialogue  boxes,  and  using  animated  icons  and  histories. 

The  final  stage  in  developing  an  educational  tool  is  the  actual  implementation  and  coding.  All  the 
plans  chosen  during  the  previous  stages  have  to  be  synthesized,  organized  and  displayed  in  such  a way  that 
gives  the  maximum  effects  to  the  users.  The  ultimate  goal  here  must  be  that  the  users  will  enjoy  what  they  are. 
By  simply  using  a computer  mouse  to  point  and  click  on  a particular  picture,  word,  or  button,  stories  and 
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information  come  alive  on  a computer  screen.  The  phrase  “edutainment”  is  the  buzzword  for  this.  The 
explained  processes  are  summarized  as  presented  in  [Fig.  1]. 


3.  Synthesizing  Constructivism,  Cognitive  Theory  and  Behaviorism 

In  developing  an  educational  tool,  a consistent  educational  theory  is  needed  so  that  we  would  not 
digress  from  the  initial  set  of  goals  during  the  entire  development  process.  A combination  of  cognitive  theory, 
behaviorism  and  constructivism  will  provide  the  ideal  setting  for  the  development  of  educational  tools.  The 
synergized  theory  emphasizes  both  accumulating  personal  experiences  before  attempting  to  learn  a new  subject 
and  by  providing  enough  exercise  problems  to  enhance  student’s  ability  to  apply  the  concept  after  learning  it. 
The  following  characteristics  from  the  current  educational  theories  are  identified  and  will  be  resolved  into  the 
new  educational  framework:  1)  From  cognitive  theory  - Habitual  process  of  perceiving  and  thinking,  Thought 
that  forms  association,  Seek  interconnection  between  connected  ideas  and  individual  analogies,  and  Awaken 
knowledge  in  long-term  memory,  2)  From  behaviorism  - Learn  by  doing,  experiencing,  and  engaging  in  trial 
and  error.  Give  repetitive  stimulus  and  get  response.  Present  similar  stimulus  material,  and  Give  some 
reinforcement  immediately  after  the  response,  3)  From  constructivism  - Constructing  knowledge  rather  than 
mere  memorization.  Provide  complex  learning  environment.  Collaborate  or  cooperative  learning.  Examine  the 
same  material  from  multiple  metaphor,  and  Invoke  student’s  reflexivity 


The  tools  provide  the  following  features,  which  would  help  the  students,  learn  mathematics  step-by- 

step. 

• Building  the  preliminary  stage(s)  at  first  so  that  the  users  remove  the  unfamiliarity  from  the  new  topic  and 
make  an  experience  by  playing  with  that  topic  without  deep  touch 

• Presenting/explaining  the  definition  in  order  to  give  the  students  the  initial  idea  what  they  are  going  to  learn 

• Giving  the  related  background,  information,  and/or  knowledge  to  enhance  the  students’  understanding 

• Doing  the  proof,  if  necessary,  for  the  students  to  consolidate  what  they  are  learned  systematically 

• Generalizing  and/or  synthesizing  the  topic,  and  discussing  the  variations  to  raise  the  students’  interest  and  to 
encourage  their  curiosity  in  the  given  topic 

• Providing  enough  assessment  sets  for  the  students  to  know  how  the  subject  they  are  learning  is  applied  to 
various  situations 

We  feel  that  in  such  a framework,  the  designer  would  work  closely  with  the  educators  and  students  in 

the  design  and  implementation  of  the  learning  tool.  [Fig.  2]  illustrates  the  steps  to  be  followed. 


4.  An  Example:  Pythagorean  Numbers 

At  Arizona  State  University  Multimedia  Systems  Laboratory,  we  have  developed  a large  number  of 
modules,  each  targeted  to  demonstrate  how  various  technologies  may  be  effectively  utilized  for  delivery  of 
learning  material  in  the  fields  of  mathematics,  chemistry  and  physics.  We  present  in  detail  one  of  the  modules. 
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namely  the  Pythagorean  Theorem.  Although  this  demonstration  program  is  targeted  high  school  students,  it 
clearly  shows  that  the  techniques  are  applicable  to  all  levels  of  sophistication. 


4.1.  Selection  of  Topic 

The  selected  topic,  the  Pythagorean  theorem,  is  one  of  the  most  popular  theorems  in  trigonometry. 
Most  students  consider  this  theorem  to  be  easy  and  memorize  only  one  equation,  However,  there 

is  much  more  that  cannot  be  compacted  into  just  one  equation.  In  fact,  this  theorem  represents  the  special  case 
of  the  law  of  cosines  which  establishes  the  general  relationship  among  the  sides  and  angles  of  a triangle. 
Therefore,  to  understand  the  Pythagorean  theorem,  students  must  understand  the  law  of  cosines  and  possess  the 
basic  knowledge  about  the  concepts  of  angles  and  sides  of  a triangle,  and  the  triangle  itself  Moreover,  the 
proof  for  the  theorem  is  not  easy  even  to  the  medium  level  of  students. 


4.2.  The  Preliminary  Stage 

At  this  stage,  the  users  can  get  the  first  experience  of  the  relationships  between  angles  and  sides  of  a 
triangle  simply  playing  with  any  vertices  in  the  given  figure.  A brief  description  is  given  in  the  left  upper 
window  so  that  the  user  will  know  what  to  do  at  each  stage.  When  the  users  move  the  vertex  from  one  place  to 
the  other  place  on  the  right  side  of  window,  they  can  see  the  corresponding  changes  of  sides,  angles,  and  the 
areas  of  three  squares  that  are  made  by  each  side  of  a triangle.  They  also  see  the  changes  of  equations  which 
are  shown  on  the  left  lower  window.  Since  the  Pythagorean  theorem  comes  into  existence  when  the  given 
triangle  is  a right  triangle,  the  sound  effect  is  added  so  that  the  users  know  when  they  have  created  a right 
triangle.  The  users  can  get  a rough  idea  of  what  they  are  going  to  learn  by  just  playing. 


4.3.  Presentation  of  the  Concept  and  Definitions 

The  users  are  probably  ready  to  learn  the  new  topic  after  getting  the  first  experience  of  the  previous 
stage.  Therefore,  the  formal  definitions  of  the  Pythagorean  theorem  and  its  general  form  “Law  of  Cosines”  are 
presented  and  explained  at  this  stage  [Fig.  3].  The  figures  and  their  written  explanation  will  help  the  users’ 
understanding  of  both  concepts. 


4.4.  Providing  Background  Information 

The  user  may  or  may  not  have  enough  knowledge  to  understand  the  new  topic.  For  such  students, 
presenting  the  basic  knowledge  is  a necessary  step  in  the  new  teaching  and  learning  tools  for  mathematics.  Of 
course,  the  students  who  know  the  basic  knowledge  well  can  skip  this  stage  and  go  on  to  the  next  stage  directly 
without  wasting  their  time.  For  the  Pythagorean  theorem,  the  users  must  know  the  angles  and  sides  of  the 
triangle  and  the  triangle  itself  Therefore,  the  explanation  of  angles  and  triangles  are  given  at  this  stage. 

At  first,  the  users  can  see  the  general  definition  of  an  angle,  the  unit  of  measurement  for  an  angle,  and 
the  various  types  of  angles  on  the  left  window  [Fig.  4].  After  getting  some  basic  concepts  about  angles,  the 
users  can  pick  any  angle  name  to  get  the  details  of  that  particular  angle.  The  red  color  is  used  for  the  text  of 
angle  names.  The  red  colored  text  indicates  hotwords  which  evoke  the  other  events  when  pressed.  The  users 
can  see  the  definition  of  the  specific  angle  and  its  figure  by  pressing  the  angle  name.  On  the  figure,  the  users 
can  play  with  the  vertices,  and  see  the  changes  in  angle.  The  change  in  angle  is  calculated  and  shown  on  the 
right  lower  window.  If  the  angle  is  out  of  range  to  the  given  angle  type,  both  a written  message  and  the  sound 
alarm  are  produced. 

The  section  for  the  triangle  is  composed  similarly  [Fig.  5].  The  definition  of  the  triangle  and  its  sides 
are  given  as  well  as  the  various  types  of  triangles.  Like  the  angle  section,  the  triangle  names  are  formed  as 
hotwords.  By  pressing  them,  the  users  can  see  the  definition  of  the  particular  triangle  and  play  with  its  vertices 
to  see  the  changes  of  angles  and  sides.  The  measurements  of  three  angles  are  shown,  and  both  the  written 
message  and  the  sound  alarm  are  produced  when  any  one  angle  has  not  satisfied  the  condition  for  the  particular 
triangle  type. 
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Figure  3.  Presenting  Definition  Figure  4.  Background  info.:  Angles  Figure  5.  Background  info.:  Triangles 


4.5.  Proving  Stage 

Many  mathematical  concepts  require  a proof.  Most  students  find  it  hard  to  prove  a mathematical 
theorem  because  a complete  understanding  is  needed  in  advance.  If  the  visual  and  auditory  effects  can  be  given 
to  each  step  as  well  as  a written  explanation,  the  users  will  easily  understand  the  processes  of  proving  theorem. 
For  the  Pythagorean  theorem,  the  proof  can  be  easily  understood  if  the  animation  technique  is  used.  [Fig.  6] 
shows  the  selected  phases  of  thirteen  steps  for  proving  the  theorem  [Golshani,  Park,  Cho,  and  Friesen,  1997].  The 
users  can  see  each  phase  of  the  proof  in  the  right  window.  Its  corresponding  explanation  is  also  given  in  written 
form.  If  the  users  cannot  understand  a certain  phase,  they  can  go  back  and  forth  until  they  understand 
thoroughly.  One  thing  that  needs  attention  in  the  figures  is  that  the  same-colored  areas  between  the  adjacent 
phases  have  the  same  area.  Of  course,  the  reason  why  they  are  the  same  is  well  explained  within  the  text. 
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Figure  6.  Animation  of  proof  Selected  Examples  of  13  Phases 


4.6.  Generalizations 

By  now,  the  user  should  be  able  to  understand  the  topic  completely  by  synthesizing  all  the  things  they 
have  learned  from  the  previous  stages.  The  users  can  construct  one  solid  idea  about  the  topic.  As  illustrated  in 
[Fig.  7],  the  student  can  synthesize  the  Pythagorean  theorem  by  playing  with  any  vertices.  According  to  the 
movement  of  vertices,  the  areas  of  three  squares  are  also  changed.  However,  the  sum  of  two  small  areas  always 
equals  the  area  of  one  big  square,  which  satisfies  the  Pythagorean  theorem. 
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In  addition,  we  present  the  following  extension  of  the  Pythagorean  theorem.  As  presented  in  [Fig.  8], 
if  the  user  draws  an  arbitrary  shape  from  any  side  of  the  given  triangle,  the  congruent  shapes  are  automatically 
drawn  from  the  other  two  sides  of  the  triangle.  A simple  animation  shows  that  the  area  of  the  shape,  which 
extends  from  the  hypotenuse  is  equal  to  the  sum  of  the  areas  of  the  shapes  which  extend  from  the  base  and 
height. 


Figure  7.  Synthesizing  Stage  Figure  8.  Variation  of  Pythagorean  Theorem 


4.7.  Assessment  Stage 

The  new  teaching  and  learning  tools  for  mathematics  must  provide  a set  of  questions  by  which  the 
student  can  assess  his  understanding  of  the  topic.  If  the  users  understand  the  fundamentals  of  the  topic,  they  can 
solve  any  questions  and  use  the  already  acquired  knowledge  in  practice.  The  exercise  problems  are  a means  to 
confirm  how  well  the  users  understand  the  given  topic.  The  users  can  improve  their  abilities  through  problems, 
which  they  appear  with  various  types  in  the  real  world. 


5.  Conclusion 

On  the  whole,  the  currently  available  educational  tools  for  mathematics  that  are  made  by  using 
computer  technology  are  not  good  when  used  as  a supplement  to  the  traditional  class.  This  is  because  there  is 
no  enough  correlation  between  the  computer  scientists  and  the  mathematics  educators.  This  highlights  the 
difficulty  in  creating  of  effective  educational  applications.  This  research  is  an  attempt  to  engraft  the  computer 
scientist’s  technical  background  to  the  field  of  education  in  order  to  produce  effective  educational  applications. 
First  of  all,  the  computer  multimedia  technology  is  fully  utilized  to  develop  new  tools  for  teaching  and  learning 
mathematics.  In  addition  to,  the  whole  process  of  development  follows  the  new  theory  that  is  formed  by 
combining  constructivism,  cognitive  theory  and  behaviorism.  The  actual  implementation  is  performed  to 
demonstrate  how  the  new  tools  should  be  developed  under  this  theory. 
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Abstract:  Described  within  this  paper  is  an  adaptive  hypermedia  system  (AHS)  that 
utilises  symbolic  AI  and  connectionist  AI  to  provide  generic  student  modelling.  The 
needs  for  generic  tutoring  systems  are  discussed,  in  terms  of  a system  that  is  applicable  to 
a multitude  of  teaching  domains,  whilst  maintaining  diagnostic  facilities  of  the  student. 

The  hypermedia  architecture  is  based  on  a semantic-network  allowing  the  use  of 
automatic  reasoning  to  produce  weighted  links.  A type  of  student  model  is  employed  to 
record  information  about  the  student  so  that  the  weighted  links  can  be  tailored  for  the 
student's  interests.  A neural  network  is  used  to  grade  the  student  into  an  ability  level 
based  upon  their  interactions  with  tutorials.  A further  neural  network  is  used  to  recognise 
the  movements  students  make  as  they  browse  the  hypermedia  and  link  it  to  tasks  and 
abilities.  This  offers  the  potential  to  extract  information  about  the  student  without  direct 
dialogues. 

Introduction 

In  the  face  of  growing  economic  pressures,  Leeds  Metropolitan  University  is  seeking  increasingly 
to  use  educational  technology  and  computer  based  learning  to  solve  the  practical  problems  of  providing  for 
increasing  student  numbers  given  static  or  decreasing  resources  (Willis  1994),  a widespread  problem 
shared  with  many  other  educational  institutions  (Sillcom  1996).  A particular  interest  concerns  the 
educational  use  of  hypermedia.  This  is  seen  as  being  potentially  educationally  valuable  in  that  there  is 
evidence  to  suggest  that  learning  is  improved  when  a student  is  allowed  to  follow  pathways  of  their  own 
choice,  at  their  own  pace  and  able  to  monitor  their  progress  by  instant  feedback  questions  (Kibby,  Mayes 
1990). 

A major  concern,  however,  with  the  educational  use  of  hypermedia  is  that  of  user  navigation.  The 
issue  is  how,  on  the  one  hand,  to  prevent  the  user  from  becoming  overwhelmed  with  information  and  losing 
track  of  where  they  are  going,  whilst  on  the  other  hand  permitting  them  to  make  the  most  of  the  facilities 
the  hypermedia  offers.  One  approach  to  remedy  this  is  to  restrict  the  number  of  links  made  available  to  the 
student.  The  concern  that  this  might  lead  to  an  impoverished  set  of  learning  opportunities  can  be  countered 
by  the  use  of  adaptive  hypermedia,  seeking  dynamically  to  configure  the  available  links  as  the  user 
proceeds. 

The  educational  importance  of  adaptivity 

Our  argument  for  the  importance  of  adaptivity  in  an  education  hypermedia  environment  is  based 
in  part  on  the  claim  of  Elsom-Cook  (1989)  that  the  perfect  tutoring  system  should  be  able  to  slide  between 
the  two  extremes  of  total  constraint  and  total  absence  of  constraint,  according  to  the  student's  needs  and 
current  state  of  knowledge.  In  a similar  vein.  Hartley  (1993),  cf  (Ford  1995)  argues  that  when  there  is  a 
mismatch  between  the  strategy  of  the  learning  system  and  learning  style  of  the  student,  performance  is 
degraded,  suggesting  a need  for  support  for  different  styles  and  viewpoints  of  users.  With  this  in  mind,  one 
of  our  aims  is  to  facilitate  hypermedia  systems  with  the  ability  to  adapt  to  their  students'  needs  as  they 
progress  through  the  systems  (Mullier  1996).  To  this  end  we  have  developed  a prototype  hypermedia  shell, 
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which  provides  a combination  of  knowledge-based  representations  (which  we  refer  to  as  "semantic 
hypermedia")  and  neural  network  models  ("connectionist  modelling"). 

We  argue  that  this  approach  overcomes  two  major  concerns  in  the  domain  of  educational 
hypermedia.  One  is  that  since  a hypermedia  learning  systems  shift  the  responsibility  for  accessing  and 
sequencing  information  from  the  teacher  to  the  student,  this  may  entail  a cognitive  overload:  "The  number 
of  learning  options  available  to  learners  places  increased  cognitive  demands  upon  the  learners  that  they  are 
often  unable  to  fulfil"  (Jonassen,  Grabinger  1990).  A particularly  important  manifestation  of  this  cognitive 
overload  occurs  when  a user  becomes  "lost  in  hyperspace",  not  finding  or  being  presented  with  the  required 
information,  a problem  caused  by  the  complexity  associated  with  "having  to  know  where  you  are  in  the 
network  and  how  to  get  to  some  other  place  that  you  know  (or  think)  exists  in  the  network"  (Conklin  1987). 
Our  prototype  seeks  to  deal  with  this  problem  essentially  by  making  more  links  available  to  students  as 
their  knowledge  of  the  domain  is  judged  to  be  improving;  in  an  attempt  to  provide  the  sliding  scale 
approach  advocated  by  Elsom-Cook,  the  novice  user  of  our  system  is  freed  from  much  of  the  complexity 
associated  with  an  unfamiliar  system  teaching  an  unfamiliar  subject,  while  the  more  advanced  student  is 
freed  from  unhelpful  constraints. 

Dynamic  linking 

In  order  to  enable  to  enable  provision  of  a suitable  regime  of  dynamic  links  to  the  student,  our 
prototype  adds  to  the  standard  hypermedia  paradigm  (i)  a student  model,  (ii)  a semantic  network  to  describe 
the  domain  and  (iii)  a control  system,  the  "hypermedia  manager",  for  regulating  the  links  offered  to  a given 
user,  in  the  light  of  information  from  the  previous  two  modules.  The  student  model  comprises  a 
connectionist  device  for  characterising  a student’s  browsing  strategies  and  a device  for  grading  the  student 
according  to  their  ability  to  answer  questions  and  complete  tutorials.  The  semantic  network  (Bey non- 
Davies  et  al  1994)  is  a formal  method  of  describing  the  domain.  Its  use  imbues  the  system  with  the  ability 
to  perform  reasoning  processes  drawn  from  the  field  of  artificial  intelligence,  which  in  turn  allows  the 
system  automatically  to  generate  links  and  ascertain  the  student's  interests.  These  elements  are  referred  to 
collectively  as  "semantic  hypermedia"  and  are  described  further  in  the  following  section.  Finally,  the 
hypermedia  manager  utilises  information  from  the  student  model  and  the  semantic  hypermedia  and 
determines  which  links  are  offered  to  the  student,  according  to  the  student's  current  profile  and  the  domain 
semantics. 

This  architecture  allows  three  levels  of  dynamic  linking,  resulting  in  a highly  adaptive  system 
from  the  user  perspective.  One  level  of  dynamic  linking  is  provided  by  the  semantic  hypermedia  element  of 
the  system.  This  is  based  on  a semantic  network  using  typed  links  such  as  is-a,  has-part  and  kind-of 
(Beynon-Davies  et  al  1 994).  Production  of  this  network  is  the  responsibility  of  the  domain  author,  but  the 
system  is  able  to  use  it  to  produce  links  between  nodes  in  the  hypermedia  system  automatically.  It  achieves 
this  by  building  direct  connections  between  nodes  which  are  already  connected  indirectly  via  the  semantic 
structure,  and  by  weighting  them  according  to  the  strength  of  the  semantic  relationship,  construed  as  the 
number  of  semantic  links  indirectly  connecting  the  two  nodes.  Weighting  the  links  enables  the  system  to 
suppress  low  weight  links  and/or  dynamically  tailor  the  displayed  links  to  a given  student  in  order  to  reduce 
complexity  for  novice  students.  This  first  level  of  linking  is  the  most  processor  intensive,  since  the  whole 
domain  must  be  processed.  However,  these  links  are  generated  before  a student  encounters  the  system. 

The  second  and  third  level,  tailor  the  links  to  an  individual  and  are  achieved  whilst  the  system  is  in 
use.  The  second  level  of  dynamic  linking  requires  a system  to  record  information  about  the  student.  The 
student  model  provides  information  about  the  student's  ability  and  their  interests  in  terms  of  previous  node 
types  visited  (these  are  stored  in  the  Student  Experience  Record).  The  hypermedia  manager  uses  this  to 
modify  the  already  present  weights  stored  with  each  link.  Thus  the  links  are  dynamically  modified  for  a 
particular  student  at  a particular  instant  in  time.  A third  method  of  dynamic  linking  allows  a human  teacher 
to  modify  the  link  weights  so  as  to  favour  certain  areas  of  the  semantic  hypermedia.  For  example,  the 
teacher  may  want  the  student  specifically  to  look  at  a certain  area  of  the  domain.  This  would  be 
accomplished  by  assigning  a priority  to  certain  class  nodes  in  the  semantic  network.  The  system  then 
increases  the  weight  for  each  instance  of  this  class,  thus  making  it  more  likely  to  be  selected. 
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Semantic  hypermedia 


The  central  feature  here  is  the  use  of  a semantic  network,  from  which  the  hypermedia  model  that 
forms  the  basis  of  the  student’s  interaction  is  generated.  The  use  of  a semantic  network  has  certain 
advantages  from  the  author's  point  of  view.  First,  "they  are  natural  to  use,  since  related  concepts  tend  to 
cluster  together  in  the  network... [and]  ...  an  incompletely  or  inconsistently  defined  concept  is  easy  to  spot 
since  a meaningful  context  is  provided  by  those  neighbouring  concepts  to  which  it  is  already  linked" 
(Conklin  1987),  cf  (Beynon-Davies  1994)).  Further,  the  use  of  the  semantic  network  forces  the  domain 
author  to  think  about  how  to  structure  each  additional  part  of  the  domain  and  allows  each  subsequent  part 
to  be  typed  and  attached  to  the  semantic  network.  Again,  use  of  a semantic  network  may  help  solve  the 
problem  of  "linkitis"  (Conklin  1987),  whereby  an  author,  not  knowing  how  to  link  a node  to  an  existing 
hypermedia  network,  errs  on  the  side  of  caution  and  links  nodes  to  other  nodes  that  may  not  be  relevant; 
this,  coupled  with  the  fact  that  most  hypermedia  systems  have  no  facility  for  qualifying  one  particular  link 
as  more  or  less  relevant  than  another,  can  add  to  the  cognitive  burden  of  the  learner. 

Importantly,  these  advantages  for  the  domain  authors  translate  into  advantages  for  the  users  of  the 
system.  Structure  in  general  is  widely  seen  as  important  in  educational  hypermedia  systems:  "the  belief  that 
hypertext  can  mimic  human  associative  networks  implies  that  an  appropriate  method  for  structuring 
hypertext  is  to  mirror  the  semantic  network  of  an  experienced  or  knowledgeable  performer  or  expert" 
(Jonassen,  Wang  1993).  Yet  more  importantly  from  the  point  of  view  of  the  current  project,  it  is  the  use  of 
a semantic  network  that  forms  the  basis  for  the  automatic  generation  of  hypertext  links,  by  utilising 
automatic  reasoning  algorithms. 

A semantic  network  is  therefore  created  for  a domain  and  substantive  "content  nodes"  are  attached 
to  it;  these  content  nodes  are  intended  to  provide  information  of  potential  educational  value  to  the  student 
and  may  include  calls  to  different  media  drivers,  eg  to  play  a sequence  of  video.  It  is  not  necessary  to  link 
content  nodes  to  other  content  nodes,  since  they  are  already  connected  indirectly  through  the  semantic 
network.  Additionally  these  indirect  links  provide  information  about  the  strength  of  the  relationship 
between  any  two  nodes  in  the  resulting  hypermedia  system.  It  also  provides  information  about  how  the 
nodes  are  related,  in  terms  of  the  link  and  class  types  connecting  them  together. 

Using  a Solar  System  domain  as  an  example,  the  nodes  "Mars"  and  "Earth"  can  be  linked  by  an  is- 
a link  to  the  class  node  Planet.  This  gives  a simple  knowledge  base,  namely  the  knowledge  that  both  Mars 
and  Earth  are  planets.  The  system  can  now  automatically  link  Mars  to  Earth.  Further  direct  links  might  be 
made  from  Phobos  to  Mars  and  from  The  Moon  to  Earth,  and  this  would  allow  the  system  to  link  Phobos  to 
The  Moon,  again  providing  a useful  link  without  any  intervention  from  the  author.  Further,  links  can  be 
given  a weight  according  to  the  length  of  the  path  that  indirectly  connects  them. 

Student  model 

The  student  model  is  made  up  of  four  sub-components: 

• Browsing  Pattern  Recogniser  (BPR) 

• The  Fuzzy  Rule  Base  (FRB) 

• Tutorial  Supervisor  (TS) 

• Student  Experience  Record  (SER) 

These  sub-components  are  described  below. 

The  use  of  neural  networks  within  the  student  model 

Both  the  TS  and  BPR  are  neural  network  devices.  We  argue  that  the  use  of  neural  networks  offers  a 
potentially  attractive  way  of  surmounting  the  difficulty  of  recognising  unclear  or  complex  patterns 
(complex  patterns  made  by  a student  using  the  hypermedia  and  a potentially  dynamic  grading  of  questions 
and  students).  We  argue  this  partly  on  the  negative  grounds  that  the  alternative,  namely  writing  production 
rules  to  recognise  these  strategies,  is  likely  to  be  "difficult,  incomplete  and  non-versatile"  (Mullier  1996, 
Tsutsui  1991).  There  are  interesting  precedents  in  the  use  of  neural  networks  for  student  modelling  (e.g. 
(Bergeron  et  al  1989,  Tsutsui  1991,  Battitti,  Serra  1991),  and  in  the  current  case  building  the  neural 
networks  could  be  expected  to  be  quicker  than  building  the  equivalent  symbolic  AI  components.  Further, 
the  resulting  networks  are  likely  to  be  very  fast  in  execution  and  therefore  ideal  for  real  time  applications 
(Chiu  et  al  1991).  Other  useful  network  properties  are  their  pattern  recognition  ability  on  imprecise  data 
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and  their  potential  ability  to  adapt  to  situations  where  a student  has  achieved  a result  by  unconventional 
means.  We  maintain  a compelling  prima  facie  case  for  investigating  their  use  within  the  student  modelling 
provisions  of  our  hypermedia  shell. 

The  Tutorial  Supervisor  (TS) 

The  TS  measures  the  ability  of  the  student  by  examining  the  student's  interactions  with  tutorial 
nodes.  Such  nodes  may  present  the  student  with  a simple  question  and  answer  session  or  may  give  the 
student  a goal  to  achieve,  requiring  the  student  to  browse  the  hypermedia  and  return  to  the  tutorial  node  to 
complete  the  task.  The  results  obtained  from  the  student's  interaction  with  a tutorial  node  are  utilised  by  the 
TS  to  update  its  current  level  for  the  student.  Clearly,  it  is  fundamental  to  this  that  tutorial  nodes  are 
provided  in  the  hypermedia  system.  Such  nodes  however  are  seen  as  valuable  in  their  own  right,  in  that 
simply  browsing  through  a hypermedia  system  on  the  part  of  a learner  may  not  be  enough  to  produce  any 
appreciable  transferral  of  structural  knowledge,  and  it  is  likely  therefore  to  be  necessary  to  provide 
exercises  that  explicitly  test  the  learner’s  knowledge  of  the  domain  (Jonassen,  Wang  1993).  Tutorial  nodes 
are  essentially  empty  boxes  into  which  a particular  tutor  can  place  suitable  opportunities  for  learner 
interaction;  they  are  likely  therefore  to  be  places  within  the  hypermedia  where  the  student  is  questioned 
about  information  stored  in  nodes  elsewhere  in  the  domain.  They  may  make  convenient  points  of  entry  into 
the  domain,  as  a student  is  given  a goal  by  the  tutorial  node,  and  be  useful  intermediate  reflection  and 
debriefing  points. 

The  TS  operates  on  the  basis  of  student  interactions  with  these  tutorial  nodes.  It  takes  the  student's 
current  level  (for  new  students  this  will  be  the  lowest  level),  the  question/tutorial’s  current  difficulty  level 
and  the  result  of  the  interaction  between  the  question/tutorial  and  the  student.  It  outputs  an  updated  student 
ability  level.  We  argue  that  a specialist  type  of  neural  network  (Kohonen  self-organising  map)  is  a useful 
means  of  accomplishing  this,  since  it  can  adapt  to  different  domains  without  the  need  for  external 
intervention  (unsupervised  learning).  Our  experiments  with  such  networks  have  provided  results  which 
clearly  demonstrate  that  the  network  is  able  to  adapt  to  different  domains,  where  similar  student  levels 
provide  different  results  (an  expert  score  for  one  domain  may  be  in  the  range  of  90-100%  say,  whereas  in 
another  domain  it  may  be  70-100).  This  on-line  adaptability  is  only  available  with  unsupervised  neural 
networks. 

In  addition  to  judging  the  ability  of  students,  the  TS  is  also  able  to  adjust  the  difficulty  rating  of 
questions.  This  is  seen  as  valuable  on  the  basis  of  the  pedagogical  principle  that  it  is  useful  to  present  to  a 
student  questions  at  or  just  beyond  their  current  ability  (Bergeron  et  al  1989).  Whilst  the  domain  author  is 
expected  to  provide  an  initial  grading  of  questions  and  tutorials,  this  grading  may  turn  out  not  to  be 
appropriate  in  practice.  The  system  is  therefore  able  to  examine  the  interactions  between  the  population  of 
students  and  the  questions.  If,  say,  the  domain  author  grades  a question  as  difficult,  but  results  with  students 
show  that  students  of  novice  ability  score  highly  with  the  question,  then  the  question  appears  to  have  been 
graded  as  being  harder  than  it  actually  is.  If  this  situation  occurs  then  the  benefits  of  providing  a student 
with  questions  at  or  just  beyond  their  ability  level  is  lost.  The  student  may  become  frustrated  by  answering 
questions  that  are  below  their  ability  and  is  therefore  less  likely  to  improve  their  ability  level  (Bergeron 
1989).  It  is  therefore  important  to  maintain  an  accurate  level  for  each  question  and  tutorial.  The  re-grading 
of  questions  is  a gradual  process,  since  it  is  unwise  to  alter  a question's  grading  with  respect  to  an 
interaction  with  just  one  student.  The  TS  is  able  to  re-grade  questions  dynamically  and  continually  without 
making  extreme  judgements  based  upon  one  observation. 

The  Browsing  Pattern  Recogniser  and  Fuzzy  Rule  Base 

The  BPR  is  designed  to  identify  movement  patterns  and  strategies  employed  by  the  student  as  they 
move  through  a hypermedia  domain.  A number  of  specific  hypermedia  browsing  strategies  have  been 
identified  by  empirical  studies  of  users  of  hypermedia  systems  (McAleese  1 993).  Examples  are  pathiness, 
ringiness,  loopiness  and  spikiness.  Such  patterns  have  been  identified  as  offering  the  potential  of  providing 
"psychologically  important  information  about  the  user"  (Canter  et  al  1985).  Recognition  of  the  strategy 
being  adopted  by  a given  user  during  a given  interaction  is  potentially  useful  to  both  the  system  and  to  a 
human  tutor,  but  tends  to  be  very  complex  (Canter  et  al  1985),  not  least  because  a user  may  adopt  more 
than  one  strategy  for  a particular  session,  and  may  appear  to  be  utilising  different  strategies  at  different 
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times  (Mullier  1996).  The  BPR  seeks  to  provide  such  recognition  and,  because  of  the  task’s  inherent 
complexity,  adopts  a neural  network  approach. 

Once  browsing  patterns  have  been  identified  then  the  system  attempts  to  utilise  the  information  to 
determine  what  it  is  the  student  is  doing.  Such  information  linking  browsing  patterns  to  particular  tasks 
does  not  currently  exist.  Therefore  the  system  provides  a unit  that  is  capable  of  extracting  this  information 
as  the  hypermedia  is  used.  The  information  linking  browsing  patterns  to  tasks  and  particular  abilities  of 
students  is  provided  by  the  tutorial  nodes  and  the  TS  respectively.  Tutorial  Nodes  induce  the  student  to 
perform  a particular  task  as  defined  by  the  author.  The  student  then  completes  the  task  using  a browsing 
pattern,  the  system  then  records  this  information.  Mapping  tasks  and  abilities  to  browsing  patterns  in  a 
direct  manner  is  not  suitable,  since  different  browsing  patterns  may  be  used  for  similar  tasks  and  vice  versa. 
Instead  it  is  necessary  to  pick  out  overall  trends  (generally  high  ability  users  use  browsing  pattern  x to 
locate  a specific  piece  of  information,  for  example)  over  a period  of  prolonged  exposure  with  many 
students.  This  is  achieved  by  the  utilisation  of  fuzzy  logic.  A Fuzzy  Rule  Base  (FRB)  is  able  to  adapt  to  a 
complex  and  changing  environment  as  the  system  is  in  use,  in  a similar  fashion  to  a neural  network.  The 
important  difference,  however,  is  that  the  rules  inside  can  be  read  and  interpreted  by  a human.  Therefore 
the  hypermedia  system  shell  (called  Hypemet)  is  a hypermedia  research  aid,  potentially  capable  of 
providing  the  important  information  describe  by  Canter  et  al  (1985). 

Several  networks  have  been  trained  and  tested  on  data  generated  from  programs  written  to  produce 
a rich  set  of  simulated  student  browsing  patterns.  Results  of  tests  of  the  networks  have  again  been 
encouraging.  For  the  BPR  a back-propagation  multi  layer  feed  forward  network  has  proved  to  be  an 
effective  network  type,  showing  high  levels  of  accuracy  in  pattern  recognition.  A Kohonen  network, 
however,  has  also  proved  able  to  recognise  browsing  patterns  and  in  addition  offers  the  promise  of 
detecting  previously  unidentified  patterns. 

The  Student  Experience  Record  (SER) 

The  SER  uses  an  overlay  model  to  build  from  a history  of  nodes  in  the  semantic  network  visited  by  the 
student.  This  can  be  seen  as  a characterisation  of  the  student’s  interests.  This  is  stored  for  eventual  use  by 
the  dynamic  linking  system,  which  seeks  to  tailor  links  more  closely  to  the  student’s  interests  as  they  use 
the  system.  The  overlay  model  records  information  about  particular  nodes  in  the  domain.  Information  such 
as  visited/unvisited  indicates  a student’s  interests.  Information  regarding  how  much  a student  knows  about  a 
particular  concept  can  also  be  inferred  from  results  with  tutorial  nodes.  The  SER  is  not  covered  in  detail 
here  since  it  draws  heavily  from  related  work  (Brusilovsky  1996). 

Conclusions  and  further  work 

This  paper  has  described  a novel  approach  to  adaptive  educational  hypermedia.  By  allowing  for  a 
sliding  scale  of  constraint,  from  complete  constraint  to  total  freedom,  upon  a student’s  exploration  of 
hypermedia  material  (in  terms  of  the  number  of  links  offered).  The  approach  currently  being  prototyped 
represents  a promising  means  of  alleviating  navigation  difficulties  whilst  allowing  the  student  to  take 
advantage  of  the  hypermedia  material,  without  swamping  them  with  too  much  information.  The  system 
currently  exists  as  a proof  of  concept  prototype  in  that  it  operates  highly  successfully  with  simulated 
student  data,  thus  validating  the  pedagogy  in  this  technical  sense.  An  immediate  task  involves  field  tests  of 
the  system.  A large  number  of  students  using  it  for  an  extended  period  is  required.  An  important  aspect  is 
the  use  of  the  system  in  a range  of  domains,  since  the  system  is  designed  specifically  to  be  applicable  to  a 
multitude  of  domains  and  its  modular  design  facilitates  the  incorporation  of  different  semantic  networks. 

In  summary,  then,  the  approach  to  adaptive  educational  hypermedia  adopted  by  our  prototype 
promises,  we  believe,  major  educational  advantages.  A range  of  interesting  and  important  issues  remain  to 
be  addressed,  however.  The  continual  need  to  provide  for  increasing  numbers  of  students  with  a wide  range 
of  abilities  makes  continued  investigation  of  such  issues  imperative. 


586 


References 


Battitti,  R.,  Serra,  R.  (1991).  Neural  Networks  for  Intelligent  Tutoring  Systems.  Artificial  Neural  Networks: 
Proceedings  of  ICANN‘91,  Kohonen  T (ed.). 

Benyon-Davies,  P.,  Tudhope,  D.,  Jones,  C.  (1994).  A Semantic  Database  Approach  to  Knowledge-based 
Hypermedia  Systems.  Information  and  Software  Technology  volume  36  no  6. 

Bergeron,  B.,  Morse,  A.,  Greenes,  R.  (1989).  A Generic  Neural  Network  Based  Tutorial  Supervisor  for 
C.A.I.  14th  Annual  Symposium  on  Computer  Applications  in  Medical  Care:  IEEE  Publishing 

Brusilovsky,  P.  (1996)  Adaptive  Hypermedia,  an  attempt  to  analyse  and  generalize.  Multimedia,  Hypermedia,  and 
Virtual  Reality.  Brusilovsky  P,  Kommers  P,  Streitz  N (edsj.  Lecture  Notes  n Computer  Science,  Vol.  1077:  Springer- 
Verlag, 

Canter,  D.,  Rivers,  R.,  Storrs,  G.  (1985).  Characterising  User  Navigation  through  Complex  Data  Structures.  Behaviour 
and  Information  Technology  Vol.  4.  Issue  2 1985. 

Chiu,  C.,  Norcio,  A.  F.,  Petrucci,  K.  E.  (1991).  Using  Neural  Networks  and  Expert  Systems  to  Model  Users 
in  an  OO  Environment,  in  IEEE  International  Conference  on  Systems.  Man  and  Cybernetics,  volume  I IT. 
IEEE  Publishing 

Conklin,  J.  (1987).  Hypertext:  An  Introduction  and  Survey.  IEEE  Computer  September  1987,  p 17-41. 

Elsom-Cook,  M.  (1989).  Guided  discovery  learning  and  bounded  user  modelling.  Self  J (ed.)  Artificial 
Intelligence  and  Human  Learning:  Intelligent  Computer-aided  Instruction:  Chapman  and  Hall 

Ford,  N.  (1995).  Hypermedia  and  Cognitive  Ergonomics  in  Engineering  Education.  Johnson  A R,  Eames  E 
W,  Flori  R E (eds.^  Proceedings  of  The  International  Conference  - Hypermedia  in  Sheffield  '95. 

Hartley,  J.  R.  (1993).  Interacting  with  multimedia.  University  Computing  No  15 

Jonassen,  D.  H.,  Grabinger,  R.  S.  (1990).  Problems  and  Issues  in  designing  Hypertext/Hypermedia  for 
Learning.  Jonassen  D H,  Mandl  H (eds  ) Hypermedia  for  Learning:  Springer-Verlag 

Jonassen,  D.  H.,  Wang,  S.  (1993).  Acquiring  Structural  Knowledge  from  Semantically  Structured 
Hypertext.  Journal  of  Computer-Based  Instruction.  Vol.  20  No.  1 pi -8. 

Kibby,  ,M.  Mayes,  T.  (1990).  Learning  about  Learning  from  Hypertext.  Jonassen  D (ed)  Hypermedia  for 
Learning:  Springer-Verlag. 

McAleese,  R.  (1993).  Navigation  and  Browsing  in  Hypertext.  In  McAleese  R.  (ed.j.  Hypertext  Theory  into 
Practice:  Blackwell  Scientific. 

Mullier,  D.  J.  (1996).  Integrating  Neural  Network  Technology  with  Hypermedia.  Henno  J (ed.^ 
Proceedings  of  Hypermedia  at  Tallinn,  Estonia 

Skillcom,  D.  B.  (1996).  Using  Distributed  Hypermedia  for  Collaborative  learning  in  Universities.  The 
Computer  Journal,  vol.  39,  no  6, 471-482. 

Tsutsui,  S.  (1991).  Knowledge  Based  versus  Neurocomputing.  1991  IEEE  International  Conference  on 
Systems,  Man  and  Cybernetics,  Vol.  Ill:  IEEE  Publishing. 

Willis,  N.  (1994).  Multimedia  in  a University  teaching/Leaming  Environment.  Proceedings  of  2nd 
Interactive  Multimedia  Symposium,  Perth,  January  1994. 


587 


Maintaining  Information  Awareness  with  Irwin 


D.  Scott  McCrickard 
College  of  Computing  and  GVU  Center 
Georgia  Institute  of  Technology 
Atlanta,  GA  30332-0280 
mccricks@cc.gatech.edu 


Abstract:  The  ongoing  explosion  of  Internet  usage  has  many  positive  effects  for  the  educational 
community,  but  it  makes  it  difficult  to  stay  updated  on  the  status  of  information  that  is  important. 
To  address  this  need,  we  created  Irwin,  a set  of  configurable  information  monitoring  tools  that 
endeavor  to  maintain  a desired  level  of  awareness  for  different  types  of  information  with  minimal 
interruption  to  other  important  tasks.  This  paper  describes  Irwin  and  examines  how  four  users  used 
it  over  a five-month  period. 


Introduction 

The  ongoing  explosion  of  Internet  usage  has  many  positive  effects  for  the  educational  community.  Email  can 
connect  colleagues  separated  by  thousands  of  miles.  Usenet  news  can  allow  professors  and  students  to  communicate 
outside  of  the  classroom.  The  World  Wide  Web  provides  both  essential  information  and  much  needed  distractions. 
But  with  these  positive  effects  come  new  problems. 

Although  this  information  is  readily  available,  it  is  updated  at  irregular  intervals,  making  it  difficult  to  keep  track  of 
changes.  For  example,  a professor  may  want  to  monitor  a class  newsgroup  on  the  days  leading  up  to  an  assignment 
due  date,  or  a researcher  may  want  to  be  updated  on  the  score  and  status  of  a ball  game  on  the  evening  before  a 
paper  due  date.  Another  problem  is  that  information  can  take  a significant  time  to  download  and  display.  Consider 
the  time  required  to  download  a weather  report  from  the  Web,  complete  with  colorful  icons  and  advertisements,  at  a 
time  when  a student  is  late  for  a class  across  campus. 

People  in  academics  often  want  or  need  to  be  aware  of  changes  in  the  types  of  information  in  these  examples,  but 
they  cannot  (or  should  not)  spend  all  of  their  time  reading  mail,  scanning  newsgroups,  and  surfing  the  Web.  One 
solution  is  to  use  visualization  and  communication  tools  to  help  maintain  a desired  level  of  awareness.  These  tools 
should  include  an  omnipresent  display  to  allow  the  user  to  check  on  the  status  of  resources  with  little  or  no  effort, 
yet  the  tools  must  use  limited  screen  space  as  the  user  will  often  want  to  use  other  applications  that  require 
significant  screen  space.  We  try  to  achieve  a balance  between  these  competing  requirements  with  our  Irwin 
information  awareness  system. 

Irwin  (Information  Resource  Watching  In  a Nutshell)  is  a set  of  tools  that  uses  graphical,  auditory,  and  textual 
communication  mechanisms  to  help  the  user  maintain  a desired  level  of  information  awareness.  This  paper  outlines 
ways  in  which  Irwin  can  assist  students  and  faculty  in  maintaining  awareness,  and  it  describes  a field  study  that 
examines  how  four  academicians  used  Irwin  to  maintain  information  awareness  over  a five-month  period. 


Irwin 


Irwin  (McCrickard  & Rowan  1996)  monitors  Internet  information  resources  and  alerts  the  user  of  updates  and 
modifications.  Irwin  consists  of  a set  of  hypertools,  small  reusable  programs  that  can  run  simultaneously  and  share 
information.  The  central  tool  in  Irwin  handles  the  visualization  and  user  interactions,  while  the  remaining  tools 
process  the  information  from  each  resource  and  update  the  visualization  tool  when  important  changes  occur. 
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The  information  resources  monitored  by  Irwin  can  include  email  folders,  Usenet  newsgroups,  Web  pages,  and 
weather  data.  The  email  and  newsgroup  hypertools  monitor  the  messages  in  a folder  or  group,  alerting  the  user  when 
new  messages  arrive.  The  Web  tool  summarizes  headers,  lists,  and  hypertext  links  on  a Web  page,  allowing  the  user 
to  monitor  news  wires  and  hotlists.  The  weather  tool  monitors  the  weather  conditions  and  forecast  for  a given  city. 

When  users  initially  start  Irwin,  it  tries  to  determine  their  informational  interests  by  examining  their  email  files  and 
Web  bookmarks.  (Since  users  are  not  required  to  answer  a list  of  questions  at  startup,  they  are  often  more  likely  to 
use  Irwin.)  The  user  can  then  configure  both  the  information  monitored  and  the  way  in  which  it  is  displayed  after 
getting  used  to  the  power  and  flexibility  of  Irwin. 

To  show  both  an  overview  and  details  simultaneously,  Irwin  uses  a confluent  zoom  display.  A confluent  zoom 
provides  multiple  views  of  a resource  simultaneously  with  each  view  having  a different  level  of  granularity.  With  a 
confluent  zoom,  the  user  can  see  overview  and  details  with  minimal  effort,  and  the  context  of  the  detailed  views  is 
maintained  at  all  times  by  the  coarser  views.  Since  intermediate  views  have  proven  useful  in  graphical  tools  such  as 
map  browsers  and  medical  imagery  (Plaisant,  Carr,  & Hasegawa  1992,  Plaisant,  Carr,  & Shneiderman  1995),  we 
expect  that  the  intermediate  views  will  smooth  the  transition  from  the  high-level  overview  to  the  detailed  zoom 
view. 


IMAGE  NOT  AVAILABLE 

The  image  displayed  here  is  a screenshot  of  Irwin.  In  its  confluent  zoom,  Irwin  employs  an  auditory  cue,  an  icons 
view,  a navigation  bar,  and  several  textual  views.  In  the  icons  view  (the  leftmost  view  in  the  above  image),  the  state 
of  each  resource  is  reflected  by  an  icon.  When  the  resource  changes,  the  appearance  of  the  icon  changes  and  an 
auditory  cue  is  played.  If  user  selects  an  icon,  the  other  views  are  updated  to  show  information  about  the 
corresponding  resource.  A navigation  bar  (second  from  the  left)  shows  a syntactic  encoding  of  the  resource  contents 
and  can  be  used  to  select  the  messages  displayed  in  the  textual  views  (the  two  rightmost  views).  Users  can  configure 
the  orientation  and  placement  of  the  icons  with  respect  to  the  other  views  and  can  even  choose  to  hide  the  other 
views  until  some  event  happens,  e.g.  an  icon  is  clicked. 


Auditory  cues 

Auditory  cues  provide  a means  to  attract  users  to  a change  that  has  occurred.  They  may  be  the  only  way  the  user 
knows  about  the  change  if  the  display  is  obscured  or  not  in  the  user's  current  line  of  sight. 

We  selected  sounds  that  are  distinctive  but  are  not  distracting  and,  most  importantly,  are  short.  Each  is  less  than  two 
seconds  in  duration,  which  we  expect  will  be  long  enough  to  be  noticed  but  not  too  long  to  become  an  annoyance.  In 
our  initial  configuration,  we  tried  to  select  sounds  that  were  indicative  of  the  resource;  for  example,  a dog  bark  for 
mail  (dogs  always  bark  at  the  mail  carrier)  and  a splat  for  news  (similar  to  a newspaper  hitting  a front  porch).  Of 
course,  the  user  can  select  from  a list  of  sounds  (we  provide  twenty)  or  create  and  use  their  own  sounds. 


Icons 

Icons  are  used  in  interfaces  because  they  provide  a universal  representation  in  a small  amount  of  space.  In  a WIMP 
environment  they  can  both  provide  information  and  invite  the  user  to  click  to  obtain  more  information. 

IMAGE  NOT  AVAILABLE 

The  image  displayed  here  shows  several  of  the  icons  used  by  Irwin.  Each  icon  is  a 16-by- 16-pixel  bitmap  that 
reflects  the  status  of  information  resources  and  provides  a gateway  to  more  detailed  views.  The  color  of  the  icons 
change  to  reflect  the  recency  of  updates  to  the  resource:  the  resource's  icon  is  originally  black,  but  when  a resource 
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is  updated  its  icon  changes  to  red.  Over  time,  if  the  resource  is  not  updated  its  icon  color  fades.  It  never  fades 
entirely  to  black  until  the  user  clicks  on  it. 

In  addition,  the  bitmap  itself  changes  based  on  the  state  of  the  resource.  The  weather  icon  changes  its  appearance  to 
a sun,  a cloud,  or  rain  to  reflect  the  current  weather  conditions,  and  the  spider  icon  changes  to  a dead  spider  when  its 
Web  site  is  not  accessible.  These  icons  show  (top  to  bottom)  the  mail,  sun,  rain,  live  Web,  and  dead  Web  bitmaps. 


Navigation  bar 

Irwin  needs  a method  for  visualizing  and  navigating  a list  of  items,  such  as  an  email  address  list  or  a list  of  news 
headers  from  a Web  page.  While  a scrollbar  is  a widely  used  navigation  tool,  it  shows  little  information  about  the 
content  of  the  list.  Irwin's  navigation  bar  leverages  the  familiar  scrollbar  metaphor  by  providing  the  same  basic  look 
and  functionality  as  a scrollbar  (arrows,  a trough,  a thumb),  but  the  trough  space  of  a navigation  bar  is  used  not  only 
for  navigating  the  list  but  also  for  providing  an  overview  of  the  contents  of  the  list.  Each  item  in  the  list  is 
represented  by  a line  in  the  trough.  The  thumb  (the  black  box  surrounding  several  graphical  lines)  indicates  the 
elements  that  are  shown  in  the  textual  listbox.  To  encode  information  about  the  list,  we  used  two  classes  of 
encodings:  syntactic  and  semantic. 

A syntactic  encoding  is  based  only  on  the  structure  of  the  word  being  encoded.  Advantages  of  syntactic  encodings 
are  that  they  require  no  user  input  and  they  are  consistent  between  sessions  and  between  users.  In  Irwin’s  syntactic 
encoding,  a word  or  phrase  is  represented  with  a series  of  blocks  of  pixels,  where  each  pixel  block  represents  a 
character  in  the  word.  To  differentiate  between  words,  Irwin  colors  the  blocks  that  correspond  to  vowels  such  that  'a' 
is  red,  'e'  is  orange,  'i'  is  yellow,  'o'  is  green,  and  'u'  is  blue. 

IMAGE  NOT  AVAILABLE 

For  example,  the  image  displayed  here  shows  a portion  of  a navigation  bar  representing  several  news  postings  by 
Annie  Anton.  Her  login  (anton)  is  represented  by  a red  block,  two  black  blocks,  a green  block,  and  a black  block. 

We  expect  the  syntactic  encoding  to  be  most  useful  for  mailing  lists  and  newsgroups  where  a small  number  of 
people  are  responsible  for  a large  amount  of  the  activity.  In  the  previous  example,  Annie  Anton  regularly  posts 
computer  jobs  and  internships  to  a local  news  group  - by  using  Irwin,  her  postings  can  be  identified  with  a glance. 
Also,  the  syntactic  encoding  works  well  for  course  newsgroups  where  the  students  post  questions  and  concerns  and 
the  professor  posts  responses  and  clarifications.  Since  the  encoding  for  the  professor  would  appear  quite  frequently, 
it  should  be  easy  to  recognize. 

A semantic  encoding  uses  information  about  the  meaning  of  each  list  item  to  create  the  encoding.  In  Irwin,  clusters 
of  related  items  are  created,  and  the  pixel  blocks  for  items  in  a cluster  are  identically  colored.  The  clusters  can  be 
created  automatically  using  the  Lance-Williams  dissimilarity  update  formula  (Frakes  & Baeza-Yates  1992),  or  the 
user  can  specify  the  clusters  using  keyword  matching.  The  Irwin  screenshot  earlier  in  this  paper  showed  a navigation 
bar  using  semantic  encoding,  where  email  messages  are  colored  based  on  the  sender  group  (faculty,  staff,  students, 
other). 

We  expect  the  semantic  encoding  will  be  most  useful  for  resources  with  large  amounts  of  information  on  a number 
of  different  topics.  For  example,  news  articles  taken  from  a Web  page  can  be  clustered  based  on  the  topics  of  the 
articles,  with  the  most  frequently  occurring  topics  being  colored.  Initially,  the  topics  can  be  created  automatically, 
then  later  the  user  may  want  to  specify  topics  of  interest. 

We  also  encode  the  time  at  which  the  message  was  stamped  by  indenting  the  message  according  to  the  hour;  for 
example,  messages  received  at  3 PM  will  be  indented  three  blocks.  This  is  intended  to  group  messages  by  arrival 
time,  thus  facilitating  searches  and  browsing.  If  users  know  the  time  at  which  a message  arrived,  they  can  identify 
the  range  of  messages  in  which  it  must  fall  using  the  navigation  bar. 
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Textual  views 


Graphics  alone  might  not  show  all  the  information  necessary  to  understand  a message  - at  some  point  a textual  view 
will  likely  help.  Irwin  incorporates  two  textual  views:  a header  list  and  a message  view.  Irwin  constructs  lists  of 
headers  for  each  resource,  the  senders  for  email  and  Usenet  news,  abbreviated  headers  for  Web  pages,  and  days  of 
the  week  for  the  weather  forecast.  If  the  user  selects  an  item  from  the  list,  the  message  view  shows  a more 
informative  summary  of  the  original  message. 

While  the  Irwin  textual  views  can  give  the  user  an  up-to-date  peek  at  the  contents  of  the  monitored  resource,  it  is  not 
meant  to  replace  the  programs  that  are  designed  for  reading,  browsing,  and  searching  the  resources.  To  assist  the 
user  in  bridging  the  gap  between  awareness  and  other  tasks,  Irwin  allows  the  user  to  jump  to  various  readers  and 
browsers  to  view  the  resources  of  interest.  Irwin  is  automatically  configured  to  jump  to  a Web  page  using  Netscape 
Navigator  if  a URL  is  referenced,  or  Irwin  can  be  configured  by  the  user  to  start  or  activate  various  email  readers, 
news  readers,  Web  browsers,  or  other  programs  and  commands. 


User  reaction 


While  studies  like  our  navigation  bar  experiment  are  helpful  in  analyzing  performance  for  certain  specific  searching 
and  identification  tasks,  the  true  test  of  tools,  particularly  for  a difficult  to  quantify  task  like  awareness,  is  the 
reaction  of  the  public.  By  offering  Irwin  to  users  and  observing  how  they  use  it,  we  hope  to  answer  some  of  these 
questions. 

Nine  faculty  members  and  graduate  students  attended  a presentation  and  demonstration  explaining  how  to  set  up  and 
use  Irwin.  Seven  people  tried  Irwin  at  least  once,  and  four  of  those  continued  to  use  it  regularly.  The  remainder  of 
this  section  summarizes  their  comments.  (A  full  report  of  their  comments  is  available  as  well  (McCrickard  1997).) 

We  asked  those  who  did  not  use  Irwin  why  they  chose  not  to.  Most  simply  did  not  feel  like  they  needed  such  a tool. 
One  person  noted  that  she  didn't  receive  much  email  and  only  rarely  browsed  the  Web  - she  simply  had  no  use  for 
tools  like  Irwin.  One  user  wanted  a less  passive  visualization  tool.  He  reported  the  he  felt  he  did  not  have  enough 
control  over  when  the  resources  were  checked.  He  seemed  more  interested  in  a typical  browsing  or  reading  tool. 

All  of  the  frequent  users  configured  Irwin  to  better  meet  their  needs,  though  most  used  it  for  several  days  first. 
Common  resources  to  monitor  were  email,  various  news  and  sports  Web  sites,  high  traffic  Usenet  news  groups,  and 
the  local  weather. 

Reaction  was  mixed  on  the  use  of  audio.  All  but  one  user  turned  the  audio  alerts  off  or  only  used  a simple  beep,  and 
the  one  user  who  did  use  it  only  used  short  sounds  (under  a second)  even  though  Irwin  provides  sounds  up  to  two 
seconds  long.  Reactions  varied  from  "I  didn't  need  it"  to  "It  would  drive  me  up  the  wall."  While  we  feel  that  audio  is 
not  for  everyone,  if  it  is  kept  short  and  distinctive  it  can  prove  to  serve  its  purpose  for  some  people. 

Most  users  did  not  use  the  navigation  bar  very  extensively.  They  typically  looked  at  Irwin  only  to  see  the  most 
recent  additions,  i.e.  those  visible  in  the  text  boxes.  One  user  noted  that  he  could  recognize  a few  patterns  and 
occasionally  would  notice  blocks  of  identical  encodings  together  (indicating  several  emails  or  news  postings  by  the 
same  person)  but  would  rarely  click  on  them  and  browse  them.  The  problem  may  be  that  the  navigation  bar  is  better 
suited  for  browsing  and  searching  tasks  than  for  awareness,  or  it  may  be  that  the  users  did  not  choose  resources  for 
which  the  navigation  bar  is  best  suited. 

After  using  Irwin  for  several  months,  the  users  had  many  suggestions  for  resources  that  they  wanted  to  monitor  but 
could  not  with  Irwin.  The  resources  include  Unix  commands  like  Ipq  and  finger  and  Web  images  like  traffic  and 
weather  maps.  The  users  wanted  more  flexibility  over  the  way  in  which  information  is  presented  - not  all  resources 
fit  in  the  list-based  concept  of  the  navigation  bar.  As  we  extend  and  improve  Irwin,  we  are  exploring  new 
identification  schemes  and  new  communication  mechanisms  to  allow  the  user  to  specify  items  of  interest  and  to 
have  the  information  delivered  in  the  most  appropriate  manner. 
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Related  Work 


A significant  amount  of  research  has  focused  on  the  filtering  and  dissemination  of  information  (Foltz  & Dumais 
1992,  Frakes  & Baeza-Yates  1992).  However,  far  less  work  has  been  done  in  understanding  how  people  need  the 
information  to  be  communicated.  Many  systems  generate  email  or  Web  pages  that  still  require  the  user  to  remember 
to  check  them.  Other  solutions  have  been  developed  for  specific  resources,  but  there  is  not  always  a direct 
application  to  a broader  range  of  resources.  We  hope  that  the  Irwin  monitoring  system  will  address  these  concerns 
by  providing  a configurable  set  of  tools  for  monitoring  interesting  resources  and  staying  aware  of  changes  that  occur 
to  them. 

The  individual  components  of  Irwin  were  influenced  by  prior  research.  Auditory  cues  have  proven  useful  in  both 
enhancing  and  replacing  visual  cues  in  user  interfaces  (Brown,  Newsome,  & Glinert  1989,  Brewster  1994).  Studies 
have  shown  that  changing  the  appearance  of  icons  can  convey  additional  information  about  their  use;  for  example, 
animated  icons  (Baecker,  Small,  & Mander  1991)  convey  more  information  about  the  functionality  of  tools  in  a tool 
palette  than  a static  image.  The  navigation  bar  concept  of  encoding  lines  of  data  using  graphical  lines  came  from 
previous  systems.  Read  wear  and  edit  wear  (Hill  & Hollan  1994)  represents  lines  of  text  with  lines  of  pixels,  where 
characteristics  like  length  and  color  correspond  to  properties  of  the  text  such  as  the  number  of  modifications.  The 
SeeSoft  (Eick,  Steffen,  & Sumner  1992)  and  RunView  (McCrickard  & Abowd  1996)  software  visualization 
programs  represent  code  with  lines  of  pixels  where  the  length  of  the  line  can  correspond  to  the  length  of  the  line  or 
function,  and  the  color  can  correspond  to  the  author,  module  name,  or  last  modification  date.  To  our  knowledge,  our 
navigation  bar  was  the  first  to  use  syntactic  encodings,  the  first  to  fully  leverage  the  scrollbar  metaphor,  and  the  first 
to  empirically  demonstrate  that  this  type  of  encoding  improves  user  performance  in  search  and  recognition  tasks. 

While  many  parts  of  the  Irwin  system  have  been  researched  individually,  we  feel  that  our  true  contribution  comes  in 
the  integration  and  evaluation.  We  hope  that  the  lessons  we  have  learned  have  built  on  the  work  of  previous 
researchers  to  provide  another  step  towards  understanding  information  awareness. 


Conclusions  and  Future  Work 


Based  on  our  experience  with  Irwin,  we  would  like  to  discuss  a few  of  the  lessons  we  learned  which  should  help  in 
the  future  development  of  information  awareness  tools. 

Irwin  users  were  willing  to  sacrifice  a little  bit  of  space  to  increase  their  awareness  of  information  resources.  Even 
those  who  did  not  use  Irwin  cited  reasons  other  than  space  limitations.  While  we  do  not  expect  that  everyone  will 
need  awareness  tools,  it  appears  that  there  is  a significant  population  that  does. 

When  using  Irwin,  most  users  utilized  some  combination  of  feedback  from  auditory  cues  and  icon  changes  to  know 
when  to  check  a resource.  Though  we  provided  auditory  cues  that  were  up  to  two  seconds  in  duration,  all  of  the 
sounds  configured  by  our  users  were  under  one  second,  and  most  used  only  a simple  beep.  Longer  sounds  seem  to 
be  too  distracting,  and  a simple  visual  change  may  suffice  to  maintain  awareness.  The  navigation  bar  seems  to  be 
better  suited  for  searching  and  browsing  tasks  than  for  general  awareness,  though  users  began  to  recognize  a few 
patterns  over  time,  and  for  certain  situations  (such  as  monitoring  a course  newsgroup  or  a listing  of  jobs)  the 
navigation  bar  may  yet  prove  to  be  essential. 

Too  often  visualizations  will  provide  complex  graphical  views  of  a data  space  but  will  make  it  difficult  to  see  the 
actual  data  that  is  represented.  Ideally,  a visualization  will  provide  a view  that  combines  graphics  and  text  dependent 
on  the  available  space,  the  needs  of  the  user,  and  the  amount  of  information  that  needs  to  be  shown.  The  confluent 
zoom  allows  the  user  to  stay  aware  of  the  status  of  several  resources  but  also  to  view  details  of  a selected  resource. 
By  providing  both  focus  and  context  in  a compact  manner,  users  can  be  free  to  focus  more  of  their  attention  on  other 
tasks. 
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Abstract:  This  paper  reports  on  research  engaged  in  the  design,  development  and 
evaluation  of  an  innovative,  interactive  learning  tool — for  Web  and  CD-ROM.  The 
learning  tool  is  the  text  analysis  object  (TAO).  TAO  facilitates  the  development  of 
text-based  extended  question-and-answer  problems.  Using  TAO,  a student  is  able 
to  generate  a more  meaningful  answer  by  articulating  their  understanding  rather 
than  merely  recognising  the  lecturer’s  representation,  as  in  multiple  choice 
questions.  TAO  facilitates  an  iterative  approach  to  knowledge  construction. 
Feedback  to  the  student  is  determined  by  using  a key  word  and  key  phrase  search, 
utilising  a two-tiered,  hierarchical  feedback  mechanism — distinguishing  between 
concepts  (major  ideas  or  principles)  and  details. 

A formative  evaluation  of  the  TAO  is  reported.  The  examples  of  TAO  relate  to 
questions  developed  for  use  in  undergraduate  medicine.  However,  the  TAO  is 
applicable  to  many  other  academic  disciplines  where  there  is  a need  to  develop 
more  complex  understanding  of  specific  content  domains. 


Introduction 

The  use  of  computer-facilitated  learning  (CFL)  for  Web  and  CD-ROM  in  higher  education  has  increased 
considerably  in  the  past  five  years.  Considerable  effort  (academic,  financial  and  time)  has  been  invested  in  the 
development  of  courseware.  However,  insufficient  effort  has  been  made  to  develop  pedagogically  sound 
interactive  learning  tools  that  have  the  potential  to  engage  students  more  actively  in  learning.  If  there  are  to  be 
any  lasting  improvements  in  student  learning  outcomes  and  the  success  of  computer-facilitated  learning  (CFL) 
there  is  a need  to: 

• embed  sound  educational  pedagogy  into  courseware;  and 

• provide  interactive  computer-based  learning  tools,  which  are  simple  for  content  experts  to  implement  in  a 
variety  of  learning  environments  supported  by  CFL. 

The  development  of  the  Text  Analysis  Object  (TAO)  is  part  of  a wider  program  being  undertaken  at  The 
University  of  Melbourne  to  develop  simple  generic  Web-based  and  CD-based  learning  and  evaluation  tools 
(Kennedy  & Fritze,  1998;  Kennedy  & Ip,  1998).  Initially,  interviews  with  academic  staff  from  a variety  of 
disciplines  were  undertaken.  Responses  indicated  that  an  interactive  learning  tool  that  could  elicit  student 
responses  of  more  complex  knowledge  (rather  than  merely  require  recognition  as  in  multiple-choice  questions 
(MCQ))  might  have  the  potential  to  enhance  student  learning  by  providing  more  meaningful  feedback  for 
students.  The  TAO  is  intended  to  facilitate  an  iterative  approach  to  learning,  provide  meaningful  feedback,  and 
require  articulation  rather  than  recognition  of  knowledge.  The  functional  aspects  of  the  TAO  include: 
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• a text  box  into  which  students  type  short,  text-based  answers  based  upon  a question  item; 

• a software  engine  (one  for  the  Web  environment  and  a second  stand-alone  version  for  CD-ROM)  to  search 
for  key  words  or  multiple  versions  of  a phrase  or  phrases  defined  by  the  lecturer  or  content  expert; 

• the  ability  to  provide  appropriate  and  meaningful  feedback  to  students;  and 

• access  to  expert  answers  and  a representative  student  exemplar  to  questions  for  comparison  and  self- 
evaluation  purposes. 

Three  perspectives  guided  the  development  of  TAO,  These  perspectives  involved: 

• a constructivist  view  of  teaching  and  learning; 

• the  needs  and  computer  literacy  skills  of  academic  staff  engaged  in  developing  courseware;  and 

• the  requirement  for  a student  to  link  anatomical  knowledge  to  a clinical  perspective. 

The  pedagogical  perspective  adopted  derived  from  a theoretical  framework  that  links  what  is  known  about 
sound  educational  practice  with  the  design  of  computer-based  learning  tools  (design  elements)  for 
incorporation  into  computer-facilitated  learning  environments  or  interactive  multimedia  (Kennedy  & 
McNaught,  1997).  The  analysis  of  text  from  student-generated  answers  was  designed  to  engage  students  more 
actively  in  the  learning  process — a move  away  from  the  click-and-browse  interface  of  many  current  CFL 
offerings.  Initially,  the  feedback  mechanism  for  the  TAO  was  established  by  interviews  with  academic  staff 
The  final  mechanism  was  determined  after  meetings  with  student  focus  groups  engaged  applying  anatomical 
knowledge  in  a clinical  context.  The  students  were  in  their  first  year  of  clinical  practice  (Year  4 of  Medicine). 
A number  of  potential  scenarios  were  constructed  and  a variety  of  feedback  mechanisms  evaluated.  Typical 
comments  from  the  students  about  the  problems  they  encountered  in  their  daily  clinical  practice  include: 

...the  difficulty  is  knowing  what  is  important  and  what  is  not  important  (anatomical  knowledge)  in  making 
a diagnosis. 

I have  forgotten  much  of  what  I learnt  in  first  and  second  year  anatomy.  ...  I find  myself  going  home  each 
night  and  relearning  the  important  concepts  (of  anatomy). 

The  TAO  facilitates  the  incorporation  of  feedback  that  links  anatomical  knowledge  to  clinical  practice,  and 
provides  a numerical  score  (specifically  requested  by  the  students  in  order  to  establish  a link  to  formal 
assessment).  The  feedback  mechanism  is  intended  to  address  the  widely  reported  failure  of  students  to  link  pre- 
clinical  knowledge  (e.g.,  anatomical  knowledge)  with  the  demands  of  being  a successful  clinician  (Balia,  1989; 
Eizenberg,  1988;  Eizenberg,  1991).  The  feedback  mechanism  is  a key  feature  of  the  TAO,  using  a hierarchical 
structure  to  separate  key  concepts  from  minor  details.  Initially,  the  TAO  software  determines  the  number  of 
key  words  or  words  or  multiple  versions  of  a phrase  or  phrases  (as  defined  by  the  content  expert)  found  in  the 
text  string  typed  in  by  the  student.  The  concepts  and  details  of  the  correct  responses  are  then  differentially 
highlighted  and  returned  to  the  student.  The  student  knows  what  parts  of  her  answer  are  correct,  and  the 
relative  importance  of  each  correct  response.  The  two  tiers  required  the  development  of: 

• TAO  software  to  differentiate  between  important  anatomical  concepts  and  what  were  merely  anatomical 
details  (ideas  not  fundamental  in  clinical  practice),  and 

• a model  for  writing  questions  which  focused  on  anatomical  concepts  important  in  clinical  diagnosis. 

These  led  to  the  development  of  a schema  for  writing  the  questions,  expert  and  student  representative  answers, 
key  words  and  key  phrases.  The  schema  also  guided  the  development  of  the  authoring  environment.  Figure  1 
(Fig.  1 ) is  a screen  capture  image  of  student  feedback  during  the  evaluation  process.  Not  shown,  is  the  video 
clip  (a  simulated  patient  complaining  of  pain  in  the  right  side  of  the  abdomen)  that  accompanies  the  text 
component  of  the  question.  With  TAO,  video,  graphics  and  sound  may  be  used  with  any  question. 

TAO  is  intrinsically  a very  simple  system  with  no  built-in  intelligence — a strength  and  a weakness.  This  is  in 
contrast  to  other  text  analysis  systems,  for  example.  Latent  Semantic  Analysis  (Landauer,  Foltz,  & Laham, 
1998b).  The  Intelligent  Essay  Assessor  (lEA)  is  one  such  tool  that  uses  LSA — relying  on  the  ability  of  the 
software  to  measure  the  coherence  of  text  by  comparing  the  degree  of  semantic  relatedness  of  adjacent  text 
segments — from  parts  of  sentences  to  paragraphs  (Landauer,  Foltz,  & Laham,  1998a). 
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Figure  1 : The  Web  version  of  TAO:  Hierarchical  feedback  linking  clinical  practice  and  anatomical  detail 

The  accuracy  of  the  IE  A in  the  grading  of  student  essays  is  comparable  to  that  of  human  markers.  However, 
the  training  of  the  lEA  requires  a very  intensive  diagnostic  process  undertaken  by  experts,  and  access  to  a large 
number  of  exemplars  of  relevant  subject  material  (e.g.,  up  to  30,000  articles).  The  lEA  makes  no  sense  of  word 
order,  syntactic  relations,  or  logic  (Landauer  et  ai,  1998b,  p.  261).  In  clinical  diagnosis,  word  order  and  logic  | 
are  critical.  The  TAO  is  designed  to  provide  formative  and  (limited)  summative  feedback  to  students.  It  is  not 
intended  for  use  as  an  assessment  tool.  Student  learning  is  enhanced  by  engaging  students  in  a reflective 
process.  In  answering  a question  a student  must  address  the  following:  How  many  of  the  major  concepts  have  I 
got  correct?;  Are  the  concepts  I articulated  actually  the  ones  required  for  a correct  clinical  diagnosis?;  How 
‘good’  is  my  answer  compared  to  the  two  representative  answers  provided?,  and.  How  does  my  answer  relate  to 
the  clinical  perspective  presented  (e.g.,  in  the  accompanying  video)?  TAO’s  strengths  are: 

• its  relative  simplicity  of  use  by  content  experts, 

• the  ability  to  focus  on  important  concepts  specified  by  the  educational  needs  of  a particular  content  domain, 

• the  hierarchical  nature  of  the  feedback  provided  (concepts  and  details),  and 

• the  provision  for  an  iterative  approach  to  learning  as  the  student  moves  towards  a final  answer. 

A content  expert  using  TAO  also  has  the  ability  to  incorporate  other  media,  video  clips,  graphics  and  sounds 
into  a question.  Video  clips  may  be  used  to  situate  the  learning  in  a clinical  context.  The  major  weaknesses  of 
developing  a system  using  TAO  are  the  need 

• to  design  the  questions  carefully,  and 

• a pre-emptive  approach  (Bain  & McNaught,  1996)  to  appropriately  express  key  words  and  phrases. 

A Functional  View  of  TAO 

The  basic  functionality  of  TAO  requires  a: 

• text  field  into  which  a student  can  type  an  answer  to  a question, 

• software  engine  to  analyse  the  student  text  and  provide  meaningful  feedback, 

• method  of  displaying  an  expert  answer,  and 

• simple-to-use  authoring  environment. 

Initially,  the  TAO  was  created  as  a server-side  object — that  is,  the  software  engine  for  the  tool  was  located  on  a 
Web-server.  This  decision  derived  from  the  need  to  have  a short  development  time,  core  functionalities  that 
could  be  developed  quickly  and  tested,  and  a tool  that  could  be  easily  maintained.  Design  of  the  Web  authoring 
environment  was  inherited  from  the  OXYGEN  software  engine  (Ip,  1998).  OXYGEN  also  supports  Learning 
Evaluation  Online  (LEO),  an  online  evaluation  tool  used  in  the  formative  evaluation  of  TAO  (Kennedy  & Ip, 
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1998).  The  second  version  for  CD-ROM  has  been  prototyped  as  a Macromedia  Director®  object  for  the  project, 
Demystify — which  focuses  on  clinical  practice  in  undergraduate  medicine. 

The  structure  of  the  authoring  environment  was  critical  because  the  expected  users  of  TAO  are  lecturers — who 
often  have  only  limited  computing  experience.  Our  prior  experience  indicated  that  a forms-based  approach  was 
appropriate.  The  authoring  environment  consists  of  input  fields  for  the: 

• reference  code  for  the  question, 

• question  stem, 

• key  words  and  multiple  versions  of  a phrase  or  phrases  based  on  an  important  concept  or  concepts, 

• key  words  and  multiple  versions  of  a phrase  or  phrases  based  on  an  detail  or  details, 

• mark  to  be  awarded  for  each  concept  (major  idea)  and  detail  (minor  idea),  and 

• expert  answer  and  an  exemplar  student  answer. 

Experience  has  shown  that  content  experts  with  minimal  computing  skills  can  format  questions  easily  for  use 
with  the  TAO.  The  structure  of  each  question  required  the: 

• separation  of  important  (clinical)  concepts  from  (anatomical)  details, 

• development  of  a number  of  alternatives  for  key  words  and  phrases,  separated  by  the  pipe  ‘|’  symbol,  and 

• organisation  of  the  expert  answer  into  point  form  which  clearly  showed  the  concepts  and  details  (a  stated 
requirement  from  the  students  in  the  focus  groups). 

A question  used  in  the  first  formative  evaluation  is  shown  in  Table  1.  The  CD-ROM  version  of  TAO  will  be 
used  in  the  Demystify  project  and  the  Web  version  in  a variety  of  courseware  in  the  Faculties  of  Medicine  at 
The  University  of  Melbourne  and  Monash  University. 

Formative  Evaluation 

Formative  evaluation  of  the  Web  version  has  been  completed.  The  student  responses  to  the  use  of  TAO  were 
generated  by  the  ‘think-aloud’  approach,  researcher  notes,  and  an  online  survey  (Likert-style  and  free  text 
response)  using  Learning  Evaluation  Online  (LEO).  One  of  the  surprise  results  of  the  evaluation  process  was 
the  considerable  discussion,  debate  and  collaborative  effort  engaged  in  by  students  in  order  to  arrive  at  an 
answer,  or  succession  of  answers.  The  students  found  using  the  TAO  straightforward.  In  each  session  students 
were  asked  to  complete  approximately  half  the  questions  (4  to  6).  This  led  to  the  unusual  phenomenon  of  the 
majority  of  students  requesting  to  stay  to  complete  the  remaining  questions  at  an  adjacent  computer  (which  they 
subsequently  did).  The  formative  evaluation  indicated  a strong  acceptance  by  students  of  the  TAO  as  an 
effective  learning  tool.  Representative  written  comments  from  the  online  survey  asking  about  the  best  features 
of  TAO  are: 

It  is  very  clinically  oriented  which  is  brilliant  since  books  do  not  often  highlight  important  principles. 

...  the  distinctions  between  concepts  and  details  in  the  question  are  critical,  therefore  very  relevant. 

The  fact  that  we  could  type  in  our  own  answers  and  for  the  program  to  analyse  what  we  got  right  instead  of 
us  choosing  from  a number  of  given  answers. 

The  use  of  pairs  of  students  enhanced -the  learning  experience  by  promoting  a collaborative  approach.  Students 
were  strongly  engaged  in  debate  and  argument  about  the  inclusion  or  exclusion  of  anatomical  concepts  and 
details.  Most  pairs  of  students  tended  to  refine  the  first  attempt  a number  of  times  before  having  the  response 
analysed.  The  concern  expressed  by  some  academic  staff  that  a student  would  merely  access  the  expert  answer 
almost  immediately  was  unfounded  for  two  reasons. 

The  first  reason  was  student  motivation — when  asked  why  they  didn’t  look  at  the  expert  answer  immediately, 
students  stated  that  doing  so  was  a ‘waste  of  time’  because  it  wouldn’t  help  them  learn.  The  second  reason  is 
more  pragmatic — TAO  requires  a student  to  click  on  a Submit  button  before  being  provided  access  to  the  Get 
Answer  button.  A similar  lack  of  concern  was  expressed  in  regard  to  the  lack  of  a spell  checking  mechanism — 
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they  believed  they  should  know  how  to  spell  the  terms.  Not  all  comments  were  positive  and  some  of  these 
responses  have  resulted  in  changes  in  functionality  of  the  TAO. 

In  response  to  ‘What  was  the  worst  feature  of  this  multimedia  program?’,  students  stated  the  following. 

No  indication  of  how  much  is  expected  in  answering  a question. 

Having  to  sift  through  (expert)  answers  to  find  what  you  hadn’t  included  (in  your  own  answer). 

Maybe  at  times  being  too  pedantic  about  details. 


Q: 0004 

In  appendicitis,  after  initial  pain  has  been  experienced,  the  pain  typically  shifts  to  the  right  iliac 
fossa.  Explain  the  anatomical  basis  for  this 

Expert 

answer 

Concepts: 

Inflammation  has  spread  from  the  appendix 
to  involve  the  adjacent  parietal 
peritoneum. 

The  parietal  peritoneum  and  the  overlying 
skin  have  a somatic  nerve  supply. 

Somatic  pain  is  usually  well  localised  as  it 
is  typically  referred  to  the  overlying  skin. 

Details: 

The  appendix  is  usually  located  in  the  right  iliac  fossa. 

The  parietal  peritoneum  of  the  right  iliac  fossa  is 
supplied  by  the  right  twelfth  thoracic  spinal  nerve,  or 
T12  or  subcostal  nerve. 

The  skin  of  the  right  iliac  fossa  is  supplied  by  the  T12 
dermatome  or  by  the  right  twelfth,  T12,  subcostal 
nerve 

Keywords 

and/or 

phrases 

Concepts 

parietal  peritoneum 

somatic  nerve  supply!  somatic  nervej 

somatic  pain 

overlying  skin 

referred  painj  referred!  refer 

Details 

twelfth  thoracic  spinal  nervej  subcostal  nerve|  T12| 

twelfth  thoracic 

appendix 

T12  dermatome  1 dermatome 
right 

Table  1 : A template  and  exemplar  question  for  using  TAO 

In  response  to  ‘Any  more  comments  or  suggestions  for  improvement?’,  students  stated  the  following. 

Providing  hints  at  second  attempts  and  . ...  having  a help  file  with  a list  of  anatomical  terms. .. 

Just  enhance  the  text  analysis  flexibility-  sample  a wide  proportion  of  possible  answers. 

In  addition  to  the  extended  comments  (above),  the  student  responses  to  bipolar  Likert-style  questions  were 
analysed.  The  mean  responses  are  shown  in  Table  2 (Tab.  2).  The  responses  were  scored:  1,  strongly  disagree; 
2,  disagree;  3,  no  opinion;  4,  agree;  and  5,  strongly  agree. 


Likert-style  question 

Mean 

The  TAO  helped  emphasise  important  and  relevant  concepts. 

4.7 

The  TAO  helped  identify  key  concepts  and  provided  insights. 

4.6 

The  TAO  helped  provide  feedback  which  was  meaningful  to  me. 

4.3 

As  a result  of  using  this  multimedia: 

I was  stimulated  to  reflect  on  what  is  important  for  me  to  understand. 

4.4 

my  intention  to  achieve  an  understanding  of  this  subject  was  enhanced. 

4.4 

Overall  I found  this  multimedia  program  to  be  worthy  of  extension  to  develop  new  modules  with  similar  uses 
of  the  TAO. 

4.7 

Table  2:  Example  student  responses  to  likert-style  questions:  Web-based  TAO 

In  response  to  these  comments  two  changes  have  been  made  to  the  TAO  software  engines.  The  mark  for  each 
question  is  displayed  with  the  question  stem  to  provide  a guide  for  the  relative  complexity  of  the  answer 
required.  The  second  change  is  more  fundamental.  The  TAO  is  to  be  incorporated  into  courseware — 
providing  access  to  hints  and/  or  online  help. 
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Future  Development 

A new  collaborative  project,  Interactive  Qualitative  Questions:  IQ^,  has  been  commenced  jointly  at  The 
University  of  Melbourne  and  Monash  University  to  develop  the  two  versions  of  TAO  further.  The  TAO  is 
being  incorporated  into  both  Web-based  and  CD-ROM  courseware — Anatomedia  (designed  to  support  the 
learning  of  anatomy),  Demystify  (which  focuses  on  clinical  practice).  The  Virtual  Radiography  Room  (a 
simulation  of  radiography  practice),  and  Gastro-intestinal  Medicine  courseware.  In  response  to  current  student 
feedback  a number  of  changes  will  be  made.  One  change  has  already  been  indicated  (a  mark).  Both  versions 
of  TAO  derive  from  courseware  based  on  components  (Ip  & Canale,  1996;  Ip,  Canale,  Fritze,  & Ji,  1997). 
Other  changes  will  be: 

• improvements  to  the  authoring  environment  by  greater  integration  of  video  and  sound  to  provide  multiple 
perspectives  for  the  questions,  and 

• generation  of  each  question  in  a separate  window. 

The  formative  and  summative  evaluation  of  student  learning  outcomes  in  both  institutions  for  both  the  Web  and 
CD-ROM  versions  of  TAO  in  the  four  projects  will  continue  in  1999. 
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Abstract:  As  attention  becomes  increasingly  focused  on  distance  education,  in  the  public  eye  and 
within  the  academic  community,  a sense  of  urgency  to  develop  on-line  course  offerings  is  taking 
hold  among  institutions  that  wish  to  be  responsive  to  student  needs  (real  or  perceived)  and  that  do 
not  want  to  be  "left  behind"  as  peer  institutions  tool  up  for  distance  instruction.  With  the  host  of 
software  tools  available,  adapting  courses  to  the  on-line  learning  environment  does  not  pose 
insurmountable  technical  obstacles,  and  can  be  almost  routine.  What  becomes  a challenge  is 
preserving  the  original  value  of  the  instructional  design  and  adapting  the  pedagogical  nuances  of  a 
course  to  best  leverage  the  modalities  of  on-line  instruction.  This  paper  illustrates  how  a graduate 
level  course  at  Teachers  College,  tailored  for  face-to-face  instruction,  was  adapted  for  distant 
learners,  and  how  the  instructional  design  was  captured  and  replicated  by  a web-based  distance 
education  authoring  tool. 


Introduction 

Recently,  Teachers  College,  Columbia  University  expanded  the  boundaries  of  its  campus  by  offering  on- 
line distance  learning  courses  for  the  first  time.  Three  graduate  courses,  in  three  different  areas,  were  selected  for 
this  pilot  initiative:  Computer-Mediated  Communication,  The  Teaching  of  Writing,  and  Instructional  Design  of 
Educational  Technology.  The  courses,  previously  designed  for  the  classroom,  were  adapted  to  on-line  settings,  and 
in  the  spirit  of  experimentation,  each  course  adopted  a different  development  path.  We  will  discuss  the  design  of  the 
course  in  Instructional  Design  of  Educational  Technology,  which  poses  several  challenges  to  the  distance  format, 
among  them,  its  emphasis  on  discussion,  the  importance  of  technology  demonstrations,  and  the  centrality  of  group 
project  work.  We  also  summarize  the  evolution  of  the  CourseMaster  Authoring  Environment,  a database-driven  tool 
that  is  derived  centrally  from  a model  of  distance  instruction  that  guided  the  construction  of  the  original  course 
application. 


Goals 


The  goals  of  this  project  were  to  build  a reproducible  distance  learning  model,  within  which  rich  classroom 
interactions  among  participants  can  be  replicated,  and  to  shed  light  on  potential  obstacles  to  the  success  of  this 
approach.  We  were  guided  in  part  by  our  previous  distance  learning  research  that  calls  for  a resource  space, 
discussion  space  and  collaboration  space  as  components  of  successful  computer-supported  distributed  learning  (Bell 
& Meyer,  1997).  In  the  pilot  course,  we  focussed  on  specific  design  techniques  for  fostering  group  interaction.  We 
developed  several  facets  of  the  course  with  community-building  in  mind;  a student  database  where  participants 
would  share  personal  profiles;  a suite  of  technology  demonstrations  and  corresponding  assignments  aimed  at  getting 
students  to  share  reactions  to  those  demonstrations;  a terminal  project  that  engaged  teams  of  students  in  remote 
collaboration;  and  regular  discussion  and  chat  sessions  to  foster  and  sustain  whatever  communal  momentum  had 
been  achieved.  The  creation  and  implementation  of  Instructional  Design  Online  (as  the  distance  version  came  to  be 
called)  presented  an  opportunity  to  invent  a web-based  learning  environment,  adapt  an  onsite  class  to  an  on-line 
course  and  capture  its  design  in  a mold,  thus  providing  a test-bed  for  studying  learning  at  a distance. 
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Pilot  Course  Design:  Instructional  Design  Online 

Instructional  Design  of  Educational  Technology  is  a graduate  class  that  surveys  contemporary  frameworks 
for  intelligent  learning  environments  and  that  engages  students  in  designing  and  executing  collaborative  projects 
(e.g.,  an  interactive  web-based  environment,  stand-alone  instructional  multimedia,  etc.).  The  class  thus  emphasizes 
learning  by  doing.  In  order  to  support  this  approach  through  computer-mediated  instruction,  the  course  site  includes 
five  sections,  each  of  which  can  be  reached  by  using  a drop-down  menu:  Information  Desk,  Course  Central, 
Instructor's  Corner,  Student  Lounge,  and  Help  Center,  Each  section  has  its  own  sub-menu  for  the  sites  located 
within  that  section. 

The  Information  Desk  includes  an  overview  of  the  course,  an  orientation  session,  a course  outline,  and 
registration  information.  The  overview  offers  a summary  of  the  class  and  a list  of  required  readings.  The 
orientation  is  intended  to  welcome  students,  guide  them  into  practicing  tasks  common  to  an  on-line  class  and 
prepare  them  with  the  tools  necessary  for  exploring  the  learning  environment.  Initial  assignments  are  designed  to 
engage  students  in  Internet  communication.  Orientation  assignments  include  viewing  a video  welcome  from  the 
instructors  (Fig.  1)  and  creating  a student  profile. 

Course  Central  consists  of  a syllabus,  on-line  demonstrations  of  educational  technology,  course  lecture 
slides,  and  course  assignments.  The  syllabus  is  divided  into  topic-specific  modules,  each  including  links  to  the 
corresponding  lecture  slides,  assignments  for  the  week  and  related  technology  demonstrations.  The  readings  for  that 
module  are  available  via  hypertext  links  to  external  web  sites. 

Web-based  educational  technology  demonstrations  represent  an  important  element  of  this  course.  The 
principles  and  frameworks  discussed  in  the  readings  can  remain  largely  static  or  abstract  in  the  absence  of 
applications  of  those  ideas  in  the  form  of  interactive  illustrations.  To  preserve  this  central  aspect  of  the  course,  we 
adapted  several  works-in-progress  at  Teachers  College  for  on-line  viewing,  including  Wx-Brief  (Bell,  Vaughn  & 
Reibel  1997),  Aviation  Story  Archive  (Bell,  Gold  & Kaplan,  1998),  and  the  Virtual  Hall  of  Fame  (Zirkel,  Bell  & 
Gold,  1998).  Each  Demonstration  is  accessible  on-line  (Fig.  2)  and  supplemented  by  an  assignment  in  which 
students  share  observations  and  critiques  of  the  demonstration. 

Lecture  Slides  are  available  in  modules,  each  consisting  of  slides  that  can  be  navigated  in  succession  or 
through  direct  links.  Assignments  are  designed  to  encourage  students  to  explore  technology  tools  and  Instructional 
Design  theories,  and  to  integrate  them  in  discussion  responses  and  project  creations.  Assignments  range  from 
regular  responses  to  issues  raised  in  the  reading  to  creating  interactive  exemplars  of  concepts  treated  during  the 
course. 

The  Student  Lounge  includes  a database  of  student  profiles,  a meeting  room  and  a student  projects  area. 
The  Student  Profiles  section  facilitates  the  distribution  and  sharing  of  individual  profile  and  contact  information.  A 
participant  database  (Fig.  3)  receives  profile  forms  submitted  by  the  student,  and  permits  the  browsing  and  searching 
of  participant  profiles,  pictures  and  contact  data.  Profiles  include  direct  links  to  student  mailboxes  and  world  wide 
web  addresses,  personal  interests,  and  academic  and  professional  histories.  The  instructors’  profiles  and  office 
hours  are  in  the  Instructors’  Comer.  The  profiles  include  Instmctor  academic  background,  links  to  personal  web 
sites,  and  office  hours  (which  are  conducted  face-to-face,  by  telephone,  via  text  chat,  or  using  audio  conferencing). 

The  Meeting  Room,  supporting  synchronous  (chat)  and  asynchronous  (bulletin  board)  interaction,  is  the 
primary  communication  area.  The  chat  room  permits  live  scheduled  group  discussions,  including  multiple 
simultaneous  one-to-one  private  chats,  about  topics  related  to  course  content,  assignments  and  group  projects. 
Class-wide  chats  are  transcribed  and  published  on  the  course  site.  The  discussion  room  supports  group 
communication  through  individual  message  postings. 

Early  in  the  course,  students  are  asked  to  form  groups  and  to  begin  proposing  final  projects.  As  the 
semester  progresses,  students  are  asked  to  prepare  incremental  reports,  such  as  a storyboard,  documenting  their 
progress  in  the  final  project.  The  Student  Projects  area  centralizes  information  regarding  final  projects,  the 
culminating  product  of  the  students'  experience  in  the  course.  Suggestions  (in  the  form  of  online  technology 
demonstrations)  are  offered  as  potential  group  projects,  though  students  are  invited  to  propose  their  own  ideas  as 
well.  A project  database  collects  and  publishes  information  about  project  group  members,  member  email  addresses, 
and  project  URLs,  so  that  group  members  and  projects  are  easily  accessible. 


Formative  Evaluation 


Participants 


The  course  was  offered  during  the  spring,  summer  and  fall  semesters  of  1998  and  is  currently  in  session  for 
the  spring,  1999  term.  Eight  students  enrolled  for  the  spring  semester,  7 students  for  the  summer,  and  17  for  the  fall. 
In  each  semester's  class,  more  than  half  of  the  students  registered  for  credit.  All  of  the  students  in  each  semester  had 
completed  or  were  pursuing  either  a Masters  or  Doctoral  degree.  Most  of  the  students  registered  for  credit  were 
part-time  students  who  work  full-time  as  either  classroom  teachers  or  technology  professionals.  Several  of  the  non- 
credit students  were  faculty  at  other  institutions  who  were  interested  in  exploring  examples  of  distance  learning 
courses.  Students  were  physically  situated  in  locations  throughout  the  U,S,  (with  some  international  participation 
from  Japan  and  Brazil), 


Methods 

Pre-  and  post-course  surveys  and  scheduled  chat  interviews  were  used  to  gather  qualitative  information 
about  on-line  students  and  on-line  student  experiences  in  the  pilot  course.  Chat  sessions,  discussion  postings  and 
group  projects  served  as  additional  evidence  documenting  student  performance.  The  objective  of  this  formative 
evaluation  was  to  learn  more  about  how  students  made  use  of  the  tools  we  supplied  with  the  course,  the  extent  to 
which  those  tools  furthered  the  students'  capacity  to  engage  in  the  class  activities,  and  to  explore  potential  research 
questions  for  further  studies.  The  study  did  not  attempt  to  measure  learning  outcomes  beyond  a qualitative 
assessment  of  their  productivity  as  required  for  grading  purposes.  Our  evaluation  was  guided  in  part  by  the 
expectation  that  understanding  the  learning  needs  of  students  is  crucial  to  successful  on-line  course  experiences 
(Reid,  1996;  Warren,  1996;  Willis,  1995),  Pre-course  surveys  were  aimed  at  gauging  student  backgrounds  and  skills 
and  their  reasons  for  enrolling  in  the  course.  Post-course  surveys  were  aimed  at  gathering  student  perceptions  about 
their  experience  in  the  course  and  the  design  of  the  learning  environment. 


Results 


Computing  ability  and  exposure  to  the  Internet  varied  widely  among  incoming  students.  In  each  class  there 
were  individuals  who  reported  that  they  did  not  feel  confident  in  their  computing  skills,  as  well  as  others  who 
reported  that  they  were  comfortable  with  a variety  of  computer  and  Internet  applications.  Variation  in  computer 
literacy  also  became  evident  as  the  semesters  progressed.  Many  of  the  individuals  who  had  expressed  a lack  in 
computing  skills  moved  through  the  syllabus  at  a much  slower  rate  and  seemed  to  be  less  involved  in  group 
communication.  There  were  also  individuals  who  came  in  with  weak  computing  skills  and  gained  proficiency  at  a 
rapid  rate.  During  one  semester,  a vision-impaired  student  participating  in  the  course  provided  valuable  feedback  on 
the  extent  to  which  our  course  site  accommodates  visually-impaired  students  in  accessing  information  and 
navigating  the  site. 

The  Discussion  room  was  the  most  active  component  of  the  courses  in  each  semester,  primarily  used  for 
discussion  about  course  assignments  and  scheduling.  Assignment  postings  took  on  a quality  that  seemed  to  be  a 
cross  between  written  language  and  spoken  conversation,  with  ideas  expressed  in  a blending  of  informal  and  formal 
language  styles,  A large  proportion  of  the  postings  in  the  discussion  threads  were  responses  to  assignments  or 
questions  from  the  Instructor  or  Teaching  Assistant,  Students  did  not  begin  to  initiate  spontaneous  postings  until 
later  in  the  semester.  The  discussion  room  was  not  used  for  dialog  with  other  students  until  the  Instructor  suggested 
that  students  respond  to  the  postings  of  other  students.  Once  this  suggestion  was  made,  peer-to-peer  interaction 
became  more  frequent  and  substantive. 

Statements  from  students  indicate  that  communication  among  students  also  took  place  via  private  email 
messages  and  by  telephone.  While  much  of  the  communication  from  the  Instructor  and  Teaching  Assistant  occurred 
in  the  public  communication  space,  spontaneous  communication  from  students  to  instructors  usually  took  place  in 
private  email  or  telephone  messages.  Email  messages  and  telephone  calls  from  students  to  the  Instructor  or  TA  were 
primarily  about  administrative  and  technical  issues,  such  as  course  credit,  and  only  occasionally  contained  content 
related  communication.  Instances  of  project  related  mail  and  phone  calls  did  occur  just  before  the  final  projects 
were  due  during  the  spring  and  summer  semesters. 

Students  in  the  spring  and  summer  semesters  did  not  frequently  meet  in  the  Chat  Room  unless  the 
Instructor  had  scheduled  the  chat.  Students  in  the  fall  course  appeared  to  be  making  more  use  of  the  Chat  Room  for 
peer-to-peer  communication.  One  fall  semester  group  met  in  the  Chat  Room  for  several  hours  a week.  The  main 
differences  between  groups  in  the  fall  course  and  groups  in  the  spring  and  summer  appeared  to  be  group  size  (the 
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fall  class  had  larger  groups)  and  individual  computing  ability  (the  fall  class  had  more  individuals  with  strong 
computing  ability).  In  each  term,  the  Chat  Room  was  used  on  just  one  occasion  as  an  entire  class  (time-zone 
differences  caused  scheduling  difficulties).  Not  all  of  the  students  attended  the  scheduled  chats.  For  the  participants 
who  did  attend,  these  sessions  were  productive.  Several  students  mentioned  that  Chat  Room  discussions  were  most 
like  the  communication  that  takes  place  in  the  classroom.  All  of  the  Chat  participants  spontaneously  shared 
thoughts  and  ideas. 

Not  all  of  the  individuals  in  the  spring  and  summer  courses  completed  a final  group  project.  The  group 
projects  that  were  completed  were  of  high  quality.  There  had  been  some  concern  about  whether  students  in  the  on- 
line class  would  have  access  to  the  same  materials  as  students  in  the  classroom,  such  as  commercial,  licensed 
authoring  software,  so  it  was  decided  that  on-line  student  projects  would  be  created  with  publicly  available 
materials.  Despite  the  potential  difference  in  student  resources,  the  on-line  students'  final  project  was  comparable  to 
final  projects  created  by  students  in  the  classroom.  Students  in  both  online  and  in  classroom  courses  produced 
original  technology-based  learning  environments  that  showed  some  integration  of  theories  discussed  in  the  course. 

Student  comments  suggested  that  they  appreciated  the  variety  of  delivery  methods  and  information 
resources  provided,  such  as  lecture  notes,  public  readings  and  discussion  threads.  However,  they  also  expressed 
frustration  in  accessing  these  resources.  Several  students  mentioned  that  they  had  consistently  had  connectivity 
problems  and  problems  during  chat  sessions  and  in  downloading  plug-ins  and  other  course  information.  Advanced 
components  took  too  long  to  download  or  made  their  computer  crash.  These  students  indicated  that  much  of  the 
problem  was  with  their  hardware.  Suggestions  were  made  to  make  graphics,  video  and  other  hard  to  access 
information  available  through  other  means.  Generally,  the  comments  acknowledged  the  importance  of  speedy  and 
reliable  connections  and  up-to-date  hardware.  Also,  based  on  the  comments  made  during  the  final  chat  session  in 
which  students  were  directly  queried  (in  the  absence  of  the  instructor),  students  appeared  overwhelmed  by  the 
technical  requirements  combined  with  the  content  of  the  course.  These  students  did  not  feel  that  their  knowledge  of 
computer  technology  supported  their  learning  needs  and  suggested  a hands-on  prerequisite,  which  would  enable 
students  to  learn  the  technology  apart  from  the  content.  All  students  who  have  completed  the  course  thus  far 
responded  that  they  would  consider  taking  another  distance  learning  course. 


Discussion 

Our  exploratory  findings  provide  some  information  about  the  needs  of  distance  learners,  how  our  design 
functions  in  practice,  and  how  to  focus  our  future  research  efforts.  While  many  distance  courses  are  promoted  to 
provide  flexibility  and  convenience,  they  may  in  fact  be  more  time  consuming  than  classroom  courses,  depending 
upon  the  skill  level  and  resources  of  individual  participants  and  the  quantity  of  participants.  For  students  who  were 
not  comfortable  with  Internet  communication  or  had  inadequate  resources,  downloading  data  and  keeping  up  with 
communication  was  more  time  consuming  than  they  had  expected.  It  is  clear  when  examining  both  student  needs 
and  instructional  design  that  explicit  information  regarding  minimum  and  suggested  hardware  and  software 
requirements  be  made  available  to  students. 

There  appear  to  be  both  similarities  and  differences  in  student  participation  in  classroom  and  distance 
learning  environments.  Similarly  to  on-site  classes,  students  who  were  not  taking  the  course  for  credit  were  not  as 
active  as  students  who  were  enrolled  for  credit.  In  contrast  to  on-site  classes,  students  in  the  on-line  class  did  not 
seem  as  accountable  for  their  work.  On-line  students  did  not  complete  all  of  the  assignments  and  did  not  attend  all 
the  scheduled  meetings.  Overall,  the  on-line  students  in  this  class  required  more  guidance  and  imperatives.  On-line 
students  did  not  seem  as  self-disciplined,  perhaps  because  they  did  not  have  physical  access  to  the  Instructor  or 
Teaching  Assistant.  Despite  the  challenges  involved  in  this  on-line  setting,  students  were  able  to  utilize  the  tools, 
navigate  and  participate  within  the  learning  environment.  Most  students  more  than  sufficiently  completed  the  entire 
course  on-line. 

These  preliminary  studies  have  been  helpful  in  our  efforts  to  develop  more  systematic  means  of 
investigation.  Based  on  student  comments  and  student  participation  in  the  spring  and  summer  courses,  a new  set  of 
database-driven  pre-course  and  post-course  surveys  was  developed  and  distributed  in  the  fall  semester  online  and 
classroom  courses.  Survey  questions  are  designed  to  capture  information  regarding  student  skill  level,  reasons  for 
taking  the  course,  predictions  about  the  learning  outcomes  and  challenges  in  on-line  courses  versus  the  learning 
outcomes  and  challenges  of  students  in  classroom  courses,  and  perceptions  of  student-professor  and  student-student 
relationships.  Questions  in  the  new  surveys  invite  both  open-ended  (as  in  previous  surveys)  answers  and 
precalculated  answers  (based  on  open-ended  answers  from  previous  semesters).  The  overall  goal  of  this  new 
addition  to  the  design  of  the  course  is  to  provide  an  improved  structure  for  examining  on-line  student  learning, 
evaluating  course  design,  discovering  student  perceptions  about  on-line  learning  in  comparison  to  classroom 
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learning  and  gathering  preliminary  evidence  about  whether  or  not  intimacy  levels  among  individuals  within  a 
distance  learning  environment  are  different  than  intimacy  levels  among  students  in  a classroom  environment. 


Bringing  the  Model  to  Life 

Upon  building  what  proved  to  be  an  effective  course  design  given  current  technological  conditions,  we  then 
successfully  crafted  CourseMaster  (Fig.  4)  by  abstracting  the  original  course  and  implementing  that  abstracted 
model  as  a set  of  re lationally-1  inked  database  templates.  The  result  is  a sophisticated  courseware  authoring  tool  that 
not  only  renders  the  course  production  process  less  mechanically  burdensome,  but  also  adheres  to  (and  derives  its 
power  from)  this  specific  design  model.  Because  it  is  created  in  a fully  web-compliant  database  environment, 
CourseMaster  can  be  used  remotely  by  authors,  and  design  commitments  are  immediately  reflected  in  the  materials 
published  on  the  web.  We  can  now  create  a new  on-line  course  modeled  on  this  design,  within  any  subject  area, 
with  relative  ease. 

CourseMaster  presents  us  with  three  research  opportunities.  First,  comparisons  can  be  made  using  the 
same  design  among  different  content  areas.  Second,  aspects  of  group  interaction  across  all  content  areas  and  design 
frameworks  can  be  explored  with  the  use  of  survey  evaluation  databases.  Third,  questions  can  be  asked  about 
instructional  design  in  terms  of  particular  on-line  learning  components,  such  as  chats,  discussion  rooms  and  visual 
aids,  by  building  identical  courses  that  vary  only  in  terms  of  the  use  of  particular  components. 


Conclusion 

One  common  deficiency  among  distance  instruction  materials  we  have  encountered  is  the  absence  of  an 
explicit  instructional  approach  (Bell  & Meyer,  1997).  This  paper  described  a distance  courseware  design  in  which 
we  adopted  a pedagogy  that  emphasizes  peer  interaction  and  group  collaboration.  An  overall  objective  of  this 
research  is  to  develop  frameworks  for  distance  course  design,  appropriate  for  given  subject  domains,  that  preserve 
the  positive  elements  of  face-to-face  instructional  designs  while  introducing  new  strategies  that  take  advantage  of 
the  properties  of  desktop  telecommunication  and  web-based  interchange.  Our  experiences  with  the  design  of 
Instructional  Design  Online  have  suggested  some  positive  aspects  of  on-line  distance  learning  and  have  indicated 
some  directions  for  future  research  and  course  development.  A principal  conclusion  is  that  successful  distance 
learning  must  be  at  least  as  (if  not  more)  firmly  grounded  in  communication  as  traditional  classroom  instruction. 
Our  success  with  the  use  of  student  profiles,  collaborative  project  work,  and  extensive  discussion  lend  supports  to 
the  claim  that  distance  learning  is  effective  to  the  degree  that  it  creates  and  maintains  learning  communities  that 
support  knowledge  construction.  The  emergence  of  collaborative  tools  that  allow  students  to  create  and  refine 
knowledge  artifacts  on-line  is  encouraging  (Harasim,  Calvert  & Groeneboer,  1996).  Web-based  communications 
technologies  will  no  doubt  continue  to  evolve  to  support  more  real  time  communication  and  a more  seamless 
interface  among  communication,  collaboration,  and  information-gathering  resources  (Harasim,  1990).  Flexible 
tools,  such  as  CourseMaster,  that  help  course  designers  to  organize  their  information  and  incorporate  emerging 
communication  and  collaborative  learning  tools  support  an  extension  beyond  the  current  model  of  an  HTML 
template  for  didactic  presentation,  toward  virtual  communities  of  learning. 
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CHANGES  IN  STUDENT  ATTITUDES  IN  A COMPUTER 
ENRICHED  ACADEMIC  ENVIRONMENT:  REPORT  FROM  A 
LONGITUDINAL  CASE  STUDY 


Abstract:  This  paper  reports  that  longitudinal  exposure  to  a computerized 

enriched  learning  environment  might  not  necessarily  change  the  attitudes  toward 
computer  enhanced  learning  unless  the  development  of  the  computerized 
enriched  environment  is  supplemented  by  the  fact  that  every  member  in  the 
environment  is  provided  with  the  tools  for  connecting  with  the  environment.  In 
this  study,  students  without  the  tool  never  felt  the  empowerment  of  access  and 
so  they  continued  to  maintain  either  a negative  attitude,  or  did  not  alter  their 
attitudes  in  a positive  way  even  when  placed  in  a networked  and  computer 
enriched  environment.  On  the  other  hand,  the  students  who  entered  the 
environment  with  a personal  laptop  computer  and  felt  immediately  connected  to 
the  environment. 


INTRODUCTION 

Based  on  the  existing  evidence  on  the  use  of  computers  in  the  higher  education  environment  and 
in  view  of  the  increasing  trend  of  all-out  computer  enrichment  programs  in  academic  institutions  we  pose 
the  following  question:  l.What  are  the  changes  in  attitudes  towards  the  use  of  computers  in  a higher 
education  environment  that  is  computer  enriched?  This  question  attempts  to  address  the  way  in  which 
wide-scale  and  long-term  exposure  to  a computer  enriched  environment  can  alter  the  overall  attitude 
towards  the  application  of  technology  in  higher  education.  As  has  been  argued  elsewhere,  it  is,  however, 
important  to  acknowledge  the  context  within  which  the  technology  is  being  applied  (Ashley,  1992;  Ahola- 
Sidaway,  1990;  Bender,  1995;  Fey,  1993;  Goulet,  1994;  Hettinger,  1995;  Mitra,  1994;  Mitra  and  Hullett, 
1997). 

METHODS  AND  SAMPLE 

The  study  was  designed  as  a longitudinal  survey  spanning  five  years  with  the  first  year  being  the 
baseline  year.  In  the  Fall  of  1995,  Wake  Forest  University  entered  into  a partnership  with  IBM  to  make  the 
institution  computer  enriched.  In  this  case,  the  process  of  enrichment  involved  several  different  initiatives 
including  extensive  networking  of  student  residence  halls,  classrooms,  the  library  and  most  office  buildings 
as  well  as  providing  all  incoming  students  (beginning  the  Fall  of  1 996)  with  an  IBM  ThinkPad  computer. 
This  strategy  ensured  that  by  Fall  of  2000  every  student  at  the  University  would  be  equipped  with  mobile 
computing  that  would  provide  easy  access  to  a local  area  network  and  then  the  Internet.  When  this  decision 
was  ratified,  a survey  design  study  was  also  initiated  to  track  the  changes  in  the  way  the  different  elements 
of  the  institution  (faculty,  staff,  students,  etc.)  react  to  the  computer  enrichment.  This  analysis  reports  from 
three  years  of  the  study  to  answer  the  two  questions  posed  earlier. 

The  survey  design  utilized  a questionnaire  that  was  constructed  after  extensive  use  of  focus  group 
discussions  to  explore  and  uncover  the  various  issues  surrounding  the  computer  enrichment  process  at  the 
University.  Focus  groups  were  conducted  with  students  who  belonged  to  the  "ThinkPad  Program" 
(henceforth  referred  to  as  the  Program)  as  well  as  the  students  who  were  not  direct  beneficiaries  of  the 
Program  but  could  utilize  the  connectivity  and  technology  advancements  in  the  classrooms.  A survey 
instrument  was  developed  and  was  used  in  the  Fall  of  1995  when  only  100  pilot  students  were  part  of  the 
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Program,  in  Spring  1997  when  all  the  First  Year  students  were  part  of  the  Program  and  in  Spring  1998 
when  all  First  and  Second  Year  students  were  part  of  the  Program.  The  three  administrations  thus  covered 
the  1995-1996  school  year  (henceforth  referred  to  as  Year  1),  the  1996-1997  school  year  (Year  2)  and  the 
1997-1998  school  year  (Year  3).  Given  the  way  the  enrichment  process  was  designed,  Year  1 included  only 
100  pilot  students  who  had  a personal  ThinkPad  and  the  University  was  barely  networked.  In  Year  2,  nearly 
90%  of  the  university  was  networked  and  all  the  Freshmen  and  100  Sophomores  had  ThinkPads  and  the 
Juniors  and  Seniors  did  not  have  personal  standardized  mobile  computers  issued  by  the  University  (such 
students  who  were  not  a part  of  the  ThinkPad  program  is  referred  to  as  ’’legacy"  students  later  in  the  paper). 
Finally,  in  Year  3 all  Freshmen,  all  Sophomores  and  100  Juniors  owned  ThinkPads  while  the  remaining 
Juniors  and  all  Seniors  were  considered  legacy  students.  The  overall  composition  of  the  three  groups  are 
summarized  in  Table  1. 


Table  1:  Composition  of  the  samples  for  Years  1,  2 and  3 


GENDER 

Year  1 

Year  2 

Year  3 

Male 

53% 

43% 

43% 

Female 

47% 

57% 

57% 

SCHOOL  YEAR 

Freshmen 

32% 

36% 

54% 

Sophomore 

25% 

24% 

27% 

Junior 

20% 

19% 

18% 

Senior 

22% 

21% 

0%* 

MAJOR 

Sciences 

23% 

25% 

22% 

Math 

2% 

4% 

3% 

Fine  Arts 

3% 

2% 

3% 

Business 

15% 

22% 

15% 

Liberal  Arts 

13% 

16% 

16% 

Social  Sciences 

17% 

19% 

22% 

Undecided 

17% 

12% 

19% 

Other 

11% 

N/A 

N/A 

* The  Seniors  were  not  sampled  in  the  Year  3 survey 

Note:  The  N/A  in  Columns  2 and  3 refers  to  the  fact  that  the  question  was  not  asked  in  Years  2 and  3 


RESULTS 

An  ONEWAY-Anova  measure  was  used  to  test  for  differences  between  groups  and  Scheffe’s  post-hoc 
test  for  homogeneity  was  utilized  to  identify  the  specific  groups  that  were  significantly  different.  The 
results  indicate  that  there  were  more  significant  difference  in  attitude  between  Year  1 and  Year  2 as 
compared  to  Year  2 and  Year  3.  As  shown  in  Table  2,  the  results  show  that  there  are  significant  differences 
in  the  way  the  students  expressed. 

Table  2:  Changes  in  attitude  mean  of  all  students  from  Year  1 to  Year  3 


Item 

Year  1 

Year  2 

Year  3 

Use  of  technology  makes  learning  easier 

3.5  (1391) 

3.4  (469) 

3.5  (271) 

I prefer  to  take  classes  where  I can  get  to  use  a computer 

2.9  (1392) 

2.9  (474) 

3.0  (271) 

I feel  comfortable  using  a computer 

3.6*  (1392) 

3.8*  (469) 

4.0*  (271) 

I have  a certain  apprehension  about  computer  use 

2.8*  (1388) 

2.4*  (469) 

2.2  (270) 

Computers  in  teaching  makes  the  learning  impersonal 

3.2*  (1392) 

2.9*  (472) 

2.7*  (271) 

Computer  use  increases  the  college  work  load 

3.1  (1392) 

3.0  (474) 

3.0(271) 

Computers  are  effective  for  communicating  wit  faculty 

3.3*  (1400) 

3.6*  (472) 

3.8*  (271) 

Communicating  with  professors  by  e-mail  is  generally  gratifying 

3.4*  (1396) 

3.7*  (472) 

3.9*  (272) 

Computers  enable  me  to  interact  more  with  professors 

3.0*  (1393) 

3.4*  (464) 

4.3*  (266) 

o 
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These  differences  include: 

1 . increasing  comfort  with  the  use  of  computers, 

2.  felt  increasingly  less  apprehensive  with  the  use  of  computers, 

3.  were  less  concerned  that  computers  in  teaching  was  making  the  teaching  process  too  impersonal, 

4.  felt  that  computers  enabled  them  to  interact  more  with  teachers, 

5.  felt  that  computers  were  effective  for  communicating  with  teachers  about  non-course  related  material, 

6.  felt  that  communicating  with  professors  with  e-mail  was  generally  gratifying. 

The  question  of  attitude  change  was  also  explored  by  comparing  specific  groups  of  students  and  the 
changes/differences  in  their  attitudes  toward  the  use  of  computers  in  education.  The  first  comparison  was 
conducted  between  the  Year  1 first  year  students,  the  Year  2 sophomore  students  and  the  Year  3 juniors. 
This  group  represents  the  legacy  students  who  entered  the  University  at  the  time  when  the  enrichment 
initiative  began.  They  did  not  feel  the  impact  of  the  Program  but  were  exposed  to  the  changing 
technological  ambience  of  the  University.  They  had  the  opportunity  to  reap  the  benefits  of  a standardized 
computer  network  on  campus.  At  the  same  time,  they  were  exposed  to  an  altering  pedagogic  condition 
where  the  faculty  were  increasingly  "computerizing”  the  curriculum  and  introducing  the  use  of  computer 
mediated  communication  as  a significant  element  of  their  teaching  process.  The  groups  were  compared 
using  an  ONEWAY-Anova  and  the  Scheffe’s  post-hoc  test  of  homogeneity.  The  results  indicate  that  the 
Year  2 Sophomores,  and  Year  3 Juniors  felt  that  the  use  of  computers  did  not  make  the  learning  process 
easier.  The  Year  1 Freshmen  reported  a mean  of  3.6  (n=449)  in  responding  to  the  item,  "increased  use  of 
technology  makes  learning  easier,”  while  the  Year  2 Sophomores  reported  a mean  of  3,2  (n=l  10)  and  the 
Year  3 juniors  reported  a mean  of  3.1  (n=48).  While  the  responses  of  the  Year  2 Sophomores  and  Year  3 
Juniors  were  not  significantly  different,  both  of  those  means  were  significantly  different  from  the  mean 
response  of  the  Year  1 Freshmen  (p<0.05).  At  the  same  time,  the  Year  3 Juniors  expressed  a significantly 
more  positive  feeling  that  computers  enables  them  to  interact  with  their  teachers,  reporting  a mean  of  4,2 
(n=48),  as  compared  to  the  means  of  3.2  (n=107)  for  the  Year  2 Sophomores  and  the  mean  of  3,2  (n=448) 
for  the  Year  1 Freshmen. 

Another  comparison  was  conducted  between  the  Year  2 Freshmen  and  all  the  other  Year  2 students. 
This  test  represented  a comparison  between  the  ThinkPad  students  and  the  legacy  students  in  Year  2,  The 
comparison  was  conducted  with  the  use  of  a t-test  statistic  with  Levine’s  test  of  equality  of  variance. 
Significant  differences  (p<0.05)  were  observed  here  with, 

1 . the  ThinkPad  students  feeling  that  the  introduction  of  computers  makes  learning  easier, 

2.  the  ThinkPad  students  preferring  to  take  classes  where  they  could  use  the  computer, 

3.  the  legacy  students  feeling  that  computers  can  make  the  learning  process  too  impersonal, 

4.  the  ThinkPad  students  feeling  that  computers  are  effective  for  communicating  with  faculty  about  non- 
course related  work, 

5.  the  ThinkPad  students  feeling  that  communicating  with  teachers  by  e-mail  is  generally  gratifying  and 

6.  the  ThinkPad  students  perceiving  that  computers  enable  them  to  interact  more  with  teachers. 

There  were,  thus,  several  instances  where  there  were  significant  differences  between  the  first  group  of 
ThinkPad  students  and  all  the  legacy  students. 

While  these  tests  represent  the  differences  between  legacy  and  ThinkPad  students,  it  was  also 
considered  important  to  compare  similar  groups  of  ThinkPad  students.  The  first  test  included  an 
independent  sample  t-test  with  Levine’s  test  for  equality  of  variance.  The  Year  2 Freshmen,  who  composed 
the  first  batch  of  ThinkPad  students  was  compared  to  the  Year  3 Sophomores  who  represented  the  same 
group  after  a year’s  exposure  to  computerized  instruction  as  well  as  after  one  year’s  use  of  the  ThinkPad. 
There  was  only  one  significant  difference  observed  in  this  comparison  with  the  Year  3 Sophomores 
reporting  a mean  of  4.3  (n=71)  in  response  to  the  item,  "computers  enable  me  to  interact  more  with 
teachers,”  as  compared  to  the  mean  of  3.7  (n=166)  for  the  Year  1 Freshmen.  A similar  comparison  between 
the  Year  2 Freshmen  and  the  Year  3 Freshmen  revealed  that  the  only  significant  difference  between  the 
first  and  second  batch  of  ThinkPad  students  was  that  the  second  batch  felt  that  computers  enabled  them  to 
interact  with  teachers  more  than  the  first  batch  of  ThinkPad  students.  The  second  group  reported  a mean  of 
4.3  (n=139)  as  compared  to  the  Year  1 Freshmen  who  reported  a mean  of  3.7  (n=166).  At  the  same  time, 
the  Year  3 Freshmen  were  not  significantly  different  on  any  attitude  measure  from  the  Year  3 Sophomores. 
There  were  thus  significant  differences  in  attitude  over  the  three  years  as  well  as  across  different  groups  of 
students  over  the  three  years.  The  specific  areas  of  difference  are  summarized  in  Table  3. 
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Table  3:  Summary  of  Findings 
Significant  changes  in  attitude  for  all  students 


Attitude  Items 

Pilot  year  to  First  year  of 
computerization  (P*  batch  of 
TP  students  on  campus) 

First  year  of 

computerization  to  second 
year  of  computerization 
(2"^*  batch  of  TP  students 
on  campus) 

I feel  comfortable  using  a computer 

More  comfortable 

More  comfortable 

I have  a certain  apprehension  about  computer  use 

Less  apprehensive 

No  Change 

Computers  in  teaching  makes  the  learning  impersonal 

Less  concerned 

Less  concerned 

Computers  enable  me  to  interact  more  with  professors 

More  enabling 

More  enabling 

Computers  are  effective  for  communicating  wit  faculty 

More  effective 

More  effective 

Communicating  with  professors  by  e-mail  is  generally 
gratifying 

More  gratifying 

More  gratifying 

Significant  changes  in  attitude  for  legacy  students 


Attitude  Items 

Pilot  year  to  First  year  of 
computerization  batch  of 

TP  students  on  campus) 

First  year  of 

computerization  to  second 
year  of  computerization 
(2"^^  batch  of  TP  students 
on  campus) 

Use  of  technology  makes  learning  easier 

Less  easy 

No  Change 

Computers  enable  me  to  interact  more  with  professors 

No  Change 

More  enabling 

Significant  Changes  in  attitude  of  ThinkPad  students 


Attitude  Items 

First  year  of  computerization 
to  second  year  of 

computerization  (2"^^  batch  of 
TP  students  on  campus) 

Computers  enable  me  to  interact  more  with  professors 

More  enabling 

Significant  differences  between  ThinkPad  and  legacy  students  observed  in  Year  2 


Attitude  Items 

ThinkPad  Students 

Legacy  Students 

Use  of  technology  makes  learning  easier 

More  easy 

Less  easy 

1 prefer  to  take  classes  where  I can  get  to  use  a computer 

More  preferable 

Less  preferable 

Computers  in  teaching  makes  the  learning  impersonal 

Less  concerned 

More  concerned 

Computers  enable  me  to  interact  more  with  professors 

More  enabling 

Less  enabling 

Communicating  with  professors  by  e-mail  is  generally 
gratifying 

More  gratifying 

Less  gratifying 

Significant  differences  between  first  batch  of  ThinkPad  students  and  second  batch  of  ThinkPad  students 

observed  in  Year  3 


Attitude  Items 

First  batch 

Second  batch 

Computers  enable  me  to  interact  more  with  professors 

Less  enabling 

More  enabling 
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LONGITUDINAL  PERSPECTIVES 


Our  results  suggest  that  all  the  three  elements  - attitudes  towards  computers  in  general,  the  role  of 
computers  in  teaching  and  the  way  computers  facilitate  the  communication  process  - tend  to  become  more 
favorable  as  an  institution  makes  itself  computer  enriched.  As  demonstrated  in  the  case  of  the  overall 
student  body  composed  of  the  legacy  and  the  ThinkPad  students,  there  is  indication  to  suggest  that  the  mix 
of  students  tended  to  become  more  positive  towards  the  way  in  which  computers  could  impact  their  college 
experience.  Overall,  the  students  became  more  comfortable  with  technology,  were  less  apprehensive 
toward  computers  and  were  less  concerned  that  the  introduction  of  computers  in  teaching  could  make  the 
learning  process  too  impersonal.  Indeed,  the  overall  student  body  agreed  that  computers  can  positively 
impact  the  way  in  which  students  and  teachers  can  interact  and  such  interactions  can  be  quite  gratifying. 
The  overall  student  body  were  not  overly  concerned  over  the  ways  in  which  computers  could  alter  the 
learning  process,  and  student  opinion  about  the  effects  of  computers  on  college  workload  did  not  change 
with  exposure.  In  many  ways,  the  general  student  body  composed  of  the  students  with  personal  machines 
and  those  without  the  machines  all  felt  that  the  computer  was  an  expected  and  "natural"  development  in  the 
pedagogic  process  and  thus  the  students  expressed  a desire  to  use  computers  in  classes.  Even  if  the  students 
did  not  change  their  opinion  about  the  way  in  which  computers  can  make  leaning  "easier,"  they  appeared  to 
agree  with  the  statement  that  computers  could  make  learning  easier.  Over  the  length  of  the  study,  the 
students  felt  that  the  use  of  computer  in  the  college  environment  can  be  perceived  to  make  the  process  of 
learning  easier. 

CROSS-SECTIONAL  PERSPECTIVES 

The  cross-sectional  perspective  looks  at  the  findings  from  the  point  of  view  of  differences  between 
the  students  in  any  particular  year,  or  the  differences  between  students  of  similar  characteristics  over  a 
couple  of  years.  This  analysis  demonstrates  that  the  legacy  students  are  less  favorable  than  the  ThinkPad 
students  in  nearly  every  attitude  towards  computer  aided  instruction.  The  legacy  students  who  realized  that 
they  would  not  benefit  from  the  enrichment  that  they  were  witnessing  around  them  were  expectedly  more 
concerned  about  the  utility  of  the  technology  in  the  teaching  environment  and  less  convinced  that 
computers  will  significantly  alter  the  ways  in  which  teachers  and  students  can  communicate.  The  ThinkPad 
students  on  the  other  hand  were  more  interested  in  being  able  to  use  computers  in  every  class  and  were  far 
more  convinced  that  the  technology  will  make  learning  and  teaching  better.  This  fundamental  difference  in 
attitude  between  the  ThinkPad  students  and  legacy  students  is  also  reflected  in  their  patterns  of  computer 
use.  The  ThinkPad  students  used  the  Internet  more  and  used  the  computer  more  for  electronic  mail  with 
teachers.  This  difference  in  patterns  of  use  is  consistent  with  the  differences  in  attitude  as  well.  Overall,  the 
ThinkPad  students  appear  to  be  far  more  positively  predisposed  towards  computers  and  thus  approach 
computer  enhanced  pedagogy  in  a more  positive  way. 

CONCLUSIONS 

A key  finding  that  emerge  from  this  data  is  that  longitudinal  exposure  to  a computerized  enriched 
learning  environment  might  not  necessarily  change  the  attitudes  toward  computer  enhanced  learning  unless 
the  development  of  the  computerized  enriched  environment  is  supplemented  by  the  fact  that  every  member 
in  the  environment  is  provided  with  the  tools  for  connecting  with  the  environment.  This  conclusion 
reaffirms  a classical  definition  of  the  networked  organization  where  it  is  assumed  that  every  member  of  the 
networked  organization  has  equal  and  democratic  access  to  the  network  (Sproull  and  Kiesler,  1992).  In  this 
study,  the  legacy  students  are  those  who  never  felt  the  empowerment  of  access  and  so  they  continued  to 
maintain  either  a negative  attitude,  or  did  not  alter  their  attitudes  in  a positive  way  even  when  placed  in  a 
networked  and  computer  enriched  environment.  On  the  other  hand,  the  ThinkPad  students  entered  the 
environment  with  a personal  laptop  computer  and  felt  immediately  connected  to  the  environment.  Indeed 
the  academic  atmosphere  was  getting  increasingly  designed  and  customized  for  the  ThinkPad  students  and 
fitting  in  there  was  not  a challenge  to  them. 

Finally,  this  study  addresses  the  question  of  effects  of  computer  enrichment  in  teaching  in  a 
circumvented  way.  In  stead  of  attempting  to  measure  learning  from  traditional  perspectives  of  grades  and 
test  scores,  we  have  measured  the  way  in  which  students  perceive  the  changes  brought  forth  in  the  learning 
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environment  with  the  introduction  of  computers.  There  is  evidence  in  the  data  that  the  ThinkPad  students 
considered  the  computer  enriched  environment  to  positively  transform  the  learning  space.  The  ThinkPad 
students  also  demonstrate  that  they  are  using  the  technology  more  for  learning-related  tasks.  The  legacy 
students,  on  the  other  hand,  do  not  demonstrate  these  tendencies.  Thus  a positive  attitude  and  increasing 
utilization  of  the  tool  by  the  ThinkPad  students  would  suggest  that  the  computer  enrichment  has  a series  of 
positive  outcomes  on  the  attitudes  toward  learning  and  the  process  of  learning  which  in  turn  can  have  a 
positive  outcome  on  learning  itself.  Some  evidence  for  this  claim  is  also  found  in  the  fact  that  the 
institutional  research  data  indicate  that  the  ThinkPad  students  of  Year  1 and  Year  2 have  some  of  the 
highest  grade  point  averages  in  several  years  and  have  the  highest  retention  rate  in  several  years.  While  it 
would  be  naiVe  to  claim  that  computer  enrichment  is  the  "cause”  for  these  trends,  it  would  also  be  incorrect 
to  discount  the  evidence  about  GPAs  and  not  relate  it  to  the  process  of  computer  enrichment.  In  many 
ways,  the  definitive  answer  about  the  effects  on  learning  is  still  being  debated,  but  the  results  here  suggest 
that  a networked  institution  where  the  students  have  easy  access  can  foster  a more  positive  attitude  toward 
the  use  of  technology  in  teaching  and  learning. 
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Abstract:  This  presentation  reports  on  an  effort  to  create  a hypermedia  supported  learning 
environment  using  problem -based  learning  for  middle  school  students  in  the  subject  matter 
area  of  space  science.  The  objectives  of  the  program  are  to  promote  problem-solving  skills,  to 
facilitate  learning  of  science  through  hypermedia  supported  cognitive  tools,  and  to  encourage 
collaborative  learning.  The  paper  discusses  the  design  and  development  of  the  learning 
environment  and  cognitive  tools  provided.  It  also  outlines  a research  agenda  for  conducting 
research  studies  using  the  environment. 


Introduction 

Enhancing  students'  higher  order  thinking  skills  has  been  a major  challenge  for  educators.  For  years, 
educators  have  been  trying  to  find  ways  to  create  learning  environments  that  can  encourage  students  to  be 
active  learners  and  develop  problem-solving  skills.  The  newly  devised  Texas  Essential  Knowledge  and  Skills, 
based  upon  the  national  educational  standards,  places  more  emphasis  on  the  integration  of  different  disciplines 
and  knowledge.  It  states  that  “In  Grade  6,  the  study  of  science  includes  conducting  field  and  laboratory 
investigations  using  scientific  methods,  analyzing  data,  making  informed  decisions,  and  using  tools...  to  collect, 
analyze,  and  record  information.  Students  also  use  computers  and  information  technology  tools  to  support 
scientific  investigations.”  This  is  a very  important  objective  to  achieve.  However,  little  instructional  materials 
are  currently  available  to  meet  such  a challenge. 

The  project  to  be  presented  is  primarily  concerned  with  these  important  educational  issues  and 
approaches  them  by  investigating  how  to  use  hypermedia  technology  and  problem-based  learning  to  enhance 
higher  order  thinking  skills  for  six  graders.  This  is  a research  and  development  project  in  that  the  need  for 
conducting  it  is  based  upon  the  current  literature  and  research  in  education  and  instructional  technology. 
Research  has  been  an  integral  part  of  the  development  both  as  formative  and  summative  evaluations.  This 
presentation  will  focus  on  the  design  and  development  of  the  project. 


Theoretical  Assumptions 

Educational  literature  indicates  that  problem-based  learning  is  an  effective  way  to  meet  the  challenge 
of  enhancing  students’  higher  order  skills.  Problem-based  learning  (PEL)  is  an  instructional  approach  that 
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exemplifies  authentic  learning  and  emphasizes  solving  problems  in  rich  contexts.  Though  originally  developed 
in  medical  education,  it  has  been  subsequently  applied  to  a wide  variety  of  professional  schools  including 
business  and  law.  Recently,  there  is  a growing  interest  among  educators  in  using  PBL  in  the  K-12  setting.  In 
problem-based  learning,  students  are  given  a problem  replete  with  all  the  complexities  typically  found  in  real 
world  situations,  and  work  collaboratively  to  develop  a solution.  PBL  provides  students  an  opportunity  to 
develop  skills  in  problem  definition  and  problem  solving,  to  reflect  on  their  own  learning,  and  to  develop  a deep 
understanding  of  the  content  domain  (Cognition  and  Technology  Group  at  Vanderbilt,  1992;  Lajoie,  1993; 
Jacobson  & Spiro,  1995). 

Three  implementations  of  problem-based  learning  are  of  particular  interest  to  this  project.  The 
anchored  instruction  by  the  Cognition  and  Technology  Group  at  Vanderbilt  used  video-based  scenarios  to 
anchor  instruction  in  authentic  problem-solving  tasks  (Cognition  and  Technology  Group  at  Vanderbilt,  1992). 
Schank’s  goal-based  scenarios  use  computer  simulations  to  embed  learning  in  problem-solving  activities 
(Schank,  Fano,  Jona,  & Bell,  1993).  Cognitive  flexibility  theory  by  Rand  Spiro  emphasizes  learning  from 
multiple  perspectives  in  ill-structured  domains  (Spiro  & Jehng,  1990).  Though  approached  in  different  ways,  all 
three  implementations  deal  with  solving  complex  problems  in  authentic  situations,  and  emphasize  the 
importance  of  learner-centered  activities. 

Educational  literature  has  demonstrated  the  benefits  of  problem-based  learning  to  enhance  students’ 
thinking  skills  (Boud  & Feletti,  1991;  Cognition  and  Technology  Group  at  Vanderbilt,  1992;  Stepien, 
Gallagher,  & Workman,  1993).  However,  research  has  also  shown  that  the  implementation  of  PBL  in  the 
classroom  is  a challenging  and  complex  task  for  both  teachers  and  students.  Learning  in  a PBL  environment  is  a 
challenge  to  the  students  because  it  requires  their  active  participation  in  a self-directed  manner.  The 
development  of  a solution  plan  requires  learners  to  sift  through  vast  amount  of  information,  distinguishing  that 
which  is  pertinent  from  that  which  is  not.  Learners  need  to  be  able  to  generate  and  evaluate  their  problem- 
solving goals,  and  organize  their  knowledge  in  ways  that  facilitate  development  of  a solution.  Teaching  in  PBL 
is  also  a challenge  because  teachers  must  not  only  be  familiar  with  the  PBL  approach  and  the  skills  needed  to 
teach  with  it,  but  also  must  be  able  to  provide  necessary  and  often  "just-in-time"  support  tailored  to  individual 
and  small  group  needs.  Effective  implementation  of  PBL  requires  necessary  scaffolding  (Koschmann,  Kelson, 
Feltovich,  & Barrows,  1996).  This  can  be  an  overwhelming  task  in  K-12  classrooms,  where  learners  may  have 
limited  self-directed  learning  skills  and  classes  may  be  large. 

The  purpose  of  this  project  is  to  create  a hypermedia-supported  problem-based  learning  environment 
for  sixth-graders.  Through  research  and  development,  it  is  our  intention  to  investigate  and  understand  in  what 
ways  the  hypermedia  technology  can  be  used  to  share  the  responsibilities  of  the  teachers  to  provide  necessary 
and  “just-in-time”  scaffolding  to  support  students’  learning  in  PBL  environments. 


Program  Description 

Designed  for  six  graders,  the  hypermedia  program  developed.  Alien  Rescue,  utilizes  video,  audio, 
animation,  text,  and  graphics  to  create  simulations  in  which  students  will  encounter  problems  in  a rich  context 
and  engage  in  scientific  inquiries.  The  simulation  begins  with  the  arrival  in  Earth  orbit  of  a group  of  six  species 
of  aliens  fleeing  the  destruction  of  their  own  planetary  system.  Their  ship  was  damaged  during  their  voyage, 
and  except  for  their  engines  and  computer  databases,  little  of  their  technology  continues  to  function.  In  order  to 
survive,  they  must  find  new  homes  on  worlds  which  can  support  them.  Having  picked  up  Earth  broadcasts,  the 
aliens  learned  our  languages  with  the  intention  of  asking  for  our  help  to  relocate  to  worlds  in  our  solar  system. 
However,  when  their  life  support  failed  completely,  the  aliens  could  only  complete  a distress  message  to  be  sent 
once  they  reached  Earth  orbit,  then  entered  a state  of  suspended  animation,  where  they  must  remain  until  they 
are  safely  relocated  to  suitable  worlds.  This  science  fiction  premise  takes  students  to  a newly  operational 
international  space  station  where  they  become  part  of  a worldwide  effort  to  rescue  alien  life  forms. 

Students  are  asked  to  take  on  the  role  of  junior  scientists  working  with  experts  aboard  the  space  station 
to  determine  the  most  suitable  relocation  site  for  each  alien  species.  To  solve  the  problem,  students  must  engage 
in  a variety  of  activities.  They  must  learn  about  the  aliens  and  identify  the  basic  needs  of  each  species.  They 
must  then  investigate  the  planets  and  moons  of  our  solar  system,  searching  them  for  possible  matches  with  the 
needs  of  the  aliens.  Students  gather  this  information  by  performing  searches  within  a database  on  our  solar 
system  and  launching  probes  that  they  have  constructed  to  gather  the  additional  information  not  available 
through  the  existing  databases.  They  must  also  engage  in  collaborative  planning  and  decision-making  as  they 
determine  how  to  use  the  resources  of  the  solar  system  effectively. 
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In  the  course  of  developing  a solution  plan,  students  learn  about  both  our  solar  system,  and  the  tools 
and  procedures  scientists  use  to  gather  that  information.  The  hypermedia  program  allows  students  to  have 
access  to  all  the  tools  and  information  needed  to  develop  a solution  plan,  but  the  program  is  structured  in  such  a 
way  as  to  not  suggest  what  that  solution  should  be.  Students  are  encouraged  to  explore  the  virtual  space  station 
as  they  determine  for  themselves  the  information  they  need  and  the  process  they  will  use  to  develop  a solution 
plan. 


Design  Features 

The  program  incorporates  design  features  that  are  supported  by  problem-based  learning  and 
hypermedia  research.  These  features  include:  (1)  situating  the  problem  in  a rich  context  and  allowing  learners  to 
engage  in  scientific  inquiries  as  experts  do;  (2)  presenting  the  problem  with  its  complexity,  yet  providing  tools 
to  support  students  in  working  with  complexity;  (3)  providing  information  in  multimedia  formats  to  allow 
dynamic  and  interactive  presentations  that  address  different  learning  styles  and  student  needs;  (4)  providing 
experts'  guidance  from  multiple  perspectives  to  facilitate  knowledge  acquisition  and  transfer;  and  (5) 
emphasizing  the  interrelated  nature  of  knowledge  through  hypertextual  nodes  and  links. 

Specifically,  The  program  provides  a range  of  cognitive  tools  to  scaffold  students'  problem-solving 
process.  The  tools  are  provided  through  the  main  environment  of  the  program,  the  international  space  station, 
and  through  the  imaginative  goggles  students  are  wearing  wherever  they  go  in  the  environment.  The  tools  are 
in  two  categories:  (1)  information  presentation  and  (2)  knowledge  construction  (see  Table  1).  With  the  tools 
provided  in  the  information  presentation  category,  students  are  required  to  locate  helpful  resources,  search  and 
research  existing  knowledge  databases,  select  relevant  information,  and  make  effective  decisions.  With  the  tools 
available  in  the  knowledge  construction  category,  students  are  to  collect  new  data,  interpret  and  organize  data, 
build  the  rationale  for  their  decisions  and  present  their  solution  plan.  Expert  modeling  and  coaching  for  problem 
generation  and  problem  solving  is  provided  by  expert  demonstrating  how  to  use  the  tools  and  stories  illustrating 
how  similar  problems  were  solved  by  scientists.  The  scripts  for  such  video  clips  were  developed  by  examining 
how  students  used  an  earlier  version  of  the  program  without  modeling,  noting  their  sources  of  difficulty,  then 
providing  modeling  to  support  them. 


. 

Information 

Presentation 

Conference 

Room 

Students  receive^l^ideo  messages  from  Earth  in  the  conference  room.  These 
messages  provide  details  about  the  problem  situation  and  help  to  direct 
student  activity.  These  messages  help  to  maintain  the  impression  that 
students  are  part  of  a scientific  community,  where  coordinated  efforts  and 
frequent  communication  are  essential  components  of  problem  solving. 

Alien 

Computer 

Because  the  alien  ship  is  damaged,  scientists  decide  to  move  the  alien 
computer  to  the  space  station.  It  contains  information  about  the  aliens:  their 
physiology,  technology,  homeworlds,  and  history.  However,  early  in  the 
program  students  learn  that  the  aliens  had  insufficient  time  to  complete  the 
translation  of  their  computer  database,  so  only  a limited  amount  of 
information  is  available.  Students  must  sift  through  the  information  to 
identify  the  needs  of  each  alien  species. 

Charts 

A variety  of  charts  are  provided  which  are  designed  to  scaffold  students’ 
efforts  to  organize  their  data  and  make  decisions. 

Solar  System 
Database 

This  contains  information  about  our  sun,  the  nine  planets,  and  ten  of  the 
moons  in  our  solar  system.  Information  useful  to  solving  the  problem  is 
provided,  but  the  database  is  structured  in  such  a way  as  not  to  suggest 
which  pieces  of  information  are  relevant  to  the  problem  or  what  the  best 
solution  to  the  problem  might  be.  The  database  includes  pictures  that 
students  can  bookmark  for  later  retrieval. 
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Messages 

Periodically  through  the  program,  students  receive  video  email.  These 
messages  are  recorded  in  text  form  in  the  message  tool  for  students  to  review 
as  they  deem  necessary. 

Mission 

Database 

Students  can  review  information  on  five  landmark  probe  missions  to  see 
how  scientists  designed  probes  to  collect  specific  types  of  data  in  the  past. 

Knowledge 

Construction 

Probe  Builder 

The  Probe  Builder  contains  lists  and  descriptions  of  real  scientific 
equipment  used  in  both  past  and  future  probe  missions  to  investigate  other 
worlds  in  our  solar  system.  Students  construct  probes  by  making  choices 
within  four  categories:  probe  type,  communication,  power  source,  and 
instruments.  Budgetary  constraints  force  students  to  consider  their  choices 
carefully  and  make  compromises. 

Launch  Room 

Students  review  the  probes  they  have  built  in  the  Probe  Builder,  and  decide 
which  ones  to  actually  send  in  the  Launch  Room.  Only  probes  that  are 
launched  show  up  in  the  Control  Room,  where  data  from  them  is  displayed. 

Control  Room 

Raw  data  collected  by  the  probes  are  displayed  in  the  Control  Room. 
Students  must  interpret  this  data  in  order  to  turn  it  into  information  that  they 
can  use  to  develop  their  solution.  As  in  all  probe  missions,  malfunctions  are 
possible,  and  poor  planning  can  result  in  mission  failure. 

Notebook 

Students  use  their  notebook  to  collect  and  organize  information  useful  in 
solving  the  problem.  Students  can  create  numerous  sections  within  their 
notebook  to  help  them  organize  their  notes.  The  notebook  was  deliberately 
designed  not  to  allow  students  to  cut  and  paste  information  contained  in  the 
rest  of  the  program,  meaning  that  students  must  type  their  own  notes  based 
on  what  they  think  is  important  enough  to  record.  A second  function  of  the 
notebook  is  the  bookmarks  feature.  Students  can  drag  pictures  and  video 
from  any  part  of  the  program  to  their  notebook.  These  media  elements  are 
bookmarked  for  later  review  and  possible  use  in  presentations. 

Presentation 

Tool 

A typical  assignment  for  students  to  be  given  within  Alien  Rescue  is  to 
create  a presentation  on  a decision  they  made  and  their  rationale  for  that 
decision.  The  presentation  tool  permits  students  to  create  a series  of  slides 
using  media  they  have  bookmarked  and  notes  from  their  notebook.  Students 
choose  from  a number  of  slide  templates,  meaning  that  slides  are  fast  and 
easy  to  make. 

Table  1.  Hypermedia-Supported  Cognitive  tools 


Development 

The  development  of  the  project  follows  a model  based  upon  practice  in  the  multimedia  industry  and 
current  literature  in  instructional  technology.  This  four-phase  model  has  the  following  phases:  (1)  planning,  (2) 
designing,  (3)  production  and  (4)  evaluation  (Liu,  Jones,  & Hemstreet,  1998).  Testing  and  revision  is  an  on- 
going process  both  from  the  teachers  and  the  students.  So  far  the  production  of  this  project  has  drawn  expertise 
from  various  fields  which  include  instructional  design,  hypermedia  research,  astronomy,  graphic  design, 
multimedia  programming,  video  and  audio,  and  writing.  It  is  a collaborative  effort  by  the  Instructional 
Technology  Program,  the  Institute  for  Technology  and  Learning,  the  Astronomy  Department,  and  the 
McDonald  Observatory  at  University  of  Texas  - Austin. 


Research  Agenda 
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Both  formative  and  summative  evaluations  are  integral  parts  of  the  project.  Two  phases  of  formative 
evaluations  have  been  conducted,  one  with  a small  group  of  sixth-graders  for  an  informal  evaluation,  and  one 
with  a larger  group  for  a formal  evaluation.  The  purpose  of  these  evaluations  was  to  understand  how  the  target 
audience  reacted  to  the  program  and  what  improvements  needed  to  be  made.  Based  upon  these  evaluations, 
revisions  have  been  made.  The  improvement  of  the  program  is  an  on-going  process,  however.  In  addition  to 
program  improvement,  research  is  currently  underway.  Present  and  future  research  studies  will  investigate  the 
research  issues: 

• the  effect  of  the  hypermedia-supported  cognitive  tools  on  students’  achievement  and  their  problem- 
solving skills, 

• the  relationship  between  the  use  of  hypermedia-supported  cognitive  tools  and  students’  achievement 

• the  possible  differential  effects  of  the  project  on  students  with  different  abilities, 

• the  effect  of  using  expert  stories  on  students’  achievement, 

• the  change  of  students’  epistemological  beliefs  as  a result  of  using  the  program. 

In  this  presentation,  we  will  share  the  design  and  development  of  the  project,  demonstrate  the  program  and 
outline  the  research  agenda. 


References 

Boud,  D.,  & Feletti,  G.  (1991).  The  challenge  of  problem  based  learning.  London:  Kogan  Page. 

Cognition  and  Technology  Group  at  Vanderbilt.  (1992).  The  Jasper  series  as  an  example  of  anchored 
instruction:  Theory,  program  description,  and  assessment  data.  Educational  Psychologist^  27(3),  291-315. 

Jacobson,  M.  & Spiro,  R.  J.  (1995).  Hypertext  learning  environments,  cognitive  flexibility,  and  transfer  of 
complex  knowledge:  An  empirical  investigation.  Journal  of  Educational  Computing  Research,  12(4),  301-333. 

Lajoie,  S.  P.  (1993).  Computer  environments  as  cognitive  tools  for  enhancing  learning.  In  S.  P.  Lajoie  and  S.  J. 
Derry  (Eds.),  Computers  as  Cognitive  Tools  (pp.26 1-288).  Hillsdale,  NJ:  Lawrence  Erlbaum  Associates,  Inc. 

Koschmann,  T.,  Kelson,  A.  C.,  Feltovich,  P.  J.,  & Barrows,  H.  S.  (1996).  Computer-supported  problem-based 
learning:  A principled  approach  to  the  use  of  computers  in  collaborative  learning.  In  T.  Koschmann  (Ed.), 
CSCL:  Theory  and  practice  of  an  emerging  paradigm  (pp.  83-124).  Hillsdale,  NJ:  Lawrence  Erlbaum 
Associates,  Inc. 

Liu,  M,  Jones,  C.,  & Hemstreet,  S.  (1998).  A study  of  the  multimedia  design  and  production  process  by  the 
practitioners.  Journal  of  Research  on  Computing  in  Education,  30(3),  254-280. 

Schank,  R.,  Fano,  A.,  Jona,  M.,  & Bell,  B.  (1993).  The  design  of  goal-based  scenarios  (Tech.  Rep.  No.  39). 
Evanston,  IL:  Northwestern  University,  The  Institute  for  the  Learning  Sciences. 

Spiro,  R.  J.,  & Jehng,  J.  (1990).  Cognitive  flexibility  theory:  Theory  and  technology  for  the  nonlinear  and 
multidimensional  traversal  of  complex  subject  matter.  In  D.  Nix  & R.  Spiro  (Eds.),  Cognition,  Education,  and 
Multimedia  (pp.  163-205).  Hillsdale,  NJ:  Lawrence  Erlbaum  Associates. 

Stepien,  W.  J.,  Gallagher,  S.  A.,  & Workman,  D.  (1993).  Problem-based  learning  for  traditional  and 
interdisciplinary  classrooms.  Journal  for  the  Education  of  the  Gifted,  76(4),  338-357. 


616 


Nano-Visualization  On  the  Web 


Laurie  Luckenbill 
Arizona  State  University 
Center  for  Solid  State  Science,  PO  Box  871704 
Tempe,  AZ,  USA  85287-1704 
Laurie.Luckenbill@asu.edu 

Kirsten  Hintze 
Arizona  State  University 
Center  for  Solid  State  Science,  PO  Box  87 1 704 
Tempe,  AZ,  USA  85287-1704 
khb@asu.edu 

B.L.  Ramakrishna 
Arizona  State  University 
Center  for  Solid  State  Science,  PO  Box  871704 
Tempe,  AZ,  USA  85287-1704 
BRamakrishna@asu.edu 

Vincent  Pizziconi 
Arizona  State  University 

Chemical,  Bio  and  Materials  Engineering  Department,  PO  Box  876006 
Tempe,  AZ,  USA  85287-6006 
Vincent.Pizziconi@asu.edu 


Abstract:  The  Interactive  Nano-Visualization  in  Science  and  Engineering  Education  (IN- 
VSEE)  project  provides  the  first  real-time  interface  for  remote  operation  of  Scanning  Probe 
Microscopy  (SPM)  over  the  World  Wide  Web  for  educational  and  training  purposes.  In 
addition  to  providing  "live  remote  control"  of  Nobel  prize-winning  microscopy  techniques  for 
doing  discovery  based  learning  of  our  material  nanoworld,  novel  and  highly  interactive 
learning  modules  are  being  developed  for  diverse  material  systems  which  demonstrate  the 
interdisciplinary  nature  of  nanotechnology.  IN-VSEE  brings  the  excitement  of  research  using 
cutting-edge  technology  to  upper-level  high  school  and  first-year  college  students  and 
teachers. 


Introduction 

Advances  in  science  and  technology  have  led  to  the  design  and  manufacture  of  materials  and  devices  of 
increasingly  smaller  dimensions.  Engineering  techniques  have  already  surpassed  the  limits  of  resolution 
afforded  by  the  naked  eye.  Since  the  1960's  device  sizes  have  rapidly  decreased  from  millimeter  (10‘^)  to 
micrometer  (lO"^)  and  now  approaching  the  nanometer  (10‘^)  scales.  They  may  possibly  decrease  to  atomic 
scale,  measured  in  the  angstrom  (10‘*^)!  Working  at  the  nanometer  scale  has  become  a reality  as  scientists  and 
engineers  strive  to  exploit  miniaturization  techniques  and  technologies  for  the  future.  These  nanotechnologies 
are  envisioned  to  become  the  manufacturing  reality  in  the  next  millenium.  In  an  effort  to  introduce  the 
fascinating  nanoworld,  Arizona  State  University  has  become  a leader  in  the  use  of  Scanning  Probe  Microscopes 
in  research  and  education.  The  Interactive  Nano-Visualization  in  Science  and  Engineering  Education  (IN- 
VSEE)  project,  led  by  ASU  is  a consortium  of  university,  community  college,  and  high  school  teachers,  as  well 
as  industrial  scientists  and  science  museum  educators.  The  IN-VSEE  team  has  developed  the  first  interface  for 
remote  operation  of  Scanning  Probe  Microscopy  (SPM)  over  the  World  Wide  Web  (WWW)  to  promote 
education  and  training  to  prepare  students  for  careers  in  the  emerging  nanotechnologies.  The  primary  goals  of 
IN-VSEE  are  to: 


o 
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♦ convey  the  excitement  of  nanoscience  and  nanotechnology,  promote  student-initiated  learning  and 
increase  interest  in  science  and  engineering  careers 

♦ teach  fundamental  concepts  in  science  and  engineering  using  visualization  techniques  to  help  students 
learn  and  integrate  difficult  concepts  more  effectively 

♦ provide  students  with  the  capability  to  routinely  explore  materials  in  three  dimensions  with  resolutions  at 
the  nanoscale  and  even  down  to  the  atomic  scale 

♦ demonstrate  the  feasibility  of  using  the  WWW  for  remote  operation  of  sophisticated  research-grade 
laboratory  instrumentation  that  can  be  used  directly  in  the  classroom. 

The  IN- V SEE  project  is  working  to  accomplish  these  goals  by: 

♦ creating  a user-friendly  interface  for  ’SPM  Live!’  remote  SPM  experimentation 

♦ developing  cross-discipline  interactive  instructional  modules  which  provide  key  material  science  and 
engineering  concepts 

♦ providing  an  ’invitation’  to  explore  the  nanoworld  via  ’SPM  Live!’ 

♦ highlighting  the  interdisciplinary  nature  of  nanotechnology 

♦ working  with  teachers  to  provide  an  introduction  to  the  technology,  as  well  as  the  use  of  this  technology 
in  teaching 

♦ obtaining  and  cataloging  exciting  images  of  materials  using  the  Nobel  prize-winning  SPM  coupled  with 
other  advanced  surface  characterization  instruments. 

To  date,  IN-VSEE  has  succeeded  in  developing  a WWW  user  friendly  interface  for  remote  control  of  a 
scanning  probe  microscope  which  is  located  on  the  Arizona  State  University  Main  campus.  Additionally,  17 
instructional  modules  are  in  the  review  phase  of  development.  Three  teacher  workshops  have  been  held  and  a 
two-day  workshop  is  scheduled  for  August  1999,  Accompanying  the  modules  is  a comprehensive  image  bank, 
accessible  through  the  IN-VSEE  Web  site,  of  micrographs  and  nanographs  created  using  various  microscopies. 


Remote  SPM 


The  computer  science  task  force  of  IN-VSEE  has  succeeded  in  creating  a WWW  interface  to  remotely  control 
the  scanning  probe  microscope  from  any  on-line  classroom  in  the  world.  Figure  1 shows  the  Remote  operator 
interface  page  for  SPM  Live!  From  this  page  the  remote  operator  can  control  the  instrument,  while  others  can 


Figure  1:  SPM  Live!  WWW  Interface 


participate  in  the  same  experiment  using  the  observers’  page.  A chat-dialog  box  allows  synchronous 
interaction  between  the  operator,  observers  and  IN-VSEE  staff  at  the  microscope  site.  Thus,  students  and 
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scientists  can  discuss  the  image  being  viewed.  Another  window  on  the  web  page  provides  real-time  video  of 
the  instrument  and  the  IN-VSEE  laboratory  in  order  to  personalize  the  viewing  experience.  The  controller 
interface  allows  many  of  the  same  capabilities  as  the  actual  instrument,  including  zoom  and  rotation. 


Instructional  Modules 

One  specific  aim  of  the  educational  modules  is  to  provide  a bridge  between  the  'virtual  classroom'  and 
laboratory.  The  modules  provide  a roadmap  for  students  on  how  to  design  and  conduct  experiments,  with  the 
end  goal  of  stimulating  interest  in  remote  operation  of  the  SPM.  The  instructional  modules  center  around  a 
common  theme  of  understanding  and  manipulation  of  natural  and  man-made  materials.  The  module  topics 
selected  utilize  interactive,  discovery-based  learning  activities  to  introduce  or  reinforce  applied  material 
concepts  within  various  disciplines  and  among  various  material  classes  over  a wide  range  in  scale.  The 
interactive  IN-VSEE  modules  provide: 

♦ reinforcement  and  integration  of  key  concepts  and  fundamental  principles  that  are  taught  in  high  school 
and  lower  division  college  science,  math  and  engineering  curricula 

♦ prospective  users  on  the  methodologies  of  experimental  design  and  encouragement  to  formulate  original 
experiments  for  remote  SPM  instrumentation  on  the  web,  through  SPM  Live! 

♦ students  with  a flavor  of  research 

♦ teachers  with  topics  designed  to  meet  national  science  education  standards. 


Figure  2:  Connections  made  within  each  module.  A material's  atomic  structure,  properties, 
processing  and  performance  are  all  related  at  the  nano-level,  as  well  as  the  macro-level. 


The  IN-VSEE  instructional  module  task  force  has  developed  a module  template  to  facilitate  module 
development  and  provide  consistency  between  modules.  The  module  structure  uses  a discovery  environment  to 
provide  background  and  relevance  to  the  use  of  the  SPM  Live!  remote  instrument.  Figure  2 represents  the 
unifying  concept  reinforced  in  each  module;  that  structure,  properties,  processing  and  performance  of  materials 
are  intimately  linked  at  all  levels  of  scale.  The  standardized  user  interface  and  layout  includes  an  attention 
grabber,  and  a navigation  panel  with  a menu  of  topics  and  links.  These  topics  and  links  include: 

♦ real  world  applications 

♦ FAQ’s  and  trivia  questions 

♦ core  material  of  text  in  images 

♦ key  concepts 

♦ SPM  relevant  to  the  module,  SPM  Live! 

The  uniform  structure  of  the  modules  facilitates  ease-of-use  between  modules.  The  generic  module  template 
will  also  assist  in  development  of  future  modules.  Figure  3 shows  a sample  module  interface. 
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Figure  3:  Instructional  module  interface.  All  modules  use  a standardized  navigation  system  for 

consistency  between  the  17  modules. 

Table  1 lists  the  titles  of  the  17  modules  currently  under  development.  The  IN-VSEE  team  is  beta  testing 
modules  in  remote  classroom  demonstrations  at  Chandler-Gilbert  Community  College,  Scottsdale  Community 
College  and  Glendale  Community  College.  Additionally,  teachers  will  evaluate  modules  at  the  August 
workshop. 


1 . Making  Sense  of  Size  and  Scale 

2.  Theory  and  Virtual  Operation  of  SPM 

3.  The  Allotropes  of  Carbon 

4.  The  Five  Kingdoms  of  Biology  (Y east) 

5.  Engineered  Materials 

6.  Biological  Structural  Materials 

7.  Why  does  a Light  Bulb  Bum  Out? 

8.  Visualizing  Properties:  Friction 

9.  The  World  of  Liquid  Crystals 


10.  Biominerals:  It's  a Hard  Life 

1 1 . The  Morphology  and  Use  of  Gold  Films 

12.  What  Is  That  in  Your  Dog's  Dish? 

13.  Osmotic  Pressure  in  Red  Blood  Cells  and  Plant 
Cells 

14.  The  Miracle  Molecule:  DNA 

15.  The  Music  of  Spheres 

16.  Iridescence 

17.  Modem  Information  Storage  Media 


Table  1:  Modules  currently  under  development 
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Teachers’  Workshops 


The  purpose  of  the  teachers’  workshops  is  twofold.  First,  teacher-training  is  provided  to  help  teachers  develop 
confidence  in  using  this  state-of-the-art  technology.  Second,  discussions  and  brainstorming  sessions  with  the 
IN-VSEE  team  are  designed  to  assist  teachers  in  exploring  ways  they  can  best  use  this  technology  to  enhance 
their  educational  objectives.  The  first  teachers’  workshop  in  October  1997,  focused  on  the  use  of  image 
processing  in  the  classroom.  Image  processing  provides  a dynamic,  hands-on  way  to  incorporate  microscopy 
images  in  the  classroom.  (Greenberg  et  al.  1993)  It  also  provides  links  to  real  world  applications  of 
microscopy.  In  this  workshop  teachers  also  provided  feedback  about  the  "Allotropes  of  Carbon"  module.  The 
second  teachers'  workshop  in  July  1998,  focused  on  ways  to  overcome  barriers  to  using  remote  SPM, 
instructional  modules,  and  microscopy  images  in  the  classroom.  The  upcoming  two-day  teachers’  workshop, 
scheduled  for  August  1999,  will  focus  on  hands-on  activities  and  beta-testing  of  instructional  modules, 
especially  the  "Making  Sense  of  Size  And  Scale"  and  the  ’’Theory  and  Virtual  Operation  of  SPM"  modules 
which  are  considered  the  keystone  modules  for  providing  background  to  students  in  understanding  the 
nanoscale. 


Image  Bank 

Also  available  on  the  IN-VSEE  Web  site  is  a database  of  more  than  400  SPM  and  other  related  microscopy 
images.  These  are  images  collected  by  the  IN-VSEE  team  during  classroom  demonstrations  and 
experimentation,  as  well  as  images  provided  by  other  scientists  at  Arizona  State  University.  The  images  come 
from  a wide  variety  of  microscopy  techniques  including  Scanning  Electron  Microscopy,  Scanning  Tunneling 
Microscopy,  Atomic  Force  Microscopy,  Magnetic  Force  Microscopy,  along  with  more  traditional  Optical 
Microscopy,  and  macroscopic  photographs.  Learning  activities  utilizing  the  images  beyond  the  instructional 
modules  are  also  being  generated. 


Conclusions 

IN-VSEE  has  pioneered  the  effort  creating  a user  friendly  interface  for  remote  operation  of  research-grade 
laboratory  instruments  i.e.,  the  Scanning  Probe  Microscope  for  educational  purposes.  To  provide  a firm 
foundation  for  students  and  teachers  about  SPM  experimentation,  17  instructional  modules  are  being  developed. 
These  modules  are  currently  being  beta-tested.  Additionally,  a ’virtual'  SPM  is  being  developed  for  the  web  site 
to  provide  students  practice  using  SPM  equipment  prior  to  doing  their  own  experiments  with  SPM  Live! 
Teacher  workshops  in  the  past  and  future,  focus  on  providing  teachers  with  training  to  use  the  technology  to 
enhance  instruction  in  their  classroom.  A database  of  images  is  available  for  downloading  to  use  off  line  or  for 
activities  on-line.  This  project  makes  available  state-of-the-art,  Nobel  prize-winning  research  tools  available  to 
students  anywhere  via  the  Internet.  These  powerful  tools  intrigue  students  in  the  process  of  scientific 
investigation  and  the  exciting  field  of  nanotechnology.  IN-VSEE  also  provides  resources  which  can  expand 
cross-discipline  curricula  of  any  school  on-line! 
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Abstract;  This  paper  presents  Colt,  a system  for  the  development  of  applications  that 
support  close  collaboration  between  two  or  more  users.  The  Colt  system  has  three  parts:  a 
design  methodology,  a software  toolkit,  and  additional  tools  for  visualization  and  analysis. 
The  software  toolkit  includes  computational  objects  that  support  simultaneous  manipulations 
from  multiple  users.  We  define  and  illustrate  "Cooperatively  Controlled  Objects"  (CCOs) 
and  we  discuss  their  design  and  development.  Finally,  a color  matching  activity  is  presented 
as  an  example  of  a collaborative  activity  that  was  built  using  Colt  and  CCOs. 


Introduction 

Developing  applications  to  support  synchronous  collaboration  by  two  or  more  users  involves  special 
challenges.  An  application  designer  needs  to  anticipate  the  interaction  between  the  user  and  the  computer  as 
well  as  the  interactions  between  the  users.  The  designer  also  needs  system  support  for  development  of  software 
that  accepts  simultaneous  input  from  multiple  users  on  the  same  machine.  The  Colt  system  is  designed  to 
address  both  of  these  needs. 

Colt  is  comprised  of  three  parts,  a design  methodology,  a software  toolkit,  and  visualization  and 
analysis  tools.  The  design  methodology  is  based  on  a series  of  questions  to  aid  developers  in  the  creative 
process  of  planning  a collaborative  activity.  The  toolkit  supports  the  implementation  of  collaborative  programs 
by  providing  a shell  application,  a hierarchy  of  "cooperatively  controlled  objects"  (CCOs),  and  support  for 
input  from  multiple  users.  CCOs  are  designed  to  support  close  collaboration  among  the  users  of  a computer 
program.  Each  CCO  contains  a mechanism  to  record  a history  of  how  users  cooperatively  controlled  the 
objects.  The  visualization  and  analysis  tools  present  different  views  of  the  object  histories.  These  tools  can  be 
employed  after  program  testing  to  determine  if  the  implementation  meets  the  original  design  criteria.  Colt  is 
unique  in  its  support  for  the  development  of  co-present  simultaneous  computer  based  activities.  Our  motivation 
for  developing  Colt  is  described  in  detail  in  (Bricker98).  Rather  than  cover  all  three  parts  of  the  Colt  system, 
this  paper  describes  only  the  software  toolkit  component. 

The  Cooperatively  Controlled  Object 

Many  common  non-computer  activities  contain  elements  of  highly  synchronized  cooperation.  Some 
activities  are  done  collaboratively  purely  for  the  social  enjoyment.  Some  are  done  because  they  are  easier  to 
complete  with  additional  help.  Unfortunately,  most  desktop  computers  today  are  designed  to  accommodate 
input  from  one  user  at  a time.  Groups  who  wish  to  collaborate  using  one  computer  are  forced  to  share  control 
awkwardly  through  a single  mouse  and  keyboard.  Social  and  physical  conflicts  over  these  input  devices  may 
arise  since  current  operating  systems  and  application  software  only  support  a single  input  stream.  One  solution 
to  the  conflict  problem  is  to  provide  a separate  physical  device  for  each  user.  Many  video  game  systems,  such 
as  the  Super  Nintendo  Entertainment  System  (SNES),  do  support  multiple  joysticks.  However,  the  multi-user 
games  for  these  systems  often  employ  either  a turn-taking  mechanism,  or  simultaneous  competitive  play.  The 
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risk  in  enforcing  turn-taking  is  that  inactive  users  will  lose  focus  on  the  task.  The  risk  in  supporting 
simultaneous  competitive  play  is  that  it  does  not  appeal  to  many  people,  particularly  young  girls.  Colt  was 
developed  to  support  simultaneous  collaborative  interactions  in  order  to  address  these  issues.  One  of  the  goals 
of  this  research  is  to  define  a class  of  interesting  objects  that  helps  to  keep  the  users  focused  during 
collaborative  activities. 

Cooperatively  Controlled  Objects  Defined 

In  object-oriented  terminology,  an  object  contains  state  and  has  behavior.  A controlled  object  is  an 
object  containing  methods  that  allow  one  or  more  users  to  manipulate  properties  of  that  object.  Typically  these 
properties  are  manipulated  directly  through  input  devices  such  as  mice,  joysticks,  keyboards,  etc.  A 
cooperatively  controlled  object  (CCO)  is  a controlled  object  that  is  designed  to  be  manipulated  simultaneously 
by  more  than  one  user  based  on  certain  relationships  that  hold  between  user  inputs  and  components  of  the 
CCO. 

Each  CCO  is  assumed  to  have  a set  of  components  or  "properties."  Each  property  may  take  on  one  of 
a possibly  infinite  number  of  values.  In  the  physical  world  some  properties,  like  the  location  of  a spot  on  the 
floor,  may  have  an  infinite  number  of  possible  values.  We  will  assume  that  the  computer  representation  is 
simplified  with  a finite  set  of  discrete  values. 

A control  mechanism  for  a CCO  provides  a means  for  a group  of  users  to  provide  a value  for  one  or 
for  several  (or  even  all)  of  the  properties  of  the  CCO.  The  control  mechanism  for  a property  in  a CCO  can  be 
described  as  a function: 

y=f((^h  vj,  (Uj,  U2,  Um)), 

where  V is  the  output,  or  new  value  of  the  property,  each  v,  is  the  input  value  of  the  properties  on  which  this 
function  depends,  and  each  Uj  is  the  input-device  state  corresponding  to  the  multiple  users  simultaneously 
manipulating  the  object.  By  providing  functions  for  each  of  the  CCO’s  modifiable  properties,  a complete 
control  mechanism  for  the  CCO  can  be  specified. 

Fine-grained  sharing  (FGS)  is  a previously  studied  approach  for  defining  one  class  of  CCO  (Shen92). 
In  FGS,  an  object  is  expressed  in  terms  of  its  properties,  each  of  which  is  controlled  by  one  of  the  users. 
Although  fine-grained  sharing  can  offer  users  a clear  delineation  of  responsibility  and  control  for  shared 
objects,  it  lacks  the  ability  to  compel  tight  cooperation  among  the  users.  Users  who  work  on  separate  pieces  of 
a "shared"  document  do  not  necessarily  coordinate  in  real  time.  Instead  the  work  is  done  asynchronously  and 
are  the  users  not  forced  to  synchronize  their  activities  to  the  same  degree  that  one  does  in  musical 
performance,  dance,  three-legged  races,  and  the  like.  Our  goal  is  to  compel  users  to  interact  synchronously. 
CCOs  support  close  collaboration  though  complex  interactions  with  objects  by  adopting  a more  general 
framework  than  fine-grained  sharing. 

Specifying  the  types  of  interactions 

Developers  need  to  specify  how  closely  the  users  in  a collaborative  activity  should  work  together.  In 
order  to  assist  in  the  specification  of  the  users’  interactions,  we  have  developed  a sub-classification  of 
synchronous  activity  based  on  how  simultaneously  the  users  interact  with  a screen  object.  An  asynchronous 
cooperative  interaction  is  one  in  which  two  or  more  users  work  independently  on  a task,  then  synchronize  or 
coordinate  their  work  by  delivering  information  through  messages  such  as  conversation,  snail  mail  or  email. 
Participants  of  this  type  of  cooperative  interaction  may  not  expect  an  immediate  response  from  questions  or 
requests.  A required  asynchronous  cooperative  interaction  is  one  in  which  each  user  must  complete  their  part 
of  the  activity  before  the  other  user  may  proceed.  An  example  of  this  type  of  interaction  in  the  real  world  is 
two  people  trying  to  go  through  a doorway  that  can  only  fit  one  person  at  a time. 

A synchronous  cooperative  interaction,  on  the  other  hand,  is  one  in  which  two  or  more  users  interact 
on  the  same  task  in  real-time.  Phone  calls  and  text  chat  are  examples  of  synchronous  cooperative  interactions. 
Often,  the  users  in  a text  chat  session  take  turns  in  writing  (the  reader  will  wait  for  the  writer  to  complete  a 
thought  before  responding),  but  they  may  also  type  at  the  same  time.  In  this  case  a chat  writer  is  not 
guaranteed  that  the  other  user(s)  are  focused  on  the  current  thread  of  conversation,  but  they  are  working 
simultaneously.  A required  simultaneous  interaction  is  one  in  which  the  users  must  manipulate  an  object  at 
the  same  time  in  order  to  make  any  progress  on  the  activity.  An  example  of  this  is  two  people  attempting  to  lift 


o 

ERIC 


G24 


a heavy  piece  of  furniture,  such  as  a piano.  An  encouraged  simultaneous  interaction  is  one  in  which  the  users 
do  not  have  to  manipulate  objects  at  the  same  time,  but  doing  so  will  allow  them  to  complete  the  task  more 
easily  or  in  a shorter  amount  of  time.  Furthermore,  their  progress  may  be  hampered  (slower)  if  the  interactions 
are  not  simultaneous.  Moving  a heavy  object  where  no  lifting  is  required  is  an  example  of  this  type  of 
interaction. 

User  Interface  Issues  for  CCOs 

A number  of  interface  issues  must  be  taken  into  consideration  when  writing  software  to  support  two 
or  more  people  working  in  a synchronous  collaborative  situation.  If  users  are  to  manipulate  a single  object 
simultaneously,  each  user  must  have  access  to  her  or  his  own  device  to  input  their  modifications.  It  is  possible, 
if  the  users  are  co-located  and  using  a single  display,  to  use  an  off-the-shelf  computer  system  with  one  mouse 
and  one  keyboard.  However,  this  typically  causes  frustration  because  of  either  an  unintuitive  interface  or  a 
contention  for  the  mouse  or  keyboard.  Obvious  solutions  include  having  the  users  interact  over  a network,  or  to 
support  multiple  input  devices  on  a single  machine,  similar  to  Multiln  (Bricker98), 

Apart  from  the  hardware,  the  software  should  indicate  objects  each  user  may  access.  Also,  some 
indication  should  be  given  to  all  users  as  to  which  user  currently  has  possession  or  is  controlling  each  object  in 
the  shared  space.  Color  usually  affords  a reasonable  sign  of  object  ownership,  particularly  if  the  color  matches 
that  of  the  owner’s  cursor. 


The  Colt  Software  Toolkit 


There  are  many  toolkits  designed  to  support  computer-based  collaboration.  These  toolkits  facilitate 
the  development  of  cooperative  software  by  providing  abstractions  for  more  difficult  concepts  in  programming 
groupware  applications,  such  as  how  to  maintain  synchronization  of  the  objects  or  views  of  objects,  and  users 
dynamically  joining  and  leaving  sessions.  Most  of  the  existing  toolkits,  such  as  Rendevous  (Hill93),  GroupKit 
(Gutwin95),  and  Habanero  (NCSA98)  support  synchronous  cooperation  over  a network.  There  are  also  a 
number  of  collaborative  systems  that  encourage  communication  at  a distance  through  common  objects 
(Gutwin95),  tasks  (Hill93)  or  physical  situations  (Bier91).  Groupkit,  in  particular,  uses  indicators  such  as 
telepointers  and  other  ’’awareness  widgets”  to  indicate  to  others  (working  at  distant  locations)  where  a user  is 
working  on  the  screen.  These  systems  permit  cooperation,  but  the  activities  in  which  they  are  used  do  not 
require  the  users  to  work  on  a task  simultaneously  or  to  communicate  frequently.  Furthermore,  none  of  these 
systems  support  co-present  interactions,. 

Colt  was  designed  to  facilitate  the  rapid  prototyping  of  applications  in  which  users  need  to  work 
simultaneously  and  interact  frequently,  Colt’s  toolkit  includes  the  Collaborative  Object-based  Application 
Program  Interface  (CO-API)  hierarchy  of  Cooperatively  Controlled  Objects  and  the  Coimage  application  shell 
with  support  for  various  multiple-user  input  solutions.  The  overall  architecture  for  the  Colt  system  is  show  in 
Figure  1. 
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Figure  1.  The  Colt  toolkit  system  architecture. 
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As  input  is  delivered  to  the  Coimage  shell  window,  it  is  diverted  to  the  Input  Translation  subsystem 
(ITS).  This  subsystem  translates  the  input-device  dependent  event  messages  into  input-device-independent 
Colt  event  messages.  Colt  messages  include  information  about  position,  button  state,  and  the  user  associated 
with  the  message.  The  activities  implemented  in  the  Coimage  shell  respond  to  the  device-independent 
messages.  The  movement  and  location  of  each  device  is  mapped  to  a separate  color-coded  cursor  on  the  screen. 
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Based  on  the  location  and  the  owner  of  a cursor,  the  activity  software  can  determine  which  objects  to 
manipulate  and  if  that  user  may  manipulate  that  object,  respectively.  In  particular,  if  a user  attempts  to 
manipulate  an  object  he  does  not  own  the  object  will  not  allow  that  user  to  control  the  object,  and  the  attempt 
to  control  the  object  is  ignored.  The  CCOs  in  the  CO-API  do  not  respond  directly  to  the  input  events,  they  only 
respond  to  changes  to  object  properties. 

The  CO-API 

The  CO-API  is  comprised  of  a number  parts:  a hierarchy  of  cooperatively  controlled  objects 
(including  object  necessary  for  specifying  access  and  recording  a history  of  events)  event,  a constraint  system, 
and  tools  for  viewing  and  transforming  bitmapped  images. 

Cooperatively  controlled  objects  are  implemented  in  our  software  environment  as  reusable  C++ 
classes,  and  they  thereby  can  inherit  state  and  behavior.  CCOs  derive  the  ability  for  sharing  from  access 
control  lists  similar  to  (Shen92).  Other  properties  of  a CCO  include  location,  size,  orientation,  coordinate 
system,  display  parameters,  and  color.  An  object  will  ignore  attempts  to  change  a property  by  users  who  do  not 
have  the  proper  privileges  (as  dictated  by  access  control  lists).  In  order  to  track  when  the  users  cooperatively 
control  objects,  each  object  in  the  CO-API  can  also  store  information  regarding  the  users’  changes  to  an  object, 
or  actions,  in  what  we  call  a history  of  actions.  The  information  stored  for  each  action  will  vary  depending  on 
how  the  object  is  manipulated,  but  typically  includes  the  changed  property  and  the  new  value,  which  user 
effected  the  change,  when  the  change  occurred  and  how  long  it  took.  The  history  of  actions  may  be  saved  and 
analyzed  after  the  activity  is  completed.  This  information  may  be  restored  at  a later  time  for  evaluation 
purposes.  The  interface  for  an  activity  can  be  designed  to  include  visualizations  that  aid  the  analysis  of 
measured  quantities  such  as  the  number  of  interactions  on  an  object,  the  total  time  of  each  interaction,  how  the 
interactions  overlapped,  or  the  order  in  which  the  users  interacted. 

In  addition  to  the  object  hierarchy,  the  Colt  system  includes  a Constraint  Manager  that  keeps  a list  of 
constraints  currently  used  in  the  Colt  system.  Each  constraint  contains  a list  of  input  objects,  or  objects  that 
may  affect  other  objects  in  the  system,  and  a list  of  output  objects,  or  objects  which  are  affected  by  changes  to 
other  objects.  At  this  time,  only  one-way  functional  constraints  are  handled. 

The  Colmage  Shell 

We  support  development  of  collaborative  activities  using  CCOs  by  providing  an  application  shell  that 
handles  the  operations,  such  as  handling  input  from  toolbars,  and  saving  and  restoring  files,  which  are  shared 
by  all  activities  built  in  the  environment. 

As  in  (Bricker95),  the  activities  supported  by  Colt  allow  each  user  to  own  an  input  device  and 
corresponding  colored  cursor  on  the  screen.  The  color  of  the  cursor  identifies  its  user.  Objects  are  displayed  in 
a user’s  color  if  only  they  are  permitted  to  manipulate  that  object.  For  example,  a tool  on  the  toolbar  is  outlined 
with  the  user’s  color  if  they  are  permitted  to  use  that  tool.  If  two  users  may  select  a tool,  it  is  outlined  with  both 
of  the  users’  colors.  The  design  and  implementation  of  an  activity  defines  how  the  tools  in  the  toolbar  are  used. 
Some  tools  simply  set  the  value  of  a property  for  an  object,  while  others  may  change  a mode  for  the  whole 
activity.  Depending  on  the  design  of  the  activity,  a user’s  cursor  may  change  shape  to  indicate  that  a user  just 
selected  an  instance  of  a particular  tool. 

The  Coimage  shell  also  supports  saving  and  restoring  data  in  a number  of  formats.  The  shell  includes 
methods  that  save  the  state  of  an  activity  as  a binary  data  file  so  that  a user  can  continue  at  a later  date.  The 
shell  also  includes  separate  methods  for  saving  a history  file  in  a tab  delimited  text  format.  This  format 
permits  an  experimenter  to  visually  read  the  data  or  analyze  it  using  another  program  such  as  a spreadsheet.  A 
copy  of  the  data  is  also  written  in  a binary  Coimage  data  file  format  whenever  the  text  version  is  written  to 
disk.  The  Coimage  shell  can  also  save  image  data  in  the  standard  Microsoft  Windows  bitmap  format,  and  in  a 
format  that  can  be  read  by  a World  Wide  Web  (WWW)  browser. 

The  Coimage  shell  also  includes  methods  that  read  and  write  a text-based  problem  description  file.  A 
problem  description  includes  an  initial  state,  goal  states  or  criterion  the  users  are  expected  to  reach,  any 
constraints  enforced  on  the  users  in  reaching  that  goal,  and  a possible  scoring  criteria.  The  current 
implementation  of  the  shell  contains  methods  that  can  be  overridden  by  an  activity  developer  to  read  and  write 
problem  description  files.  Most  activities  implemented  with  Colt  to  date  support  a problem  file  that  contains 
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the  number  of  users,  the  initial  state  of  the  program,  a text  description  of  the  goal  state(s)  that  can  be  presented 
to  the  user,  and  the  scoring  criterion. 

The  Input  Translation  Subsystem 

The  input  translation  subsystem  (ITS)  was  developed  to  abstract  away  the  details  of  any  specific  input 
device.  Device  dependent  input  events  delivered  to  the  Coimage  shell  are  diverted  to  the  ITS,  where  they  are 
translated  into  input-device  independent  messages.  These  Colt  messages  include  information  about  the  mouse 
position,  the  state  of  the  mouse  buttons,  and  the  user  associated  with  the  message.  Thus,  the  application 
developer  only  needs  to  respond  to  the  one  set  of  messages. 

The  system  requirements  for  the  toolkit  are  Pentium  based  PC  platforms  running  Microsoft  Windows 
3.1  or  Windows  95.  For  the  co-present,  single  display  situation,  we  currently  support  the  Access.Bus  multiple 
input  system  from  Computer  Access  Technology  Corporation  for  Windows  3.1  and  Microsoft  Sidewinder^'^ 
Game  Pads  for  Windows  95.  The  Coimage  Shell  supports  distance  collaboration  with  Microsoft  NetMeeting™ 
conferencing  technology.  NetMeeting  supports  audio,  video,  file  and  data  transfer,  as  well  as  a rudimentary 
sharing  (turn-taking)  mechanism  for  single-user  applications  that  are  not  designed  to  be  manipulated 
simultaneously. 

Activities  that  employ  cooperative  control 

We  have  developed  a number  of  activities  that  employ  CCOs  using  the  Colt  system  that  include  a 
Collaborative  "Drawing"  Activity,  based  on  the  Etch-a-Sketch  toy  developed  by  Ohio  Arts,  a collaborative 
puzzle  activity,  a coordination  game  based  on  a game  where  two  people  each  use  a chopstick  to  pick  up  a 
"bean,"  and  a color  matching  activity.  These  first  three  of  these  activities  are  described  in  detail  in  (Baker97 
and  Bricker98). 

The  Color  Matcher  activity,  as  described  in  (Bricker95),  was  originally  implemented  in  Microsoft’s 
Visual  Basic  with  the  Multiln  system  to  support  the  Access.Bus  multiple  input  devices.  Multiln  can  only  be 
used  under  Windows  3.1,  and  we  wished  to  conduct  a user  study  on  a version  of  this  activity  under  Windows 
95  (for  more  information  on  the  user  study,  see  (Bricker98)).  The  original  version  of  the  Color  Matcher 
activity  took  an  undergraduate  student  approximately  ten  weeks  to  implement.  The  re-implementation  of  the 
activity  using  Colt  took  the  author  only  a few  days.  This  version  included  an  interface  that  mirrored  the 
original  version  as  well  as  a new  CCO  method  for  selecting  the  users’  color,  shown  in  Figure  2.  In  this 
activity,  the  users’  color  is  set  according  to  the  color  of  a selected  pixel  in  a bitmapped  image  (e.g.,  a palette). 
The  pixel  location  is  set  by  the  location  of  the  centroid  of  a triangle.  Each  user  controls  the  vertex  of  the 
triangle  that  is  colored  to  correspond  to  the  color  of  their  cursor.  Although  it  would  be  impractical  to  have  a 
bitmapped  image  of  a full  24  bit  color  palette,  the  Windows  256  color  palette  affords  a reasonable 
approximation  for  the  purposes  of  the  activity.  The  users’  color  in  the  Colt/CCO  version  of  the  activity  is 
scored  in  the  same  way  as  the  original  application  - based  on  the  distance  from  the  target  color  in  RGB  color 
space. 
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Figure  2.  The  Color  Matcher  activity  user  interface. 
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Conclusions  and  Future  Work 


We  have  presented  the  toolkit  part  of  the  Colt  System,  as  well  as  a collaborative  activity  that  was  built 
using  it.  The  software  toolkit  includes  Cooperatively  Controlled  Objects,  which  generalize  fine-grained  shared 
objects  in  being  manipulated  by  more  complicated,  higher  degree-of-freedom  methods. . 

The  Colt  system  was  tested  in  the  development  of  the  drawing,  jigsaw,  and  chopstick  activities. 
Further  testing  needs  to  be  done  with  developers  to  improve  the  system.  There  are  many  enhancements  we’d 
like  to  add  to  the  software  toolkit,  including  adding’ support  for  other  types  of  multiple  input  devices,  such  as 
via  the  Universal  Serial  Bus,  and  support  for  multi-way  constraints.  The  CCOs  designed  to  date  can  also  be 
ported  to  other  platforms,  such  as  Java,  support  distance  collaboration  in  WWW  based  activities.  We  also  wish 
to  continue  to  expand  the  CCOs  in  CO-API  to  support  other  types  of  collaborative  interactions  and  activities. 
Colt  could  be  used  to  implement  collaborative  versions  of  popular  games  such  as:  Tetris™,  where  one  person 
controls  side-to-side  motion  while  the  other  controls  rotation.  Additionally,  CCOs  could  be  used  in  3D  virtual 
worlds,  or  first-person  perspective  games  such  as  Doom™.  An  example  of  a CCO  in  such  a game  is  a 
cooperatively  controlled  latch  on  a door  where  users  may  be  required,  encouraged,  discouraged  or  forbidden 
from  manipulating  the  latch  at  the  same  time. 
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Abstract:  We  examined  the  externalized  knowledge  structures  elicited  from  subjects  who  used 
the  multimedia  program  BrainStorm:  An  Interactive  Neuroanatomy  Atlas  under  two  contexts.  In 
the  problem-solving  context,  student  pairs  were  asked  to  solve  a clinical  case  resulting  from 
cranial  nerve  injury.  In  the  second,  information-gathering  context,  student  pairs  were  given 
traditional  questions  to  answer  about  these  cranial  nerves.  We  assessed  student  knowledge  using 
individual  pre-  and  post-tests,  and  concept-based  Pathfinder  Networks  (PFNETS),  and  surveyed 
students’  opinions  of  the  contexts.  Although  the  survey  seemed  to  indicate  a preference  for  the 
problem-based  context,  the  preference  was  not  universal,  and  only  marginally  significant.  The 
information  gathering  context  yielded  higher  gain  scores,  and  generated  PFNETS  more  similar 
to  the  instructors’  relative  to  the  results  from  the  problem-solving  context.  In  addition,  students 
who  were  in  pairs  with  at  least  one  graduate  student  were  more  likely  to  yield  higher  post-test 
scores  and  PFNETS  that  resembled  the  instructors’  networks. 


The  Role  of  Technology  for  Problem-Based  Learning 

Computer  technology  may  prove  to  be  an  important  ally  in  support  of  PBL,  and  may  help  to  alleviate 
several  of  the  shortcomings  associated  with  it.  By  providing  an  authoritative  source  that  is  readily  available 
(theoretically)  at  any  time,  it  allows  an  instructor  to  provide  more  one-on-one  attention  to  those  students  who  require 
it,  thus  potentially  lowering  the  cost  of  the  time-intensive  aspects  of  PBL.  It  may  also  be  used  to  alleviate  the 
occasional  problems  regarding  structured  feedback  that  are  sometimes  associated  with  PBL;  this  readily  available 
source  of  information  might  reduce  the  fhistration  that  some  students  feel  when  trying  to  solve  problems  in  a PBL 
environment.  In  addition,  computer  programs,  to  varying  degrees,  can  be  used  independently  or  collaboratively,  both 
in  and  out  of  classrooms.  Both  Linn  (1992)  and  Okada  and  Simon  (1997)  recommend  the  use  of  student  pairs  as  an 
effective  alternative  to  group  learning  that  incorporates  the  advantages  of  group  learning,  while  minimizing  its 
disadvantages.  The  latter  reported  that  pairs  of  subjects  were  more  successful  than  individuals  while  interacting  with 
a computer  simulated  genetics  laboratory  during  a discovery  activity.  White  and  Frederiksen  (1998)  report  that 
collaborating  with  a higher-achieving  partner  while  engaging  in  reflective  activities  may  significantly  improve  the 
performance  of  lower  achieving  students.  Just  this  type  of  effective  combination  can  occur  in  heterogenous  pairs  of 
students  engaged  in  problem  based  learning. 

Very  few,  if  any,  studies  have  investigated  how  multimedia  use  in  a problem-based  format  may  differ  from 
its  use  in  the  service  of  more  traditional  school  tasks— such  as  information  gathering— with  regard  to  measures  of 
problem  solving  ability,  domain  knowledge,  and  knowledge  organization,  etc.  The  paucity  of  research  in  this  area,  in 
part,  motivated  the  present  study. 

The  Utility  of  Pathfinder  Associative  Networks 

One  of  the  characteristics  of  PBL  that  accounts  for  the  advantages  attributed  to  it  may  be  the  way  resultant 
knowledge  is  organized  (Norman  & Schmidt,  1992).  This  conjecture  has  support  from  Durso,  Rea,  and  Dayton 
(1994),  who  used  Pathfinder  Associative  Networks  (“PFNETS”;  Schvaneveldt,  1990)  to  measure  the  knowledge 
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organizations  of  subjects  solving  an  insight  problem,  relative  to  those  who  were  given  the  information  as  a story 
(rather  than  as  a problem  to  solve).  Pathfinder  networks  reflect  general  conceptual  proximities  that  are  empirically 
derived  from  subjects’  ratings  of  the  relatedness  of  pairs  of  terms.  Each  network  representation  (PFNET)  yields  a 
“concept  map”  of  a subject’s  knowledge  organization  (e.g..  Figure  1).  Durso,  et  al.  (1994)  report  that  people  who 
solved  an  insight  problem  had  a significantly  different  knowledge  organization,  as  measured  by  PFNETS,  than  did 
those  who  (a)  did  not  solve  it  or  (b)  were  presented  with  the  information  in  a non-problem  format. 


Figure  1.  A student’s  PFNET  for  Exercise  1. 

Additionally,  Goldsmith  and  Johnson  (1990)  used  PFNETS  to  assess  the  knowledge  structures  of  students 
over  the  course  of  learning  in  a conventional  statistics  classroom-and  to  compare  the  students’  structures  with  those 
of  the  instructors,  over  time.  They  found  that  the  knowledge  representations  as  generated  by  Pathfinder  offered  a 
useful  assessment  of  classroom  learning  and  that  the  correlation  between  a student’s  structure  and  an  instructor’s 
was  a good  index  of  how  much  the  student  learned  about  the  domain  of  study. 


Results 

Pre-  and  Post-Tests 

Paired  sample  t-tests  revealed  significant  score  differences  between  the  set  of  corresponding  pre-test  and 
post-test  items  for  all  combinations  of  exercises  and  context  (p's  < .005,  see  Table  1).  The  difference  in  the  means 
on  the  (full)  post-tests  between  the  two  exercises  indicates  Exercise  2 was  more  difficult,  in  terms  of  proportion 
correct  (Exercise  1 = .74;  Exercise  2 = 54;  p<  .001),  but  the  context  in  which  students  learned  the  material  had  no 
effect  on  these  post-test  scores  (PS=.62;  IG  = .66).  However,  the  information  gathering  context  yielded  higher  gain 
scores  than  the  problem  solving  context  when  both  exercises  are  combined  (PS  gain  = .15;  IG  gain  = .23;  p=.06; 
marginally  significant).  Additionally,  analyses  of  the  post-test  item  scores  by  type  of  question  (recall,  analysis,  or 
integration)  or  topic  (anatomy,  function,  or  symptoms)  revealed  no  significant  context-based  differences  in  score. 
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Exercise 

Context 

Pre- 

Test 

Mean 

Post-Test 

Items 

Mean 

Mean 

Difference 

Gain 

SD 

Paired  T- 
test 

p value 

1 

PS 

0.68 

0.83 

0.15 

0.13 

3.854 

.003 

IG 

0.67 

0.89 

0.22 

0.12 

6.000 

.000 

2 

PS 

0.41 

0.57 

0.16 

0.16 

4.528 

.001 

IG 

0.42 

0.67 

0.25 

0.25 

4.672 

.001 

Overall 

0.55 

0.74 

0.19 

0.14 

9.048 

.000 

Table  1.  Corresponding  Pre-Test  versus  Post-Test  Item  Performance  by  Problem  Solving  (PS)  and  Information 
Gathering  (IG)  Contexts,  as  well  as  by  Exercise. 


Exercise 

PFNET 

PS  Mean 

IG  Mean 

F 

Significance 

1 

Instructor  1 

.4521 

.4629 

0.038 

.847 

Instructor  2 

.4765 

.6461 

3.106 

.094 

Instructor  3 

.3624 

.5029 

5.053 

.037 

Instructor  4 

.4777* 

.4775 

0.000 

.997 

Instructor  5 

.4533 

.6108 

4.189 

.055 

2 

Instructor  1 

.4375 

.4863 

0.588 

.452 

Instructor  2 

.4238* 

.3976 

0.464 

.504 

Instructor  3 

.4219 

.4707 

0.688 

.416 

Instructor  4 

.4520 

.5072 

0.877 

.360 

Instructor  5 

.4181 

.4684 

0.667 

.424 

Wilcoxon  Signed  Rank  Test 

*1  higher/1  tie 

8 higher 

Z=-2.395 

.017 

Table  2.  Correlations  of  Student  and  Instructor  PFNETS  by  Problem  Solving  and  Information  Gathering  Contexts 
Pathfinder  Networks 

Student  pathfinder  networks  (PFNETS)  were  compared  to  instructors’  PFNETS  on  both  individual-instructor 
and  a combined-instructor  (averaged)  bases.  A significant  difference  among  the  correlations  and  instructional 
contexts  was  found  for  Exercise  1 regarding  the  major  instructor  for  the  course  (Instructor  3 in  Table  2),  In  this 
case,  students  who  learned  the  material  in  the  information  gathering  context  produced  PFNETS  that  yielded  higher 
correlations  with  that  instructor’s  PFNET  than  those  learning  in  the  problem  solving  context  (.50  vs.  .36;  p<M5). 
Two  additional  experts  (Instructors  2 and  4 in  Table  2)  yielded  a marginally  significant  result  for  Exercise  1 
(p=094  and  .055).  More  generally,  a Wilcoxon  Signed  Ranks  Test  showed  that  overall,  the  information  gathering 
PFNETS  correlated  more  highly  with  instructor  PFNETS  than  did  the  problem-solving  PFNETS  (Z=  -2.395; 
p=.0}7).  Comparisons  among  the  instructors’  PFNETS  revealed  significant  correlations  for  every  comparison  (r's 
ranging  from  .40  to  .93;  p <0.01 ).  In  addition,  we  observed  a significant  meta-correlation  between  students’  post- 
test scores  and  their  PFNET  correlations  with  the  instructors.  For  both  exercises,  students  who  scored  higher  on 
the  post-test  had  generally  significantly  higher  PFNET  correlations  with  the  instructors’  PFNETS  {meta-r’s  ranged 
from  .33  to  .85;  see  Table  3). 


Exercise 

Instructor 

1 

2 

3 

4 

5 

Combined 

1 

Post-Test 

.77* 

.59* 

.83* 

.68* 

.85* 

.83* 

2 

Post-Test 

.61* 

.33 

.50* 

.54* 

.58* 

.55* 

Table  3.  Pearson  (Meta-)Correlations  of  Post-test  Performance  with  Student/Instructor  PFNETS  Correlations 
(^significant  at  the  .001  level;  1-tailed). 

Pair  Composition 

Across  both  exercises,  pair  composition  had  significant  effects  on  student  performance,  both  on  the  post-test 
score  proportions  and  on  student/instructor  PFNET  correlations  (see  Table  4).  Although  there  were  no 
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differences  in  the  pre-test  scores  of  undergraduates  based  on  pair  membership,  those  who  were  paired  with 
graduate  students  performed  better  on  the  outcome  measures  (i.e.,  both  gain-scores  and  PFNET  correlations) 
than  their  counterparts  who  were  paired  with  fellow  undergraduates  (see  Table  5).  In  fact,  graduate  students  in 
mixed  pairs  also  benefitted  more  from  the  mixing  when  comparing  their  PFNET  correlations  with  the 
instructors’  relative  to  those  in  graduate-graduate  pairs  (r=.60  vs  .50; p < .05). 


Exercise 

Measure 

Undergrads 

Grads 

Mixed 

F 

Significance 

1 

Post-Test  Score 

.64 

.76 

.79 

4.850 

.020 

PFNETS  Correlation 

.32 

.54 

.60 

13.634 

.000 

2 

Post-Test  Score 

.44 

.59 

.58 

3.988 

.036 

PFNET  Correlation 

.34 

.49 

.45 

5.737 

.011 

Overall 

Post-Test  Score 

.54 

.68 

.69 

4.526 

.017 

PFNET  Correlation 

.32 

.49 

.57 

18.002 

.000 

Table  4.  Differences  in  Pair  Performances  based  on  Composition.  “Score”  indicates  mean  proportion  of  correct 
answers,  and  “Correlation”  indicates  comparison  with  experts’  combined  PFNETS. 


Measure 

Homogenou 
s Pairs 

Mixed 

Pairs 

F 

Significance 

Undergrads 

Pre-Test  Score 

.48 

.49 

.002 

.969 

Gain-Score 

.14 

.30 

4.742 

.045 

PFNET  Correlation 

.33 

.53 

16.402 

.001 

Graduates 

Pre-Test  Score 

.59 

.59 

.003 

.956 

Gain-Score 

.19 

.20 

.044 

.836 

PFNET  Correlation 

.50 

.60 

4.437 

.046 

Table  5.  Graduate  and  Undergraduate  Performance  Based  on  Pair  Membership.  “Score”  indicates  mean 
proportion  of  correct  answers,  and  “Correlation”  indicates  comparison  with  experts’  combined  PFNETS. 

Survey/Questionnaire 

On  the  questionnaire  that  elicited  student  opinions  about  the  relative  merits  of  each  of  the  tasks,  students 
agreed  that  both  methods  (1)  helped  in  comprehending  and  remembering  the  material,  (2)  allowed  for  success  in 
accomplishing  the  task,  and  (3)  should  be  incorporated  as  regular  course  activities.  Fourteen  of  the  26  items 
yielded  statistically  significant  findings.  Students  were  more  likely  to  agree  that  problem-solving,  rather  than 
information  gathering,  a)  contributed  to  the  integration  of  the  material,  b)  promoted  reflection,  c)  provided  insight 
into  applications,  d)  supported  collaboration,  e)  required  the  use  of  prior  knowledge,  f)  piqued  curiosity,  g) 
encouraged  deep  understanding,  h)  provided  contextual  meaning,  i)  helped  to  better  understand  the  lab,  j) 
prompted  additional  questions,  k)  provided  relevance,  and  1)  was  a meaningful  activity  (all p's  < .05).  Conversely, 
students  were  more  likely  to  agree  that  information  gathering,  rather  than  problem-solving  encouraged  both  rote 
memorization  and  attention  to  detail  {p's  < .05).  However,  while  students  seemed  to  prefer,  in  general,  the 
problem-solving  method,  this  finding  was  only  marginally  significant  on  a five  point  scale  (in  which  5=  “strongly 
agree”  and  1=  “strongly  disagree”:  PS  mean  =3.55,  IG  mean=2.77;  p=.108). 

With  regard  to  their  opinions  about  the  multimedia  program,  63%  of  students  disagreed  or  strongly  disagreed- 
while  only  one  student  agreed-with  the  statement,  “Learning  this  material  could  have  just  as  easily  been  done  by 
using  a textbook.”  Indeed,  82%  agreed  or  strongly  agreed  (with  no  one  in  disagreement)  with  the  statement,  “The 
program  provided  support  that  would  be  difficult  to  obtain  through  other  methods.” 

On  the  survey  questions  that  elicited  their  attitudes  about  working  on  each  of  the  tasks,  students  were  more 
likely  to  agree  that  they  enjoyed  the  problem  assignment  (relative  to  the  IG  assignment),  and  were  interested, 
motivated,  and  involved  in  the  problem.  Still,  although  79%  of  respondents  disagreed  or  strongly  disagreed  with 
the  statement,  “The  questions  were  too  difficult  to  answer  using  the  program,”  significantly  fewer  (59%)  disagreed 
that  “the  problem  was  too  difficult  to  answer  using  the  program”  (5  point  scale:  IG  = 2.23,  PS  = 2.59,  t-  -2.16,  p < 
.05).  The  opinion  that  the  problem  was  too  difficult  to  solve  was  modulated  by  the  particular  exercise;  students 
who  received  the  problem  as  their  second  exercise  were  more  likely  to  view  it  as  difficult  (El  =2.4  vs  E2=3.0; 
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p=.024).  Students  overwhelmingly  agreed  or  strongly  agreed  that  solving  case  problems  encourages  the 
integration  of  concepts  (96%),  whereas  only  27%  agreed  that  answering  direct  questions  does  so  (t=4.27J, 
p<Ml). 


References 

Albanese,  M.  A.  & Mitchell,  S.  (1993).  Problem-based  learning:  A review  of  literature  on  its  outcomes  and 
implementation  issues.  Academic  Medicine,  68,  52-8 1 . 

Coppa,  G.  & Tancred,  E.  (1995).  BrainStorm:  An  Interactive  Neuroanatomy  Atlas  [Computer  Program]. 
Developed  by  SUMMIT  Technology  Group  at  Stanford  University.  St.  Louis,  MO:  Mosby 

Durso,  F.  T.,  Rea,  C.  B.,  & Dayton,  T.  (1994).  Graph-theoretic  confirmation  of  restructuring  during  insight 
Psychological  Science,  5,  94-98. 

Goldsmith,  T.  E.,  & Johnson,  P.  J.  (1990).  A structural  assessment  of  classroom  learning.  In  R.W.  Schvaneveldt, 
(Ed.),  Pathfinder  Networks:  Studies  in  Knowledge  Organization  (pp.  315-322),  Norwood,  NJ:  Ablex  Publishing 
Corp.. 

Linn,  M.  C.  (1992).  The  Computer  as  learning  partner:  Can  computer  tools  teach  science?  In  K.  Shiengold,  L.  G. 
Roberts,  & S.  M.  Malcolm  SM.  (Eds.),  This  Year  in  Science  1991:  Technology  for  Teaching  and  Learning  (pp.  31- 
69).  Washington,  DC:  American.  Association  for  the  Advancement  of  Science. 

Nii,  L.  J.,  & Chin,  A.  (1996).  Comparative  trial  of  problem  based  learning  versus  didactic  lectures  on  clerkship 
performance.  American  Journal  of  Pharmaceutical  Education,  60,  162-164. 

Norman  G.  R.  & Schmidt,  H.  G.  (1992).  The  psychological  basis  of  problem-based  learning:  A review  of  the 
Qv'idQncQ.  Academic  Medicine,  67,  557-65. 

Okada,  T.,  & Simon,  H.A.  (1997).  Collaborative  discovery  in  a scientific  domain.  Cognitive  Science,  21,  109-146. 

Schvaneveldt,  R.  W.,  Editor  (1990).  Pathfinder  Associative  Networks:  Studies  in  Knowledge  Organization.  Ablex 
Publishing  Corp;  Norwood,  NJ.  Series  title:  Ablex  series  in  computational  sciences. 

Solomon,  P.,  & Finch,  E.  (1998).  A qualitative  study  identifying  stressors  associated  with  adapting  to  problem- 
based  learning.  Teaching  and  Learning  in  Medicine,  10,  58-64. 

Weinert,  F.  E.,  & Helmke,  A.  (1995).  Learning  from  wise  Mother  Nature  or  Big  Brother  instructor:  The  wrong 
choice  as  seen  from  an  educational  perspective.  Educational  Psychologist,  30,  135-142. 

White,  B.Y.,  Frederiksen,  J.R.  (1998).  Inquiry,  modeling,  and  metacognition:  Making  science  accessible  to  all 
students.  Cognition  and  Instruction,  16,  3-1 18. 

Acknowledgments 

The  authors  thank  the  following  for  their  valuable  comments  in  the  preparation  of  this  manuscript:  Bernard  Gifford,  Arthur 
Shimamura,  Michelle  Million,  and  members  of  the  Reasoning  Research  Group  of  the  University  of  California,  Berkeley. 


633 


The  Crucial  Roles  of  the  Instructional  Designer  and  the  Subject  Matter 

Expert  in  Multimedia  Design 


Mike  Keppell 

Biomedical  Multimedia  Unit,  Faculty  of  Medicine,  Dentistry  and  Health  Sciences 
The  University  of  Melbourne 
Australia 

m.keppell@medicine. unimelb.edu.au 


Abstract:  This  paper  examines  a study  which  developed  and  began  testing  a process  to  assist 
instructional  designers  in  eliciting  unfamiliar  content  from  subject  matter  experts  and  in 
conceptualizing  that  content.  It  explored  the  development,  trial,  training  and  application  of  the 
“Content  Production  Process”  (CPP).  Three  major  insights  were  gained  into  the  use  of  the  CPP. 

Firstly,  the  novice  instructional  designer  found  that  knowledge  map  construction  was  useful  for 
assisting  him  to  think  carefully  about  the  unfamiliar  content.  Secondly,  the  knowledge  map 
appears  to  have  been  a useful  ’’communication  prop”  for  assisting  the  instructional  designer  in 
interacting  with  the  subject  matter  experts.  Thirdly,  the  instructional  designer  found  that 
combining  a “teachback”  procedure  with  the  knowledge  map  provided  a powerful  means  of 
checking  his  understanding  of  the  content  against  that  of  the  expert.  The  CPP  offers  a process 
for  improving  the  interviewing  process  in  multimedia  instructional  design. 

Nature  of  the  Problem 

Imagine  attempting  to  design  an  instructional  unit  in  an  area  in  which  you  have  no  understanding  or 
expertise.  How  would  you  approach  the  unfamiliar  materials?  What  are  your  choices?  You  can  read  textbooks  or 
reference  materials  or  you  could  talk  to  experts  in  the  area.  Subject  matter  experts,  however,  are  often  busy 
people.  How  can  you  optimize  your  time  spent  with  the  subject  matter  expert?  How  can  you  elicit  a sufficient 
quantity  and  quality  of  information  so  that  instructional  design  is  possible?  What  questions  would  you  ask  of  the 
subject  matter  expert?  This  paper  addresses  the  relationship  between  the  instructional  designer  and  subject  matter 
expert  and  examines  more  particularly  a possible  method  for  eliciting  and  conceptualizing  unfamiliar  content 
knowledge  from  the  subject  matter  expert  so  that  efficient  and  effective  instructional  design  can  proceed. 

Research  on  instructional  design  has  not  adequately  addressed  the  conceptualization  by  the  instructional 
designer  of  unfamiliar  content  as  presented  by  a subject  matter  expert.  Many  researchers  in  the  field  of 
instructional  design  acknowledge  the  importance  of  this  conceptualization  (Brien  & Towle,  1977;  Bratton,  1981, 
1983;  Cram,  1981;  Wallington,  1981;  Rutt,  1985;  Armstrong  & Sherman,  1988;  Morrison,  1988a;  Tessmer, 
1988a,  1988b;  Rodriguez,  Stephens  & Arena,  1991;  Davidove,  1993;  Maple,  1994;  Lee,  1994;  Ingram,  Heitz, 
Reid,  Walsh  & Wells,  1994;  Yancey,  1996);  however,  only  general  heuristics  and  suggestions  have  been  offered. 
There  appears  to  have  been  no  systematic  efforts  to  develop  a comprehensive  strategy  for  the  conceptualization 
and  elicitation  of  subject  matter  knowledge. 

This  crucial  skill  of  an  instructional  designer  is  a potential  ’’bottleneck"  for  the  development  of 
instructional  materials  in  project-based  multimedia  environments.  For  example,  an  instructional  designer  may  be 
required  to  design  multimedia  in  content  areas  as  diverse  as  mining,  engineering,  taxation,  alumina  processing, 
beef  production  and  medicine.  It  is  not  possible  to  be  conversant  in  all  of  these  diverse  content  areas,  and  the 
instructional  designer  must  rely  on  subject  matter  experts  to  assist  with  this  content.  Without  an  effective  strategy 
for  interacting  with  the  subject  matter  expert,  valuable  time  will  be  lost  understanding  and  organizing  the  content, 
hence  the  need  for  an  efficient  and  effective  method  to  assist  in  the  instructional  designer-subject  matter  expert 
interaction  (Keppell,  1997). 

Instructional  Designers  and  Subject  Matter  Experts 

Instructional  Designer 
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Instructional  designers  tend  to  be  process-oriented  individuals  as  they  can  apply  instructional  design 
principles  to  a wide  range  of  content  areas.  The  designer  usually  begins  by  analyzing  goals,  needs  and  learner 
characteristics  in  an  attempt  to  understand  the  instructional  problem.  The  scope  and  content  of  the  subject  are 
then  identified.  These  areas  represent  the  "problem  space"  of  the  instructional  designer.  One  of  the  designer's 
main  jobs  is  to  select,  sequence,  synthesize  and  summarize  the  content  for  instructional  purposes  and  deliver  the 
desired  product.  ‘‘Despite  having  no  content  expertise,  the  designer  has  a set  of  representations  based  on  a ** design 
nwdeV*  which  can  guide  the  development  of  an  effective  training  program”  (Nelson,  Magliano  & Sherman,  1988, 
p,  32)  (italics  added). 

The  design  model  is  the  designer's  accumulated  knowledge  of  how  instruction  should  be  developed.  It  is 
the  amalgamation  of  general  experience,  educational  background  and  instructional  experience.  The  design  model 
is  analogous  to  a script.  Scripts  are  knowledge  packages  in  memory  which  allow  individuals  to  understand 
routine  activities  (e.g.  eating  in  restaurants,  visiting  the  dentist).  They  are  prototypical  or  stereotypical 
information  useful  in  everyday  occurrences.  Schank  and  Abelson  (1977)  suggest  that  we  have  hundreds  of 
stereotypical  situations  coded  in  memory,  each  having  idiosyncratic  variations.  The  designer  has  a generic  script 
which  can  be  applied  with  variations  to  new  instructional  problems.  This  design  model  is  one  of  the  most 
important  characteristics  required  by  the  designer  in  approaching  a new  instructional  problem.  The  advantage  of 
the  generalized  script  is  that  it  is  adaptable  to  new  instructional  problems.  The  concept  of  the  script  suggests  how 
a designer  can  approach  unfamiliar  content.  The  aim  of  the  designer  in  the  interaction  with  the  subject  matter 
expert  will  be  to  "formulate  a working  content  structure  within  which  the  information  and  skills  to  be  taught  can 
be  formed  into  a sequence  and  hierarchy"  or  other  appropriate  structure  (Wallington,  1981,  p.  30). 

Subject  Matter  Expert 

As  the  name  suggests  the  subject  matter  expert  (SME)  or  the  content  expert  is  an  authority  on  a 
particular  domain  of  knowledge  from  whom  the  designer  is  attempting  to  elicit  knowledge.  The  SME  may  be  the 
client  in  the  business,  academic  or  military  setting.  Nelson,  Magliano  and  Sherman  (1988)  suggest  that  "experts'" 
knowledge  structures  are  more  highly  organized  and  well  integrated"  (p.  30)  than  those  of  novices.  “Experts  tend 
to  "chunk"  or  organize  information  into  more  highly  structured  patterns  and  to  complete  the  task  more  quickly 
than  novices.  Experts  also  appear  to  represent  problems  differently  than  novices  because  of  their  superior  ability 
to  recognize  patterns,  infer  relationships,  disregard  irrelevant  information,  and  recall  similar  problems  from  past 
experience”  (Nelson,  Magliano  & Sherman,  1988,  p.  33).  One  of  the  SME’s  functions  in  training  is  to  provide 
accurate  content  to  the  design  team.  In  this  study  SME’s  have  two  roles;  (1)  providing  a clear  description  and 
explanation  of  the  content  area  being  examined,  and  (2)  assisting  the  designer's  conceptualization  by  clarifying 
and  verifying  the  content. 


Phases  of  the  Study 

Phase  1 : Development  of  the  Process 

The  CPP  was  developed  by  using  an  eclectic  approach.  Ausubel’s  concept  of  "intellectual  scaffolding" 
forms  an  important  part  of  the  CPP  (Ausubel,  1960,  1963,  1968).  When  instructional  designers  are  working  with 
SME's  their  aim  is  to  create  a conceptual  scaffold  and  then  attach  content  elicited  in  the  subsequent  interactions 
with  the  SME.  As  designers  create  this  conceptual  scaffold  (knowledge  map)  and  elaborate  the  content  they  begin 
to  conceptualize  the  relationships  within  the  content.  It  seemed  that  some  form  of  graphic  organizer  might  be  an 
appropriate  tool  for  representing  successive  iterations  of  the  instructional  designer's  attempts  to  conceptualize 
unfamiliar  content,  and  of  the  various  forms  available  the  knowledge  map  was  selected  (Breuker,  1984; 
Dansereau,  1991a,  1991b).  In  addition  to  helping  the  instructional  designer  conceptualize  unfamiliar  content 
(Wedman,  1 987),  it  was  thought  that  there  would  emerge  a second  advantage  of  making  the  instructional 
designer’s  perceptions  public  for  the  subject  matter  expert  to  examine. 

Phase  2:  Personal  Trial  of  the  CPP 
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The  personal  trial  of  the  CPP  helped  to  crystallize  the  researcher’s  conceptualization  of  the  process. 
Initially,  it  was  necessary  to  keep  pace  with  the  SME  and  attempt  to  obtain  an  overview  of  the  content  area.  A 
second  phase  began  with  the  SME  viewing  the  map  and  understanding  the  role  of  the  instructional  designer 
within  the  interaction.  At  this  point  in  the  interaction  the  SME  appeared  to  become  more  cooperative  and 
provided  more  detailed  explanations  because  a certain  rapport  had  been  developed.  A third  phase  involved  both 
the  designer  and  SME  collaborating  to  achieve  the  goals  of  the  interaction.  The  fourth  phase  centered  on  cultural 
understanding  with  the  instructional  designer  feeling  more  effective  in  conceptualizing  the  content  when  the 
content  could  be  viewed  from  the  SME’s  point  of  view.  After  the  trial  an  effort  was  made  to  formalize  the  process 
to  allow  it  to  be  taught  to  another  instructional  designer. 

Phase  3:  Training  a Novice  Designer  to  Use  the  CPP 

The  training  program  addressed  both  the  declarative  and  procedural  knowledge  required  by  a novice 
designer  when  working  with  SME’s  in  a ’’real”  setting.  The  novice  designer  was  trained  for  a total  of  thirty-three 
hours  in  all  aspects  of  the  CPP.  Personally  teaching  the  process  to  John  provided  the  researcher  with  the 
opportunity  to  observe  the  process  in  action  and  document  the  proceedings.  John  achieved  competency  in  all 
aspects  of  the  CPP  as  it  was  conceived  at  that  point  and  was  ready  to  work  with  SME’s  in  a real  setting.  He  had 
shown  competence  in  both  declarative  and  procedural  knowledge  in  relation  to  the  CPP. 

Phase  4:  Case  Stories 

These  case  stories  investigated  the  designer-SME  interaction  within  the  real-world  context  of  a militaiy 
setting.  They  attempted  to  focus  on  discoyery,  insight  and  understanding  from  the  perspectives  of  the  designer 
and  the  SME’s  being  studied.  In  this  way  they  attempted  to  ’’illustrate  conclusions  to  which  the  author  ...  was  ... 
already  committed”  (Biddle  & Anderson,  1986,  p.  238).  Such  case  stories  are  intended  to  provide  new  insights 
and  understandings  and  ”...  not  provide  conclusions,  however,  that  reflect  evidence”  (p.  238). 

This  case  focussed  on  how  one  instructional  designer  aided  by  the  CPP  interacted  with,  elicited  and 
conceptualized  unfamiliar  content.  Through  the  process  of  the  case  it  was  possible  to  examine  the  situation  in 
depth,  detail  and  from  a holistic  perspective.  This  portrayal  provided  a solid  basis  for  examining  what  took  place 
in  the  interaction.  The  following  important  areas  emerged  during  the  case:  (1)  Strategies  for  conceptualizing  the 
content,  (2)  the  knowledge  map  as  a communication  prop,  (3)  the  teachback  function  and  the  knowledge  map. 

Strategies  for  Conceptualizing  the  Content 

Throughout  the  five  interviews  in  the  case  John  appeared  to  become  more  adaptable  and  flexible  in  his 
interactions  with  the  SME’s.  As  the  interviews  progressed,  John’s  ability  to  react  to  the  situation  was  more 
noticeable.  He  appeared  more  comfortable  with  clarifying  the  content  when  he  was  unsure  of  its  meaning.  In  his 
initial  interactions,  John  often  allowed  Steve  (first  SME)  to  complete  a lengthy  explanation  before  he  sought 
clarification.  In  later  interviews  with  Bob  (second  SME),  John  was  able  to  seek  more  regular  explanations.  As 
John  became  more  conversant  and  more  confident  with  the  interaction  his  questioning  also  became  more  specific. 

John’s  preferred  approach  to  gaining  an  initial  understanding  of  the  content  was  to  obtain  an  overview  of 
the  area.  He  appeared  to  need  to  see  the  conceptual  terrain  of  the  content  area  before  examining  the  area  in  more 
depth.  This  was  consistent  with  the  author’s  own  application  of  the  CPP.  John  commented  that  he  needed  to  do 
this  so  that  he  could  see  the  extent  of  the  information  that  needed  to  be  covered.  It  may  represent  a heuristic  that 
designers  need  to  use  in  the  process  of  interacting  with  SME’s. 

John  also  found  that  the  knowledge  map  was  a useful  tool  which  assisted  him  when  wrestling  with 
unfamiliar  content  and  displaying  the  content  to  the  SME.  John  suggested  that  the  construction  of  the  knowledge 
map  ’’forced  me  to  think  about  what  was  said”  and  ’’focuses  my  thinking”.  The  use  of  the  knowledge  map 
appeared  to  assist  John  in  thinking  more  carefully  about  the  content.  Holley  and  Dansereau  (1984,  p.  8)  suggest 
that  the  creation  of  a knowledge  map  may  force  the  constructor  to  process  the  content  in  ’’greater  semantic 
depth”.  Furthermore,  Holley  and  Dansereau  also  suggest  that  reorganizing  the  information  in  the  form  of  a 
knowledge  map  may  activate  both  the  spatial  and  verbal  processing  systems,  allowing  the  developer  to  gain  a 
deeper  understanding  of  the  information. 
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The  Knowledge  Map  as  a Communication  Prop 


The  use  of  the  knowledge  map  in  the  designer-SME  interview  may  act  as  a communication  "prop"  to 
improve  the  interpersonal  nature  of  the  interview.  The  interaction  with  the  SME  can  sometimes  be  a stressful 
experience  which  may  be  lessened  by  the  use  of  a visual  prop  in  the  form  of  the  knowledge  map.  The  knowledge 
map  appeared  to  assist  John  in  handling  the  sometimes  difficult  interpersonal  aspects  of  the  interview.  "And  it 
really  helps  me,  because  when  you're  dealing  with  someone  who  you  don't  really  know  very  well ...  I'm  nervous 
...  and  rather  than  sort  of  concentrating  and  talking  to  him  and  looking  at  him  in  the  eye,  I could  focus  attention 
on  the  map". 

The  knowledge  map  may  also  help  to  focus  the  attention  of  the  SME  and  designer  on  the  most  important 
parts  of  the  content.  Accurate  content  on  the  map  may  not  require  further  attention.  It  may  not  be  necessary  to 
further  explore  this  information  because  the  SME  has  verified  the  accuracy  of  the  content.  John  used  the  map  in 
this  sense  by  check-marking  accurate  information  on  the  map  as  it  was  examined  by  the  SME.  In  this  sense  the 
map  may  act  as  a communication  device  that  confirms  the  knowledge  shared  by  the  designer  and  SME.  The  focus 
of  the  interview  can  then  be  concerned  with  the  areas  that  are  most  in  need  of  attention. 

The  knowledge  map  appeared  to  assist  both  the  SME  and  designer  in  recognizing  gaps  in  the  flow  of  the 
sequence  and  in  the  information;  for  example,  Steve  instantly  recognized  the  absence  of  safety  procedures  in  the 
map  John  had  created  on  the  C7  rifle.  Likewise,  once  John  had  created  the  map  on  landmines,  he  himself,  was 
able  to  pinpoint  gaps  in  the  information  that  he  addressed  in  the  subsequent  iteration.  McAleese  (1988),  in  the 
context  of  developing  expert  systems,  suggests  that  experts  were  able  to  recognize  and  pinpoint  inconsistencies 
on  a concept  map. 

Lambiotte,  Dansereau,  Cross  and  Reynolds  (1989,  p.  332)  suggest  that  knowledge  maps  are 
"computationally  efficient"  in  that  they  "facilitate  faster  search  and  recognition  of  relevant  information"  (p.  332), 
Efficiency  is  defined  in  terms  of  how  the  knowledge  map  representation  assists  attention  focusing,  knowledge 
assimilation  and  knowledge  searching  of  new  information.  The  presentation  of  the  knowledge  map  to  the  SME 
may  activate  both  the  spatial  and  verbal  systems  thus  increasing  the  processing  efficiency  of  the  information. 

The  Teachback  Function  and  the  Knowledge  Map 

In  the  initial  conception  of  the  CPP  it  was  envisaged  that  the  designer  would  stop  at  certain  points 
throughout  the  interaction  and  teach  the  material  back  to  the  SME.  This  proved  far  too  ambitious  for  a designer 
who  is  totally  unfamiliar  with  a content  area.  It  was  not  possible  for  the  designer  to  gain  sufficient  familiarity 
with  the  content  to  teach  back  the  material  to  the  SME.  John  also  felt  uncomfortable  with  this  practice.  “I  still 
think  the  toughest  nut  to  crack  for  me  is  the  teachback,  I could  just  go  over  it  and  kind  of  lecture  him,  but  I really 
felt  uncomfortable  doing  that”.  Therefore  it  seems  more  reasonable  to  assume  that  the  designer  must  concentrate 
on  obtaining  the  big  picture  and  must  postpone  paraphrasing  and  asking  in-depth  questions  until  a later  point  in 
the  interaction.  A great  deal  of  reflection  and  problem-solving  was  required  to  develop  a conceptualization  of  the 
SME's  content  area. 

In  spite  of  the  original  prediction,  interspersing  the  teachback  procedure  (Gregory,  1986;  Pask,  1975)  in 
the  interviews  may  actually  interrupt  the  SME  in  the  midst  of  an  explanation  and  prove  counterproductive  to  the 
overall  goal  of  the  elicitation  process.  A SME  may  require  total  concentration  when  explaining  a complex 
sequence.  If  the  instructional  designer  interrupts  the  SME,  the  SME  may  lose  the  train  of  thought  and  be 
prevented  from  explaining  crucial  information.  The  SME  may  also  simply  become  irritated  at  being  interrupted, 
thus  affecting  the  rapport  between  the  designer  and  the  SME. 

It  would  appear  that  the  teachback  procedure  is  more  appropriately  used  at  the  beginning  of  a 
subsequent  interview.  With  the  aid  of  the  knowledge  map,  John  found  this  a powerful  means  of  checking  his 
understanding  of  the  content  against  the  expertise  of  the  SME.  John  gathered  the  information  in  the  interview, 
reflected  on  the  content,  constructed  the  map  and  then  taught  the  material  back  to  the  SME  at  the  beginning  of 
each  interview.  This  proved  an  effective  adaptation  of  the  CPP  over  what  was  originally  conceived. 

Conclusion 
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This  study  explored  the  development,  trial,  training  and  application  of  a process  to  elicit  and 
conceptualize  unfamiliar  content.  In  particular  the  study  examined  the  personal  strategies  used  by  the  designer  in 
handling  unfamiliar  content.  A secondary  purpose  was  to  examine  the  effectiveness  of  the  process  to  train 
designers  in  elicitation  and  conceptualization  processes. 

This  study  has  examined  the  ffont-end  investigation  of  eliciting  and  conceptualizing  unfamiliar  content 
by  an  instructional  designer.  The  work  is  significant  as  it  has  carefully  documented  the  interactions  of  a novice 
designer  with  SME’s  in  a real  context.  The  process  appears  promising  in  assisting  the  conceptualization  of 
unfamiliar  content  and  improving  the  designer-SME  interaction.  Further  research  needs  to  be  undertaken  with  a 
variety  of  designers  and  SME’s  to  determine  the  applicability  of  this  process  for  the  field  of  instructional  design. 

Epilogue — Personal  Note 

Since  conducting  this  study  the  researcher  has  employed  the  CPP  to  interact  with  over  forty-six  subject 
matter  experts  in  the  oil  and  gas  industry  in  Canada  from  July  1993  to  July  1994.  Content  areas  included 
cementing,  acidizing,  fracturing,  coil  tubing,  pipeline  pigging,  hydrostatic  testing,  vehicle  inspections,  nitrogen 
and  CO2.  Comments  by  the  SME's  in  relation  to  this  process  were  very  favourable.  SME's  were  often  puzzled  by 
the  accuracy  of  the  information  documented  in  the  training  manuals  by  an  individual  who  was  totally  unfamiliar 
with  the  content.  The  operational  procedures  for  twelve  oil  and  gas  operations  relied  exclusively  on  the  input  of 
SME’s.  The  training  materials  were  developed  without  actually  observing  the  operations  within  the  field  setting. 
During  1 994, 1 also  worked  with  seven  content  experts  in  therapeutic  massage.  My  role  was  to  assist  in  the 
formulation  of  the  curriculum.  From  August  1994  to  February  1998  in  Australia,  I worked  with  over  thirty 
subject  matter  experts  in  the  development  of  multimedia  training  materials  for  industry  and  university-based 
clients.  Some  of  these  content  areas  included  open  cut  coal  mining,  underground  coal  mining,  automotive 
practices,  alumina  processing  and  engineering.  This  collaborative  approach  has  proven  to  be  successful  in  the 
completion  of  the  technical  projects.  From  February  1 998  I have  been  working  with  medical  subject  matter 
experts  at  a medical  school  in  Australia.  I have  utilised  principles  from  the  CPP  to  interact  with  over  thirty 
medical  experts  in  developing  multimedia  modules  for  use  by  medical  students.  Clients  appear  to  "buy  into"  the 
process  once  they  realize  that  their  expertise  drives  the  project.  The  role  of  the  instructional  designer  is  to  filter 
and  shape  the  content  for  the  audience.  My  own  experience  in  working  with  SME’s  has  been  favorable.  In  the 
project-driven  environment  in  which  I work,  the  identification  and  interaction  with  the  SME  is  a major  factor  that 
determines  the  success  or  failure  of  the  project.  For  me  the  CPP  has  continued  to  "work"  and  provide  a valuable 
contribution  to  the  design  and  development  of  multimedia  modules. 
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Abstract:  This  paper  details  the  dimensions  of  a formative  evaluation  program  developed  to 
refine  and  improve  computer  aided  learning  (CAL)  modules  in  the  Faculty  of  Medicine,  Dentistry 
and  Health  Sciences  at  the  University  of  Melbourne.  Evaluation  criteria  were  developed  in  three 
domains  and  are  broadly  based  on  the  characteristics  of  a CAL  module.  The  program  of  evaluation 
which  has  been  developed  incorporates  a variety  of  perspectives  and  a number  of  data  collection 
techniques  or  methodologies.  The  program  of  formative  evaluation  is  divided  into  four  stages  and 
is  integrated  with  the  development  cycle  of  a CAL  module.  Each  stage  employs  a different 
methodology.  An  advantage  of  this  program  of  formative  evaluation  is  its  applicability  in  a number 
of  multimedia  education  development  environments. 


Introduction 

A number  of  developments  in  recent  years  have  led  to  the  need  for  a more  formal  approach  to  the 
evaluation  of  multimedia  technologies  within  the  Faculty  of  Medicine,  Dentistry  and  Health  Sciences  at  the 
University  of  Melbourne.  First,  consistent  with  the  University  wide  push  to  develop  multimedia  teaching 
methods  in  higher  education,  more  energy  has  been  directed  into  developing  multimedia  education  packages 
for  courses  offered  across  the  Faculty.  Second,  as  of  semester  one  1999,  a new  problem-based  curriculum  was 
introduced  for  students  completing  a degree  in  Medicine  at  the  University  of  Melbourne.  A key  feature  of  the 
new  medical  curriculum  is  the  emphasis  placed  on  computer  aided  learning  (CAL)  which  is  consistent  with  a 
self-directed  approach  to  teaching  and  learning.  This  self-directed  approach  will  complement  traditional 
lectures  and  practical  sessions.  Seventeen  multimedia  education  projects  were  undertaken  in  1998  by  the 
Faculty,  the  majority  of  which  will  be  used  to  develop  computer  aided  learning  modules  to  support  the  new 
medical  curriculum. 

In  order  to  assist  with  the  design  and  development  of  multimedia  technologies,  the  Biomedical 
Multimedia  Unit  (BMU)  was  established  in  the  Faculty.  In  addition  to  assisting  with  project  management, 
instructional  design,  graphic  design  and  multimedia  authoring,  part  of  the  BMU’s  role  was  to  assist  Faculty 
members  with  the  evaluation  of  their  multimedia  projects.  This  paper  reports  on  how  the  BMU  approached  the 
formative  evaluation  of  multimedia  technologies  being  developed  in  the  Faculty  and  the  rationale  associated 
with  it.  Three  evaluation  dimensions  were  considered  instrumental  to  an  effective  program  of  formative 
evaluation.  These  were  (i)  the  criteria  for  evaluation  (ii)  the  participants  in  the  evaluation  and  (iii)  the  data 
collection  techniques  or  methods.  While  many  researchers  have  considered  these  dimensions  more  generally,  it 
was  thought  that  an  effective  program  of  evaluation  required  a more  specific  approach,  and  an  articulation  of 
the  interaction  between  these  variables.  In  addition,  it  was  thought  that  the  program  of  formative  evaluation 
should  be  integrated  with  the  CAL  module  development  cycle. 


Deflning  Evaluation  Criteria 

In  general  terms  evaluation  has  been  conceptualised  as  either  formative  or  summative.  Knapper  (1980) 
has  noted  the  debate  as  to  whether  educational  evaluation  should  focus  on  the  end  product  of  instruction  and  its 
impact  or  effectiveness  (summative  evaluation)  or  whether  evaluation  should  concern  itself  with  the  process  of 
refining  and  improving  instruction  (formative  evaluation).  While  the  goals  of  these  two  approaches  may,  at 
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times,  be  incompatible,  a comprehensive  program  of  evaluation  should  attempt  to  cover  both  areas.  That  is,  it  is 
important  to  attempt  to  refine  and  improve  a CAL  module  through  formative  evaluation  and  it  is  also  crucial  to 
document  the  impact  and  effectiveness  of  a CAL  module  through  summative  evaluation  once  it  has  been 
produced.  The  focus  of  this  paper  is  on  the  formative  evaluation  process  and  thus  concerns  the  refinement  and 
improvement  of  a CAL  module. 

If  the  aim  of  formative  evaluation  is  the  refinement  and  improvement  of  a CAL  module,  then  the  way 
in  which  this  can  be  brought  about  must  be  determined.  The  starting  point  for  this  process  is  making 
judgements  about  the  “quality”  (Dick  & Carey,  1991)  or  the  “value”  and  “worth”  (Cuba  & Lincoln,  1981)  of  a 
CAL  module.  Evaluators  in  the  field  of  multimedia  education  are,  as  a result,  faced  with  a question.  What 
criteria  will  be  used  to  judge  the  value  and  worth  of  a CAL  module?  Will  these  criteria  be  based  on  students’ 
learning  outcomes  or  will  they  be  based  on  the  characteristics  of  the  learning  module  itself?  Researchers  have 
used  both  these  sets  of  criteria  in  the  formative  evaluation  process,  but  often  place  different  degrees  of  emphasis 
on  each.  For  example,  Dick  and  Carey  (1991)  argue  that  while  formative  evaluation  is  aimed  at  revising  the 
module  and  is  an  iterative  process,  it  primarily  concerns  the  “collection  and  interpretation  of  learner 
performance  data”  (p.  261).  Alternatively,  Hannafin  and  Peck  (1988)  have  developed  formative  evaluation 
criteria  based  predominantly  on  the  characteristics  of  the  CAL  and  issues  associated  with  its  implementation. 
Others,  such  as  Reeves  and  Lent  (1984)  present  evaluation  criteria  based  both  on  educational  outcomes  and 
characteristics  of  the  CAL  module. 

There  are  a number  of  difficulties  with  restricting  formative  evaluation  criteria  solely  to  learning 
outcome  measures.  As  will  be  argued  below,  formative  evaluation  of  multimedia  can  begin  before  the  content  of 
instruction  has  reached  the  computer  screen.  Pragmatically,  it  may  not  be  feasible  to  assess  students’  learning 
outcomes  if  the  module  is  far  from  completion.  If  learning  outcomes  are  used  as  formative  evaluation  criteria 
other  problems  emerge.  If  the  proposed  learning  objectives  are  not  found  to  be  adequately  met  it  may  be 
difficult  to  determine  why  this  is  the  case.  That  is,  using  this  form  of  assessment  it  is  difficult  to  determine  the 
value  of  discrete  features  of  the  CAL  module.  Are  the  learning  objectives  not  being  met  because  of  the  illogical 
sequence  of  content,  the  poor  navigation  system  in  the  module,  or  the  inability  of  the  module  to  engage  the 
learner?  This  does  not  imply  that  the  assessment  of  students’  learning  outcomes  is  not  integral  to  the  evaluation 
of  CAL  modules.  Rather,  it  is  argued  that  the  assessment  of  students’  learning  outcomes  may  be  more 
appropriately  applied  in  the  final  stages  of  formative  evaluation  or  during  a program  of  summative  evaluation. 

An  alternative  to  basing  formative  evaluation  criteria  on  learning  outcomes  is  basing  them  on  the 
characteristics  of  the  CAL  module.  If  a program  of  formative  evaluation  is  predominantly  based  on  the 
characteristics  of  the  CAL  module,  this  program  could  lead  seamlessly  into  a program  of  summative  evaluation 
where  the  assessment  of  learning  outcomes  would  become  paramount.  If  it  is  accepted  that  formative  evaluation 
can  be  based  primarily  on  the  characteristics  of  a CAL  module,  researchers  and  evaluators  need  to  determine 
suitable  criteria  with  which  to  judge  CAL  modules.  These  criteria  should  reflect  the  mechanisms  or  attributes 
of  a CAL  module  which  theoretically  assist  student  learning.  Numerous  researchers  in  the  past  have  developed 
such  criteria  based  on  theories  and  models  of  learning  and  instruction. 


Criteria  used  to  Evaluate  CAL  Modules  in  the  Faculty 

The  criteria  formed  to  evaluate  the  development  and  design  of  CAL  modules  in  the  Faculty  were  based 
on  theories  of  learning  and  instruction  (eg.  Gagne,  1985;  Hannafin  & Peck,  1988;  Hannafin  & Reiber,  1989a, 
1989b;  Herrington  & Oliver,  1997;  Oliver  & Herrington,  1995;  Overbaugh,  1994;  Park  & Hannafin,  1993; 
Reigeluth,  1983;  Schmidt,  1993:  Shuell,  1980)  and  the  criteria  researchers  have  used  to  evaluate  multimedia 
programs  in  the  past  (Hannafin  & Peck,  1988;  Kearsley  1985;  Reeves  & Harmon,  1994;  Schueckler  & Shuell, 
1989;  Shuell  & Schueckler,  1989).  Primary  and  common  features  which  emerged  from  this  literature  review 
were  taken  to  form  the  formative  evaluation  criteria.  These  features  were  classified  into  three  general  domains: 
Instructional  and  Conceptual  Design,  Interface  and  Graphic  Design  and  User  Attitudes  and  Affect. 

A review  of  the  characteristics  of  the  specific  sub-criteria  contained  in  each  of  these  three  evaluation 
domains  is  presented  elsewhere  (Kennedy,  Petrovic  & Keppell,  1 998),  however,  a summary  is  presented  below. 
Five  sub-criteria  are  contained  in  the  domain  of  Instructional  and  Conceptual  Design:  introductory  objectives 
and  directions,  navigation  and  orientation,  interactivity,  sequencing  and  structure,  and  consistency  between 
learning  objectives  and  content  of  instruction.  The  sub-criteria  in  the  domain  of  Interface  and  Graphic  Design 
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were  initially  organised  by  the  primary  features  of  the  interface;  namely,  colour,  text,  structure  and  sound, 
animations  and  graphics.  Each  of  these  interface  components  is  evaluated  through  general  questions  relating  to 
issues  such  as  useability,  consistency  and  clarity.  Other  than  this  more  general  evaluation,  specific  principles  of 
“best  practice”  were  determined  for  each  component  of  the  interface  on  the  basis  of  previous  research.  The 
degree  to  which  particular  CAL  modules  conform  to  these  notions  of  “best  practice”  is  also  evaluated.  The  final 
domain,  User  Attitudes  and  Affect,  incorporated  aspects  of  the  first  two  domains.  Evaluation  centres  on  student 
users  and  asks  them  to  evaluate  aspects  of  Instructional  and  Conceptual  Design  and  Interface  and  Graphic 
Design.  In  addition,  users  are  asked  to  evaluate  modules  more  generally,  commenting  on  criteria  such  as 
effectiveness,  usefulness,  enjoyability,  appeal,  user-friendliness,  relevancy  and  engagement 


Methodology  of  Evaluation 

Once  the  formative  evaluation  criteria  were  determined  a method  of  evaluation  needed  to  be  decided 
upon.  Within  the  field  of  education,  a considerable  amount  has  been  written  on  the  methodology  associated 
with  formative  evaluation  generally,  and  of  multimedia  in  particular  (see  Alexander  & Hedberg,  1998;  Dick  & 
Carey,  1991;  Draper,  1995,  1997;  Flagg,  1990;  Cuba  & Lincoln,  1981;  Hannafin  & Peck,  1988;  Knapper, 
1980;  Reeves,  1989,  1993a,  1993b;  Reeves  & Lent,  1984;  Wills  & McNaught,  1996).  From  a review  of  this 
literature  the  methodology  associated  with  formative  evaluation  was  seen  to  have  two  key  components:  the 
individuals  who  participate  in  the  evaluation  and  the  data  collection  techniques  employed. 

Evaluation  Participants 

It  is  generally  recognised  that  the  participants  in  formative  evaluation  can  be  “experts”  and  potential 
“users”  (Dick  & Carey,  1991;  Flagg,  1990;  Hannafin  & Peck,  1988;  Reeves,  1993b;  Reeves  & Lent,  1984). 
Flagg  (1990)  categorised  experts  into  four  areas  including  subject  matter,  media,  design,  and  utilization 
experts.  Similarly,  Reeves  (1993b)  argued  that  experts  such  as  content  or  subject  matter  experts,  instructional 
experts  and  teachers  and  trainers  can  all  provide  valuable  and  unique  contributions  to  the  development  of  a 
CAL  module.  In  an  educational  setting  “users”  are  usually  thought  of  as  students  who  will  eventually  complete 
the  module. 

There  were  a number  of  reasons  to  consider  a variety  of  perspectives  in  the  formative  evaluation 
process.  First,  it  seemed  that  some  criteria  could  not  be  reliably  evaluated  by  users  while  others  could  not  be 
reliably  reflected  upon  by  experts.  For  example,  it  is  unlikely  that  a content  expert,  who  has  invested  much  time 
and  effort  in  creating  a multimedia  module,  will  indicate  that  the  module  contains  a lot  of  distracting  or 
irrelevant  information.  Similarly,  it  may  be  difficult  for  a student  to  judge  how  the  structure  and  content  of  the 
module  enhanced  the  communication  of  the  learning  objectives.  The  second  reason  to  evaluate  CAL  modules 
from  a variety  of  perspectives  was  the  advantage  of  highlighting  any  disparities  which  exist  between  the 
perceptions  of  CAL  module  users  and  content  experts  or  developers.  Some  researchers  have  remarked  that 
educational  and  content  experts  may  agree  that  a particular  feature  of  a CAL  module  is  worthwhile  only  to  find 
that  the  same  feature  is  seen  to  be  irrelevant,  distracting  or  confusing  for  the  learner  (Reeves,  1993b).  For  these 
reasons  it  was  decided  that  the  program  of  formative  evaluation  in  the  Faculty  would  include  four  perspectives: 
those  of  content  experts,  graphic  design  experts,  educational  experts  and  CAL  module  users. 

Evaluation  Data  Collection  Techniques 

Just  as  evaluators  have  suggested  a variety  of  perspectives  in  the  evaluation  process,  a number  of  data 
collection  techniques  have  also  been  advocated.  Dick  and  Carey  (1991)  suggest  the  use  of  one-to-one  trials, 
small  group  trials  and  field  trials  as  evaluation  procedures.  Reeves  and  Lent  (1994)  on  the  other  hand  argue 
that  formative  evaluation  should  be  based  on  internal  reviews  and  operational  testing  involving  questionnaires, 
expert  review,  observation  and  interviews.  It  is  important  to  use  a variety  of  measures  or  data  collection 
techniques  in  the  process  of  evaluation.  This  enables  evaluators  to  obtain  an  indication  of  the  validity  of  the 
assessment  of  constructs  through  the  processes  of  triangulation  and  bracketing.  Often  this  process  will  aid  in 
the  interpretation  of  evaluation  results. 
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However,  before  a decision  can  be  made  about  the  type  of  methodology  to  be  employed,  a number  of 
pragmatic  issues  needed  to  be  considered.  Reeves  and  Lent  (1984)  noted  that  “Unfortunately,  few  CBI 
development  projects  have  the  budget  or  time-line  to  spend  large  amounts  of  money  and  time  on  these 
[evaluation]  procedures”  (p.  195).  Commonly  formative  evaluation  is  constrained  by  the  availability  of  time 
and  money.  An  additional  constraint  in  the  current  context  was  the  availability  of  students  for  evaluation.  Due 
to  the  large  number  of  multimedia  projects  being  developed  in  the  Faculty,  CAL  module  users  were  regarded  as 
a scarce  resource.  The  developers  of  the  program  of  evaluation  were  wary  of  over-evaluating  students,  and  in 
the  long  term,  undermining  the  reliability  of  their  responses.  There  was  a need,  therefore,  to  develop  a method 
of  formative  evaluation  which  accommodated  these  restrictions  yet  produced  data  which  was  able  to  assist  in 
the  refinement  and  improvement  of  the  CAL  modules  being  developed.  Keeping  these  constraints  in  mind,  four 
means  of  data  collection  were  determined  for  the  current  program  of  formative  evaluation:  expert  review, 
questionnaire,  focus  group  and  observation. 

Stages  of  Formative  Evaluation 

The  program  of  formative  evaluation  was  divided  into  four  stages.  These  stages  were  integrated  with 
the  development  cycle  of  a CAL  module  (see  Figure  1).  It  should  be  noted  from  Figure  1 that  the  four  stages  of 
evaluation  overlap  and  that  as  a module  approaches  the  beta  version  in  the  development  cycle  an  initial 
assessment  of  users’  learning  outcomes  would  begin.  A number  of  evaluation  resources  have  been  developed  to 
be  used  in  the  evaluation  program,  including  qualitative  and  quantitative  template  questionnaires  and 
checklists  (Kennedy,  1998).  There  is  a reliance  on  questionnaires  and  expert  review  as  the  major  form  of  data 
collection  as  these  are  the  most  cost-effective  and  convenient  forms  of  data  collection.  Rather  than  being 
prescriptive  to  Faculty  staff  about  the  use  evaluation  resources,  an  array  of  resources  was  developed  to  account 
for  different  evaluation  questions  and  methodologies. 
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Figure  1 : The  stages  of  the  proposed  formative  evaluation  program  integrated  with  the  CAL  module 
development  cycle. 

The  first  two  stages  of  the  program  of  evaluation  (Stages  A and  B)  consist  of  internal  reviews  of 
instructional  and  conceptual  design  and  interface  and  graphic  design.  These  reviews  are  both  formal  (written 
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reports)  and  informal  (verbal  discussion  and  feedback)  and  are  carried  out  by  content  experts,  graphic  design 
experts  and  instructional  designers  associated  with  the  module’s  development.  Stage  C is  the  most  extensive 
stage  of  the  program  of  formative  evaluation  and  incorporates  both  instructional  and  conceptual  design  and 
interface  and  graphic  design.  Reviews  are  conducted  externally  (that  is,  by  individuals  not  associated  with  the 
module’s  development)  and  from  a number  of  perspectives  (content  experts,  graphic  design  experts, 
instructional  designers  and  student  users).  In  the  final  stage  (Stage  D)  the  domain  of  User  Attitudes  and  Affect 
is  evaluated  and  is  restricted  to  the  perspectives  of  student  users.  Three  forms  of  data  collection  are  used  in  this 
stage  (questionnaire,  focus  group  and  observation). 


Implications  and  Applications 

The  utility  of  the  proposed  program  of  formative  evaluation  lies  in  the  fact  that  it  is  not  dependent  on 
the  content  area  of  a particular  learning  module,  and  therefore  may  be  easily  applied  in  any  context  where 
educational  software  is  being  developed.  The  criteria  for  evaluation,  rather  than  being  based  on  students’ 
learning  outcomes,  are  based  on  the  characteristics  of  CAL  modules.  This  allows  the  program  to  be  easily 
transferred  to  other  educational  contexts.  While  the  program  of  evaluation  was  developed  to  evaluate  a number 
of  CAL  modules  across  a variety  of  disciplines,  it  should  be  noted  that  it  can  also  be  applied  by  smaller 
development  teams.  The  program  of  formative  evaluation  outlined  above  provides  a solid  foundation  for  the 
development  of  a commensurate  program  of  summative  evaluation.  It  is  hoped  that  when  used  together  these 
evaluation  programs  will  foster  the  development  of  educationally  effective  CAL  modules. 
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Abstract:  This  paper  attempts  to  justify  the  need  for  the  use  of  alternative  teaching  models  in  non- 
classroom-based instruction  (NCBI).  Disproportionally  high  drop-out  and  failure  rates,  along  with 
unnecessary  constraints  from  behavioral  teaching  models  necessitate  the  examination  of  alternative 
models.  Social  models  such  as  small  group  discussion  and  information  processing  models  such  as 
concept  attainment  may  more  effectively  utilize  new  delivery  methods  such  as  the  World  Wide 
Web.  More  research  using  these  teaching  models  may  aid  instructional  designers  in  selecting  the 
most  appropriate  instructional  model  for  a particular  audience  and  instructional  problem. 


Introduction 

As  the  United  States  moves  from  an  economy  based  on  manufacturing  to  one  based  on  information,  the 
need  for  information-literate  employees  is  rapidly  growing.  Public  education  is  currently  under  scrutiny  from 
business  and  industry,  and  is  being  criticized  for  being  slow-moving  and  overly  conservative,  particularly  in  terms 
of  implementing  computer  technology.  Commercial  institutions  expect  the  same  productivity  gains  in  education  that 
have  occurred  in  business.  Computer  instruction  and  the  Internet  in  particular  has  been  widely  promoted  as  the 
ultimate  instructional  tool  to  replace  schools  and  teachers  (Perelman,  1994).  Some  promoters  of  computers  and 
networks  in  education  believe  that  traditional  classroom-based  instruction  will  become  unnecessary  as  computers 
become  less  expensive  and  more  ubiquitous.  More  research  on  non-traditional  educational  methods  is  certainly  in 
order  in  response  to  cries  for  educational  reform. 

Non-classroom-based  instruction  (NCBI)  can  be  defined  as  any  planned  learning  environment  that  is  not 
designed  to  occur  in  the  traditional  classroom.  NCBI  includes  all  forms  of  distance  education,  as  well  as  campus- 
based  self-paced  courses  and  other  self-paced  learning  environments.  This  type  of  instruction  is  becoming  much 
more  common,  particularly  due  to  the  growth  of  computer  networks  (Harasim,  1995).  With  the  rapid  expansion  of 
NCBI  there  is  a need  to  examine  its  effectiveness  in  terms  of  instructional  design.  A primary  method  of  evaluating 
various  forms  of  NCBI  lies  in  the  examination  of  the  teaching  model  employed  in  the  design  of  the  instruction. 

Teaching  models  are  approaches  to  teaching  with  underlying  theoretical  support  that  are  designed  to  bring 
about  a particular  type  of  learning  (Joyce,  1992).  Joyce  groups  the  various  teaching  models  into  "families”  based  on 
the  learning  theories  from  which  the  models  have  been  derived.  A learning  theory  can  be  defined  as  a set  of 
principles  and  observations  derived  from  research  that  support  a hypothesis  of  how  learning  takes  place  (Ormrod, 
1995).  Models  for  delivery  have  been  constructed  around  the  tenets  of  certain  learning  theories.  According  to  Joyce 
(1992),  teaching  models  can  be  grouped  into  four  "families".  These  families  include  behavioral  models,  social 
models,  information-processing  models,  and  personal  models.  Following  is  a brief  summary  of  three  of  these 
families  of  teaching  models.  Personal  models  will  not  be  discussed  as  they  are  beyond  the  scope  of  this  paper. 

Behavioral  Teaching  Models 

On  the  foundation  of  the  work  of  Skinner,  a large  number  of  approaches  to  learning  have  been  developed, 
each  taking  advantage  of  the  human  being’s  ability  to  modify  behavior  in  response  to  tasks  and  feedback.  These 
models  are  used  in  a wide  variety  of  applications,  from  teaching  information  to  changing  habits,  decreasing  phobias, 
and  learning  to  control  one’s  own  behavior. 

One  of  the  important  applications  of  behavioral  systems  theory  is  in  the  development  of  systems  that  enable 
learning  tasks  to  be  regulated  according  to  the  progress  of  the  learners  and  which  teach  students  to  pace  themselves 
for  optimal  performance.  Often  these  systems  organize  material  to  be  learned  in  small  sequenced  instructional 
modules  that  are  presented  to  the  students  with  assessments  of  learning  embedded  in  them  (Joyce,  1992).  Some  of 
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these  behaviorally-based  teaching  models  include  mastery  learning,  direct  instruction,  contingency  management, 
and  self-training  through  simulation. 

Social  Teaching  Models 

These  teaching  models  capitalize  on  our  nature  as  social  creatures  to  further  learning  and  to  expand  our  ability  to 
relate  productively  to  one  another.  The  models  range  from  the  simple  processes  or  organizing  students  to  work 
together  to  elaborate  models  that  teach  democratic  social  organization  and  the  analysis  of  major  social  problems  and 
critical  social  values  and  issues.  The  social  models  of  teaching  are  constructed  to  take  advantage  of  the  "synergy” 
phenomenon  - the  collective  energy  generated  by  group  work.  The  simplest  forms  of  cooperative  learning  organize 
students  to  help  one  another  respond  to  the  cognitive  and  social  tasks  presented  to  them.  More  advanced  models 
prepare  students  for  life  in  a democratic  society  --  an  idea  developed  by  Dewey  (1916). 

Many  developers  of  the  social  models  believe  not  only  that  they  have  developed  important  additions  to  the 
storehouse  of  models  but  also  that  the  current  state  of  affairs  — teacher-dominated  recitation  — is  actually  counter- 
productive for  individuals  and  society  by  depressing  learning  rates,  creating  an  unnatural  and  even  antisocial 
climate,  and  failing  to  provide  opportunities  for  young  people  to  maximize  their  potential  through  cooperation. 
Current  social  teaching  models  to  be  considered  include  partners  in  learning  (dyads),  cooperative  learning,  role 
playing,  and  jurisprudential  inquiry  (Joyce,  1992). 

Information  Processing  Teaching  Models 

Information  processing  theory  is  concerned  with  how  humans  acquire,  process,  and  remember  information. 
This  is  contrasted  with  behaviorism’s  concern  with  measurable  behavior  change  and  lack  of  concern  over  mental 
events  or  cognition.  A number  of  teaching  models  seek  to  discover  these  mental  events  and  create  effective 
environments  for  processing  information  (Pressley,  1995). 

These  models  are  designed  to  increase  students’  ability  to  seek  and  master  information,  organize  it,  build 
and  test  hypotheses,  and  apply  what  they  are  learning  in  their  independent  reading  and  writing  and  their  exploration 
of  themselves  and  the  world  about  them.  Some  of  these  models  induce  the  students  to  collect  information  and  build 
concepts.  Others  teach  them  to  profit  from  direct  instruction  through  readings,  lectures,  and  instructional  systems. 
Some  of  the  information  processing  models  include  thinking  inductively,  concept  attainment,  memorization, 
advance  organizers,  and  inquiry  training  (Joyce,  1992). 


Teaching  Models  for  NCBI 

The  preceding  teaching  models  have  been  widely  used  in  traditional  classroom-based  educational 
environments.  However,  many  instructional  situations  and  opportunities  exist  outside  the  school  classroom.  These 
instructional  situations,  identified  as  non-classroom-based  instruction  (NCBI)  operate  outside  of  traditional  face-to- 
face  interaction  with  an  instructor,  so  alternative  delivery  methods  must  be  considered.  NCBI  has  been  and  is 
currently  being  delivered  via  print,  radio,  television,  and  computer  network.  Following  is  a discussion  of  the 
teaching  models  employed  within  these  delivery  methods. 

NCBI  Delivered  via  Print 

NCBI  consists  of  distance  education  courses  as  well  as  school  or  campus-based  self-paced  courses. 
Distance  education,  traditionally  defined,  is  an  environment  in  which  instructor  and  audience  are  separated  by  time 
or  distance  or  both.  While  NCBI  may  seem  to  be  a recent  phenomenon,  print-based  correspondence  courses  have 
been  in  existence  for  over  150  years  ^olmberg,  1996).  These  early  examples  of  NCBI  consisted  of  courses  offered 
through  newspapers  and  through  printed  mailings  using  the  postal  service.  Print-based  correspondence  courses  are 
still  used  today  although  they  are  now  commonly  combined  with  electronically-delivered  material  (Moore,  1995). 

Print-delivered  NCBI  has  traditionally  relied  upon  behaviorally-based  teaching  models,  particularly  direct 
instruction  and  mastery  learning  (see  Price,  1 996).  The  common  design  uses  short  modules  of  instructional  content 
followed  by  a self-test  and  then  a formal  assessment,  most  often  using  techniques  that  require  low-level  processing, 
such  as  multiple  choice  tests,  matching,  and  filling  in  blanks.  Course  objectives  are  stated  in  behavioral  terms.  The 
student  may  conduct  an  internal  dialogue  with  the  material,  but  has  little  to  no  interaction  with  a live  instructor  or 
with  other  students.  Collaborative  learning  is  discouraged  in  these  instructional  situations. 
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The  most  common  model  for  print  course  development  is  the  author-editor  model.  Authors  (often 
professors)  are  contracted  to  write  a course,  which  is  then  edited  by  specialists  within  the  correspondence  school  or 
independent  studies  department.  In  most  cases  the  correspondence  material  consists  of  units  that  include  a statement 
of  lesson  objectives,  a reading  assignment,  a commentary  and  a study  guide,  and  a written  assignment  to  be 
completed  (Moore,  1993). 

NCBI  Delivered  via  Electronic  Communications 

Electronic  communications  for  education  also  began  in  the  1920s  with  radio  programs.  These  programs 
most  often  focused  on  fundamental  instruction,  usually  in  language  and  mathematics  (Cuban,  1986).  In  the  1930s 
experimental  television  programs  were  being  produced  at  a few  pioneering  universities,  and  programming  expanded 
in  the  next  few  decades.  These  radio  and  television  programs  were  not  interactive,  with  no  student  feedback  or 
discussion  with  the  instructor.  Content  was  commonly  delivered  by  a "talking  head"  — meaning  an  instructor  faced 
the  camera  or  microphone  and  delivered  a prepared  lecture  (Cuban,  1986).  This  delivery  method  operates  within 
artificial  constraints  that  are  not  set  in  place  by  the  television  medium. 

Satellite  technology,  developed  in  the  1960s  and  made  cost  effective  in  the  1980s,  enabled  the  rapid  spread 
of  instructional  television.  Televised  courses  have  now  become  more  interactive  with  the  deployment  of  distance 
education  equipment  including  cameras  and  microphones  for  the  students  at  remote  and  local  sites  as  well  as  for  the 
instructors.  Still  the  teaching  model  most  commonly  used  is  direct  instruction,  with  the  instructor  lecturing  to  a 
group  of  students  whether  in  person  or  at  a remote  site  (Gilbert,  Temple,  & Underwood,  1991). 

The  last  ten  years  have  seen  an  exponential  growth  of  microcomputers  in  homes  and  schools  in  the  United 
States  (Williams,  1996).  As  microcomputer  access  has  grown,  so  has  computer-based  instruction  (CBI).  CBI  began 
as  an  outgrowth  of  Skinner’s  programmed  instruction.  Because  of  this,  much  of  CBI  has  followed  the  drill-and- 
practice  behavioral  model  (Price,  1991).  The  development  of  the  CD-ROM  greatly  increased  the  storage  capacity  for 
commercial  and  educational  software,  and  this  increased  storage  has  led  to  the  creation  of  multimedia  titles  that  are 
rich  in  sound,  video,  and  animation.  While  these  attributes  in  themselves  do  not  lead  to  alternative  teaching  models, 
multimedia  titles  tend  to  be  more  learner-centered  in  terms  of  user  control  and  non-linear  progression  through  the 
material. 

The  distribution  of  Apple  Computer’s  HyperCard  authoring  environment  contributed  to  the  development  of 
hypertext  and  hypermedia  programs  in  CBI.  Proponents  of  these  programs  claim  that  they  model  human  associative 
memory  and  thus  can  serve  as  powerful  cognitive  amplifiers  (Conklin,  1987).  Hypertext  CBI  is  related  more  to 
information  processing  than  behavioral  models,  and  these  types  of  applications  have  become  much  more  prevalent 
with  the  growth  of  the  World  Wide  Web. 

NCBI  Delivered  via  Computer  Networks 

With  the  widespread  proliferation  of  personal  computers  and  the  growth  of  the  Internet,  online  distance 
education  is  a rapidly  growing  segment  of  NCBI  that  serves  as  an  educational  alternative  to  traditional  face-to-face 
instruction.  The  ubiquity  and  ease  of  use  of  the  World  Wide  Web  has  led  to  a large  expansion  of  online  courses  in 
higher  education.  In  the  United  States,  the  American  Open  University,  Nova  University,  and  the  University  of 
Phoenix  have  been  traditional  leaders  in  providing  distance  education.  These  and  many  other  universities  are  now 
offering  hundreds  of  courses  online  (Lintz  & Tognotti,  1996). 

Computer  networks  offer  new  opportunities  for  collaboration  in  NCBI.  Protocols  and  software  tools  such  as 
e-mail,  mailing  lists,  newsgroups,  chat,  and  instant  messaging  allow  several  modes  of  interaction.  Interaction  modes 
can  be  classified  as  one-to-one,  one-to-many,  and  many-to-many  (Harasim,  1990).  The  Internet  is  a medium  that 
removes  many  of  the  constraints  of  print,  radio,  and  television  delivery  mediums.  Communication  can  occur  in  real 
time  or  asynchronously,  allowing  interaction  with  both  instructor  and  fellow  students.  Distributing  material  over 
networks  is  much  less  expensive  than  television  production  and  more  expedient  than  print  correspondence.  It  would 
seem  to  be  a superior  medium  for  many  types  of  instruction.  But  as  in  print  correspondence  and  earlier  electronic 
media  in  their  initial  years  of  use,  network-delivered  NCBI  has  relied  primarily  on  behavioral  teaching  models,  most 
often  direct  instruction  and  mastery  learning.  This  is  evidenced  by  current  Web-based  online  course  offerings  (see 
WebLeaming,  1998;  Real  Education,  1998). 
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Current  Practice 


Many  educational  institutions  as  well  as  businesses  and  other  organizations  are  providing  Web-based 
courses  for  students  and  employees.  However  a large  segment  of  this  non-classroom-based  online  coursework 
remains  exclusively  in  the  domain  of  the  behavioral  family  of  teaching  models,  despite  evidence  of  the  effectiveness 
of  alternate  teaching  models. 

The  effectiveness  of  the  current  state  of  NCBI  is  under  scrutiny,  particularly  as  institutions  prepare  to 
embark  upon  major  distance  education  initiatives.  Drop-out  rates  for  correspondence  courses  are  disproportionately 
high  (Kember,  1989).  Similarly,  in  a pilot  study  the  researcher  found  that  students  in  a campus-based  self-paced 
course  demonstrated  unusually  high  failure  rates.  The  nature  of  NCBI  places  responsibility  for  learning  upon 
students  who  may  have  never  faced  such  requirements. 

For  behavioral  teaching  models  there  are  other  concerns  beyond  high  failure  and  drop-out  rates.  In  the 
context  of  an  online  course  these  models  do  not  take  advantage  of  the  available  attributes  of  the  http  protocol.  These 
attributes  include  the  availability  of  asynchronous  interaction  and  discussion  with  the  instructor  or  classmates,  and 
hypertext  associational  linking  between  related  topics.  This  linking  may  mirror  the  hypothesized  information 
processes  in  human  cognition.  Behavioral  models,  therefore,  work  within  artificial  constraints  not  imposed  by  the 
http  protocol. 

The  dominance  of  behavioral  models  in  NCBI  can  be  explained  in  terms  of  misperception  and 
convenience.  Behavioral  teaching  models  are  perceived  to  present  low  instructor  demands  in  terms  of  time  and 
design  expertise.  Behavioral  models  such  as  direct  instruction  and  mastery  learning  seem  to  be  simply  and 
conveniently  translated  into  printed  or  online  "study  guides"  which  lead  the  learner  step-by-step  through  the 
particular  instructional  content.  Often  these  materials  are  organized  into  instructional  "modules"  which  can  be  easily 
assessed  for  effectiveness  (see  Price,  1996). 

In  order  to  achieve  the  interaction  required  by  social  and  information  processing  teaching  models,  NCBI 
(whether  print-based  or  Internet-based)  is  perceived  to  require  greater  instructor  time  and  resources,  as  well  as  more 
advanced  design  and  programming  techniques.  The  decision  to  use  behavioral  teaching  models  in  NCBI  may 
therefore  be  based  on  perceived  convenience  rather  than  effectiveness  and  appropriateness  toward  reaching  the 
instructional  goal. 

Teaching  models  from  the  information  processing  and  social  interaction  families  do  not  necessarily  require 
greater  instructor  time  or  resources  to  implement.  Alternative  online  course  design  and  specialized  software  present 
an  opportunity  to  use  these  models  efficiently.  These  alternative  designs  need  to  be  compared  and  contrasted  with 
designs  Ifom  behavioral  teaching  models  such  as  direct  instruction  and  mastery  learning  which  currently  dominate 
NCBI  environments. 


Alternative  Teaching  Models  for  NCBI 

The  following  two  teaching  models  are  proposed  as  alternatives  to  the  behavioral  models  widely  used  in 
NCBI.  Small  group  discussion  is  trom  the  social  interaction  family  of  models  and  concept  attainment  is  from  the 
information  processing  family  of  models. 

Small  Group  Discussion 

The  challenge  of  the  current  state  of  affairs  is  to  design  effective  NCBI  using  teaching  models  beyond 
simple  direct  instruction  or  mastery  learning.  The  evidence  is  largely  affirmative  that  cooperative  groups  do  result  in 
improved  learning.  Classrooms  organized  so  that  students  work  in  pairs  and  larger  groups,  tutor  each  other,  and 
share  rewards  are  characterized  by  greater  mastery  of  material  than  the  common  individual-study  and  recitation 
pattern.  Also,  the  shared  responsibility  and  interaction  produce  more  positive  feelings  toward  tasks  and  others, 
generate  better  intergroup  relations,  and  result  in  better  self-images  for  students  with  histories  of  poor  achievement. 
The  results  generally  affirm  the  assumptions  that  underlie  the  use  of  cooperative  learning  methods  (Sharan,  1990). 

Beyond  traditional  face-to-face  instructional  situations,  the  effectiveness  of  group  discussion  and 
collaboration  has  also  been  shown  effective  in  distance  education.  With  Internet  delivery  there  are  many  cooperative 
learner  approaches  for  effective  instructional  environments.  These  include  online  mentorships,  tutor  support, 
informal  peer  interaction,  and  expert  forums.  Online  group  learning  structures  include  online  seminars,  small  group 
discussions,  learning  partnerships  and  dyads,  learning  circles,  and  teaching  and  presentations  by  the  learners.  Role 
playing  simulations,  online  debates,  and  informal  chat  areas  are  also  viable  instructional  environments  (Harasim, 
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1995).  Many  of  these  learner  approaches  can  be  classified  as  online  asynchronous  group  instruction.  Specialized 
network  software  such  as  the  IdeaWeb  (Ahem,  1992)  can  be  used  for  small  group  discussion  and  collaboration  in 
online  environments  as  a type  of  computer-mediated  communication  (CMC).  The  IdeaWeb  represents  an 
opportunity  to  utilize  the  advantages  of  the  http  protocol  in  creating  a collaborative  learning  environment  within 
NCBI. 

Concept  Attainment 

Among  the  information-processing  models  found  within  traditional  education,  the  concept  attainment 
model  is  a viable  choice  for  instructional  use  in  NCBI.  The  concept  attainment  model  of  instruction  can  be  defined 
as  the  search  for  and  listing  of  attributes  that  can  be  used  to  distinguish  exemplars  from  non  exemplars  of  various 
categories.  The  model  requires  students  to  compare  and  contrast  exemplars  that  contain  characteristics  (called 
"attributes")  of  the  concept  with  exemplars  that  do  not  contain  those  attributes.  Exemplars  are  a subset  of  data  from 
a data  set.  Positive  exemplars  are  the  subset  examples  that  share  one  or  more  defining  characteristics  that  are 
missing  in  the  other  examples.  It  is  by  comparing  the  positive  exemplars  and  contrasting  them  with  the  negative 
ones  that  the  concept  is  learned. 

Learning  by  concept  attainment  involves  two  phases.  First,  the  concept  name  is  presented  along  with 
labeled  examples  of  positive  and  negative  exemplars.  The  learner  integrates  these  exemplars  and  generates  a 
hypothesis  about  the  concept  definition.  The  student  then  states  a possible  definition  in  terms  of  essential  attributes. 
Secondly,  the  student  identifies  additional  unlabeled  examples  as  positive  or  negative  exemplars  of  the  concept.  The 
student  hypothesis  and  definition  is  either  confirmed  or  corrected,  and  a correct  definition  of  the  concept  is  given. 
The  student  then  generates  more  concept  examples.  By  following  this  model  the  student  learns  the  concepts  related 
to  the  instructional  content  (Joyce,  1992).  Using  HTML,  the  author  has  previously  developed  concept  attainment 
lessons  for  use  in  an  NCBI  environment. 


Conclusion 

This  paper  attempts  to  justify  the  need  for  the  use  of  alternative  teaching  models  in  non-classroom-based 
instruction  (NCBI).  The  author  contends  that  behavioral  teaching  models  are  dominant  in  NCBI  and  that  high  failure 
and  drop-out  rates,  along  with  unnecessary  constraints  from  the  behavioral  models,  necessitate  the  examination  of 
alternative  models  of  teaching.  This  paper  examines  two  alternate  teaching  models:  small  group  discussion  from  the 
social  models  classification,  and  concept  attainment  from  the  information-processing  models  classification. 
Instructional  systems  using  these  models  can  be  asynchronous  and  do  not  necessarily  require  additional  instructor 
time  or  resources  in  terms  of  class  management.  Instructional  effectiveness  can  potentially  be  increased  without 
significantly  increased  instructor  resources.  Empirical  research  on  teaching  models  should  be  conducted  to  examine 
the  level  of  effectiveness  of  these  particular  teaching  models  for  NCBI.  This  type  of  research  will  aid  course 
designers  in  selecting  the  most  appropriate  instructional  model  for  a particular  audience  and  instructional  problem. 
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Abstract:  This  study  investigated  a virtual  partnership  between  educational  technology  graduate  students 
in  France  and  Texas.  The  partnership  was  designed  initially  for  the  students  to  communicate  their 
conceptions  of  educational  technology  primarily  via  a Web  message  board.  We  used  qualitative  research 
methods  to  identify  the  Texas  students'  descriptions  of  their  participation  in  and  reactions  to  the  partnership 
and  their  ideas  about  educational  technology  in  the  U.S.  The  investigation  tested  an  existing  model  of  six 
design  considerations  for  computer  conferencing  (grading  system,  grouping,  collaboration,  relevance, 
learner  control,  and  technological  preparation)  and  four  design  considerations  for  cross-cultural 
participation  via  communication  technologies  (communication  and  interaction,  language,  content,  and 
representation  form).  We  discovered  that  unless  each  design  consideration  is  addressed,  cross-cultural 
partnership  via  telecommunications  may  be  unsuccessful. 


Introduction 

Graduate  students  preparing  to  work  in  the  field  of  educational  technology  and  multimedia  subsequently 
find  positions  as  administrators  and  faculty  members  in  K-12  settings,  community  colleges  and  universities;  and  as 
trainers  and  instructional  designers  in  the  public  and  private  sectors.  Students  graduating  from  programs  in 
educational  technology  and  multimedia  require  specialized  telecommunications  and  multimedia  skills  and  face  an 
increased  emphasis  on  international  communication  in  their  work.  Romiszowski  (1997)  identifies  critical  thinking, 
or  creative  problem  solving,  of  "paramount  importance  for  the  knowledge  worker  of  the  future  and,  therefore, 
ultimately,  for  the  employability  of  the  human  race"  (p.  33).  A study  of  Texas  employers  identified  multilingual 
skills  as  vital  and  increasingly  required  for  success.  Other  requisite  skills  include:  written  and  verbal  presentation 
skills,  ability  to  think  logically,  sound  judgment,  ability  to  work  in  multicultural  teams,  flexibility  in  adapting  to 
change,  and  ethical  decision  making  (Zey,  Luedke,  & Murdock,  1999).  In  preparation  for  such  work,  graduate 
students  can  expand  their  horizons  about  the  role  of  educational  technology  and  multimedia  in  an  international 
arena  by  using  the  Internet  to  communicate  and  collaborate  across  international  boundaries.  In  this  cross-classroom 
partnership,  graduate  students  in  educational  technology  at  Texas  A&M  University  (TAMU)  in  the  US  and  the 
University  of  Poitiers  (UP)  in  France  used  telecommunications  to  share  ideas  on  topics  related  to  their  studies. 

The  Project 

Graduate  students  in  educational  technology  at  TAMU  in  the  US  and  at  UP  in  France  were  linked  through 
an  electronic  communication  exchange  designed  to  expand  participating  students'  conceptions  about  the  graduate 
programs  and  the  role  of  educational  technology.  The  course  for  TAMU  students  began  at  the  end  of  August, 
whereas  the  UP  students  began  their  course  six  weeks  later.  The  TAMU  students  met  weekly  via  interactive 
videoconference  and  FirstClass  computer  conference  software  to  discuss  topics,  post  assignments,  and  develop 
small  group  projects.  The  UP  students,  on  the  other  hand,  were  in  a traditional  face-to-face  class:  lectures,  labs, 
problem-solving  using  varied  technologies;  the  course  culminated  with  a three-month  internship  in  a company. 
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The  TAMU  and  UP  educational  technology  instructors  had  collaboratively  developed  the  partnership 
activities,  which  included:  1)  development  of  student  biographies  as  individual  Web  pages,  2)  class  discussions  at 
both  universities,  3)  a one-hour  videoconference,  4)  three  discussions  via  message  boards  on  the  Web  over  a six- 
week  period  among  mixed  groups  of  students,  and  5)  a follow-up  survey.  The  TAMU  instructor  assigned  10%  of 
her  students’  grades  to  this  project,  whereas  the  UP  instructor  informed  his  students  that  participation  in  the  project 
was  required  but  did  not  assign  specific  points. 

The  first  activity  was  for  the  students  to  create  and  publish  their  individual  biographies  on  the  Web.  The 
second  activity,  class  discussions  at  both  universities,  addressed  the  logistics  of  the  partnership  activities, 
guidelines  for  using  the  Web  board,  and  communication  issues  between  the  two  universities.  The  third  activity  was 
a one-hour  interactive  videoconference,  which  included  the  instructors  and  nearly  all  of  the  students  from  both 
countries.  The  videoconference  included  introductions  of  faculty  and  students,  and  small  group  discussions.  Each 
group  of  eight  students  took  turns  sharing  backgrounds  and  interests  with  each  other  as  well  as  making  plans  for 
their  Web  discussions.  The  videoconference  culminated  with  a demonstration  of  the  Web  message  board. 

The  fourth  activity  was  a series  of  discussions  on  a Web  message  board  about  the  roles  of  educational 
technology.  The  TAMU  instructor  developed  six  identical  Web  boards,  which  allow  users  to  post  new  messages, 
reply  to  questions  and  other  replies,  and  search  for  a string  of  characters  within  messages.  Separate  boards  were 
built  for  each  group  of  eight  students,  four  from  each  university.  We  planned  that  during  a six  week  period,  the  six 
groups  would  discuss  three  topics  for  two  weeks  each:  1)  the  role  of  educational  technology  at  their  universities;  2) 
the  role  of  educational  technology  in  their  countries;  and  3)  the  formation  of  their  own  conceptions  of  educational 
technology.  Six  TAMU  students,  one  from  each  group,  were  designated  to  moderate  the  first  discussion  topic  for 
their  groups,  thereby  modeling  appropriate  computer  conferencing  behaviors  for  the  UP  students,  whose 
coursework  had  only  recently  begun.  The  TAMU  instructor  first  entered  the  welcome  messages  and  instructions. 
Then  six  student  moderators  each  entered  the  first  set  of  three  discussion  questions  regarding  educational 
technology  in  the  universities.  During  the  first  two-week  period,  each  TAMU  student  entered  at  least  one  reply  to 
the  three-part  discussion,  while  most  students  replied  several  times.  Then  they  waited  for  responses  from  UP. 
However,  only  one  UP  student  participated,  replying  to  each  of  the  three  questions  in  her  discussion. 

The  final  activity  was  a follow-up  survey  administered  in  March  1999.  The  TAMU  students  received  their 
surveys  by  email,  and  the  UP  students  received  theirs  on  paper.  The  survey  collected  demographic  data, 
investigated  conceptions  about  educational  technology  in  the  two  countries,  and  sought  reactions  to  participation  in 
the  partnership. 

Methods 

Participants 

This  partnership  included  TAMU  in  College  Station,  in  central  Texas,  and  the  UP,  in  central  France. 
Participants  included  24  graduate  students  from  TAMU  and  22  from  UP.  With  one  exception,  the  TAMU  students 
were  enrolled  in  a semester-long  course  in  educational  technology  earning  master's  or  doctoral  degrees  in 
education.  All  22  UP  students  were  post-graduates  in  the  master’s  degree  program  consisting  of  coursework  on 
campus  followed  by  internships  throughout  the  country.  The  UP  students,  who  were  primarily  from  France  and 
included  several  students  from  Romania,  remained  as  a cohort  throughout  the  year-long  program. 

Data  Sources 

The  data  sources  included  the  messages  from  the  six  Web  boards  and  the  follow-up  survey  results  from  17 
TAMU  students.  We  downloaded  the  messages  from  the  six  Web  board  discussions,  and  we  coded  the  responses  on 
the  surveys.  The  data  were  coded  according  to  T for  TAMU,  a student  number  between  1 and  24,  and  ”s"  for 
survey  or  "w"  for  Web  board.  We  analyzed  the  data  by  using  the  constant  comparative  method. 

Research  Questions  and  Procedures 

The  initial  research  questions  dealt  with  an  investigation  of  students'  conceptions  of  educational  technology  in 
the  two  countries  and  identification  of  the  conceptions  that  were  shared  across  borders.  However,  because  of  the 
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lack  of  participation  by  UP  students,  we  changed  our  research  focus  to  the  actual  partnership  experience.  We 
investigated  the  following  topics: 

1 , What  were  TAMU  students’  conceptions  about  educational  technology? 

2,  How  did  TAMU  students’  describe  their  participation  in  and  reactions  to  the  partnership? 

In  addition,  we  wanted  to  identify  key  ingredients  of  successful  cross-cultural  partnerships.  Therefore,  after 
analyzing  the  data,  we  critiqued  the  data  based  on  two  design  models,  one  of  computer  conferencing  and  the  other 
of  cross-cultural  collaboration.  Doing  so  allowed  us  to  identify  which  factors  were  not  addressed  in  the  partnership 
and  therefore  may  have  been  responsible  for  the  failure  to  achieve  the  objectives. 


Results 

Based  upon  the  existing  data,  we  report  here  on  findings  regarding  the  partnership.  The  TAMU  students 
reported  spending  approximately  1.5  hours  in  class  discussing  aspects  of  the  partnership.  The  means  of  student 
hours  spent  on  the  three  activities  associated  with  the  partnership  follows;  creating  a Web-based  biography  (mean 
= .87  hours),  videoconferencing  (mean  = 1.02  hours),  and  Web  board  discussion  (mean  = 1.44  hours).  Therefore, 
the  mean  amount  of  time  spent  on  the  partnership  per  student  was  2.42  hours.  All  TAMU  students  created  a Web- 
based  biography  and  participated  in  the  Web  board.  Although  the  videoconference  was  held  on  a Saturday  and  was 
optional,  20  of  the  24  TAMU  students  attended. 

Conceptions  of  Educational  Technology:  The  US  Perspective 

Students  perceived  the  role  of  educational  technology  in  schools,  business,  and  government  as  a growing 
entity  for  facilitating  learning,  supporting  communication,  and  conducting  research.  In  schools,  educational 
technology  enhances  student  learning  primarily  through  application  to  problem  solving,  a skill  that  they  will 
transfer  to  situations  outside  of  school.  The  TAMU  students  identified  problems  associated  with  technology  in  the 
schools.  “Schools  are  quickly  realizing  how  important  technology  has  become... In  many  cases,  however,  teachers 
are  not  trained  on  how  to  use  the  equipment.  They  also  may  not  be  trained  on  how  to  integrate  the  technology  into 
their  teaching  scheme”  (T19s).  Another  student  expressed  concern  that  some  people  believe  technology  “could 
’replace’  a teacher”  (T2s).  In  business,  educational  technology  facilitates  training  of  personnel,  sharing  of  ideas 
through  telecommunications  and  presentations,  and  providing  instructional  programs  and  testing  methods.  The 
government  sponsors  "public  awareness  and  informational  programs  and  ...is  attempting  to  help  all  schools  access 
the  Internet  in  order  for  all  children  to  have  access  to  information  and  experience  with  technology”  (T16s). 

Students  reported  forming  their  ideas  about  educational  technology  by  discussing  issues  with  their 
classmates,  professors,  colleagues,  and  others  both  face-face  and  through  listservs;  reading  literature  in  the  field; 
experiencing  educational  technology  hands-on;  conducting  research;  and  solving  educational  and  technological 
problems.  They  acknowledged  the  importance  of  educational  technology  and  maintained  a positive  attitude  that 
makes  them  receptive  to  learning:  “1  realize  more  and  more  each  day  the  importance  of  learning  to  use  it  so  1 can 
function  in  today’s  society”  (T7s).  They  identified  various  elements  in  the  popular  culture  that  helped  them  shape 
their  ideas  about  educational  technology:  television,  magazines  (including  trade  magazines),  film,  and  videos. 

The  Partnership  Experience:  The  US  Perspective 

The  videoconference  was  the  most  popular  activity  of  the  partnership.  Students  enjoyed  “being  able  to 
interact  with  people  from  a different  culture  and  setting”  (Tils)  and  “being  able  to  SEE  the  French  students” 
(TlOs).  One  student  suggested  that  future  follow-up  videoconferences  should  be  held.  Two  students  liked  sharing 
biographies  best  of  the  activities,  as  one  of  them  explained:  “1  enjoyed  reading  the  biographies  ...and  finding  out 
how  their  preparation  was  different  from  ours”  (T22s). 

TAMU  students  agreed  that  the  concept  of  cross-cultural  learning  about  educational  technology  was  good 
but  that  in  reality  the  experience  was  a great  disappointment.  The  videoconference  built  up  students’  hopes  that 
they  could  communicate  with  students  in  France.  However,  when  TAMU  students  did  not  receive  responses  to  their 
questions,  they  expressed  frustration  of  checking  the  Web  board  frequently  and  finding  no  responses  from  UP. 

The  two  TAMU  students  who  expressed  positive  experience  with  the  Web  board  imparted  value  on  the 
process  of  cross-cultural  communication  itself  They  noted  the  power  of  international  dialogue,  saying:  “It ...  forces 
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you  to... be  precise  in  how  you  are  saying  what  it  is  that  you  want  to  say”  (T16s)  and  determining  "the  best  way  to 
organize  the  discussion  to  make  it  easy  to  follow  for  new  users”  (T22s).  Both  students  examined  their  own 
processes  and  used  the  partnership  to  refine  their  approaches. 

The  TAMU  students  suggested  that  the  instructors  “make  sure  both  groups  are  willing  to  follow  through 
with  what  the  collaboration  entails”  (Tils).  They  offered  these  suggestions  to  enhance  collaboration:  (a)  clarify 
expectations  to  learners  by  setting  "a  minimum  number  of  entries  in  a set  time  period”  (T9s)  and  maintain  contact 
with  each  other;  (b)  arrange  for  sufficient  access  to  labs  and  equipment  as  well  as  training  in  using  technology;  and 
(c)  enforce  communication  through  guidance  and  grading  of  the  activity. 

The  students  felt  that  they  had  done  everything  possible  to  encourage  the  UP  students.  They  suggested  that 
the  UP  students  demand  lab  time,  be  proactive,  and  make  more  effort  to  communicate.  Despite  limited 
communication,  TAMU  students  learned  from  the  partnership.  Several  students  commented  that  a successful 
partnership  involves  careful  planning  and  a commitment  from  both  sides.  However,  one  noted  that  “even  the  best 
planned  initiative  can  be  short-circuited  by  one  problem  - no  open  labs  in  Poiters”  (T22s).  Other  students 
acknowledged  the  power  of  technology  to  bring  people  together  from  distant  comers  of  the  world.  “In  about  an 
hour  I had  met  a bunch  of  nice  people  on  the  other  side  of  the  globe.  I saw  their  faces,  their  names  and  I heard 
them  talking,  just  like  if  they  had  been  in  the  same  room  with  me”  (T14s). 

Through  observations,  email  exchanges,  and  comments  from  the  TAMU  students,  we  identified  probable 
reasons  for  the  failed  electronic  exchange.  Limited  access,  limited  time  to  reply  to  messages,  lack  of  understanding 
the  power  of  asynchronicity  or  how  to  use  the  Web  board,  and  lack  of  course  credit  for  participation  all  led  to  the 
UP  students'  setting  the  partnership  low  on  the  priority  list  of  course  activities. 


Discussion 

The  purpose  for  identifying  the  reasons  that  the  partnership  was  ineffective  for  the  goal  of  investigating 
cross-cultural  understandings  of  educational  technology  was  to  provide  a scholarly  analysis  of  why  this  particular 
cross-cultural  partnership  did  not  achieve  its  goals.  The  broader  purpose  was  to  discover  the  key  ingredients 
needed  for  successful  partnerships  across  cultures.  The  literature  describes  cross-cultural  partnerships  and  suggests 
successful  designs  of  such  partnerships  via  telecommunications.  This  literature  is  related  to  the  following  topics: 
(a)  educational  partnerships,  (b)  designing  for  computer  conferencing,  and  (c)  cross-cultural  communication. 

Educational  Partnerships 

The  literature  on  university-school  partnerships  reveals  important  guidelines  for  successful  partnerships  of 
various  types.  Tushnet  (1993)  identifies  the  following  characteristics  of  successful  partnerships:  a history  of 
collaboration;  addressing  real  problems  as  they  occur;  mutual  respect  for  each  other;  and  discussions  with  all 
players  about  the  content  of  activities.  Prior  to  the  partnership,  the  TAMU  and  UP  instructors  had  not  shared  their 
content  or  ways  to  use  technology.  Once  problems  such  as  unequal  contributions  by  the  students  began  to  surface, 
the  instructors  failed  to  communicate  regularly  with  each  other  and  thus  didn't  address  the  problems. 

Designing  for  Computer  Conferencing 

Increased  use  of  computer-mediated  communication  for  teaching  and  learning  has  created  a developing  body 
of  literature  on  designing  for  online  environments.  Researchers  propose  six  design  considerations  for  computer 
conferencing:  two  that  involve  administrative  design  considerations— grading  system  and  grouping— and  four 
instructional  design  considerations— collaboration,  relevance,  learner  control,  and  technological  preparation 
(Cifuentes,  Murphy,  Segur,  & Kodali,  1997).  Each  design  considerations  will  be  described  and  analyzed  in  terms 
of  the  partnership. 

The  two  administrative  design  considerations  were  grading  system  and  grouping.  Research  shows  that 
student  participation  rates  are  influenced  by  whether  participation  is  voluntary  or  required  (Wells,  1992).  In 
partnerships,  grading  should  include  a similar  emphasis  on  both  sides,  and  professors  should  collaborate  on  a 
grading  scheme  for  all  students.  In  this  partnership,  the  grading  emphasis  was  different,  with  the  TAMU  instructor 
assigning  actual  point  values  for  participation,  and  the  UP  instructor  requiring  the  activity  but  not  placing  point 
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values  on  it.  The  second  factor,  grouping,  concerns  size,  location  of  the  members,  and  composition  of  the  groups. 
Decisions  about  group  formation  are  best  made  prior  to  partnership  activities.  The  instructors  were  effective  in 
establishing  small  groups  and  combining  computer  conferencing  expertise  and  command  of  written  English. 

The  four  instructional  design  considerations  were  collaboration,  relevance,  learner  control,  and 
technological  preparation.  Collaboration  involves  a group  created  to  perform  a task  or  solve  a problem  together 
(Slavin,  1983).  Romiszowski  (1997)  asserts  that  the  most  effective  types  of  teaching-learning  techniques  to  foster 
critical  thinking  skills  are  "experiential  exercises  followed  by  interpersonal  interaction  in  small  groups,  and  with 
facilitators  to  guide  the  group  towards  useful  conclusions"  (p.  33).  Creating  and  playing  adventure  games  on  the 
Web,  for  example,  have  been  found  to  improve  foreign  language  skills  (Vilmi  & Malmi,  1996).  In  this  study,  the 
participants  did  not  share  a common  task  to  be  performed. 

Relevance  is  necessary  for  meeting  learners'  needs  and  allowing  them  to  profit  directly.  Relevance  is  related 
to  authentic  activities,  the  "ordinary  practices  of  the  culture"  (Brown,  Collins,  & Duguid,  1989).  Romiszowski 
(1997)  recommends  several  authentic  activities:  small-group  discussions,  simulation  games,  project-based  work, 
and  collaborative  problem-solving  activities.  The  TAMU-UP  partnership  might  have  benefited  from  doing  shared 
tasks  in  small  groups  rather  than  discussing  educational  technology. 

Learner  control  is  typically  characterized  by  learners  making  choices  in  the  pacing,  sequence,  and  selection 
of  instructional  materials.  Giving  learners  control  over  their  learning  environment  is  empowering,  as  Cifuentes  et 
al.  (1997)  noted  in  regard  to  pre-service  teachers  who  were  allowed  to  co-moderate  their  own  conferences.  Had  the 
students  in  the  partnership  between  TAMU  and  UP  been  given  the  opportunity  to  select  their  own  discussion 
moderators  and  share  the  moderating  responsibilities  between  sites,  they  might  have  felt  more  empowered. 

Technological  preparation  is  imperative  in  helping  students  be  competent  with  the  technology.  Hillman, 
Willis,  and  Gunawardena  (1994)  assert  that  "user-interface  interaction"  is  critical,  particularly  with  high 
technology  communication  devices.  They  suggest  methods  that  foster  comfort  with  a new  technology  like  having 
students  play  with  the  tools  in  a game-like  activity  before  using  the  tools  for  course  content.  The  emphasis  on 
technological  preparation  and  uses  of  telecommunications  should  be  similar  at  both  sites.  The  TAMU  students  had 
been  using  computer  conferencing  for  eight  weeks  by  the  time  of  the  videoconference,  whereas  UP  students  hadn't 
used  telecommunications  other  than  the  Web. 

As  Cifuentes  et  al.  explain,  "The  design  concern  for  including  collaboration  and  relevance  is  to  provide  a 
social  learning  context  in  which  students  can  work  productively  with  each  other  and  internalize  their  own 
individual  learning.  Similarly,  student  motivation  can  be  ensured  by  allowing  some  learner  control  in  the  design. 
Finally,  adequate  technological  preparation  empowers  students  to  successfully  complete  the  tasks  associated  with 
computer  conferencing  (1997,  p.  186). 

Cross-cultural  Communication 

While  the  six  design  considerations  were  tested  with  university  students  in  the  US,  they  may  also  be 
appropriate  for  cross-cultural  partnerships.  Collis  and  Remmers  (1997)  report  that  at  least  four  issues  are  related  to 
effective  cross-cultural  Web  design:  communication  and  interaction;  language;  content;  and  representation  form. 
These  four  issues,  when  interfaced  with  the  six  design  considerations,  may  hold  the  key  to  successful  cross-cultural 
partnerships.  First,  communication  and  interaction  are  easily  misinterpreted  across  cultures.  For  example,  in  cross- 
cultural  contexts  more  communication  and  interaction  are  not  necessarily  better  than  less,  and  well-structured 
communication,  moderated  by  an  individual  with  standing,  may  be  preferable  for  wider  audiences.  In  the  TAMU- 
UP  partnership,  the  two  instructors  could  moderate  the  discussions  themselves.  Second,  because  language  includes 
verbal  and  non-verbal  communication,  those  developing  cross-cultural  sites  must  be  sensitive  to  cultural 
differences  in  communication  styles.  Issues  pertaining  to  selection  of  language(s),  translation  by  multilingual 
people,  and  communication  protocols  all  present  potential  problems.  Combining  nationalities  in  project-based 
learning,  such  as  building  Web  sites  together  in  teams,  could  enhance  the  learning  environment.  Perhaps  both 
TAMU  and  UP  instructors  could  incorporate  project-based  learning  projects.  Third,  partnership  leaders  must 
choose  course  content  where  the  cross-cultural  aspects  are  of  minimal  relevance  or  are  integral  to  the  content.  Like 
the  previous  example,  developing  a Web  site  in  multi-cultural  teams  may  provide  a more  level  playing  field. 
Fourth,  representation  form  needs  to  be  considered  in  relation  to  the  use  of  visualizations  to  replace  or  supplement 
text.  Visual  resources,  while  becoming  more  widely  used  due  to  multimedia  advances,  are  subject  to  potential 
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cross-cultural  misinterpretation.  For  instance,  only  the  UP  students  had  access  to  multimedia  in  developing  their 
Web  biographies;  if  all  participants  were  able  to  be  creative,  the  bios  would  seem  less  one-sided. 


Conclusion 

This  investigation  was  based  on  TAMU  students'  conceptions  of  educational  technology  and  their 
explanations  for  the  failed  partnership.  The  investigation  led  to  the  verification  of  two  existing  models  that 
explicate  key  design  and  cross-cultural  considerations  for  use  in  cross-classroom  and  international  environments. 
The  first  model  provides  six  design  considerations:  grading  system,  grouping,  collaboration,  relevance,  learner 
control,  and  technological  preparation  (Cifuentes  et  al,,  1997),  The  second  model  provides  four  cross-cultural 
considerations:  communication  and  interaction;  language;  content;  and  representation  form  (Collis  & Remmers, 
1997).  Determining  the  delicate  balance  between  and  among  these  factors,  particularly  in  cross-cultural  settings, 
will  pave  the  way  for  identifying  key  pedagogical  techniques  in  making  meaning.  This  study  demonstrates  the 
power  of  networking  learners  from  across  the  world  to  learn  about  and  from  the  perspectives  of  each  other,  A 
TAMU  student  summed  up  the  experience  by  acknowledging  that  "instructional  technology  is  world-wide,  not  just 
here  in  America.  We  have  become  a global  society  and  every  country  is  facing  similar  technological  issues”  (T7s), 
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Abstract;  Multimedia  enables  novel  ways  of  representing  information,  most  strikingly  as 
animations,  but  we  lack  good  detailed  accounts  of  its  value  for  learning.  The  main  study 
reported  here  investigates  the  value  of  animation  for  students  aged  thirteen  and  fourteen, 
learning  the  principles  of  cardiac  circulation.  The  study  compared  the  effects  of  media  type, 
paper-based  diagram  versus  computer-based  animation,  and  of  design  of  the  learning  task, 
structured  versus  open,  in  a 2 x 2 design.  Learning  was  measured  by  completion  of  a diagram 
of  blood  flow  through  the  heart.  Statistical  analysis  showed  a significant  overall  effect  of  task 
design  but  not  of  media  type.  Further  analysis,  however,  showed  that  the  animation  produced 
distinctive  kinds  of  errors  and  a second  study  looked  in  more  detail  at  the  effects  of  working 
with  an  animation.  Results  suggest  it  has  the  effect  of  making  learners  over-confident  but  can 
ultimately  provide  some  benefits  over  static  formats. 


Introduction 

The  arrival  of  multimedia  provides  new  ways  of  representing  information  through  different  media 
formats,  such  as,  text,  audio,  graphics.  Advances  in  technology  have  enabled  diagrammatic  representation  in 
alternative  forms,  such  as  animation.  Multimedia  also  provides  the  opportunity  for  innovative  ways  of 
interacting  with  information,  not  available  with  traditional  media  formats.  A pervasive  assumption  is  that  the 
more  complex  and  varied  the  technology  the  better  the  interaction  and  the  better  the  learning.  Large  et.  al.  (1994) 
claims  “It  is  frequently  claimed  by...  producers  (and  emphasised  by  reviewers)  that  multimedia  capabilities 
enhance  educational  value”  (p.528).  However,  there  is  no  body  of  empirical  evidence  that  this  assumption  is 
generally  valid. 

Multimedia  can  be  seen  as  a particular  form  of  external  representation,  such  as  diagrams,  which 
supports  external  cognition  (Zhang  and  Norman  1994).  Diagrams  can  be  used  to  convey  "whole  processes  and 
structures,  often  at  great  levels  of  complexity"  (Winn  1987,  p.l53),  and  may  reduce  cognitive  processing,  as 
many  important  aspects  for  comprehension  are  perceptually  available  in  the  diagram  and  do  not  need  to  be 
worked  out  (Larkin  and  Simon  1987).  Diagrams  should,  therefore,  facilitate  understanding,  as  information  is 
more  explicit  and  requires  less  inferencing.  It  has  also  been  noted  that  different  representational  formats  of  the 
same  structure  “can  activate  completely  different  processes”  (Zhang  and  Norman  1994  p.  118).  Different 
processes  in  turn  may  result  in  different  conceptual  understanding.  Thus,  the  representational  format  may  be 
critical  in  the  way  comprehension  is  actualised. 

Advances  in  graphical  technology  provide  the  facility  for  dynamic  processes  to  be  represented  in 
animated  as  well  as  static  form,  with  the  assumption  that  the  ability  of  an  animation  to  show  motion  more 
explicitly  should  be  beneficial  for  learning.  However,  research  investigating  both  animated  and  static  diagrams, 
shows  no  comprehensive  evidence  of  improved  understanding,  suggesting  that  provision  of  a moving  image 
does  not  necessarily  facilitate  overall  conceptual  understanding.  Several  researchers  have  postulated  the 
complexities  of  processing  dynamic  illustration  and  multimedia  (e.g.  Kaiser  et.  al.  1992,  Hoogeveen  1997,  Large 
et.  al.  1994,  Stenning  1998).  This  research  suggests  that  animation  may  be  problematic  for  comprehension  for 
several  reasons:  use  of  animation  on  diagrams  results  in  an  unmanageable  possible  increase  of  perceptually 
available  information,  especially  where  multidimensional  dynamics  are  involved  (e.g.  Kaiser  et  al  1992);  use  of 
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multimedia  representations  can  result  in  cognitive  /memory  overload  due  to  too  much  information  (Kaiser  et.al. 
1992),  due  to  problems  in  having  to  integrate  multiple  representations  not  always  simultaneously  available  on 
the  screen  (Rogers  and  Scaife  1997,  Hoogeveen  1997).  Interestingly,  Kaiser  et  al.  (1992)  found  that  where  only 
one  dimension  of  the  dynamic  information  was  present  then  animation  facilitated  accurate  observation.  This 
suggests  that  ’parsing’  of  dynamic  events  is  important  in  facilitating  correct  dynamical  judgement. 

To  investigate  the  cognitive  value  of  graphical  representations  this  study  uses  the  ’Cognitive 
Interactivity*  framework  developed  by  Scaife  and  Rogers  (1996).  This  emphasises  the  interplay  between  internal 
and  external  representations  on  cognition  (i)  how  new  information  is  integrated  with  existing  knowledge  and  re- 
represented and  (ii)  the  identification  of  the  cognitive  benefits  and  costs  of  particular  forms  of  representations 
identifying  the  properties  of  external  representations  in  terms  of  their  computational  offloading  - the  ways  in 
which  different  forms  constrain  the  amount  of  cognitive  effort  required  to  solve  informationally  equivalent 
problems.  An  example  is  the  way  that  different  graphical  forms  of  representations  of  the  same  information  will 
limit  the  kind  of  inferences  that  are  likely  to  be  made,  thus  guiding  thinking  towards  the  concept  to  be  acquired. 


Study  1 

The  aim  of  this  study  was  to  investigate  pupil  learning  from  diagrams  using  either  a static  or  animated 
format.  Diagrammatic  representation  of  blood  flow  through  the  heart  was  used  in  both  CD  ROM  and  paper 
format,  to  compare  animated  with  static  representation  of  the  concept.  As  CD  ROM  presentation  allows  pupils 
more  freedom  in  exploring  information,  whereas  formal  teaching  situations  use  specific  task  questions,  two 
different  forms  of  task  presentation  were  used.  In  the  open  task  pupils  were  asked  to  use  the  diagram  to  find  out: 
how  the  blood  flows  in  through  and  out  of  the  heart;  about  blood  that  is  rich  in  oxygen,  and  blood  that  has  less 
oxygen  and  about  the  different  chambers  of  the  heart,  and  structured.  In  the  structured  task  pupils  were  given  a 
worksheet  with  specific  aspects  for  them  to  investigate  designed  collaboratively  with  a teacher.  Thus,  the  effect 
of  different  formats  of  the  same  graphical  representation  and  different  task  presentations  on  conceptual 
understanding  of  a dynamic  process  were  investigated.  Participants  were  taken  from  two  community  colleges 
and  one  comprehensive  school  in  Sussex.  No  pupils  had  completed  formal  teaching  on  cardiac  circulation. 


Method 

One  hundred  and  twelve  pupils  aged  thirteen  and  fourteen,  from  two  Sussex  community  colleges 
participated  in  the  study.  Fifty  three  were  male  (mean  age  = 13  years  10  months;  sd=3.2  months),  and  59  were 
female  (mean  age  = 13  years  1 1 months;  sd=3.6  months).  The  independent  variables  were  media  format  (CD 
ROM  or  static)  and  task  type  (open  or  structured).  The  dependent  variable  was  the  test  score.  A pilot  study  was 
undertaken  to  ensure  that  information  was  appropriate  and  to  formulate  task  instructions  and  the  test.  In  the  main 
study  half  of  the  participants  worked  with  information  using  the  computer  (animated)  the  other  half  worked  with 
the  paper,  (static)  format.  Half  of  each  of  these  groups  worked  from  the  ‘open’  task,  the  other  half  used  the 
structured  worksheet.  All  participants  worked  for  fifteen  minutes  on  their  allocated  task,  followed  immediately 
by  the  test.  Time  allocation  was  decided  as  a combination  of  ensuring  adequate  time  for  collecting  necessary 
information  and  restrictions  from  school  curriculum  activities.  The  teacher  allocated  pupils  who  worked  well 
together  into  pairs. 

Animated  condition:  The  ‘heart’  section  of ‘What’s  the  Secret?’  CD  ROM  from  3M  Learning  Software 
was  used.  The  programme  covered  varying  aspects  about  the  heart,  including  animated  video  presentation  of 
blood  flow.  Information  needed  for  overall  understanding  was  found  in  separate  parts  of  the  programme.  Pupils 
were  guided  to  the  relevant  areas  prior  to  beginning  their  own  investigations,  due  to  time  restrictions,  and  pupils 
could  ask  for  guidance  during  the  task  if  they  were  unable  to  find  the  information  they  needed.  Participants 
worked  in  pairs  for  fifteen  minutes,  prior  to  completing  individual  assessments. 

Static  condition:  Three  sequential  static  colour  printed  diagrams  were  taken  from  the  heart  section  of 
‘What’s  the  Secret’  CD  ROM.  This  design  ensured  coverage  of  information  was  apparently  equivalent  to  the  CD 
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Figure  1 : Example  of  test  diagram. 
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ROM  information  needed  to  complete  the  test.  One  class  was  given  the  ‘open’  task,  the  other  used  the  structured 
worksheet.  Participants  worked  with  the  information  in  pairs  for  fifteen  minutes,  prior  to  completing  individual 
assessments. 

Post  test:  This  consisted  of  a black  and  white  version  of  the  diagram  taken  from  the  CD  ROM.  Pupils 
had  to  complete  the  diagram  within  five  minutes,  the  task  being  to  use  arrows  to  show  where  the  blood  enters, 
flows  through,  and  leaves  the  heart,  using  conventional  colouration,  a blue  pen  for  deoxygenated  blood  and  a red 
pen  for  oxygenated  blood.  Pupils  were  also  required  to  put  the  following  eight  labels  - ‘to  body’,  ‘from  body’, 
‘to  lungs’,  ‘from  lungs’,  ‘right  atrium’,  ‘left  atrium’,  ‘right  ventricle’,  ‘left  ventricle’  - in  the  appropriate  places 
(see  figl).  Use  of  information  and  notes  was  not  permitted  for  the  test. 


Results 

Scoring  Method 

Four  different  aspects  of  the  diagram,  relating  appropriately  to  the  task  and  the  structured  work  sheet, 
were  chosen  for  analysis.  One  point  was  allocated  for  each  of  the  following;  (i)  oxygenated  and  deoxygenated 
blood  represented  in  separate  sides  of  the  heart;  (ii)  depicting  different  types  of  blood  in  the  correct  sides;  (iii) 
correct  labelling  using  all  eight  labels;  (iv)  correct  depiction  of  direction  of  blood  flow  using  arrows,  or  from 
labelling  if  arrows  were  not  used.  Subjects  could  therefore  receive  a score  from  0 to  4.  An  interobserver 
reliability  score  of  95%  was  achieved. 

The  data  were  analysed  to  determine  whether  media  presentation  or  task  type  affected  learning  about 
blood  flow  through  the  heart,  and  to  determine  the  kinds  of  inferences  made  from  the  diagrams.  The  data  are 
presented  in  three  sections:  analyses  of  the  effects  of  both  variables  on  the  overall  test  score;  analyses  of  the 
effects  of  variables  on  selected  areas  of  the  test  from  which  scores  were  obtained;  and  analyses  of  errors  made, 

1 ,  Analysis  of  both  variables  on  the  overall  test  score,  A two  way  independent  analysis  of  variance 
(ANOVA)  was  performed  on  the  test  scores.  No  significant  main  effect  of  media  format  was  found.  There  was  a 
significant  main  effect  of  task  type  (F=5,08;  df  =1,1;  p<0.02),  Scheffe  (p<  0,0215);  those  using  a structured 
worksheet  performed  significantly  better  than  those  using  an  open  task,  regardless  of  media  presentation.  No 
interaction  was  found  between  the  two  variables. 


2,  Analysis  of  the  effects  of  variables  on  selected  areas  of  the  test  A series  of  two-tailed  chi-square 
analyses  were  performed  separately  on  the  four  selected  parts  of  the  test  to  determine  whether  media 
presentation  or  task  type  affected  performance  in  particular  areas  of  the  test.  No  significant  differences  were 
shown  in  any  section:  Blood  in  separate  sides,  chi-square  0.017,  d.f  = 1,  not  significant;  Blood  in  correct  sides, 
chi-square  = 0.013,  d.f  = 1,  not  significant;  Labels,  chi-square  = 0.505,  d.f  = I,  not  significant;  Directionality, 
chi-square  = 0.108,  d.f  = 1,  not  significant. 

3.  Analysis  of  errors  made.  Despite  the  absence  of  overall  significant  differences,  an  inspection  of 
individual  scores  showed  that  pupils  made  more  errors  with  labelling  and  direction  of  blood  flow.  Analysis  on 
these  aspects  revealed  differences  in  errors  of  interpretation  of  blood  flow.  The  two  most  noteworthy  errors 
were;  showing  blood  to  flow  into  the  heart  deoxygenated  but  to  flow  out  oxygenated  or  vice  versa;  and 
perceiving  the  blood  to  enter  the  heart  through  the  ventricles  and  leave  via  the  atria.  A chi-square  was  performed 
to  determine  whether  pupil  interpretations  were  affected  by  the  two  independent  variables,  and  was  found  to  be 
significant  for  ‘in  deoxygenated/out  oxygenated’  (chi-square  = 6.00;  d.f  = 1;  Fisher  exact  p<0.0222),  and 
approaching  significance  for  ‘blood  entry  through  ventricles’  (chi-square  = 4.03;  d.f  = 1;  Fisher  exact  p<0.0608). 


Discussion 

The  results  showed  no  significant  difference  in  overall  test  performance  between  computer  and  paper 
diagrams.  However,  significant  differences  of  task  type  were  evident.  As  diagrams  contain  large  amounts  of 
information  in  one  representation  the  learner  must  decide  which  aspects  are  important  and  in  which  order  to 
’read’  information.  Sequencing  the  information  may  reduce  confusion  and  enable  focus  of  attention  on  relevant 
aspects.  The  worksheet  may  have  provided  this  by  guiding  pupils  to  specifically  focus  on  smaller  components  of 
information  and  to  order  salient  related  pieces  of  information. 


6G0 


Although  analysis  of  accurate  depiction  of  each  of  the  four  chosen  aspects  showed  no  significant 
differences,  pupils  in  all  conditions  made  most  errors  with  blood  flow  and  labelling.  This  suggests  that  these 
concepts  were  less  clear  from  the  diagrams  or  were  more  conceptually  complex  to  grasp.  Analysis  of  these 
aspects  in  terms  of  errors  made  showed  significant  differences  across  conditions.  Representation  of  blood 
flowing  into  the  heart  oxygenated  and  out  deoxygenated,  or  vice  versa  was  more  prevalent  in  the  animated 
condition.  This  seems  to  attribute  to  the  heart  a similar  functional  role  as  the  lungs.  Depiction  of  blood  flowing 
in  via  the  ventricles  and  out  via  the  atria  was  predominant  in  the  static  condition. 

These  findings  may  be  explained  by  aspects  of  graphical  representation,  in  particular  re- representation 
and  computational  offloading  (Scaife  and  Rogers  1996).  Re-representation:  the  results  suggest  that  different 
representations  not  only  influence  whether  problem  solving  is  easier  or  more  difficult,  but  also  generate  different 
understanding,  confirming  that  different  representational  formats  of  a common  structure  “can  activate 
completely  different  processes''  (Zhang  and  Norman  1 994,  p.  1 1 8).  In  this  study  the  animated  version  shows 
simultaneous  movement  of  all  aspects  of  blood  flow.  Multidimensional  dynamics  are  not  sequenced  into 
separate  dynamical  processes  (as  advocated  by  Kaiser  et,  al.  1992),  and  result  in  a confused  presentation  of 
movement.  Thus,  ‘direction'  could  be  misconstrued  as  blood  flowing  in  one  colour  and  out  another  colour, 
resulting  in  the  confusion  of  oxygenated  in,  deoxygenated  out.  The  static  diagram  did  not  explicitly  show 
direction  of  blood  flow,  which  had  to  be  inferred  from  the  labels  (fig.l).  Without  attendance  to  these  labels 
direction  could  also  be  misconstrued. 

Computational  offloading;  overall,  improved  understanding  of  the  circulation  in  the  heart  was  not 
apparent  from  the  use  of  animation,  supporting  evidence  that  multidimensional  problems  are  not  aided  by 
animation  (Kaiser  et.  al.  1992).  Too  much  movement  of  different  parts  and  different  types  of  blood  made  it 
difficult  to  see  where  each  element  was  going  and  what  each  part  was  doing.  Pupils  made  comments  such  as 
‘stop  moving’  and  used  the  diagram  in  stationary  format  to  work  out  the  blood  flow.  Although  use  of  animation 
may  be  assumed  to  increase  perceptually  available  information  thus  reducing  cognitive  processing,  cognitive 
effort  and  ‘working  out’  of  dynamics  may  be  reduced  to  such  a degree  that  learning  is  ineffectual. 

Although  animation  may  be  more  explicit  “our  perceptual  appreciations  do  not  spontaneously  form  the 
basis  of  our  conceptual  understanding  of  dynamics”  (Kaiser  et.  al.  1992,  p.686).  Anderson  (1995)  proposes  that 
conceptual  knowledge  is  based  on  the  meaning  of  a representation.  Errors  in  pupil  learning  suggest  that 
comprehension  of  the  function  of  the  heart  or  reasoning  behind  the  blood  flow  was  lacking.  This  information 
was  not  explicit  in  the  information  in  either  presentation  format,  therefore  the  meaning  of  the  representation  was 
not  evident.  If ‘meaning’  is  important  in  overall  conceptual  understanding  then  awareness  of  the  rationale  behind 
the  concept  may  constrain  certain  aspects  of  the  process.  Knowing  the  function  of  the  heart,  or  each  part  of  the 
heart,  would  serve  to  constrain  the  inferences  or  information  gleaned  from  the  diagram.  For  example,  if  the 
function  of  the  ventricles  to  ‘pump’  blood  out  of  the  heart  is  made  explicit,  then  this  constrains  the  way  that  the 
blood  is  likely  to  flow,  consequently  affecting  diagram  interpretation.  The  context  and  function  of  a represented 
structure  may  critically  influence  the  effectiveness  of  a graphical  representation.  Not  only  may  it  be  important  to 
investigate  the  external  graphical  constraints  (Scaife  and  Rogers  1996)  of  a representation,  but  also  the  internal 
cognitive  constraints. 

These  aspects  confirm  cited  complexities  of  animation  and  multimedia.  Stenning  (1998)  suggests  that 
such  complexity  may  be  a result  of  the  following  characteristics  of  animation;  i)  evanescence,  the  problem  of 
memory  load  is  enhanced  by  the  'transient  form'  of  animation,  which  means  that  all  sequences  of  movement  need 
to  be  held  in  memory  to  integrate  with  new  pieces  of  information  to  understand  the  process;  ii)  control,  the 
evanescent  form  means  that  learners  are  unable  reaccess  pieces  of  information  as  they  are  no  longer  perceptually 
available.  Learners  therefore  have  no  control  over  what  information  is  viewed,  or  for  how  long,  as  information  is 
continuously  passing  by.  Large  et  al  (1994)  proposes  that  "animation  improves  performance  when  used  as  an 
interactive  dynamic,  continuously  changing  depending  on  subject  input"  (p.  527,  emphasis  added);  iii) 
expressiveness;  Animation  demands  a more  expressive  representation,  as  all  aspects  of  a process  must  be  shown 
simultaneously,  making  focus  on  one  aspect  difficult.  So:  how  do  pupils  work  with  animated  and  static 
representations.  Do  these  properties  of  animation  complicate  learning?  Do  static  diagrams  give  clearer 
understanding? 


Study  2 

Study  1 demonstrated  that  different  representational  formats  of  the  same  process  result  in  different 
understanding,  and  suggests  that  learning  a dynamic  concept  is  not  necessarily  facilitated  by  animation  alone, 
comprehension  being  dependent  on  other  factors.  Study  2 set  out  to  examine  more  specifically  the  differences  in 
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ways  that  pupils  worked  with  animation  and  static  diagrams  of  blood  flow,  which  parts  they  found  hard  or  easy 
to  understand,  and  investigate  cited  claims  about  animation. 


Method 

Twenty  two  pupils  aged  13  and  14  years  (mean  age  = 14  years  1 month)  participated  in  the  study. 
Thirteen  were  male  and  nine  female.  This  study  used  verbal  protocols  of  pupils  working  in  pairs  (as  allocated  by 
the  teacher)  either  with  a computer  presented  animated  diagram  or  with  a static  paper  diagram.  All  participants 
worked  with  the  researcher  for  approximately  half  an  hour  on  their  allocated  information,  but  those  using  the 
static  diagram  were  also  given  an  opportunity  to  briefly  work  with  the  animated  version,  to  investigate  any 
further  information.  All  sessions  with  pupils  were  video-recorded. 

Animated  condition:  The  same  CD  ROM  programme  was  used  as  in  study  1,  but  pupils  worked  only 
with  the  animated  diagram.  Text  information  was  obscured  and  introduced  only  if  pupils  found  it  too  difficult  to 
retrieve  information  from  the  diagram  alone.  Pupils  interacted  with  the  programme  only  by  clicking  to  view  the 
animation  again. 

Static  paper  condition:  This  consisted  of  a single  diagram  from  study  1,  initially  without  accompanying 
text.  Pupils  were  provided  with  pens  to  make  notes  on  the  information  if  they  chose.  One  of  each  pair  was  given 
a few  minutes  to  study  the  diagram  alone.  Time  allowed  was  dependent  on  the  individual  pupil,  but  generally  did 
not  exceed  five  minutes.  Pupil  1 was  then  required  to  explain  to  pupil  2 what  the  diagram  was  showing.  The 
interview  proceeded  differently  with  each  pair  according  to  the  information  relayed  back  from  the  pupils.  Pupils 
participated  in  discussions  between  themselves.  The  researcher  intervened  only  when  pupils  were  unable  to 
glean  any  more  information,  by  asking  open  questions  or  more  specific  questions  depending  on  pupil  progress. 


Results  and  Discussion 

Preliminary  analysis  confirms  previous  caution  that  animation,  can  be,  as  in  this  case,  too  complex  - 
being  too  fast,  lacking  in  learner  control,  and  impeding  attention  focus,  and  suggests  that  animation  alone  may 
be  insufficient  to  aid  comprehensive  learning  of  a dynamic  process.  Animation  appeared  to  generate  artificially 
high  confidence  levels,  increasing  complexity  and  preventing  learners  from  paying  appropriate  attention  to  the 
information.  For  example,  children  refused  the  opportunity  to  view  the  animation  more  than  once  despite 
evidence  of  miscomprehension.  This  suggests  that  animation  provides  inappropriate  computational  offloading 
precluding  the  possibility  of  effective  learning,  by  giving  the  learner  false  certainty  of  gained  knowledge.  Only 
when  learners  had  to  explain  the  process  or  were  asked  specific  questions,  were  they  aware  of  their  lack  of 
understanding.  Extemalisation  of  knowledge  may  be  an  important  part  of  the  learning  process  (Chi  1998),  with 
diagrammatic  learning  as  well  as  textual.  Only  one  pupil  was  able  to  give  a full  description  of  blood  flow  after 
initial  viewing.  Two  pairs  made  inaccurate  interpretations  of  flow,  perceiving  blood  flowing  in  one  colour  but 
flowing  out  the  other.  This  suggests  that  they  perceived  oxygen  exchange  taking  place  inside  the  heart. 
Moreover,  these  pupils  stated  that  the  function  of  the  heart  was  to  ‘clean  the  blood’  i.e.  to  oxygenate  the  blood. 
This  equates  with  errors  found  in  study  1,  also  suggesting  that  meaning  and  interpretation  of  a representation 
may  be  related. 

Inability  to  focus  attention  was  apparent  when  pupils  were  asked  questions  about  specific  aspects.  For 
example,  all  pupils  primarily  focused  on  blood  flow.  After  one  more  viewing  four  of  the  pairs  noticed  the  heart 
contracting.  No  pupils  spontaneously  noticed  valve  action  and  had  to  be  specifically  directed  to  this  component. 
Even  then  three  pairs  needed  to  watch  the  animation  twice  more  to  procure  appropriate  information.  Pupils 
themselves  expressed  difficulty  with  focusing,  suggesting  that  directing  attention  to  pertinent  pieces  of 
information  is  important.  With  each  viewing  of  the  animation  pupils  were  able  to  explain  blood  flow  in  more 
detail,  and  combine  this  information  with  contraction  and  valve  action.  Thus,  they  appear  to  take  in  small 
amounts  of  information  at  one  time,  gradually  combining  their  knowledge  with  previous  pieces  of  information, 
which  highlights  the  importance  of  parsing  or  sequencing  knowledge  acquisition. 

Despite  some  disadvantages,  animation  may  provide  benefits  over  use  of  static  diagrams  alone.  When 
attention  was  focused  and  pupils  were  directed,  animation  imparted  more  information  about  the  dynamics  of  the 
system  than  pupils  could  obtain  from  a static  diagram,  for  example,  they  could  construct  clearer  models  about 
how  the  valves  worked  and  how  the  heart  'pumped'.  However,  in  both  of  these  studies  interaction  with 
information  was  severely  limited.  Future  research  is  continuing  to  investigate  the  effects  of  different  levels  of 
interaction  with  such  diagrams  on  learning,  for  example,  slowing  down  the  animation,  stopping  facilities. 
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focusing  attention  on  salient  aspects  of  information  in  an  appropriate  order,  the  effect  of  construction  and 
simulation  on  comprehension  of  such  a system.  This  research  will  also  investigate  pertinent  aspects  of  learning, 
such  as  the  effects  of  extemalisation  by  interacting  with  information. 
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Abstract:  In  this  paper,  we  describe  our  efforts  to  introduce  desktop  videoconferencing  into 
schools.  Our  particular  project  focus  is  the  support  of  distributed  collaborative  learning  between 
K-12  science  classrooms.  We  outline  an  educational  argument  for  the  use  of  videoconferencing 
and  go  on  to  present  a case  study  of  the  educational  videoconferencing  experiences  in  our  project. 
We  particularly  identify  and  describe  the  kinds  of  technical  difficulties  that  can  be  encountered 
and  the  way  we  overcame  most  of  these  problems.  Once  the  students  began  to  use  the 
technology  meaningfully  to  support  inter-school  collaborative  learning  a whole  host  of  interesting 
educational,  organizational  and  other  issues  began  to  emerge.  We  highlight  the  most  important 
of  these  issues.  Finally,  we  consider  some  of  the  issues  influencing  teacher  acceptance  of 
videoconferencing. 


Introductio 

We  consider  K-12  education  to  be  an  important  but  largely  unexplored  area  for  the  application  of 
collaborative  technology.  Videoconferencing  has  been  around  for  over  three  decades  but  in  that  time  it  has  enjoyed 
little  commercial  success  or  widespread  acceptance.  In  this  paper,  we  report  on  our  efforts  to  introduce  low-cost 
Internet-based  desktop  videoconferencing  (DVC)  into  K-12  science  classrooms  as  a basis  for  Computer  Supported 
Collaborative  Learning  (CSCL).  Desktop  videoconferencing  does  not  use  specially  equipped  rooms  or  studios  to 
facilitate  telelectures;  it  uses  the  desktop  personal  computer  as  the  basis  for  synchronous  multimedia 
communication  with  remote  others.  There  has  been  little  published  research  into  the  introduction,  use,  and 
acceptance  of  DVC  to  support  distributed  collaborative  learning  in  and  between  K-12  classrooms.  Most  research  in 
the  general  area  of  educational  videoconferencing  has  concentrated  on  distance  education  (usually  at  university 
level).  We  make  a clear  distinction  between  CSCL  and  distance  education.  Whereas,  CSCL  emphasizes 
supporting  peer  interaction,  distance  education  is  more  concerned  with  supporting  instructor-student  interaction. 
CSCL  is  a relatively  new  area,  and  those  working  in  this  area  are  still  involved  in  the  process  of  defining  its 
boundaries  (Bannon  1995).  It  includes  collaborative  learning  through  a computer,  around  a computer,  and  even 
w>ith  a computer.  Our  project  focus,  and  the  focus  of  this  paper,  is  on  collaborative  learning  through  networked- 
computers,  but  because  most  classroom  computers  have  multiple  users,  there  is  usually  also  an  element  of 
collaboration  around  a computer  involved. 


The  Educational  Value  of  Videoconferencing 

We  believe  there  are  sound  educational  reasons  for  incorporating  desktop  videoconferencing  into  everyday 
classroom  activities.  Over  the  last  decade  or  so,  there  has  been  a growing  awareness  of  the  link  between  learning 
and  context.  Learning  is  now  widely  seen  as  inherently  social  and  taking  place  within  a community  of  learners 
(Brown  & Campione  1996).  Individual  cognitive  development  is  largely  shaped  and  determined  by  the 
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characteristics  and  potential  of  the  various  communities  of  which  one  is  a member  and  by  the  role  or  identity  one 
assumes  within  these  communities.  Unfortunately,  the  kinds  of  mental  activities  encouraged,  developed,  and 
valued  in  school  communities  appear  to  be  quite  different  from  those  valued  outside  of  school  (Resnick  1 987).  If 
we  consider  schools  as  communities  of  practice,  the  primary  practice  appears  to  be  schooling  (i.e.  activities  and 
skills  that  only  have  meaning  within  schools).  Cognition  outside  of  schools  is  typically  socially  distributed,  linked 
closely  to  tool  manipulation,  situation  specific,  and  involves  contextualized  reasoning  or  reflection  in  action.  In 
contrast,  school-developed  knowledge  and  skills  are  rarely  capable  of  being  meaningfully  applied  in  other  contexts. 
In  other  words,  school  learning  is  often  inauthentic. 

Communication  technology  and  associated  activities  afford  the  opportunity  to  "virtually"  extend  school- 
based  communities  of  learning  (Eales  & Byrd  1997).  Authentic  activities  are  often  collaborative  and  children 
benefit  when  their  learning  involves  meaningful  interaction  with  their  peers.  Learning  can  also  be  enhanced  when 
children  engage  in  learning  activities  with  students  of  other  ages.  With  younger  children,  they  reinforce  their  own 
learning  through  having  to  articulate  and  demonstrate  their  knowledge,  while  with  older  children,  they  receive 
assistance  from  a more  advanced  learner's  perspective.  School-based  educational  activities  can  also  be  stimulated 
and  enhanced  by  contact  with  domain  experts  from  outside  the  school  system,  for  example,  scientists, 
mathematicians,  or  writers.  These  representatives  of  real-world  communities  of  practice  not  only  provide  expert 
advice,  they  also  act  as  role  models  and  guides.  Generally,  "outsiders",  experts,  local  community  members,  or 
parents  can  be  a source  of  authentic  influence  on  school  practices,  which  can  help  to  develop  more  meaningful  and 
robust  cognitive  abilities  among  students. 

Various  technologies,  such  as  the  World  Wide  Web,  e-mail,  and  interactive  chat  can  contribute  to 
extending  learning  communities.  But  videoconferencing,  particularly  in  its  support  of  social  presence  (Short  et  al 
1976)  and  mutual  knowledge  (Krauss  & Fussell  1990),  may  be  a uniquely  suited  technology  for  extending  the 
learning  boundaries  of  the  classroom.  Daly-Jones  et  al.  (1998)  found  that  when  more  than  one  person  is  at  each 
end  of  a conference,  video  makes  a significant  difference  to  the  fluency  of  interaction  and  awareness  of  remote 
partners.  However,  the  effective  use  of  videoconferencing  in  authentic  collaborative  learning  situations  still 
remains  a largely  unexplored  area.  Our  project  seeks  to  contribute  to  the  development  of  understanding  in  this 
area. 


Early  Experiences  with  Videoconferencing 

The  context  for  our  investigation  is  a large  interdisciplinary  project  group  from  Virginia  Tech  and  the 
Montgomery  County  Public  Schools  (Virginia)  supported  by  a major  award  from  the  U.  S.  National  Science 
Foundation.  The  Learning  in  Networked  Communities  (LiNCj  project  seeks  to  leverage  the  network  infrastructure 
brought  to  the  County  by  the  Blacksburg  Electronic  Village  (BEV)  (Carroll  & Rosson  1996).  A recent  report 
(Elswick  1998)  describes  Blacksburg  as  "America's  most  wired  town",  with  60%  of  residents  and  70%  of  businesses 
currently  on-line.  As  part  of  the  infrastructure  support  for  the  BEV,  all  the  schools  in  the  county  are  connected  to 
the  Internet  by  T1  ethemet  lines  (capable  of  transmitting  data  at  around  1.5  million  bps). 

Our  particular  interest  is  the  development  of  network  support  for  middle  and  high  school  science  education 
with  a special  emphasis  on  collaborative  learning.  Apart  from  our  interest  in  educational  videoconferencing  we 
have  also  undertaken  extensive  research  into  the  participatory  design  of  educational  software  and  the  development 
of  Java-based  tools  for  collaborative  learning.  The  project  members  include  four  science  teachers  from  four 
different  schools,  two  high  schools  and  two  middle  schools.  One  of  the  high  schools  and  one  of  the  middle  schools 
are  together  on  the  same  campus  in  a rural  part  of  the  county.  The  other  two  schools  are  in  the  Town  of  Blacksburg 
and  are  approximately  one  mile  (1.6  km)  apart.  The  distance  between  the  rural  and  the  town  schools  is 
approximately  12  miles  (19  km).  DVC  has  been  used  to  support  a variety  of  different  interactions,  including  peer 
interactions  between  students  at  different  middle  schools,  cross-age  interactions  between  middle  school  students 
and  high  or  elementary  school  students,  and  expert  or  mentor  interactions  between  middle  school  students  and  non- 
teacher adult  mentors. 

Our  videoconferencing  experiences  in  the  LiNC  project  can  be  divided  into  two  quite  clearly  defined  phases. 
Our  early  experiences  (phase  1),  although  they  demonstrated  a potential  interest  from  teachers  and  students  were 
dogged  by  technical  problems.  Our  later  experiences  (phase  2)  have  demonstrated  that  most  of  the  technical 
problems  can  be  solved  allowing  us  to  begin  to  study  the  educational,  organizational  and  other  implications  of 
videoconferencing  in  the  classroom. 

Our  early  experiences  using  videoconferencing  in  the  classroom  demonstrated  the  potential  of  the  technology 
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but  also  highlighted  a number  of  significant  problems.  On  the  one  hand,  students  were  highly  motivated  towards 
videoconferencing  and  the  teachers  felt  that  it  was  a positive  educational  experience,  while  on  the  other  hand, 
technical  problems  abounded  and  clearly  interfered  with  the  quality  of  the  educational  experience.  These 
conflicting  reactions  are  summed  up  by  the  comments  from  one  middle  school  student: 

"Videoconferencing  is  a lot  of  fun.  It's  almost  like  sitting  in  the  same  room  with  the  person  that  you're 
talking  to.  But  I do  think  that  it  could  be  better.  Not  only  the  computer,  but  things  like  volume  or  the 
background  noise,  the  speed  at  which  the  sight  and  sound  come,  and  the  amount  of  time  we  were  on  the 
computer." 

Reliability  Problems  - Our  early  videoconferencing  sessions  between  students  at  different  schools  often 
involved  a number  of  system  crashes.  We  had  tested  the  hardware,  software,  and  educational  activities  beforehand 
in  the  classrooms,  but  a condition  that  only  occurred  during  actual  school  classes  caused  the  computers  to  crash. 
After  every  crash,  the  computers  would  have  to  be  "hard-rebooted",  that  is,  turned  off  and  then  turned  on  again. 
Quite  apart  from  the  obvious  loss  of  time  (around  5 minutes)  involved  in  every  crash  and  reconnection,  such 
incidents  severely  disrupted  the  flow  of  communication  and  concentration  of  the  students.  On  this  particular 
system,  the  first  indication  of  an  impending  crash  was  the  freezing  of  the  received  video  image,  the  audio 
connection  would  be  lost  a short  time  afterwards.  This  allowed  just  enough  time  for  the  students  to  transmit  a 
distress  message  usually  along  the  lines  of  "your  faces  are  frozen".  Not  surprisingly  the  students’  reactions  to 
system  breakdowns  were  disappointment  and  disillusionment.  During  one  of  the  breaks  in  communication  caused 
by  a crash,  one  high  school  student  said  to  another,  '7  think  it  would  be  easier  to  go  over  there  in  person.  ” 

Audio  Problems  - Audio  quality  was  a constant  problem.  Quality  was  variable  but  usually  poor.  Sometimes  a 
system  crash  resulted  in  better  audio  quality  when  reconnection  was  achieved.  The  particular  videoconferencing 
system  we  were  using  lacked  any  way  of  adjusting  the  audio  levels.  Getting  closer  to  the  speakers  and  microphone 
(as  well  as  shouting)  became  the  only  method  of  varying  the  levels  of  sent  or  received  audio.  Audio  levels  were 
such  a constant  problem  that  many  of  the  groups  assigned  a "designated  listener"  to  press  their  ear  against  one  of 
the  computer  speakers  and  then  relay  the  incoming  audio  to  the  other  students  in  the  group.  Another  response  to 
poor  audio  quality  was  to  move  communication  to  the  chat  (text)  channel.  Unfortunately,  classrooms  appear  to  be 
inherently  noisy  places  often  with  poor  acoustic  properties.  Unlike  audio  quality,  the  quality  of  the  video  did  not 
seem  to  be  an  important  issue  for  the  students. 

The  ''Do  You  See  What  I See"  (DYSWIS)  Problem  - Activities  between  students  using  videoconferencing  often 
involve  coordinated  use  of  applications  running  on  the  two  computers.  In  the  particular  system  we  were  using  there 
was  no  way  for  one  group  of  students  to  see  or  control  what  was  on  the  other  group's  computer  screen.  This  leads 
to  a common  situation  where  one  group  has  to  describe  to  the  other  group  just  what  they  should  have  on  their 
screen.  In  reply  the  other  group  then  has  to  try  to  describe  what  they  actually  have  on  their  screen.  Students  were 
often  not  familiar  with  the  names  of  programs  or  the  terms  for  screen  features  (e.g.,  icon,  dialog  box).  In  such 
circumstances  a common  reaction  is  to  pick  up  the  camera  and  point  it  at  the  screen. 

Camera  Framing  Problems  - The  standard  desktop  videoconference  cameras  are  naturally  built  for  office  use. 
If  a camera  is  placed  on  the  top  of  the  computer  monitor,  it  captures  the  image  of  a seated  user  quite  well. 
However,  in  the  classroom,  computers  often  have  two,  three,  or  more  users.  Students,  either  by  design  or  accident, 
start  to  disappear  from  the  transmitted  video  image  as  the  communication  session  progresses.  Sometimes,  all  that 
can  be  seen  of  a group  of  students  is  perhaps  an  ear  and  an  elbow. 


Overcoming  the  Technical  Problems 

Our  early  experiences  using  desktop  videoconferencing  in  the  classroom  were  often  depressing  and 
frustrating  for  students,  teachers,  and  researchers  alike.  Clearly,  without  project  motivation  there  was  little 
likelihood  of  the  teachers  incorporating  videoconferencing  into  their  normal  educational  practices.  However,  these 
early  experiences  did  allow  us  to  clearly  identify  the  principal  practical  problems  related  to  using  this  technology  in 
classrooms.  Before  we  began  the  second  phase  of  videoconferencing  we  tested  a number  of  different 
videoconferencing  programs,  cameras,  video  capture  cards,  and  microphones  in  different  settings.  Each  of  the  four 
classrooms  in  the  project  is  now  equipped  with  5 computers  (200Mhz  Pentium  PC’s).  Three  of  the  computers  in 
each  classroom  are  equipped  for  desktop  videoconferencing.  This  involves  the  fitting  of  a video  capture  card  and 
the  provision  of  a camera,  a microphone,  and  videoconferencing  software  (Microsoft  NetMeeting  2.1).  We  also 
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have  computers  at  the  university  equipped  for  videoconferencing.  These  can  be  used  for  communication  between 
experts  or  mentors  at  the  university  and  the  classrooms,  or  for  project  personnel  to  communicate  with  teachers. 
Most  of  our  videoconferencing  has  been  point-to-point,  but  we  are  currently  experimenting  with  multi-point 
conferences  in  some  instances. 

The  audio  quality  and  audio  control  in  our  second  phase  DVC  system  has  been  extremely  good.  However, 
classrooms  can  still  be  noisy  places  and  background  noise  is  still  a problem.  To  address  this  problem  we  have 
provided  inexpensive  unidirectional  microphones  with  a very  short  effective  pick-up  range.  The  disadvantages  of 
these  microphones  are  that  they  have  to  be  held  in  the  hand  close  to  the  mouth.  This  reduces  the  ability  for 
someone  using  the  microphone  to  control  the  computer  with  two  hands.  It  also  means  that  whoever  holds  the 
microphone  can  monopolize  the  communication.  On  the  other  hand  these  small  inexpensive  microphones  are 
remarkably  effective  at  cutting  out  background  noise. 

The  "Do  You  See  What  I See"  problem  has  also  been  solved  by  the  DVC  software  we  are  now  using. 
Students  can  now  share  any  application  with  other  students,  for  example,  show  them  web  pages  or  word  processing 
documents.  Students  can  also  collaborate  while  using  an  application,  which  means  that  control  can  pass  back  and 
forth  between  the  collaborating  parties.  Students  also  use  the  collaborative  whiteboard  and  text  chat,  and  pass 
information  between  schools  by  point-to-point  file  transfer. 

The  only  technical  problem  that  still  remains  an  issue  is  camera  framing.  While  most  of  the  groups  have  3 
or  4 members  the  number  of  students  appearing  in  the  video  window  is  typically  around  1 .25.  Ideally  we  would 
like  a wide-angle  camera  that  can  still  sit  on  the  top  of  the  monitor  (the  only  logical  place  to  position  the  camera). 
We  have  experimented  with  the  use  of  the  video  image  from  our  wide-angle  video  cameras  used  for  data  capture  as 
feed  into  the  DVC  transmitted  video  image.  This  tends  not  to  provide  very  much  detail  and  can  not  of  course  be 
considered  a permanent  solution.  We  continue  to  experiment  with  different  cameras  and  camera  positions  in  an 
attempt  to  improve  this  situation.  We  have  also  experimented  with  a hand-held  camera  for  close-ups  of  student 
models  and  other  physical  objects.  This  involves  an  additional  camera  on  a long  cable  connected  to  the  computer 
via  a switch-box,  allowing  students  to  switch  between  the  monitor-mounted  camera  and  the  hand-held  camera. 

Computer  systems  used  in  schools  have  to  be  more  than  just  reliable;  they  need  to  be  robust  to  withstand 
intentional  and  unintentional  misuse  by  students.  The  revised  hardware- software  combination  used  for 
videoconferencing  in  phase  2 of  the  LiNC  project  has  proved  to  be  remarkably  reliable  and  robust  and  has  solved 
most  of  the  previous  technical  problems.  System  crashes  are  now  extremely  rare.  Students  now  quite  happily  use 
DVC  with  little  support  or  intervention  from  project  personnel,  and  confidence  in  DVC  technology  has  even 
developed  among  the  teachers  again. 


Beyond  the  Technical  Problems 

Our  initial  phase  1 videoconferencing  experiences  in  the  project  forced  us  to  focus  on  the  various  technical 
problems.  In  phase  2 of  the  project  the  technology  has  now  become  largely  transparent  for  the  students  allowing 
them  to  concentrate  on  their  education  activities.  Below  a certain  technical  threshold  the  student  discourse 
appeared  to  be  mainly  concerned  with  the  technology  itself,  but  above  this  threshold  users  concentrated  on  the  tasks 
in  hand.  The  significant  reduction  in  technical  problems  has  allowed  us  to  begin  to  focus  on  the  many  interesting 
educational,  organizational  and  other  issues  now  emerging  from  the  project.  We  highlight  some  of  the  most 
immediately  apparent  issues  in  this  section. 

Around  the  same  time  that  we  changed  the  videoconferencing  technology,  the  teachers  involved  in  the 
project  initiated  a change  in  educational  direction.  Previously,  our  use  of  technology  in  the  classroom  had 
primarily  focused  on  one-off  “experimental”  situations  occurring  every  few  months.  The  teachers  felt  that  an 
ongoing  collaborative  project-based  approach  would  provide  a more  meaningful  basis  for  considering  the 
introduction  of  videoconferencing  and  other  network-based  tools  into  the  classrooms. 


Educational  Issues 

Student  motivation  is  an  important  factor  in  education.  Most  students  involved  in  our  project  show  a 
considerable  interest  in  videoconferencing  in  the  classroom.  However,  some  students  show  an  active  interest  in 
videoconferencing  (they  want  to  start  communicating  as  soon  as  possible)  whereas  other  students  are  only  passively 
interested  (they  like  to  observe  but  not  take  part  in  the  interaction). 
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Perhaps  the  most  significant  educational  issue  to  emerge  from  our  introduction  of  DVC  into  the  classroom 
was  that  many  of  the  most  active  and  competent  DVC  users  were  what  might  be  termed  “average”  students.  The 
experienced  teachers  in  the  project  first  highlighted  this  characteristic  of  videoconferencing.  These  students  are 
often  hampered  in  school  activities  by  poor  literacy  skills.  For  example,  one  middle  school  student  when  asked 
which  method  of  communication  he  preferred  replied:  ‘‘The  video  [DVC],  because  you  actually  get  a chance  to  see 
and  talk  to  the  person  rather  than  spending  a lot  of  time  typing,  ” Videoconferencing  introduces  a new  form  of 
communication  into  the  classroom  which  requires  new  skills.  Many  of  those  that  demonstrated  competency  in  this 
area  were  students  who  normally  do  not  get  the  opportunity  to  excel  in  the  classroom.  Student  motivation 
developed  during  videoconferencing  also  appeared  to  be  transferred  to  areas  where  literacy  skills  are  more  central. 
For  example,  students  coordinated  videoconferencing  sessions  via  e-mail  messages  and  presented  their  final  project 
reports  in  the  form  of  web  pages. 

Not  all  students  are  comfortable  with  videoconferencing.  Most  of  our  videoconferencing  sessions  involved 
middle  school  students.  These  students  are  around  1 3 years  old  and  this  can  be  a difficult  time,  particularly  with 
regard  to  concerns  about  self-image.  One  feels  that,  ideally,  many  students  would  like  to  be  able  to  see  and  hear  the 
students  at  the  other  end  of  the  connection  without  being  seen  or  heard  themselves.  We  observed  one  particular 
videoconferencing  session  between  two  middle  school  groups  where  self-consciousness  was  very  much  in  evidence. 
This  was  the  first  time  this  group  had  experienced  videoconferencing.  In  a session  that  lasted  approximately  20 
minutes,  the  monitor-mounted  camera  was  moved  1 9 times  by  the  students.  The  camera  view  was  usually  moved 
away  from  the  person  doing  the  moving.  Students  also  made  comments,  such  as  “get  the  camera  away  from  me”. 
Students  in  this  group  also  showed  audio  self-consciousness.  The  microphone  changed  hands  at  least  27  times 
during  this  session.  It  was  also  offered  to  other  students  on  many  more  occasions  but  refused.  Side  comments 
during  the  session  included:  “What?  Someone  else  speak  to  them”,  “I  don’t  understand  Southern  accents”,  and  “I 
don’t  want  it  [the  microphone]  give  it  to  her”.  In  addition,  we  also  observed  many  examples  of  nervous  behavior, 
including  adjusting  hair,  covering  the  mouth,  and  giggling.  On  other  occasions  we  have  observed  videoconferences 
where  students  move  communication  to  the  text  chat  channel  apparently  to  deliberately  avoid  talking  into  the 
microphone.  One  of  the  most  common  ways  that  groups  seem  to  deal  with  this  self  consciousness  issue  is  to 
designate  one  student  to  act  as  spokesperson  for  the  group.  The  whole  issue  of  student  attraction  and  fear  towards 
videoconferencing  needs  further  investigation. 


Organizational  Issues 

We  have  found  that  the  educational  value  of  videoconferencing  is  highly  dependent  on  the  suitability  of  the 
collaborators  and  the  underlying  basis  for  the  collaboration.  You  cannot  just  connect  people  together  and  expect 
them  to  meaningfully  collaborate.  One  of  our  student  projects  involved  a group  of  students  at  one  school  building 
the  arm  of  a working  robot  and  another  group  at  a different  school  building  the  mobile  base.  Clearly  such  an 
activity  requires  that  the  two  groups  actively  collaborate.  The  development  of  such  collaboration-dependent 
activities  can  have  a positive  influence  on  the  perceived  educational  value  of  DVC. 

Three  obvious  coordination  problems  are  immediately  apparent: 

1.  Teachers  need  to  establish  contact  with  remote  others  (teachers  or  experts)  who  can  organize/provide 
meaningful  educational  interactions  for  their  students.  These  contacts  not  only  have  to  be  educationally 
compatible,  they  also  need  compatible  videoconferencing  facilities. 

2.  Assuming  teachers  have  been  able  to  make  contact  with  suitable  others,  they  then  have  to  negotiate  a basis  or  a 
context  for  the  use  of  videoconferencing  in  their  respective  educational  practices.  This  is  no  simple  process. 
Teachers,  even  those  teaching  the  same  subject  to  the  same  age  students,  have  very  different  teaching  styles 
and  related  practices. 

3.  Time  is  very  important  in  middle  and  high  schools.  Classes  change  according  to  the  clock.  We  have  found 
that  even  in  the  same  county,  schools  may  have  different  time  slots  for  the  classes  throughout  the  day.  Trying 
to  organize  a time  when  groups  from  two  different  locations  can  take  part  in  synchronous  communication  can 
be  extremely  difficult. 

In  the  LiNC  project,  these  coordination  issues  have  been  less  apparent  because  the  project  itself  has  provided  a basis 
and  a structure  for  coordination  between  the  various  teachers. 
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We  see  desktop  videoconferencing  as  a valuable  educational  tool  particularly  suitable  for  supporting  distributed 
collaborative  learning  between  schools  and  between  schools  and  the  community.  However,  teachers  have  a 
considerable  amount  of  autonomy  in  their  own  classrooms  and  any  kind  of  significant  educational  reform, 
including  the  widespread  acceptance  of  new  technology,  is  notoriously  difficult.  Our  studies  suggest  that  desktop 
videoconferencing  supports  distributed  collaborative  learning  surprisingly  well.  Even  though  the  technology  was 
presumably  developed  with  office  environments  in  mind.  However,  the  suitability  of  the  technology  and  the  high 
motivation  levels  amongst  students  does  not  guarantee  that  DVC  will  be  widely  accepted  in  schools.  We  have 
described  the  possible  technical  problems  in  some  detail  and  have  gone  on  to  suggest  some  of  the  organizational 
issues  that  also  need  to  be  surmounted.  Our  experiences  suggest  that  desktop  videoconferencing  can  make  a 
significant  and  special  contribution  to  learning  in  schools,  however,  there  are  many  factors  that  may  hinder  the 
widespread  acceptance  of  this  technology. 
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Abstract:  It  seems  desirable  that  learners  do  not  only  passively  browse  through  course  mate- 
rial, but  are  also  able  to  actively  work  with  it  by  adding  own  material,  annotating  existing  doc- 
uments, creating  own  views  of  the  material  and  exchanging  documents  with  fellow  learners. 
We  have  developed  two  tools  that  allow  for  such  active  individual  and  collaborative  use  of 
course  material.  “Semantic  Spaces”  allow  learners  to  Organize  hypermedia  documents  accord- 
ing to  their  individual  view  while  not  interfering  with  the  “real”  link  structure  of  the  course 
material.  These  spaces  are  represented  as  maps  and  serve  for  orientation  as  well  as  navigation 
purposes.  A collaborative  virtual  environment  has  been  developed  to  allow  for  synchronous  as 
well  as  asynchronous  multi-user  collaboration.  Both  concepts  have  been  implemented  and 
tested  as  prototypes  that  can  be  easily  integrated  into  standard  or  advanced  web  technology.  An 
evaluation  will  yield  first  results  by  the  beginning  of  1999. 


1 Introduction 

The  World  Wide  Web  has  become  one  of  the  major  ways  to  distribute  course  material  to  students.  However,  this 
material  is  usually  only  available  for  browsing  purposes.  While  the  original  idea  of  hypertext  allowed  readers  to 
take  an  active  role  in  the  creation  and  extension  of  hypertext  documents  (cf  Bush  1945,  Nelson  1980),  the  World 
Wide  Web  has  ignored  this  concept.  Material  presented  on  standard  web  servers  is  usually  static  in  the  sense  that 
readers  cannot  actively  work  with  it:  Because  links  are  embedded  into  the  documents  and  are  always  visible  to 
everybody,  readers  are  not  allowed  to  link  from  existing  documents  to  material  that  they  feel  is  important  for  the 
learning  context,  nor  can  they  annotate  existing  documents.  Also,  restructuring  of  existing  material  is  only  possi- 
ble in  very  simple  ways  (by  bookmarking)  or  by  writing  own  documents  that  are  external  to  the  server.  While 
with  advanced  technologies  like,  e.g..  Hyperwave  (cf  Maurer  1996)  some  of  these  restrictions  have  been  remo- 
ved, the  problem  of  allowing  the  learner  to  create  their  own  view  to  existing  material  has  not  been  solved. 

We  believe  this  problem  to  be  closely  related  to  navigation  and  orientation  problems  in  hypertexts  that  have 
often  been  reported  (see,  e.g.,  Conklin  1987)  and  are  of  particular  importance  in  learning  contexts  where  learners 
have  to  construct  knowledge  from  large  bodies  of  material  presented  to  them.  Semantic  Spaces  allow  users  to  cre- 
ate their  own  maps  of  the  hypermedia  material,  thus  documenting  their  individual  understanding  of  the  informa- 
tion space.  Objects  recorded  on  the  map  can  be  connected  to  the  “real”  objects  of  the  hypermedia  system.  When 
users  move  through  the  material,  their  current  position  can  be  visualized  in  the  Semantic  Space.  First  experiments 
with  mock-ups  suggest  that  users  get  a better  overview  of  the  hypermedia  material  and  have  a better  sense  of  ori- 
entation. A tool  that  allows  creating,  modifying  and  sharing  of  Semantic  Spaces  has  been  implemented  as  a proto- 
type and  is  currently  being  tested.  An  evaluation  and  further  developments  are  planned  for  1 999. 

While  this  tool  may  help  learners  to  work  with  hypermedia  course  material  individually,  collaboration 
between  fellow  learners  is  one  major  aspect  in  teaching  and  learning  processes.  When  learning  with  hypermedia 
systems  in  a collaborative  environment,  students  need  to  communicate  their  position  and  their  understanding  of 
the  documents  to  each  others  in  order  to  start  a dialogue  about  the  material  provided  by  the  system.  Thus,  learners 
should  be  supported  in  sharing  perspectives  on  course  material  (cf  Wan  & Johnson  1994).  Norman  (Norman 
1 994)  reports  that  special  tools  are  needed  for  communicating  the  current  position  and  knowledge  of  the  structure 
of  the  material  used:  “Pointing  out”  the  user’s  current  position  or  sharing  their  knowledge  with  other  people  is 
impossible  without  shared  knowledge  of  the  structure  of  the  system.  In  the  project  sTEAM,  we  have  developed 
such  a tool  as  a client-server  application  that  is  based  on  a MUD  (multi-user  dungeon)  environment.  Learners  may 
meet  in  “rooms”  within  that  system  and  share  documents  as  well  as  discuss  topics,  thereby  implementing  asyn- 
chronous as  well  as  synchronous  cooperation  facilities.  A prototype  is  currently  being  evaluated. 
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In  the  next  section  we  will  shortly  discuss  the  possibilities  and  problems  of  creating  individual  views  on 
course  material  and  cooperating  with  fellow  learners  when  working  with  standard  tools.  We  will  also  discuss  fea- 
tures of  more  advanced  learning  environments.  The  following  two  sections  will  then  describe  the  two  approaches 
of  individualization  and  cooperation  of  web  based  material  and  compare  them  to  related  work.  The  prototypes 
will  be  described.  The  last  section  will  summarize  the  main  results  and  describe  the  plans  for  further  development 
and  evaluation  of  the  tools. 


2 Working  actively  and  collaboratively  with  documents 

Many  hypermedia  systems  under  development  today  are  used  in  education.  The  most  promising  aspects  of  hyper- 
media use  are  in  our  opinion  (Keil-Slawik,  Klemme  & Selke  1996): 

• the  integration  of  all  materials  used  in  the  learning  process; 

• the  availability  of  these  materials  for  all  forms  and  phases  of  learning,  without  the  need  to  use  different 
media  in  different  situations; 

• hypermedia's  capability  of  being  a repository  for  all  learning  situations,  with  the  option  of  reusing  hyper- 
media units  in  different  situations  and  contexts. 

Hypermedia  systems  for  this  purpose  need  to  accommodate  large  and  frequently  changing  bodies  of  documents. 
This  requires  powerful  tools  allowing  users  to  move  through  the  hypermedia  system,  interact  with  it  and  other 
users,  and  customize  it  for  their  personal  needs. 

While  today  we  see  more  and  more  courses  offering  material  via  the  World  Wide  Web,  it  was  already  in  1 993 
that  Ben  Shneiderman  coined  the  phrase  of  “education  by  engagement  and  construction”  (Shneiderman  1993).  He 
stressed  the  fact,  that  students  on  the  one  hand  have  to  interact  with  the  material — which  means  not  just  clicking 
and  hopping  from  page  to  another — and  on  the  other  hand  need  to  interact  with  each  others.  Also,  in  the  develop- 
ment of  the  Intermedia  system  the  developers  state  that  “hypertext  systems  that  support  multiple  users  allow 
researchers,  professors,  and  students  to  communicate  and  collaborate  with  one  another  within  the  context  of  a 
body  of  scholarly  materiar  (Yankelovich  et  al.  1988,  p.  81).  This  is  definitely  a potential  of  hypermedia  systems, 
the  importance  of  which  is  obvious  from  a constructivist  viewpoint.  However,  we  find  little  of  that  in  the  courses 
we  nowadays  see  on  the  web. 

This  is  mainly  due  to  the  fact  that  the  World  Wide  Web  in  fact  is  mainly  used  for  the  mere  distribution  of  doc- 
uments. Because  of  the  Web’s  and  HTML’s  architecture,  the  material  can  only  be  read.  Students  are  not  able  to 
create  personal  hyperlinks  or  organize  the  documents  in  an  individual  manner  apart  from  setting  bookmarks,  a 
technique  that — according  to  questionnaires  to  our  students — is  heavily  used  by  learners.  For  security  reasons, 
many  universities  do  not  even  allow  students  to  publish  their  own  documents  on  the  faculty’s  web  server.  Annota- 
tions to  documents  were  possible  with  Mosaic,  a browser  popular  in  the  old  days  of  the  web.  However,  users 
could  only  create  annotations  on  a per-document  basis  in  an  additional  file.  With  the  Netscape  and  Microsoft 
browsers  becoming  more  prominent,  the  situation  became  even  worse  since  these  browsers  do  not  support  annota- 
tions at  all.  As  a consequence,  the  Web  often  serves  as  a printing-on-demand  device  for  text  documents.  Coopera- 
tion is  also  only  possible  in  a very  limited  way:  by  sending  documents  or  links  to  other  users. 

These  limited  capabilities  suggest  the  use  of  more  advanced  web  servers  like,  e.g..  Hyperwave  (see  Maurer 
1 996)  that  allow  users  to  actively  work  with  documents  (cf.  Brennecke,  Schwolle  & Selke  1 997).  As  compared  to 
traditional  Web  servers.  Hyperwave  offers  several  important  advantages.  The  sophisticated  user  management  and 
integrated  management  of  access  rights  makes  it  possible  to  establish  a private  area  for  each  student  where  they 
can  file  their  own  documents  and  create  a personal  view  of  the  available  documents.  With  Hyperwave’s  concept 
of  collections  different  views  can  be  created  without  installing  real  copies  of  documents.  In  combination  with  the 
integrated  link  management,  links  always  point  to  the  most  recent  version  of  a document  even  when  the  material 
has  been  revised.  However,  these  views  are  still  more  or  less  lists  of  links.  A graphical  representation  of  the  docu- 
ment set  can  either  be  produced  with  a “local  map”  of  links  (see  next  section)  or  hand-crafted  clickable  images 
that  are  difficult  to  maintain. 

Hyperwave  also  offers  limited  cooperation  functionality  through  access  rights.  Users  can  decide  who  is 
granted  read  or  write  access  to  their  documents.  Other  systems  coming  from  the  CSCW  area,  such  as  Lotus  Notes, 
offer  sophisticated  workflow  mechanisms.  However,  collaboration  in  learning  contexts  is  usually  very  different 
from  cooperative  processes  in  a company,  as  both,  the  formation  of  groups  of  learners  and  the  collaboration  based 
on  documents,  are  rather  informal.  Also,  groups  only  exist  for  a very  short  period  of  time  and  members  may  join 
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or  leave  within  very  short  time  intervals.  For  this  reason,  the  systems  mentioned  so  far  are  only  of  limited  use  in 
collaborative  learning  environments.  In  the  web,  there  is  no  relation  between  the  documents  and  the  people  work- 
ing with  them:  E.g,,  it  is  not  possible  to  obtain  any  information  about  people  who  have  read  a particular  docu- 
ment. 

3 Individualization  of  course  material:  Semantic  Spaces 

It  has  been  claimed  that  link  structures  in  hypermedia  systems  map  well  onto  users’  cognitive  structures  (Jonassen 
1 986).  However,  this  hypothesis  is  questioned  by  many  experts  in  the  field  (see,  e,g.,  Duffy  & Knuth  1 990),  The 
link  structure  itself  is  based  on  the  syntactic  structure,  i,e,  the  interrelationships  defined  on  the  document  space  by 
hyperlinks.  This  must  not  be  confused  with  the  semantic  structure  of  the  contents  of  hypermedia  documents,  i.e, 
the  interrelationships  of  the  concepts  presented,  since  these  are  in  most  cases  not  the  same  (Whalley  1990,  p,  63), 

The  insight  into  these  structures  is  particular  to  a user  and  cannot  easily  be  transferred  to  others.  Specifically 
non-experts  need  help  finding  out  about  the  concepts  covered  in  the  documents  and  they  are  not  necessarily  inter- 
ested in  the  form  of  presentation  as  represented  in  the  link  structure.  In  particular,  users  new  to  a subject  have 
problems  discovering  the  underlying  structures  and  finding  their  way  through  the  material.  Understanding  of  a 
subject  area  develops  gradually.  Users  continuously  extend  their  models  as  they  find  more  information.  These 
models  are  different  from  the  relationships  explicated  by  nodes  and  links  in  the  hypermedia  material,  so  the  stu- 
dents’ model  can  be  completely  wrong  and  will  (hopefully)  be  corrected  later. 

These  problems  of  individualization  and  orientation  are  amplified  in  large  interwoven  hypermedia  systems 
such  as  the  World  Wide  Web,  which  span  huge  numbers  of  hosts  and  reference  material  of  different  contexts.  In 
order  to  make  efficient  use  of  a hypermedia  system,  a consistent  hypermedia  working  environment  needs  to  be 
provided  to  the  learners.  It  should  offer  tools  for  information  selection,  navigation  through  the  system,  and  docu- 
menting the  findings  and  results.  Utting  and  Yankelovich  (Utting  & Yankelovich  1989)  described  and  tested  dif- 
ferent tools  such  as  local  and  global  maps  for  solving  navigational  problems.  However,  the  proposed  tools  proved 
either  to  be  insufficient  in  scope  or  too  complex  to  be  understood  and  efficiently  handled  by  the  users. 

Semantic  Spaces  may  help  to  overcome  these  problems,  A Semantic  Space  is  a two-  or  three-dimensional 
map  that  represents  the  user’s  individual  understanding  of  the  information  space.  It  is  closely  intertwined  with  the 
objects  of  the  hypermedia  systems  and  serves  as  a tool  for  the  active  exploration  of  hypermedia  systems.  As  users 
move  through  the  hypermedia  system,  they  enter  concepts  and  relationships  that  they  think  are  significant  into  the 
map.  The  concepts  displayed  in  a Semantic  Space  are  thus  not  necessarily  related  to  the  nodes  and  links  in  the 
hypertext,  but  represent  the  user’s  current  knowledge  and  understanding  of  the  system.  Users  can  therefore 
explicitly  document  their  personal  understanding  of  the  part  of  the  system  they  have  explored. 

Items  and  links  in  the  Semantic  Space  can  be  created  interactively  by  naming  the  concept  and  target  docu- 
ment or  by  dragging  a document  icon  into  the  map,  automatically  connecting  the  object  in  the  Semantic  Space 
with  the  documents.  The  interrelationships  between  the  concepts  in  the  Semantic  Space  and  the  documents  of  the 
“real”  hypermedia  system  are  maintained  by  specially  labelled  bi-directional  hyperlinks  connecting  concept  and 
document.  With  the  help  of  these  links,  the  position  of  the  user  within  the  hypermedia  system  can  be  visualised  in 
the  current  Semantic  Space  when  the  user  moves  through  the  hypermedia  system.  They  will  be  altered,  moved  or 
deleted  as  the  process  of  understanding  continues.  A set  of  tools,  similar  to  those  of  standard  desktop  environ- 
ments, for  grouping,  moving,  deleting,  etc.  provide  for  quick  and  direct  manipulation.  Additional  drawing  and 
writing  tools  allow  for  annotation  and  graphical  markup  (arrows,  boxes,  etc,). 

By  creating  maps  of  their  own,  users  can  develop  and  visualize  a spatial  understanding  of  the  hypermedia 
system.  This  intensifies  the  users’  involvement  and  can  thus  foster  their  understanding  of  the  overall  context 
(Shneiderman  1 993).  The  general  layout  of  the  map  remains  persistent  until  the  users  decide  to  change  it  because 
it  no  longer  represents  their  understanding  of  the  document  space.  The  maps  themselves  are  also  documents 
within  the  hypermedia  system.  They  can  be  inter-linked  with  the  other  documents  or  be  referenced  by  other 
Semantic  Spaces,  thus  creating  a mesh  or,  if  required,  hierarchy  of  maps.  Since  they  are  persistent  over  time  and 
can  be  shared  with  other  users,  they  can  be  used  in  group  learning  situations  to  assemble  and  document  the 
group’s  common  knowledge  of  the  information  space.  Furthermore,  they  can  provide  starting  points  for  the  fur- 
ther exploration  of  the  system.  Collaborative  design  of  Semantic  Spaces  may  thus  foster  understanding  and  learn- 
ing in  groups. 

A tool  for  the  creation  and  usage  of  Semantic  Spaces  has  been  implemented  on  top  of  the  hypermedia  system 
Hyperwave  using  a combination  of  Java  and  JavaScript.  Users  can  use  their  standard  Netscape  browser  to  retrieve 
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documents  from  the  hypermedia  system.  A separate  map  viewer  communicates  with  the  Navigator  windows  indi- 
cating changes  in  location.  By  selecting  an  element  in  a map  the  user  may  also  cause  the  browser  to  display  the 
corresponding  document. 

The  concept  of  Semantic  Spaces  is  described  in  more  detail  in  (Klemme,  Kuhnert  & Selke  1998).  In  support- 
ing constructive  navigation  in  hypermedia  systems,  they  are  similar  to  the  NESTOR  system  (Zeiliger  1998). 
Whereas  in  NESTOR  there  is  an  automatic  generation  based  on  the  user’s  activities,  in  Semantic  Spaces  the  users 
explicitly  have  to  create  their  own  visualization.  Evaluations  may  show  if  there  are  significant  advantages  in 
either  of  the  approaches.  Semantic  Spaces  may  also  be  compared  to  concept  maps  (Kremer  1997).  However,  they 
do  not  impose  any  rules  on  how  to  represent  objects  and  relationships.  Also,  while  providing  an  abstraction  from 
the  underlying  hypermedia  database,  Semantic  Spaces  still  serve  as  navigational  and  orientational  means  by  pro- 
viding location  feedback. 

4 Collaborative  Virtual  Environments:  sTEAM 

With  the  emergence  of  local  area  networks  and  the  massive  growth  of  the  Internet,  communication  games  on 
these  networks,  the  so-called  MUDs  (multi  user  dungeons),  were  developed.  While  many  of  these  are  text-orien- 
ted, recent  games  of  this  type  also  support  graphical  user  interfaces.  MUDs  are  based  on  real-time  communication 
and  interaction  of  multiple  users  moving  through  a “virtual  world”  which  is  organized  in  “rooms”  or  “places”. 
MUDs  are  based  on  a client-server  architecture,  allowing  players  with  different  computer  systems  to  participate  in 
these  games  from  different  places.  MUDs  are  on  the  one  hand  multi-user  capable,  on  the  other  hand  authorised 
players,  the  so-called  wizards,  can  restrict  access  to  the  game.  They  can  grant  or  deny  access  to  specific  rooms  to 
a certain  group  of  players  and  can  also  extend  the  “world”.  In  such  games  direct  communication  and  interaction  in 
real  time  are  the  main  goals.  Groups  of  players  can  communicate  directly  and  influence  each  others  in  their  game 
actions,  while  the  communication  flow  is  unrestricted.  Since  the  architecture  of  MUDs  is  open,  the  games  can  be 
modified  and  adapted  to  new  situations  during  the  current  session  even  while  the  system  is  running.  So  at  any 
time  during  the  game  new  situations  can  be  created,  with  which  the  groups  of  players  have  to  cope.  A number  of 
“serious”  applications — usually  called  “collaborative  virtual  environments”  (CVE) — have  been  developed  on  top 
of  this  architecture  (see,  e.g.,  Donath  1995). 
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Our  goal  was  to  combine  the  advantages  of  classical  MUDs  and  Hypermedia  database  systems  with  each  oth- 
ers, allowing  users  to  share  documents  in  a virtual  world  and  to  communicate  with  other  users  while  working  with 
certain  documents  (see  Bollmeyer  1997).  The  system  developed  is  called  sTEAM  and  uses  the  paradigm  of  vir- 
tual rooms  representing  a document  space.  People  can  meet  in  virtual  rooms,  interchange  documents  and  commu- 
nicate with  each  others.  The  rooms  can  be  seen  as  areas  of  interest  or  as  given  topics  of  a lecture.  In  these  areas 
the  stored  objects  are  files,  which  all  have  some  topic  or  subject  in  common.  Exits  connecting  different  rooms  rep- 
resent hyperlinks.  The  big  advantage  of  this  connecting  technique  is  that  they  represent  a semantic  context  with- 
out having  to  represent  a planar  layout  of  rooms  that  can  be  represented  by  a planar  map. 

Among  the  system’s  major  new  aspects  are  its  abilities  for  easily  constructing  new  rooms.  As  virtual  rooms 
can  be  created  by  any  user  on-the-fly,  document  collections  can  be  structured  easily  and  personally  or  collabora- 
tively  by  the  students.  For  example,  when  students  discuss  different  approaches  to  a certain  topic  during  a tutorial 
or  lecture  on,  say,  user  interface  design,  they  can  create  a new  room  and  meet  there.  They  may  then  search  for 
examples  of  different  user  interfaces  on  the  web  and  collect  these  (as  hyperlinks  or  as  copies  of  the  documents)  in 
this  new  room.  After  they  finished  their  work  they  could  either  leave  this  room  and  make  it  accessible  for  other 
students  interested  in  the  same  topic,  or  take  just  a few  objects  to  their  private  rooms  and  delete  this  temporarily 
needed  room  again. 

Groups  of  students  and  teachers  can  thus  build  their  individual  and  cooperative  learning  spaces.  Hypermedia 
documents  can  be  transported  easily  between  the  various  rooms.  Messages  and  attachments  can  readily  be  stored 
in  such  learning  places  and,  most  important,  the  learners  may  meet  in  groups  within  the  system.  The  system  may 
either  be  used  within  a scenario  where  students  meet  “in  real  life”,  using  the  rooms  in  the  way  described,  or  within 
a scenario  where  students  do  not  meet  and  make  use  of  the  integrated  facilities  enabling  them  to  synchronously 
communicate  with  each  others.  Currently,  only  a text-based  chat  tool  is  integrated,  but  audio  or  video  connections 
may  be  implemented  in  the  future.  Thus,  collaboration  is  enhanced  through  a powerful  CSCW  system  regardless 
of  whether  people  meet  in  the  same  place  or  work  from  different  places. 

The  sTEAM  system  currently  exists  as  a prototype  consisting  of  a client  written  in  Java  and  a web  server 
implemented  in  the  MUDs  own  object-oriented  programming-language  LPC  using  the  Dworkins  Generic  Driver 
(DGD),  a modified  well-known  MUD  server.  This  system  allows  runtime  modifications  of  all  objects  in  the  envi- 
ronment. To  create  their  individual  environment  inside  sTEAM,  the  user  can  employ  the  pre-de fined  objects, 
derive  their  own  objects  from  existing  ones,  or  create  new  ones  by  implementing  them  in  LPC.  Many  different 
new  objects  may  be  created  during  a long  term  use  of  the  system. 

All  sTEAM  objects  are  stored  on  the  server.  Currently,  all  data  management  is  covered  by  the  sTEAM  server. 
Using  an  interface  that  is  currently  under  development,  it  will  be  possible  to  connect  sTEAM  to  a database  in  the 
future,  thus  allowing  the  use  of  multimedia  documents  stored  in  that  database  or  of  hypermedia  objects  stored  on 
a Hyperwave  server.  Thereby,  sTEAM  can  easily  be  integrated  into  existing  web  environments.  All  objects  like 
hypertexts,  images,  links,  users  etc.  have  a number  of  common  attributes  that  are  used  to  control  the  details  about 
objects  such  as  name,  access  rights,  owner,  creation  date,  expiration  date  etc.  and  methods.  In  addition,  some 
objects  have  special  attributes  and  methods  such  as  an  associated  MIME-Type  and  a download  method  for  docu- 
ments or  a method  “talk  to”  for  user  objects.  Users  can  also  add  their  own  generic  attributes  to  objects.  By  default, 
the  user  object  has  a virtual  container  object,  that  can  be  seen  as  a “suitcase”  that  the  user  carries  along.  Any 
objects  they  want  to  take  along,  even  exits,  can  be  stored  in  this  suitcase  and  stay  there  until  taken  out.  sTEAM 
features  a sophisticated  security  scheme  resembling  that  of  Hyperwave  with  one  significant  difference:  User 
groups  can  be  defined  by  any  user,  as  they  are  just  objects  on  the  server  which  have  attributes.  Just  as  rooms,  user 
groups  can  thus  be  created  on-the-fly.  So,  if  some  team  of  students  decides  that  they  are  a group  which  needs  to 
share  access  rights,  they  create  a group  object. 

The  client  deals  as  the  interface  between  the  user  and  the  functionality  given  by  the  server  and  has  to  be  easy- 
to-use  and  platform  independent.  The  last  requirement  was  the  reason  for  its  implementation  in  Java.  In  order  to 
create  an  easy-to-use  interface  we  restricted  the  view  of  the  virtual  environment  to  a simple  2d-representation. 
The  view  is  made  up  of  three  areas,  showing  the  users  currently  in  this  room,  the  interior  of  the  room,  i.e.  the 
objects  including  exits  to  other  rooms,  and  the  communication  area.  Here,  the  user  can  type  text  to  chat  and  talk  or 
communicate  in  other  text  based  ways.  All  of  these  objects,  even  the  users,  are  subject  to  direct  manipulation. 
E.g.,  you  can  “talk  to”  other  users,  “download”  documents,  or  move  to  other  rooms  with  just  a mouse  click  on  the 
respective  object.  More  sophisticated  functions  including  creation  of  new  objects  and  file  upload  can  be  accessed 
by  menus. 

According  to  the  view  presented  above  we  are  developing  sTEAM  under  two  main  aspects.  As  a first  goal, 
sTEAM  will  serve  as  a research  platform  for  studying  new  forms  of  cooperation  and  collaboration  between  stu- 
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dents.  sTEAM  closes  the  gap  between  classical  information  and  communication  technology  allowing  the  users  to 
actively  structure  their  document  space  while  setting  up  communication  channels  between  them.  As  a second 
aspect,  the  flexible  design  allows  the  fast  and  easy  creation  of  special  clients  to  study  questions  of  human  com- 
puter interaction  or  new  forms  of  collaboration  technology. 


5 Conclusion 

We  have  described  two  concepts  for  individual  and  collaborative  use  of  hypermedia  course  material.  Semantic 
Spaces  allow  learners  to  organize  hypermedia  documents  according  to  their  individual  needs.  These  spaces  serve 
as  means  of  orientation  and  navigation,  A collaborative  virtual  environment  called  sTEAM  establishes  a virtual 
working  environment  in  which  groups  can  meet  to  work  on  certain  topic  areas.  Social  contacts  between  students 
can  thus  be  augmented  by  the  system.  Both  systems  have  been  implemented  as  prototypes  and  will  consequently 
be  evaluated.  Future  plans  include  the  integration  of  Semantic  Spaces  and  sTEAM  to  allow  for  an  easy  transition 
between  individual  and  group  learning  processes. 
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Abstract:  The  Multibook  project  of  the  Darmstadt  University  of  Technology  builds  a webbased  adaptive  hypermedia 
teaching  and  learning  environment  for  multimedia  and  communication  technology.  Hereby,  the  demands  of  diverse  user 
groups,  user  levels  and  especially  of  diverse  learning  strategies  are  taken  into  account.  Besides  information  gained  from 
the  interaction  with  the  user,  the  system  uses  standardized  content  relations  and  meta-information  to  adaptively  compile  a 
selection  from  the  set  of  available  information  units.  A subset  of  the  meta-information  represents  the  compiled  lesson  on  a 
high  level  and  is  presented  to  the  learner  as  a dynamically  generated  table  of  contents. 

1.  Motivation 

"Imagine  everything  is  available  and  tied  together.  Grand  visions  come  to  mind  of  what  things  will  be  like  when  ‘ifs 
all  there  and  linked’.  The  thought  of  that  great  body  of  material  calls  to  us,  calls  to  us  like  an  ocean.  But  that 
ocean  of  universal  hypertext  is  not  enough;  we  want  free  sailing  on  it."[10] 

So  why  should  we  restrict  this  demanded  freedom  of  navigation?  There  are  many  applications  where  it  makes  sense 
to  guide  the  user  to  a certain  amount.  One  of  these  are  learning  environments.  If  the  use  of  such  a system  is  supposed 
to  lead  to  an  intersubjectively  measurable  learning  effect,  it  must  be  guaranteed  that  the  user  has  seen/leamt  certain 
parts  of  information,  and  doesn't  get  completely  lost  on  his/her  way.  Hence  the  decision  where  to  go  cannot  be 
entirely  up  to  the  user. 

The  solution  we  are  going  for  is  to  offer  a special  form  of  guided  tours  comparable  to  those  of  a classical  hypermedia 
system.  Ifs  our  concern  to  build  a system,  in  which  we  can  generate  tours  dynamically.  We  offer  these  guided  tours 
to  the  user  in  form  of  tables  of  contents,  that  structures  a relevant  subset  of  the  material.  We  see  this  as  a suitable  way 
to  overcome  the  gap  between  a linear  ordered  text  and  free  navigation. 


2.  Related  Work 

Recently,  ideas  from  intelligent  tutoring  systems  (ITS)  and  from  hypermedia  have  been  brought  together  opening  the 
field  of  adaptive  hypermedia.  This  synthesis  responds  to  the  specific  strengths  and  weaknesses  of  both  approaches. 
ITS  [1],[4],[15]  provide  a high  level  of  guidance  in  learning,  they  model  and  control  the  entire  learning  process  in 
great  detail.  A domain  model  representing  all  facts  of  the  field  that  are  to  be  learnt,  usually  forms  the  background  for 
a model  of  the  learner's  knowledge  and  knowledge  acquisition.  Free  exploration  by  the  learner  does  not  play  an 
important  role  in  ITS,  the  navigation  decisions  are  very  much  up  to  the  system.  As  discussed  in  [6],  most  of  the 
existing  ITS  systems  are  environments  for  coding  and  testing  a specific  programming  language.  Furthermore,  users 
often  need  considerable  time  for  learning  how  to  use  an  ITS,  because  the  development  of  an  intuitive  user  interface 
for  ITS  systems  seems  to  be  a very  difficult  task.  Hypertext  and  hypermedia  systems  exploit  the  nature  of  different 
media  such  as  text,  pictures,  audio,  video  or  simulations  as  a medium  for  making  differentiated  statements  and 
communicating  less  structured  knowledge.  In  addition,  a hypermedia  system  offers  more  than  predefined  learning 
paths  - by  selecting  different  nodes  in  different  order,  the  individual  learners  produce  a multitude  of  paths  through 
the  material.  The  drawback  of  these  systems  is  that  the  learning  process  cannot  be  controlled  in  a well  defined  way. 
This  results  in  insufficient  guidance.  In  particular  when  used  for  educational  purposes,  hypermedia  systems  are 
striving  for  a higher  degree  of  control  [7], [3].  We  regard  as  decisive  the  step  of  adding  conceptual  information  on  top 
of  the  hypermedia  chunks,  being  the  basis  for  their  intelligent  selection  and  sequencing.  Connecting  the  concepts 
with  semantic  rather  than  didactic  relations  that  would  already  imply  sequences  or  dependencies  among  the  concepts 
[2]  provides  yet  a higher  degree  of  flexibility  - this  way,  the  concept  space  lends  itself  to  realizing  different  learning 
strategies  and  goals  but  also  to  tasks  like  information  retrieval. 

The  Interbook  project  is  a well  known  adaptive  hypermedia  system,  which  also  makes  use  of  a concept  space,  called 
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domain  model  [8].  Interbook  supports  navigation  through  the  lesson  with  adaptive  annotation,  showing  the  type  and 
the  educational  state  ("ready  to  be  learnt",  "recommended",  "not  ready  to  be  learnt")  of  each  offered  link.  Interbook 
is  based  on  the  domain  modelling  approach  of  ELM- ART,  a WWW  version  of  ELM-PE,  that  is  currently  one  of  the 
most  advanced  intelligent  learning  environments  for  programming.  In  the  online  description  of  Interbook  the  term 
"13-textbooks"  (integrated  + interactive  + intelligent  textbook)  is  suggested  for  approaches  which  integrate  on-line 
representation  of  learning  material  with  the  interactivity  of  problem  solving  environments  and  intelligence  of  ITS.  In 
this  sense,  the  Multibook  project  can  be  seen  as  a specific  type  of  an  13-textbook,  although  our  application  domain 
brings  along  some  different  problems  which  requires  slightly  different  approaches  and  techniques,  which  we  will 
describe  in  the  remainder  of  this  article. 


3.  Multibook 

3.1  Constraints  and  Requirements 

The  content  of  the  Multibook  system,  currently  being  developed  at  the  TU  Darmstadt,  is  the  printed  book 
"Multimedia:  Computing,  Communications  & Applications"  by  Ralf  Steinmetz  and  Klara  Nahrstedt  consisting  of 
about  1200  pages,  and  a selection  of  Java  applets  [13].  The  aim  of  Multibook  is  to  have  individual  views  on  this 
material  according  to  the  needs  and  preferences  of  the  individual  users.  A linear,  printed  book  does  not  satisfy  these 
requirements:  It  does  not  adapt  flexibly  to  the  level  of  difficulty,  the  learning  aim,  the  learning  strategy  and  the  media 
preferences  of  the  specific  user.  Apart  from  that,  many  aspects  of  multimedia  technology  can  be  explained  better 
using  motion  and  interactivity.  A pure  hypertext  or  hypermedia  system  does  not  satisfy  these  requirements  either.  A 
hypertext  is  static,  in  that  the  text  is  either  heavily  linked  and  there  is  a suitable  path  through  it  for  every  user  - then 
the  user  might  be  confused  by  the  number  of  possibilities  and  not  be  able  to  find  this  path.  Or  there  are  only  a few 
links,  then  it  will  probably  not  satisfy  the  demands  of  each  individual  user.  An  ITS  system  does  satisfy  these 
requirements,  but  these  systems  only  work  in  areas  which  are  highly  structured,  such  as  Mathematics  and  especially 
programming  languages.  The  level  of  control  and  the  detailed  knowledge  of  the  state  of  the  user  can't  be  achieved  in 
our  subject  domain.  To  formally  model  everything  that  can  be  said  about  multimedia  in  texts  or  images  would  not  be 
a feasible  approach.  Also  the  extent  and  the  extensibility  of  our  material  and  the  amount  of  text  describing  it  are 
constraints  which  make  it  impossible  to  guide  the  user  in  such  a controlled  way. 


3.2  Approach 

The  Multibook  system  has  multiple  facets:  it  provides  an  intelligent  testing  environment  for  the  user,  offers  search 
facilities  for  a direct  access,  etc.  In  this  article,  however,  we  will  focus  on  the  main  task:  To  find  and  to  propose  a 
suitable  path,  hence  a suitable  composed  lesson.  To  be  able  to  acomplish  this,  the  content  and  meta  information 
concerning  the  content  must  be  represented  in  a formal  way,  that  enables  the  system  to  operate  on  it.  As  a result  the 
system  can  play  the  role  of  a guide  for  the  user.  In  order  to  find  a suitable  path,  the  system  makes  use  of  a user 
profile  and  of  the  meta  information.  In  the  beginning,  the  profile  is  filled  with  the  demands  and  preferences  of  the 
learner.  There  are  two  spaces  in  which  the  knowledge  base  is  modelled:  a domain  model,  which  we  call  the 
ConceptSpace,  and  the  MediaBrickSpace,  where  knowledge  is  represented  in  small  modular  pieces  of  text,  in  images 
etc.  The  concepts  and  the  actual  content  are  therefore  separated.  The  ConceptSpace  represents  the  domain,  the 
MediaBrickSpace  is  a set  of  possible  explanations  of  this  domain.  The  concepts  are  comparable  to  possible  chapter 
headings  and  are  interconnected  by  semantic  relations.  There  are  different  possibilities  to  choose  a set  of  relations. 
The  granularity  of  the  model  relates  to  the  fact  that  we  are  interested  in  roughly  structuring  the  domain  rather  than 
completely  representing  all  facts.  In  the  vocabulary  of  Knowledge  Engineering,  this  makes  our  ConceptSpace  a 
terminological  rather  than  a axiomatic  ontology  [12],  We  are  aware  of  the  benefits  that  the  extended  use  of  verb 
concepts,  and  the  expression  of  a greater  variety  of  facts  they  allow,  could  have  for  our  domain.  However,  the  current 
model  is,  for  simplicity  reasons,  mainly  built  around  noun  concepts  [5].  Semantic  relations  strongly  guide  the 
composition  of  lessons  but  do  not  predetermine  a unique  set  of  topics  and  the  order  of  these,  to  make  application  of 
different  learning  strategies  possible 

In  the  following  we  will  describe  in  an  exemplary  way  the  rules  that  operate  on  the  different  categories  of  the  profile 
and  on  the  relations  in  the  ConceptSpace  to  compile  a adequate  outline  for  a lesson. 

The  rules  which  are  responsible  for  choosing  the  global  structure  of  the  lesson  use  the  semantic  relations 
corresponding  to  the  learning  aim.  Note  that  these  rules  are  not  a hardwired  part  of  the  system.  At  this  stage,  we 
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modeled  some  rather  straightforward  didactic  rules  ourselves,  later  we  want  to  give  the  domain  experts  working  with 
the  system  the  possibility  to  specifiy  their  own  rules.  Examples  for  rules  using  the  semantic  relations:  In  his/her 
profile,  the  user  is  characterized  as  a student  learning  for  exams.  In  this  case  the  system  will  search  for  concepts 
which  are  related  to  the  chosen  topic  by  the  relations  "uses",  ”part_of’  and  "is_a".The  selection  of  the  topics  has  to 
be  coordinated  with  their  order  arising  from  the  chosen  learning  strategy,  e.g.,  hierachical:  Definition  - broader  term- 
component  - application.  In  other  cases  the  selection  of  the  topics  uses,  on  a small  scale,  inferences  on  the 
ConceptSpace.  Example:  The  topic  is  related  to  another  by  the  relation  "problem".  Then  the  system  would  most 
probably  find  a concept  representing  a solution  for  this  problem  among  the  more  specific  concepts  (i.e.  one  that  is 
connected  by  a "is_a"  relation).  The  lessons  are  finally  put  together  by  combining  a set  of  media  bricks.  They  can  be 
in  form  of  text  or  other  multimedia  elements  such  as  images,  graphics,  video  and  audio  streams  and  - with  the  main 
focus  - animations  realized  as  Java  applets.  Also  these  multimedia  elements  satisfy  the  requirements  of  modularity. 
This  requires  that  the  format  of  the  media  bricks  enables  the  system  to  describe  the  content,  grade  of  detail,  and  the 
underlying  pedagogical  concept  of  the  media  brick.  Thus  it  is  possible  to  integrate  a media  brick  in  a lesson, 
independent  of  the  kind  of  media.  Particular  emphasis  of  our  work  lies  on  the  issue  of  coherence  [14].  When  a user 
of  a hypertext  does  not  have  the  possibility  to  choose  the  pages,  i.e.  he/she  cannot  establish  the  relation  between  the 
parts  of  the  lesson  by  himself/herself,  he/she  is  more  likely  to  expect  a coherent  lesson,  a lesson  where  the  relations 
between  the  parts  are  obvious,  although  they  are  put  together  by  someone  else.  For  this  purpose  the  media  bricks  are 
not  only  linked  to  the  corresponding  concept  but  also  interconnected  in  the  MediaBrickSpace  by  rhetorical  relations 
based  on  the  Rhetorical  Structure  Theory  [9]  [11].  Examples  for  such  rhetorical  relations  are  "deepen"  or  "explain". 

It  is  a major  task  of  the  system,  to  make  this  relations  explicit. 

In  general,  the  rules  for  building  the  lesson  out  of  the  relevant  media  bricks,  have  to  use  the  rhetorical  relations  and 
the  characteristic  of  the  media  bricks,  in  order  to  match  the  users  level  of  difficulty,  media  preferences  and  coherence 
expectations.  Simultaneously,  they  have  to  work  off  the  structure  of  the  lesson  compiled  earlier  and  to  fulfil  the 
demands  of  the  user’s  learning  strategy.  Note  that  the  learning  strategy  requires  rules  working  on  both  spaces.  These 
goals  are  not  always  easily  harmonized:  The  system  will,  for  instance  tend  to  select  as  the  next  media  brick  one  that 
is  connected  to  the  current  one  but  also  one  that  is  connected  to  the  next  topic  in  the  planned  structure. 


4.  A Table  of  Contents  for  each  User! 

As  an  example  how  we  overcome  the  gap  between  free  navigation  and  static  guidance  by  using  the  meta-information 
described  above,  we  suggest  a dynamically  generated  table  of  contents.  Dynamically  generated  tables  of  contents 
combine  the  best  of  both  worlds:  The  interests  of  the  user  are  made  manifest  by  the  outline,  the  structure,  of  the 
lesson.  This  task  can  be  achieved  by  applying  the  user  profile  instead  of  constraining  the  user  to  choose  the  media 
bricks  by  himself/herself  As  with  a guided  tour  the  lessons  are  coherent,  and  more  security  is  given  to  a sensible 
course  of  text/media.  Rules  working  on  the  ConceptSpace  are  responsible  for  creating  the  outline  according  to  the 
user  profile;  rules  working  on  the  MediaBrickSpace  are  responsible  for  the  coherence  of  the  lesson. 


4.1  Tables  of  Contents 

Tables  of  contents,  what  are  they  and  what  function  do  they  have?  Generally,  a table  of  contents  is  the  rough 
structure  of  lessons.  Any  kind  of  table  of  contents  is  motivated  by  semantic  relations.  A table  of  contents  gives  the 
reader  an  overview  of  the  lessons  and  shows  the  view  the  author  has  of  the  subject.Therefore  we  can  consider  the 
table  of  contents  as  a path  through  a net  of  concepts.  We  observe  that  not  every  path  can  serve  as  a table  of  contents, 
nor  does  every  structure  of  a topic  make  sense.  Obviously  some  proven  patterns  occur  quite  often.  These  characterize 
a sensible  structure  for  texts.  We  give  three  examples: 

A structure  based  on  part_of  relations: 

1 . Media  Server 

1.1.  Disc  Controller 

1.2.  Storage  Management 

1.3.  File  System 

1.3.1.  Allocation  Table 

1.3.2.  Access  Control 

1 .4.  Memory  Management 

The  subchapters  describe  the  components  of  a media  server,  the  sections  of  "File  System"  represent  the  components 
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of  a file  system. 

A structure  based  on  is_a  relations: 

1 .  Image  Compression 

1.1.  JPEG 

1.2.  PNG 

1.3.  TIFF 

Here  the  subchapters  are  examples  of  the  topic  of  the  chapter. 

A structure  based  on  proceeds  relations: 

1.  JPEG 

1.1.  Picture  Preparation 

1.2.  Picture  Processing 

1.3.  Quantization 

1.4.  Entropy  Encoding 

The  structure  of  these  view  follows  the  order  of  the  process  itself 

The  structure  in  each  of  these  three  examples  is  a paths,  that  is  a tree,  even  leading  to  multiple  trees  or  forests.  It  is 
not  necessary  that  the  structure  is  based  only  on  one  kind  of  relation,  hybrid  composition  can  also  be  observed  quite 
often.  There  are  several  reasons  why  it  makes  sense  to  provide  explicit  guiding  for  the  user  in  the  form  of  a table  of 
contents: 

1 . A user  in  a learning  environment  has  to  learn  certain  topics,  if  he/she  wants  it  or  not. 

2.  The  user  may  not  have  the  necessary  overview  of  the  area  to  choose  the  relevant  concepts. 

3.  The  user  is  not  able  to  compose  a lesson  according  to  a learning  strategy  by  himself/herself 

4.  The  knowledge  base  is  too  broad  to  be  managed  effectively  without  help  from  the  system.  The  system  can  select 
and  combine  the  media  bricks  faster  and  more  efficiently  than  the  user. 

5.  A table  of  contents  offers  the  user  some  sort  of  orientation.  He/She  can  see  exactly  where  he/she  is  and  to  which 
context  the  current  topic  belongs. 

In  the  Multibook  system  there  are  also  situations  where  "classical"  free  navigation  is  the  appropriate  strategy:  The 
user  selects  the  topics  being  relevant  for  him/her  from  a net  of  content  objects,  which  is  a simplified  version  of  the 
ConceptSpace  where  the  relations  are  omitted.  The  net  offers  the  user  an  overview  of  the  area  with  connections 
between  the  topics  beyond  the  linear  order  of  a table  of  contents  in  a printed  book  and  many  hypertext  systems.  The 
user  can  choose  the  topics  without  working  on  the  media  bricks,  thus  simplifying  the  complexity  of  the  knowledge 
bases. 


4.2  Dynamic  Table  of  Contents 

Since  the  aim  of  the  Multibook  system  is  to  be  adaptive  to  the  needs  and  preferences  of  the  specific  user  we  need 
more  than  just  one  table  of  contents:  We  need  a table  of  contents  for  each  single  user.  This  cannot  be  obtained  by 
having  several  static  tables  of  contents.  If  the  set  of  media  bricks  would  be  static  we  could  precompute  every 
possible  way  through  the  material.  This  means,  every  possible  learning  strategy  etc.,  could  be  realized  and  presented 
to  the  user.  But  since  our  system  is  open  for  extensions  of  both  ConceptSpace  and  MediaBrickSpace  including  media 
bricks  from  external  sources,  we  must  be  able  to  generate  the  tours  through  the  material  dynamically  for  every  user. 
Our  system  is  able  to  fulfil  this  task  because  of  the  partition  into  ConceptSpace  and  MediaBrick 
Space.  The  concepts/topics  are  modelled  separately  from  the  descriptions,  therefore  every  concept  can  be  picked  up 
to  be  the  main  topic.  The  semantic  relations  which  allow  the  diverse  structuring  of  the  topics  make  it  possible  to  tell 
the  difference  between  sensible  and  senseless  structures,  i.e.  table  of  contents.  The  ConceptSpace  with  its  manifold 
semantic  relations  is  a starting  point  for  different  and  meaningful  ways  to  construct  tables  of  contents.  Topics  which 
are  chapter  headings  in  one  lesson  can  be  subchapters  in  another. 


679 


Figure  1 . 1 JPEG  as  main  chapter  described  by  its  components 
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Figure  1.1  JPEG  as  a subchapter  of  compression 


Figure  1 .3  JPEG  as  main  chapter  described  by  its  applications 
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In  Figure  1 different  views  of  the  complex  JPEG  are  shown.  The  topics  selected  for  the  outline  are  highlighted  in  the 
figures.  In  Figure  1.1  JPEG  is  defined  by  its  componenents,  in  Figure  1.2  JPEG  is  a subchapter  of  Compression  and 
is  explained  in  the  context  of  MPEG.  In  Figure  1 .3  JPEG  is  the  main  topic  of  the  chapter  and  is  described  first  by  its 
applications  where  MPEG  is  mentioned,  then  in  detail  by  its  components 

5.  Conclusions  and  Outlook 

The  learning  strategies  we  have  implemented  so  far  (problem  oriented  vs.  hierachical)  are  exemplary.  Our  aim  is  not 
to  prove  whether  a certain  strategy  is  under  certain  circumstances  better  than  another.  We  rather  want  to  provide  a 
tool  for  educationalists  to  conduct  these  kind  of  experiments.  They  can  test  their  theories  about  learning  strategies  by 
applying  their  research  results  to  our  system.  The  system  is  flexible  for  changing  the  rules  in  both  spaces  which  are 
responsible  for  composing  the  lessons.  As  we  only  recently  finished  a prototype  of  our  system,  an  empirical 
evaluation  is  now  on  its  way:  In  a broad  field  trial  at  the  end  of  this  year  with  the  students  of  our  institute,  we  will 
investigate  to  what  extent  the  individual  tables  of  contents  help  to  navigate  on  a hypermedia  system.  A major 
problem  which  needs  further  investigation  is  how  to  produce  the  media  bricks.  To  write  a text  in  a modular  way  is 
quite  unaccustomed,  also  a deeper  understanding  of  the  relations  is  necessary  to  bind  new  media  bricks  in  the 
MediaBrickSpace. 


Acknowledgements 

The  authors  would  like  to  thank  the  Bundesministerium  fur  Bildung,  Wissenschaft,  Forschung  und  Technologie 
(BMBF)  which  partially  funded  the  research  project  and  the  Volkswagen  Stiftung.  We  owe  also  thank  to  our 
colleagues  Abdulmotaleb  El-Saddik,  Martin  Wessner,  Rudiger  Pfister  and  especially  to  Jorg  Haake  for  their  helpful 
comments. 


References 

[1]  Andersen  J.R.,  Boyle  C.F.  Corbett  A.,  and  Lewis  M.;  Cognitive  modeling  and  intelligent  tutoring;  Artificial  Intelligence,  42 

[2]  Beaumont  I.  H.;  Interactive  Anatomy  Tutoring  Systems  ANATOM-TUTOR;  in  Brusilovsky  P.,  Kobsa  A.,  Vassileva  J.; 
Adaptive  Hypertext  and  Hypermedia,  Kluwer  Academic  Publishers,  1 998 

[3]  Brusilovsky  P.,  Methods  and  Techniques  of  Adaptive  Hypermedia;  in  Brusilovsky  P.,  Kobsa  A.,  Vassileva  J.;  Adaptive 
Hypertext  and  Hypermedia,  Kluwer  Academic  Publishers,  1998 

[4]  Devedzic  V.,  and  Debenham  J.;  An  Intelligent  Tutoring  System  for  teaching  formal  languages;  In  Proceedings  of  the 
Intelligent  Tutoring  Systems  Conference,  San  Antonio,  USA  1 998 

[5]  Fischer  D.H.,  From  Thesauri  towards  Ontologies?,  in  Hadi  M.,  Maniez  J.  and  Pollit  St.  A.;  Structures  and  Relations  in 
Knowledge  Organization,  Proceedings  of  the  5 th  ISK  Conference,  1998 

[6]  F.P.  Deck,  J.A.  McHugh;  A review  and  analysis  of  tools  for  learning  programming;  in  Proceedings  of  the  EdMedia,  Freiburg, 
Germany  1998 

[7]  Greer  J.,  McCalla  G.,  Cooke  J.  et  al;  The  Intelligent  Helpdesk:  Supporting  peer-help  in  a university  course;  In  Proceedings  of 
the  Intelligent  Tutoring  Systems  Conference,  San  Antonio,  USA  1998 

[8]  InterBook  Home  Page  http://www.contrib.andrew.cmu.edu/^lb/InterBook.html 

[9]  Mann,  W.C.,  Thomson  S.A.,  Rhetorical  Structure  Theory:  A theory  of  text  organization;  Technical  Report  RS-87-190, 
Information  Science  Institute,  USC  1ST,  USA  1 987 

[10]  Nelson  T.H.,  The  Call  of  the  Ocean:  Hypertext  Universal  and  Open  Hyper  Age,  Vol.  1,  No.  2,  May- June,  1988 

[11]  Nicholas  N.;  Parameters  for  Rhetorical  Structure  Theory  Ontology; 
http://www.arts.unimelb.edu.au/Dept/LALX/publications/rst_ontology.html 

[12]  Sowa  J.,  Ontologies  for  Knowledge  Sharing,  Manuscript  of  the  invited  talk  at  TKE'96,  1996 

[13]  Steinmetz  R.,  Nahrstedt  K.,  Multimedia:  Computing  Communications  & Applications  2nd  edition,  to  be  published,  Prentice 


[14]  Thiiring  M.,  Hannemann  J.,  Haake  J.;  Hypermedia  and  Cognition:  in  Designing  for  Comprehension;  Designing  Hypermedia 
Applications,  Communications  of  the  ACM,  1995 

[15]  Dynamic  Course  Generation  on  the  WWW;  In  Proceedings  of  the  ATED'97  Workshop  on  Intelligent  Educational  Systems 
on  the  World  Wide  Web,  1997 


Hall,  1998 


ERIC 


Automated  Coursework  Assessment  over  the  Internet 


Roger  Moore*and  David  Marshall 
Department  of  Computer  Science 
Cardiff  University 
PO  BOX  912 
Cardiff 
CF2  3XF 

email:  {sCTn6rTn,dave}@cs.  cf.ac.uk 

Abstract 

The  automatic  assessment  of  coursework  can  have  significant  advantages,  especially  when  dealing 
with  large  class  sizes.  Several  automated  marking  systems  have  been  developed  in  recent  years 
[BBF93,  BBF'^Oda,  BBF'^Odb,  LJ95,  TR96],  the  Ceilidh  system  probably  the  most  popular  and 
significant.  All  the  systems  have  basically  required  the  user  to  manually  submit  prepared  pieces  of 
coursework  to  the  system.  In  many  the  cases  this  is  perfect  method  for  submission  of  coursework. 
However,  with  the  possibilities  offered  with  internet  connections  alternative  methods  where  the 
coursework  may  be  meirked  directly  over  internet.  This  is  especially  necessary  if  the  coursework 
involves  testing  some  networking  component  (e.^.  Are  Web  page  links  (URLs)  valid  and  working,  or 
do  CGI  scripts  operate  properly).  This  paper  addresses  the  practical  issues  involved  in  supporting  the 
provision  of  such  a system.  The  system  is  demonstrated  with  implementations  of  marking  students 
Web  Pages  for  HTML  content  and  Perl/ CGI  scripts  for  dynamic  correctness  and  syntax.  However,  we 
emphasise  that  general  principles  of  using  the  internet  as  means  for  automated  coursework  assessment 
and  general  practical  solutions  to  this  problem  are  presented. 

1 Introduction 

The  automatic  assessment  of  coursework  can  have  significant  advantages,  especially  when  dealing  with 
large  class  sizes.  It  is  evident  that  not  all  pieces  of  coursework  can  be  adequately  assessed  by  computer. 
However,  with  careful  consideration  and  design  of  questions  a wide  variety  of  topics  can  be  tested. 
Several  automated  marking  systems  have  been  developed  in  recent  years  [BBF93,  BBF‘^94a,  BBF‘^94b, 
LJ95,  TR96],  the  Ceilidh  system  probably  the  most  popular  and  significant.  All  the  systems  have 
basically  required  the  user  to  manually  submit  prepared  pieces  of  coursework  to  the  system.  In  many 
the  cases  this  is  perfect  method  for  submission  of  coursework.  However,  with  the  possibilities  offered 
with  internet  (or  other  network)  connections  alternative  methods  where  the  coursework  is  placed  on  web 
pages  or  specified  directories  and  the  work  is  marked  over  the  network.  This  is  especially  necessary  if 
the  coursework  involves  testing  some  networking  component  {e.g.  Are  Web  page  links  (URLs)  valid  and 
working,  or  do  CGI  scripts  operate  properly  when  called  over  the  Web). 

The  provision  of  automatic  assessment  can  offer  enhanced  possibilities  both  for  self-study  and  for  class 
administration.  Some  of  these  possibilities  are  listed  below: 

•A  large  number  of  exercises  can  be  assembled,  which  students  attempt  and  submit  interactively 
via  the  courseware.  These  exercises  are  then  marked  by  computer  and  the  marks  returned  to 
the  students.  In  many  cases,  the  marks  can  be  returned  almost  immediately.  Breakdowns  of  the 
marks  can  usually  be  supplied  — thereby  providing  additional  feedback  to  the  students.  Model 
solutions  can  also  be  provided.  Many  more  exercises  can  be  attempted  and  submitted  by  students 
than  is  possible  using  conventional  lecture  techniques  — and,  of  course,  this  method  is  suitable  for 
distance  learning.  The  burden  of  course  administration  is  also  significantly  reduced. 

•Similar  submission  and  marking  procedures  can  be  adopted  for  assessed  exercises.  Additional 
(electronic)  gathering  of  marks  is  easily  facilitated. 

*This  project  was  funded  by  a Nuffield  Undergraduate  Research  Bursary  (Grant  No.  AT/ 100/98/0433) 
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•Checking  for  plagiarism  can  be  more  comprehensive  and  consistent. 

Clearly  different  methods  of  assessment  and  submission  are  required  for  different  disciplines. 

In  recent  years,  the  Ceilidh  automatic  program  marking  system  has  been  a popular,  stand-alone  means 
of  supporting  a wide  variety  of  programming  language  courses  [BBF93,  BBF'^94a,  BBF'^94b]).  We  have 
used  Ceilidh  extensively  for  C programming  courses  at  our  institution  for  many  years.  Current  research, 
we  are  undertaking,  is  investigating  the  possibility  of  utilising  advanced  Ceilidh  features  to  enhance  the 
marking  process  described  in  this  paper.  Ceilidh  is  now  quite  widely  used  in  many  programming  and 
software  engineering  courses  — both  in  the  UK  and  overseas.  In  addition  to  administering  coursework 
submission  and  marking,  the  full  Ceilidh  system  also  supports  lecture  notes.  Unfortunately,  the  lecture 
support  structure  does  not  fit  in  with  our  philosophy  of  delivering  courseware  over  the  WWW,  which 
has  always  been  a major  aim  of  the  courseware  we  have  developed  [Mar95,  Mar98a,  Mar98b,  MH96a, 
MH96b,  MH97].  However,  the  automated  coursework  marking  tools  are  extremely  powerful.  We  have 
(for  the  C course)  provided  an  integrated  Web-based  courseware  environment  that  called  upon  Ceilidh’s 
facilities  for  marking  students’  programs.  Essentially  a Web  form  interface  has  been  provided  for  students 
to  submit  their  programs.  A similar  system  is  now  provided  by  the  original  Ceilidh  developers  (see 
http:  / /www.cs. nott.ac.uk/ ~ceilidh/). 

Other  relevant  systems  for  automated  program  assessment  include  PASS  [TR96]  which  has  addressed 
wider  issues  of  syntax  and  grammar  analysis  of  submitted  (C)  programs  and  BOSS  [LJ95]  which  has 
provided  a user-friendly  interface  for  online  submission  and  supports  several  programming  based  courses. 
However,  the  submission  mechanism  over  the  Web  is  not  always  ideal.  We  have  found  from  our  experi- 
ences with  C coursework  online  submission  that  it  is  cumbersome  for  students  to  develop  their  programs 
externally  of  the  courseware  the  ensure  that  work  correctly  (compiling,  debugging,  test  runs)  and  then 
submit  by  essentially  cutting  and  pasting  the  source  code  into  a Web-based  form.  More  importantly  in 
order  to  assess  that  certain  pieces  of  coursework  work  according  to  specification  they  may  actually  need 
to  test  to  run  over  a network.  A system  is  there  required,  with  many  general  applications,  which  in 
networked  based  and  user  simply  provide  solutions  to  pieces  of  coursework  by  placing  on  a Web-page  or 
in  a directory  which  can  be  accessed  via  a network.  A few  systems  for  assessing  the  validity  of  HTML 
for  Web  Browser  Support [Add98]  and  generally  interpreting  HTML[CAS98,  EWS98]  have  also  appeared 
on  the  WWW  recently. 

The  principle  aim  of  this  paper  is  an  investigation  into  the  practical  issues  involved  in  supporting  the 
provision  of  such  a system.  The  system  is  demonstrated  with  implementations  of  marking  students  Web 
Pages  for  HTML  content  and  CGI/Perl  scripts.  However,  general  principles  of  using  the  internet  as  means 
for  automated  coursework  assessment  and  general  practical  solutions  to  this  problem  are  presented. 

2 The  Assessed  Coursework  — Internet  Computing 

With  over  100  students  taking  the  Internet  Computing  (Year  1,  BSc  Course),  an  alternative  to  ‘mark- 
ing by  hand’  was  required.  Several  assessed  exercises  of  coursework  to  test  students  understanding  of 
other  internet  concepts  such  as  searching,  ftp  and  email  [Mar98b].  However  the  major  piece  of  assessed 
coursework  involved  developing  a Web  page  and  writing  some  CGI/Perl  programs  with  a HTML  form 
interface.  By  careful  consideration  of  the  web  page  content  and  CGI-based  tasks  it  was  possible  to  devise 
effective  exercises  that  could  be  marked  online. 

Students  were  set  specific  tasks  to  achieve  in  HTML  for  their  Web  page,  such  as  having  to  include 
certain  HTML  formatting  including  images,  and  Web  links  on  their  Web  pages.  For  example  students 
were  requested  to  put  their  name,  college  address  etc.  in  certain  font  styles  and  formats,  and  to  include 
an  image  of  themselves.  Students  were  also  required  to  find  and  provide  (external)  links  to  web  pages 
that  address  a particular  topic.  By  choosing  topics  that  have  an  abundance  of  web  pages  devoted  to 
the  topic  (VRML,  Perl  and  Video  were  used  this  year)  one  can  ensure  that  students  can  easily  find  such 
pages  on  the  Web.  Also  it  is  not  unreasonable  to  assume  that  Web  pages  should  include  the  topic  name 
in  Web  page  headers,  title  or  main  text  which  can  easily  be  searched  for  to  verify  the  relevance  of  the 
Web  page. 

CGI  programming  exercises  were  devised  such  that  they  have  a specified  input  and  output.  Therefore 
the  CGI  scripts  can  be  called  via  a Perl  script  with  arguments  specified  within  the  URL  and  the  output 
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checked  by  Perl  pattern  matching.  The  content  of  the  Perl  source  may  also  be  checked.  The  CGI 
programming  exercises  that  were  currently  set  involved  producing  a simple  online  calculator  or  image 
browser  in  Perl  which  were  automatically  marked.  The  marking  system  developed  is  modular  and  other 
exercises  are  planned  as  alternatives  in  the  near  future. 

3 Assessing  Web  Page  and  CGI  Programming  Exercises 

As  mentioned  above  students  were  set  specific  Web  programming  tasks  to  achieve  using  HTML  and 
CGI/Perl.  Basically  the  marking  system  involves  scanning  the  students’  Web  pages  to  verify  content 
and  links  by  simple  pattern  matching.  Web  links  can  be  verified  by  following  the  URL  given  and  checking 
if  the  Web  link  exists  and  if  the  content  is  valid.  Links  to  the  CGI/Perl  exercise  were  also  identified  and 
followed.  The  CGI/Perl  programs  were  then  analysed  for  dynamic  (runtime)  correctness  and  syntax, 
Marks  are  accumulated  by  the  system  as  set  criteria  are  met.  A configuration  file  for  each  exercise  is 
used  to  set  the  appropriate  mark  values  for  each  task.  The  system  is  implemented  using  Perl. 

3.1  Restrictions  Imposed  by  Current  System 

In  order  to  achieve  the  above  automated  marking  some  (not  unreasonable)  restrictions  have  had  to  be 
applied.  These  restriction  do  not  pose  and  real  limit  to  our  system  and  can  be  enforced  as  part  of  the 
exercise  as  described  below: 

•Students  Exercise  Home  page  must  have  specified  location  and  content  — All  students  have  allo- 
cated web  space  which  can  be  easily  accessed  in  a methodical  manner  (e.^. 

http://www.cs.cf.ac.uk/User/Student-Name).  Therefore  a student  class  list  is  assembled  offline 
and  this  list  is  read  by  the  marking  system.  For  each  student  in  turn  the  system  accesses  their  web 
space  and  searches  for  a prescribed  web  page  (exercise.html  for  example).  The  student  web  page 
location,  system  configurations  and  exercise  names  are  input  variables  to  the  system.  In  order  that 
the  CGI  programming  exercises  may  be  located  these  must  also  be  specified.  Since  the  students 
have  been  asked  to  produce  a HTML  page  that  must  adhere  to  certain  criteria  this  insistence  of 
providing  links  can  be  readily  into  incorporated  this  part  of  the  assessment. 

Other  systems  [Add98,  CAS98,  EWS98]  for  more  general  assessment  of  HTML.  In  the  future  we 
intend  to  incorporate  some  of  these  ideas  into  our  system  to  make  it  more  robust  and  general. 
Our  system  implements  a very  simple  method  of  parsing  the  HTML  that  basically  verifies  that  all 
HTML  tags  match  up  and  then  checks  for  specific  criteria  above  to  mark  the  page. 

•Student  CGI  program  links  must  have  specified  name  and  method  of  parameter  passing  — In 
order  for  the  link  to  be  found  from  the  exercise  page  the  exercise  must  have  a link  that  contains 
a specified  name  link  {e.g.  Perl  exercise  1).  CGI  programs  must  also  adhere  to  the  standard 
get/post  methods  of  parameter  passing. 

•Student  CGI  programs  must  have  (or  be  designed  to  produce)  standard  output  — In  order  to  mark 
the  program  dynamically  the  exercises  need  to  be  designed  so  that  specific  output  is  produced 
which  can  easily  be  marked.  This  is  basically  a similar  requirement  to  the  Ceilidh  and  other 
program  marking  systems.  The  dynamic  output  part  of  the  marking  scheme  is  essentially  achieved 
by  pattern  matching  using  regular  expression.  We  use  the  powerful  built  in  regular  expression 
capabilities  of  Perl  to  perform  our  dynamic  marking  of  programs. 

3.2  The  Automated  Marking  System 

We  now  outline  the  basic  algorithms  for  the  modules  that  perform  the  marking  of  the  HTML  and 
CGI/perl  scripts: 

Configuration  and  Main  Module  (Fig.  1)  — This  module  basically  reads  various  configuration  files, 
sets  up  the  appropriate  Web  links  to  students’  web  pages,  calls  the  appropriate  HTML/CGI  mark- 
ing modules  and  finally  assembles  the  final  marking  statistics.  Having  assembled  various  web  links 
it  is  relatively  straightforward  to  download  web  page  source  to  internal  structures  in  the  main 
module  using  the  standard  Perl  User  Agent  package  [Per  98]. 

HTML  Marking  Module  (Fig.  2)  — Students  are  instructed  (as  part  of  the  exercise  requirement)  to 
constrict  a web  page  of  specified  content  and  of  a specified  name.  The  exercise  page  is  therefore 
downloaded,  stored  internally  in  the  module  and  then  assessed  for  style  and  content.  Not  that  we 
are  not  marking  general  HTML  style  but  marking  a page  against  a prescribed  format.  The  format 
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For  each  student  in  class  do 
begin 

Build  home  page  link 
Get  Home  page 
Find  exercise  1 (HTML) 

Download  Exercise  1 page 
Mark  HTML  page  style  and  content 
Find  CGI/Perl  exercise  links 
Download  HTML  Form  exercise  page 
Mark  CGI/Perl  exercise 
end 


Figure  1:  Main  HTML  and  CGI  marking  algorithm 

is  designed  so  as  to  give  students  some  individuality  to  their  page  design.  Some  general  HTML 
page  style  is  checked  such  as  overall  page  length  (no  more  than  1.5  pages)  ajid  that  the  number 
of  images  and  size  of  the  page  downloaded  (including  image  size)  is  not  too  large  (30  Kbytes). 
Web  page  tags  are  identified  and  matched  according  to  the  specified  requirements.  Students  are 
also  required  to  find  and  provide  (external)  links  to  web  pages  that  address  a particular  topic  {e.g. 
VRML,  Perl  and  Video).  Web  links  are  found  in  the  HTML  exercise  page,  checked  for  keywords 
matching  respective  topics  and  the  the  web  link  downloaded.  The  downloaded  web  page  was  then 
searched  for  occurrences  of  the  keyword.  Thus,  we  can  test  and  mark  for  the  validity  of  the  Web 
link  and  the  relevance  of  the  web  page  to  the  topic. 


Parse  HTML: 

Count  number  of  lines,  estimate  page  length 
Check  HTML  tags  mark  according  to  exercise  criteria 

Parse  graphics: 

Find  image  tags  in  page 

Download  images  in  turn 

Check  number  of  images  vs  size  of  files 

Parse  URLs: 

For  each  line  of  page 
Check  for  keyword/URL  link 
Download  URL  link  page 

Check  URL  for  Keyword  in  Title,  Header,  Main  text 


Figure  2:  HTML  marking  algorithm 

CGI/Perl  Marking  Module(Fig.  3)  — Links  to  the  CGI/Perl  scripts  must  be  specified  from  the 
exercise  home  page.  The  link  to  a HTML  form  interface  is  found,  downloaded  and  then  parsed 
to  establish  the  name  and  URL  of  the  associated  Perl  script  and  the  name/value  pair  parameter 
passing  format.  The  Perl  program  is  then  marked  in  two  ways: 

•The  Perl  program  is  run  over  the  Internet  to  check  that  it  can  run  as  a CGI  script  — The 
URL  of  the  Perl  is  formed  and  the  argument  list  is  formed  from  the  name/value  pair  list 
and  appended  as  an  explicit  call  {e,g,  http://my_perLurl/my_perLcgi?namel=vahiel).  The 

output  of  the  perl  script  is  marked  for  dynamic  correctness. 

•The  Perl  source  code  is  downloaded  and  code  is  more  rigoiirously  tested  for  dynamic  cor- 
rectness and  syntax  in  a similar  fashion  to  the  Ceilidh  system [BBF93,  BBF"*'94a,  BBF"*'94b]. 
Several  test  runs  against  set  test  data  are  performed.  Marks  are  awarded  and  accumulated  for 
passing  each  test.  The  precise  weighting  for  each  mark  is  specified  by  a configuration  variable 
for  the  test  and  reflects  the  appropriate  level  of  the  test. The  Perl  syntax  is  check  by  using 
the  Perl  warning  diagnostic  command  line  option  (perl  -w).  In  reality  our  implementation 
is  a simple  version  of  Ceilidh’s  more  powerful  syntax  and  error  checking.  The  perl  -w  com- 
mand performs  a UNIX  lint  (line  interpreter)  type  checking  enabling  us  to  check  for  program 
errors,  unused  variables  etc.  This  diagnostic  checking  process  generates  many  standard  error 
messages  which  can  be  tested  for  (simple  pattern  matching).  Marks  are  deducted  for  the  each 
error  found  which  are  again  preconfigured  and  reflect  the  appropriate  severity  of  the  error. 
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Main: 

Begin 

Analyse  HTML  form  source  for: 
name/value  pair  structure 
find  Perl  script  name  and  URL 

Download  Perl  script  source 
If  Perl  file  in  CGI  directory 
Test^cript_over_internet 
Analyse.Perl^cript 
End  of  Main 

Test^cript-overJnternet: 

Construct  Perl  calling  structure  from  name/value  pair  and  Perl  script  URL 
Execute  Perl  script  over  the  internet 
Analyse  feedback  for  correct  answer 

Analyse.Perl^cript: 

Analyse  for  dynamic  correctness: 

Run  script  with  a variety  of  input  test  data 
Analyse  output  for  correct  answer 
Mark  accordingly 

Analyse  script  syntax  and  efficiency: 

Run  perl  script  locally  with  -w  option 

Parse  output  for  error  messages 

(Syntax,  unused  variables  etc) 

deduct  marks  accordingly  for  respective  errors 


Figure  3;  Form/CGI  marking  algorithm 


4 Conclusions  and  Summary 

The  system  has  been  implemented  and  tested  on  test  user  web  pages  and  last  year’s  Internet  Computing 
class.  The  basic  framework  and  approach  has  been  demonstrated  to  work.  The  system  is  still  however 
basic.  The  biggest  limitation  is  that  only  a small  number  of  exercise  modules  have  been  developed 
— 1 specific  HTML  and  2 Perl  marking.  However,  the  system  has  been  designed  to  be  modular  and 
it  is  a simple  task  to  develop  new  modules.  New  module  development  is  currently  in  progress.  One 
limitation  in  the  Perl  marking  program  is  that  small  customised  Perl  modules  need  to  be  developed 
to  mark  each  CGI/Perl  programming  exercise.  This  is  because  each  exercise  may  have  diflFerent  form 
interface  requirements  and  there  is  a need  to  test  each  program  over  a CGI  network  and  locally.  Prom 
the  lessons  learned  so  far  an  extras  level  of  generalisation  of  this  process  may  lead  to  a more  streamlined 
and  simpler  approach  to  Perl  program  assessment  module  development.  We  are  currently  investigating 
means  of  achieving  this.  We  have  used  the  Ceilidh  automated  programming  marking  system  [BBF93]  in 
other  courses.  Currently,  we  are  developing  scripts  so  that  Ceilidh  may  be  used  to  assist  in  a more  formal 
assessment  of  Perl  programming  style  (and,  maybe,  HTML)  programs.  We  also  intend  to  incorporate 
more  general  HTML  assessment  ideas  [Add98,  CAS98,  EWS98]  into  our  system. 

In  summary  we  have  outlined  a system  that  is  novel  in  that  it  is  attempting  to  assess  coiirsework  over 
the  Internet  which  may  lead  to  an  alternative  means  of  submission  than  current  systems.  Furthermore, 
such  a system  is  requirement  when  internetworking  skills  need  to  be  tested.  We  have  addressed  the 
practical  issues  involved  in  supporting  the  provision  of  such  a system  and  demonstrated  the  system  with 
implementations  of  marking  students  Web  Pages  for  HTML  content  and  Perl/CGI  scripts  for  dynamic 
correctness  and  syntax.  However,  we  believe  that  the  basic  approach  has  outlined  general  principles  of 
using  the  Internet  as  means  for  automated  coursework  assessment. 

The  Internet  Computing  course  is  available  on  the  Web  at  the  URL: 

http://www.cs.cf.ac.uk/Dave/Intemet/  and  anyone  interested  in  obtaining  the  marking  system 
should  contact  the  authors  by  email. 
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Abstract:  As  part  of  a case  study  of  electronic  community  building  in  a college  freshman 
dorm,  discussion  list  messages  were  analyzed  quantitatively  and  qualitatively.  Messages  were 
categorized  by  social  purpose  (housekeeping,  social  dialogue,  or  critical  dialogue)  and  as 
either  forwarded  messages  or  original  messages.  Forwarded  messages  included  jokes, 
polemics  about  politics  or  social  issues,  and  electronic  chain  letters  deemed  offensive  by  some 
participants  - e-mail  forms  that,  along  with  metadiscussion,  have  been  widely  perceived  as 
distractions  and  impediments  to  serious  online  discussion.  A series  of  discussion  threads 
about  gender  issues,  however,  reveals  that  such  “noise  in  the  channel”  can  frequently  lead  to 
constructive  critical  dialogue,  both  online  and  offline,  in  the  context  of  a residential  student 
community. 


Introduction 

Many  studies  have  analyzed  electronic  newsgroups,  scholarly  mailing  lists,  and  other  discussion 
groups  as  virtual  communities  (e.g.,  Rheingold  1994;  Herring  1996).  Other  studies  have  looked  at  classroom 
uses  of  computer-mediated  communication  (CMC)  and  its  effects  on  learning  communities  (e.g.,  Cooper  & 
Selfe  1990;  Harris  & Wambeam  1996;  Colomb  & Simutis  1996).  Only  recently,  however  - in  new  contexts 
such  as  fully-wired  college  dorms  — have  we  had  the  chance  to  study  the  interplay  between  virtual  and  face-to- 
face  community  in  residence-based  communities.  What  are  the  social  effects  and  potentialities  of  CMC  when 
participants  also  live  and  study  together?  Critics  fear  that  students  who  spend  more  and  more  time  in  front  of 
computer  screens  - even  “talking”  to  their  teachers,  classmates,  and  dormmates  online  instead  of  in  person  - are 
becoming  more  isolated.  According  to  the  New  York  Times,  “Some  scholars  say  ‘plug  per  pillow’  campuses  are 
undermining  the  ideal  of  a residential  college”  (Gabriel  1996).  Former  hacker-turned  technology  critic  Clifford 
Stoll  says,  “We’re  turning  colleges  into  cubicle-directed  electronic  experience  and  denying  the  importance  of 
learning  to  work  closely  with  other  students  and  professors,  and  developing  social  adeptness”  (qtd.  in  Gabriel 
1996). 

For  such  critics,  the  whole  idea  of  a dorm  e-mail  discussion  list  would  seem  to  be  antisocial  or,  at  best, 
superfluous.  But  as  I have  argued  elsewhere,  the  fears  of  Stoll  and  others  are  based  on  dubious  either-or  and 
zero-sum  assumptions,  the  faulty  notion  that  computer  activities  and  community-building  activities  must  be 
mutually  exclusive  (Holeton  1997).  On  the  contrary,  residents  of  the  dorm  I studied  used  their  discussion  list  in 
highly  social  ways  and  even  found  the  list  more  valuable  than  traditional  or  face-to-face  media  for  purposes 
such  as  discussing  social  and  political  issues.  Their  live  dorm  programs  and  their  hallway  and  dinner 
discussions  often  complemented  the  e-mail  list,  and  vice-versa  (Holeton  1997).  In  this  kind  of  hybrid 
residential- virtual  community,  participants  share  dorm  hallways  and  lounges,  bathrooms,  mealtimes,  social 
events,  and  intramural  sports  teams  as  well  as  the  electronic  discussion  space.  With  the  familiarity  and 
intimacy  that  are  missing  from  distance-based  virtual  communities,  we  should  not  only  expect  less  rudeness  or 
flaming  online  but  we  should  also  question  previous  assumptions  about  forwarded  e-mail  trivia,  chain  letters, 
and  metadiscussion  - in  particular  the  widespread  perception  that  “in  written  communication  these  messages 
constitute  noise  in  the  channel  that  interrupts  discourse”  (Korenman  & Wyatt  1996,  p.  239).  The  poor 
reputation  of  metadiscussion  in  virtual  communities  is  encapsulated  in  a widely-distributed  e-mail  joke  entitled 
“How  many  Internet  mail  list  subscribers  does  it  take  to  change  a light  bulb?”  Answer:  1,331: 

1 to  change  the  light  bulb  and  to  post  to  the  mail  list  that  the  light  bulb  has  been  changed 

14  to  share  similar  experiences  of  changing  light  bulbs  and  how  the  light  bulb  could  have  been 
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changed  differently 

7 to  caution  about  the  dangers  of  changing  light  bulbs 

27  to  point  out  spelling/grammar  errors  in  posts  about  changing  light  bulbs 

53  to  flame  the  spell  checkers 

156  to  write  to  the  list  administrator  complaining  about  the  light  bulb  discussion  and  its 
inappropriateness  to  this  mail  list.. .. 

[etc.;  about  fifteen  additional  categories  of  responses  typically  follow] 

Must  forwarded  messages,  inane  or  offensive  jokes,  and  metadiscussion  serve  only  as  time-wasting 
distractions,  irritants,  and  bandwidth  hogs?  To  what  extent,  instead,  can  they  function  as  constructive 
discussion  prompts  or  make  positive  contributions  to  residential  and  virtual  community  building? 


About  the  Study 

Rinconada  House,  Wilbur  Hall,  is  an  all-freshman  dorm  of  94  students  (including  5 upperclass 
residence  staff)  at  Stanford  University,  where  during  the  1990s  all  dorms  were  wired  with  in-room  ethemet 
connections  to  complement  networked  residence  computer  clusters.  The  study  is  based  on  a complete  archive 
of  over  1200  messages  posted  to  the  dorm  e-mail  list  for  the  academic  year  1995-96.  Rinconada’s  history 
includes  the  use  of  dorm  e-mail  discussion  lists  for  a number  of  years  and  its  1 993  claim  as  the  first  college 
dorm  with  a home  page  on  the  web.  My  wife  and  I served  as  Rinconada’s  Resident  Fellows,  faculty  leaders  of 
the  residence  staff  who  live  in  an  adjacent  cottage,  from  1990  to  1997,  and  thus  - especially  where  qualitative 
judgments  are  made  --  I am  necessarily  a participant-observer  in  the  study.  All  the  house  residents,  including 
the  staff,  were  subscribed  to  the  discussion  list.  The  full  study,  including  data  from  which  this  paper  is  derived, 
is  available  on  the  web  at  http://www.stanford.edu/'^holeton/wired~pages/wired~main.html. 

The  messages  were  broadly  categorized  according  to  three  larger  social  purposes,  which  suggest  a 
rough  hierarchy  from  lower-order  to  higher-order  uses  of  CMC: 

1 . Housekeeping  (lost  and  found,  arranging  meeting  times,  etc.) 

2.  Social  dialogue  (publicizing  dorm  or  university  events  and  programs,  sharing  outside  interests, 
relieving  stress,  etc.) 

3.  Critical  dialogue  (discussing  social,  political,  academic,  or  intellectual  matters;  discussing  dorm 
community  issues  or  controversies;  etc.)^*^ 

Generally  speaking,  critical  dialogue  is  the  type  of  discussion  encouraged  in  academia,  where  matters  of  social 
or  intellectual  importance  are  negotiated  in  a reasoned,  collegial  way,  and  its  inherent  value  in  learning 
communities  will  be  presumed.  Metadiscussion  — discussion  about  the  medium  itself  or  how  the  list  should  be 
used,  or  “talk  about  talk”  (Korenman  & Wyatt  1996,  p.  238)  - was  classified  as  critical  dialogue.  Messages 
were  also  categorized  as  either  (a)  forwarded  messages  or  (b)  original  messages  or  replies.  Forwarded 
messages  are  those  passed  on  to  the  list  by  members  but  whose  content  originated  from  a person  or  organization 
outside  the  group  (the  list  was  closed  to  unsolicited  junk  e-mail  or  direct  “spam”  from  Internet  mass-mailers). 
In  the  many  cases  where  forwarded  content  was  framed  by  the  sender’s  own  commentary,  I made  a judgment 
about  whether  the  primary  content  of  the  message  was  the  forwarded  material  or  the  original  material. 

After  measuring  the  various  proportions  of  these  message  categories,  I analyzed  the  relationship 
between  forwarded  messages  and  sustained  threads  of  the  higher-order,  critical  dialogue.  I also  considered  the 
effects,  in  my  observation,  on  the  face-to-face  community.  My  discussion  focuses  on  the  most  sustained  critical 
dialogue  of  the  year,  a series  of  related  threads  about  gender  issues  that  comprised  dozens  of  messages  and 
lasted  most  of  autumn  quarter. 


Findings  and  Discussion 

In  addition  to  the  numerous  political  calls  to  action,  urban  legends,  electronic  chain  letters.  Top  Ten 
lists,  and  jokes  that  have  proliferated  on  the  Internet,  a whole  genre  of  undergraduate  e-mail  humor  has 


[1]  In  designing  these  categories  I was  influenced  by  Ziv’s  ‘Taxonomy  of  Communicative  Purposes”  (Ziv  1996,  Table  2,  p. 
247). 
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developed  as  college  students  got  online  throughout  the  1 990s.  Typical  titles  in  this  genre  include  “50  things  to 
tell  your  professor  when  you  turn  your  exam  in  late”  and  ‘TOO  ways  to  torture  your  roommate.”  A more 
insidious  subgenre  includes  material,  often  with  humorous  intent,  deemed  offensive  by  women,  men,  specific 
ethnic  groups,  gay  or  lesbian  people,  or  other  social  groups.  These  messages  are  widely  forwarded  among 
college  students  around  the  world,  and  the  Rinconada  list  was  no  exception. 

In  terms  of  larger  social  purpose,  from  lower-order  to  higher- order  uses  of  CMC,  1 3%  of  the  total 
messages  in  1995-96  were  classified  as  housekeepings  57%  as  social  dialogues  and  30%  as  critical  dialogue. 
One-quarter  (n=306  out  of  1244  or  24.6%)  of  the  total  messages  were  classified  as  forwarded  messages.  The 
vast  majority  of  forwarded  messages  fell  into  the  social  dialogue  category  (and  more  specifically,  the  sub- 
categories “Sharing  outside  interests”  or  “Relieving  stress”).  Of  the  711  social  dialogue  messages,  269  or 
nearly  38%  were  forwarded  messages.  See  Figure  1 . 


Figure  1:  Proportions  of  Forwarded  Messages  by  Social  Purpose.  The  two  arrows 
indicate  that  forwarded  messages  in  the  social  dialogue  category  frequently  led  to 
critical  dialogue  messages  or  to  “answering”  forwards. 

The  majority  of  this  busy  traffic  in  forwarded  messages  were  ignored  by  other  participants,  so  initially 
we  might  be  tempted  to  view  them,  as  others  have,  as  “noise  in  the  channel.”  Anecdotally,  some  residents 
reported  being  put  off  by  common  Internet  banalities  such  as  those  described  above.  On  the  other  hand,  I 
would  argue  that  there  is  some  inherent  value,  in  a diverse  educational  community,  in  participants  being 
exposed  to  one  another’s  range  of  interests  and  passions  as  reflected  in  forwarded  e-mail  messages.  Further,  a 
lack  of  online  response  doesn’t  necessarily  mean  that  a joke  or  piece  of  trivia  wasn’t  appreciated  or  didn’t 
provide  some  needed  levity.  Residents  frequently  made  offline  responses  to  online  chatter,  a phenomenon  that 
should  be  studied  further  as  a potential  contributor  to  community  building.  Online,  some  participants  answered 
forwards,  playfully  or  seriously,  with  another  forwarded  message  taking  an  alternate  or  opposing  view.  For 
example,  a jocular  list  entitled  “How  to  tell  if  you’re  a Republican”  would  be  answered  by  an  equally-jocular 
“How  to  tell  if  you’re  a Democrat.”  Sometimes  this  kind  of  sparring  led  to  a substantive  discussion  thread 
when  participants  framed  their  answering  forward  with  extensive  editorial  comments. 

Although  many  forwarded  messages  went  unanswered,  the  most  sustained  threads  of  critical  dialogue 
that  developed  over  the  year  all  began  as  reactions  to  forwarded  messages.  As  we  might  expect,  the  most 
controversial  or  inflammatory  messages  - those  with  the  most  loaded  social  or  political  content  of  interest  to 
undergraduates  - evoked  the  most  reaction,  including  forwards  about  sex,  rape,  neo-Nazis,  and  free  speech. 
The  remarkably  constructive  quality  of  the  dorm’s  reactions  to  these  prompts,  which  dispute  the  claims  of 
pessimists  that  virtual  communities  are  inevitably  being  broken  down  by  “flamers,  cranks,  fetishists  and 
monomaniacs”  (Chapman  1995),  is  revealed  especially  in  the  autumn-long  discussion  threads  about  gender 
issues. 


Subject:  Pm  disgusted  (fwd) 
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In  October,  1995,  an  infamously  misogynistic  e-mail  originating  at  Cornell  University  called  ‘Top  75 
reasons  why  women  (bitches)  should  not  have  freedom  of  speech”  was  widely  disseminated  at  colleges.  It 
evoked  a furor  that  made  national  headlines  (e.g.  Grunwald  1995).  Some  of  the  more  politely-phrased 
“reasons”  included,  “If  she’s  in  the  kitchen  like  she  should  be,  no  one  can  hear  her  anyway”  (no.  2),  “If  she 
can’t  speak,  she  can’t  cry  rape”  (no.  38),  and  “Only  one  set  of  lips  should  be  moving  at  a time”  (no.  43). 
Phyllis^’ ^ forwarded  the  message  to  the  Rinconada  list  with  the  subject  header  “I’m  disgusted,”  initiating  a 
month-long  discussion  thread. 

Vigorous  metadiscussion  about  the  appropriateness  of  posting  the  “75  reasons”  message  to  the  list 
ranged  from  Duncan’s  plea  to  “think  twice  before  passing  this  kind  of  thing  on”  to  defenses  of  freedom  of 
speech.  The  staff  did  not  intervene  in  this  discussion  but  purposely  let  residents  negotiate  norms  for  the  e-mail 
list,  attempt  to  establish  shared  values,  and,  hopefully,  assert  collaborative  ownership  of  this  electronic  space. 
Their  initial  success  is  indicated  by  Adelle’s  assessment  of  the  metadiscussion  (i.e.,  meta-metadiscussion?):  “It 
was  nice  that  for  once  people  expressed  their  ideas  about  e-mail  without  becoming  rude  or  over  personal.  I 
guess  people  are  starting  to  become  more  considerate  of  others  over  the  e-mail.  What  was  so  hard  about  doing 
this?” 

The  “four  players  of  Cornell”  who  composed  the  controversial  message  (and  even  put  their  real  names 
on  it)  got  flamed  from  far  and  wide,  issued  apologies,  and  faced  disciplinary  charges.  Meanwhile  at  Stanford 
on  the  Rinconada  list,  Gregory  started  a serious  discussion  of  gender  issues  by  objecting  to  how  such  incidents 
reinforce  negative  stereotypes  about  men.  Several  other  men  participated  thoughtfully  in  this  thread.  For 
example,  Duncan  wrote,  “On  behalf  of  my  gender  I would  like  to  emphasize  that  the  majority  of  men  do  not 
think  this  way  about  women,  that  many  find  it  sick  that  some  people  actually  perpetrate  such  filth,  and  that  the 
guys  who  wrote  this  in  the  first  place  are  probably  loving  every  angry  response  that  they  receive.” 


Subject:  Re:  Something  Serious  To  Think  About  (fwd) 

Less  than  two  weeks  after  the  “75  reasons”  forward,  an  e-mail  chain  letter  originally  called  “YOUR 
PARENTS  WERE  RIGHT  ABOUT  THIS  ONE”  was  forwarded  to  the  list  by  Mona  with  the  new  subject 
header  “Something  Serious  To  Think  About.”  Interestingly,  Faith  had  previously  sent  this  message  only  to  the 
women  of  the  dorm,  and  now  Mona  prefaced  her  posting  with  explicit  appeals  to  her  male  dormmates  for  a 
continuation  of  reasoned  discussion: 

To  all  the  women  of  the  dorm,  sorry  for  cluttering  up  your  e-mail  with  a repeated  message,  but  I 
wanted  to  send  this  to  the  guys,  and  this  was  way  easier.  Guys,  please  don’t  take  this  as  an  attack 
on  males,  cuz  it’s  not.  I just  thought  it  was  an  important  message  for  the  state  of  the  world  we  live 
in,  regardless  of  gender.  Faith  was  great  enough  to  send  this  to  all  the  girls,  and  I thought  there 
might  be  guys  in  the  dorm  who  would  appreciate  it  as  well  (for  male/female  friends,  for 
themselves,  or  just  as  a reminder  that  you  can’t  trust  everyone  in  this  world,  unfortunately). 

Mona  concluded  her  persuasive  preface  with  gender-neutral  appeals  to  dorm  spirit  and  Stanford  spirit  (“Rinc 
Rules  and  Weenies  Got  Roasted!”  — a reference  to  the  recent  Big  Game  football  victory  over  Cal-Berkeley). 
The  forwarded  narrative  itself  depicts  a generic  date  rate  (“He  knocks  on  the  door.  She  doesn’t  really  know  him 
well,  but  her  friends  party  with  him  often.  So,  she  smiles  and  settles  back  onto  her  bed  as  he  walks  in....”), 
offers  moral  lessons  (“we  all  learned  in  kindergarten  - ask  before  taking,  and  don’t  touch  what  isn’t  yours”), 
and  exhorts  readers  to  spread  the  word: 

This  is  a chain  letter.  Send  one  to  the  people  you  care  about,  or  are  afraid  of.  In  any  school,  in  any 
country.  Please  write  the  name  of  your  school  at  the  bottom,  and  place  an  X beside  it  if  someone 
you  know  has  been  a victim  of  assault  or  rape.  [A  long  list  of  colleges  and  universities  marked 
with  Xs  follows.] 

In  response,  the  previous  thread  about  male  stereotyping  evolved  into  a discussion  about  rape,  with 
extensive  contributions  from  Gregory,  Ronald,  Buff,  Betty,  and  others.  Gregory  and  Ronald  argued  that  the 
generic  date  rape  in  the  narrative  grossly  oversimplified  what  can  be  a complex  series  of  miscommunications. 


[1]  All  names  are  pseudonyms. 


691 


Buff  sent  a four-page  self-described  “treatise”  on  rape  rebutting  Gregory’s  and  Ronald’s  arguments,  Ronald 
sent  a rebuttal  to  the  rebuttal,  and  so  on.  Six  days  into  the  thread,  Betty  tried  to  get  the  discussion  refocused  on 
basic  feelings  and  human  communication,  suggesting  that  the  argument  had  gotten  too  analytical  and  theoretical 
- a classic  example  of  what  some  have  identified  as  a female  communication  style  (e.g.,  Tannen  1994) 
contrasted  with  the  male  “agonistic,”  warlike  debate  style  (Ong  1981): 

I think  it’s  fine  to  analyze  all  the  little  points  of  the  law  and  of  ethics  on  a theoretical  level,  but  it 
seems  to  me  that  the  real  issue  here  is  a lot  more  broad.  It’s  about  respect  and  communication. 

You  should  respect  yourself  enough  to  both  voice  your  desires  or  concerns  and  then  respect  your 
partner  enough  to  ask  about  their  feelings  and  listen  to  their  response. 


Subject:  75  REASONS.  PLEASE  FORWARD,  (fwd) 

Yet  another  forwarded  message  - “75  Reasons  Why  Angry  Cornell  Women  (Your  Worst  Nightmare) 
Are  Exercising  Their  Freedom  of  Speech,”  a reply  to  the  original  inflammatory  “75  reasons”  list  - re-ignited  the 
gender  threads  again  in  early  December.  It  was  posted  by  Carolyn,  Rinconada’s  Residence  Computer 
Coordinator  (RCC),  an  upperclass  member  of  the  residence  staff  The  Cornell  women’s  message  cited  a 
number  of  statistics  from  the  Women’s  Action  Collective,  establishing  a scholarly  tone.  Some  men  on  the 
Rinconada  list,  such  as  King,  reacted  defensively:  “Men  are  often  depicted  as  villains  that  suppress  and  abuse 
women. . ..  I want  to  question  the  validity  of  some  of  the  statistics  given  in  the  message.”  Ronald  questioned  the 
Women’s  Action  Collective’s  statistics  by  offering  his  own  research  and  citations  from  the  Western  Journal  of 
Medicine,  The  New  Republic,  and  other  sources,  as  in  a formal  academic  paper.  And  Betty  and  Clarisse 
challenged  the  men  with  a thread  called  “Could  we  just  forget  semantics  for  a minute?” 

In  a virtual  touche  that  seemed  to  go  largely  under- appreciated  by  the  men,  Clarisse  connected  the 
political  discussion  about  gender  directly  to  the  dorm  face-to-face  community,  chastising  the  “guys”  for 
referring  to  Carolyn,  the  RCC,  as  “The  Computer  Lady.”  Although  Clarisse  qualified  her  objection  as  a 
“random  thing,”  it  wasn’t  random  at  all,  because  of  course  Carolyn  had  posted  the  Cornell  women’s  rebuttal. 

Along  with  Carolyn,  the  rest  of  the  residence  staff  had  many  opportunities  to  make  similar  connections 
between  the  virtual  and  the  residential  community.  We  organized  formal  residential  education  programs  with 
gender  themes,  such  as  a consciousness-raising  exercise  led  by  the  Resident  Assistants  called  “Gender 
Alliances.”  Not  surprisingly,  numerous  conversations  at  meals  and  late  at  night  in  the  hallways  extended  and 
complemented  the  online  discussions.  Such  conversations  fed  back  into  the  e-mail  discussion  list  as  more 
participants  got  involved;  60%  of  the  dorm’s  residents  participated  in  critical  dialogue  threads  at  one  time  or 
another. 

Moreover,  the  value  of  all  this  online  discussion  about  gender  issues  was  by  no  means  limited  to  those 
who  did  most  of  the  “talking.”  A survey  I conducted  near  the  end  of  the  year  showed  that  even  those  who 
participated  little  online  were  active  as  readers  of  the  list  and  thought  it  was  very  useful  for  discussion  of  social 
and  political  issues.  These  so-called  “lurkers”  were  not  necessarily  inactive  in  the  face-to-face  community  - 
and  vice-versa,  the  most  active  e-mail  participants  were  not  all  similarly  active  in  physical  dorm  activities 
(Holeton  1997).  Thus,  online  interaction  and  offline  interaction  together  offered  residents  more,  often  mutually 
reinforcing  ways  of  communicating,  exploring  and  negotiating  ideas,  and  getting  to  know  one  another.  In  these 
complex  and  sometimes  surprising  ways,  the  “four  players  of  Cornell,”  the  “Angry  Cornell  Women,”  and  the 
purveyors  of  other  chain  letters  and  forwarded  messages  helped  enhance,  rather  than  diminish,  the  community 
experience  at  Rinconada  House. 


Conclusions 

A dorm  e-mail  discussion  list  or  other  electronic  discussion  space  can  be  very  useful  for  the  kinds  of 
critical  dialogue  encouraged  in  academia.  At  least  in  this  higher  education  context,  the  kinds  of  forwarded  e- 
mail  trivia,  controversial  messages,  and  metadiscussion  that  have  disrupted  other  virtual  communities  need  not 
preclude  higher-order  uses  of  CMC.  Messages  forwarded  by  participants  from  outsiders,  even  if  annoying, 
preachy,  or  distasteful,  can  prompt  substantive  discussion  and  serve  as  valuable  tools  for  building  community. 
On  the  Rinconada  1995-96  e-mail  list,  the  most  engaging  and  successful  critical  dialogue  threads  of  the  year 
started  as  reactions  to  controversial  forwarded  messages. 


er|c 


692 


With  the  proliferation  and  specialization  of  e-mail  lists  in  the  late  1990s,  some  college  dorms  have 
begun  separating  discussion  lists  from  announcement-only  lists.  This  kind  of  strategy  for  filtering  out 
forwarded  messages  might  result  in  missed  opportunities  for  higher-order  electronic  discussion  and  associated 
benefits  for  the  community. 

Hybrid  virtual-residential  communities  like  today’s  fully-networked  college  dorms,  where  residents 
have  a good  deal  at  stake  in  their  relationships  both  online  and  offline,  will  become  increasingly  common  in  the 
near  future.  Teachers,  residential  educators,  list  administrators,  and  others  with  experience  in  such  hybrid 
communities  should  continue  to  explore  the  effects  and  best  uses  of  electronic  discussion  and  other  emerging 
technologies. 
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Abstract:  Distance  Learning  has  become  a standard  practice  in  educational  institutions.  This 
paper  describes  an  action  -oriented  approach  to  that  enables  anyone,  from  faculty  members  to 
administrators,  to  become  effective  change  agents  for  DL.  Five  factors  are  discussed: 

a.  Understanding  your  Institution:  change  agents  understand  their  institution,  often 
by  taking  an  outside  view. 

b.  Designing  for  “Fit”:  successful  DL  efforts  are  aligned  with  an  institution’s 
processes,  from  curriculum  changes  to  the  strategic  plans  and  objectives  of 
stakeholders  at  all  levels. 

c.  Designing  for  Quality;  Faculty  as  Intellectual  Capital:  utilizing  the  faculty 
throughout  the  DL  implementation  process  can  create  faculty  buy-in.  Faculty  also 
can  provide  a perspective  on  the  deliveiy  technology  chosen. 

d.  Designing  for  Accreditation:  as  more  programs  move  online,  it  is  necessaiy  to  be 
aware  of  the  expectations  of  accrediting  organizations. 

Designing  for  Integration  and  Acceptance:  activities  that  smooth  the  way  for  DL 
acceptance  inside  the  school  are  informational  activities,  publishing,  clarifying 
intellectual  ownership,  and  providing  a technology  development  center. 


Introduction 

As  Distance  Learning  continues  to  become  a mainstream  educational  practice,  schools  are  addressing  it  less  as  an 
innovation  and  more  as  a challenge  to  existing  institutional  cultures.  Those  who  first  taught  DL  classes  and  designed 
delivery  systems  are  now  becoming  spokespersons  and  advocates  to  those  in  their  schools  who  feel  threatened,  are 
not  convinced,  or  are  simply  not  interested. 

However,  the  appearance  of  online  courses,  certificates  and  degrees,  many  blessed  by  traditionally-oriented 
accrediting  bodies,  is  creating  a new  land  rush:  rather  than  the  question  Should  we  Go  Online,  the  question  in  many 
schools  has  been  jump-started  to  How  Quickly  Can  We  Get  There?  Acceptance  by  accreditors,  combined  with  the 
ubiquity  of  the  World  Wide  Web  and  other  online  course  presentation  and  management  tools,  is  creating  a race  to 
mount  online  programs  of  all  sorts. 

Such  rapid  changes  can  create  tension  in  the  culture  of  institutions.  Instead  of  the  meandering  pace  of  change  that 
schools  are  used  to.  Distance  Learning  has  seemingly  appeared  in  the  US  without  precedent  or  warning  (its 
worldwide  history  notwithstanding),  elbowing  aside  accepted  practices  of  development  and  delivery.  As  well,  DL  is 
refocusing  debate  on  issues  such  as  intellectual  property,  course  ownership  and  faculty  workloads. 

Fortunately,  the  knowledge  level  of  decision-makers  is  on  the  rise.  We’re  moving  from  the  notion  of  solitary  faculty 
members  designing  online  courses  to  an  awareness  that  viable  distance  programs  involve  interactive,  long-term 
efforts  among  diverse  institutional  constituencies. 
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This  paper  discusses  an  action-oriented  approach  for  DL  advocates  and  change  agents  to  use  as  they  integrate 
quality-oriented  distance  delivery  courses  and  programs  into  an  existing  school  world.  After  presenting  3 potential 
DL  scenarios,  the  following  issues  are  discussed: 

Understanding  your  Institution 
Designing  for  “Fit” 

Designing  for  Quality;  Faculty  as  Intellectual  Capital 

Designing  for  Accreditation 

Designing  for  Integration  and  Acceptance 


Scenarios 

a.  One  or  a few  faculty  members  create  and  present  courses  online,  probably  in  a bulletin  board,  email,  or  perhaps 
World  Wide  Web  environment. 

This  leads  to  . . . 

b.  The  school  begins  to  support  the  development  and  presentation  of  courses,  certificates  and  (increasingly) 
degrees  online,  in  varied  delivery  modes.  There  may  be  some  support,  but  efforts  are  still  by  individuals  or 
small  groups. 

c.  The  school,  recognizing  the  sea  change  and  the  potential  that  distance  education  represents,  builds  multifaceted 
distance  delivery  into  its  mission  and  its  long-and-short-term  strategic  planning.  Distance  technologies  and 
development  are  line  items  in  the  budget.  The  institution  does  not  immediately  buy  into  one  delivery  mode,  but 
understands  that  tools  and  approaches  continue  to  evolve.  Faculty  and  development  staffs  are  adequately 
supported  in  development  and  research,  and,  in  return,  they  operate  with  reasonable  performance  expectations. 
Developers  are  encouraged  to  understand  and  use  not  only  the  “new  traditional”  technologies  (conferencing, 
Web  and  Web-enhanced),  but  emerging  technologies. 

"C"  is  very  probably  fantasy.  It’s  more  than  likely  that  you  and  your  institution  are  at  a middle  point,  with  perhaps 
some  distance  education  implementation  in  place,  probably  not  enough  staff  and  budgetary  support,  and  an 
undifferentiated  sense  of  urgency  at  the  administrative  level  that  DL  programs  need  to  be  developed  and 
implemented. 

Theoretically,  strategic  planning  drives  institutional  activities.  In  reality,  projects  that  are  successful  tend  to  be 
incorporated  into  strategic  planning  after  the  fact,  depending  on  the  degree  of  their  success  and  the  fit  they  find  in  a 
given  institution.  With  that  in  mind,  let’s  discuss  a number  of  activities  (and  thought  processes)  at  several  levels  that 
can  help  the  change  agent  move  her  or  his  institution  toward  successful  DL  planning  and  integration.  First,  we  need 
to  revisit  our  institutional  culture. 


Understanding  your  Institution 

Why  is  understanding  your  institution  important?  The  development  process  may  be  driven  by  the  DL  bandwagon, 
but  it’s  still  directed  by  the  many  well-known  dynamics  of  organizational  change.  These  forces  (resistance, 
withholding  support,  differences  in  expectations  between  decision-makers  and  implementers,  conflicts  among 
constituents,  a “show  me”  attitude)  are  well  known  and  must  be  recognized  to  be  effectively  addressed.  Since  any 
culture  tends  to  be  invisible  to  those  within  it,  the  change  agent  who  is  a member  of  an  institutional  community 
should  first  step  back  and  take  a look  at  his  or  her  school,  with,  as  the  comedian  Gallagher  says,  “new  eyes.” 


Designing  for  Fit 
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A subsequent  step  is  to  develop  meta-objectives  for  the  distance  program.  Consider  the  institution’s  mission  and 
existing  programs,  and  position  DL  to  align  with  them.  This  is  a fundamental  step  in  creating  a long-term  support 
dynamic.  Work  with  stakeholders  at  all  levels,  from  Board  of  Trustees  members  to  staff,  if  possible. 

Position  any  existing  DL  courses  and  programs  in  such  a way  that  they  correlate  with  the  emerging  objectives  of  the 
institutional  effort.  Ideally,  this  sort  of  strategic  planning  should  be  addressed  before  nuts-and-bolts  development 
begins,  but  in  practice,  development  frequently  involves  aligning  any  new  programs  with  an  institution’s  existing 
culture.  Regardless  of  the  situation,  the  change  agent  should  focus  on  growing  the  distance  program  both  within  the 
institutional  culture  and  with  an  eye  toward  development  and  marketing  avenues. 


Developing  for  Quality;  Faculty  as  Intellectual  Capital 

For  long  term  credibility  and  success,  courses  and  programs  should  be  of  the  highest  quality.  Institutional  and 
programmatic  quality  descriptors  may  grow  out  of  DL  planning,  but  faculty  members  should  be  at  the  front  line  of 
quality  at  the  course  content  and  presentation  level.  Faculty  represent  the  intellectual  and  educational  capital  of  the 
institution,  and  their  contributions  should  be  nurtured  and  protected  as  much  as  possible. 

While  faculty  members  are  an  important  resource,  they  may  resist  becoming  involved  with  DL  for  many  reasons.  An 
important  conflict  that  invariably  arises  is  the  traditional  notion  that  intellectual  capital  resides  with  the  individual 
instructor,  and  is  usually  manifested  in  the  face-to-face  classroom.  Expanding  this  paradigm  to  fit  teaching  in  the 
largely  unknown  territory  of  virtual  space  is  a complex  issue,  and  it  encapsulates  other  questions  of  course 
ownership  and  intellectual  property.  Working  supportively  with  faculty  is  important.  It  will  help  resolve  such  issues, 
and  will  demonstrate  to  the  body  of  faculty  members  that  their  contributions  to  the  integrity  of  the  institution  are 
understood  and  valued. 

Another  benefit  of  this  perspective  is  that  an  intellectual  capital  approach  to  planning  and  design  can  help  forestall 
the  inevitable  temptation  to  choose  technology  delivery  mode  over  course/program  content.  If  the  institution  first 
settles  on  a delivery  mode  and  provider,  (which  administrators  will  tend  to  do)  then  content  delivery,  along  with  the 
entire  look  and  feel  of  your  institutional  program,  will  be  constrained  by  the  system  that  is  chosen. 

If  institutional  constraints  do  make  it  necessary  to  choose  a standardized  delivery  system,  be  sure  to  engage  faculty 
members  and  developers  in  understanding  how  the  system  allows  for  the  use  of  technology  features  as  they  evolve. 
Currently,  audio/ video,  web  interfaces  and  the  incorporation  of  course  management,  multimedia  production  and 
presentation  are  features  that  are  supported  by  some  systems,  and  not  by  others.  In  the  near  future,  any  substantial 
delivery  system  will  have  to  support  video  conferencing,  interactive  database  access,  streaming  media,  multimedia, 
QT/AVI  audio/video  and  World  Wide  Web  access,  and  users  will  have  to  know  how  to  incorporate  them  into 
content  and  course  presentation. 

Implementing  a flexible  system  is  important  for  another  reason.  Whatever  delivery  system  is  chosen,  it  will  become 
a traditional  organizational  entity,  which  will  then  resist  subsequent  intrusions  of  new  technologies  and  ideas. 


Developing  for  Accreditation 

Designing  for  quality  implies  attention  to  translating  courses  and  programs  into  online  environments,  not  only  to 
attract  students  but  also  to  maintain  standards  that  will  satisfy  your  school’s  accrediting  body. 

Accreditation  standards  for  online  programs  are  still  a murky  issue,  although  standards  are  emerging,  and 
completely  online  certificates  and  degrees  are  appearing.  Since  institutional  accreditation  is  frequently  involved,  DL 
developers  should  be  in  close  contact  with  the  school’s  accrediting  body.  The  Western  Association  of  Schools  and 
Colleges,  for  example,  currently  allows  programs  to  offer  up  to  50%  of  courses  in  existing  programs  via  online 
delivery  modes  before  accreditation  review  is  called  for.  At  that  point,  incremental  changes  or  proposals  for  online 
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programs  must  go  before  the  Substantive  Change  committee  for  scrutiny.  (These  requirements  are  in  flux,  and  may 
have  changed  by  the  time  this  is  published) 


Designing  for  Integration  and  Acceptance 


This  concluding  section  discusses  a number  of  activities  and  processes  that  can  be  undertaken  on  an  ongoing  basis. 
These  are  some  of  the  “little  things”  that  will,  in  the  long  run,  help  you  to  open  doors  for  DL  development,  and  that 
will  grow  credibility. 

• Work  toward  incorporating  DL  into  the  university’s  administrative  and  financial  structure.  There  should  be  distinct 
budget  lines  for  DL  support,  delivery  and  design.  The  DL  entity  should  have  representation  on  faculty  and 
institutional  governance  bodies,  and  should  report  institutionally  at  the  Vice  President  level. 

• Develop  a communication  process  that  shares  information  and  invites  involvement  from  the  university  community. 
Giving  community  members  ready  access  to  information  about  development  will  encourage  awareness  and  buy-in. 

Newsletters,  flyers,  memos  and  in-house  press  releases  are  effective,  but  more  impact  will  be  created  by  using  the 
technology  itself  as  a primary  communication  vehicle,  perhaps  via  an  institutional  Intranet , groupware  activities  or 
an  electronic  bulletin  board.  A good  introductory  use  of  such  technology  is  newsgroups,  or  perhaps  designing  online 
textbook  order  forms  that  faculty  are  expected  to  use. 

• Implement  a user  Development  Center.  Whatever  form  such  a center  takes,  it  should  be  equipped  with  high-end 
technology  tools  sufficient  to  allow  DL  developers  to  experiment,  practice  and  apply  technology  ideas. 
Technological  and  pedagogical  support  should  be  provided.  Such  a center  will  have  practical  application,  but  the 
real  impact  will  be  its  demonstration  of  the  school’s  commitment  to  Distance  Learning. 

• Promote  ongoing  faculty  involvement  and  ownership  of  course  materials. 

Placing  courses  online  for  the  first  time  usually  involves  moving  existing  content  to  text-based  online  activities.  This 
is  time  consuming  but  relatively  easy,  and  a traditional  course  can  be  translated  to  a reasonably  effective  online 
course  this  way. 

Ideally,  this  first  time  translation  is  only  the  first  step  in  growing  a sophisticated  online  course  that  utilizes  the  many 
technology  tools  available.  Faculty  should  be  involved  (and  supported)  in  course  development  over  time.  If 
ownership  and  subsequent  development  of  online  courses  are  handed  off  to  the  distance  learning  entity  as  soon  as  a 
course  is  transferred,  then  quality  may  flag,  because  the  interest  and  expertise  of  the  primary  instructor  will  be  lost. 

• DL  practitioners  should  be  encouraged  to  demonstrate,  present  and  publish,  in  both  local  and  larger  venues. 

• Encourage  early  adopters  and  innovators,  wherever  they  are  found.  They  are  role  models  for  others,  and  they  will 
continue  to  demonstrate  how  technology  can  be  used  in  obvious  and  less-than-obvious  ways.  If  they  are  involved  in 
development  or  if  they  use  the  Development  Center,  not  only  their  discoveries  but  also  their  processes  for  using 
technology  can  be  codified  and  shared  with  others. 

• Share  progress  internally  and  in  the  public  community.  This  can  mean  providing  copy  for  student  and  faculty 
newspapers  as  well  as  local  media.  Schools  are  notoriously  memo-centric,  so  non-traditional  venues  for  sharing 
successes  should  be  chosen  to  ensure  attention. 

• Value  and  use  technology  innovation  wherever  it  is  found  and  whoever  provides  it.  Technology,  concept  and 
content  skills  may  be  found  everywhere  in  a school  community.  Integrate  the  skills  and  efforts  of  those  innovators 
into  DL  program  development. 


Conclusion 
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Finding  keys  to  unlock  a school’s  culture  is  a difficult  challenge,  particularly  when  changes  are  occurring  quickly, 
under  the  impetus  of  external  pressure.  College  teachers,  staff  and  administrators  who  believe  in  DL  are  finding  that 
they  have  to  quickly  develop  and  apply  new  skills  outside  the  envelope  of  their  normal  job  descriptions.  Any 
expertise  they  develop  has  to  quickly  be  shared  with  colleagues,  in  order  to  make  cultural  changes  happen.  But 
Distance  Learning,  whether  we  think  of  it  as  a new  paradigm  or  one  of  many  extensions  of  the  old,  is  clearly 
broadening  educational  horizons.  Advocates  for  Distance  learning  who  understand  its  potential  can  provide  the 
leadership  to  move  their  colleagues  and  schools  toward  successful  integration  of  this  exciting  new  educational 
opportunity. 
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Abstract:  The  present  study  describes  a prototypal  system  for  machine  tools  teaching  in  a 
Virtual  Reality  environment  integrated  with  hypermedia.  The  goal  is  to  lead  students  not 
only  to  understand  the  structure  and  functioning  of  the  lathe,  but  also  to  use  such  a machine. 
To  reach  this,  the  prototype  tries  to  foster  conceptual  changes  in  students’  mental  models 
and  to  increase  students’  control  over  the  learning  process.  A series  of  experimental  tests 
have  been  carried  out  to  assess  the  educational  validity  of  the  instructional  tool  which  has 
been  devised.  The  aim  of  the  first  phase  of  the  project  was  to  assess  whether  students  could 
use  easily  the  virtual  lathe  and  to  verify  what  they  learn  about  such  a machine.  The  purpose 
of  the  second  phase  was  to  focus  on  some  critical  aspects  of  the  learning  process  activated  by 
using  the  virtual  lathe  in  order  to  realize  which  are  the  instructional  procedures  which  give 
the  best  learning  outcomes. 


Current  Needs  in  Engineering  Instruction 

There  are  many  multimedia  teaching  aids  available  for  engineering  education.  However,  the  problems 
currently  encountered  in  the  use  of  such  instructional  tools  account  for  the  computer  packages’  reduced 
interactivity,  for  the  impossibility  of  on-line  assessment  of  students’  competence  levels  and  of  relevant 
feedback,  for  the  difficulty  to  integrate  the  presentation  of  notions  and  concepts  with  the  active  exploration  that 
software  allows,  and  so  on.  Furthermore,  educational  packages  often  lack  of  user  friendliness,  are  not  always 
adapted  to  the  needs  of  either  teachers  or  learners,  the  networking  is  rarely  sufficient,  and  traditional 
pedagogical  methods  have  certain  difficulties  in  integrating  the  contributions  of  didactic  multimedia.  In 
addition,  the  products  designed  and  distributed  are  seldom  modular,  open-ended,  and  sufficiently  flexible  to  fill 
the  different  goals  of  the  users.  Finally,  there  have  been  few  attempts  to  validate  experimentally  the  efficacy  of 
computer-assisted  courses. 

To  devise  better  products  for  teaching  and  learning  scientific  and  technological  topics,  collaboration 
among  Information  Technology  engineers,  didactics  experts,  cognitive  psychologists,  and  software  developers 
is  needed.  In  this  way  users  will  have  validated  products  available  along  with  technical  and  pedagogical 
assistance  from  specialised  institutions. 

In  order  to  reach  this  goal,  accurate  researches  are  requested,  particularly  in  three  fields: 

- in  the  cognitive  sciences  field,  to  the  extent  that  it  is  in  particular  a question  of  increasing  the  pertinence  of 
the  trainee  model  by  observing  and  reconstructing  students’  spontaneous  learning  process; 
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- in  the  information  science  field,  in  order  to  improve  traceability  and  guidance  of  individual  activities  or  joint- 
solving training  pursued  by  the  student; 

- in  the  expert  systems  and  artificial  intelligence  field,  to  allow  the  creation  of  innovative  telematic  procedures 
which  can  amplify  the  range  of  their  use. 

The  results  of  this  research  will  favour  the  definition  of  a new  pedagogical  approach  which  will  be 
more  individualized  and  interactive  both  for  students  and  for  teachers.  More  precisely,  at  the  service  of  didactic 
innovation,  the  instructional  tools  to  be  devised  should  allow  personalised  training  and  the  learner's  activity 
should  be  privileged  by  favouring  an  immediate  evaluation  of  his  or  her  actions,  among  other  things,  as  well  as 
the  apprehension  of  all  the  information  available  in  the  learning  environment.  In  other  words,  the  development 
of  didactic  frameworks  corresponding  to  the  students'  customised  requirements,  either  in  person  or  remotely,  in 
different  teaching  contexts  are  needed.  Moreover,  a methodology  for  these  innovative  practices  based  on 
interaction,  tracing  and  intelligent  guidance  to  improve  every  student's  performance  should  be  shared.  For 
example,  new  teacher- training  methods  and  the  elaboration  of  new  learning  situations  which  call  on  games, 
exercises,  interactivity  and  problem  solving  should  be  developed.  Further  innovations  targeted  are  the 
adaptability  of  the  media  to  different  contents,  cultural  environments  and  cognitive  profiles. 

The  purpose  of  the  project  described  in  this  paper  is  to  present  an  instructional  tool  to  be  used  in  engineering 
education  which  integrates  hypermedia  and  virtual  reality  and  to  discuss  the  results  of  some  experimental 
investigations  aimed  at  evaluating  its  pertinence  and  at  assessing  which  are  the  best  procedures  to  train 
students.  The  long-term  goal  of  the  project  is  to  produce,  thanks  to  the  progress  achieved  both  in  software  tools 
and  in  learning  devices,  an  high-quality  course  to  disseminate  innovative  methods  for  engineering  teaching 
accessible  to  the  greatest  number  and  at  reduced  cost. 


The  Virtual  Factory 

The  possibility  to  use  a Virtual  Reality  system  for  teaching  and/or  training  surely  offers  a series  of 
advantages:  learning  efficacy,  safety  in  the  interaction  (because  the  knowledge  objects  are  virtual)  and 
cheapness  of  didactic  mission  (because  the  computer  system  replaces  the  didactic  equipment  and  the 
machinery,  generally  existing  in  the  technical/scientific  laboratories).  The  efficacy  of  the  didactic  mission 
increases  because  with  Virtual  Reality  it  is  possible  to  integrate,  but  not  to  replace,  the  traditional  knowledge 
processes,  based  on  a symbolic-reconstructive  approach  (for  example,  using  books,  language  or  logic 
connections),  with  the  natural  knowledge  process  based  on  a perceptive-motor  approach.  Finally,  Virtual 
Reality  in  the  field  of  engineering  is  thought  to  make  easy  learning  and  training  in  using  machine  tools;  it 
utilizes  different  teaching  techniques  thanks  to  different  didactic  paths  depending  on  the  kind  of  user  (student, 
worker)  and  on  his/her  cultural  and  vocational  level.  The  representation  techniques  of  Virtual  Reality  allow  to 
model  all  elements,  that  are  the  objects  of  lessons,  at  different  complexity  stages,  to  foster  a progressive 
learning  of  the  subjects. 

On  these  grounds,  the  project  aims  to  realize  a prototypal  system  to  teach  the  utilization  of  machine  tools 
through  a desktop  Virtual  Reality-hypermedia  integrated  environment.  The  prototype  regards  a manual  centre 
lathe  [Boer  et  al.  1997],  It  will  allow,  on  one  hand,  to  implement  in  a virtual  environment  typical  exercises  of  a 
manufacturing  laboratory,  eventually  performable  with  the  aid  of  the  hypermedia  environment,  and,  on  the 
other  hand,  to  set  a graphic  computer  network  that  is  able  to  share  interactively  the  same  synthetic 
environment.  The  virtual  lathe  is 

addressed  to  students  that  attend  the  last  three  years  of  technical-industrial  schools  and  engineering  faculty 
courses.  This  application  could  easily  be  introduced  in  the  industry  (maintenance  courses,  NC-machine  tools 
programming,  products  marketing). 

The  prototypal  system  consists  of  a traditional  centre  lathe  (fundamental  machine  tool,  both  historically  and 
industrially,  mainly  dedicated  to  the  machining  of  cylindrical  workpieces),  in  the  virtual  environment,  and  of 
the  master  environment,  which  contains  technical  information  and  didactic  instructions.  The  two  environments 
have  been  integrated  to  work  together. 

The  project  is  based  upon  a logical  structure  called  A,I,A,D,I,  (Italian  acronym  of  “Direct  and  Indirect 
Learning  Integrated  Environment”).  It  consists  of  two  main  environments  (Virtual  Reality  and  hypermedia), 
plus  a third  environment  which  has  the  function  of  introducing  the  student  into  the  system  and  of  allowing  to 
choose  among  some  proposed  alternatives,  regarding  the  type  of  exercise  and  the  learning  modality. 
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The  lathe  consists  of  a set  of  components  (structures,  piece  supports,  parts  for  the  information  transfer  and 
control)  that  receive  power,  data,  and  materials  with  the  goal  to  modify  their  physical  characteristic  by  a 
sequence  of  operations.  From  the  operative  point  of  view,  the  machine  tool  presents  some  mechanical 
components  that  are  in  relative  movement  and  therefore  it  is  necessary  to  define  their  own  kinematics 
characteristics  and  their  movement  synchronization  according  to  the  specific  operation.  A set  of  technological 
parameters  comes  into  play;  these,  if  rightly  fixed  and  linked,  allow  the  reaching  of  the  productive  goal. 

The  Virtual  Reality  environment  allows  to  learn  the  machine  tool  (declarative  knowledge)  and  to  learn  its 
use  (procedural  knowledge)  within  a realistic  situation,  performing  practical  operations,  both  physical  (i.e., 
piece  positioning  on  machine  tool  and  cutting  tool  movement)  and  conceptual  (i.e.,  choice  of  cutting  tool  from 
the  virtual  store,  choice  of  cutting  speed).  First  the  lathe  has  to  be  prepared  with  the  necessary  accessories;  then 
all  the  necessary  parameters  have  to  be  set;  finally,  the  user  can  perform  the  assigned  task  according  to  the 
drawing  previously  showed.  The  virtual  machine  gives  a feedback  at  each  step. 

The  design  of  the  hypermedia  environment  is  based  on  the  concept  of  “structured  approach”;  it  may  reduce 
the  risk  of  “getting  lost  in  hyperspace”,  that  is  not  to  know  how  to  go  back  to  the  starting  point  and  the  risk  of 
obtaining  a so  called  “spaghetti  document”,  that  is  a document  containing  a lot  of  pages,  each  one  with  a short 
piece  of  information,  which  doesn’t  let  the  user  learn  effectively  the  concept.  The  chosen  model  for  this 
application  is  called  HDM+  (Hypermedia  Design  Model).  The  hypermedia  environment  represents  the 
knowledge  store/manager  of  the  system  and  is  comparable  with  the  role  of  teacher.  It  lets  the  student  learn 
through  texts,  pictures,  movies,  graphical  simulations,  and  Virtual  Reality  or  real  situation  demos.  It  is 
subdivided  into  three  logical  parts:  glossary,  hypermedia,  and  simulation.  The  student  can  access  the 
hypermedia  environment  on  his  request;  the  structure  of  the  student-computer  interaction  is  open-ended.  Five 
different  kinds  of  lesson  are  implemented  in  the  system:  theoretical,  demonstration,  guided  practicing,  free 
practicing,  and  examination;  student  is  allowed  to  choose  the  kind  of  lesson  and  change  it  at  any  time. 
Theoretical  lesson  is  worked  out  only  in  the  hypermedia  environment;  the  system  explains  the  information 
related  to  the  chosen  item.  In  the  same  environment  the  demonstration  lesson  is  available;  it  consists  of  two 
demos  related  to  the  chosen  manufacturing  cycle.  In  the  guided  practicing  the  student  performs  the 
manufacturing  cycle  step  by  step  under  suggestions  and  control  of  the  system;  each  step  consists  of  three 
phases:  “what  to  do”,  “how  to  do”,  and  “to  do”.  In  the  free  practicing  the  student  performs  the  chosen  task  in 
the  Virtual  Reality  environment  and  he  or  she  can  take  information  from  the  hypermedia  environment,  and  the 
system  shows  the  results  at  the  end  of  job  or  stops  the  lesson  if  mistakes  occur.  Finally,  the  examination  lesson, 
is  similar  to  the  previous  kind  of  lesson  except  the  hypermedia  environment  is  inaccessible. 

The  prototypal  system  has  been  realized  on  a personal  computer  (Pentium  processor,  32MB  RAM)  using 
VRT  4-00  Superscape,  for  the  implementation  of  the  virtual  environment,  and  Multimedia  Toolbook  4.00  for 
the  realization  of  the  hypermedia  environment. 


An  Experimental  Study 


Aim 

A pilot  study  demonstrated  that  the  machine  prototype  enables  students  to  learn  what  a lathe  is  and  how  it 
works.  The  next  step  of  the  project  was  to  devise  the  best  procedure  to  train  students.  As  far  as  this  need  was 
concerned,  a critical  question  to  be  addressed  was:  is  it  better  that  students  begin  their  training  by  navigating 
the  virtual  lathe  or  by  facing  the  hypermedia?  This  was  crucial  in  order  to  identify  the  relevant  steps  of  the 
learning  process  in  which  students  have  to  be  engaged. 

To  answer  the  above  mentioned  question,  an  experiment  was  carried  out  by  analyzing  a specific  learning 
situation,  that  is,  the  set  up  of  the  lathe. 


Method 

Three  groups  of  novice  university  students  were  assigned  to  the  following  learning  conditions: 
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Virtual  Reality  - Lathe  = participants  navigated  freely  the  virtual  lathe  before  reading  hypertextual  information; 
Virtual  Reality  - Neutral  = participants  navigated  freely  a virtual  environment  (a  room)  with  no  connection 
with  the  lathe  before  reading  hypertextual  information; 

Hypermedia  = participants  read  the  hypertext  about  the  lathe  without  having  navigated  the  virtual  environment 
before. 

Indices  were  obtained  from  the  students: 

- by  observing  their  behavior 

- by  administering  questionnaires  about: 

. computer  using  skills 

. naive  representation  of  the  lathe 
. knowledge  presented  through  the  hypermedia 
. the  role  of  Virtual  Reality  in  education 

The  sequences  of  tasks  for  each  group  were  the  following: 

A)  Virtual  Reality  - Lathe  condition 

- computer  using  skills  questionnaire 

- questionnaire  about  the  representation  of  the  lathe  (1) 

- free  navigation  of  the  virtual  lathe 

- questionnaire  about  the  representation  of  the  lathe  (2) 

- experience  with  hypermedia 

- questionnaire  about  hypermedia  information 

- set  up  of  the  lathe  in  the  virtual  environment 

- questionnaire  about  the  representation  of  the  lathe  (3) 

- questionnaire  about  the  role  of  Virtual  Reality  in  education 

B)  Virtual  Reality  - Neutral  condition 

- computer  using  skills  questionnaire 

- questionnaire  about  the  representation  of  the  lathe  (1) 

- free  navigation  of  the  virtual  room 

- questionnaire  about  the  representation  of  the  lathe  (2) 

- experience  with  hypermedia 

- questionnaire  about  hypermedia  information 

- set  up  of  the  lathe  in  the  virtual  environment 

- questionnaire  about  the  representation  of  the  lathe  (3) 

- questionnaire  about  the  role  of  Virtual  Reality  in  education 

C)  Hypermedia  condition 

- computer  using  skills  questionnaire 

- questionnaire  about  the  representation  of  the  lathe  (1) 

- experience  with  hypermedia 

- questionnaire  about  hypermedia  information 

- set  up  of  the  lathe  in  the  virtual  environment 

- questionnaire  about  the  representation  of  the  lathe  (3) 

- questionnaire  about  the  role  of  Virtual  Reality  in  education 

Instruments  employed  in  the  investigation  can  be  shortly  described  as  follows. 

a)  The  computer  using  skills  questionnaire  asked  students  about  their  knowledge  and  previous  experience  with 
Information  Technology;  some  abilities  concerning  computer  use  were  assessed  through  self-evaluation. 

b)  The  questionnaire  about  the  representation  of  the  lathe  focused  on  some  critical  questions.  For  example, 
students  were  asked  to  describe  a lathe,  to  explain  how  it  works,  to  list  the  kinds  of  manufactured  good  for 
which  it  can  be  employed,  why  it  may  be  dangerous,  which  abilities  are  requested  to  use  it.  The  same 
questionnaire  was  administered  twice  (in  the  Hypermedia  condition)  or  three  times  (in  the  Virtual  Reality 
condition). 
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c)  The  questionnaire  about  hypermedia  information  consisted  of  open  questions  about  specific  aspects  of  the 
structure  and  functioning  of  the  lathe  which  were  explained  in  the  hypermedia  environment. 

d)  The  questionnaire  about  the  role  of  Virtual  Reality  in  education  reported  a list  of  24  statements,  such  as 
“Virtual  Reality  requires  to  plan  actions”,  “Virtual  reality  is  good  for  intuitive  people”,  “Virtual  Reality  yields 
incidental  learning”.  Students  had  to  rate  on  a 5-point  scale  their  agreement  with  each  statement. 

Operations  required  to  the  students  during  the  experiments  were  the  following. 

a)  Free  navigation  of  the  virtual  environment:  basic  instructions  for  navigation  were  provided  and 
exemplified;  then  students  were  trained  in  exploring  virtual  environment  by  allowing  them  to  navigate  ad 
libitum  the  Virtual  Reality  prototype. 

b)  Experience  with  hypermedia:  students  could  navigate  the  hypertext  about  the  lathe  in  order  to  achieve 
competence  to  carry  out  the  set  up  of  the  lathe. 

c)  Set  up  of  the  lathe  in  the  virtual  environment:  students  employed  the  virtual  machine  tool  to  set  it  up. 
Students’  behavior  was  observed  and  categorized. 


Results 

Responses  given  to  the  computer  using  skills  questionnaire  showed  that  familiarity  with  computer  and 
ability  levels  were  equally  distributed  between  the  three  groups  of  participants  and  that  the  subsequent  behavior 
during  the  experience  with  the  virtual  lathe  did  not  depended  on  the  variables  measured  by  this  questionnaire. 

Responses  given  to  the  questionnaire  about  the  representation  of  the  lathe  were  converted  into  scores;  the 
highest  was  the  score,  the  highest  was  the  quality  of  the  answer.  Data  indicated  that  at  the  beginning  of  the 
experiment  both  the  Virtual  Reality  and  the  Hypermedia  students  had  no  adequate  notions  about  what  a lathe  is 
and  how  it  works. 

After  the  free  navigation  of  the  virtual  lathe.  Virtual  Reality  - Lathe  students  learned  some  core  concepts 
about  the  lathe. 

Knowledge  presented  through  hypermedia  was  acquired  by  the  three  groups  in  the  same  manner;  in  fact,  the 
analysis  of  the  responses  given  to  the  questionnaire  about  hypermedia  information  revealed  no  differences 
between  the  three  groups  of  learners. 

No  differences  between  the  three  conditions  during  the  set  up  of  the  lathe  performed  in  the  virtual 
environment  emerged  also  by  the  observation  of  the  students’  behavior. 

At  the  end  of  experience  Virtual  Reality  - Lathe  students  showed  a higher  increase  in  the  adequacy  of  their 
conception  of  the  lathe  (measured  by  the  questionnaire  about  the  representation  of  the  lathe)  as  compared  to 
the  Virtual  Reality  - Neutral  and  Hypermedia  students 


Conclusions 

The  main  findings  of  the  experiment  can  be  summarized  as  follows: 

- the  virtual  lathe  prototype  can  be  used  to  teach  what  is  a lathe  and  how  such  a machine  works; 

- the  integration  of  Virtual  Reality  and  hypermedia  allows  novice  students  with  no  previous  training  in 
engineering  to  understand  the  essential  feature  of  the  lathe; 

- improvements  in  both  declarative  and  procedural  knowledge  emerged  after  a short  experience  with  the  virtual 
lathe  prototype; 

- the  initial  naive  representation  of  the  lathe  is  restructured  thanks  to  the  experience  in  the  virtual  environment 
and  a more  relevant  notion  is  acquired  after  the  free  navigation. 

- learning  is  enhanced  when  the  exploration  of  the  virtual  lathe  precedes  the  presentation  of  the  hypertextual 
information. 


Further  Developments 

On  the  ground  of  the  experimental  work  that  has  been  carried  out,  the  final  part  of  the  project  aims  to 
elaborate  and  ensure  the  distribution  of  a multimedia  and  virtual  reality  training  tool  validated  on  the  basis  of  a 
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full-scale  evaluation  of  the  cognitive  processes  involved.  This  product  focus  above  all  on  customized  teaching 
and  the  personal  attitude  for  active  self-training;  its  intention  is  to  establish  an  interactive  environment  for  each 
learner  which  favors  the  immediate  evaluation  of  his  or  her  activities,  as  well  as  providing  information 
accessible  locally  or  at  a distance  within  a given  context  by  multiplying  the  interpersonal  communications 
possibilities  thanks  to  the  electronic  network.  It  also  imply  that  teachers  must  receive  appropriate  training 
which  leads  them  to  the  most  effective  mastery  possible  of  flexible,  open-ended  software  packages  and  reinforce 
their  role  as  pedagogical  project  initiators  and  mediators,  indispensable  in  the  progress  of  the  training  process. 
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Abstract:  When  a Web-based  classroom  teacher  manages  a Web-based  distance  learning  system, 
there  can  be  a substantial  burden  of  preparation  work.  Hence,  an  instructor  needs  a Web  site 
manager  to  help  him/her  use  tools  to  manage  the  Web-based  classroom.  In  fact,  some  of  the  more 
tedious  work  can  be  avoided  by  using  proper  collaborative  learning  strategies.  To  use 
collaborative  learning  strategies  on  Web,  a Web-based  classroom  teacher  can  use  advanced 
technology  for  managing  learners’  portfolios,  controlling  learning  flow,  and  constructing 
collaborative  learning  activities;  and  he/she  can  also  refine  existing  collaborative  learning 
strategies,  which  reduce  the  requirements  of  the  advanced  technology.  Furthermore,  refining 
existing  collaborative  learning  strategies  can  make  it  easier  to  implement  them  in  a Web-based 
classroom.  Although  there  are  methodologies  for  constructing  a Web-based  collaborative  learning 
system,  none  specifically  address  the  possibility  of  refining  collaborative  learning  strategies  to 
work  with  Web  features.  The  paper  surveys  some  notable  collaborative  learning  strategies  as  well 
as  proposing  guidelines  and  recommendations  for  how  a Web-based  classroom  teacher  can  refine 
those  strategies  for  use  in  a Web-based  classroom  through  our  technical  perspective. 


Introduction 

“Issues  of  determining  which  strategies  and  how  they  are  used  in  classrooms,  as  well  as  matching  proven 
strategies  to  classroom  needs  and  measures  of  intervention  fidelity,  are  targeted  as  primary  areas  in  need  of 
research.”  (Gunter  & Denny,  1996)  Traditionally,  an  instructor  manages  a classroom  by  proper  strategies.  The 
properties  of  strategies  that  an  instructor  uses  to  manage  a classroom  include  group,  activity,  agenda,  role, 
responsibility,  interdependence,  etc.  Those  strategies  are  called  classroom  management  strategies.  Many  educators 
agree  that  using  collaborative  learning  strategies  can  reduce  an  instructor’s  efforts  in  classroom  management.  At 
the  same  time,  collaborative  learning  strategies  are  also  required  to  implement  a Web-based  distance  learning 
environment  to  promote  learning  motivation  and  learning  performance  (Leidner,  & Fuller,  1997).  Hence, 
guidelines  are  necessary  for  an  instructor’s  training  needs  in  using  collaborative  learning  strategies  on  the  Web 
(Sherry  & Morse,  1996).  However,  in  order  to  construct  a Web-based  environment  for  collaborative  learning,  an 
instructor  must  make  great  efforts  to  solve  technical  problems  of  using  Web  features  for  collaborative  requirements. 

To  construct  a Web  environment  for  collaborative  learning,  an  instructor  first  needs  to  explore  the  various 
possible  collaborative  learning  strategies  and  extract  their  common  characteristics.  Then,  an  instructor  may  want  to 
see  comparisons  between  the  characteristics  of  collaborative  learning  strategies  and  features  of  the  Web.  For 
instance,  an  instructor  may  want  to  know  the  answers  to  questions  such  as  ‘Are  existing  Web  technologies 
sufficient  for  developing  a virtual  space  for  collaborative  learning?’,  ‘What  are  the  efforts  required  for  an  instructor 
to  manage  the  Web-based  classroom  after  constructing  it?’,  ‘Which  kind  of  collaborative  learning  strategies  are 
most  effective  on  the  Web?’,  ‘Does  using  collaborative  learning  strategies  in  a Web  classroom  work  as  well  as  in  a 
traditional  classroom?’,  and  etc..  However,  both  instructor  and  Web  site  manager  have  no  guidelines  about  the 
relations  between  the  technological  efforts  and  their  effectiveness  before  constructing  a Web-based  collaborative 
learning  environment.  Thus,  there  should  be  reports  to  depict  the  relationships  between  technological  efforts  and 
collaborative  learning  strategies  from  both  technical  and  pedagogical  perspectives.  Thereafter,  a distance  learning 
instructor  can  coordinate  with  a Web  site  manager  to  construct  an  effective  Web-based  classroom  with  less  effort. 

The  features  for  a collaborative  learning  strategy  include  heterogeneous  grouping,  interdependence, 
individual  accountability,  and  group  processing  (Warschauer,  1997).  An  instructor  may  ask  the  Web  site  manager 
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to  construct  all  of  these  features  on  the  Web-based  classroom.  For  instance,  an  instructor  may  want  to  use  Team 
Game  Tournament  (TGT)  collaborative  learning  strategies.  However,  a Web  server  is  inherently  passive,  without  a 
grouping  mechanism,  so  that  it  is  very  difficult  to  construct  collaborative  learning  activities.  This  problem  is  called 
the  learning  activity  construction  problem.  Furthermore,  a TGT  strategy  must  control  learning  flow  among 
learning  activities  for  its  regrouping  requirements.  That  problem  is  called  the  learning  flow  management  problem. 
Besides,  the  logs  of  a Web  server  can  not  provide  enough  information  for  an  instructor  of  the  Web-based  classroom 
to  make  evaluations.  That  is  called  the  portfolio  management  problem.  To  solve  those  problems  from  a purely 
technical  perspective,  our  previous  research  used  active  database,  workflow,  and  data  cube  technology  to  enhance 
the  capability  of  a Web  server  (Chang  & Chen,  1997;  Chang  et  al.,  1998). 

Frequently  asked  questions  about  our  previous  research  are  “Is  so  much  efforts  always  needed  for  a Web 
site  manager  to  satisfy  an  instructor’s  requirements?”,  or  “Does  an  Web  site  manager  need  to  construct  all  features 
of  existing  collaborative  learning  strategies  for  an  instructor  to  begin  practicing  collaborative  learning  activities  on 
the  Web?”.  In  fact,  the  Web  site  manager’s  burden  can  be  greatly  reduced  if  the  instructor  can  make  an  agreement 
about  learning  strategies  with  learners.  Hence,  the  Web  site  manager  can  temporarily  ignore  that  requirement  and 
put  more  effort  on  other  more  important  requirements.  To  reduce  the  Web  site  manger’s  efforts,  three  aspects  of  an 
instructor’s  requirements  for  using  collaborative  learning  strategies  on  Web  should  be  considered  for  refinements. 

First,  an  instructor  using  a Web-based  distance  learning  system  has  the  difficulty  of  constructing 
collaborative  learning  activities  because  the  Web  system  does  not  support  role  assigning,  and  learners’  behavior 
recording.  If  an  instructor  wants  to  heterogeneously  group  learners  in  a Web-based  classroom,  the  instructor  needs 
to  know  ‘how  was  a learner’s  learning  performance  previously?’;  ‘Which  kind  of  role  is  suitable  for  a learner?’; 
‘What  is  the  capability  of  a role,  called  individual  accountability,  for  a learner  in  a heterogeneous  group?’,  and  so 
on.  The  present  Web  system  can  neither  answer  those  questions  by  retrieving  information  from  the  large  amount  of 
logs,  nor  endow  learners  with  a role.  Hence,  a Web  site  manager  may  be  asked  to  develop  programs  for  retrieving 
meaning  from  the  logs  of  an  existing  Web-server.  These  issues  are  referred  as  the  heterogeneous  grouping 
requirements  for  constructing  activities.  However,  a teacher  will  have  difficulty  in  constructing  a collaborative 
learning  activity  because  the  Web  does  not  support  a collaborative  style  of  working.  The  teacher  must  arrange  the 
style  of  collaborative  work  and  define  what  a group  member  can  do.  Furthermore,  an  instructor  should  be  able  to 
reuse  previous  learning  activities  to  construct  other  learning  activities.  These  are  referred  to  specification  and 
regulation  problems  in  the  activity  construction. 

Second,  an  instructor  does  not  have  effective  tools  to  control  the  learning  flow  to  allow  learners’ 
interdependence.  Some  works  of  previous  studies  have  attempted  to  manage  the  learning  flow  of  a learner  on  the 
Web,  for  instance  the  Guided  Tour  for  Courseware  (Hauck,  1996)  and  Footsteps  for  a Quiz  Activity  (Nicol, 
Smeaton,  & Slater,  1995).  Those  works  devised  mechanisms  for  generating  dynamic  hyperlinks  or  views  of  a Web 
site  to  help  the  Web  site  manger  to  manage  learners’  learning  flow  for  collaborative  learning.  Thus,  the  Web  site 
manager  can  easily  generate  a Web  page  with  specific  learning  flows  instead  of  rewriting  a Web  page  while 
instructors  modify  the  collaborative  learning  strategies.  However,  that  kind  of  solution  requires  a workflow 
mechanism  and  Web  site  manager  who  is  skilled  at  transferring  the  learning  flow  of  a collaborative  learning 
strategy  into  the  workflow.  Furthermore,  certain  kind  of  learning  flows  are  difficult  to  transfer  into  the  workflow 
mechanism.  These  issues  are  called  interdependence  requirements  for  managing  learning  flow.  A teacher  must 
maintain  the  progress  of  an  activity  because  network  tools  can  not  automate  the  agenda  of  an  activity.  In  addition, 
progress  in  agenda  of  an  activity  may  depend  on  certain  tasks  that  learners  must  finish.  However,  an  instructor  can 
not  always  monitor  the  progression  of  a collaborative  learning  activity.  These  are  called  automating  schedule 
problems  of  the  learning  flow  issues. 

Third,  to  analyze  a learner’s  behaviors,  an  instructor  must  make  a great  effort  to  find  pedagogical 
meanings  in  the  large  volume  of  learners’  records.  Recording  and  diagnosing  learners’  behavior  is  tedious  work  for 
an  instructor  in  a distance  learning  environment;  especially  to  analyze  the  interaction  patterns  of  collaborative 
learning.  Although  learners’  behavior  records  may  have  been  properly  recorded  and  analyzed,  an  instructor  can  not 
easily  determine  the  group  processing  from  learners’  portfolios  to  refine  grouping  strategies  or  learning  flow  of  the 
collaborative  learning  strategy.  This  issue  may  be  called  the  group  processing  requirements  for  portfolio 
management.  Consequently,  there  should  be  a mechanism  for  completely  recording  group  processes  in  a 
collaborative  learning  activity.  The  mechanism  should  be  able  to  represent  all  the  used  data,  including  generated 
and  navigated.  We  call  this  issue  the  recording  and  analyzing  problem  of  the  portfolio  management  issues. 

In  other  words,  a Web  site  manager  or  a distance  learning  instructor  must  make  great  effort  to  construct 
and  manage  the  additional  components  of  a Web  server  due  to  the  properties  of  collaborative  learning  strategies. 
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To  reduce  management  efforts  for  collaborative  learning  on  Web,  this  paper  proposes  guidelines  for  refining 
collaborative  learning  strategies  to  ease  the  heterogeneous  grouping,  interdependence,  and  group  processing 
requirements.  Although  the  proposed  guidelines  are  not  generally  applicable  to  all  collaborative  learning  strategies, 
they  do  extend  our  previous  research.  The  point  here  is  that  the  Web  site  manager  does  not  have  to  remember,  or 
be  bothered  with  intricate,  yet  irrelevant,  information;  he  can  remain  focused  on  the  validation  task  at  hand.  Hence, 
the  Web-based  collaborative  learning  environment  will  be  more  easily  constructed  by  a Web  site  manager  and 
managed  by  a distance  learning  instructor. 

An  illustrative  example  and  guidelines 

It  is  assumed  that  an  instructor  would  like  to  construct  a collaborative  learning  strategy,  for  this  example 
TGT(Team  Game  Tournament),  on  Web.  The  three  steps  of  one  round  for  the  TGT  collaborative  learning  strategy 
are:  (1)  divide  learners  into  heterogeneous  groups  for  collaboration,  (2)  regroup  learners  into  tournament  for  the 
honor  of  their  own  groups,  (3)  return  to  their  own  group  for  reflection.  A learner  will  get  his/her  points  for  his/her 
group,  which  is  determined  in  step  one,  in  order  of  his/her  performance  in  a tournament  group.  In  next  round, 
learners  will  be  assigned  to  suitable  tournament  group  according  their  performance  in  last  round.  Hence,  learners 
will  help  each  other  in  the  first  step  to  get  points  for  their  own  group.  The  purpose  of  step  two  is  to  let  every  learner 
have  the  opportunity  to  get  points  for  their  own  group  because  step  two  assigns  learners  to  a groups  of  learners 
with  similar  levels.  The  last  step  enables  learners  to  reflect  on  their  behavior  by  returning  to  their  own  group  and 
discussing  their  performance  with  collaborative  partners. 

A Web  site  manager  should  first  construct  the  required  learning  activities  for  TGT,  that  is  discussion 
activity  and  tournament  activity.  For  instance,  the  discussion  activity  needs  heterogeneous  learners  to  help  each 
other  learning.  Initially,  the  distance  learning  instructor  should  divide  learners  into  heterogeneous  groups 
according  their  previous  learning  performance.  Thereafter,  the  Web  site  manager  should  develop  mechanism  for 
heterogeneous  grouping,  such  as  behavior  recording,  online  testing,  and  so  on.  Furthermore,  the  Web  site  manager 
has  to  endow  learners  with  suitable  capabilities  in  an  activity,  that  is  the  concept  of  role.  Then,  the  agenda  of  the 
collaborative  learning  activity  should  be  defined  and  obeyed.  A Web  site  manager  has  to  develop  or  use  additional 
tools  to  satisfy  the  requirements  for  constructing  activities  because  Web  features  do  not  innately  include  the 
functionality  for  constructing  activities.  In  summary,  a distance  learning  instructor  may  ask  the  Web  site  manager 
to  be  responsible  for  the  following  tasks:  (1)  Constructing  the  role  mechanism  in  a collaborative  learning  activity, 
(2)  Automating  and  coordinating  the  agenda  for  every  group  in  the  learning  activity,  (3)  Learners’  portfolio 
analysis.  All  the  above  tasks  are  very  complex  for  implementation  on  the  Web.  Hence,  it  may  not  be  best  to  directly 
transform  the  collaborative  learning  strategies  in  a classroom  to  a Web-based  learning  environment. 

The  following  sections  depict  the  guidelines  for  refining  collaborative  learning  strategies  to  reduce  the 
complexity  of  implementing  them  on  Web,  First,  the  role  mechanism  in  a collaborative  learning  activity  can  be 
replaced  by  embedding  autonomic  systems  in  the  collaborative  learning  strategies.  Then,  group  moderators  can 
help  coordinate  the  learning  flow  among  groups.  Finally,  embedding  a feedback  role  can  help  instructors  to 
determine  the  group  processes  in  collaborative  learning  activities. 

Autonomic  systems  instead  of  role  mechanism 

In  practice,  an  instructor  can  reduce  both  construction  and  management  efforts  by  embedding  autonomic 
systems  into  a collaborative  learning  strategy.  For  instance,  learners  can  divide  themselves  into  heterogeneous 
group  by  ranking  each  other.  An  instructor  can  ask  every  learner  to  give  a score  to  the  other  learners,  who  are  ever 
in  the  same  discussion  activity  or  tournament  activity,  as  the  source  for  ranking.  Instead  of  providing  a mechanism 
for  online  testing,  the  Web  site  manager  just  provides  the  functionality  for  learners  to  rank  the  learning 
performance  of  each  other.  Furthermore,  heterogeneous  grouping  requires  a learner’s  background  information, 
such  as  race,  first  language,  majority,  etc.  Most  of  the  information  can  not  be  easily  collected  by  distance  learning 
instructors,  but  learners  will  know  the  background  of  each  other  and  will  reveal  the  knowledge  through  the  ranking 
function. 

The  autonomic  system  should  consider  the  following  function: 
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♦ Ranking.  This  functionality  of  the  autonomic  system  can  be  used  for  heterogeneous  grouping  because  the 
ranking  results  will  categorize  learners.  For  instance,  an  instructor  may  ask  learners  to  rank  each  other 
from  degree  A to  D about  their  mastery  of  a specific  topic.  Then,  the  instructor  can  use  the  ranking 
results  to  group  learners.  An  instructor  needs  to  ask  learners  to  rank  only  their  partners  in  a group  or  a 
competing  group  because  a learner  can  not  know  the  learning  degrees  of  others  that  he/she  never 
contacted.  The  instructor  can  normalize  the  ranking  results  to  suit  the  required  number  of  members  in  a 
group  if  the  ranking  results  are  presented  in  a percentage  form. 

♦ Voting.  The  functionality  of  the  voting  mechanism  is  similar  with  the  ranking  functionality,  but  the 
purpose  of  voting  functionality  is  to  choose  the  most  suitable  learner  for  a specific  role  from  a group 
according  personal  characteristics.  Instead  of  assigning  roles  to  learners,  this  issue  lets  learners 
recommend  someone  for  a role.  Hence,  the  instructor  or  Web  site  manager  need  not  make  efforts  to 
arrange  every  learner  to  his/her  suitable  role.  For  instance,  the  moderator  is  one  of  the  roles  in  the 
discussion  activity  of  the  TGT  strategy.  Because  a moderator  should  be  responsible  for  maintaining  the 
agenda  in  time  and  coordinating  with  other  moderators,  learners  should  vote  for  a learner  with  proper 
characteristics  to  play  this  role. 

♦ Accountability.  This  functionality  means  that  a learner’s  role  determines  his/her  capabilities.  In  general, 
the  collaborative  learning  system  has  to  determine  what  a role  does  something  (Chang  & Chen  1997). 
The  instructor  can  design  daily  routines  for  each  role  to  maintain.  Consequently,  every  learner  should  be 
responsible  for  some  tasks  that  can  only  be  executed  by  the  capabilities  of  his/her  playing  role. 

Consequently,  both  a Web  site  manager’s  burden  and  an  instructor’s  Web-based  classroom  management  efforts  can 
be  reduced  by  the  aforementioned  autonomic  functionality. 

Moderator  coordination  supports  learning  flow  management 

Subsequently,  a Web  site  manager  should  determine  the  learning  flow  among  learning  activities  according 
to  the  learning  flow  of  TGT  strategy.  In  other  words,  the  Web  site  manager  should  develop  a mechanism  to 
coordinate  learners  of  an  activity  with  other  learners  of  the  same  activity.  For  instance,  the  group  that  first  finishes 
the  discussion  activity  has  to  wait  for  other  groups.  In  our  previous  research,  this  interdependent  relationship  is 
called  and-join  learning  flow,  which  means  that  the  subsequent  learning  activity  is  activated  only  after  all  the  prior 
learning  activities  are  finished.  And-join  is  one  of  the  learning  flow  types  that  the  instructor  or  Web  site  manager 
should  make  efforts  to  control.  Hence,  we  previously  presented  a model  for  the  instructor  to  depict  the  learning 
flow  in  the  requiring  collaborative  learning  strategy,  as  well  as  a methodology  for  the  Web  site  manager  to  manage 
the  learning  flow  based  on  workflow  technology  (Chang  & Chen,  1998). 

The  requirements  of  managing  learning  flows  for  collaborative  learning  includes  regrouping  learners, 
learning  activity  coordination,  and  maintaining  strategy  constraints.  After  a collaborative  learning  activity  is 
finished,  learners  of  the  previous  learning  activity  may  be  arranged  into  different  groups,  a process  called 
regrouping  learners.  For  instance,  learners  in  the  same  discussion  group  of  the  TGT  strategy  will  be  distributed  to 
different  tournament  groups.  Some  collaborative  learning  strategies  need  to  construct  the  interdependence 
relationships  among  learning  activities;  hence  learning  activity  coordination  is  necessary.  For  instance,  in  the  TGT 
strategy  the  tournament  activity  can  not  start  until  all  previous  discussion  activities  are  finished.  Sometimes  the 
collaborative  learning  strategy  should  be  modified  before  being  applied  to  some  situations,  which  is  called 
maintaining  strategy  constraints.  For  instance,  an  instructor  can  not  directly  use  the  TGT  strategy  on  thirteen 
learners  because  the  number  of  learners  can  not  be  exactly  grouped. 

In  addition  to  using  a technical  perspective,  a distance  learning  instructor  can  reduce  gaps  between  the 
requirements  of  managing  learning  flow  and  workflow  technology  by  adding  a coordinate  subsystem  into  his/her 
collaborative  learning  strategy.  Hence,  the  requirements  for  learning  flow  management  can  be  largely  reduced  by 
the  coordination  of  learners  play  a specific  role  in  every  group.  If  the  moderator  of  the  group  can  notify  other 
moderators  of  other  groups,  the  Web  site  manager  can  ignore  the  and-join  learning  flow  and  make  more  efforts  to 
manage  other  kind  of  learning  flow.  The  moderator  of  the  notified  group  must  agree  to  reply  to  the  notification 
after  finishing  the  current  learning  activity.  Similarly,  moderators  can  handle  the  or-join  learning  flow 
requirements,  which  means  that  the  subsequent  learning  activity  is  activated  by  coordinating  agenda  if  one  of  the 
previous  learning  activities  is  finished.  Consequently,  learners  are  interdependent  because  they  should  notify  or 
wait  for  the  notification  to  do  something  in  the  learning  flow. 


708 


Learners’  assessments  support  evaluating  group  processing 


Finally,  the  Web  site  manager  must  record  learners’  behavior  and  learning  performance  during  an  activity 
to  permit  the  heterogeneous  grouping  and  learning  flow  managing  requirements  in  next  stage.  Although  a Web 
server  can  provide  system  log  files,  such  as  learners’  access  records,  a distance  learning  instructor  requires  a report 
with  pedagogical  meaning.  Hence,  the  Web  site  manager  has  to  develop  or  use  tools  for  monitoring,  managing, 
and  mining  learners’  portfolios.  Furthermore,  the  Web  site  manager  should  provide  a friendly  interface  for  an 
instructor  to  understand  the  generated  reports.  The  purpose  of  the  friendly  interface,  called  multidimensional 
analysis  in  our  previous  works,  is  to  demonstrate  reports  in  any  style  asked  by  instructors.  However,  the  issues 
observed  by  an  instructor  are  dynamic  and  can  rarely  be  predicts.  Hence,  we  use  the  data  cube  technology  to 
support  the  Web  site  manager  providing  report  service  to  instructors, 

A distance  learning  instructor  can  help  to  reduce  the  Web  site  manager’s  portfolio  management  efforts  by 
refining  the  collaborative  learning  strategies.  In  our  opinion,  there  are  other  ways  to  support  an  instructor 
recording  learners’  behavior,  analyzing  behavior  records,  and  making  decisions.  These  additional  guidelines 
include  daybook,  moderators’  comments,  and  learners’  questionnaires. 

♦ A distance  learning  instructor  can  first  design  a daybook  according  the  pedagogical  issues  that  he/she 
wants  to  observe.  One  specific  role  of  any  learning  activity  should  take  the  accountability  to  write  down 
the  events  related  to  the  items,  which  is  defined  by  the  instructor,  in  the  daybook.  Hence,  a distance 
learning  instructor  can  review  the  key  processes  of  collaborative  learning  by  reviewing  the  daybook  that 
is  written  by  the  student  with  a specific  role.  Furthermore,  the  workload  of  the  Web  site  manager  is 
reduced  by  developing  tools  to  help  the  instructor  design  the  daybook  instead  of  developing  methodology 
to  retrieve  pedagogical  meaning  from  learners’  behavior  records.  Although  we  proposed  in  previous 
research  a methodology  to  solve  the  learners’  portfolios  management  issue  by  data  cube  and 
multidimensional  analysis  technology,  the  Web  site  manager  can  not  master  those  skills  over  the  short 
term. 

♦ Moderators’  comments  can  be  used  to  indicate  possible  directions  for  behavior  records  analysis  because  a 
moderator  will  realize  the  problems  in  his/her  group.  A distance  learning  instructor  can  easily  get  the 
moderators’  comments  either  by  periodic  email  or  a Web  page  for  feedback.  Hence,  a distance  learning 
instructor  can  avoid  using  the  cube  operations  for  multidimensional  analysis  but  get  the  same  requiring 
results. 

♦ The  results  of  learners’  questionnaires  can  be  used  to  verify  moderators’  comments.  Furthermore,  a 
distance  learning  instructor  needs  not  reconstruct  the  collaborative  learning  processes  before  making 
decisions  using  the  verification  from  learners’  questionnaires  results.  Consequently,  the  instructor  can 
free  the  Web  site  manager  from  the  tedious  tasks  of  computing  cube  visualization. 

To  sum  up,  an  instructor  may  expect  simply  transfer  existing  collaborative  learning  strategy  into  the  Web  format 
and  get  similar  effects  on  learners.  From  the  perspective  of  the  instructor,  four  features  should  be  implemented  on 
the  Web,  that  is  heterogeneous  grouping,  interdependence,  individual  accountability,  and  group  processing.  From 
the  perspective  of  the  Web-site  manager,  the  collaborative  learning  strategies  will  involve  technical  requirements 
including  activity,  learning  flow,  and  portfolio  management.  As  in  the  examples  mentioned  above,  an  instructor 
can  motivate  learners’  autonomic  behavior  by  refining  collaborative  learning  strategies  to  reduce  classroom 
management  efforts  and  technical  requirements.  Table  1 illustrates  the  relationships  among  learners’  types  of 
autonomic  behavior,  collaborative  learning  strategies,  and  ways  to  reduce  technical  requirements. 


Technical  requirements 

Characteristics  of  collaborative  learning  strategy 

Suggested  refinements 

Assign  group 

Heterogeneous  grouping 

Ranking 

Assign  role 

Voting 

Endow  capability 

Individual  accountability 

Accountability 

Agenda 

Interdependence 

Notifying  others 

Automate  agenda 

Notified  by  others 

Repository  for  behavior 

Designing  daybook  and 

records 

Group  processing 

Appending  daybook 

709 


Multidimensional  analysis 

Moderators’  comments 
and  learners’  questionnaires 

Table  1:  Relationships  between  technical  requirements  and  refinements  of  collaborative  learning  strategies. 

Conclusions 

This  paper  provides  guidelines  for  a Web  site  manager  to  coordinate  with  a distance  learning  instructor 
for  refining  collaborative  learning  strategy.  Consequently,  the  refined  collaborative  learning  strategy  will  reduce 
the  Web  site  manager’s  efforts  in  constructing  the  collaborative  environment  on  the  Web,  and  also  reduce  the 
distance  learning  instructor’s  efforts  in  managing  the  virtual  classroom  on  the  Web.  This  paper  also  briefly 
introduces  the  required  technology  and  the  method  of  constructing  a Web-based  collaborative  learning  system  from 
our  previous  research.  However,  that  purely  technical  solution  for  constructing  Web-based  collaborative  learning 
environment  would  be  a burden  for  the  Web  site  manager  due  to  its  tedious  technical  tasks.  Based  on  the 
characteristics  of  collaborative  learning  strategy,  we  present  guidelines  for  refining  collaborative  learning  strategies 
from  our  technical  perspective.  A collaborative  learning  example,  the  Team  Game  Tournament,  is  used  to  illustrate 
how  the  guidelines  reduce  the  work  needed  for  implementation.  Hence,  a distance  learning  instructor  can  use 
existing  collaborative  learning  strategies  and  shape  them  to  be  suitable  for  implementation  on  the  Web  with 
relatively  little  effort.  Most  important  of  all,  the  guidelines  indicate  the  directions  for  the  instructor  and  Web  site 
manager  to  achieve  a common  grand  point  of  constructing  a Web-based  collaborative  learning  environment. 
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Abstract:  This  paper  describes  a study  that  examined  how  an  Internet  telecommunications  tool,  EdPsy 
Web,  was  used  to  support  a pre-service  teacher  training  course  at  the  level  of  higher  education  has 
changed  the  quality  of  social  and  intellectual  interactions  that  were  different  from  what  has  happened  in  a 
traditional  classroom  setting.  Findings  of  the  study  showed  that  students  had  seen  the  value  of  and 
demonstrated  a positive  attitude  toward  network  learning  after  using  Edpsy  Web  to  create  discourses. 
When  assignments  were  delivered  in  an  electronic  form,  students  were  motivated  to  see  and  compare  what 
other  students  had  accomplished  and  thus  were  more  aware  of  what  they  had  achieved.  This  network 
learning  environment  provides  more  opportunity  for  collaborative  thinking  and  knowledge  sharing.  The 
analysis  of  electronic  discourse  explicitly  demonstrated  the  multiple  thread  nature  of  electronic  message 
interactions.  In  general,  this  network-enhanced  classroom  setting  offered  an  alternative  pattern  of 
interaction  which  differed  from  the  traditional  face-to-face  setting  in  some  ways,  but  not  all. 


Introduction 

There  has  been  dynamic  growth  in  the  utilization  of  Internet  telecommnications  in  the  area  of  higher 
education.  Researchers  have  indicated  that  the  utilization  of  telecommunications  technology  in  education  will 
have  significant  impacts  on  instructional  activities  (Roberts,  Blakeslee,  Brown,  & Lenk,  1990).  When  compared 
with  traditional  face-to-face  communication,  computer-mediated  communication  functions  differently.  This  new 
medium  has  its  own  rules  or  patterns  for  managing  temporal,  spatial,  and  social  aspects  of  interactions  (Beals, 
1992;  Harasim,  1990;  Kiesler,  Siegel,  & Mcauire,  1984).  It  allows  for  asynchronous  interactions. 
Asynchonicity  expands  user  control  over  the  time  of  interaction,  thereby  increasing  the  time  available  to  read  or 
re-read  a message  in  order  to  carefully  formulate  a proper  response,  the  result  being  a more  fruitful  and 
thoughtful  discussion  around  a number  of  interrelated  topics.  The  record  of  dialogue  interactions  can  be  a rich 
source  of  ideas  setting  the  stage  for  knowledge  building  activities.  Furthermore,  the  place-independent  feature  of 
the  telecommunications-based  learning  environment  has  the  capability  of  linking  persons  from  different 
locations  for  the  purpose  of  communication.  Compared  to  traditional  face-to-face  communication,  asynchronous 
network  communication  operates  in  the  absence  of  social  context  cues,  such  as  facial  expressions,  eye  contact, 
and  voice  intonation.  Participants  must  make  their  communication  clear  and  understandable.  Transcripts  of 
network  conversations  provide  an  explicit  record  of  interactions  for  users  to  interpret  and  trace  the  progress  of 
intellectual  interactions.  Finally,  the  telecommunications-based  learning  environment  tends  to  reduce 

differences  in  social  status  and  prestige,  thereby  providing  a more  egalitarian  context  for  social  interactions. 
This  may  lead  to  more  open  and  spontaneous  participantion.  Based  on  its  powerful  capabilities  of  information 
management  and  unique  communicative  style,  the  telecommunications-based  learning  environment  may  have 
different  impacts  on  student  learning  and  motivation. 

Facilitating  collaborative  interactions  through  computer  network 

Recent  studies  on  human  cognition  have  viewed  learners  as  agents  in  a distributed  cognitve  system  which 
includes  physical  objects,  symbolic  representations,  people  in  social  relationships,  and  features  in  physical 
situations,  all  of  which  play  a part  in  the  accomplishment  of  cognitive  activities  (Oshima,  Scardamalia,  & 
Breiter,  1996;  Pea,  1993).  This  view  of  human  cognition  implies  that  learning  no  longer  belongs  to  an  individual 
enterprise,  but  to  a joint  effort  made  by  learners  and  their  surrounding  environment.  Theoretically,  the 
classroom  can  be  regarded  as  a distributed  system  where  students,  teachers,  symbols  and  artifacts  interact.  The 
classroom  discouse  provides  a collective  zone  of  proximal  development  that  supports  student  learning  (Brown, 
Ash,  Rutherland,  Nakagawa,  Gordon,  & Campione,  1992). 

Researchers  have  recognized  that  tools  offered  by  computer-mediated  communication  (CMC)  not  only  can 
increase  the  learner’s  access  to  information  but  can  also  faciliate  knowledge  building  activities.  Studies  in  the 
Internet-infused  learning  environment  indicated  that  the  utilization  of  telecommunication  tools  in  teaching  may 
influence  student  learning  regarding  science  (Linn  & Songer,  1991;  Songer,  1996).  The  telecommunication- 


based  learning  environment  provides  a chance  for  students  to  get  access  to  real-time  resources  and  firsthand 
information.  Students  have  greater  opportunity  to  participate  in  dialogue  with  distributed  peers.  The  flux  of 
personal  experiences,  scientific  information  and  expert  knowledge  infused  into  the  learning  environment  has  the 
potential  to  help  students  develop  a rich  explanatory  base  for  learning  and  application  of  knowledge.  Studies 
have  shown  that  students  in  the  network  learning  environment  demonstrated  significantly  greater  understanding 
of  scientific  concepts  than  did  students  in  traditional  classroom  settings  (Songer,  1996),  greater  motivation  to 
interact  and  share  ideas  with  other  students  (Ruberg,  Moore,  & Taylor,  1996),  and  more  thoughtful  and  self- 
regulatory  learning  behaviors  (Scardamalia  & Bereiter,  1991). 

The  telecommunications  tool  has  demonstrated  potential  as  a medium  for  sustaining  and  expanding  zones 
of  proximal  development,  freeing  teachers  from  the  burden  of  being  the  sole  source  for  transmitting  knowledge 
and  allowing  the  learning  community  to  extend  beyond  the  boundary  of  classroom.  Studies  have  indicated  that 
telecommunications  techonology  can  faciliate  thinking  and  help  shape  thought  (Brown  et  al.,  1992;  Scardamalia 
& Breiter,  1991).  Through  the  computer-mediated  learning  environment,  participatory  activities  in  which 
students  could  practice  cognitive  strategies,  communicate  understanding,  clarify  their  positions  and  develop 
ideas  with  their  partners  are  more  likely  to  be  facilitated. 

Previous  research  made  a claim  that  the  telecommunications  technology  could  be  a powerful  tool  for 
collaborative  interactions  and  for  building  a community  of  learners,  however,  few  demonstrated  how  the 
mechanism  underlying  the  process  of  collaborative  interactions  could  provide  opportunities  to  facilitate  learning, 
and  how  participants  were  motivated  to  learn  in  the  computer  network  environment.  In  viewing  collaborative 
interactions  in  the  telecommunications-based  learning  environment  are  essential  for  students  to  develop  their 
understanding  and  experience,  this  research  attempted  to  explore  two  problems,  namely  (a)  did  a network- 
enhanced  classroon  setting  actually  motivate  students  to  interact  and  share  knowledge?  and  (b)  did  distinct 
learning  behaviors  exhibited  by  students  exist  in  a network-enhanced  classroon  setting?  Answers  to  these  two 
problems  could  help  us  not  only  understand  how  a telecommunications-based  learning  environment  facilitates 
group  interaction  and  collective  knowledge  building,  but  help  develop  an  effective  learning  environment  as  well. 

Method 

The  study  was  conducted  in  an  instructional  setting  where  college  students  interacted  and  discussed 
educational  psychology  problems  with  an  Internet  tool.  Three  key  evaluation  methodologies,  survey,  discouse 
analysis  and  message  flow  analysis,  were  used  to  investigate  social  and  intellectual  interactions  and  assess  the 
educational  value  of  this  network-enhanced  learning  environment. 

Participating  subjects 

The  subjects  were  58  students  registered  at  the  National  Central  University,  Taiwan.  They  were  from 
different  departments  and  participated  in  an  educational  psychology  course  designed  as  one  of  the  prerequisites 
for  training  pre-service  teachers.  There  were  22  males  and  36  females. 

The  network-enhanced  learning  environment 

An  Internet  telecommunications  tool  called  EdPsy  Web  was  used  in  this  study.  The  role  and  purpose  of 
this  Internet  tool  was  not  to  replace  but  to  enhance  and  transform  existing  instructional  activities  into  more 
powerful  ones.  There  were  ten  functions  provided  by  Edpsy  Web  (Figure  1). 

Participant  information.  Students  could  access  information  about  their  teacher  by  clicking  the  “Know 
your  Teacher”  button.  Information  regarding  the  teacher’s  educational  backgrounds,  research  activities,  and 
publications  were  avabilable  for  the  students’  reference.  The  tool  also  allowed  students  to  access  fellow 
students’  personal  information  through  the  “Know  your  Classmates”  button. 

Instructional  activities.  An  on-line  course  syllabus  was  available  for  students  to  consult  at  any  time 
through  the  “Course  Syllabus”  button.  The  system  also  provided  an  on-line  discussion  function  through  the 
“On-line  Problem  Solving”  button.  By  using  this  button,  students  could  post  ideas  and  peruse  other  students’ 
opinions  regarding  a particular  problem  or  issue.  All  written  assignments  could  be  delivered  to  and  evaluated 
by  the  teacher  electronically  through  the  “Assignments”  button.  Students  could  use  this  function  to  see  the 
teacher’s  evaluative  comments  as  well  as  other  students’  works.  The  “Exchange  Idea”  button  allowed  students 
to  exchange  ideas  regarding  the  course.  The  “Reference”  button  provided  lists  of  reference  materials  relevant  to 
the  course  content,  such  as  books,  journals,  and  brief  introductions  of  preeminent  educational  psychologists. 

Instructional  administration.  Activities  regarding  the  course  was  announced  and  accessed  through  the 
“Announcements”  button.  Students  could  check  their  grades  by  clicking  the  “Check  your  Performance”  button. 
Finally,  students  could  send  their  suggestions  regarding  the  course  through  the  “Your  suggestions”  button. 
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Figure  1:  The  Edpsy  Web  Internet  tool. 

Procedure 

In  order  to  have  this  functioning  network  community  implemented  successfully,  the  teacher  managed  a 
reliable  computer  network  for  students  access  and  designed  well-specified  tasks  to  be  accomplished  by  students 
(Riel  & Levin,  1985).  At  the  beginning  of  the  semester,  the  teacher  spent  one  week  teaching  students  to  learn  to 
use  Edpsy  Web.  Students  were  requested  to  enter  their  personal  information  into  the  system  and  all  written 
assignments  were  transmitted  to  the  teacher  electronically. 

After  the  mid-term  exam,  two  on-line  problem  solving  activities  were  arranged  and  each  one  lasted  two 
weeks.  Problems  were  posted  electronically.  Students  were  required  to  post  their  ideas  and  discuss  solutions  to 
the  problems  with  their  classmates  using  this  Internet  tool.  They  were  allowed  to  collect  information  from 
different  sources.  The  teacher  did  not  take  any  action  to  moderate  or  intervene  in  student  discussions.  The  two 
activities  were  held  in  two  different  formats  commonly  used  in  traditional  classroom  settings.  Firstly  students 
were  requested  to  pose  their  opinions  individually.  The  problem  contained  the  issue  of  whether  speed  reading 
can  really  assist  students  to  study  and  acquire  knowledge.  Secondly  students  were  requested  to  study  in  a 
group.  Six  groups  were  formed  and  each  was  required  to  figure  out  opinions  before  posting  it  on  EdPsy  Web. 
The  problem  contained  the  issue  of  whether  an  objective  evaluation  systems  could  be  developed  to  assess 
student  learning  in  an  ill- structured  domain,  such  as  history,  geography  and  civics. 

Data  source 

A post-course  questionnaire  was  developed  to  assess  student  attitudes  toward  this  learning  experience. 
The  questionnaire  contained  21  statements  with  a five-point  Likert  scale  in  which  1 indicated  strong 
disagreement,  3 neutral  and  5 strong  agreement.  The  questionnaire  also  included  an  open-ended  question  which 
would  solicit  the  students  specific  opinions  about  EdPsy  Web.  Fifteen  senior  students  who  needed  to  participate 
in  graduation  exams  dropped  the  second  on-line  problem  solving  activity  and  did  not  complete  the  questionnaire. 

All  student  on-line  behaviors  were  recorded  by  Edpsy  Web,  including  the  time  and  frequency  they  logged 
into  the  system.  Students’  written  assignments  and  their  ideas  during  the  two  on-line  problem  solving  activities 
were  all  collected  by  the  system. 

Analysis  and  results 

In  response  to  the  two  research  problems  addressed  above,  our  data  analysis  focused  on  four  questions 
related  to  emergent  learning  in  the  Edpsy  Web  learning  environment:  (a)  Did  students  hold  a positive  attitude 
toward  learning  with  Edpsy  Web?  (b)  Were  students  of  different  genders  motivated  differently  to  use  Edpsy 
Web  in  relation  to  their  prior  Internet  experience?  (c)  Did  students  demonstrate  particular  learning  behaviors 
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during  the  on-line  problem  solving  activities  with  Edpsy  Web?  (d)  Did  on-line  problem  solving  activities  impact 
their  understanding  specific  learning  issues  related  to  educational  psychology? 

Student  attitudes  and  the  motivational  differences 

Statistical  analyses  of  students’  responses  on  the  post-course  questionnaire  showed  that  mean  scores  of  21 
items  were  above  3,  indicating  that  in  many  circumstances  students  had  a positive  attitude  toward  learning  with 
Edpsy  Web.  An  2x2  ANOVA  analysis  of  student  responses  was  made  along  with  two  demographic  variables: 
gender  and  prior  Internet  experience  (Table  2).  Results  indicated  that  the  two  variables  significantly  determined 
the  students’  attitude  toward  Edpsy  Web.  Students  who  were  frequent  Internet  users  (at  least  3 hours  a week) 
tended  to  be  more  motivated  and  had  a more  positive  attitude  toward  learning  with  Edpsy  Web  than  infrequent 
ones  (less  than  3 hours  a week),  F = 5.49,  p < .05.  Males  tended  to  have  a more  positive  attitude  than  females,  F 
= 8.24,  p < .05.  There  was  an  interaction  effect  between  the  gender  and  the  prior  Internet  experience  variable,  F 
= 7.60,  p < .05.  Females  who  were  frequent  users  tended  to  have  a more  positive  attitude  than  female  infrequent 
users.  No  significant  difference  in  attitude  between  male  frequent  and  infrequent  users. 


Gender 

Male  students 

Female  student 

Internet 

Experience 

More  than  3 hrs  per  week 

86.09’*'  (9.83) 

94.00  (3.61) 

Less  than  3 hrs  per  week 

86.63(11.72) 

80.69  (11.68) 

Note.  Mean  scores  with  standard  deviation  in  parenthesis. 


Table  2:  Student  attitudes  toward  learning  with  EdPsy  Web  as  a function  of  their  gender  and  Internet 
experience. 

Student  on-line  learning  behaviors 

Analysis  of  login  data  can  help  recognize  student  on-line  learning  behaviors.  The  58  students  had  logged 
into  Edpsy  Web  for  more  than  950  times  across  10  weeks  of  use.  Three  hundred  and  ninety-eight  logins  were 
clustered  during  the  two  on-line  problem  solving  activities.  Eighty-three  messages  were  posted  across  these  two 
on-line  activities,  of  which  80  individual  messages  were  for  the  first  activity  (average  1.38  per  person)  and  13 
group  messages  for  the  second  (average  2.17  per  group).  For  the  first  activity,  all  students  posted  messages. 

Impacts  of  on-line  problem  solving  activities 

In  order  to  explore  how  the  on-line  problem  solving  activities  impact  student  learning,  transcripts  of  all 
written  messages  were  analyzed  and  an  attempt  was  made  to  draw  up  a typology  of  electronic  messages  related 
to  the  educational  values  they  displayed.  The  analysis  of  transcripts  of  the  first  on-line  problem  solving  activity 
indicated  that  the  independence  and  initiative  of  the  students  in  searching  out  different  sources  of  information  to 
develop  their  ideas  during  the  on-line  discussion.  For  example,  one  male  student  wrote  a message  drawn  from 
his  personal  learning  experience  about  speed  reading; 

“...I  received  speed  reading  training  while  I was  preparing  the  national  extrance  exam  during  the  senior  high. 

I really  feel  that  speed  reading  training  helps  me  read  faster,  but  it  may  overlook  comprehension.  I also  feel 
that  I have  acquired  lots  of  knowledge  about  study  skills  from  the  training.  Although  speed  reading  skill  can 
be  acquired  within  a short  period  of  time,  but  you  still  need  to  keep  practicing  in  order  to  maintain  the  skill. 

In  my  experience,  a speed  reading  training  course  provides  different  reading  skills  such  as  skimming  by 
gaining  control  of  eye  movements,  reading  without  sub  vocalizing,  range  reading  technique,  meta  guiding  with 
photographic  memory,  mind  mapping,  etc...” 

Some  of  the  students  wrote  messages  to  express  their  ideas  based  on  their  knowledge  learned  from  the 
textbook.  Part  of  them  drew  information  from  what  they  had  learned  in  order  to  defend  why  speed  reading  can 
really  work  to  facilitate  quick  learning  of  text  material.  Part  of  them  drew  theories  to  deprecate  the  educational 
value  of  speed  reading.  For  example,  a student  drew  a memory  theory  explaining  why  speed  reading  can  not 
facilitate  deep  learning  and  processing  information: 

“...In  1972,  Craik  and  Lockhart  proposed  the  theory  of  levels  of  processing  in  that  information  can  be 
processed  in  human  memory  at  either  the  syntactic  level  or  the  semantic  level.  Processing  information  at  the 
syntactic  level  becomes  superficial  processing.  It  is  difficult  to  keep  information  that  is  processsed  at  the 
syntactic  level  in  the  memory  for  a longer  period  of  time  for  later  retrieval.  Speed  reading  training  only 
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teaches  students  how  to  draw  quick  attention  to  learning  materials,  but  does  not  allow  time  for  deep 
information  processing.  Therefore,  speed  reading  does  not  help  individuals  to  retain  information  in  the 
memory  for  a longer  period  of  time...” 

Several  students  got  information  from  materials  outside  the  course  or  from  the  Internet.  The  following 
excerpts  indicated  that  students  drew  information  from  the  Internet: 

“...  Many  people  believe  that  speed  reading  is  simply  skimming  or  scanning  materials,  and  that  reading 
material  slowly  is  the  best  way  to  subsume  printed  information.  Such  an  assumption  is  incorrect.  Skimming 
may  be  appropriate  for  reading  new  material.  Surprisingly,  the  more  quickily  you  read,  the  better  you  can 
memorize  things.  But  it  is  important  to  comprehend  what  you  read. 

One  efficient  way  to  read  material  is  mind-mapping,  a multi-dimentional  mnemonic,  in  that  you  make  use  of 
the  structure  of  paragraphs  and  their  placment  in  the  text  to  improve  your  reading  efficiency,  but  not  each 
word  literally.  For  example,  you  just  make  up  a memory  word  for  the  main  theme  of  each  paragraph  and 
relate  them.  Reading  paragraph  by  paragraph  easily  helps  you  understand.  It  is  not  helpful  in  increasing 
comprehension  when  you  Just  read  material  word  ‘ by  word...  Resources: 

http://www.study.com.tw/txt3.htm....” 

Analysis  of  transcripts  also  found  that  there  were  no  particular  persons  who  dominated  or  took  the  lead 
during  the  dicussion.  The  message  flow  analysis  indicated  the  number  of  messages  issued  and  the  number  of 
student  logins  per  day  were  significantly  correlated,  r = 81,  p<  .000.  Interestingly,  the  number  of  student  logins 
and  messages  issued  dramatically  jumped  each  Tuesday  when  the  course  was  held.  A similar  pattern  repeated 
in  the  second  week  of  on-line  activity.  Obviously,  the  ebb  and  flow  of  messages  corresponded  to  the  ebb  and 
flow  of  school  activity. 

Previous  research  on  on-line  discussion  indicated  at  least  three  types  of  roles  could  be  identified  when 
tracing  specific  themes  as  discussants  wove  their  way  through  messages:  idea  introducer,  idea  developer,  idea 
synthesizer  (Vallee,  Johansen,  Lipinski,  & Wilson,  1974).  Based  on  this  idea,  an  analysis  of  the  major  role  that 
each  message  played  was  made  for  the  first  on-line  activity.  As  Figure  2 shows,  at  the  initial  stage  of  activity, 
students’  messages  mainly  dealt  with  idea  introduction  and  development.  Some  of  the  students’  messages 
introduced  about  what  speed  reading  is.  Other  students  then  drew  certain  psychological  theories  from  textbooks 
to  support  or  deprecate  previously  initiated  ideas.  As  more  new  ideas  were  introduced,  more  students  began  to 
add  information  to  elaborate  upon  those  ideas  during  the  middle  stages  of  the  activity.  Almost  60%  of  the  major 
role  played  by  messages  as  an  idea  developer  from  day  4 to  day  12.  The  role  of  idea  synthesizer  became  more 
apparent  at  the  end  of  the  activity.  As  shown  in  Figure  2,  a map  was  plotted.  Messages  sharing  the  same  theme 
were  clustered  around  each  day  and  those  clusters  containing  messages  with  the  same  theme  were  linked  with 
lines  across  the  duration  of  problem  solving  activity.  A main  idea  could  be  elaborated  into  more  specific  ideas 
which  later  may  became  main  ideas.  The  map  demonstrated  the  “multiple  thread”  nature  of  electronic  message 
interactions  and  that  different  ideas  were  being  pursued  in  parallel  (Quinn,  Mehan,  Levin,  & Black,  1983). 


Duration  of  case  discussion 


Figure  2:  The  multi-thread  characteristics  of  electronic  messages  flow  in  the  first  problem  solving  task. 


Discussion 

Findings  of  the  study  indicated  that  students  had  a positive  attitude  toward  network  learning  after  they  had 
used  the  Internet  tool,  EdPsy  Web,  to  assist  learning,  social  interactions,  group  problem  solving,  assisgnment 
delivery,  and  knowledge  sharing.  Students  were  highly  motivated  to  have  a system  that  allowed  them  to  send 
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their  assignments  electronically,  to  read  their  teacher’s  evaluative  comments,  and  to  compare  other  students’ 
works  in  an  electronic  form.  In  most  traditional  classroom  settings,  students  usually  submit  assignments  to  their 
teachers  physically,  and  receive  feedbacks  from  them  without  having  opportunity  to  understand  what  they  had 
acomplished  in  comparison  with  other  students.  In  EdPsy  Web,  however,  students  had  opportunty  to  see  and 
compare  what  other  students  had  performed  so  that  they  could  be  more  sensitive  to  what  they  had  achieved. 
This  provided  opportunity  for  students  to  develop  a metacognitive  skill  to  self-monitor  and  self-evaluate  their 
learning  behaviors  which  may  lead  to  a more  effective  process  of  knowledge  acquisition. 

Results  of  this  study  support  the  hypothesis  that  two  variables,  gender  and  prior  Internet  experience,  are 
significantly  related  to  attitude  toward  this  Internet  tool.  Since  boys  are  exposed  to  computers  at  an  earlier  age 
and  more  offen  than  girls,  they  have  more  opportunity  to  use  the  Internet.  Therefore,  they  are  more  apt  to  take 
advantage  of  the  tool  to  learn.  With  less  time  exposure  to  the  Internet,  most  female  students  were  not  familiar 
with  the  functions  afforded  by  the  tool  so  they  did  not  fully  appreciate  it  nor  did  they  take  advantage  of  it.  This 
may  have  a bearing  on  male  and  female  students’  attitudes  toward  the  tool. 

Analysis  of  the  on-line  problem  solving  activity  indicated  that  students  could  fully  take  advantage  of  the 
telecommunications  tool  to  generate  and  express  their  ideas.  According  to  results  of  our  survey,  students 
preferred  the  on-line  problem  solving  activity  to  be  held  on  an  individual  basis  rather  than  in  a group.  This  may 
indicate  that  students  were  motivated  to  use  the  tool  to  express  their  own  thoughts  and  learned  with  it 
individually.  The  data  explicitly  showed  that  there  was  no  group  leader  to  dominate  and  take  the  lead  during  the 
on-line  learning  activity  and  all  students  posted  their  messages.  Evidently,  the  network  learning  environment 
provided  a more  egalitarian  climate  for  social  and  intellectual  interaction. 
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Abstract:  An  interdisciplinary  on-line  project,  connecting  two  high  schools  from  diverse 
school  districts,  positively  influenced  student  attitudes  and  resulted  in  a collaborative 
publication  on  the  World  Wide  Web.  Comparisons  of  the  frequency  distributions 
resulting  from  administration  of  an  Attitude  Survey  in  the  form  of  a Likert-type  scale 
revealed  increases  in  positive  student  attitudes  toward  English  and  science,  connecting 
the  disciplines,  technology,  and  collaborating  with  students  of  different  socioeconomic 
and  ethnic  groups.  Barriers  to  other  formal  statistical  analyses  of  the  data  included:  the 
unforeseen  changes  in  population  of  the  two  classes;  small  sample  size;  and 
insurmountable  technological  problems  in  one  of  the  schools.  Further  research  is 
recommended  to  examine  similar  interventions. 


Introduction 

Making  high  school  education  work  in  a changing  culture  necessitates  linking  learning  to  life. 
Because  of  the  rise  in  the  number  of  problems  associated  with  urban  high  schools,  such  as  gang  activity  and 
difficulties  in  dealing  with  increasingly  diverse  ethnic  and  linguistic  school  populations,  high  school 
teachers  have  become  aware  of  the  importance  of  making  school  relevant  to  students’  lives.  Further,  they 
are  starting  to  recognize  the  need  to  emphasize  an  interdisciplinary  curriculum  organized  around  questions 
students  consider  important,  rather  than  a strictly  subject-based  curriculum.  A number  of  innovations  in 
high  school  curricula  integrate  subject  areas  to  make  “...connections  between  bits  of  knowledge  and  skills 
from  different  realms,  organized  around  [such]  focusing  questions...”  (Clark  & Agne,  1997,  p.  xv)  as 

• How  does  a civilization  come  into  being? 

• What  does  it  take  to  solve  a major  environmental  problem? 

• What  do  I need  to  know  to  thrive  in  a technological  society?  (Clark  & Agne,  1997,  p.  xv) 

Students  must  link  facts  and  ideas  from  disciplines  in  order  to  answer  these  questions,  as  well  as  use 
higher-order  thinking  skills,  and  apply  abstract  ideas  to  real  problems  in  real  situations. 

In  attempting  to  explore  these  and  other  higher  order  questions,  the  infusion  of  computer 
technology  into  the  classroom  can  enhance  classroom  communications  and  resources.  Computer-mediated 
communication,  in  particular,  can  “...virtually  bring  the  world  into  the  classroom,  engage  students  in 
collaborative  processes,  enable  them  to  access  experts  worldwide,  and  to  make  each  student  an  active 
member  of  the  international  community.”  (Schrum  & Berenfeld,  1 997,  p.  28). 

The  social,  as  well  as  academic  benefits  of  using  computer-mediated  communication  are  abundant. 
Social  barriers  break  down,  as  students  from  different  communities,  cultures,  and  socioeconomic  classes 
collaborate  with  each  other  to  discover  answers  to  such  important  questions. 


The  Project 

A grant  from  the  Corporation  for  Public  Broadcasting  (1997)  to  encourage  the  infusion  of 
technology  into  the  curriculum  enabled  the  collaboration  among  the  School  of  Education  at  California  State 
University,  Dominguez  Hills,  and  two  high  schools  in  two  different  districts  during  the  1997-98  school 
year.  Students  from  the  tenth-grade  “Integrated  Science”  class  at  West  Torrance  High  School  in  the 
Torrance  Unified  School  District,  collaborated  with  eleventh-grade  English  students  at  Manual  Arts  High 
School  in  the  Los  Angeles  Unified  School  District,  on  an  interdisciplinary  problem-based  project  that 
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involved  research,  inter-school  communication  via  e-mail,  creative  production  and  publication  on  the 
World  Wide  Web,  as  well  as  social  action. 

The  goal  of  the  project  was  to  involve  students  and  teachers  in  interdisciplinary  teaming  and 
infusing  computer  technology  into  the  high  school  curriculum.  Technology  provided  the  students  with 
increased  motivation,  as  they  researched  and  wrote  for  a wide  audience  (on  the  Internet),  and  examined 
issues  of  scientific  and  social  import.  Teachers  used  constructivist  methods  that  empowered  their  students, 
and  provided  exemplary  models  to  be  used  with  university  credential  candidates  enrolled  in  Secondary 
Interdisciplinary  Methods  courses  and/or  Educational  Technology  courses. 

The  high  school  students,  from  two  very  socioeconomically  and  ethnically  different  schools, 
collaborated  with  one  another  on-line  to  write  and  do  research,  in  order  to  build  a home  page  that  provided 
a forum  for  information  and  creative  writing  about  scientific  issues  of  import  to  humankind. 

Manual  Arts  High  School  is  located  in  the  center  of  Los  Angeles,  commonly  referred  to  as  the 
“inner  city,”  and  West  Torrance  High  School  is  located  in  a suburb  adjacent  to  the  city  of  Los  Angeles. 
Both  are  considered  ethnically  diverse,  though  with  different  ethnic  populations;  Manual  Arts  High  School 
is  primarily  Latino  and  Black,  and  West  Torrance,  primarily  Asian  and  Caucasian.  There  is  a large  gap 
between  the  schools  with  respect  to  students’  socioeconomic  backgrounds. 

The  following  were  the  objectives  in  the  year-long  project: 

• to  increase  students’  awareness  and  understanding  of  the  world  around  them; 

• to  make  students  cognizant  of  the  issues  arising  from  the  environment  and  scientific  discoveries,  as 

well  as  their  ethical  and  moral  implications; 

• to  stimulate  students  to  explore  possible  solutions  to  moral  dilemmas  posed  by  the  issues; 

• to  encourage  students  to  use  concepts  inherent  in  real  issues  to  write  fiction  and  poetry; 

• to  improve  students’  research  and  writing  skills; 

• to  motivate  students  to  do  further  reading  of  science  fiction,  poetry,  and  material  concerning  the  issues; 

• to  promote  inter-school  cooperation; 

• to  bridge  the  gap  between  socioeconomically  and  ethnically  diverse  student  populations; 

• to  involve  students,  teachers,  and  pre-service  teachers  in  research  which  makes  use  of  computers,  CD- 
ROMs,  and  the  World  Wide  Web; 

• to  update  equipment  at  both  schools  to  complete  the  project; 

• To  stimulate  discussion,  further  reading  and  exploration  of  social  action  by  students  and  others 
accessing  the  resulting  Web  site;  and 

• To  create  a constructivist  learning  environment  that  supports  inquiry,  interdisciplinary  teaming,  and  the 
integration  of  technology  into  the  curriculum. 

As  a first  step,  both  classes  took  an  attitude  survey  (see  Tab.  1)  at  the  beginning  of  the  project  to 
determine  how  they  felt  about  several  things:  studying  English  and  science;  using  technology;  and  working 
with  students  who  were  different  from  themselves.  This  survey  was  to  be  administered  again  at  the  very 
end  of  the  project  to  determine  if  there  were  any  significant  changes. 

Next,  the  Integrated  Science  class  did  research  both  in  traditional  methods  and  on  the  World  Wide 
Web  to  write  scientific  briefs  on  issues  related  to  their  studies,  issues  that  have  moral  and  ethical 
implications,  that  involve  conflict,  and  are  of  serious  import  to  humankind.  Although  originally  conceived 
to  involve  a biology  class,  the  issues  in  integrated  science  broadened  the  scope  of  possibilities.  After 
brainstorming  during  the  first  few  weeks  of  school,  the  Science  students  came  up  with  the  following: 

1 . Space  exploration 

• Mir  Space  station  (out  of  control,  or  having  alien  contact) 

• Mars  Lander  (discovery  of  life  form) 

• People  living  in  space  (stations) 

• Asteroid  impact  Specific  environmental  topics 

• Effects  of  El  Nino 

• Global  warming 

• Nuclear  war  or  nuclear  accident  at  San  Onofre 

• Ozone  depletion 

• Effects  of  a “supernova”  on  Earth 

2.  Animals 

• Communication  with  dolphins 

• Communication  with  apes  and  chimpanzees 

• Effects  of  solar  flares  on  the  Earth 
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• Another  ice  age 
3.  Human  physiology 

• Ebola  virus 

• Curing  cancer 

• diet  medications 


Attitude  Survey 


Directions:  For  each  of  the  items  below,  write  the  number  that  best  describes  your  opinions.  There  are  no  right  or  wrong  answers;  we 
are  merely  interested  in  how  high  school  students  feel  about  these  issues. 

• Not  at  all  (1) 

• A little  (2) 

• Somewhat  (3) 

• Quite  a bit  (4) 

• Very  much  (5) 


Catci^oiy 

Item 

Tlie  Disciplines 

1. 

I think  English  and  science  are  two  subjects  that  relate  to  each  other. 

2. 

I think  English  is  an  important  subject. 

1 

3. 

I enjoy  English  class. 

4. 

I think  science  is  an  important  subject. 

A 

5. 

I enjoy  science  classes. 

ife'  ■ 

6. 

School  would  make  more  sense  if  teachers  from  different  subjects  would  work  together  to 

plan  projects  for  the  students. 

7. 

Teachers  from  different  subjects  should  work  together  to  plan  student  projects  so  that  the 

students  could  make  more  connections  between  the  different  topics  they  are  studying. 

pifr  T ' 

8. 

Research/library  skills  are  important  in  studying  science. 

9. 

Reading  is  important  in  studying  science. 

10. 

Writing  is  important  in  studying  science. 

11. 

Word  processing  is  an  important  skill  in  studying  science. 

/Techjnology 

5. 

I feel  comfortable  working  with  computers. 

6. 

7. 

I feel  comfortable  exploring  the  Intemet/World  Wide  Web. 
I feel  comfortable  doing  using  e-mail. 

8. 

I think  learning  to  use  technology  is  important. 

9. 

I think  technology  is  useful. 

10. 

I think  learning  how  to  communicate  via  e-mail  is  important. 

11. 

I think  learning  to  do  teleconferencing  is  important. 

12. 

I think  communicating  via  e-mail  is  a useful  skill. 

13. 

I think  teleconferencing  is  a useful  skill. 

14. 

I like  working  with  computers/technology  while  I learn  science  and/or  English. 

Work  Situations 

6. 

I feel  comfortable  working  alone  on  assignments  in  class. 

Ml 

7. 

I feel  comfortable  working  on  assignments  in  a small  group  in  class. 

8. 

I feel  comfortable  working  with  students  from  another  class  in  school. 

9. 

I feel  comfortable  working  with  students  from  another  school. 

10. 

I feel  comfortable  working  with  students  from  other  neighborhoods. 

11. 

I feel  comfortable  working  with  students  from  other  geographical  areas. 

1 

12. 

I feel  comfortable  working  with  students  from  other  races  and/or  cultures. 

Table  1:  Attitude  Survey 


The  science  students  then  sent,  via  e-mail,  the  “scientific  briefs”(see  Tab.  2),  which  were  uniform 
in  content,  to  the  students  in  the  “Contemporary  Composition”  class  at  Manual  Arts  High  School.  Again 
through  email  and  one  videoconference  (which  unfortunately  had  to  be  aborted  due  to  lack  of  appropriate 
wiring)  students  from  both  classes  attempted  to  explore  solutions  to  the  issues,  possible  avenues  for  social 
action,  and  “what  if*  scenarios.  The  students  in  the  English  class  then  took  the  issues  in  the  “scientific 
briefs,”  and  created  science  fiction  short  stories  based  upon  them.  In  preparation  for  the  creative  writing, 
English  students  did  their  own  research,  reading  works  of  science  fiction  authors  who  have  dealt  with  such 
issues. 

English  students  edited  the  articles  to  be  published  on  the  Web  site  for  mechanical  and 
organizational  content.  Similarly,  the  science  students  edited  the  English  students’  stories  for  scientific 
accuracy.  Both  classes  then  published  their  works  on  the  Internet,  on  a Web  site  provided  by  Manual  Arts 
High  School.  After  the  publication  of  the  first  issue,  the  intent  was  originally  to  solicit  contributions  from 
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students  at  other  schools  for  contributions  to  other  issues.  Unfortunately,  the  actual  amount  time  both 
classes  needed  to  complete  the  steps  in  the  project  exceeded  the  amount  estimated;  therefore  only  the  initial 
round  of  stories  was  published. 


Scientific  Brief 


Issue: Explain  why  this  topic  is  an  issue: 

Explain  the  controversy  over  this  issue: 

Background:  Give  a brief  history  of  this  issue  (When  and  where  did  it  become  an  issue?  Facts  leading  up  the  present  situation.) 

Current  status  of  issue:  What  has  been  accomplished?  Where  are  we  now  in  relationship  to  this  issue?  Has  any  progress  been  made? 
What? 

Possible  solution(s)  to  this  issue: 

Suggest  a “what  if’  for  a possible  story,  poem,  or  play  about  this  issue: 

Table  2:  Template  for  Scientific  Briefs 

The  project  Web  site  is  http://www.lausd.kl2.caMs/Manual_Arts_HS/CPB/Science_Fantasy.html. 
A Hyperlink  to  the  project  web  site  is  also  available  from  the  author’s  Home  Page  at  California  State 
University,  Dominguez  Hills,  http://www.csudh.edu/soe/faculty/nmach.htm. 

As  the  culminating  event,  in  late  May  there  was  a face-to-face  meeting  of  both  classes,  together 
with  their  teachers  and  the  university  sponsor  at  a luncheon  at  California  State  University,  Dominguez 
Hills.  A guest  speaker,  Mr.  Scott  Greenberg,  got  the  students  involved  in  an  interactive  writing  workshop. 
Mr.  Greenberg,  is  a humorist,  screenwriter,  speaker  and  leadership  consultant  who  has  inspired  young 
people  across  the  country  to  overcome  obstacles  and  achieve  their  dreams.  Students  met  at  the  University 
Student  Union,  where  they  mingled  over  lunch.  They  then  were  put  into  heterogeneously  composed  groups 
in  order  to  participate  in  the  writing  workshop. 

At  the  end  of  the  year,  both  classes  again  completed  the  attitude  survey  (see  Tab.  1)  to  see  if  there 
had  been  any  significant  change  in  any  of  the  variables  (see  Fig.  1). 


Findings 

In  addition  to  administering  the  Attitude  Survey  (Tab.l)  at  both  the  beginning  and  end  of  the 
school  year,  the  teachers  compared  student  work,  student  journals  and  other  classwork  from  fall  to  spring, 
for  improved  factual  content  and  writing  skills.  Both  high  school  teachers  also  informally  compared  the 
participating  classes  to  their  other  classes  with  regard  to  student  motivation  and  involvement,  on  such 
variables  as  attendance;  quality  of  work  submitted  (content/writing  skills);  and  participation  in  projects. 

Results  of  the  Attitude  Survey  are  indicated  in  the  charts  in  Figure  1,  below,  and  at 
http://www.csudh.edu/soe/faculty/NmachAACE/Jigure3.html.  Although  they  were  from  two  different 
schools  and  two  different  subjects,  the  two  classes  were  combined  due  to  the  small  number  of  students  in 
the  sample,  and  for  purposes  of  comparison  between  pre-  and  post-project  attitudes.  There  were  some 
students  who  took  only  the  pre-test,  but  for  one  reason  or  another  did  not  continue  to  participate  in  the 
project;  they  therefore  were  not  included  in  the  post-test.  In  toto,  39  students  completed  the  pre-test  and  33 
completed  the  post-test.  The  author  therefore  looked  at  frequency  distributions  in  the  form  of  percentages 
of  students  who  responded  with  specific  ratings  on  a one-to-five  point  Likert  scale  to  the  various  questions. 

Some  interesting  Pre-  and  Post-project  differences  emerged  when  comparing  the  mean 
percentages  of  students’  ratings  in  the  area  of  technology  in  general.  When  all  the  items  were  combined  in 
this  area,  numbers  of  students  using  a rating  of  “5”  rose  slightly  (from  44%  to  48%),  though  they  were 
initially  somewhat  high.  “Comfort  Level  with  Computers”  also  showed  a rise  in  number  of  students  giving 
ratings  of  “4”and  “5,”  with  a corresponding  decrease  in  those  with  ratings  of  “1,”  “2,”  and  “3.”  Ratings  of 
“4”  for  this  question  rose  from  41%  to  48%,  and  ratings  of  “5”  rose  from  18%  to  30%.  The  increase  is 
similar  when  looking  at  “Comfort  Level  Using  E-mail,”  with  those  rating  it  with  “5”  rising  from  33%  to 
45%.  In  other  areas,  numbers  of  students  selecting  “4”  and  “5”  combined,  for  attitudes  toward  the 
disciplines,  in  this  case  English  and  science,  as  well  as  interdisciplinary  work,  showed  a slight  increase, 
rising  from  57%  to  61%.  Similarly,  more  students  selected  these  same  ratings  for  attitudes  toward  working 
with  others,  particularly  those  from  other  ethnic  and  socioeconomic  groups,  with  those  selecting  “4”  or  “5” 
combined  rising  from  58%  to  64%. 
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Figure  3: Bar  graphs  Depicting  Pre-  vs.  Post-Project  Attitudes  - percentages  of  students  responding 
with  #1-5  on  Attitude  Survey  (in  color  at  http://www.csudh.edu/soe/faculty/NmachAACE/figure3.html) 
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Informal  feedback  from  students  was  highly  positive.  Students  came  away  from  the  face-to-face 
meeting  with  changed  opinions  of  the  ethnically  and  socioeconomically  different  groups  than  they  had 
originally  held,  and  increased  levels  of  self  esteem.  One  student  actually  commented  that  the  guest  speaker 
at  the  face-to-face  meeting  “changed  her  life.”  Both  teachers  in  the  two  high  schools  reported  higher  than 
normal  attendance  on  the  days  students  were  to  work  on  the  project,  particularly  on  days  when  e-mail  and 
an  attempted  video  conference  were  to  take  place.  Teachers  noticed  slight  increases  in  student  grades  when 
compared  with  their  other  classes,  but  also  a tremendous  increase  in  student  motivation,  as  evidenced  by 
student  attendance,  small  and  large  group  participation,  and  submission  of  classwork. 


Conclusions 

In  sum,  it  appears  that  in  this  project,  electronic  collaboration  positively  affected  student  attitudes 
toward  English  and  science,  connecting  the  disciplines,  technology,  and  collaborating  with  students  of 
different  socioeconomic  and  ethnic  groups.  The  increases  in  attitude  ratings  were  most  evident  at  the 
higher  end  for  most  categories,  strongly  suggesting  noticeable  differences  in  those  areas.  On-line 
communication  is,  after  all,  both  color-  gender-blind,  thus  capable  of  dispelling  stereotypes  about  students 
from  other  areas,  other  ethnic  and  socioeconomic  groups.  To  the  students,  the  meaningful  real-world 
connections  of  an  interdisciplinary  project,  as  well  as  the  knowledge  that  their  work  would  be  published  on 
a Web  site,  motivated  them  to  apply  themselves  to  the  study  of  both  English  and  science,  and  positively 
influenced  their  ability  and  comfort  level  in  using  technology. 

Unfortunately,  certain  problems  prevented  the  emergence  of  statistically  significant  findings,  most 
notably:  (1 ) the  size  differences  between  pre-and  post-test  populations  of  the  two  classes  prevented  stricter 
comparisons  using  raw  data  and  t-tests  to  prove  significance;  (2)  the  small  sample  precluded  making 
generalizations  from  the  data  collected;  and  (3)  one  school  was  not  fully  wired  for  Internet  access,  thus 
making  it  difficult  to  carry  out  the  teleconference  successfully. 

It  was  ironic  that  it  was  the  suburban  school  that  was  not  fully  wired,  while  the  "inner  city" 
school"  was  extremely  well  equipped.  This  is  the  reverse  of  what  would  be  expected,  since  the  inequities  in 
school  funding  usually  expected  in  the  two  types  of  schools  ("inner  city"  vs.  "suburban")  usually  have 
repercussions  in  their  technological  sophistication.  Our  "suburban"  school,  however,  may  not  be 
representative,  since  it  has  a large  working  class  population  that  has  been  rather  conservative  in  defeating 
school  bond  issues.  Similarly,  our  "inner  city"  school  may  not  typify  the  inner  city  schools  in  Los  Angeles, 
because  of  the  unusual  number  of  technology  grants  it  has  received. 

Further  research,  with  improved  experimental  design  in  such  areas  as  using  a larger  population  of 
students,  comparisons  with  control  groups  and  stricter  controls  on  administration  of  attitude  surveys,  would 
increase  the  general izability  of  results. 
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Abstract:  The  web-like  nature  of  multimedia  projects  differs  significantly  from  traditional 
linear  environments,  and  navigation  in  this  new  terrain  relies  heavily  on  abstract  thinking, 
involving  spatial  ability.  This  study  examined  the  use  of  a familiar  analogy  in  assisting  the 
conceptual  understanding  of  hypertext  and  multimedia  environments.  The  outcomes  of  this 
research  suggest  that  familiar  analogies  may  enhance  perceptual  understanding  in  computing 
novices,  and  encourage  these  students’  use  of  the  new  media  for  academic  writing. 

1.  Introduction 

Research  has  shown  that  the  use  of  word  processors  in  student  essay  writing  in  secondary  and  higher  education 
has  brought  minimal  benefit,  and  has  done  little  to  aid  the  planning  and  revision  of  texts.  The  reasons  lie 
chiefly  in  the  computing  technology  itself  When  composing  text  on  the  computer,  the  vision  of  the  writing  is 
limited  to  the  current  screen  view,  and  preceding  passages  are  obscured.  The  scrolling  back  and  forth  required 
to  locate  an  earlier  sentence  or  paragraph  creates  in  many  writers  a form  of  cognitive  motion  sickness.  More 
serious  still,  the  writer  has  little  chance  of  developing  a conceptual  map  of  the  whole  piece,  leading  to  what  has 
been  described  as  a 'text  sense'  problem;  that  is,  a lack  of  'knowing'  or  recalling  the  text  brought  about  by  the 
difficulty  of  gaining  a 'global  perspective'  of  a piece  of  writing  which  is  essential  for  its  structural  reform 
(Eklundh  1990). 

These  difficulties  have  caused  many  students  to  spurn  the  use  of  computers  as  writing  tools,  and  they  persist  in 
writing  their  first  and  subsequent  drafts  by  hand,  and  then  use  a word  processor  only  to  produce  a final  version 
of  their  writing.  Thus  the  revision  of  previous  drafts  remains  a surface,  cosmetic  process,  most  often  without 
structural  alterations,  despite  the  obvious  usefulness  of  the  word  processor's  'cut  and  paste'  facilities. 

Other  strands  of  writing  research  have  led  to  the  design  of  hypertext  systems  which  assist  the  cognitive 
mapping  of  writing  from  source  documents,  and  which  allow  the  planning  stage  of  writing  to  proceed  in  non- 
linear fashion.  However  useful  these  systems  may  be,  the  movement  from  linear  computer  environments  to  the 
web-like,  branching  structures  of  hypertext  and  multimedia  formats  requires  a conceptual  leap  in  cognition 
which  depends  to  a great  extent  on  spatial  ability.  This  facet  of  human  cognition  is  under-developed  in  many 
students  (females  in  particular)  due  to  differential  socialization  and  mathematical  experiences,  and  is  an 
anomaly  which  apparently  disappears  with  practice  (Hyde  et  al.  1990). 

Computing  educators,  who  understandably  are  caught  up  in  the  excitement  of  the  educative  potential  of 
multimedia  formats,  must  not  take  for  granted  a student's  easy  passage  from  the  old  and  familiar  linear  formats 
of  reading  and  writing  to  these  new,  non-linear  presentations  of  information.  This  paper  describes  how  the  use 
of  a familiar  analogy,  in  this  case,  the  format  of  information  presentation  in  popular  newsmagazines,  has 
assisted  students  in  achieving  the  conceptual  transition  from  the  linear  to  the  non-linear,  and,  at  the  same  time, 
has  both  improved  student  academic  writing,  and  revealed  to  the  students  the  narrative  potential  of  writing  in 
multimedia  formats. 


2.  The  use  of  analogy  in  the  conceptual  understanding  of  computer  systems 

Learning  with  computers  often  creates  problems  for  students  inexperienced  in  their  use,  and  anxiety  can 
develop  into  avoidance  (Harrington  et  al.  1990).  Familiar  analogies  have  been  used  by  computer  designers  to 
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reduce  anxiety  and  to  increase  the  ’user-friendliness’  of  their  product,  for  example,  Apple’s  (and  now  Windows’) 
’drag  and  drop’,  icon-based  filing  system  for  electronic  documents  uses  familiar  analogies,  in  which  the  user 
drags  a file  to  a familiar  folder  icon,  or  to  a rubbish  bin  icon,  an  arrangement  which  is  far  easier  to  understand 
and  use  than  the  former  PC  filing  method  of  creating  directories  and  sub-directories. 

Analogical  reasoning  has  a long  history  in  human  cognition  and  scientific  thought.  Centner  & Jeziorski  (1993) 
define  the  central  idea  of  analogy  as  the  mapping  of  knowledge  from  one  domain  (the  base)  into  another  (the 
target).  However,  in  processing  analogy,  people  implicitly  focus  on  certain  similarities  and  discard  others. 
These  writers  cite  a bright  student  reading  the  analogy,  ’a  cell  is  like  a factory’,  and  suggest  she  is  unlikely  to 
see  a cell  as  constructed  of  masonry  and  steel,  but  more  likely  will  consider  the  similarities  of  the  processes  of 
factories  and  cells,  that  is,  the  intake  of  resources  to  maintain  operations  and  generate  products. 

The  research  literature  supports  the  use  of  familiar  analogies  in  promoting  understanding  of  new  concepts 
especially  in  novices  (Hesse  & Klecha  1990).  Analogies  act  as  'advance  organizers’  which  allow  learners  to 
'map  the  new'  on  to  familiar  material,  although  there  is  a danger  that  the  particular  quality  of  an  analogy  may 
bias  how  new  material  is  understood,  and  the  transferability  of  learning  may  also  be  impeded  if  the  analogy  is 
insufficiently  transparent. 

Russon,  et  al.  (1994)  cite  a small  body  of  work  relating  to  the  use  of  analogies  in  computer  education  for  novice 
learners,  including  Beard,  et  al.  (1987)  who  found  that  paperwork  analogies  were  helpful  in  students’ 
understanding  of  simple  database  retrieval.  Russon  and  his  colleagues  (1994)  discovered  that  changing  one 
aspect  of  computer  exposure,  in  this  case  the  way  computer  material  is  taught  by  the  use  of  analogy,  had 
beneficial  effect  on  skill  acquisition  and  the  development  of  confidence  in  computer  novices.  These  various 
writers  identify  the  value  of  analogies  in  assisting  learners  to  link  new  with  familiar  material,  thereby  fostering 
both  a sense  of  relevance  and  reducing  anxiety. 

3.  Multimedia:  a conceptual  approach 

Douglas  (1993)  notes  that,  as  an  environment  for  reading  and  writing,  hypertext  represents  an  intriguing  and 
compelling  paradox,  for  while  its  content  looks  like  the  printed  word,  its  technological  apparatus  places  the 
printed  word  in  an  environment  almost  free  of  the  physical  constraints  and  characteristics  traditionally 
associated  with  printed  text.  The  problem  of  how  the  unfamiliar  structure  of  hypertext  might  best  be  conveyed 
to  students  as  they  explore  different  ways  of  writing  with  computers  suggests  the  need  for  a familiar  analogy  to 
aid  the  conceptual  understanding  of  hypertext  and  multimedia  formats. 

The  genre  of  newsmagazine  writing  presents  itself  as  a powerful  analogy  (and  conceptual  tool)  for 
understanding  the  complexities  of  these  new  writing  environments,  as  well  as  their  potential  to  revolutionise 
our  traditional  notions  of  text.  Newsmagazines  represent  a new  yet  familiar  form  of  electronic  writing. 
Formatted  with  page  layout  software  similar  to  that  used  by  student  writers,  their  structure  and  visual 
presentation  allows  easy  analysis  of  the  text.  That  these  magazines  are  composed  and  typeset  centrally,  and 
then  transmitted  electronically  to  regional  areas  of  the  world  for  the  addition  of  local  content,  is  compatible 
with  students’  understanding  of  online  communications  and  co-operative  writing.  However,  it  is  these 
magazines’  style  of  information  presentation  which  makes  them  an  even  more  effective  analogy  for  students  in 
their  understanding  of  new  ways  of  writing,  and  reading  electronically. 

The  newsmagazine  presents  its  information  in  fragmented  but  connected  pieces,  and  departs  from  a text-only 
approach  to  communicate  with  various  formats  of  text  and  graphics,  a visual  variety  which  has  been  created 
with  the  purpose  of  gaining  attention  and  facilitating  understanding.  Designed  to  be  read  in  trains  and  buses  as 
much  as  in  lounge  rooms,  the  newsmagazine  allows  readers  different  entry  and  exit  points  from  its  stories  by 
segmenting  them  into  main  text  and  commentaries.  This,  of  course,  is  the  nature  of  hypertext  and  multimedia 
presentations  of  information,  where  the  reader  has  a similar  choice. 

In  newsmagazines,  these  commentaries  which  exploit  the  facilities  of  page  layout  software,  exist  on  a different 
narrative  level  than  the  main  text,  thus  creating  an  illusion  of  web-like  connections  with  the  main  story.  As 
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such,  these  ’layered’  texts  represent  a powerful  analogy,  which  aids  students’  understanding  of  the  electronic 
construction  of  hypertext  and  multimedia  environments,  as  well  as  the  opportunities  they  present  for  the  reader 
to  depart  from  the  traditional  linear  path  of  reading.  These  facets  of  newsmagazines  were  used  as  conceptual 
tools  in  the  study  outlined  below. 


4.  The  Study 

This  paper  describes  a study  of  Arts  undergraduates  at  the  University  of  South  Australia,  who,  in  writing  with 
page  layout  software,  moved  beyond  mere  page  design  to  exploit  this  software’s  potential  for  displaying  the 
’layering’  of  information.  Using  a newsmagazine  style  of  writing,  typified,  for  example,  by  Time  and  Newsweek 
magazines,  the  students  took  a word  processed  story  and  transformed  it  into  a layered  text  which  simulated  a 
hypertext  or  multimedia  environment. 

The  news-magazine  genre  is  familar  to  most  students  who  are  avid  consumers  of  magazines.  Time  and 
Newsweek  magazines  are  known  to  students,  (if  not  as  well  as  the  more  ’pop  culture’  adolescent  magazines), 
and  are  readily  available.  At  the  start  of  the  study,  the  unique  features  of  these  magazines’  presentation  of 
information  were  analysed.  For  example,  in  a recent  story  on  heart  attacks,  the  Time  magazine  writers  departed 
from  their  main  theme  of  the  incidence  and  cost  to  society  of  the  prevalence  of  heart  disease,  to  create  various 
subthemes,  (or  layers),  to  the  main  story.  One  such  subtheme  was  a list  of  recovery  chances,  based  on  the 
closeness  to  medical  assistance  at  the  onset  of  the  attack.  Another  was  a commentary  on  risk  factors  in  the 
prevalence  of  heart  disease,  related  to  personality-typed  individuals,  while  another  significant  section  was  an 
interview  with  patients  who  had  made  an  excellent  recovery  after  suffering  a heart  attack. 

In  Time  magazine,  through  desktop  publishing  techniques,  the  themes  in  this  article  had  been  integrated  on  the 
same  or  adjoining  pages,  but  were  separated  by  textual  and  graphical  means,  such  as  the  main  columned  text’s 
weaving  its  way  through  'islands’  created  by  graphics  and  photographs,  and  by  the  separate,  coloured  and  boxed 
panels  of  text  which  contained  the  subthemes.  Here  was  choice  for  the  reader,  either  to  follow  the  main  text 
through  the  pages  to  its  conclusion,  or  to  digress  to  the  accompanying  subthemes,  and  later,  maybe,  to  once 
more  resume  the  main  story. 

So,  Time  magazine  presented  a practical  example,  if  in  a contradictory  and  linear  print  form,  of  an 
electronically  layered  hypertext.  That  is,  instead  of  the  subthemes  being  hidden  from  view  in  electronic  layers 
distant  from  the  main  text,  the  page  layout  of  these  articles  provided  a hypertextual,  visual  example  on  the  one 
plane.  In  this  way,  this  journalistic  model  created  a conceptual  analogy  of  hypertext  for  students  yet  to 
undertake  projects  in  multimedia  presentation. 

The  writing  task  undertaken  by  the  students  involved  transforming  into  a layered  text  the  article  written  by  van 
Gelder,  entitled  The  Strange  Case  of  the  Electronic  Lover,  which  is  reproduced  in  Dunlop  & Kling  (1991). 
Briefly,  the  story  involves  deception  on  the  Internet.  A male.  New  York,  psychiatrist  masquerades  as  a disabled 
female,  Joan,  and  develops  online,  electronic  relationships  with  a host  of  unsuspecting  women.  The  motives  of 
Alex  (not  his  real  name)  for  these  several  months  of  deception  are  uncertain.  Whether  Alex  saw  himself  as  an 
earlier  Tootsie  or  Mrs  Douhtjire  and  could  argue  some  valid  professional  reason  for  his  actions,  or  whether  he 
sought  only  personal  gain,  remains  a mystery.  Certainly,  Alex  (in  a complicated  sub-plot)  developed  a sexual 
relationship  with  one  of  his  new  female,  ’on-line’  friends,  and  eventually  was  unmasked.  An  interesting 
outcome  was  the  reaction  of  Alex’s  ’victims’.  Most  did  not  condemn  him,  but  spoke  only  of  the  pleasure  and 
benefit  of  their  electronic  interaction  with  Joan  (Alex’s  female  persona). 

First  the  students  analysed  the  story  of  the  electronic  lover,  and  its  many  facets  became  visible  as  layers,  or 
stories  within  stories.  As  they  set  about  planning  the  four  pages  allowed  for  their  review  essay,  there  was  much 
to  interest  both  male  and  female  students,  and  their  responses  to  Alex’s  deception  were  varied.  Some  were 
offended  by  the  deception,  while  others  were  more  tolerant,  and  these  divergent  opinions,  interestingly,  were 
represented  not  only  in  words  but  in  the  scanned  photographs  which  they  chose  to  represent  Alex. 


725 


Some  students  created  an  ethical  debate  as  the  framework  for  their  review,  some  highlighted  gender  issues, 
while  others  extended  their  review  to  the  wider  concerns  of  behaviour  in  global  communication  networks. 
Others,  still,  followed  the  journalistic  view  that  here  was  an  interesting  story  whose  nuances  could  best  be 
highlighted  by  the  interplay  of  banner  headlines,  subheads  and  pullquotes  with  the  text,  and  with  typography 
and  graphics  chosen  for  visual  impact,  in  page  layout  software  which  allowed  these  differing  story  lines  to  be 
given  individal  emphasis  in  ways  not  available  in  the  linear,  wordprocessed,  original  version  of  van  Gelder's 
article. 

So,  page  layout  software  allowed  the  composition  of  diverse  pages,  each  exploiting  text  and  graphics  to 
construct  'commentaries'  on  the  main  story.  Importantly,  the  students  created  their  own  fragmented,  but 
connected,  textual  format,  which  was  a hypertext,  but  in  a single  plane. 

There  was  a significant  increase  in  student  understanding  of  the  meanings  and  issues  contained  in  the  story  as 
they  wrote  with  software  which  allowed  the  visual  representation  of  its  various  facets,  and  the  structural 
planning  of  their  review  essay.  Before  writing  electronically,  the  students  created  pencilled  drafts  of  the  layout 
of  each  of  their  four  pages,  much  as  an  architect  would  make  preliminary  sketches  of  individual  living  or 
working  spaces  before  proceeding  to  the  formal  plans  of  a building.  Of  importance  here  was  the  visual 
advantage  of  seeing  discrete  but  connected  pages  of  text,  a view  which  is  denied  to  the  writer  when  working  in 
the  linear,  medium  of  wordprocessing.  Of  great  significance  was  the  students'  planning,  not  for  the  purpose  of 
page  layout  only,  but  as  an  integral  part  of  their  story-telling,  an  element  so  frequently  missing  from  the  work 
of  student  writers  who  work  with  word  processors. 

After  completing  their  desktop  publication,  the  students  were  asked  to  describe  how  the  story  might  have  been 
enhanced  further  in  a multimedia  format,  for  example,  with  the  use  of  audio/visual  interviews  with  Alex's  so 
called  'victims',  or  with  ethical  debates,  or  street  interviews  which  would  act  as  straw  polls  of  citizen  opinion. 
There  was  an  imaginative  response  from  these  students  which  demonstrated  not  only  their  acknowledgement  of 
the  potential  of  writing  in  a multimedia  format,  but  also  their  conceptual  understanding  of  how  this  could  be 
accomplished. 


5.  Conclusion 

Hypertext  and  multimedia  systems  present  advantages  and  disadvantages  for  readers  and  writers.  Theoretical 
arguments  insist  that  hypertext  is  a system  which  offers  an  absence  of  direction,  where  textual  meaning  can  be 
suspended  on  an  immediate  level,  thus  allowing  its  readers  to  compile  their  own  experience  of  the  text  as  a 
whole.  Such  is  the  case  with  the  fragmented  but  connected  texts  of  the  newsmagazine  writing  genre  which 
allow  discontinuous  reading  without  loss  of  meaning.  However,  the  problem  for  writers  who  use  hypertext  is 
the  need  for  a conceptual,  navigational  map  so  that  information  might  be  juxtaposed  without  loss  of  meaning  in 
an  environment  of  transparent  windows,  paths  and  links,  and  one  which  lacks  the  traditional  structures  of 
chapters  and  closure. 

The  students  in  this  study  demonstrated  the  usefulness  of  page  layout  software  in  providing  a pathway  to  the 
conceptual  understanding  of  writing  with  multimedia,  as  well  as  their  enthusiasm  for  writing  in  this  format. 
Reluctant  writers  seemed  better  able  to  cope  with  writing  smaller,  but  related  pieces  of  text,  and  enjoyed 
communicating  with  graphics  as  well  as  text. 

The  outcomes  of  the  study  suggest  the  beneficial  effect  of  a familiar  analogy  in  building  conceptual 
understanding  in  computing  novices  who  are  about  to  undertake  further  study  and  projects  in  the  non-linear 
environments  of  hypertext  and  multimedia.  Also  suggested  by  this  research  is  the  potential  of  multimedia  for 
writers  and  readers  to  interact  more  fully  with  the  layers  of  information  in  a textual  piece.  Certainly,  in  this 
study  there  was  a significant  increase  in  student  understanding  of  the  meanings  and  issues  contained  in  the 
story  as  they  used  software  which  allowed  the  visual  representation  of  these  layers  of  meaning.  As  well,  the 
planning  and  revision  processes  in  student  academic  writing  were  encouraged  by  the  rough  sketches  made  by 
the  students,  which  acted  as  conceptual  maps  as  well  as  the  structural  underpinning  of  their  writing.  * 
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Abstract:  This  paper  describes  the  development  of  a self-paced  multimedia  learning  resource 
designed  to  facilitate  the  transition  of  graduating  nurses  into  the  workplace.  The  resource  uses 
an  authentic  case  to  simulate  the  complexities  of  life  in  a typical  hospital  ward,  and  place 
graduating  nurses  in  the  role  of  problem-solvers.  Problem  solving  in  the  simulation  is  based  on 
a rich  repertoire  of  cases  and  stories  that  have  been  extracted  from  the  experiences  of  expert 
practitioners.  This  case-based  reasoning  architecture  reflects  a model  of  learning  where 
graduating  nurses  are  coached  in  the  development  of  decision-making  skills  within  the  context 
of  a contrived  but  an  authentic  presenting  problem.  Outcomes  of  formative  evaluation  carried 
out  in  a semi-structured  format  with  individuals  and  small  groups  of  nurse  practitioners  have 
revealed  a positive  disposition  towards  the  resource  and  the  approach  to  learning  it  utilises. 


Educational  Problem 

Current  practices  in  the  preparation  of  nursing  students  for  a successful  transition  into  the  workplace  is 
reported  as  being  ineffective.  Meanwhile,  the  Graduate  Nursing  and  Preceptor  Programs  offered  by  the  hospitals 
(at  least  in  the  Australian  context)  to  facilitate  the  transition  of  graduate  nurses  into  the  workplace  are  under 
tremendous  strain.  It  has  been  suggested  that  in  order  to  prepare  better  nursing  graduates  for  the  challenges  of 
the  workplace,  there  must  be  alternative  ways  of  developing  the  decision  making  skills  of  graduating  nurses  in 
situ.  This  implies  improved  collaboration  between  the  employing  organizations  and  Nursing  education 
institutions  to  develop  additional  strategies  to  formal  classroom  and  clinical  education  processes  that  could  be 
used  by  the  graduating  nurses  in  their  workplace  in  a self-paced  and  self-instructional  environment.  Moreover, 
this  alternative  solution  to  the  continuing  concern  for  the  successful  transition  of  graduating  nurses  into  the 
workplace  had  to  be  solidly  grounded  in  the  authentic  problems  and  situations  of  their  daily  routines.  The 
learning  tasks  had  to  be  immediately  relevant  and  meaningful  to  them,  not  contrived  or  removed  from  their 
workplace  environment.  The  learning  architecture  used  would  be  powerful  if  it  benefited  from  the  experiential 
knowledge  base  of  practitioners. 


Instructional  Solution 

In  attempting  to  address  this  need,  we  developed  a self-paced  and  self-instructional  multimedia 
learning  environment  using  the  case  based  reasoning  architecture.  The  material  presented  on  this  learning 
environment  attempts  to  simulate  the  complexities  of  life  in  a typical  hospital  ward,  and  in  so  doing  making  the 
education  of  graduate  nurses  and  students  case-based  and  authentic. 

Learning  activities  incorporated  in  the  courseware  require  users  to  make  decisions  about  the  best 
course  of  action,  and  source  of  information  regarding  each  case  or  problem.  Users  are  also  able  to  discuss  the 
cases  presented  to  them  in  the  multimedia  environment  and  reflect  on  how  they  might  have  addressed  the 
situation.  This  learning  architecture  reflects  a situated  cognitive  model  of  learning  where  students  and 
graduating  nurses  are  coached  in  the  development  of  their  strategies  for  recognizing  learning  opportunities  and 
critical  thinking  with  the  help  of  authentic  cases. 

We  have  not  recommended  throwing  out  all  the  other  resources  and  abandoning  students  in  a \velter  of 
diverse  information  sources  presented  in  this  multimedia  environment.  On  the  contrary,  we  suggest  leading 
students  very  carefully  through  unstructured  problem  situations  from  multiple  perspectives  and  sources  of 
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information,  providing  careful  instructional  feedback,  not  only  on  content  mastery  but  also  on  the  skills  of 
information-processing,  critical  thinking,  and  clinical  decision-making. 


Implications  for  Advances  in  Learning 

This  project  proceeded  from  the  realisation  that  current  practices  in  the  preparation  of  nursing  students’ 
successful  transition  to  the  workplace  upon  graduation  are  ineffective  and  deficient.  Moreover,  it  was  of  the 
view  that  more  of  the  same  kind  of  education  was  not  going  to  be  very  useful  and  argued  for  a radical  shift  in 
the  approach  to  this  component  of  nursing  education.  A shift  which  would  combine  powerful  educational 
technologies  and  proven  learning  strategies  to  build  a technology  enhanced  learning  environment.  This 
environment  is  innovative  in  two  ways.  First,  it  integrates  powerful  technology  with  case  based  reasoning  in  an 
integrated  learning  environment.  This  adds  to  the  currently  burgeoning  enthusiasm  in  the  use  of  interactive 
multimedia  applications  for  enhancing  learning  and  teaching  effectiveness.  Secondly,  it  comprises  a significant 
shift  away  from  current  practices  of  teaching  and  learning  towards  one  that  is  case-based.  In  this  environment 
student  assessment  is  situation-specific  and  as  such  authentic. 


Theoretical  Foundations 

There  is  considerable  support  for  the  belief  that  learning  and  teaching  is  most  efficient  and  effective 
when  it  is  situated  in  realistic  settings  where  learners  are  clear,  not  only  about  the  reasons  for  learning  but  the 
context  or  the  ecology  of  their  learning  environment.  This  view  of  learning  is  in  contrast  with  the  notion  that 
subject  matter  content  is  something  that  can  be  represented  in  schemas  stored  in  memory  and  retrieved  in  certain 
ways  when  needed.  The  foundations  of  these  beliefs  are  derived  from  cognitive  science  and  artificial 
intelligence,  and  trace  back  to  Gestalt  Psychology  and  the  works  of  Wertheimer,  Kohler,  and  Koffka,  who  were 
German  Psychologists  of  the  early  20*^  Century  (Bower  and  Hilgard,  1981).  Gestalt  psychologists  argued  in 
favour  of  the  role  of  insight,  perception  and  reflection  in  the  learning  process  as  opposed  to  association  based 
primarily  on  past  experience,  such  as  that  proposed  by  Thorndike,  Skinner  and  Pavlov  (Bower  and  Hilgard, 
1981).  This  philosophy  of  learning  and  teaching  translates  into  the  premise  that  to  educate  and  facilitate  learning 
we  must  create  situations  that  are  not  only  motivating  and  challenging,  but  that  necessitate  learning  of  facts, 
principles  and  procedures.  One  tried  and  tested  way  of  doing  this  is  through  a Goal-Based  Scenario  (GBS). 

A GBS  is  essentially  a contrived  situation  in  which  learners  assume  a main  role.  Their  "goal"  as  part  of 
this  role  is  to  accomplish  the  mission  or  task  associated  with  that  main  role  in  the  scenario.  In  order  to  achieve 
this  goal  the  learner  will  need  to  acquire  particular  skills  and  knowledge.  This  is  where  the  learning  is  taking 
place.  Goals  in  this  context  refer  to  the  successful  completion  of  the  task  at  hand  not  the  achievement  of  grades. 
A GBS  serves  to  both,  motivate  learners  and  also  give  them  the  opportunity  to  "learn  by  doing".  As  long  as  a 
goal  is  of  inherent  interest  to  learners,  and  the  skills  needed  to  accomplish  those  goals  are  the  targeted  learning 
outcomes,  we  have  a match  and  a workable  GBS.  The  important  idea  here  is  that  a GBS  is  organized  around 
"performance"  skills  and  the  end  result  is  a student  who  can  perform  the  specified  task. 

Every  aspect  of  human  behaviour  involves  the  pursuit  of  goals.  Sometimes  these  goals  are  simple,  like 
brushing  your  teeth  to  prevent  decay;  sometimes  they  are  quite  subconscious,  like  searching  for  similar 
experiences  when  you  encounter  a new  experience.  Sometimes  they  are  quite  complex,  like  trying  to  build  high 
quality  software  to  effect  change  in  the  education  system.  When  goals  are  simple,  we  really  don’t  think  about 
them  much.  When  they  are  subconscious  we  don’t  think  about  them  at  all.  And,  when  they  are  complex,  we 
may  think  about  them,  but  find  the  going  so  rough  that  we  hone  in  on  the  simplest  ones  and  lose  the  forest  for 
the  trees. 

But  understanding  how  people  pursue  goals  is  a critical  aspect  of  understanding  cognition.  For 
computers  to  really  understand  human  stories,  they  need  a complete  model  of  the  goals  that  people  pursue,  the 
plans,  the  use,  and  the  complexities  that  arise.  The  issue  is  this.  If  goals  underlie  human  behaviour  to  the  extent 
that  we  cannot  understand  a story  or  what  someone  says  or  what  someone  wants,  without  a clear  assessment  of 
the  underlying  goals  and  the  interaction  of  those  goals,  then  it  follows  that  goals  are  at  the  root  of  human 
learning.  Why  would  anyone  learn  anything  if  not  to  help  in  the  pursuit  of  a goal?  Why  would  anyone  try  to 
understand  anything  if  not  because  they  had  the  goal  of  learning  new  information  from  what  they  were  trying  to 
understand?  The  desire  to  change  one’s  knowledge  base,  to  comprehend  what  is  going  on  about  you,  and  to 
learn  from  experience,  are  all  pretty  much  different  ways  of  saying  the  same  thing.  And,  all  of  these  are  goal- 
directed.  If  goals  are  at  the  base  of  the  human  thought  process,  then  it  follows  that  learning  must  be  a goal- 
dominated  arena  as  well. 
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The  intent  of  a Goal  Based  Scenario  is  to  provide  motivation,  a sense  of  accomplishment,  a support 
system,  and  a focus  on  skills  rather  than  facts.  Facts  can  be  deceptive.  They  give  the  sense  of  knowing  without 
the  significance  of  knowing.  Understanding  why  you  are  doing  something,  having  a clear  goal  that  is  more  than 
the  recitation  of  facts,  truly  knowing  why  and  wanting  to  know  more  so  that  one  can  become  curious  about 
more  “whys”  is  what  learning  is  all  about.  Goal  Based  Scenarios,  interrupted  by  good  telling  of  important  cases, 
offer  a reasonable  framework  for  courses  that  are  meant  to  be  the  means  of  education. 


Learning  Architecture 

Perhaps  the  most  harmful  misconception  people  have  about  intelligence  is  that  being  smart  comes  from 
knowing  a lot  of  rules.  Behind  this  notion  is  the  sense  that  reading  a lot  of  textbooks  and  absorbing  what  they 
say  will  lead  one  to  become  an  expert.  While  it  does  make  sense  to  say  that  intelligence  comes  from  knowledge, 
most  of  that  knowledge  in  practice  looks  quite  a bit  different  than  what  you  find  in  a textbook.  The  architecture 
of  this  learning  environment  follows  from  the  premise  that  if  we  are  to  prepare  better  graduate  nurses  for  the 
challenges  of  the  contemporary  workplace,  we  must  shift  our  focus  from  a content-centred  to  a case-based 
reasoning  approach.  The  case-based  reasoning  approach  is  based  on  the  principles  of  a situated  cognitive  model 
of  learning  (Schank,  1997,  1990;  Schank  and  Cleary,  1995).  The  primary  propositions  of  the  situated  cognitive 
model  of  learning  are  outlined  by  Savery  and  Duffy  (1995)  as  follows: 

• understanding  is  gained  through  our  interactions  with  authentic  cases  and  in  situ; 

• cognitive  conflict  is  the  stimulus  for  learning,  and  also  determines  the  organization  and  nature  of  what 
is  learned. 

In  this  multimedia  learning  environment  case-based  reasoning  is  used  to  improve  current  instructional  practices 
in  the  education  of  nurses  for  their  transition  into  the  workplace  (see  Figure  1).  The  intent  of  this  model  is  to 
present  students  with  a contrived  but  an  authentic  scenario,  which  offers  them  an  opportunity  to  learn  in  a safe 
environment,  and  by  making  mistakes  without  injury  to  real  human  patients.  We  argue  that  mistakes  offer  real 
opportunities  for  learning  when  these  are  accompanied  by  timely  and  potent  feedback.  We  will  now  describe 
this  model  in  some  detail. 


I Clinical  Decision  Making  with  Case-Based  Reasoning 


• Learners  encounter  the  problem  situation  as  they  enter  the  learning  environment. 

• They  deal  with  the  problem  (in  a safe  environment)  with  help  from  experts  in  the  form  of  their 
experiences  and  stories,  and  also  documentation  and  other  resources  available  in  electronic  form. 

• Their  "goal"  in  this  simulation  is  to  develop  an  action  plan  for  managing  the  patient's  situation. 

• This  action  plan  is  considered  at  the  “case  conferences”  where  feedback  is  provided. 

Phase  I:  Case  Encounter 

• Learners  encounter  the  case  at  handover  where  they  are  explained  its  history  and  pathology. 

Phase  II:  Understanding  Problem 

Precipitating  event 

Identifying  its  causes 

Managing  the  situation 

Learner  encounters  the  precipitating 
event. 

Learner  seeks  to  locate  the 
cause  of  precipitating  event. 

Learner  attempts  to  manage  the 
situation. 

Phase  II:  Seeking  Solutions 

Becoming  aware 

Asking  questions 

Reasoning 

Learners  listen  to  the  stories  and 
experiences  of  expert  practitioners. 

They  ask  experts  questions 
about  their  work  experiences. 

Learners  attempt  to  reason  on  the 
basis  of  the  stories  of  experts. 

Phase  III:  At  the  Case  Conferences 

Raising  issues 

Asking  questions 

Reasoning 

Learners  explore  new  and  related 
issues  to  the  problem  by  reviewing 
sources  of  information. 

They  ask  experts  additional 
questions  about  their  work 
experiences. 

Learners  develop  their  final  action 
plan  based  on  the  stories  of  expert 
practitioners. 

Phase  IV:  Developing  an  Action  Plan 

• Learners  submit  their  action  plans  to  supervisors  and  receive  feedback  on  their  decision  making. 

Figure  1:  Clinical  decision  in  nursing  practice  with  case-based  reasoning.  The  phases  in  the  figure  guide  the 
procedure  for  attempting  to  use  the  repository  of  cases. 
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Case  Encounter 


As  users  enter  this  learning  environment  they  are  presented  with  a clinical  case  in  the  form  of  a 
scenario,  A guide  (which  is  always  available)  welcomes  users  and  informs  them  of  their  immediate  goal.  In  this 
scenario  learners  are  required  to  make  clinical  decisions  on  the  basis  of  information  that  is  available  to  them. 
This  information  is  presented  in  the  form  of  documentation  and  expert  knowledge  (which  are  encapsulated  as 
stories)  to  manage  a crisis  situation.  Users  begin  by  attending  a handover^  which  is  a regular  event  of  a nurse's 
daily  routine  where  relieving  nurses  are  updated  on  the  current  situation  of  their  patients  and  this  is  where  the 
user  encounters  the  case.  As  in  a real  hospital  setting  after  handover  the  learners  move  on  to  attend  to  routine 
nursing  care  activities  and  meeting  patients’  needs  by  administering  medications  and  ensuring  patients'  comfort. 
Soon  after  this  users  are  met  with  a "precipitating  event", 

A precipitating  event  in  this  instance  is  an  emergency  situation  that  causes,  or  has  the  potential  to  cause 
a chain  of  events.  It  requires  the  learner  to  make  complex  decisions  under  the  pressure  of  time.  Within  this 
learning  environment  though,  time  is  not  a variable  because  this  is  a contrived  learning  situation  in  which  users 
have  the  opportunity  to  review  relevant  documentation  and  seek  advice  from  experts  if  necessary,  on  the  best 
practice  before  making  decisions.  This  becomes  the  "goal"  or  mission  of  users,  which  is  not  to  be  confused  with 
a "learning  outcome".  A learning  outcome  is  a skill  that  learners  will  develop  (such  as  decision  making)  as  they 
seek  to  fulfil  their  goal  or  mission.  In  order  to  achieve  this  goal  or  mission,  the  first  thing  the  learner  must  do  is 
to  understand  the  situation  and  control  it  because  the  situation  has  the  potential  to  deteriorate,  and  then  develop 
an  appropriate  action  plan  to  manage  the  situation. 


Managing  the  Situation 

In  the  first  instance,  the  learner  must  do  everything  that  is  necessary  to  manage  the  crisis  situation 
before  recommending  an  action  plan.  In  order  to  do  this,  it  is  necessary  that  the  learner  understand  the  crisis, 
including  its  causes. 

In  this  scenario  the  learner  is  presented  with  a situation  in  which  Mr.  George  Parker  (the  patient)  is 
experiencing  an  anaphylactic  reaction  to  a drug  administered  and  to  which  he  has  been  previously  sensitised.  To 
diagnose  this  situation  accurately,  the  learner  needs  to  assess  the  patient's  condition.  In  order  to  arrive  at  a 
correct  diagnosis,  learners  can  access  a whole  range  of  information  including  documentation  on  hospital 
procedures/protocols  and  video-clips  of  interviews  with  expert  practitioners  (experienced  nurses)  on  appropriate 
procedures  to  follow  or  not  to  follow  under  such  circumstances. 

After  diagnosis  the  learner  must  take  appropriate  action  to  manage  the  crisis  situation  by  generating  an 
action  plan.  But  before  the  learner  develops  an  action  plan,  they  are  prompted  by  the  learning  system  during 
each  step  of  the  transaction  to  identify  and  sequence  (from  lists)  appropriate  actions  that  are  necessary  for 
correct  diagnosis  and  management  of  the  anaphylactic  reaction. 

For  example,  during  diagnosis  the  system  asks  the  learner,  'What  would  you  be  looking  for  to  confirm 
that  George  Parker  is  at  risk  of  an  anaphylactic  reaction?''  It  allows  the  learner  to  choose  from  a list  of  past 
events  or  patient  characteristics  the  appropriate  triggers  that  would  confirm  that  this  patient  is  indeed  at  risk  of 
an  anaphylactic  reaction  that  is  about  to  take  place.  This  action  plan  requires  the  learner  to  make  decisions  about 
prioritising  and  delivering  care  that  is  appropriate,  given  the  circumstances. 

Users  are  able  to  make  these  decisions  after  having  listened  to  the  experiences  of  expert  practitioners. 
These  experiences  (reported  as  vignettes)  are  presented  in  the  form  of  stories  and  made  available  to  users  as 
they  make  their  decisions.  This  is  where  the  learning  is  taking  place  for  the  learner. 

In  real  life,  nurses  do  not  have  the  time  in  a crisis  situation  to  consult  anything  because  of  the  pressure 
of  time  associated  with  the  situation.  In  this  context  however,  they  are  allowed  the  opportunity  to  stop  and 
reflect  upon  each  possible  action  they  can  take,  and  seek  advice  and  information,  review  the  underlying 
pathology  of  the  case,  and  policies  and  protocols  governing  such  situations  before  including  them  as  part  of 
their  action  plan. 


Reflecting  at  Case  Conference  I 

When  an  action  plan  has  been  developed,  users  proceed  to  a case  conference.  This  is  a place  where 
users  have  the  opportunity  to  reflect  upon  their  own  action  plans  and  that  of  others.  There  is  the  opportunity 
here  to  engage  in  collaborative  negotiation  of  meanings,  questioning,  critique,  and  commenting  on  alternative 
approaches  of  care  that  is  deemed  appropriate  to  the  case.  There  is  the  opportunity  here  for  invoking  cognitive 
conflict  in  the  learner,  which  has  the  potential  to  lead  to  changing  perceptions  that  result  in  learning. 
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Reflecting  at  Case  Conference  II 


The  second  case  conference  offers  an  opportunity  for  learners  to  obtain  feedback  on  their  revised 
action  plans.  This  has  the  potential  to  lead  to  further  cognitive  conflict  for  learners  and  lead  to  further  questions 
and  critique  that  can  result  in  learning. 


Developing  an  Action  Plan 

In  the  final  stages,  users  ought  to  be  in  a position  to  develop  an  action  plan  that  is  based  on  informed 
decision  making  and  one  that  is  realistic  and  acceptable  to  their  supervisors.  By  now  it  will  be  clear  to  users  that 
the  process  of  reaching  this  stage  is  more  important  than  having  the  action  plan  accepted.  The  architecture  of  the 
model  was  designed  to  develop  in  learners  clinical  decision  making  skills.  And  it  sought  to  achieve  that  by 
focusing  and  urging  users  to  learn  from  the  experiences  of  expert  practitioners.  Its  expressed  intent  was  to 
expose  nurses  to  the  process  of  clinical  decision  making  and  encourage  them  to  make  decisions  on  this  basis  so 
that  this  process  is  automated  for  them.  Ultimately,  within  this  case-based  reasoning  environment  and  through 
the  use  of  critical  thinking  users  should  hone  their  problem  solving  skills  and  ability,  in  making  appropriate 
clinical  judgments  concerning  patient  status. 


Development 

The  development  of  this  interactive  multimedia  courseware  product  adopted  a user-oriented  approach 
(Goodyear,  1995)  which  comprises  ongoing  testing  and  formative  evaluation  of  the  prototype  by  experts  in 
multimedia  courseware  development,  content  experts  and  a selected  sample  of  intended  users.  This  multimedia 
environment  is  being  developed  using  Asymetrix’s  Multimedia  Toolbook™,  a quasi-object-oriented,  event- 
driven  development  system  for  Microsoft  Windows™.  Toolbook^^  combines  database  functionality,  text 
manipulation,  hypertext,  drawing  capabilities  as  well  as  a full-featured  programming  language  called 
OpenScript™  which  allows  the  developer  to  program  object  behaviour.  Toolbook™  offers  all  the  advantages  of 
a prototyping  tool  and  is  best  suited  in  projects  where  user  initiated  changes  need  to  be  made  during  the 
development  process. 

This  prototype  is  currently  undergoing  a series  of  iterations  of  progressive  development.  The  highly 
modular  nature  of  object-oriented  programming  allows  the  developer  to  test  each  object  and  its  behavioural 
characteristics.  This  learning  system  contains  both  generic  content  about  the  diagnosis  and  management  of 
anaphylaxis  as  well  as  specific  procedures  and  protocols  that  are  specific  to  St  Vincent’s  Hospital  in 
Toowoomba,  Queensland,  Australia,  which  has  served  as  the  reference  site.  Furthermore,  in  order  to  achieve  the 
most  realistic  nursing  clinical  decision  making  environment  possible,  all  participants  in  the  video  clips  used  in 
this  multimedia  courseware  are  nursing  practitioners  who  have  volunteered  their  time. 


Procedure 

Much  of  human  reasoning  is  case-based  rather  than  rule-based.  When  people  solve  problems,  they  are 
frequently  reminded  of  previous  problems  they  have  faced  in  similar  situations.  Most  people  have  experienced 
problems  brought  about  in  their  daily  life.  For  instance,  you  are  in  a queue  at  the  ATM  to  withdraw  money  and 
you  realise  that  you  forgot  your  PIN  number,  you  are  reminded  of  other  times  you  have  been  in  similar 
situations.  While  rushing  to  catch  a train  you  realise  that  you  do  not  have  the  right  change  to  purchase  a pass 
from  the  ticket  machine,  you  are  reminded  of  other  times  when  you  have  been  similarly  caught  out.  People 
constantly  experience  such  "remindings”,  comparing  one  experience  to  another  so  as  to  learn  from  both. 

Often  we  believe  that  our  mind  is  wandering,  as  it  seems  to  flit  from  thought  to  thought,  leading  us  in 
directions  that  often  seem  irrelevant  to  our  needs  at  the  time.  But  the  reminding  process  is  reflective  of  our 
mind's  constant  search  for  past  information  to  help  in  processing  new  information.  In  effect  we  are  creating 
theories  about  the  minute  details  in  the  world  around  us,  trying  to  create  a theory  of  banking  procedures  that  will 
help  us  to  select  the  right  one  and  better  function  in  the  world.  We  are  constantly  accumulating  cases  and 
comparing  those  to  the  cases  we  have  accumulated  in  an  effort  to  understand  the  next  case  that  will  appear. 

In  line  with  this  process,  the  first  step  in  the  development  procedure  for  this  project  was  to  develop  and 
shoot  the  precipitating  event.  The  experiencing  of  anaphylaxis  was  chosen  as  the  event  because  it  represents  a 
reasonably  complex  occurrence  in  a hospital  as  well  as  a common  scenario  in  the  community,  for  example, 
children  receiving  immunisation  at  the  clinic,  or  being  stung  by  a bug  etc.  Except  for  the  patient  experiencing 
anaphylaxis,  all  other  participants  in  this  video  shoot  were  educators  and  health  professionals.  The  next  major 
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task  was  to  capture  a large  enough  repository  of  the  experience  of  nurse  practitioners  in  the  form  of  stories. 
Gathering  this  repository  was  no  mean  task.  Interviewing,  filming,  and  editing  of  these  stories  took  up  much 
time.  Indexing  of  the  database  of  these  stories  was  an  ongoing  exercise. 

The  model  of  learning  outlined  in  Figure  1 guided  the  development  of  the  learning  transaction.  This 
unfolded  in  several  phases.  Within  the  multimedia  simulation,  users  could  move  easily  from  activity  to  activity 
towards  their  ultimate  goal,  which  was  the  successful  management  of  the  presenting  problem.  Some  of  these 
activities  comprised  selecting  from  a number  of  decision  options,  dragging  and  dropping  them  in  appropriate 
boxes,  deciding  to  administer  a particular  treatment  or  not,  deciding  to  call  for  help  or  not,  choosing  to  consult 
expert  video  clips  for  advisement  or  not,  and  making  decisions.  Resources  apart  from  expert's  stories  such  as 
procedures  and  other  reference  manuals  had  to  be  identified  and  collected  for  the  reference  of  users. 


Evaluation 

Our  approach  to  the  monitoring  and  evaluation  of  the  outcomes  of  this  project  was  utilisation  focussed. 
As  such  our  focus  was  on  the  use  and  utility  of  the  instructional  innovation  for  nursing  students,  their  lecturers, 
graduate  nurses,  and  other  stakeholders  such  as  the  employing  institutions.  Evaluation  of  the  courseware 
prototype  is  continuing.  Formative  evaluation  of  courseware  has  been  carried  out  with  small  groups  of  potential 
users  including  a small  group  (n=3)  of  subject  matter  experts  who  were  asked  to  validate,  among  other  things, 
the  authenticity  of  anaphylaxis.  A semi-structured  Expert  Group  Checklist  has  been  used  for  gathering  the 
comments  of  the  expert  group.  A semi-structured  questionnaire  including  an  Interface  Rating  Scale  has  been 
used  to  collect  data  from  the  group  of  potential  users.  A larger  scale  implementation  evaluation  of  this 
courseware  is  planned.  This  will  involve  observations,  focus  groups  and  use  of  semi-structured  questionnaires. 

Results  of  the  formative  evaluation  carried  out  so  far  show  that: 

• users  found  the  learning  program  easy  to  use  and  navigate, 

• the  majority  did  not  find  the  information  overwhelming,  and  the  content  pitched  at  a 
appropriate  level  for  the  target  group, 

• the  curriculum  was  well  balanced. 

With  regards  to  interface  design: 

• users  felt  that  the  screen  design  was  pleasing  and  appropriate  use  of  graphics  was  made, 

• the  clarity  of  information  presentation  was  high. 

On  the  whole  they  found  the  program  enjoyable.  Subjects  were  asked  how  interesting  did  they  find  the 
material  on  each  one  of  the  sections  on  the  CD?  Material  on  the  CD  was  itemised  to  include:  Handover, 
attending  to  patient,  listening  to  stories,  making  decisions,  and  preparing  the  report. 

• The  majority  reported  each  one  of  these  sections  either  interesting  or  very  interesting. 

Subjects  were  also  asked  to  rate  the  usefulness  to  them  of  these  same  attributes. 

• All  of  them  rated  these  attributes  either  useful  or  very  useful. 

In  summary,  what  did  they  like  most  about  the  multimedia-based  simulation?  These  included  simple 
access  to  different  components,  more  interesting  and  retained  information  in  comparison  with  reading  a journal 
article  or  book,  realistic  presentation,  interactive  patient  observation. 
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Abstract.  In  1 945  Vannevar  Bush  created  a theoretical  machine  that  constructs  and  stores  knowledge  as  trails 
through  information.  This  presages  Web-based  learning  and  points  out  that  authoring  and  delivery 
environments  should  be  based  on  information  and  knowledge  constructs.  Unfortunately,  current  Web-based 
learning  environments  are  based  on  the  manipulation  of  data.  This  approach  links  data  rather  than  ideas,  forces 
authors  to  think  about  form  rather  than  pedagogic  substance,  and  makes  it  almost  impossible  to  extend  the 
functionality  of  commercially  available  learning  environments  in  new  and  creative  ways. 

An  information-based  approach  can  be  realized  using  the  object-oriented  paradigm  applied  first  to 
instructional  design  and  then  to  the  software  built  to  support  it.  This  note  summarizes  our  work  in  this 
direction.  We  treat  the  problem  of  hyperlinks  with  conclusions  similar  to  those  of  Hermann  Maurer's 
Hyperwave  group  and  illustrate  how  the  object-oriented  approach  can  add  extensibility  to  on-line  learning 
environments. 


Hyperlinks,  Data,  and  Information 

When  thinking  about  pedagogy  it  is  useful  to  distinguish  among  data,  information,  and  knowledge.  In  (Robson 
1999)  we  illustrate  these  epistemological  categories  by  looking  at  tabular  data  that  is  to  be  included  in  a 
document  or  lecture.  The  main  point  is  that  forcing  an  author  to  enter  data  into  table  cells  via  a GUI  fixes  the 
form  in  which  the  data  is  displayed.  This  makes  it  impossible  to  specify  context  sensitive  rendering  methods 
and  thereby  inhibits  the  ability  of  an  author  to  think  in  terms  of  connections  among  ideas.  Instead,  the  author  is 
forced  to  spend  time  on  the  format  used  to  display  data.  A table  or  graph  is  a means  to  convert  data  into 
information  and  the  problem  we  encounter  is  that  the  authoring  system  confuses  the  two. 

Another  fundamental  example  of  this  same  phenomenon  is  the  notion  of  a hyperlink  itself  It  has  been  pointed 
out  by  that  standard  hyperlinks  carry  with  them  the  same  flaws  that  GO  TO  statements  carry  in  programming 
languages.  To  quote  from  (Lennon  & Maurer  1996), 

There  is  certainly  no  significant  data  model  for  WWW,  and  in  many  ways  first  generation 
systems  may  rightly  be  likened  to  novice  programs  that  used  too  many  GO  TO  statements. 

Manifestations  of  these  flaws  are  the  prevalence  of  broken  links,  the  inability  of  users  to  annotate  documents, 
and  the  difficulty  of  creating  symbolic  links  by  traditional  means.  The  computer  science  analysis  of  the 
hyperlink  problem  is  that  hyperlinks  should  be  object-oriented  instead  of  procedural  in  nature.  This  approach  is 
taken  by  Kappe,  Maurer,  and  Tomek  in  their  work  on  the  Hyperwave  system  developed  at  the  Institute  for 
Information  processing  and  Computer  supported  new  Media  in  Graz,  Austria  (Maurer  1 999). 

Whereas  we  agree  with  the  Hyperwave  solution,  from  the  pedagogic  perspective  the  underlying  problem  lies  in 
the  confusion  of  data  with  information.  In  the  minds  of  authors,  hyperlinks  are  symbolic  and  informational.  For 
example,  an  instructor  might  wish  to  insert  a link  to  a brief  biography  of  Linus  Pauling  into  a Web  page 
discussing  Vitamin  C.  The  instructor  may  or  may  not  have  a specific  biography  in  mind.  The  URL  should 
really  be  “link  that  displays  a brief  biography  of  Linus  Pauling''  rather  than  something  like 
http://bio.linus.pauling.edu.  By  hard-coding  the  URL  of  a link,  information  is  replaced  by  its  instantiation  and 
becomes  data.  This  is  not  what  we  want. 

Vannevar  Bush's  memex  machine  (Bush  1945)  is  considered  to  be  a visionary  description  of  the  Web  (Berners- 
Lee,  1 995).  It  is  actually  an  even  more  visionary  description  of  an  on-line  learning  environment.  Below  is  a 
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quote  from  the  memex  article  that  describes  the  use  of  the  machine.  Note  how  the  owner’s  goal  is  described 
using  the  words  ’’interest"  and  "study",  which  implies  a quest  for  knowledge,  and  that  this  knowledge  is 
constructed  by  finding  information  (books  and  articles)  and  linking  this  information. 

The  owner  of  the  memex,  let  us  say,  is  interested  in  the  origin  and  properties  of  the  bow  and 
arrow.  Specifically  he  is  studying  why  the  short  Turkish  bow  was  apparently  superior  to  the 
English  long  bow  in  skirmishes  of  the  Crusades.  He  has  dozens  of  possibly  pertinent  books 
and  articles  in  his  memex.  First  he  runs  through  an  encyclopedia,  finds  an  interesting  but 
sketchy  article,  leaves  it  projected.  Next,  in  a history,  he  finds  another  pertinent  item,  and 
ties  the  two  together.  Thus  he  goes,  building  a trail  of  many  items.  Occasionally  he  inserts  a 
comment  of  his  own,  either  linking  it  into  the  main  trail  or  joining  it  by  a side  trail  to  a 
particular  item.  When  it  becomes  evident  that  the  elastic  properties  of  available  materials 
had  a great  deal  to  do  with  the  bow,  he  branches  off  on  a side  trail  which  takes  him  through 
textbooks  on  elasticity  and  tables  of  physical  constants.  He  inserts  a page  of  longhand 
analysis  of  his  own.  Thus  he  builds  a trail  of  his  interest  through  the  maze  of  materials 
available  to  him. 

We  observe  in  (Robson  1999)  that  learning  is  the  process  of  acquiring  or  constructing  knowledge  and  that  many 
components  of  teaching  deal  primarily  with  information.  As  in  the  Vannevar  Bush  description,  a key 
component  is  the  forming  of  cognitive  relationships  among  pieces  of  information.  Yet  the  vast  majority  of 
current  on-line  educational  resources,  the  current  versions  of  memex,  operate  strictly  in  the  data  domain.  In 
memex  terms  they  are  fixated  with  the  physical  production  of  the  books  and  the  fonts  used  for  annotations. 

This  has  many  negative  consequences.  One  is  that  too  much  effort  is  spent  on  form  and  format  and  too  little  on 
pedagogic  substance.  This  contributes  to  the  large  cost  of  developing  Web  courses  or  course  components  at 
Universities  (Arenson  1998  and  Harapnuik  et.al.  1998).  Another  consequence  is  that  authors  and  designers 
cannot  interact  with  learning  environments  in  a language  built  around  concepts  of  teaching  and  learning.  This  is 
typified  by  the  need  to  make  hard  links  instead  of  symbolic  cognitive  links.  But  the  worst  effect  is  that  teachers 
and  authors  are  using  technology  that  dictates  and  limits  on-line  pedagogic  methods  instead  of  using 
technology  which  both  serves  existing  pedagogy  and  encourages  the  development  of  new  instructional 
strategies.  We  will  discuss  that  now. 


The  Limits  of  Commercial  Environments. 

Confusing  data  with  information  can  make  more  work  for  the  author,  but  the  real  problem  with  the  functional 
nature  of  current  authoring  environments  is  the  lack  of  extensibility.  Towards  the  end  of  Bush’s  memex  article 
(Bush,  1945)  he  speculates  that 

Wholly  new  forms  of  encyclopedias  will  appear,  ready  made  with  a mesh  of  associative 
trails  running  through  them,  ready  to  be  dropped  into  the  memex  and  there  amplified. 

The  Web  reflects  this  in  its  ability  to  integrate  new  sites,  but  the  tools  that  exploit  the  Web  for  pedagogic 
purposes  are  limited  to  the  functionality  that  the  creators  envisioned.  To  illustrate  this,  suppose  that  we  are 
using  a commercial  course  management  system  that  offers  all  of  the  standard  amenities:  user  types,  quizzes, 
navigation,  record-keeping,  chat,  email,  and  so  on.  Suppose  further  that  we  would  like  to  use  this  on-line 
environment  to  do  some  new  things,  for  example  to 

(A)  display  student  progress  towards  a learning  objective  as  a bar  chart, 

(B)  pair  students  and  have  them  exchange  papers  and  critique  each  others  papers,  or 

(C)  have  different  students  see  different  examples  on  the  same  Web  page  depending  on  their  scores  on  a 
particular  quiz. 

These  are  all  things  we  have  done  using  the  Web  and  that  represent  sound  pedagogy.  (A)  was  suggested  to  the 
author  by  a high  school  guidance  counselor  and  was  implemented  in  a mathematics  Web  site.  The 


implementation  uses  nothing  more  complicated  than  an  image  tag  with  the  width  parameter  read  in  from  a 
database.  (B)  is  something  we  have  done  using  QuestWriter  software  developed  at  the  author’s  home 
institution  (Bogley  et  al.  1996).  In  our  implementation,  the  computer  takes  care  of  all  of  the  management 
including  pairing  off  students,  delivering  electronic  versions  of  papers  and  comments,  and  sending  email 
reminders  when  a student  falls  behind  on  an  assignment.  (C)  is  an  example  of  adaptive  pages.  Adaptive  pages 
have  been  implemented  in  a number  of  academically  developed  environments  including  Interbook  (Interbook 
1996)  and  QuestWriter  (QuestWriter,  1998). 

Despite  all  of  this,  the  creators  commercial  environments  have  not  included  any  of  the  functionality  represented 
by  (A),  (B),  or  (C).  It  might  be  possible  to  bend  the  environments  to  implement  the  associated  pedagogical 
ideas,  but  it  would  be  very  hard  to  do  so  neatly  and  impossible  to  truly  add  new  functionality. 


An  Object-oriented  Model 

If  our  goal  is  to  create  an  environment  that  operates  on  a higher  level  than  data  manipulation  and  that  permits 
users  or  designers  to  add  new  functionality,  then  we  need  a new  underlying  model.  In  (Robson,  1999)  we 
propose  such  a model.  We  start  with  an  object-oriented  approach  to  instructional  design  that  is  independent  of 
the  medium  in  which  it  is  realized  but  works  very  well  for  the  Web.  At  its  core  is  the  notion  of  a learning 
object.  Examples  of  learning  objects  include  "explanations",  "assessments",  "lectures",  and  "applets". 

Anything  that  an  instructor  or  student  might  use  as  a path  to  learning  is  a candidate  for  a learning  object.  In  our 
model  the  designer  (or  author)  defines  learning  objects  and  manipulates  their  methods  and  properties.  Methods 
might  include  ways  of  rendering  an  object  in  various  media,  and  properties  might  include  other  objects.  An 
important  property  of  an  object  is  a relationship  to  another  object,  often  rendered  as  a hyperlink  in  the  Web 
context.  This  allows  us  to  incorporate  the  symbolic  and  cognitive  notions  of  links  discussed  earlier  in  this  note. 

In  (Robson,  1998)  we  illustrate  how  this  object-oriented  model  can  be  implemented  in  a primitive  way.  We 
discuss  a few  Web  sites  with  ad-hoc  authoring  environments  built  using  a simple  database  and  Web  integration 
software  (Robson  & Whitesitt  1999).  In  these  environments  the  author  can  associate  an  external  link  to  the 
content  of  a page,  or  a learning  objective  to  a lecture,  by  using  a pull-down  list  of  available  links.  Although  this 
is  just  a first  step,  it  brings  out  the  underlying  philosophy  that  the  role  of  the  author  is  to  specify  relationships 
among  objects  and  not  to  worry  about  how  those  relations  will  be  rendered. 


Extensibility 

Our  primitive  attempts  do  not  illustrate  the  possibility  of  creating  an  extensible  environment.  We  believe  this  is 
not  only  possible  but  is  where  the  next  generation  of  instructional  management  systems  must  go.  Although  we 
have  built  none  of  this,  and  therefore  must  admit  the  possibility  of  many  hidden  difficulties,  it  is  worthwhile 
speculating  about  the  shape  and  form  of  such  an  environment.  What  we  envision  is  an  environment  in  which 
every  component  can  be  accessed  with  full  authoring  and  design  privileges  by  an  appropriately  authenticated 
user.  The  various  learning  objects  would  be  displayed  graphically.  Properties  could  be  assigned  from  lists  and 
sub-objects  could  be  defined  with  automatic  inheritance  of  properties. 

To  illustrate  what  could  be  done  with  this  environment,  start  with  your  favorite  course  management  system  and 
imagine  that  all  components  are  object-oriented.  As  an  exercise,  suppose  we  wanted  to  construct  a new 
component  that  supports  group  work.  We  would  need  a way  to  form  groups,  a way  to  facilitate  group 
communication,  a way  to  give  out  projects,  and  a way  to  determine  scores  on  the  projects.  Of  these,  the  part 
that  is  most  obviously  missing  is  the  one  that  forms  groups  of  students,  but  we  could  start  with  the  "class" 
object’.  A "group"  is  just  a "class"  within  a "class".  We  should  therefore  be  able  to  suitably  define  a new 
"group"  object  that  would  inherit  properties  from  the  "class"  object.  This  would  immediately  give  a 


’ Most  academically-oriented  management  systems  include  a notion  of  a "class"  (a  class  of  students,  not  a class 
in  the  programming  sense)  as  a top-level  construct. 
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'‘communication”  property  that  could  be  realized  as  a chat  room  and/or  threaded  email,  an  "assessment” 
property  that  would  allow  us  to  give  and  score  assignments  with  deadlines,  and  so  on.  The  assignments 
associated  to  our  new  "group”  object  should  probably  be  re-named  "projects”.  We  would  also  want  to  change 
some  of  the  rendering  properties  to  give  the  group  project  a distinctive  look  and  we  might  want  to  associate  text 
or  interactive  exercises  to  "projects”.  All  of  this  could  be  done  without  much  difficulty  because  our  envisioned 
object-oriented  authoring  tool  supports  precisely  these  types  of  transactions. 

In  our  own  real  life  implementation  of  a limited  group  learning  environment  we  relegate  all  management  tasks 
to  the  computer.  It  would  not  be  so  easy  to  program  our  environment  to  automatically  create  student  groups  and 
send  email  reminders  to  non-participating  members  unless  similar  functionality  was  already  present  in  our  base 
learning  environment.  This  points  out  the  need  for  a true  programming  environment  within  an  authoring 
environment.  High-powered  industrial  products,  for  example  Macromedia  products,  have  this  feature,  but 
academically-oriented  management  systems  generally  do  not.  This  is  an  issue  for  another  time  and  place. 


Conclusion 

We  have  traced  a path  that  starts  with  the  realization  that  current  on-line  learning  environments  are  grounded  in 
the  manipulation  of  data  as  opposed  to  the  communication  of  information  and  construction  of  knowledge.  As 
has  been  pointed  out  by  the  computer  science  community,  even  the  hyperlink  suffers  from  this  defect.  Our  path 
takes  us  through  some  of  the  difficulties  this  causes  and  proposes  an  object-oriented  approach  that  allows  us  to 
overcome  these  difficulties.  The  environment  we  propose  is  built  around  learning  objects  and,  unlike  any  of 
which  we  currently  know,  has  the  potential  to  be  extensible.  Authors  and  designers  can  use  it  to  create  new  on- 
line learning  objects  from  old  ones.  Despite  the  fact  that  this  journey  goes  back  at  least  as  far  as  Vannevar  Bush, 
we  feel  we  are  just  at  the  beginning. 
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Abstract;  This  paper  investigates  methods  to  maximise  the  benefits  of  interactivity  in  the 
context  of  computer-based  learning  applications.  To  achieve  this,  the  analysis  initially 
reviews  the  factors  which  to  date  have  defined  interactivity  and  some  recent  assessments  of  its 
impact,  with  a subsequent  assessment  of  interactivity  in  terms  of  three  alternative  components 
- narrative,  play  and  agents.  Commentary  on  perceived  problems  with  interactivity  are  also 
discussed.  Based  on  this  analysis,  the  major  finding  is  that  mutual  reciprocity,  a critical 
element  of  interactivity,  is  not  manifested  in  applications  and  that  the  introduction  of  elements 
of  narrative  and  play  have  the  potential  to  alter  the  dynamic  of  the  interactive  process.  Of 
greater  impact  too  is  the  potential  to  integrate  the  interplay  between  designer  and  learner  into 
the  interactive  engagement.  By  integrating  these  elements,  a revised  model  of  the  interactive 
environment  is  proposed,  based  on  mutual  adaptation  by  designer  and  learner.  By  developing 
our  understanding  of  the  complexity  of  interacting  elements  which  impact  the  interactive 
process,  we  will  take  another  step  towards  a truly  interactive  environment. 


Reassessing  Interactivity 

The  Interactive  World 

Interactivity,  in  the  context  of  computer-based  learning,  ranges  from  simple  navigation  of  web-pages  to 
immersion  in  interactive  virtual  worlds.  It  is  often  portrayed  as  the  distinguishing  factor  of  the  new  media,  with 
the  assumption  that  "interactivity  in  a computer  product  means  that  the  user,  not  the  designer,  controls  the 
sequence,  the  pace,  and  most  importantly,  what  to  look  at  and  what  to  ignore"  (Kristof  & Satran,  1995:35). 
While  this  holds  true  for  recreational  or  information-seeking  activities,  the  value  of  control  for  educational 
pursuits  is  less  clear,  as  there  are  conflicts  between  the  value  of  control  for  new  learners  as  well  as  questions 
regarding  the  overall  prescriptions  for  control  identified  through  research  findings  (Reeves,  1993). 

The  overall  aim  of  this  discussion  is  to  develop  a better  understanding  of  interaction  in  terms  of  the  factors  which 
inhibit  or  enhance  content  engagement,  and  the  communication  or  transfer  which  results  in  deep  processing  and 
consequential  learning.  When  considering  technology-based  learning  the  significant  factor  is  the  engagement 
which  takes  place  between  the  learner  and  the  content,  and  the  extent  to  which  there  is  a change,  manifested 
through  learning,  skill  formation  or  knowledge  acquisition  resulting  from  that  engagement. 

However,  interactivity  as  a determinant  of  effective  learning  has  yet  to  be  fully  established,  based  on  analysis  of 
a range  of  sources  including  theoretical  limitations,  research  credibility  and  the  variety  of  interactive  constructs. 
For  example,  Whitby  (no  date)  describes  the  vision  of  the  future  home  with  a video  wall  of  sports,  interactive 
access  to  player  profiles  and  holographic  video-phones.  While  acknowledging  the  popular  appeal  of  this 
scenario,  Whitby  (no  date:l)  questions  whether  people  really  wish  to  interact  that  way  and  concludes: 

...  we  are  rushing  to  implement  interactive  CDs,  cable  shows  and  personal  electronics  in  the 
crudest  ways  without  pausing  to  consider  whether  an  improved  medium  will  result. 

Storytelling  and  narrative  lie  at  the  heart  of  all  successful  communication.  Crude,  explicit, 
button  pushing  interaction  breaks  the  spell  of  engagement  and  makes  it  hard  to  present 
complex  information  that  unfolds  in  careful  sequence. 

The  problems  confronting  educational  technology  developers  are  clearly  defined  - how  to  develop  computer- 
based  environments  to  engage  the  learner  in  effective  instructional  communication  without  generating 
interactive  interference!  In  addition,  Whitby  (no  date)  also  introduces  the  notion  that  storytelling  and  narrative 
are  critical  determinants  of  communication,  which  is  the  ultimate  goal  for  educational  multimedia  applications. 
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Interactive  Challenges 

In  addition  to  the  impact  of  internet  communication  in  the  tertiary  environment,  the  development  of  stand-alone 
computer-based  learning  applications  is  widespread  and  often  a critical  business  risk.  As  we  continue  to  research 
and  develop  computer-assisted  learning  applications,  it  becomes  clear  that  there  remains  much  to  learn  about  the 
nature  of  interactivity  and  implementing  its  components  effectively  and  appropriately.  In  noting  that  complexity 
comes  with  the  freedom  associated  with  learner  control,  Kirsh  (1997)  asserts  that  there  are  additional  restrictions 
resulting  from  the  scripting  of  applications  which  require  the  user  to  adapt  to  their  structure,  suggesting  that: 

Since  interactive  interfaces  ought  to  foster  this  type  of  coordination  between  improvisation 
and  planning  we  need  to  discover  better  theories  of  what  is  involved  in  the  dynamic  control  of 
inquiry,  line  of  thought,  and  action  more  generally.  We  need  to  discover  more  open-ended 
models  of  coherence  and  narrative  structure.  (Kirsh,  1997:81) 

While  acknowledging  that  one  solution  is  to  scaffold  a learning  environment  to  support  rather  than  direct,  Kirsh 
(1997:83)  argues  that  an  analysis  of  the  nature  of  interactivity  in  terms  of  reciprocity  between  the  two  parties 
(designer  and  user)  suggests  that  "computer  interfaces  are  rarely  interactive  because  the  programs  that  drive 
them  are  rarely  intelligent  enough  to  behave  as  tacit  partners".  In  this  instance,  as  well  as  the  observation  by 
Whitby  (no  date),  the  concept  of  narrative  for  coherence  is  introduced,  which  Kirsh  (1997)  beleives  might  be 
resolved  through  the  application  of  a modified  decision-cycle  theory. 

From  a different  perspective,  Bardini  (1997)  compared  the  concepts  of  association  and  connection  in  relation  to 
interaction  with  hypertext  and  hypermedia  environments,  observing  that  most  implementations  to  date  have  been 
associationist,  extending  the  argument  to  consider  the  relationship  of  the  main  protagonists  (agents)  in  the 
interactive  process.  Ascribing  the  process  of  delegation  and  inscription  to  both  the  designer  and  user,  "the  degree 
of  interactivity  of  the  interface  can  be  seen  as  the  relative  opportunity  for  both  user  and  designer  to  take  part  in 
the  two  dimensions  of  the  representation  process"  (Bardini,  1997:12).  While  this  supports  the  position  of  Kirsh 
(1997),  it  also  highlights  the  potential  of  creating  gulfs  (Norman  - cited  in  Booth,  1989)  which  might 
compromise  mutual  engagement  of  the  user  and  designer  - the  ideal  of  the  interface  as  a socially  constructed 
narrative  (Bardini,  1997). 

In  brief,  these  two  observations  suggest  that  interactivity  remains  an  uncertain  concept  and  that  developing 
applications  in  such  a way  that  the  learner  is  integrated  into  some  form  of  narrative  may  lead  to  implementations 
which  reduce  the  gulf-potential  between  user  and  designer.  However,  it  is  also  necessary  to  appreciate  that  there 
are  many  factors  that  contribute  to  effectiveness  of  computer-based  learning.  The  following  section  reviews 
these  dimensions,  with  specific  emphasis  on  the  relationship  between  learning  theory  and  subsequent  learning. 


Learning  Theory  and  Interactivity 

To  assess  the  relationship  between  learning  and  interactivity  in  a computer-based  environment,  it  is  necessary  to 
examine  the  extent  to  which  the  various  approaches  to  learning  provide  guidance  or  indices  of  relative  and 
appropriate  interactions  to  foster  learning.  By  way  of  example,  Craik  & Lockhart  (1972)  suggest  that  the 
conditions  of  learning  can  affect  whether  surface  or  deep  learning  results.  According  to  Kearsley  (1998),  the 
levels  of  processing  approach  views  stimulus  information  as  being  processed  at  multiple  levels  simultaneously 
depending  upon  its  characteristics,  and  that  deeper  processing  will  result  in  more  being  remembered.  In  addition, 
the  meaning  of  the  presentation  will  affect  the  extent  to  which  material  will  be  remembered;  similarly  the  less 
meaning  implies  greater  attention  and  therefore  an  interruption  to  the  processing  flow. 

For  the  development  of  a learning  task  within  this  paradigm,  educators  are  required  to  present  information  both 
meaningful  and  relevant,  aiming  to  ensure  that  attention  is  focused  on  the  content  without  interference. 
Transposed  to  a computer-based  environment,  the  context  within  which  the  material  is  presented  is  alien,  at  least 
on  the  first  encounter,  and  therefore  attention  can  be  required  to  de-construct  the  content  from  the  interface, 
metaphor  or  context.  If  attention  is  too  focused  on  the  de-construction  process,  it  can  be  argued  that  the 
consequential  learning  might  be  interrupted  or  de-emphasised,  generating  more  superficial  rather  than  deep 
learning.  This  reinforces  the  potential  of  interactivity  to  actually  interfere  with  the  learning  process. 

From  a human-computer  interface  perspective.  Booth  (1989)  identifies  that  users  can  have  mapping  difficulties 
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when  accessing  computer  systems  (matching  the  information  presented  with  their  current  experience),  which 
have  been  described  by  Norman  (cited  in  Booth,  1989)  as  being  gulfs  that  prevent  users  from  dealing  easily  and 
efficiently  with  computer-based  tasks.  The  gulf  of  execution  exists  when  the  user  knows  what  is  to  be  achieved 
but  does  not  know  which  physical  variable  to  adjust  or  in  what  way  to  adjust  them  and  the  gulf  of  evaluation 
exists  where  the  system  has  altered,  usually  as  a result  of  the  user’s  actions,  but  the  user  cannot  easily  understand 
the  change  in  that  system’s  state.  The  assertion  by  Craik  & Lockhart  (1989)  that  meaning  is  critical  to  deep 
processing  underlies  the  importance  of  effective  design  of  the  human-computer  interface  if  successful 
engagement  is  to  occur.  The  interaction  of  the  user  in  manipulating  the  system  forms  a precursor  to  their 
interaction  with  the  content  and  thus  any  disturbance  in  the  manipulation  may  have  subsequent  consequences 
with  that  engagement.  This  example  highlights  the  impact  one  approach  can  have  on  the  way  we  think  about 
interactions  and  supports  further  investigation  of  a wider  range  of  learning  theories  and  paradigms  to  help  clarify 
the  links  between  interactivity  and  learning.  However,  this  form  of  analysis  is  beyond  the  scope  of  the  present 
discussion. 


Interactivity  and  Narrative 

Introduction 

The  overt  issues  faced  by  prospective  designers,  developers  and  implementors  of  educational  software  is  often 
the  look  and  feel  of  the  product.  What  is  often  omitted  is  an  analysis  of  aspects  of  the  total  interactive  experience 
that  will  maximise  engagement  (manifested  through  learning,  knowledge  acquisition  or  skill  development)  and 
the  overall  communication  of  content.  However,  it  is  proposed  that  the  frequent  development  in  the  base 
technology  (mainframe,  desk-top,  on-line)  have  taken  precedence  over  developing  a comprehensive 
understanding  of  the  means  by  which  the  technology  might  enhance  the  learning  process. 

As  continually  evolving  field,  educational  technology  is  often  subject  to  creativity  and  experimentation  as  the 
most  legitimate  means  by  which  its  potential  can  be  realised.  At  the  same  time,  there  is  considerable  evidence  to 
support  the  conclusion  that  we  do  not  yet  fully  understand  its  similarity  to  or  differences  from  current  means  of 
educational  communication  (Plowman,  1996b;  Kirsch,  1997).  There  is  no  doubt  that  much  of  the  development 
has  assumed  that  the  technology  is  simply  an  automated  form  of  current  practice,  and  this  may  in  part  be 
responsible  for  the  often  stated  comment  that  educational  technology  has  failed  to  live  up  to  its  original 
expectations.  In  other  words,  should  the  implementation  of  educational  resources  on  a computer-based  medium 
be  a replication  of  current  media,  or  are  we  dealing  with  a new  environment  that  has  not  yet  been  fully  explored 
in  terms  of  interface  to  and  interaction  with  its  human  users? 

There  is  a growing  body  of  literature  and  discussion  which  supports  the  latter  position,  through  a re-assessment 
of  the  way  in  which  we  conceptualise  the  use  of  interactivity  in  educational  multimedia  applications.  One  of  the 
most  interesting  aspects  of  this  research  is  the  comparison  of  traditional  story-telling  techniques  (narrative  and 
play)  with  those  presented  through  the  technological  medium.  While  the  complex  framework  which  surrounds 
the  study  of  narrative  and  story-telling  should  not  be  trivialised,  in  the  context  of  the  growing  demand  for 
technology-based  solutions  to  learning,  the  possibilities  provide  a novel  and  refreshing  perspective. 


Interactive  Impact 

Based  on  the  previous  discussion,  there  is  an  indication  that  if  interactivity  has  the  potential  to  distract,  the  use  of 
narrative  may  be  useful.  Narrative  can  be  viewed  from  a simplistic  context  of  representing  a linear  storyline 
(Plowman,  1 996a);  it  can  also  be  perceived  in  the  way  it  is  deconstructed  in  terms  of  how  the  story  is  told,  the 
way  it  is  received,  what  meanings  it  can  have  and  the  specific  social,  cultural  and  technological  context  in  which 
it  is  told  (Humphreys,  1997b).  An  extension  of  this  analysis  is  the  assertion  that  narrative  enables  construction  of 
mental  models  of  the  situation  and  environment  (Bower  & Morrow,  1990).  Therefore  it  can  be  predicted  that 
narrative  may  assist  meaning,  reduce  the  impact  of  interactive  interference  and  provide  the  necessary  framework 
to  promote  learning  amongst  diverse  groups  of  learners. 

But  in  what  way  can  narrative,  as  currently  understood,  relate  (or  integrate)  with  the  values  of  an  interactive 
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world?  Josephson  (1997)  suggests  that  we  are  still  learning  to  define  a new  media  literacy,  and  by  defining  it  we 
are  also  creating  it.  If  we  can  define  an  effective  interactive  narrative,  then  we  can  begin  to  project  what  will 
happen  to  that  narrative  as  the  audience  moves  from  being  "actively  engaged  on  an  interpretive  level  to  actively 
intervening  in  its  representation"  (Humphreys,  1997a).  In  a comprehensive  analysis  of  narrative  and  interactivity. 
Plowman  (1996a:92)  observes  that 

Narrative  coherence  is  identified  here  with  a lack  of  redundancy  and  a fixed  sequence. 

Interactive  multimedia  (IMM)  programmes  challenge  these  traditional  definitions  of  narrative 
because  it  can  be  suspended  or  altered  at  discrete  decision  points,  the  foci  of  interactivity, 
and  a rearrangement  of  discrete  elements  gives  rise  to  new  text  and  new  meanings.  While  the 
concepts  of  wholeness,  unity  and  coherence  of  meanings  are  not  fashionable  in  a post-modern 
world,  in  "educational  multimedia...  the  notion  of  multiple  interpretations  has  different 
implications,  particularly  for  comprehension  and  cognition  ". 

Because  these  decision  points  represent  the  foci  of  interactivity,  and  a rearrangement  (through  branching  or 
learner  control)  of  discrete  elements  gives  rise  to  new  text  and  new  meanings,  the  implication  in  educational 
multimedia  of  multiple  interpretations  becomes  critical,  particularly  in  relation  to  comprehension  and  cognition. 
While  "narrative  isn’t  just  a shaping  device:  it  helps  us  think,  remember,  communicate,  and  make  sense  of 
ourselves  and  the  world"  (Plowman,  1996a:3),  when  contrasted  with  its  implementation  in  an  interactive 
environment,  its  efficacy  is  uncertain.  So  can  narrative  and  interactivity  co-exist?  Plowman  (1996b)  argues  that 
interactivity  is  at  odds  with  our  expectations  of  traditional  narrative  forms  and  communication.  As  disruption  of 
the  narrative  is  strongest  at  the  foci  of  interactivity,  interactive  multimedia  development  should  be  considered  in 
terms  of  how  it  can  be  integrated  into  the  underlying  narrative.  However,  by  examining  techniques  for  measuring 
interactivity  and  comparing  these  with  the  basic  narrative  structures,  the  potential  for  the  new  media  to  combine 
and  incorporate  effective  narrative  was  examined  by  Plowman  (1996a)  who  concluded  that,  with  young  learners 
at  least,  a meaningful  narrative  was  beneficial  for  learning. 

In  an  alternate  view  of  interactivity,  Hilf  (1997:7)  states  that  "through  the  interrupted  narrative,  the  learner  learns 
more  about  the  story  and  characters  through  their  own  interaction".  Whereas  Plowman  (1996a,  1996b) 
recognised  the  negative  interference  potential  of  interactivity,  Hilf  (1997)  declares  the  opposite.  Clearly  there  are 
disparate  views  on  narrative  and  interactivity,  and  their  combined  impact  on  the  communication  of  content; 
whether  an  interactive  device  contributes  to  engagement  and  meaning  or  generates  an  interruption  to  that  process 
is  the  underlying  reason  for  considering  the  impact  of  narrative  on  the  overall  process. 

In  his  analysis,  Hilf  (1997)  introduces  four  narrative  structures:  linear  (where  the  user  is  guided  from  beginning 
to  middle  to  end),  interrupted  (where  the  narrative  is  halted  while  problems,  tests  or  some  other  form  of 
interaction  is  implemented),  branching  (where  the  user  has  the  option  to  choose  from  a range  of  paths)  and 
object-oriented  (where  elements  within  the  narrative  can  be  controlled  or  defined  by  the  user,  thereby  impacting 
on  other  users  within  the  system).  In  terms  of  interative  learning  environments,  it  would  appear  that  branching 
narrative  is  termed  menu  selection  and  interrupted  narrative  the  tutorial  model  in  which  a pre-designed  sequence 
is  interspersed  with  interactivity  in  the  form  of  the  question-response-feedback  loop.  The  object-oriented 
narrative  conforms  to  the  delegation/inscription  factors,  at  least  in  terms  of  the  user  (Bardini,  1997). 


From  Narrative  to  Play 

So  does  narrative  interfere  with  or  promote  engagement  during  an  interactive  learning  session?  The  answer  is 
likely  to  be  both,  but  possibly  to  the  detriment  of  the  final  outcome.  An  extension  of  the  narrative  aspects  of 
interactivity  are  proposed  by  Humphreys  (1997a).  Like  Plowman  (1996b),  she  suggests  that  giving  the  audience 
(or  user)  choices  can  disrupt  the  sequence  of  events,  affecting  the  final  closure  of  the  narrative.  From  another 
perspective,  "as  the  level  of  interactivity  increases  and  the  amount  of  agency  for  the  user  structured  into  the  piece 
increases,  the  amount  of  ‘retelling’  done  decreases"  (Humpherys,  1 996a:6). 

To  provide  an  answer  to  this,  Humpherys  (1997b)  explored  the  concept  of  play  and  play  theory,  which  she 
suggests  are  closely  related  to  narrative,  and  notes  that  to  maximise  audience  engagement  through  interactivity 
requires  consideration  of  agency,  narrative  structure,  emotional  engagement  and  construction  of  meaning.  The 
consequences  of  this  conundrum  may  find  some  solutions  in  play  theory,  which  Humpherys  (1997b:9)  suggests 
can  provide  a "framework  which  accommodates  the  audience  or  user  into  the  process  of  engagement  with 
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interactive  media  in  ways  that  narrative  theory  finds  difficult".  While  narrative  is  about  the  experience  of  a 
recounting  of  a story,  play  is  the  experience  of  enacting  a story. 

IMAGE  NOT  AVAILABLE 

Figure  1:  Intersection  and  Parallel  Concepts  of  Interactivity 

A preliminary  comparison  between  narrative  and  play  interactivity  is  represented  in  Fig.  1 . The  top  sample  of  an 
interactive  sequence  represents  a narrative  regularly  intersected  or  interrupted  by  an  interaction  whereas  the 
bottom  sample  depicts  a constant  interaction  between  user  and  story.  While  play  in  terms  of  the  instructional 
game  has  long  been  recognised  as  a valid  model  (Alessi  & Trollip,  1991),  the  early  implementations  were  not 
contextualised  in  narrative.  By  integrating  play  elements  such  as  friendship,  risk,  problem  solving,  competition 
and  creativity,  Humpherys  (1997b:9)  suggests  that  "play  theory  offers  a framework  which  accommodates  the 
audience  or  user  into  the  process  of  engagement  with  interactive  media  in  ways  that  narrative  theory  finds 
difficult".  More  importantly,  in  the  context  of  educational  applications  using  multimedia,  Humpherys  (1997:1 1) 
speculates  that  “Interactivity  produces  for  the  user  of  media  a different  relationship  to  story.  This  shift  in 
relationship  may  be  able  to  be  framed  as  a shift  from  narrative,  as  an  experience  of  recounting  a story,  to  play,  as 
an  experience  of  enacting  a story”. 

The  notion  that  multimedia  applications  for  learning  should  focus  on  either  narrative  or  play  to  enhance 
engagement  implies  that  the  interactivity  provided  to  users  must  therefore  be  integrated  in  such  a way  that  it  not 
only  provides  opportunity  to  reinforce  the  specific  learning  objectives  but  also  to  maintain  the  user’s 
participation  in  the  story. 


Multilinear  Narrative 

Rather  than  comparing  the  traditional  linear  narrative  with  non-linear  interactive  multimedia,  Johnson  & Olivia 
(no  date)  use  the  term  multi-linear.  This  term  implies  a range  of  equally  plausible  paths  which  may  be  taken 
through  the  application  and  offer  the  suggestion  (with  respect  to  internet  sites  specifically)  that  beginnings  and 
endings  should  be  replaced  with  the  terms  entrance  and  exit.  In  discussing  the  relationship  between  the  content 
and  the  medium,  Johnson  & Olivia  (no  date)  cite  the  work  of  Liestol  1994: 105: 

The  screen  occupies  a third  position,  between  the  three  dimensions  of  space  and  the  one 
dimension  of  time.  The  screen  and  what  it  presents  is  a manifestation  of  the  present,  between 
past  and  future.  Therefore  the  movement  from  space  to  time  and  the  reduction  from  three 
dimensions  to  one  both  halt  at  the  position  of  the  screen  and  its  flatland  of  two  dimensions. 

Obviously  the  design  and  composition  of  elements  on  the  screen  are  of  central  importance  to 
any  critical  study  of  hypermedia  texts. 

This  introduces  a further  variable  to  the  equation  of  effective  interaction  with  respect  to  the  user  having  to 
develop  the  skill  to  either  adapt  to  changing  dimensions  or  to  create  a new  dimension  by  which  they  interpret  the 
technology  interface. 


Conclusion 

Interactivity  is  often  portrayed  as  the  crucial  element  of  the  new  technology,  and  yet  recent  research  has 
demonstrated  that  there  is  still  much  to  understand  about  the  ways  in  which  the  interactive  process  facilities 
access  to  technology,  especially  in  the  context  of  computer-based  learning  applications.  As  Plowman 
(1996a:  102-103)  states, 

disruption  of  the  narrative  is  strongest  at  the  foci  of  interactivity  ...  (which)  should  be 
considered  in  terms  of  how  they  can  be  integrated  into  the  overall  narrative  and  how  they  can 
be  used  as  a way  of  stimulating  interest  in  the  unfolding  narrative  ...  by  considering  the 
interrelationship  of  narrative,  linearity  and  interactivity  and  their  design  implications  we  can 
help  learners  to  make  sense  of  interactive  multimedia. 

And  it  is  our  challenge  to  develop  applications  which  minimise  the  potential  for  interactive  interference.  By 
considering  the  concepts  of  narrative  and  play  in  association  with  the  links  between  the  designer  and  user,  a 
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model  of  adaptive  applications  which  cater  for  both  designer  and  user  is  proposed  as  a means  to  enhance  the 
interactive  process. 
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Abstract:  This  paper  describes  an  interdisciplinary  design  studio  that  utilizes  concepts  and 
methodologies  intended  to  create  a comprehensive  approach  to  the  organization  of  building 
design  through  the  merger  of  several  techniques.  These  techniques  include 
1)  collaborative  learning  and  design,  2)  using  multi-tasking  workstations,  3)  hypertext 
learning  modules  and  courseware,  and  4)  interdisciplinary  team  teaching.  The  studio  merges 
the  preceding  teaching  paradigms,  building  on  current  research  and  the  experience  of  the 
faculty.  The  studio  also  establishes  the  premise  that  architectural  design  studio  and  engi- 
neering laboratories  (structural  and  mechanical)  will  need  to  be  organized  across  depart- 
mental boundaries  as  team  oriented  activities.  Learning  modules  are  being  developed  in  a 
multimedia  format,  using  analog  video  and  hypertext.  The  modules  are  expected  to  be  com- 
pleted using  digital  video,  allowing  latitude  for  evolution  of  the  material.  The  paper  dis- 
cusses the  continual  shift  between  synthetic  and  analytic  processes  in  the  context  of  problem 
solving.  Also  discussed  are  methods  of  representation,  design  assignments  and  design  meth- 
ods and  process. 


The  Problem 

Architects  and  engineers,  who  need  to  interact  during  their  professional  career  in  order  to  build  any  kind  of 
complex  building,  are  educated  entirely  separately.  “Over  the  past  century,  increased  movement  toward  con- 
centration within  an  academic  discipline  has  taken  charge  of  the  curriculum,  as  well  as  serving  to  compart- 
mentalize the  professoriate  and  the  institution.”  (Landry  94)  The  vertical  separation  of  disciplines  occurs  in 
most  universities.  This  suggests  a need  for  modification  of  the  curriculum  and  teaching  methods,  and  devel- 
opment of  the  delivery  of  course  material.  Fortunately,  this  comes  during  a period  of  reflection  in  schools  of 
engineering  and  architecture  when,  as  Mitgang  suggests,  there  are  “growing  doubts  over  whether  the  tradi- 
tional educational  environment  is  preparing  students  for  a rapidly  changing  world  outside.”  While  “schools 
remain  wedded  to  shopworn  traditions”  there  seems  to  be  a growing  malaise  about  the  role  of  design  as  the 
centerpiece  of  architectural  education.  (Mitgang  97)  Recognition  of  the  problem  also  comes  at  a time  when 
new  teaching  methods  are  emerging. 

The  problem  with  the  vertical  separation  between  disciplines  is  that  students  from  each  discipline  learn  to 
solve  their  part  of  the  problem  independently.  This  separation  discourages  them  from  understanding  the  rela- 
tionship among  the  problem  components.  The  students  usually  complete  their  projects  without  having  the  op- 
portunity to  gain  insight  to  the  trade-offs  required  for  an  optimal  solution.  While  courses  in  each  department 
include  information  about  the  other  disciplines,  each  is  taught  in  a manner  which  tends  to  diminish  the  im- 
portance of  integration.  Moreover,  the  ultimate  professional  relationship  among  the  disciplines,  which  in  the 
building  industry  consists  of  teams,  is  downplayed  by  the  students  working  as  individuals  in  their  classroom 
experience.  Research  outside  of  the  architectural  and  engineering  professions  suggest  that  “future  work  situa- 
tions are  likely  to  use  a complex  mixture  of  different  information  channels,  including  video  conferencing,  e- 
mail,  small  group  work,  and  on-line  searches.”  (Kilker  & Gay  94)  In  other  words,  information  and  designing 
systems  are  being  developed  which  rely  on  new  modes  of  interaction,  intertwining  the  sociological  and  tech- 
nological aspects  of  the  design  process.  These  are  the  conceptual  underpinnings  of  the  development  of  a new 
approach. 
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Designing  the  Solution 

The  major  effort  in  trying  to  refine  elements  in  the  curriculum  (particularly  in  different  departments)  depends 
upon  what  might  be  termed  “changing  the  culture”  of  the  curriculum.  To  create  these  changes  an  architectural 
design  studio/engineering  laboratory,  TOTAL  Studio,  is  being  organized  comprehensively,  across  departmental 
boundaries.  The  course  is  being  changed  from  independently  organized  activity  to  a team  oriented  activity. 
The  hypothesis  is  that  a concurrent  and  collaborative  design  environment  will  add  to  the  problem-recognition 
and  problem-solving  abilities  of  the  engineering  and  architecture  students.  In  most  workplaces  in  the  building 
industry,  problems  solving  and  design  require  collaboration  among  members  of  a group.  These  activities  re- 
quire that  people  share  information  and  coordinate  their  activities  in  a setting  that  allows  for  immediate  inter- 
action (Kilker  & Gay  94).  Although  the  design  and  production  of  buildings  traditionally  requires  collaboration, 
the  work  is  done  serially,  with  drawings  passing  among  the  professions  and  each  adding  their  information  and 
recycling  through  the  process  until  the  project  is  completed.  No  methods  of  optimization  are  applied  because 
of  the  way  the  design  process  is  structured.  Optimal  solution  spaces  are  closed  off  by  the  time  each  new  part  of 
the  process  is  introduced.  Therefore,  the  focus  of  the  studio  is  around  the  activities  in  which  the  students  can 
engage  to  help  construct  a comprehensive  knowledge  base  necessary  to  design,  optimize  and  build  complex 
structures. 

The  impact  of  the  studio  is  to  improve  the  education,  professional  behavior  and  attitude  of  students  as  they 
prepare  for  various  aspects  of  the  building  industry.  The  students  have  the  opportunity  to  see  how  the  separate 
courses  they  have  taken  in  architecture,  structural  engineering,  and  building  performance  are  integrated.  They 
can  observe  that  contemporary  construction  is  not  a simple  separate,  sequential  process,  but  rather  a system 
characterized  by  integration  and  a search  for  optimal  solutions.  When  design  is  objectively  considered  as  an 
iterative,  multifaceted  process,  and  a series  of  problem  solving  sequences,  a significant  paradigm  shift  can 
occur.  Emphasizing  this  way  of  thinking,  we  discovered,  is  imperative  because  the  way  engineering  and  ar- 
chitecture students  learn  is  different. 

Our  goal  is  to  create  a completely  computer  mediated  environment,  where  students  work  on  their  designs  and 
problems,  communicate  with  each  other  both  locally  and  at  a distance,  and  receive  courseware  and  criticism  at 
the  workstation.  One  innovative  aspect  of  Total  Studio  is  that  it  is  interdisciplinary,  and  has  been  designed 
from  the  beginning  to  provide  the  format  for  the  perpetuation,  replication  and  dissemination  of  the  studio  in  a 
continually  upgradeable  hypertext  format.  The  studio  focus  has  moved  away  from  the  traditional  piecemeal 
design  methodology. 

Implementing  the  Solution 

The  Underlying  Pedagogy 

The  study  of  building  design  must  be  rooted  in  a general  theory  of  building  science  in  which  architectural 
space  and  form,  structure,  and  the  environmental  aspects  of  the  building  envelope  are  considered  inseparably. 
Optimization  is  a complex  problem  to  solve.  “In  architecture,  aesthetics  deals  with  the  way  buildings  look:  the 
skin,  form,  site,  and  overall  image  within  the  culture  to  which  they  belong.  In  engineering,  technology  is  in- 
formed by  modem  science  and  is  systems  and  process  oriented.  Controlling  building  environments  (both  ac- 
tively and  passively)  can  produce  more  insightful  architecture  when  an  understanding  of  the  aesthetics  of  the 
building  envelope  (static)  is  paired  with  an  appreciation  of  the  technology  of  the  building  systems  (dynamic).” 
(Greenwood  et  al  97)  The  studio  focuses  on  the  development  of  structural,  energy  and  spatial  intuitions  and 
the  relationship  among  them.  Study  of  highly  indeterminate  structures  is  crucial  for  understanding  building 
frame  design.  (Black  and  Duff  94)  For  energy  analysis  and  building  performance,  it  is  preferable  to  analyze  a 
skin-dominated  building  so  that  relationship  of  form  to  energy  flow  can  be  considered.  One  goal  of  the  faculty 
is  to  identify  projects  at  the  appropriate  scale  to  study  these  relationships.  We  are  currently  using  an  elemen- 
tary school  project  which  seems  to  have  the  requisite  components  and  degree  of  difficulty. 

Collaboration  and  the  matrix  of  studio  organization 

Collaborative  design — thus  collaborative  learning  about  a project — ^is  a new  concept  for  students  in  the  studio. 
A very  serious  introduction  is  required  to  get  the  students  working  collaboratively.  Collaborative  learning  is 
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“...a  learning  process  that  emphasizes  group  or  cooperative  efforts  among  faculty  and  students,  active  partici- 
pation and  interaction  on  the  part  of  both  students  and  instructors,  and  new  knowledge  that  emerges  from  an 
active  dialog  among  those  who  are  sharing  ideas  and  information.”  (Turoff  95)  While  group  criticism  and 
group  research  is  a normal  activity  in  the  design  studio,  designing  together  is  not.  Turoff,  is  his  years  of  devel- 
opment of  computer-mediated  environments  at  NJIT  concludes  that  “in  many  learning  situations  it  has  been 
observed  that  two  people  working  together  at  a computer  learn  more  working  together  than  either  one  sepa- 
rately. It  is  this  ability  to  share  the  actual  interactive  process  of  ‘creating  the  painting’  that  this  approach  en- 
tails.” (Turoff  95)  This  suggests,  as  Feisel  points  out,  that  “we  need  to  design  an  educational  process  that  in- 
volves students  in  one  another’s  learning  and  rewards  mutual  accomplishment.”  (Feisel  94)  The  studio  adapts 
these  concepts  and  builds  on  them. 

To  facilitate  the  collaborative  idea  of  teaching  and  learning,  the  studio  is  organized  around  a matrix  of  groups 
and  teams.  Groups  are  defined  as  a number  of  students  organized  to  develop  techniques  and  learn  a particular 
set  of  the  task.  Teams  are  defined  as  a number  of  students  assembled  to  apply  techniques  and  complete  a de- 
sign task.  Each  student  is  a member  of  one  group  and  one  team.  The  sets  of  teams  and  groups  act  as  support 
clusters  for  each  student.  In  recent  semesters,  the  four  by  four  matrix  of  teams  and  groups  has  been  applied. 
The  teams  and  groups  for  the  first  part  of  the  semester  are  four  member  each.  When  the  major  design  project 
begins  the  teams  of  four  are  divided  into  teams  of  two.  This  affords  the  opportunity  to  modify  teams  to  improve 
interpersonal  relationships  among  the  members.  The  team  organization  depends  heavily  on  the  mix  of  stu- 
dents. The  current  mix  is  two  architecture  students  working  with  one  engineering  students.  Each  semester 
slight  variations  on  the  model  have  to  be  developed  to  accommodate  enrollment. 

The  Learning  Groups 

In  theory,  the  learning  groups  should  be  divided  among  the  three  disciplines  involved  in  the  project:  architec- 
tural design,  structural  design,  and  mechanical  design.  Each  learning  group  will  be  led  by  an  instructor  who 
will  develop  the  educational  modules  associated  with  that  discipline.  The  members  of  each  learning  group  will 
be  responsible  for  learning  the  software  packages  associated  with  specific  segments  of  the  problem  and  also  be 
encouraged  to  share  their  knowledge.  As  the  experiment  evolves,  different  uses  of  learning  groups  are  devel- 
oped. For  example,  last  semester  project  research  assignments  were  made  by  the  learning  groups;  this  semester 
software  instruction  is  carried  on  in  learning  groups. 

The  Design  Teams 

The  design  teams  develop  the  building  design  as  a whole.  During  their  sessions  with  the  design  instructor  they 
work  together  on  the  development  of  the  project,  each  member  looking  at,  and  engaging,  the  design  from  the 
perspective  of  their  learning  group  and/or  discipline.  Architecture  students  and  engineering  students  work 
together  on  the  design,  each  naturally  contributing  from  the  point  of  view  of  their  background.  While  it  is  in- 
tended that  contributions  to  the  design  are  made  across  disciplines,  there  has  been  a tendency  for  the  engi- 
neering students  to  wait  for  the  architecture  students  to  finalize  their  designs  before  contributing  their  engi- 
neering expertise.  Changing  this  attitude  toward  design  is  central  to  the  success  of  the  project.  This  year  we 
have  separated  the  engineering  students  by  semester  (structural  engineering  in  the  fall;  mechanical  engineer- 
ing in  the  spring.) 

The  Simulation,  Animation  and  Modeling  Laboratory  (SAML) 

The  studio  is  held  in  an  advanced  graphics  computer  laboratory  with  video  equipment  available  for  various 
levels  of  production.  The  laboratory  has  been  developed  with  support  from  NJIT  and  the  National  Science 
Foundation  (NSF)  (Jackson,  97a,  97b).  The  computers  are  networked  with  adjacent  computer  laboratories  and 
the  Internet.  The  laboratory  is  also  adjacent  to  the  Multimedia  Internet  Delivery  and  Production  Studio 
(MIDPS)  which  is  part  of  the  New  Jersey  Center  for  Multimedia  Research  (NJCMR).  This  provides  the  faculty 
with  a set  of  authoring  tools,  and  other  multimedia  production  equipment  which  aids  in  the  development  of  the 
courseware.  Combined  with  our  existing  video  equipment,  MIDPS  augments  our  facilities  to  provide 
webcasting. 
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The  Four  Modes  of  Representation 

Although  the  studio  is  designed  to  be  in  a multimedia  environment,  an  important  idea  is  to  dampen  the  use  of 
the  computer  as  the  central  tool.  The  computer  is  considered  on  equal  footing  with  all  the  tools  in  the  toolbox. 
This  is  accomplished  directly  through  assignments  which  emphasize  the  four  modes  of  representing  architec- 
ture. The  notion  of  multiple  representations  of  information  is  central  to  the  course  and  is  discussed  thoroughly 
with  the  students  at  the  beginning  of  the  semester.  Narrative  descriptions  of  initial  and  final  concepts  are  re- 
quired, as  are  short  written  reports  on  field  trips  and  research  issues.  Building  class  site  models  and  chipboard 
study  models  in  the  traditional  way — especially  as  a comparison  to  the  3D  computer  models — ^is  emphasized. 
Students  learn  to  study  their  work  simultaneously  via  the  computer  models  and  via  physical  models  and 
sketches  to  augment  their  visual  perception  and  other  cognitive  skills.  Research  suggests  that  students  relate  to 
what  they  see  on  the  screen  better  if  they  can  relate  it  to  previous  experiences.  (Roberts,  et  al)  Our  experience 
suggests  that  the  shifting  from  one  mode  of  representation  to  another,  while  difficult  for  some,  offers  the  op- 
portunity to  broaden  the  perceptual  understanding  of  the  design  problem  and  its  solutions.  Modal  shifting  also 
prevents  the  students  from  getting  mentally  fatigued  by  working  on  the  project  from  one  point  of  view.  The 
most  difficult  shift  occurs  between  the  design  mode  and  the  analytical  mode. 

Final  student  projects  are  presented  in  two  ways.  The  first,  for  an  oral  presentation  of  the  project  before  a 
group  of  critics,  designs  are  presented  on  traditional  ‘boards’  which  allows  computer  models  to  be  rendered, 
printed,  and  thus  communicated  traditionally.  This  also  gives  the  students  the  opportunity  to  mix  their  media, 
drawing  on  strengths  and  skills  they  may  have  previously  acquired.  The  second  mode  of  presentation  is  on  the 
Internet  both  as  digital  images  and  as  VRML  models.  To  link  these  two  methods  we  have  experimented  with 
webcasting  the  final  review,  both  as  a means  of  organizing  the  work  and  disseminating  innovative  methods. 
The  first  webcast  was  merely  a window  into  the  studio.  Currently  we  are  more  prepared  to  modify  our  methods 
of  digital  presentation  to  meet  the  challenges  of  a new  media. 

The  Results 

The  studio  is  presently  being  supported  through  funding  from  the  National  Science  Foundation  as  An  Interdis- 
ciplinary Virtual  Laboratory  for  Engineering  and  Architecture.  The  studio  was  offered  in  the  academic  year  of 
1996  and  the  spring  of  1997  as  a pilot  effort,  and  is  operating  currently  as  a “proof  of  concept”.  The  following 
items  have  been  implemented:  1)  collaborative  learning  and  design,  2)  computer  mediated  environment,  and 
3)  interdisciplinary  teaching,  coursework  and  design.  Deep  within  this  seemingly  complex  learning  environ- 
ment outlined  are  elements  of  the  traditional  design  studio.  The  semester  is  divided  into  an  analysis  phase  and 
a design  phase  with  a few  short  research  assignments  interspersed.  The  goal  is  that  the  whole  experience  can 
be  compressed  into  one  semester.  The  current  two  semester  sequence  is  designed  to  emphasize  structural  issues 
in  the  fall  semester  and  building  performance  issues  in  the  spring.  Development  of  a preliminary  syllabi 
quickly  dispatched  the  idea  that  such  a holistic  approach  could  be  accomplished  in  one  semester  until  some 
experience  has  been  gained  in  the  development  of  the  delivery  system.  However,  the  organization  of  the  three 
academic  units  involved  suggests  implementing  the  course  in  one  semester.  The  difficulty  of  a one  semester 
course  is  that  there  is  too  much  to  learn  and  apply  in  one  semester.  Learning  and  application  can  only  take 
place  if  the  students  have  the  proper  preparation  prior  to  enrolling  in  the  course.  Effecting  one  element  of  the 
curriculum  is  one  thing;  creating  a downward  ripple  effect  is  another  matter. 

Collaborative  learning 

From  the  faculty  point  of  view  dividing  the  class  into  teams  has  two  major  effects:  1)  on  the  downside,  it  in- 
creases management  time  to  continually  monitor  the  interpersonal  problems  which  occur  as  people  adjust  to 
working  together;  2)  on  the  upside,  it  reduces  the  number  of  critiques  and  increases  the  time  available  for  each 
critique.  Moreover,  it  increases  the  students’  effective  working  time  because  they  need  to  manage  each  other 
and  spend  time  in  the  studio  discussing  the  problem  and  working  together.  In  general,  students  are  very  favor- 
able toward  collaboration.  Exit  interviews  and  course  evaluations  indicate  a strong  interest  in  learning  to  work 
together. 
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The  computer-mediated  environment 

The  multimedia  material  is  currently  under  development.  Our  project  had  access  to  adjacent  mutlimedia  proj- 
ects and  have  benefited  from  that  synergy. (Bengu  95;  Bengu  & Swart  97)  The  computer-mediated  environ- 
ment works  as  follows:  Course  material  created  on  the  word  processor  is  converted  to  HTML,  edited  to  add 
dynamic  elements,  stored  on  the  server,  accessed,  displayed  and  disseminated  through  the  browser.  This  pro- 
vides the  student  and  others  continual  access  to  all  of  the  course  material  as  it  is  posted  and  modified.  Dis- 
semination of  course  materials  becomes  seamless.  Remote  access  and  email  allow  the  team  members  to  work 
at  a distance.  The  NJIT  library  catalog  is  also  available  at  the  workstation  so  that  references  to  books  and  arti- 
cles can  be  located  immediately  by  the  student. 

We  have  found  that  it  is  imperative  that  faculty  take  a hands  on  approach  to  imparting  computer  skills.  Com- 
ments from  the  students  indicate  they  are  more  confident  in  the  instructors  who  are  hands  on  and  display  their 
skills  directly  rather  than  those  who  are  more  aloof  and  directive  in  their  approach.  The  experience  of  the  last 
few  semesters  suggests  that  teaching  small  groups  of  students  detailed  methods  of  access  to  the  computer  sys- 
tems diffuse  information  among  the  students  more  rapidly  than  imparting  the  information  to  the  whole  class  in 
a formal  setting.  The  students  learn  more  quickly  in  informal  settings  in  which  they  are  able  to  communicate 
directly  with  their  peers.  Research  by  Roberts  et  al  also  suggest  that  of  the  critical  variables  for  productive 
learning,  “the  most  important  is  the  faculty’s  pedagogical  style  in  their  direct  teaching  and  the  student  interest 
in  the  subject.  The  faculty  must  be  sensitive  to  both  the  need  to  empower  students  exploration  by  providing 
them  with  the  skills  they  need  to  explore...  through  direct  teaching  as  well  as  allowing  and  encouraging  stu- 
dents to  do  their  own  exploring.”  (Roberts  et  al)  This  requires  continual  observation  of  the  skill  acquisition  of 
the  students. 

Problems 

The  development  of  Total  Studio  is  an  evolving  effort.  The  imprimatur  of  NSF  funding  has  been  a great  aid 
to  fostering  collaboration  among  the  academic  units.  A review  of  the  four  semesters  of  experience  have  yielded 
the  following  findings:  1)  students  are  slow  to  conceptualize  the  multitasking  potential  of  their  workstations, 
and  therefore  under  utilize  the  potential  of  the  UNIX  environment,  2)  there  are  only  a finite  number  of  teach- 
ing hours  in  a studio  and  having  to  spend  time  teaching  software  subtracts  from  the  time  used  for  teaching 
principles  of  design,  and  giving  individual  design  criticism,  and  3)  engineering  students  and  architecture  stu- 
dents organize  their  work  and  their  thought  processes  in  an  entirely  different  manner  which  will  require  fur- 
ther analysis  and  adjustment  on  the  part  of  the  faculty. 

While  working  on  teams  is  generally  not  part  of  the  studio  culture,  most  teams  have  operated  successfully  in 
recent  semesters.  The  reason  for  lack  of  success  in  teamwork  seems  to  be  based  on  the  conflict  in  personality, 
differences  in  work  ethic  and  habits,  the  lack  of  experience  in  working  on  teams,  and  a mismatch  in  skill  lev- 
els between  team  members.  In  anonymous  evaluations,  the  students  commented  that  “I  liked  the  team  thing  as 
an  idea.  It  didn’t  work  out  for  me  though  because  I had  a hard  time  with  my  partners.  I do  think  in  the  future, 
team  design  is  a good  idea.”  Or  “Groups  of  peopled  helped  but  caused  many  disagreements  which  slowed 
down  progress.  Although  the  thought  of  putting  people  together  was  a good  one.”  The  extended  history  of  the 
studio  as  a collaborative  one  aids  in  the  development  of  teamwork  because  it  generally  excludes  students  who 
might  undermine  the  concept. 

Conclusion 

The  metaphor,  changing  the  culture  of  the  curriculum,  may  seem  slightly  pedantic;  however,  methods  of 
teaching  have  been  passed  from  generation  to  generation  in  the  most  unquestioning  manner.  New  technolo- 
gies, as  precursors  of  changing  methods,  are  often  resisted.  The  efforts  to  change,  sometimes  abetted  by  the 
layers  of  accrediting  organizations,  are  held  back  by  the  ‘viscosity’  in  the  organization.  Creating  a change  that 
evolves,  rather  than  changes  abruptly,  gives  the  results  an  opportunity  to  become  integral  to  the  culture  and 
reaffirmed  by  tradition.  Our  team  of  faculty  expect  criticism  from  our  colleagues,  from  professional  organiza- 
tions with  long  standing  paradigms  and  vocabularies  holding  antithetical  views,  and  even  from  the  students. 
We  are  abetted  by  the  support  of  outside  funding,  and  the  interest  of  professional  organizations,  both  of  which 
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add  an  imprimatur  to  our  efforts.  At  the  core  of  our  activity  is  the  goal  of  helping  to  “lead  the  professional  to  a 
future  of  greater  relevance  and  responsibility.”  (Mitgang  97  p.  124) 

References 

Black,  G.R.  and  Duff,  S.  (1994)  “A  Model  for  Teaching  Structures:  Finite  Element  Analysis  in  Architectural 
Education”,  Journal  of  Architectural  Education^  Vol.  48,  No.  1,  September. 

Bengu,  G.  (1995)‘Tnteractive  Multimedia  Course  on  Manufacturing  Processes  and  Systems”,  International 
Journal  of  Engineering  Education(\iEE),  Vol.  1 1,  No.  1. 

Bengu,  G.  and  Swart,  W.  (1997)  “Interactive  Multimedia  Courseware  over  Internet”,  IEEE  Transactions,  Spe- 
cial Issue  on  Education. 

Feisel,  L.D.,  (1994)  “Engineering  Curricula:  Beginning  a New  Era”,  Project  Impact:  Disseminating  Innova- 
tion in  Undergraduate  Education,  Conference  Proceedings,  NSF. 

Greenwood,  J.B.,  Jalalzadeh,  A. A.,  and  Wagner,  G.K.  (1997)  “Aesthetics  and  Technology:  Cultivating  a 
Common  Ground”,  Architecture:  Material  and  Imagined^  Proceedings  of  the  85  th  ACSA  Annual  Meeting, 
Dallas,  TX,  March. 

Jackson,  B.  (1997)  “The  Development  of  TOTAL  Studio:  A Comprehensive  Computer-Mediated  Studio  Envi- 
ronment for  Architecture  and  Engineering”,  Architecture:  Material  and  Imagined^  Proceedings  of  the  85  th 
ACSA  Annual  Meeting,  Dallas,  TX,  March. 

Jackson,  B.  (1997)  “The  Evolution  of  TOTAL  Studio:  A Comprehensive  Digital  Studio  Environment  for  Ar- 
chitecture and  Engineering”,  Reflections  on  Heritage  and  Modernity,  Proceedings  of  the  Northeast  Regional 
Meeting  of  ACSA,  Newport,  October. 

Kilker,  J.  and  Gay,  G.  (1994)  “Information  Seeking  by  Novices  in  a Collaborative  Multimedia  environment”, 
IMG  Working  Paper  94-1,  The  Interactive  Multimedia  Group,  Cornell  University. 

Landry,  S.  (1994)“The  Nature  of  Efforts  to  Effect  Interdisciplinary  Reform:  Drivers,  Barriers,  and  Strategies 
for  Dissemination,”  Project  Impact:  Disseminating  Innovation  in  Undergraduate  Education,  Conference  Pro- 
ceedings, NSF. 

Mitgang,  Lee.  D.  (1997)  “Saving  the  Soul  of  Architectural  Education”,  Architectural  Record,  May. 

Roberts,  N.,  Blakeslee,  G.  and  Barowy,  W.  “The  Dynamics  of  Learning  in  a Computer  Simulation  Environ- 
ment”, Journal  of  Science  Teacher  Education,  Vol.  7,  Kluwer  Academic  Publishers. 

Turoff,  M.  (1995)“Designing  a Virtual  Classroom™”,  1995  International  Conference  on  Computer  Assisted 
Instniction  ICCAI  ‘95,  Taiwan,  March. 

Acknowledgments 

This  paper  and  the  laboratory  are  partially  supported  by  the  National  Science  Foundation,  Grant  #DUE-9650748,  Devel- 
opment of  Interdisciplinary  Courses  and  Laboratory  Facilities  and  Grant  #DUE  9752459,  An  Interdisciplinary  Virtual 
Laboratory  for  Engineering  and  Architecture.  My  collaborators  in  the  project  are  Professors  Edward  Dauenheimer  and 
Erv  Bales,  both  of  New  Jersey  Institute  of  Technology.  Additional  support  has  been  received  from  Professor  Golgen 
Bengu,  Alfred  Greenberg,  and  the  staff  of  Engineering  Computing  at  NJIT.  Programming  assistance  on  the  website,  which 
can  be  observed  at  http://www-ec.njit.edu/ec_info/imagel/text_files/hp_la.html,  is  from  graduate  students  Abhijeet 
Shinde  and  Gamal  Ghabour.  Incidentally,  the  website  is  a dynamic  place,  continually  changing  as  courseware  and  student 
work  is  added  and  modified. 


730 


Architectural  Aspects  of  an  Interactive  Multimedia  Environment  for 
Teaching  Chemistry  in  Secondary  Education 


Abstract:  In  this  paper  we  present  a multimedia-based  system  that  facilitates  teaching 
chemistry  to  pupils  of  Hellenic  (Greek)  high  schools  over  a LAN.  The  system  uses  a variety 
of  multimedia  objects.  An  important  aspect  of  the  system  is  the  central  control  the  teacher  can 
have  over  some  aspects  of  a lesson.  So,  the  teacher  can  enable  the  section  to  be  taught  and 
disable  the  rest  ones,  and  design  and  deliver  on-line  tests  to  the  pupils.  The  pupils  are  then 
able  to  submit  their  answers  to  a central  database.  This  is  achieved  by  using  dynamic 
hyperlinking  implemented  as  remote  SQL  queries  over  a LAN. 


1.  Introduction 

Over  the  past  few  years,  new  technologies  have  been  used  in  producing  educational  tools.  The  use  of  multimedia 
technology  has  given  a great  impetus  to  the  training  evolution  in  education.  Multimedia  systems  provide  an 
integrated  environment  for  the  creation,  storage  and  presentation  of  a variety  of  media  types  (e.g.  text,  graphics, 
video  and  audio)  enabling  the  development  of  a great  number  of  successful  prototype  educational  applications 
(Hopper  1990),  (Fox  1991),  (Narasimhalu  & Christodoulakis  1991),  (Faconti  1994). 

In  most  cases,  information  technology  is  concerned  with  heterogeneous  data  both  static  and  dynamic. 
Existing  LAN  (Local  Area  Network)  development  tools  support  efficient  solutions  for  the  integration  of 
multimedia  interfaces  (static  information)  and  database  systems  (mainly  dynamic  information)  with  impressive 
results  (Narasimhalu  & Christodoulakis  1991). 

Today  in  Hellas  (Greece),  computer  technology  (e.g.  multimedia  systems)  is  not  extensively  used  for 
educational  purposes  in  subjects  other  than  informatics  in  Secondary  Education.  When  used,  it  is  usually  done  on 
stand  alone  computer  systems.  However,  due  to  recent  introduction  of  LANs  in  computer  labs  in  Hellenic  high 
schools  (Garofalakis  et  al.  1998),  the  need  for  educational  software  that  exploits  LAN  capabilities  has  been 
arisen. 

This  paper  presents  the  architecture  of  an  interactive  multimedia-based  environment  to  be  used  as  an  assistant 
for  teaching  chemistry  in  a classroom  over  a LAN.  It  incorporates  a multimedia  application,  for  manipulating 
static  information,  and  a relational  database,  dealing  with  dynamic  information.  These  two  components 
intercommunicate  via  an  application  server  over  a LAN,  The  end-users  (teacher  and  pupils)  can  access 
corresponding  available  information  (static  and  dynamic).  The  static  information  resides  in  the  multimedia 
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application  and  is  to  be  used  during  a chemistry  course.  The  dynamic  information  is  related  to  the  tests  designed 
and  delivered  by  the  teacher  and  the  answers  given  by  the  pupils  and  resides  in  the  database. 

The  paper  is  organised  as  follows.  Section  2 presents  the  abstract  architecture  and  design  aspects  of  the 
system.  Its  functionality  and  interaction  with  the  users  are  discussed  in  section  3.  Section  4 deals  with  the 
implementation  aspects  of  the  system,  while  the  last  section,  Section  5,  presents  the  conclusions  drawn  and 
directions  for  further  work. 

2.  System  Description 

The  system  is  composed  of  three  basic  components  (see  Fig,  1),  the  multimedia  application,  the  application 
server  and  the  relational  database.  The  multimedia  application  (front-end)  interacts  with  the  human  users  and 
the  administrator  of  the  system.  The  application  server  acts  as  an  intermediate  between  the  multimedia 
application  and  the  database  by  conveying  queries  from  the  former  to  the  latter  over  a LAN,  The  database  (back- 
end) accepts  data  from  various  sources  (mainly  dynamic  information),  processes  it,  stores  the  results  and  pipes  it 
to  the  front-end,  when  requested. 

The  system  topology  is  illustrated  in  Fig.  2,  where  the  front-end  and  the  back-end  of  the  system  are  indicated. 
The  database  resides  on  the  central  PC  of  the  LAN  (server),  whereas  a copy  of  the  multimedia  application  and 
the  application  server  exists  on  each  of  the  rest  PCs  (clients). 


Multimedia 

Application 


Figure  1 : The  architecture  of  the  system 


2.1  Multimedia  application 

The  static  information  provided  by  the  system  is  included  in  the  multimedia  application  and  presented  in  a 
variety  of  ways,  such  as  hypertext,  special  sounds,  appropriate  images  and  animations,  hypermedia  object, 
educational  games  or  quizzes,  video  recordings  of  real  chemical  experiments  and  interactive  simulated  chemical 
experiments.  Every  screen  follows  one  of  some  predefined  presentation  screen  patterns.  Common  to  all,  on  the 
top  area  of  the  screen  the  logo  of  the  application,  at  its  left,  and  the  title  of  the  specific  section  or  subsection,  at 
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its  right,  lies.  Also,  at  the  bottom  of  the  screen,  there  are  several  buttons  (or  options)  facilitating  the  interaction 
between  the  user  and  the  application.  The  main  content  is  displayed  at  the  central  area  of  the  screen,  the  active 
screen  (see  Fig. 3). 
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Figure  3:  A screen  for  topic  selection  in  a subsection 
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Navigation  scheme  is  a mixture  of  the  hierarchical  and  linear  models.  The  content  is  distributed  in  sections, 
subsections  and  topics.  Its  topic  comprises  a series  of  screens  containing  information  in  any  combination  of  the 
above  mentioned  ways  of  presentation.  The  user  determines  the  presentation  flow  in  a degree,  by  using  hot  area 
selections  as  well  as  buttons.  For  example,  in  Fig.  3 there  are  three  hot  (selection)  areas,  one  for  each  of  the  three 
topics.  On-line  help  is  available,  usually  via  messages  at  the  bottom  of  the  active  screen,  where  necessary,  thus 
making  use  of  the  system  as  simple  as  possible  to  even  a novice  user. 

There  are  different  components  of  the  interface  of  the  system  (front-end),  related  to  the  administrator  and  the 
end-users  (teacher  and  pupils)  of  the  system.  These  components  are  related  to  the  user  views,  described  in 
section  3.  The  interface  of  the  multimedia  application  allows  use  of  both  the  static  and  the  dynamic  information. 
Access  to  dynamic  information  is  achieved  via  dynamic  hyperlinking.  That  is,  links  to  objects  are  defined  not  as 
local  references  but  as  remote  queries  to  the  database.  This  means  that  some  hyperlinks  are  established 
dynamically,  at  run-time,  assuming  that  the  information  is  structured  appropriately  in  the  database.  This  reduces 
the  cost  of  static  hyperlinking  and  results  in  ease  of  tailoring,  since  it  permits  data/schema  update  and  evolution 
(needed  for  on-line  applications)  via  the  database.  In  addition,  it  makes  the  presentation  stable,  since  the 
hyperlinked  objects  are  subject  to  the  integrity  control  of  the  database. 

Dynamic  information  concerns  a)  activation  of  the  section  to  be  taught  and  deactivation  of  the  rest  ones,  b) 
creation,  modification  and  selection  of  multiple-choice  type  questions  for  each  section  to  form  a test,  c)  storage 
of  the  answers  of  the  pupils  to  the  test  and  d)  access  authorisation  checking. 

2.2  Application  Server 

The  Application  Server  comprises  the  ODBC  Driver  Manager,  the  SQL  components  and  the  ODBC  drivers. 
SQL  queries  to  the  database  are  directed  via  the  ODBC  Driver  Manager  to  the  corresponding  ODBC  driver  and 
then  to  the  database.  ODBC  Driver  Manager  is  necessary  in  case  we  have  more  than  one  ODBC  driver  for  each 
of  the  administrator,  the  teacher  and  the  pupil.  This  happens  when  we  have  more  than  one  database  or  database 
partition.  In  case  we  have  a single  database,  ODBC  manager  degenerates  to  direct  connections  of  the  ODBC 
drivers  to  the  corresponding  SQL  componenents. 

The  three  components  of  the  SQL  server,  that  are  related  to  the  user  views  (section  3),  are  as  follows: 

• SQL  component  for  the  administrator: 

This  component  is  responsible  for  the  security  of  the  system,  since  it  enables  the  administrator  to  establish 
passwords  for  the  pupils  and  the  teacher.  The  passwords  are  stored  in  the  database.  It  also  deals  with 
database  updates. 

• SQL  component  for  the  teacher: 

This  component  allows  the  teacher  to  take  control  over  some  aspects  of  a lesson.  The  teacher  can  enable  the 
section  to  be  taught  and  disable  the  rest  ones.  Also,  it  allows  the  teacher  to  handle  the  set  of  the  test 
questions.  He/she  can  insert  new  questions,  modify  existing  ones,  delete  old  ones  and  select  a subset  of  them 
to  set  up  a particular  test. 

• SQL  component  for  the  pupils: 

This  component  is  responsible  for  checking  pupil  identification,  by  examining  their  particulars,  like  name 
and  password.  Also,  it  has  the  task  of  on-line  delivering  the  test  designed  by  the  teacher.  Finally,  it  is 
responsible  for  recording  the  answers  of  the  pupils  in  the  database. 

It  is  the  last  two  components  where  dynamic  hyperlinking  is  mainly  used. 

3.  User  Views 

We  can  distinguish  four  different  ways  information  can  be  viewed,  that  is  four  basic  views,  as  far  as  the  users  are 
concerned: 

• General  view:  In  the  general  view,  the  user  can  access  all  available  chemical  information  by  navigating 
through  the  sections,  subsections  and  the  topics  of  the  multimedia  application.  For  this  purpose,  the  screen  is 
divided  into  hot  spots  (specific  regions)  for  every  section  or  subsection  and  the  user  can  select  a specific 
region  and  access  the  related  information  (Fig.  3). 

• Administrator  view:  In  the  administrator  view,  the  user  can  establish  passwords  for  the  users  of  the  system 
and  update  information  in  the  database. 
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• Teacher  view:  In  the  teacher  view,  the  user  is  able  to  specify  the  content  of  a lesson,  design  and  insert  new 
questions  in  the  question  set,  select  a subset  of  the  existing  questions  and  deliver  it  to  the  pupils  and  view 
their  answers. 

• Pupil  view:  In  the  pupil  view,  the  user  can  identify  himself/herself  to  the  system  and  answer  to  the  test 
questions  that  have  been  delivered  by  the  teacher. 

The  administrator  of  the  system  can  use  only  the  administrator  view.  The  teacher  can  use  the  teacher  view, 
the  general  view  and  the  pupil  view,  whereas  the  pupil  can  use  the  general  view  and  the  pupil  view.  A third 
person  can  use  only  the  general  view. 

4.  Implementation  aspects 

The  system,  in  the  form  of  an  advanced  prototype,  has  been  developed  under  the  Windows  95  operating  system 
and  has  been  tested  on  a variety  of  workstations.  The  multimedia  application  was  developed  using  the 
Macromedia  Authorware  and  Macromedia  Afterburner  authoring  tools  that  use  icon-based  scripting.  Some 
additional  code  was  written  using  Visual  Basic. 

The  database  application  was  developed  using  the  Access  RDBMS.  The  image  format  is  BMP  while  the 
videos  are  formatted  as  AVI  and  compressed  using  the  Intel  Indeo(R)  Video  R3.2(32)  encoder.  The  animations 
are  formatted  as  AVI  files  as  well.  Finally,  any  sounds  were  digitised  and  compressed  into  WAV  files. 

The  application  that  handles  the  communication  between  the  user  interface  and  the  database  comprises  a 
number  of  SQL  components.  In  particular  we  used  the  ODBC  functions  (ODBCOpen,  ODBCClose  and 
ODBCExecute)  that  Authorware  offers  for  opening  a database,  closing  a database  and  executing  insertions, 
deletions  and  updates  respectively.  These  functions  are  called  via  calculated  icons.  Dynamic  hyperlinking  is 
implemented  by  connecting  Authorware  parameters  to  database  items. 

5.  Conclusions  and  further  work 

In  this  paper  we  present  the  architectural  aspects  of  a multimedia-based  system  that  facilitates  teaching 
chemistry  over  a LAN  to  high  school  pupils.  It  enables  a high  level  computer-based  teacher-pupil  interaction 
using  multimedia  and  hypermedia  technology. 

An  important  feature  of  our  system  is  the  use  of  dynamic  hyperlinking,  which  facilitates  on-line  control  in 
some  aspects  of  a lesson  by  the  teacher.  The  teacher  can  disable  all  irrelevant  sections,  so  that  the  pupils  cannot 
“play  around”  the  application,  but  concentrate  on  the  specific  section.  Also,  the  teacher  can  design  new 
questions,  review  or  delete  old  ones  and  select  which  questions  will  form  a test  for  a certain  lesson.  All  these  are 
done  from  a unified  user  interface,  offering  different  views  to  different  users. 

We  intend  to  improve  our  system  in  two  directions.  One  concerns  extension  of  the  flexibility  of  the  teacher, 
in  making  changes  to  the  content  of  the  application  other  than  to  questions,  such  as  to  text,  images,  videos  etc,  in 
a predefined  way.  To  this  end,  object-oriented  database  and  authoring  technologies  may  be  used  (Thuraishngham 
1992).  The  other  concerns  use  of  the  application  over  the  Internet,  additionally  equipped  with  teleconference 
capabilities  to  provide  a kind  of  a virtual  high  school  class  (see  e.g.  Hiltz  & Wellman  1997).  To  this  end,  a 
number  of  technical  problems  should  be  solved  (Wilson  1994). 
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Abstract:  Educational  hypermedia  authoring  process  remains  as  a critical  point.  This  kind 
of  application  can  assume  different  formats  depending  on  the  chosen  learning  focus.  To 
build  a system  of  patterns,  we  analyzed  contemporary  theories  and  drew  two  hypermedia 
categories:  exploratory  discovery  and  guided  navigation.  We  identified  the  main  modeling 
problems  related  to  the  formats  and  write  corresponding  solution,  using  design  patterns 
concepts.  We  are  finishing  and  refining  the  patterns  and  describing  others  to  compose  a 
system  of  patterns  for  educational  hypermedia  modeling. 


Building  hypermedia-based  educational  applications  is  not  easy.  Besides  a multidisciplinary  team,  an 
educational  application  must  reflect  the  chosen  educational  goals  and  the  learning  environment.  Learning 
environment  must  be  specified  a priori  in  educational  software  life  cycle,  and  it  mirrors  an  educational 
philosophy  and  the  elected  learning  theory.  However,  expert  educators  and  hypermedia  designers  never  build 
their  new  applications  from  scratch.  They  usually  reuse  successful  ideas  they  have  used  in  the  past.  In 
traditional  model  of  education,  pedagogical  strategies  and  goals  are  imposed  by  designers.  Now, 
Constructivism  is  in  the  front  end  of  the  education,  and  designers  can  not  impose  a specific  learning  model 
but  allow  students  to  construct  their  own  knowledge.  While  traditional  instructional  design  is  based  on  a 
linear  and  hierarchical  relationship,  new  instructional  design  aims  to  allow  students  to  select  and  develop  their 
own  strategies,  searching  for  new  knowledge  domains.  Our  research  is  to  develop  patterns  for  modeling 
educational  hypermedia  applications,  considering  that  instructional  technologies  are  now  centered  in  the 
process,  looking  forward  for  interactive  learning  environments  creation.  Intended  goal  is  to  obtain  a 
satisfactory  formal  level,  so  it  can  be  reused  by  teachers  and  hypermedia  designers. 

This  paper  presents  a set  of  navigational  patterns  for  helping  educational  hypermedia  designers.  The  work  is 
organized  as  following:  section  2 briefly  discusses  learning  theoretical  foundations  to  construct  educational 
hypermedia  scenarios;  section  3 shows  how  to  minimize  problems  met  in  developing  educational  application, 
using  concepts  of  design  patterns.  Conclusions  and  future  works  are  pointed  out  in  the  last  section. 


In  the  nineties,  changes  in  education  focus  can  be  pointed  as  a remarkable  fact.  The  challenge,  today,  is  to 
design  computational  environment  to  support  the  knowledge  construction,  instead  of  continuing  to  develop 
software  based  on  contents  memorization  and  repeatness.  The  breaking  up  with  overshot  issues  has  brought 
out  important  impacts  on  educational  software.  New  instructional  design  encapsulates  hypermedia  features,  as 
the  non  linear  format  and  the  free  choice  of  navigational  paths. 

Moreover,  educational  hypermedia  can  adopt  different  pedagogical  formats,  such  as  impromptu  discovery, 
free  exploratory  discovery,  guided  navigation  and  navigation  over  enchained  sequences  of  information.  To 
support  these  formats,  learning  and  methodological  guidelines  must  be  available  to  help  designers’  work. 


^ This  paper  has  been  partially  funded  by  CYTED-  Sub-  project  VII  - SHMI 


1.  Introduction 


2.  Learning  Theoretical  Foundations 


Learning  theories  reflect  dichotomous  visions  of  how  the  learning  process  happens.  On  one  extreme,  there  are 
the  closed  approaches,  such  as  the  Behaviorism  viewpoint  and,  on  the  other  extreme,  there  are  the  opened 
approaches,  such  as  the  Socio-Interactionism.  Between  them,  we  can  outline  Gagne’s  Neo-Behaviorism  and 
Piaget  and  Bruner’s  Constructivism. 

Behaviorists  consider  individuals  a passive  organism,  governed  by  external  environment  stimulus.  Learning 
is  a permanent  change  in  a specific  behavior.  Behavior  is  only  what  can  be  externally  observed  and  measured, 
and  it  only  changes  facing  reinforce  contingencies. 

Neo-Behaviorist  theory  also  considers  the  learning  an  external  behavior,  which  can  be  observed  and  generate 
permanent  changes.  Moreover,  in  the  theory  scope,  mind  internal  functioning  is  considered.  Gagne  is  the  most 
representative  theorist  of  Neo-Behaviorist  school  and  to  him  learning  is  a behavior  change,  followed  by 
changing  permanency.  Learning  can  be  activated  by  a large  range  of  stimulation  from  individuals  interaction 
with  external  environment.  The  stimulation  is  considered  the  learning  process’s  input,  able  to  produce 
behavior  changes,  observable  as  human  performance  - the  output.  Intellectual  skill  complies  with  a 
hierarchical  sequence:  from  simple  chains,  concepts  and  rules  to  problem  solving.  Simples  intellectual  skills 
can  be  composed  of  simplest  skills,  that  are  their  pre-requirements,  resulting  in  a skills  frame,  called  learning 
hierarchy. 

Constructivist  school  considers  that  learning  process  is  a (re)construction  inside  individuals,  through  external 
environment  interactions.  The  student  is  the  active  agent  in  the  learning  process,  experimenting,  researching, 
questioning  and  developing  the  reasoning.  Teacher  must  play,  at  same  time,  the  role  of  the  learning  facilitator, 
moderator,  promoter  and  challenger  of  new  experiences  and  must  be  able  to  propose  strategies  and  ways  that 
allow  students  to  search  and  find  answers.  Piaget  and  Bruner  are  important  ones  in  Constructivism  school.  An 
usual  criticism  to  Piagetian  Constructivism  is  referred  to  learning  as  individual  construction  viewpoint. 
Constructivism  seems  to  be  uninterested  with  cultural  and  social  conditionings.  Vygotsky  is  the  most 
representative  theorist  of  learning  as  a social  construction,  done  by  the  history  and  by  the  culture. 

Socio-Interactionism  theory  points  that  thoughts  are  gradually  built  on  a historical  and  social  environment. 
Social  interactions  play  fundamental  role  in  cognitive  development.  Development  always  happens  firstly  in 
social  setting,  among  people,  and  after  in  the  individual  setting,  inside  himself.  The  child  faces  three  cognitive 
development  stages,  extensible  to  all  learners:  real  development  level  - determined  by  individual  capacity  of 
solving  independently  problems;  potential  development  level  - determined  by  individual  capacity  of  solving 
problems,  under  supervision  of  more  capable  individual  or  in  cooperation  with  more  capable  peers;  and  zones 
of  proximal  development  - intermediate  development  stage,  situated  among  real  development  level  and 
potential  development  level.  Zones  of  proximal  development  can  be  improved  through  social  interactions, 
i.e.,  adult  help  or  peers  cooperation  can  develop  intellectual  skills. 

It  is  complex  to  draw  educational  events  strictly  based  on  theoretical  foundations,  but  it  much  more  complex 
to  develop  educational  software  under  these  foundations.  Applying  theoretical  basis  in  the  development  of 
traditional  educational  software  never  was  usual,  since  the  development  process  always  was  ad-hoc. 
Therefore,  there  are  a new  scenario  - the  diffusing  of  educational  hypermedia  applications  and,  more  recently, 
the  increasing  hypermedia  publishing  on  the  Web.  It  is  requiring  both  theoretical  and  methodological  supports 
to  all  development  phases,  mainly  to  earliest  ones.  Educational  software  designers  need  to  understand  the 
learning  process,  to  integrate,  since  the  beginning  of  the  development,  a consistent  view  of  how  and  under 
which  circumstances  people  learn. 

Behaviorism  and  Neo-Behaviorism  theories  are  concerned  to  facts  memorization,  through  the  absorption  of 
small  and  enchained  information  portions.  Constructivism  hypothesis  is  concerned  to  learning  by  doing  and 
by  acting  and  transforming  the  environment,  through  intellectual  challenges  and  puzzles.  Socio-Interactionism 
rationale  is  concerned  to  peers’  collaboration,  where  the  “more  capable  peer”  helps  the  other  peers  to 
overcome  current  knowledge  lacking.  Behaviorism,  Neo-Behaviorism  and  Constructivism  conceive  the 
learning  process  as  an  individual  process,  while  the  Socio-Interactionism  understands  it  as  a historical, 
temporal  and  collective  process.  Navigating  over  small  pieces  of  enchained  information  is  typically  an 
educational  hypermedia  application  based  on  the  Behaviorist  approach.  Guided  navigation  and  navigating  for 
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hierarchical  paths  can  be  seen  as  an  educational  hypermedia  application  based  on  the  Neo-Behaviorist 
approach.  The  impromptu  discovery,  through  the  free  navigation  on  the  Web,  the  incidental  learning,  and  the 
exploratory  discovery  are  typical  formats  of  educational  hypermedia  built  under  the  Constructivism  point-of- 
view.  Navigation  based  on  cooperative  work,  on  intense  peers’  collaboration  and  on  the  assumption  of 
learning  is  a social  and  shared  process  are  educational  formats  adopted  by  the  Socio-Interactionism  approach. 

Not  always  educational  hypermedia  applications  can  be  modeled  under  determined  theoretical  foundations. 
Piaget  and  Vygotsky  approaches  can  not  be  modeled  according  to  traditional  instructional  design  techniques. 
These  approaches  emphasize  opened  ways  of  navigating,  that  can  not  be  modeled.  When  we  try  to  model 
these  approaches,  we  often  make  them  closed,  contradicting  the  theoretical  principles.  On  the  other  hand, 
educational  hypermedia  based  on  Behaviorism  foundations  can  be  easily  modeled.  Moreover,  this  approach  is 
outdated,  proposing  poor  and  limited  instructional  events. 


3.  Design  Patterns  in  Educational  Hypermedia  Modeling 

A pattern  is  the  abstraction  from  a concrete  form  that  keeps  recurring  in  specific  non-arbitrary  contexts.,  the 
notion  of  a pattern  is  geared  toward  solving  problems  in  design.  A pattern  is  more  than  just  a battle-proven 
solution  to  a recurring  problem.  The  problem  occurs  within  a certain  context,  and  in  the  presence  of  numerous 
competing  concerns.  The  proposed  solution  involves  some  kind  of  structure  that  balances  these  concerns,  or 
"forces,"  in  the  manner  most  appropriate  for  the  given  context.  Using  the  pattern  form,  the  description  of  the 
solution  tries  to  capture  the  essential  insight  that  it  embodies,  so  that  others  may  learn  from  it,  and  make  use 
of  it  in  similar  situations.  The  pattern  is  also  given  a name,  which  serves  as  a conceptual  handle,  to  facilitate 
discussing  the  pattern  and  the  jewel  of  information  it  represents.  So  a definition  that  more  closely  reflects  its 
use  within  the  patterns'  community  is:  a pattern  is  a named  nugget  of  instructive  information  that  captures  the 
essential  structure  and  insight  of  a successful  family  of  proven  solutions  to  a recurring  problem  that  arises 
within  a certain  context  and  system  of  forces.  From  this  background,  we  describe  a set  of  patterns  related  to 
problems  met  in  educational  hypermedia. 

We  have  studied  the  authoring  process  of  educational  hypermedia  applications  under  two  learning  theories  - 
the  Bruner’s  Constructivism  and  the  Gagne’s  Neo-Behaviorism.  These  approaches  can  be  modeled,  because 
both  Bruner  and  Gagne  have  an  instructional  theories.  To  start  the  building  of  a patterns  system  we  firstly 
selected  two  educational  hypermedia  categories:  the  exploratory  discovery  based  on  Bruner  theory  and  the 
guided  navigation,  based  on  Gagne  assumptions.  Second,  we  identified  the  main  modeling  problems  related  to 
the  two  formats.  Then,  we  were  able  to  write  down  our  patterns. 

Independently  the  chosen  learning  theory,  a new  hypermedia  design  problem  arose  due  to  the  increasing 
publishing  in  the  Web:  the  Internet  functionality.  For  each  problem,  we  present  a pattern,  using  design 
patterns  concept.  The  patterns  are  composed  of  a name,  a brief  description,  the  problem,  the  related  forces, 
the  proposed  solution  and  the  known  uses.  Our  work  is  a work-in-progress  yet,  since  some  patterns  have  not 
achieved  the  necessary  formalism.  Moreover,  we  are  completing  and  refining  the  written  patterns,  aiming  to 
test  and  validate  them.  Other  related  patterns  will  be  described,  in  order  to  compose  a preliminary  pattern 
system  for  educational  hypermedia  modeling. 

At  the  moment,  we  wrote  six  patterns:  Avoiding  disorientation  and  Minimizing  cognitive  overhead, 
concerning  the  Exploratory  Discovery  Pedagogical  Approach,  Enriching  Navigational  Paths  and  Improving 
browsing,  concerning  the  Guided  Navigation  Pedagogical  Approach,  Minimizing  answer  time  and  Assuring 
access  to  informational  links,  concerning  the  Networking  Functionality. 

Navigation  in  Exploratory  Discovery  Pedagogical  Approach 

Navigation  in  exploratory  discovery  pedagogical  approach  provides  student  only  the  educational  goals  and  the 
navigational  spaces.  The  advantage  met  in  this  approach  often  becomes  a serious  problem,  since  student  can 
get  lost,  feeling  disoriented  and  presenting  cognitive  overhead.  As  solution,  we  propose  the  patterns 
“Avoiding  disorientation”  and  “Minimizing  cognitive  overhead” 
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A VOIDING  Disorient  A tion 

Description:  This  pattern  is  addressed  to  avoid  user’s  disorientation  in  educational  hypermedia  applications 
based  on  free  discovery. 

Problem:  How  can  we  avoid  that  the  user  get  lost  in  an  educational  hyperspace? 

Forces  - Hypermedia  users  get  lost  with  poorly  structured  information  landscapes  and  without  appropriate 
navigational  tools.  Incidental  learning  should  be  maintained,  since  it  is  an  important  and  fun  way  of  learning. 
Navigational  space  should . be  maintained,  but  the  student  can  improve  his/her  learning  environment. 
Navigational  design  should  allow  the  use  of  cognitive  strategies  related  to  solve  problems  and  take  decisions. 
Collaboration  and  communication  among  peers  are  important  educational  issues.  Social  knowledge 
construction  is  a relevant  point  for  the  new  world  social  organization 

Solution  - Incorporate  navigational  facilities  to  the  hypermedia  design  to  provide  the  accessing  to  relevant 
nodes  and  informational  contexts.  Use  graphic  spatial  representation  to  aid  the  user’s  orientation  and 
bookmarks,  annotations,  navigational  maps  (local  and  global)  or  conceptual  spaces  of  navigation,  searching 
by  meaningful  keywords  to  orient  the  browser.  Dispose  cooperative  virtual  spaces  and  browsing  tools. 
Consider  the  available  platform  during  the  design  phase.  If  the  design  is  for  the  Web,  frames,  navigational 
paths,  pathfinders,  HTML  form  for  annotations,  search  engines  are  practical  solution.  On  stand-alone 
platform,  navigational  paths,  no  ambiguous  interface  solution  and  annotation  individual  notecards  are 
effective  solutions. 

Kjiown  Uses  - Museums  - Le  Louvre  and  Library  - Internet  Public  Library  - http://  www.ipl.org/ref/ 

Related  patterns:  Minimizing  Cognitive  Overhead 

Minimizing  Cognitive  Overhead 

Description:  This  pattern  is  addressed  to  avoid  student’s  cognitive  overhead  while  browsing  in  information 


Problem:  How  can  we  design  an  educational  hypermedia  to  avoid  that  time  demands  becomes  high  and  the 
student  becomes  unmotivated,  fells  lost  and  the  educational  goals  fail? 

Forces  - Besides  the  challenge  of  solving  proposed  problems,  the  student  faces  the  navigational  design  when 
browsing  an  educational  hypermedia.  Cognitive  overhead  happens  when  the  student  does  not  feel  how  the 
informational  space  is  organized,  its  depth  and  extension.  Incidental  learning  should  be  maintained,  since  it  is 
an  important  way  of  learning  and  fun.  Exploratory  opportunities  should  be  preserved. 

Solution  - Plan  the  hypermedia  design  using  facilities  in  order  to  provide  access  to  relevant  marks.  Use 
pathfinders,  search  engines  or  mechanisms,  index,  embedded  menus,  bookmarks,  annotations,  navigational 
maps.  Index,  embedded  menus  and  accessing  keys  resolve  partially  cognitive  overhead  problems.  Key  points 
must  be  signed  by  special  marks. 

Known  Uses  - Museums  - Le  Louvre  and  Library  - Internet  Public  Library-  http://www.ipl.org/ref/ 

Related  patterns:  Avoiding  Disorientation 


In  guided  navigation,  instructional  contents  are  offered  in  little  portions,  though  embedded  menus.  Entry 
encyclopedia  and  text  book  are  the  metaphors  commonly  adopted.  Students  exceptionally  get  lost,  but  their 
navigational  spaces  are  very  restricted,  causing  lack  of  motivation.  As  solution,  we  propose  the  patterns 
“Enriching  navigational  paths”  and  “Improving  browsing”. 

Enriching  Na  viga  tional  Pa  ths 

Description:  This  pattern  is  addressed  to  enrich  navigational  paths  to  improve  the  learning  environment  and 
the  student  choices. 

Problems  - How  can  we  guarantee  that  powerful  navigational  paths  are  offered? 

Forces  - Very  restricted  information  landscapes  make  the  student  gathers  inappropriate  range  of  knowledge. 
Guided  navigation  guaranties  that  the  relevant  paths  will  be  browsed 

Tours,  tracks  and  pre-defined  paths  are  typical  design  decision  in  this  approach.  It  assures  user’s  orientation 
and  can  help  educational  goals  achievement.  If  the  educational  goals  are  related  to  improve  skills  and  to  solve 
very  specific  tasks,  this  approach  can  be  useful 


landscapes.  - 


Guided  Navigation  Pedagogical  Approach 


Solution  - Take  in  count  the  possibilities  of  the  student  to  expand  the  learning  environment  by  himself, 
offering  additional  information.  Make  sure  that  an  alternative  path  always  return  to  the  main  one.  Learning 
environment  enrichment  can  be  reached  by  offering  paths  with  additional  information.  Additional 
information  paths  must  be  planned  and  linked  to  the  main  path. 

Known  Uses  - Education  - Cooperative  Distance  Learning  - (http://venus.rdc.puc-rio.br/kids/kidlink/ 
acd/acd_index.htm) 

Related  pattern:  Improving  browsing 

Improving  browsing 

Description:  This  pattern  is  addressed  to  improve  browsing  possibilities  in  guided  navigation  approach,  in 
order  to  enrich  the  information  landscapes. 

Problems  - How  can  we  avoid  poor  browsing  and  restrict  learning  environment  in  a guides  navigation 
approach? 

Forces  - Guided  navigation  assures  user’s  orientation  and  often  do  not  cause  cognitive  overhead.  This 
pedagogical  approach  that  supports  this  kind  of  learning  often  lets  the  student  boring  and  unmotivated. 
When  the  hypermedia  planning  is  based  on  closed  tours,  it  may  result  in  a poorly  browsing,  restraining  the 
learning  environment  Tours,  tracks  and  pre-defined  paths  assure  user’s  orientation  and  can  help  educational 
goals  achievement.  If  the  educational  goals  must  be  reached,  guided  navigation  is  a good  design  decision.  If 
the  educational  goals  are  related  to  improve  skills  and  to  solve  very  specific  tasks,  this  approach  can  be 
useful.  A guided  navigation  guaranties  the  accessing  to  main  path  related  to  the  problem  solving.  Time 
demands  to  find  and  retrieve  effective  information  is  short. 

Solution  - Costume  the  browsing  considering  learning  speed,  learning  context  (levels),  individual  annotations 
of  documents,  so  the  student  can  feel  responsible  to  some  decisions  taken.  The  designer  must  take  in  count  the 
student’s  possibilities  to  expand  the  learning  environment  by  himself,  offering  them  additional  information 
choices.  It  must  offer  navigational  level  portraying  a range  of  choices  to  students  with  different  knowledge 
and  skills,  at  distinct  moments.  It  must  use  a comprehensive  set  of  navigational  tools,  such  as  personal  notes, 
reports  and  databases  facilities  to  individualize  the  navigation. 

Known  Uses  - Education  - Cooperative  Distance  Learning  (http://venus.rdc.puc-rio.br/kids/kidlink/ 
acd/acd_i  ndex.htm) 

Related  pattern:  Enriching  navigational  paths 


Networking  Functionality 

The  Internet  is  growing  fast  and  becoming  more  and  more  sophisticated.  It  brings  out  serious  problems,  such 
as  long  answer  time,  links  cut  and  access  denied.  As  solution,  we  propose  the  patterns  “Minimizing  answer 
time”  and  Assuring  access  to  informational  links 

Minimizing  ANSWER  time 

Description:  Due  to  current  networking  bandwidth,  informational  links  often  present  a long  answer  time.  The 
user  becomes  boring,  unmotivated,  frustrated  and  anxious.  It  makes  the  user  jump  the  node  and  choose 
another  link. 

Problem  - How  can  we  minimize  the  long  answer  time  when  we  access  a informational  link? 

Forces  - The  answer  time  delay  can  be  minimized  if  the  users  know  what  is  going  on  with  the  information 
retrieval. 

Solution  - If  the  link  belongs  to  the  application  scope  (internal  link),  the  designer  must  analyze  the  media 
benefits  (video,  animation,  images)  to  decide  about  its  importance  to  the  users  interaction  and  learning 
process.  The  designers  can  use  smaller  images,  with  low/medium  resolution,  use  small  video  sequences.  If 
the  link  is  Web-site  (external  link)  the  designer  must  provide  information  about  the  time  spent  for  information 
retrieval.  He  can  also  give  the  user  the  choice  of  accessing  or  not  the  information.  Moreover,  he  should 
provide  similar  information  as  internal  links. 

Assuring  access  to  informational  links 
Name:  Internet  functionality 

Description:  Some  Internet  functionality  can  get  the  user  lost  and  confuse,  since  he  can  meet  cut  informational 
links,  with  access  denied  and  with  different  interfaces  layout. 
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Problem  - How  can  we  avoid  that  the  user  get  lost  and  confuse  facing  not  linked  informational  nodes,  denied 
access  and  different  interface  layout? 

Forces  - All  users  can  publish  and  cut  out  pages  in  the  Internet,  at  any  time.  The  page  belongs  to  him,  so  he 
can  also  deny  the  user’s  access  any  time.  The  Internet  is  in  frequent  changing,  with  the  addition  or 
suppression  of  links.  Moreover,  its  dynamism  is  a useful  feature  for  gathering  new  information. 

Solution  - For  each  particular  situation  use  one  of  the  solutions  below:  Link  cut  - store  external  links  and 
information  into  databases  in  its  server.  Therefore  high  storage  cost,  lost  of  performance  and  maybe  requires  a 
hard  programming  step  or  phase.  Access  denied  - be  sure  that  all  external  links  are  available.  If  the  links  were 
not  available  after  implementation,  find  and  present  similar  links.  Interface  layout  - search  sites  with  simple 
and  clear  interface  layout  and  that  do  not  confuse  the  user  or  make  him  tired.  Final  solution  to  the  Internet 
functionality  related  problems  will  only  come  with  network  stability  and  the  definition  of  clear  access  rules. 


Conclusions  and  Future  Works 

Changing  on  the  educational  process  focus  has  impacting  educational  software  development.  Now,  software 
design  must  encapsulate  hypermedia  features,  but  educational  hypermedia  applications  can  assume  different 
pedagogical  categories.  To  support  the  new  scene,  learning  and  methodological  guidelines  must  be  available 
to  help  designers’  work.  An  accurate  authoring  process  of  educational  hypermedia  applications  can  result  in 
an  appropriate  navigation.  In  this  sense,  we  are  proposing  a meta-model  for  helping  all  phases  of  the 
authoring  process  plus  a set  of  pedagogical  guidelines  expressed  as  design  patterns.  Developers  and  teachers 
care  for  this  conceptual  information  and  the  pattern  system  will  be  reused  according  to  specific  need.  Our 
work  is  a work-in-progress,  moreover,  we  are  finishing  and  refining  the  written  patterns,  aiming  to  test  and 
validate  them.  Other  related  patterns  will  be  described,  in  order  to  compose  a preliminary  pattern  system  for 
educational  hypermedia  modeling. 
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Abstract:  Web  University  (WU)  is  a project  based  on  the  concept  of  virtual  university  where 
distance  participants  communicate  over  networks  by  using  applications  such  as  email,  World 
Wide  Web  (Web),  multicast  interactive  videoconferencing  and  video-on-demand.  The  project 
uses  national  research  networks  and  European  wide  high  speed  Internet  connections. 

The  Web  University  started  as  a collaborative  pilot  project  between  CERN  and  Finnish 
universities  and  extended  to  other  countries  in  1998.  This  article  describes  how  the  WU-pilot 
has  used  public  domain  software  for  creating  interactive,  real  time  distance  learning  and 
working  environment  and  video-on-demand  services  on  FUNET  (Finnish  University  and 
Research  Network).  The  WU-pilot  has  tested  and  applied  new  technologies  to  the  service  of 
distance  learning  and  working  as  efficiently,  user-friendly  and  economically  as  possible. 


1 Web  University  (WU)  project 

1.1  Goals  and  background 

The  Web  University  project  (WU)  works  as  a virtual  university.  It  started  on  transferring  the  latest  results  of 
high-energy  physics  directly  to  universities.  Physics  researchers  can  update  their  knowledge  by  participating  in 
CERN  (European  Laboratory  for  Particle  Physics)  seminars  and  experiments  from  their  personal  workstations 
over  real-time  broadband  network  based  on  ATM  (Asynchronous  Transfer  Mode)  and  Internet  technology. 
Teaching  is  interactive  and  it  is  in  principle  targeting  researchers  and  postgraduate  students,  but  some  courses  in 
physics  and  information  technology  have  been  additionally  offered  to  undergraduate  students  and  special  groups, 
such  as  journalists  and  general  public. 

The  participating  universities  have  selected  lectures  and  seminars  from  the  WU  home  page  on  the  World  Wide 
Web  (Web).  WU  has  transmitted  the  lectures  by  the  means  of  real-time  videoconferencing  from  CERN  to 
participating  sites  over  the  European  and  national  broadband  research  networks.  These  transmissions  have  been 


CERN  home  page  http://www.cem.ch/ 

WU  home  page  http://wwwcs.cem.ch/,  projects,  Web  University 
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interactive.  In  addition,  the  learning  has  been  independent  of  place  and  time  because  the  real-time  sessions  have 
been  recorded  on  a server.  All  the  time,  WU  has  tested  and  used  the  latest  technology. 

Distance  work  has  been  started  with  user  tests,  where  the  supervisor  observes  and  communicates  with  test  users 
over  the  network.  The  supervisor  has  been  at  CERN,  in  Switzerland,  and  the  test  users  in  Helsinki  Institute  of 
Physics,  in  Finland.  These  user  tests  have  been  part  of  the  development  and  evaluation  work  of  a Web  based 
document  management  system  (more  Eronen  1 998). 

The  WU  project  started  as  a collaborative  pilot  between  CERN  and  Finland  in  1995.  Since  1997,  the  activities  in 
the  WU  project  at  CERN  were  planned  and  conducted  in  collaboration  with  Finnish  distance  learning  projects, 
Finnish  Distance  Learning  in  Multimedia  Network  project  (ETAKAMU,  1996-1999)  and  Finnish  Open  Learning 
Environment  (OLE,  1999-2002).  Since  1998,  the  real-time  sessions  have  been  transmitted  at  the  same  time  to 
Finland,  Italy,  Holland  and  Slovak. 

In  particular,  the  following  Finnish  companies  are  funding  the  WU  project:  Technology  Development  Center 
(Tekes),  Center  for  Scientific  Computing  (CSC),  Helsinki  Institute  of  Physics  (HIP)  and  HPY/  Finnet  Group. 
CSC  is  the  enterprise  of  the  Ministry  of  Education  and  it  is  operating  the  Finnish  University  and  Research 
Network  (FUNET).  Helsinki  Institute  of  Physics  coordinates  cooperation  between  CERN  and  Finland  and  does 
high  energy  physics  research  in  Finland. 

WU  is  open  to  all  universities  and  research  centers  to  participate.  So  far,  real-time  distance  participants  have 
been  from  the  following  1 1 universities  and  institutes:  in  Finland,  the  University  of  Helsinki/Helsinki  Institute  of 
Physics,  Helsinki  University  of  Technology,  the  University  of  Jyv^kyla,  Tampere  University  of  Technology, 
Pori  School  of  Technology  and  Economics,  the  University  of  Oulu  and  Oulu  Institute  of  Technology;  in  Holland, 
the  University  of  Twente;  in  Italy,  the  University  of  Rome  2 "Tor  Vergata"  and  the  High  Energy  Physics 
Laboratory  of  Pisa  and  in  Slovak,  Pratislav  University  of  Technology. 

1.2  Communication  networks 

The  Web  University  project  started  by  transmitting  CERN  lectures  to  Finnish  universities  over  ISDN  but  since 
June  1996  the  videoconferencing  has  been  done  over  IP/ATM,  as  that  was  when  the  European  Commission 
launched  its  telecommunication  programs  based  on  broadband  network  services. 

The  Web  University  project  became  a customer  of  JAMES  (Joint  ATM  Experiment  on  European  Services)  in 
1996  and  used  a point  to  point  connection  with  a communication  speed  1-  2 Mbps  between  CERN  in 
Switzerland  and  Helsinki  in  Finland.  Since  1998,  WU  has  used  TEN-34  and  TEN- 155  and  the  seminars  have 
also  been  transmitted  to  Italy,  Holland  over  TEN  and  to  Slovak  over  a private  network. 

The  lectures  were  multicast  in  each  country  by  the  national  research  network  operators.  FUNET,  the  Finnish 
University  and  Research  Network  has  multicast  lectures  to  universities  in  Finland.  The  communication  speed  of 
the  FUNET  backbone  network  is  155  Mbps  and  mainly  based  on  ATM. 

1.3  Software 

The  learning  and  working  environment  of  WU  is  based  on  tools  of  the  MBONE  technology.  At  the  beginning  of 
the  project,  the  connection  for  the  real-time  videoconferencing  was  created  manually  by  Rtptrans  software  and 
digital  recordings  were  also  done  manually.  Since  the  autumn  1998,  the  connection  has  been  created 
automatically  by  the  Virtual  Room  Videoconferencing  System  (VRVS).  VRVS  is  Web  based  and  it  is  being 
developed  at  CERN.  Digital  recording  and  playback  have  also  been  done  by  VRVS.  The  recordings  have  been 
copied  to  the  WrtpVod  server  of  FUNET  for  retransmissions  on  demand. 

1.4  What  do  you  need  to  participate  in  WU? 

The  project  discovered  that  every  country,  that  wants  to  participate  in  WU,  requires: 

1.  Universities  and  research  centers,  which  are  connected  to  the  European  academic  and  research  infrastructure 
and  are  willing  to  join  the  project.  These  universities  should  be  interested  in  using  or  producing  freely  available 
courses  as  part  of  an  academic  distance  curriculum.  Moreover,  these  universities  and  research  centers  should  be 
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motivated  to  receive  or  distribute  the  latest  research  results  and  support  local  costs. 

2.  At  least  one  technical  person  and  one  educational  contact  person,  who  co-ordinate  activities  in  the  country. 
The  technical  contact  person  should  be  involved  with  a national  network  provider. 

3.  A technical  contact  person  and  an  educational  contact  person  for  each  institute. 

2 Results  and  conclusions 

CERN  is  a research  laboratory  where  the  best  specialists  of  scientific  institutes  and  universities  often  come  to 
give  lectures.  WU  has  made  these  seminars  available  for  researchers  and  students  in  remote  universities. 
Moreover,  it  has  enabled  researchers  and  their  students  even  at  small  universities  to  attend  the  CERN  seminars 
from  their  own  university.  They  can  participate  in  interactive,  real-time  seminars  from  their  personal  computers 
or  local  auditoriums  and  ask  questions  from  the  lecturer  either  as  the  audience  at  CERN  or  by  using  email.  WU 
has  transmitted  more  than  100  hours  of  different  kind  of  lectures  and  seminars  to  the  universities.  If  participants 
are  occupied  during  the  real-time  session,  they  can  study  the  subject  by  using  the  video-on-demand  service  of 
FUNET  some  hours  later  when  the  time  is  convenient  for  them. 

In  this  context,  WU  found  out  how  to  make  a successful  distance  learning  session  by  studying  distance  students’ 
attitudes  and  by  evaluating  distance  learning  (Rinta-Filppula  1998). 

These  findings  proved  that,  at  the  beginning,  the  pedagogical  expectations  should  not  be  too  high,  as  the  students 
need  time  to  adapt  to  a new  situation  and  learning  environment.  In  addition,  it  turned  out  that  it  is  difficult  create 
an  interactive  distance  learning  session  because  a lecturer  and  distance  students  are  different  every  time  and  to 
thus,  they  do  not  know  each  other. 

It  appears  that  undergraduate  students  need  academic  credits  as  a motivator  but  postgraduate  students,  on  the 
contrary,  are  interested  in  the  latest  research  results  even  without  getting  any.  Academic  freedom  gives  students 
many  possibilities  to  select  study  courses.  Undergraduate  students  are  unlikely  to  be  willing  to  spend  their  time 
looking  at  a video  screen  and  studying  in  a foreign  language,  English,  if  they  do  not  get  any  credits. 

2.1  How  to  make  face  to  face  studies  successful  distance  learning 

Distance  learning  sessions  can  be  planned  for  distance  learning  or  they  can  be  a part  of  face  to  face  learning, 
where  the  students  participate  over  networks.  This  article  concentrates  on  the  latter  case,  where  it  is  very 
important  to  provide  lecturers  with  preliminary  information  about  the  distance  students  in  order  to  give  them  a 
possibility  to  adapt  their  lectures  for  the  distance  learners.  The  feedback  collected  has  shown  that  the  lecturers 
should  more  commit  themselves  to  adapt  their  presentation  for  distance  participants.  On  the  other  hand,  the 
lecturers  would  need  support  and  help  how  to  apply  the  technology  from  pedagogical  point  of  view. 

Distributed  real-time  sessions  are  differently.  Distance  participants  have  proved  to  be  especially  satisfied  if  the 
speaker  is  well-known  (e.g.  a Nobel  price  winner  or  a Web  inventor)  and  they  can  communicate  real-time  with 
him  or  her.  In  addition,  they  can  later  integrate  the  presentation  to  their  teaching  or  studies  because  it  is  available 
on  the  network  based  multimedia  service. 

Flexible  learning  should  be  planed  different  if  the  distance  learners  are  allowed  to  include  the  sessions  to  their 
studies  and  obtain  credits.  In  this  case,  learning  process  should  start  before  the  real  time  session  to  enable  to  take 
full  advantage  of  the  possibilities  that  modem  learning  theories  and  technology  provide.  This  can  not  happen 
without  an  agreement  and  cooperation  of  a local  teaching  staff  in  a distance  institute.  A network  base  multimedia 
service  built  on  video-on-demand  service,  course  discussion  channel,  knowledge  exchange  of  the  best  practices 
of  distributed  programs  etc.  is  an  essential  part  of  distance  learning  but  even  more  important  is  to  develop  and 
adapt  teamwork  and  active  learning  methods  to  new  high  speed  network  environment.  This  can  take  place  for 
example,  as  follows: 

A lecturer  who  will  give  a presentation  in  a real-time  session  publishes  his  abstract  and  reference  material  on  the 
Web  in  advance.  In  distance  universities,  a local  teaching  staff  and  their  students  agree  on  draft  plans  of  distance 
student  teamwork,  work  methods  and  evaluation.  The  drafts  will  be  published  on  the  Web  in  a course  discussion 
channel.  The  students  study  the  reference  material  before  the  interactive,  real-time  session  gets  started.  After  the 
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session,  the  plans  of  the  distance  groups  will  be  published  on  the  Web  and  the  students  will  work  with  their 
group  presentation  papers  by  using  teamwork  tools,  project  tools  etc.  They  will  then  publish  their  papers  on  the 
Web  and  present  them  in  a real-time  virtual  seminar,  where  they  also  will  act  as  opponents  to  the  other  students. 
This  is  not  only  an  excellent  possibility  for  students  to  practice  project  and  teamwork  but  also  planning  and 
giving  a presentation  in  an  international  conference.  In  addition,  through  the  student  works,  the  teachers  can 
enrich  their  teaching  methods.  After  a virtual  seminar,  the  students  will  publish  the  second  part  of  their  distance 
work,  i.e.  the  evaluation.  It  can  consist  of  e.g.  a report  of  their  work  progress,  their  individual,  as  well  as  group 
analyses  of  what  they  have  learnt,  their  group  analysis  about  learning  and  work  practices  and  recommendations 
for  getting  better  learning  results.  It  should  be  noticed  that  the  students  are  constantly  expected  to  pay  attention 
to  justify  their  opinions,  in  order  to  derive  full  advantage  from  the  learning  process. 

The  virtual  learning  described  above  can  take  place  only  in  universities  (e.g.  Finnish  universities  and  the 
university  of  Twente  in  Holland  (Collis  & Fisser  1998))  where  the  teachers  have  applied  teleteaching,  teamwork 
tools,  project  learning  and  other  active  learning  methods  as  a part  of  their  curriculum.  The  pedagogical 
background  of  the  model  lays  on  the  seven  properties  of  meaningful  learning  (Pohjolainen  1999)  that  are  suitable 
for  lifelong  learning  - from  comprehensive  school  to  university  level  and  to  adult  learning  - independent  of  time 
and  place.  This  kind  of  virtual  learning  carries  out  the  active  learning  (Jonassen  1995). 
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Abstract 

The  Internet  is  the  largest  computer  system.  The  net  or  cyberspace  provides  much  useful  information  as  we 
navigate  URL’s,  browsers  and  hyperlinks.  With  advanced  technology  comes  new  legal  issues  to  battle. 
Copyright  law  is  at  the  forefront  of  cyberspace  debate.  Thus,  we  need  to  examine  and  stay  abreast  of  issues  so 
important  to  all  those  being  affected  by  this  area  of  law. 

In  this  work  I will  endeavor  to  explain  the  latest  legislation  regarding  copyright  in  cyberspace.  I hope  that 
such  information  will  prove  to  be  a resource  as  well  as  a guide  to  all  those  interested  in  traversing  these 
navigable  waters. 


On  October  28,  1998  President  Clinton  signed  a bill  providing  new  game  rules  for  the  treatment  and 
respecting  of  online  copyrighted  material.  The  Digital  Millennium  Copyright  Act  (hereinafter  referred 
to  as  the  DMCA),^ 

passed  both  houses  of  the  one  hundred  and  fifth  (105^*^)  Congress  earlier  in  the  month. ^ 

The  DMCA  adds  two  (2)  new  chapters  to  Title  1 7 as  it  strengthens  international  law  worldwide  and 
protects  domestic  technology. 

President  Clinton  was  please  with  the  measure.^ 

The  one  hundred  and  fifty  (150)-page  document  divides  into  five  (5)  titles  in  Fig.  1 below: 


Fig.l:  THE  DMCA 


Note'.  Except  for  Title  I (Treaty),  each  the  following  are  effective  upon 
enactment: 


Title  /:  Implementation  of  two  (2)  treaties  dealing  with  digital  issues, 

copyright  protection  and  management  systems. 

Title  II\  Limitation  of  Online  iniringement  liability  for  ISPs  ( Internet 
Service  Providers)  ( reducing  legal  uncertainties  regarding  such 
items  as  digital  networks,  strengthening  anti-online  piracy, 
outlining  copyright  owners’  notification  procedures,  defining 
university  liability,  and  creating  a “safe  harbor”  for  ISPs  in 
four  (4)  situational  activities): 
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(1)  Conduits  (provision  of  materials 

transmission,  routing  and 
connections) 

(2)  System  Caching  (temporary  or 

intermediate  materials 
storage) 

(3)  User  Storage  (materials) 

(4)  Information  Locators  (linkage  provision) 


Note:  1 and  2:  transmission  must  be  initiated  by  a third  party. 

3 and  4:  requires  the  ISP  to  be  without  knowledge  or  having  reason 
to  know  of  any  infringement,  to  obtain  no  direct  financial 
benefit  and  to  not  change  the  materials. 

Tit/e  ///:  *The  Computer  Maintenance  Competition  Assurance  Act” 

( formerly  H.R.  72)  (creation  of  an  exception  for  temporary 
computer  program  reproduction  in  maintenance/repair). 

T/t/e  IV\  “Miscellaneous  Provisions”  (distance  education,  exemption 
for  libraries/arc  hives,  ephemeral  (momentary)  recordings). 

Title  V\  “The  Vessel  Hull  Design  Protection  Act” 

(formerly  H.R.  2696)  (creation  of  new  protections  for  boat  hull 
designs,  effective  for  two  (2)  years  only)."* 


Oddly  enough  Congress  handed  down  the  measure  around  Columbus  Day.  It  would  seem  Americans  were 
in  store  for  more  than  celebrating  the  discovery  of  the  country.  Although  technically  the  Senate  still  must 
ratify  international  pacts  before  governments  of  the  world  give  credence  to  the  measure,  the  law  does 
prepare  for  the  ratification  and  execution  of  two  (2)  treaties  regarding  WIPO,  The  World  Intellectual 
Property  Organization.  In  December  1996  over  one  hundred  and  fifty  (150)  countries  agreed  on  WIPO  at 
a conference  on  digital  information  and  copyrights  in  Geneva.  The  first  treaty  clears  up  digital  authors’ 
rights.  The  second  pact  focuses  upon  the  Internet  and  sound  recordings.  Thirty  (30)  nations  must  ratify 
the  agreement  for  it  to  be  effective  globally.^ 

Internet  service  providers,  software  industry  groups,  music/movie  companies  and  the  like  herald  the 
DMCA.  Attorney  Jonathan  Band,  a partner  in  the  Washington,  D.C.  office  of  San  Francisco’s  Morrison 
& Forester,  L.L.P,  practices  copyright  law.  He  states: 

The  Digital  Millennium  Copyright  Act  accomplishes  four  things...  First,  it  implements 
the  World  Intellectual  Property  Organization  treaties,  thus  harmonizing  U.S.  copyright 
law  with  international  law.  Second,  the  DMCA  establishes  ’’safe  harbors"  for  online 
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service  providers  who  unknowingly  transmit  copyrighted  works.  Third,  the  act  permits 
the  copying  of  software  during  computer  maintenance.  Finally,  the  DMCA  facilitates 
Internet  broadcasting.^ 

However,  members  of  the  academic  and  research  communities  express  mixed  feelings  about  the  measure. 
Some  claim  the  DMCA  would  hinder  concepts  of  fair  use  and  other  acceptable  means  of  validly  utilizing 
copyrighted  materials.  Most  library  organizations  opposed  the  measure  citing  it  does  not  contain  many 
desired  provisions.  Specifically,  according  to  Professor  Bob  Oakley,  Library  Director  of  the  Georgetown 
University  Law  Center,  H.R.  2281  will  be  a hindrance  to  reading,  browsing,  classroom  teaching  and 
applying  of  fair  use  standards: 


HR  2281,  as  drafted,  would  grant  copyright 
owners  a new  and  unrestricted  exclusive  right  to 
control  access  to  information  in  digital  works 
which  could  negate  one  of  the  most  basic 
principles... the  ability  to  gain  access  information 
in  published  or  publicly  available  works. ..^ 

Many  groups  voiced  support  and  opposition  for  the  bill.  ^ The  Digital  Future  Coalition,  a forty-two  (42) 
member  organization  comprised  of  non-profit  and  for-profit  entities  interested  in  intellectual  property  law 
in  the  digital  era,  opines: 

In  the  final  version  of  the  DMCA,  Congress  recognized  the  importance  of  ensuring 
balance  in  the  treaty  implementing  legislation.  The  DMCA  safeguards  such  crucial 
activities  as  computer  security  testing,  reverse  engineering  to  achieve  interoperability, 
the  protection  of  personal  privacy,  parental  supervision  of  minors  on  the  Internet  and  the 
preservation  of  materials  by  libraries  and  archives.  It  also  assures  the  availability  of  the 
next  generation  of  consumer  electronics  and 

computer  products.  In  addition,  it  provides  a mechanism  to  assure  the  continued  vitality 
of  the  fair  use  privilege  enjoyed  by  teachers,  students,  library  patrons  and  all  other 
information  users.  These  provisions  represent  a dramatic  departure  from  earlier  drafts  of 
the  legislation....*® 

Another  bill.  The  Digital  Era  Copyright  Enhancement  Act,  H.R.  3048,  is  still  awaiting  approval.  The 
measure  included  an  increase  for  the  preservation  of  author’s  rights  similar  to  the  provisions  in  the  Berne 
Convention  for  the  Protection  of  Literary  and  Artistic  Works.  In  March  1989  the  United  States  joined  the 
Convention  that  produced  a multinational  treaty.** 


H.R.  3048  amends  the  first  sentence  of  section  107  of  The  Copyright  Act  by  adding  “analog  or  digital”  in 
applying  fair  use  to  all  transmissions.*^  In  actual  comparison,  H.R.  3048  includes  many  safeguards 
lacking  in  H.R.  2281.  Among  them  include  legal  protections  with  allowances  for  the  creation  of 
incidental  copies,  fostering  of  distance  learning*^  protecting  of  consumers  against  so-called  shrink  wrap 
licenses,  negating  of  ephemeral  copyright  liability  (i.e.  making  copies  incidentally  in  relation  to  computer 
operation),  preserving  of  microfiche  technologies,  exempting  library/archive  uses  (allowing  for  the 
making  of  temporary  computer  program  copies  and  utilizing  a “knowledge”  test  as  the  basis  for  civil 
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liability;  note  that  criminal  liability  is  not  available  at  this  level)'"^,  and,  most  importantly  for  many,  the 
preserving  of  fair  use  doctrinal  principles.*^  Indeed  the  two  bills  have  been  very  much  compared  and 
contested  of  late. 

H.R.  3048  indeed  expands  the  fair  use  doctrine  toward  notions  that  library,  teaching  and  research 
purposes  fall  into  acceptable  practices  in  relation  to  technological  advances.  Penalties  apply  against 
infringers  rather  than  those  on  the  fringe  that  may  wish  to  legally  and  ethically  utilize  materials.*’ 

According  to  the  Software  Publishers  Association,  copyright  piracy  costs  over  eighteen  (18)  billion  dollars 
worldwide.*^  Regarding  penalties  and  liabilities,  in  general,  the  DMCA’s  Sections  1203  and  1204 
imposes  updated  standards  and  gives  guidance  for  works  on  the  Net  especially  regarding  criminality. 
Plaintiffs  recovering  successfully  for  wrongdoing  have  the  choice  of  illegally  obtained  profits,  statutory 
damages  or  injunctive  relief  In  a year  and  a half,  it  will  be  unlawful  to  create  or  sell  any 
technology  used  to  break  copyright  protection  devices.  It  also  will  be  illegal  to  commit  acts  of 
circumvention  as  each  will  carry  statutory  damages  of  twenty-five  hundred  dollars  ($2500.00).*^  And,  in 
a limited  manner,  willful  and  purposeful  defendants  could  face  serious  criminal  penalties  of  several 
hundred  thousand  for  each  violation.’^ 

The  DMCA  contains  many  criminal  impositions  for  conduct  unrelated  to  infringement.  For  example 
section  1 04  provides  for  violations  regarding  circumvention  of  copyright  protection  systems  in  section 
1201  or  integrity  of  copyright  management  in  1202.  Anyone  who  violates  either  section  for  “purposes  of 
commercial  advantage  or  private  financial  gain”  faces  up  to  five  hundred  thousand  (500,000)  dollars  or 
imprisonment  of  up  to  five  (5)  years,  or  both  for  the  first  (1^*)  offense.  The  penalty  is  up  to  one  million 
(1 ,000,000)  dollars  and  up  to  ten  (10)  years,  or  both,  for  subsequent  offenses.’*  DMCA  sections  1204  a & 
b.  Courts  will  have  the  power  to  award  triple  damages  against  repeat  offenders.  A five  (5)  year  statute  of 
limitations  applies  here.”  H.R.  3048,  on  the  other  hand,  applies  civil  relief  rather  than  criminal 
punishment. 

And,  H.R.  3048  provides  that  "no  person,  for  the  purpose  of  facilitating  or  engaging  in  an  act  of 
infringement,  shall  engage  in  conduct  so  as  knowingly  to  remove,  deactivate  or  otherwise  circumvent  the 
application  or  operation  of  any  effective  technological  measure  used  by  a copyright  owner  to  preclude  or 
limit  reproduction  of  a work  or  a portion  thereof."”  The  DMCA  makes  it  illegal  to  "manufacture,  import, 
offer  to  the  public,  provide  or  otherwise  traffic  in  any  technology,  product,  service,  device,  component  or 
part  thereof  " that  may  be  used  to  circumvent  a technological  protection  measure.”  The  law  affords 
protection  toward  encryption  research,  security  testing  and  related  matters.  And  devices  as  to  safeguard 
against  child  pornography  on  the  Internet  were  given  exemption  in  this  respect. 

Conclusion 

There  are  several  bills  such  as  H.R.  3048  still  on  Congress’s  plate.  The  DMCA  is  a massive  complexity  of 
rules  and  regulations.  If  we  are  to  advance  in  the  digital  age,  we  must  have  a compromise  between  right 
and  rule.  It  will  be  a test  of  time  as  to  whether  or  not  the  Clinton  administration’s  efforts  will  be  a cure  or 
a curse  for  the  new  millenium  in  copyright.  The  only  way  to  examine  the  DMCA’s  validity  is  by  trial 
and  error.  Unfortunately  we  will  be  cursed  with  many  of  each. 


* Dr.  Robert  N.  Diotalevi  is  Director  of  Legal 
Studies  and  Associate  Professor  at  The  College  of 
West  Virginia.  He  has  been  a lawyer  for  thirteen 
years.  He  possesses  four  (4)  degrees  and  has  been 
internationally  published. 
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Abstract:  This  paper  addresses  the  question  of  the  noticed  resistance  of  many  learners,  and 
indeed  teachers,  to  the  use  of  computer-mediated  discussion  in  the  support  and  delivery  of 
learning  programmes,  and  their  preference  for  face-to-face  modes  of  teaching.  It  applies 
some  of  the  insights  gained  from  post-structuralist  theories  of  language  in  order  to  approach 
a fuller  understanding  of  this  preference,  and  concludes  that  a fundamental  re-thinking  of 
our  understanding  of  the  educational  experience,  and  its  cultural  context,  is 
necessary  if  we  are  fully  to  engage  with  the  opportunities  presented  by  computer- 
mediated  approaches  to  learning. 

Introduction 

This  paper  originates  in  attempts  to  explain,  at  a practical  level,  the  perceived  disinclination  of  learners  to 
participate  in  computer-mediated  conferencing  (CMC)  in  contrast  to  traditional  face-to-face  teaching.  This 
apparent  privileging  by  learners  of  face-to-face  environments  in  our  own  experience  of  managing  CMC 
programmes,  in  addition  to  similar  learner  resistance  reported  by  other  providers  (Akers  1997;  Finkelstein  & 
Dryden  1998;  Grint  1989;  Von  Priimmer  1995)  led  us  to  seek  theoretical  explanations  of  such  preference,  the 
better  to  be  able  to  deal  with  related  pedagogical  considerations.  Earlier  explanatory  models  have  attempted  to 
account  for  face-to-face  preference  through,  for  example,  the  idea,  derived  from  social  presence  theory,  of 
reduced  social  cues  (Gundewara  1994),  or  of  the  ephemeral  nature  of  ‘orality’  (Zuboff  1988).  In  psychology, 
more  determinist  social  learning  approaches,  derived  from  the  psychological  Theory  of  Mind  (TOM),  suggest 
that  we  are  hardwired  for  social  interaction,  with  the  mind  expecting  to  find  intersubjectivity.  Most  pf  these 
studies,  it  seems  to  us,  lack  explanatory  power  in  relation  to  CMC  and  pedagogical  implications  for  the 
management  and  practice  of  learning  in  CMC  environments.  This  is  not  surprising,  since  such  perspectives 
reside  within  a realist/humanist  tradition  which  by  definition,  as  will  be  explained  below,  always  cedes 
precedence  to  face-to-face  approaches  to  teaching  and  learning. 

There  is  need  for  a theorised  account  of  CMC  from  an  alternative  perspective.  Given  that  CMC  is  essentially  a 
to/-based  environment,  one  in  which  learning  is  mediated  entirely  through  the  processes  of  reading  and 
writing,  a much  more  compelling  approach  may  be  derived  from  the  insights  gained  from  post-structuralist 
theories  of  language,  in  particular  the  work  of  the  French  philosopher,  Jacques  Derrida.  Our  claim  is  that  the 
privileging  of  face-to-face  learning  over  CMC  derives,  from  the  privileging  of  speech  over  writing  that  Derrida 
has  forcefully  identified  as  the  most  central  hierarchical  opposition  in  the  history  of  Western  metaphysics.  The 
privileging  of  speech,  moreover,  is  associated  with  the  notion  of  personal  presence.  As  Feenberg  has  noted: 

In  our  culture  the  face-to-face  encounter  is  the  ideal  paradigm  of  the  meeting  of  minds.  Communication  seems  most 
complete  and  successful  where  the  person  is  physically  present  ‘in’  the  message.  This  physical  presence  is  supposed 
to  be  the  guarantor  of  authenticity:  you  can  look  your  interlocutor  in  the  eye  and  search  for  tacit  signs  of  truthfulness 
or  falsehood,  where  context  and  tone  permit  a subtler  interpretation  of  the  spoken  word.  (Feenberg  1989) 

Such  a position  appears  so  commonsensical  within  traditional  western  culture  that  it  comes  as  something  of  a 
shock  to  find  Derrida  describing  speech  as  another  form  of  ‘writing’  and  denying  its  status  as  a more  authentic 
source  of  ‘truth’.  This  paper  will  argue  that  resistance  to  the  loss  of  face-to-face  teaching  can  be  seen  as 
attributable  to  a generally  unexpressed  awareness  that  CMC  undermines  the  position  of  speech  over  writing, 


and  in  doing  so  also  undermines  the  entire  realist  and  humanist  perspective  from  which,  in  Western  culture,  we 
construct  ideas  of  truth,  presence  and  subjectivity. 

These  are  indeed  large  and  no  doubt  provocative  claims.  Hence  priority  is  assigned  below  to  a careful 
exposition  of  the  philosophical  basis  of  our  argument.  The  final  purpose  of  the  paper,  however,  is  to  return  to 
the  pedagogical  considerations  from  where  the  discussion  originated,  and  to  describe  what  in  our  view  are 
crucial  implications  for  the  roles  of  tutor,  moderator  and  learner  in  CMC  environments,  and  even  for  the  nature 
of  the  institutions  in  which  such  programmes  operate. 


Language  and  the  dispersed  nature  of  meaning 

To  approach  the  issue  of  why  CMC  participants  seem  uneasy  with  the  loss  of  face-to-face  teaching  it  is  first 
necessary  to  ‘problematise’  the  nature  of  language.  The  seeming  nature  of  language  as  a transparent  lens 
through  which  we  perceive  ‘reality’  rather  than  as  the  medium  through  which  realities  are  constructed  is  part  of 
a process  operating  within  CMC  which  we  are  here  concerned  to  challenge.  It  is  easy  to  forget  that  the 
computer-mediated  learning  environment  is  essentially  a collection  of  written  texts  — a ‘written  world’,  to  use 
Feenberg’s  phrase.  Though  we  often  refer  to  participants  in  CMC  environments  as  ‘speaking’,  ‘chatting’, 
‘talking’,  having  a ‘voice’  and  engaging  in  ‘conversation’  and  ‘discussion’,  and  refer  to  them  collectively  as  an 
‘audience’  that  ‘hears’,  such  metaphors  are  themselves  examples  of  the  privileging  of  speech  over  writing 
mentioned  earlier. 

The  full  impact  of  post-structuralist  thought,  and  its  relevance  to  computer-mediated  learning,  can  be 
approached  through  the  work  of  the  structuralist  linguistic  theoretician  Ferdinand  de  Saussure.  Saussure  views 
language  as  a system  of  signs;  each  sign  is  composed  of  a signifier  (a  word,  written  or  spoken)  and  a signified 
(the  concept  or  meaning)  (Saussure  1974).  The  relation  between  the  signifier  and  the  signified  is  arbitrary. 
There  is  no  intrinsic  or  given  connection,  for  example,  between  the  word  ‘mouse’  and  the  small  mammal.  If 
there  were,  the  word  for  ‘mouse’  would  be  the  same  in  every  language,  and  ‘mouse’  would  not  also  mean  ‘a 
small  device  which  controls  the  cursor  on  a PC  screen’.  What  is  more,  the  meaning  of  a sign  is  not  complete  in 
itself  - it  has  meaning  only  because  of  its  difference  from  other  signs.  For  example,  ‘mouse’  only  means 
‘mouse’  because  it  is  not  ‘house’,  ‘mousse’  or  ‘louse’.  Despite  the  fact  that  language  appears  transparent  (the 
connection  between  the  word  ‘mouse’  and  the  concept  of  a mouse  seems  clear  and  unproblematic  to  the  user  of 
the  language)  the  construction  of  meaning  is  actually  complex;  language  becomes  a system  of  differences  with 
no  positive  terms,  meaning  can  only  be  a product  of  social  and  historical  convention. 

Further,  language  does  not  provide  us  with  a set  of  terms  or  labels  which  we  use  to  name  entities  which  already 
exist  independently  in  the  world.  Rather  language  precedes  the  existence  of  these  entities,  enabling  us  to  make 
the  world  intelligible  by  differentiating  between  concepts  in  a way  which  is  particular  to  our  own  cultural 
context.  For  example,  in  Welsh  the  colour  ‘glas’  (‘blue’)  includes  elements  which  in  English  would  be 
identified  as  ‘green’  or  ‘grey’.  There  is  no  objective  experience  of  the  colour  ‘blue’  - like  language  itself,  colour 
terms  form  a system  of  differences  which  seem  ‘natural’  but  are  in  fact  enabled  or  constructed  by  the  language 
itself  (Belsey  1980).  The  language  forms  our  view  of  the  world,  and  therefore  forms  us  as  conscious,  social 
entities. 

Post-structuralist  theorists  such  as  Derrida  take  these  ideas  further.  They  highlight  what  is  implicit  in 
Saussure’s  theory,  which  is  that,  if  every  sign  gets  its  meaning  from  the  fact  that  it  is  not  all  other  signs,  every 
sign  consists  only  of  an  infinite  web  of  differences.  In  other  words,  because  the  meaning  of  a sign  consists  of 
what  the  sign  is  not,  meaning  is  never  fully  present  and  complete  in  the  sign  - rather  it  is  dispersed  among  all 
the  other  signs  in  existence.  Meaning  cannot  be  nailed  down,  it  is  inherently  unstable,  ‘never  fully  present  in 
any  one  sign  alone,  but  rather  a kind  of  constant  flickering  of  presence  and  absence  together’  (Eagleton  1983). 

The  implications  of  these  ideas  are  serious  for  traditional  systems  of  thought.  Language  loses  its  status  as  a 
handy  tool  which  enables  us  to  speak  about  and  reflect  on  ‘reality’.  To  use  language  at  all  involves  us  in  a 
dispersal  of  meaning  throughout  the  infinity  of  the  system  of  signs.  And  since  we  ourselves  are  in  a sense  our 
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language  (it  is  the  means  through  which  our  world  and  our  place  within  it  are  constructed),  the  status  of  our 
identities  (or  selves)  as  stable  and  unified  is  also  under  question. 


Logocentrism  and  the  metaphysics  of  presence 

Derrida,  whose  brand  of  post-structuralist  philosophy  is  termed  ‘deconstruction’,  sees  the  history  of  Western 
philosophy  as  being  based  in  ‘logocentrism’  - the  belief  in  some  centre,  some  ultimate  Word  (‘/ogo^’),  truth  or 
reality  which  is  not  subject  to  this  dispersal,  which  anchors  all  thought,  language  and  experience,  and  provides 
an  ultimate  meaning  which  cannot  be  questioned.  Notions  of  God,  Truth,  the  Self,  Nature  and  so  on  can  all  be 
seen  as  having  their  roots  in  logocentrism  which,  for  Derrida,  is  closely  bound  up  with  what  he  calls  the 
‘metaphysics  of  presence’. 

Presence  appears  as  the  things  we  see;  as  the  immediacy  of  the  ideas  and  representations  we  have;  as  substance, 
essence  and  existence;  as  the  temporal  present,  the  now  of  the  moment,  as  the  self-presence  of  consciousness,  of 
subjectivity,  as  one’s  own  inner  experiences  and  ideas  etc...  Presence  constitutes  the  essential  truth  of  reality;  hidden 
from  our  sight  possibly,  but  present  in  its  entirety  to  some  divine  mind,  (ibid) 

Post-structuralist  thought  stresses  that  any  such  transcendental  meaning  is  fictional,  in  the  sense  of  being 
constructed.  There  is  no  concept  which  exists  outside  systems  of  thought  and  language;  there  is  no  concept 
which  is  not  involved  in  the  infinite  play  of  meaning.  In  order  to  function  socially  we  do  make  temporary 
determinations  of  meaning  but  meaning  itself  is  never  determinate  (Derrida  1978). 

Textuality 

Ideas  about  the  nature  of  text  itself  are  radically  affected  by  the  ideas  of  the  post-structuralists.  Texts  are  no 
longer  seen  as  closed,  stable  entities,  holding  within  them  definite  meanings  which  the  reader  is  required  to 
decipher.  Rather,  they  become  a temporary  arena  for  the  endless  play  of  signifiers  - they  cannot  be  tied  down  to 
any  one  meaning,  but  remain  open  and  pluralistic,  subject  to  multiple  interpretation.  All  texts  are  woven  out  of 
other  texts  in  that  every  element  of  them  refers  to  other  writings  which  surround,  precede  or  follow,  much  as 
the  sign  itself  is  composed  of  traces  of  every  other  sign.  This  is  broadly  referred  to  as  ‘inter-textuality’. 

The  relevance  of  the  post-structuralist  vision  of  language  to  the  study  of  electronic  texts  such  as  those  found  in 
CMC  - and,  increasingly,  in  web-interfaced  versions  where  CMC  texts  become  increasingly  hypertextual  and 
linked  - becomes  clear.  Eagleton’s  account  of  the  play  of  language  from  a post-structural  perspective  is 
powerfully  suggestive  of  the  kind  of  interplay  found  in  CMC  discussion. 

Instead  of  being  a well-defined,  clearly  demarcated  structure  containing  symmetrical  units  of  signifiers  and  signifieds, 
[language]  now  begins  to  look  much  more  like  a sprawling  limitless  web  where  there  is  a constant  interchange  and 
circulation  of  elements,  where  none  of  the  elements  is  absolutely  definable  and  where  everything  is  caught  up  and 
traced  through  by  everything  else.  (Eagleton  1983) 

What  is  more,  the  traditional  hierarchical  relationship  between  texts  is  undermined.  Where  traditionally  a so- 
called  central  or  ‘primary’  text  takes  a central  position,  provides  a focus  for  discussion  and  interpretation  in  the 
form  of  marginal  or  ‘secondary’  texts,  deconstructionists  would  claim  not  only  that  the  central  can  become 
marginal  (and  vice  versa)  but  that  the  very  difference  between  the  two  is  brought  into  question.  Since  both  are 
subject  to  infinite  play  of  meaning  and  interpretation,  we  cannot  say  where  one  ends  and  the  other  begins. 
When  we  apply  this  insight  to  the  realm  of  CMC  intertextual  ‘discussion’  we  can  immediately  conceive  of  a 
questioning  of  traditional  assumptions  relating  to  which  ‘voice’  or  text  is  authoritative  and  takes  precedence 
over  the  others.  A levelling  effect  in  the  relationship  between  tutor  and  student  can  perhaps  be  seen  as  one 
inevitable  consequence  of  the  move  from  the  face-to-face  interchange  to  one  taking  place  in  a CMC 
environment. 
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Speech,  synchronicity  and  presence  in  CMC 


In  addition  to  its  essentially  textual  nature,  CMC  is  characterised  in  most  instances  of  its  use  by  its  other 
distinctive  feature  asynchronicity.  It  is  not  difficult  to  see  how  asynchronicity  can  become  identified  in  the 
minds  of  learners  with  inauthenticity. 

Traditionally  speech  is  favoured  as  the  concept  which  best  represents  the  semantic  relationship  of  language  to  thought 
in  virtue  of  its  own  essential  and  immediate  causal  proximity  both  to  the  thoughts  which  give  it  its  meaning  and  to  the 
thinker  having  those  thoughts,  expressing  these  meanings...  (Parker  1997) 

Speech  appears  to  emanate  directly  from  its  ‘author’-  the  presence  of  the  author  is  self-evidently  required  for 
speech.  In  this  regard  presence,  in  the  sense  of  physical  presence,  requires  synchronicity.  If  a logocentric 
perspective  is  maintained  then  asynchronous  activity  undermines  such  truth  at  a glance,  and  the  house  of  cards 
collapses.  And  of  course  writing  is  by  definition  asynchronous  in  nature,  traditionally  seen  as  being  a step 
removed  from  the  ‘author’  in  a way  that  speech  is  not.  Hence  text-based  conferencing  comes  to  be  seen  as 
‘inauthentic’. 

CMC,  of  course,  is  heavily  and  obviously  mediated  both  through  written  text  and  computer  technology.  Face-to- 
face,  in  comparison,  appears  deceptively  to  approximate  to  the  ideal  speech  situation.  It  is  of  course  no  such 
thing  and  is  as  mediated,  though  by  no  means  as  self-evidently,  through  linguistic  signs  and  signifiers  that  are 
as  independent  of  the  self  and  as  dependent  on  a linguistic  system  and  interpretation  as  any  written  text. 


Bivalence,  realism,  and  deconstruction 

As  Culler  has  pointed  out,  Derrida’s  claim  is  twofold: 

First  the  moment  of  speech,  or  rather  the  moment  of  one’s  own  speech,  where  signifier  and  signified  seem 
simultaneously  given,  where  inside  and  outside,  material  and  spiritual  seem  fused,  serves  as  a point  of  reference  in 
relation  to  which  all  these  essential  distinctions  can  be  posited.  Second,  this  reference  to  the  moment  of  one’s  own 
speech  enables  one  to  treat  the  resulting  distinctions  as  hierarchical  oppositions,  in  which  one  term  belongs  to 
presence  and  the  logos  and  the  other  denotes  a fall  from  presence.  To  tamper  with  the  privilege  of  speech  would  be  to 
threaten  the  entire  edifice.  (Culler  1983) 

Culler  draws  attention  here  to  a final  key  concept  in  our  argument,  what  he  terms  ‘hierarchical  oppositions’, 
derived  from  Derrida’s  notion  of  bivalence.  In  Western  thought  and  reason,  Derrida  claims,  bivalence  has 
consistently  served  to  sustain  a central  conceptual  hierarchy  of  terms.  Western  philosophy  has  historically 
operated  through  a network  of  conceptual  hierarchies,  dependent  upon  the  function  of  binary  oppositions,  in 
which  one  pole  of  the  binary  opposition  is  always  privileged  over  the  other. 

In  a traditional  philosophical  opposition  we  have  not  a peaceful  co-existence  of  facing  terms  but  a violent  hierarchy. 
One  of  the  terms  dominates  the  other... occupies  the  commanding  position.  To  deconstruct  the  opposition  is  above  all, 
at  a particular  moment,  to  reverse  the  hierarchy.  (Derrida  1981) 

As  Culler  emphasized,  the  dominant  term  is  always  that  which  ‘belongs  to  presence  and  the  logos’.  Such  binary 
oppositions  underpin  traditional  Western  ideologies  and  value  structures  such  as  realism  in  philosophy, 
humanism  in  the  arts  and  positivism  in  science.  Examples  of  such  oppositions  would  be:  mind^ody, 
male/female,  white/black,  self/other,  nature/culture,  presence/absence,  theory/practice  and  so  on.  In  each  of 
these  cases,  the  latter  term  is  seen  as  a deviation  from  or  perversion  of  the  former.  As  Parker  suggests: 

The  hierarchical  ordering  is  a symptom  of  the  way  in  which  a text  assumes  the  normality  of  one  pole  of  the  opposition 
and  sees  the  other  as  simply  a negative,  a distortion,  a perversion  or  a parasite  of  the  original  concept.  Texts, 
consequently  cannot  be  viewed  as  possessing  a structure  which  is  natural,  value-neutral,  uniform.  Instead,  the  textual 
structure  is  an  economy  in  which  the  currency  of  a concept  is  determined  by  its  place  among  other  concepts  within  a 
system  of  value  exchange  and  difference.  (Parker  1997) 
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Implications  for  teaching  and  learning  using  CMC 


If  we  consider  the  discourse  of  education  in  textual  terms  it  is  interesting  to  speculate  what  might  be  the  binary 
oppositions  that  would  emerge  and  what  normality  would  be  ‘assumed’,  in  Parker’s  phrase.  A tentative 
conceptual  hierarchy  might  include  the  following: 


• teaching/leaming 

• tutor/student 

• research/teaching 

• education/training 

• face-to-face/distance-mediated 

• synchronicity/asynchronicity 


• speech/writing 

• author/reader 

• cognitive/affective 

• individual/collective 

• disciplinar(it)y/interdisciplinarity 

• singularity/multiplicity 


• local/global 

• institutional/personal 

• rationality/irrationality 

• competition/collaboration 

• unity/fragmentation 


Again,  it  is  reasonable  to  assume  that  dominance  and  presence  reside  in  the  former  terms.  It  is  not  surprising, 
for  example,  to  find  that  in  quantitative  research  into  face-to-face  tutorial  group  interaction,  the  amount  of 
teacher  talk  massively  outweighs  that  of  student  talk  (Foster  1981). 

We  can  only  conclude  that  a dispensation  such  as  that  above  is  not  only  the  product  of  an  existing  realist- 
positivist  academic  culture  but  also  serves  to  sustain  such  a culture.  Traditional  academic  institutions  that 
maintain  such  hierarchical  bi valences  as  teaching/leaming,  tutor/student,  author/reader  will  always  favour 
face-to-face  modes  of  learning,  not  only  because  such  a mode  privileges  the  role  of  teacher  over  taught  and 
institution  over  person  but  because  ‘realism  is  situated  as  having  a phonocentric  bias  which  is  symptomatic  of  a 
deeper  commitment:  logocentrism  or  the  metaphysics  of  presence.’  (Parker  1997) 

George  Landow  (1997)  has  commented  on  the  way  in  which  CMC  and  other  new  learning  technologies  pose  a 
challenge  to  traditions  which,  as  we  have  attempted  to  show,  have  their  basis  in  logocentrism  and  the 
metaphysics  of  presence,  through  enabling  and  indeed  requiring  ways  of  working  which  are  traditionally 
resisted  in  the  academic  context.  Studying  in  the  electronic  arena  encourages  collaborative  rather  than 
competitive  approaches,  the  violation  of  disciplinary  boundaries  and  the  re-thinking  of  the  tutor/student 
relationship. 


Just  as  in  the  literary  domain  Roland  Barthes  famously  announced  in  1967  the  Death  of  the  Author,  it  may  well 
be  that  reports  of  the  Death  of  the  Tutor  have  not  yet  been  sufficiently  exaggerated  as  the  new  medium  replaces 
this  hierarchical  role  with  the  more  democratic  one  of  moderator/facilitator  who  acts  as  co-leamer.  But  the 
online  moderator  needs  to  acknowledge  that  identities  created  online  may  well  differ  from  face-to-face 
identities,  whilst  remaining  equally  valid.  Turkle  (1995)  provides  lucid  insights  into  the  creativity,  energy, 
irony  and  playfulness  of  online  identity  and  behaviour. 

The  moderator  role  in  CMC  assumes  critical  importance  in  this  respect.  As  online  writing  allows  for  the 
possibility  of  much  greater  freedom  and  elaboration  of  identity,  the  unified  self  assumed  within  face-to-face 
encounters  may  fragment  into  many  facets  or  faces  in  online  environments.  The  loss  of  one  face  may  permit  the 
emergence  of  several.  As  Baltz  has  pointed  out: 

instead  of  identity  having  the  status  of  an  initial  given  (with  which  the  communication  usually  begins),  it  becomes  a 

stake,  a product  of  the  communication.  (Baltz  1984:185) 


To  have  a centre  is  important  but,  as  Derrida  has  commented  in  respect  of  the  notion  of  ‘de-centering’,  the 
centre  is  a function,  not  a being.  In  the  same  way,  online  moderation  is  a function  that  does  not  necessarily 
have  to  be  dominated  by  one  individual,  traditionally  the  tutor. 

At  a practical  level  of  operation  CMC  offers  an  unprecedented  opportunity  for  the  encouragement  of  the  active, 
empowered  reader/leamer.  CMC  and  related  hypertextual  technologies  resist  the  grip  of  logocentrism  in  a new 
way,  and  provide  opportunities  for  learners  (and  ‘tutors’)  to  work  explicitly  with  ideas  of  plurality  and 
multiplicity  that  have  been  historically  denied,  and  which  could  well  be  emancipatory  in  their  effects.  In  order 
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to  fully  explore  these  effects  we  need,  however,  to  change  our  perspective,  understanding  and  expectations  of 
educational  experience  and  purpose.  If  we  do  not,  face-to-face  modes  will  always  remain  dominant  - working 
within  the  existing  realist  paradigm  will  never  lead  to  a questioning  of  its  own  (occluded)  assumptions. 

But  the  alternative  will  be  challenging,  risky  and  difficult.  The  effects  of  post-structuralist  thought  are 
unsettling.  It  does  seem  to  threaten,  in  Culler’s  phrase,  ‘the  entire  edifice’,  much  in  the  manner  perhaps  in 
which  Darwinism  threatened  the  edifice  of  prevailing  Victorian  thought  and  values.  And  unsettling  the 
fundamental  values  of  our  students  - and  colleagues  - is  not  something  to  be  countenanced  lightly,  however 
educationally  well  intentioned.  We  do  not  underestimate  he  potentially  negative  aspects  of  online  interaction, 
nor  do  we  ignore  the  capacity  of  virtual  environments  to  be  colonised  in  the  interests  of  managerialist 
imperatives.  Our  intention  is  to  lay  emphasis  on  the  emancipatory  effects  of  CMC. 

The  strategy  of  a deconstructivist  approach  is  to  undertake  a reversal  of  hierarchical  opposition,  not  to  replace 
one  generalising  conceptual  hierarchy  with  another,  but  to  subvert  the  original  logocentric  stance  by  bringing 
various  realities  into  play.  CMC  is  ideally  suited  for  this  purpose.  In  similar  fashion  it  has  been  our  intention  in 
this  paper  to  provoke  debate  through  challenging  the  realist  perspective  by  drawing  it  into  play  with 
alternatives.  We  hope  that  colleagues  will  enter  into  the  spirit  of  plurality  we  wish  to  foster  by  reciprocating  the 
challenge. 
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Designing  Effective  Learning  Environments  for  Distance  Education: 
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Abstracts:  Rapidly  developing  technologies  do  not  themselves  create  more  interactive  distance 
education  learning  environments.  Investigations  of  teaching  practices  coupled  with  a review  of  the 
literature  have  led  us  to  establish  three  major  aims  to  guide  in  the  design  of  coursework  and  the 
selection  of  distance  education  technologies.  These  aims  are  to  promote  learner  ownership,  to  foster 
situated  learning,  and  to  encourage  collaborative  problem  solving.  This  article  describes  the 
different  types  of  distance  education  technologies  we  use  and  how  these  technologies  are  integrated 
to  meet  these  aims. 


The  Collaborative  Teacher  Education  Program  (CTEP)  at  Indiana  University  was  created  twelve  years  ago 
to  address  the  critical  shortage  of  special  education  teachers  in  Indiana  (Knapczyk  & Rodes  1995;  Knapczyk,  Rodes, 
& Brush  1994).  The  lack  of  fully  certified  teachers  in  many  communities  is  compounded  by  the  difficulty  teachers 
have  obtaining  university  training.  By  offering  coursework  via  distance  education,  CTEP  allows  in-service  teachers 
to  complete  the  entire  36-credit  hour  requirement  for  earning  special  education  licenses  in  their  communities. 

Over  the  years,  the  communication  technologies  used  in  CTEP  have  advanced  from  speaker  phones  and  the 
U.S.  Postal  Service,  to  computer-based  audiographics  and  fax  machines,  to  multi-point  videoconferencing,  web 
conferencing  and  e-mail.  New  technologies  and  increasing  capabilities  of  interactivity  are  providing  new  tools  for 
distance  education.  However,  the  advancement  of  technology  alone  does  not  automatically  guarantee  a more 
interactive  and  effective  learning  environment. 

Investigations  of  teaching  practices  coupled  with  a review  of  the  literature  associated  with  teacher 
education  have  led  us  to  establish  three  major  aims  to  guide  in  the  design  of  coursework  and  practicum  activities  and 
the  selection  of  distance  education  technologies.  These  aims  are  to  promote  learner  ownership,  to  foster  situated 
learning,  and  to  encourage  collaborative  problem  solving.  One  challenge  that  has  remained  constant  from  the 
beginning  of  CTEP  is  tailoring  the  capabilities  of  new  technologies  to  create  an  effective  learning  environment  that 
meets  these  aims.  In  the  next  section  we  describe  the  different  types  of  distance  education  technologies  we  use  and 
discuss  how  these  technologies  are  integrated  to  meet  our  program  aims. 


Distance  Education  Technologies 


Audiographics 

We  began  the  program  using  audiographics  to  support  in-class  communication  with  students. 
Audiographics  involves  two  telecommunications  links,  one  that  connects  computers  via  modems  and  a second  link 
that  provides  an  audioconferencing  medium  through  a normal  telephone  connection.  Although  audiographics  is 
usually  overlooked  in  the  rush  to  more  up-to-date  video  options,  its  low  cost  and  broad  delivery  capabilities  make  it 
still  viable  for  programs  that  emphasize  outreach  or  low  instructor-to-student  ratios  (Knapczyk  & Rodes  1995). 

One  real  advantage  we  found  when  using  audiographic  technology  is  that  the  limitations  of  a non-video 
format  forced  us  to  adopt  effective  teaching  strategies  that  are  often  overlooked  in  campus-based  courses.  Rather 
than  lecturing  over  the  speakerphone,  we  learned  to  encourage  the  trainees  at  the  remote  site  to  lead  discussions, 
explain  key  point  from  the  text,  and  take  part  in  small-group  activities  that  relate  to  the  projects  they  are  conducting 
in  their  classrooms.  To  facilitate  this  process,  we  had  trainees  take  turns  acting  as  on-site  coordinators,  and  we 
contacted  them  in  advance  to  help  them  prepare  to  lead  the  class.  We  also  found  it  very  important  to  include  specific 
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written  guidelines  for  any  class  activities  and  text  materials  that  are  closely  adapted  to  the  needs  of  the  course,  as 
there  is  little  opportunity  for  the  kind  of  lecture  that  takes  place  in  campus-based  courses  to  provide  background 
information. 


Two-Way  Videoconferencing  System 

Two  years  ago  we  began  offering  coursework  using  a large-group  videoconferencing  system  that  employs 
high-speed  telephone  lines  to  transmit  quality  video  and  audio.  Both  origination  and  remote  sites  have  large-screen 
monitors,  speakers,  and  built-in  cameras  and  microphones.  In  addition,  the  sites  also  have  computers,  document 
cameras,  and  videotape  units  for  displaying  graphics,  hard  copies  or  videos.  This  system  has  allowed  us  to  connect 
two  or  more  remote  groups  for  our  class  sessions. 

Videoconferencing  equipment  offers  the  obvious  advantage  of  trainees  and  instructors  being  able  to  see  one 
another  and  follow  visual  cues.  Seeing  the  instructors  makes  students  much  more  comfortable,  and  it  is  much  easier 
to  explain  issues  and  spot  problems.  Even  so,  we  have  made  a deliberate  effort  not  to  lose  the  insights  we  gained 
from  our  experience  with  audiographics.  We  still  have  many  student-centered  activities  and  discussions,  and  we 
involve  trainees  as  much  as  possible  in  running  the  class,  including  operating  the  video  equipment.  The  importance 
of  using  text  materials  that  are  functional  arid  closely  adapted  to  the  course  remains  the  same  with 
videoconferencing  system.  Although  our  trainees  can  now  see  us,  they  are  just  as  likely  as  students  using 
audiographics  to  become  restless  if  we  do  not  keep  them  actively  involved  in  class  activities. 


Web-Based  Conferencing  System 

Web-based  conferencing  provides  a powerful  internet  tool  for  creating  a more  interactive  and  less  isolated 
experience  for  distance  learners.  We  have  incorporated  a web-based  conferencing  system  into  CTEP  to  encourage 
interactions  among  students  across  training  sites  to  help  make  them  feel  more  a part  of  a class  group.  We  are  using 
an  asynchronous  web-based  conferencing  system,  called  Alta  Vista  Forum.  All  discussion  on  the  conferencing 
system  is  directed  toward  existing  course  content  and  practicum  projects:  rather  than  using  this  opportunity  to  create 
extra  work  for  learners,  we  use  it  to  help  improve  their  understanding  and  application  of  the  work  they  do  through 
collaborative  learning.  We  created  teams  of  4-5  students  across  sites  to  encourage  closer  ties  between  our  remote 
sites.  We  have  also  provided  a "chat  room"  and  "teachers'  lounge"  to  give  students  open  forums  for  socializing  with 
one  another  and  for  discussing  more  personalized  issues  and  concerns. 


Support  Technologies 

Simple  technologies  such  as  fax  machines  and  e-mail  are  among  the  most  reliable  and  flexible  tools  we 
have  for  communicating  with  distance  learners.  Both  of  these  technologies  give  us  continuous  access  to  our  trainees 
and  provide  for  rapid  asynchronous  communication  between  university  instructors  and  learners. 


Integrated  Use  of  Technologies  to  Meet  Program  Aims 


Fostering  Ownership  among  Learners 

A strong  sense  of  ownership  is  a critical  factor  for  learners  in  distance  education  settings.  Learners  who  are 
not  in  close  physical  proximity  to  their  instructors  must  often  work  more  independently  than  learners  in  traditional 
settings  (Kember  1995).  In  fact  many  distance  educators  attest  that  independence  and  a strong  sense  of  ownership 
are  essential  predictors  of  success  in  distance  education  learners  (Riddle  1994;  Wagner  & McCombs  1995).  But 
distance  education  must  also  cater  to  the  needs  of  students  who  may  not  have  such  a strong  natural  inclination  to 
ownership  and  self-sufficiency.  Instructors  and  instructional  designers  need  to  make  a deliberate  effort  to  promote 
and  encourage  ownership  among  students,  especially  for  those  who  may  be  used  to  teacher-directed  instruction 
where  they  make  few  choices  for  themselves. 
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To  encourage  ownership  in  CTEP,  we  emphasize  new  roles  of  instructors  and  learners  in  the  beginning  of 
the  semester.  We  use  trainees  as  student  coordinators  to  oversee  such  instructional  aspects  of  the  class  sessions  as 
working  out  grouping  arrangements  for  activities,  assisting  with  the  actual  course  instruction  and  duplicating  and 
passing  out  papers.  Further,  we  structure  class  sessions  so  learners  act  as  on-site  facilitators  to  lead  discussions  and 
present  material  from  the  text  rather  than  having  the  campus-based  instructors  do  these  tasks.  Since  we  have  more 
than  one  remote  sites  in  a live  class,  we  have  the  on-site  coordinators  ask  the  other  site  to  present  ideas  and 
examples  instead  of  us  directing  the  activity  in  order  to  promote  greater  communication  across  groups.  A typical 
class  period  involves  several  small-group  work  and  discussion  activities,  with  instructors  acting  as  consultants  more 
than  as  presenters.  We  have  found  that  adult  learners  bring  a wealth  of  professional  skill  and  experience  with  them 
to  their  classes.  Distance  education  can  capitalize  on  these  experiences  in  a manner  that  facilitates  the  mechanics  of 
course  delivery,  enriching  the  content  and  teaching  interactions  of  the  class  sessions  and  more  closely  involving 
teachers  in  their  professional  development. 


Promoting  Situated  Learning 

A major  criticism  of  many  current  pedagogies  is  that  they  are  too  abstract,  removed  as  they  are  from  real 
world  experience  (Brown,  Collins,  & Duguid,  1989).  An  important  concern  for  educators  is  the  degree  to  which 
classroom  learning  transfers  to  external  situations  in  which  the  application  of  knowledge,  skills,  and  attitudes  is 
appropriate  (Oliver  & Reeves  1996).  The  field  of  teacher  education  has  also  shared  this  similar  concern.  Many 
teacher  education  programs  have  no  effective  way  of  ensuring  that  students  learn  to  apply  course  concepts  to  their 
actual  teaching  situations  (Cole  1992;  Dessent,  1987).  Students  may  enroll  in  a practicum  or  student  teaching 
experience  after  their  course  work  is  complete,  and  may  work  in  another  teacher's  classroom  rather  than  a setting  of 
their  own. 

Distance  education  can  be  designed  to  offer  students  an  opportunity  to  work  on  authentic  tasks  in  a real- 
world  setting  in  which  students  construct  knowledge  for  themselves.  This  type  of  instruction  is  referred  to  as 
situated  learning  (Winn  1993),  because  the  process  of  constructing  new  knowledge  is  situated  in  meaningful, 
relevant  contexts.  To  promote  situated  learning  in  CTEP,  we  assign  practicum  projects  that  serve  as  a focus  of  the 
trainees'  learning  experiences.  We  structure  the  practica  and  academic  coursework  as  a single  unified  instructional 
activity  so  teachers  practice  new  methods  as  we  cover  them  in  the  classes.  They  learn  to  integrate  these  approaches 
with  their  existing  routines,  combining  their  new  skills  with  the  skills  they  already  have  to  form  increasingly 
complex  and  unified  teaching  practices.  At  each  stage  of  training,  teachers  learn  how  to  fully  integrate  theoretical 
concepts  into  teaching  practices  that  match  their  particular  job  situations. 

We  further  ensure  situated  learning  by  visiting  the  remote  sites  so  we  can  obtain  firsthand  information 
about  the  circumstances  in  which  the  trainees  teach.  Meeting  with  students  in  person  at  their  work  sites  helps 
instructors  better  understand  their  practicum  situation,  which  in  turn,  enables  instructors  to  give  detailed  and 
situation-specific  guidance  and  feedback  to  each  trainee.  By  combining  e-mail  and  fax  transmissions  with  site 
visitations,  we  are  able  to  bring  a high  level  of  supervision  not  just  to  their  understanding  of  course  concepts,  but  to 
their  actual  application  of  these  concepts  in  real-life  settings. 


Encouraging  Collaboration  among  Teachers  and  Facilitating  School  Change 

Research  in  teacher  education  has  shown  the  value  of  within-school  professional  team  building  and 
collaboration  among  teachers  (Friend  & Cook  1993;  Koehler  & Baxter  1997).  Fullan  (1991)  argued  that  school 
improvement  is  most  surely  and  thoroughly  achieved  when  teachers  engage  in  "frequent,  continuous  and 
increasingly  concrete  and  precise  talk  about  teaching  practice"  (p.  78)  with  other  teachers.  Today  each  special 
education  teacher  must  operate  as  part  of  a team  in  many  aspects  of  his  or  her  role  (Polloway  & Patton  1997). 
However,  traditional  models  of  teacher  preparation  usually  do  not  include  strategies  for  promoting  collaborative 
problem  solving  with  other  adults. 

Because  CTEP  is  field-based,  we  attract  teachers  who  work  in  the  same  schools  or  in  the  same  school 
district.  We  take  advantage  of  our  trainees’  shared  circumstances  by  incorporating  collaborative  team  building 
techniques  into  the  very  structure  of  our  coursework.  In  CTEP  collaborative  techniques  serve  both  as  means  to 
encourage  interaction  among  learners  in  a live  class  and  as  ends  to  help  them  collaborate  with  other  adults  at  their 
job  sites.  As  stated  by  Johnson  and  Johnson  (1987)  and  Slavin  (1990),  when  instructional  delivery  systems  are 
structured  to  foster  cooperative  learning,  learners  benefit  both  instructionally  and  socially. 


780 


The  technologies  we  use  help  us  structure  a variety  of  in-class  and  out-of-class  activities  in  which  teachers 
describe  their  teaching  experiences,  brainstorm  solutions  to  real-life  problem  situations,  and  help  one  another  apply 
and  adapt  the  course  concepts  to  their  job  circumstances.  For  example,  as  out-of-class  activities  we  are  using  web- 
based  conferencing  to  promote  collaborative  learning.  While  in-class  groups  are  formed  according  to  shared 
interests  or  circumstances  within  the  same  remote  site,  groups  for  web  activities  are  formed  across  multiple  sites  so 
teachers  can  benefit  from  an  even  broader  and  more  diverse  range  of  perspectives.  To  structure  these  discussions,  we 
require  students  to  participate  in  three  kinds  of  interaction:  posting  examples  from  their  projects,  discussing  course 
content,  and  consulting  with  peer  teams  to  generate  a fuller  range  of  ideas  for  their  field-based  projects.  In  each  of 
these  formats,  the  asynchronous  environment  gives  trainees  the  opportunity  to  review  other  teachers'  work,  evaluate 
their  comments  and  develop  their  own  understanding  of  course  concepts  and  procedures. 

We  have  found  that  eliciting  a quality  discussion  on  the  web  requires  careful  preparation  and  being 
cautious  about  when  to  ask  learners  to  post  examples  of  their  work.  Instructors  must  decide  whether  to  post  models 
of  assignments  and  ask  trainees  to  give  only  appropriate  examples,  or  to  have  all  learners  post  work-in-progress  and 
then  attach  instructors'  comments  so  learners  do  not  accidentally  consult  nonexamples.  Instructors  also  need  to  make 
sure  that  discussion  topics  are  content-based  and  that  the  outcomes  of  the  activities  are  reasonably  beneficial  to  all 
trainees;  the  topics  should  not  be  too  abstract,  general,  or  contrived.  In  addition,  the  way  learners  are  supposed  to 
reply  to  each  other  also  must  be  clearly  specified  so  trainees  do  not  just  post  short  "I-agree-with-you"  type 
comments  to  fulfill  the  requirement.  Furthermore,  instructors  should  also  be  careful  that  learners  do  not  go  too  much 
ahead  of  the  course  schedule  and  suggest  ideas  solely  based  on  their  own  teaching  habits,  which  might  not  be 
consistent  with  the  goals  of  the  course.  Finally,  like  any  other  learning  activity,  instructors  must  monitor  and 
evaluate  web  activities  and  provide  feedback  to  the  class,  if  not  to  each  learner. 


Conclusion 

Rapidly  developing  technologies  do  not  themselves  create  more  interactive  distance  education  learning 
environments.  One  challenge  that  has  remained  constant  from  the  beginning  of  our  program  is  adapting  new 
technologies  to  match  our  instructional  aims.  Distance  education  has  traditionally  been  more  successful  with 
students  who  exhibit  a strong  sense  of  ownership:  students  who  are  independent  and  highly  motivated.  In  CTEP  we 
have  attempted  to  structure  course  and  practicum  activities  to  build  this  type  of  ownership  in  students  with  all  types 
of  learning  characteristics.  We  have  also  encouraged  a level  of  collaboration  in  which  trainees  can  take  advantage 
of  the  wealth  of  experiences  each  brings  with  them  into  the  program.  In  CTEP  collaborative  techniques  serve  both  as 
means  to  encourage  interaction  among  learners  in  a live  class  and  as  ends  to  help  them  become  more  effective 
teachers  at  their  job  sites. 


References 

Brown,  J.,  Collins,  A.,  & Duguid,P.  (1989).  Situated  cognition  and  the  culture  of  learning.  Educational  Researcher,  18  (1),  32- 
41. 

Cole,  R.  (1992).  The  land  of  the  little  red  schoolhouses.  Educational  Horizons.  70.55-59. 

Dessent,  T.  (1987).  Making  the  Ordinary  School  Special.  London:  Falmer. 

Friend,  M.  & Cook,  L.  (1992).  Interactions:  Collaboration  Skills  for  School  Professionals.  New  York:  Longman. 

Fullan,  M.  (1991).  The  new  meaning  of  educational  change.  New  York,  NY:  Teachers  College  Press. 

Johnson,  D.,  & Johnson,  R.  (1987).  Learning  together  and  alone  2nd  ed.).  Englewood  Cliffs,  NJ:Prentice  Hall. 

Kember,  D.  (1995).  Open  learning  courses  for  adults:  A model  of  student  progress.  Englewood  Cliffs,  NJ:Educational 
Technology  Publications. 

Knapczyk,  D.  & Rodes,  P.  (1995).  Effecting  change  in-service  teachers  through  distance  education.  Technology  and  Teacher 
Education.  3.  47-55. 


781 


Knapczyk,  D.,  Rodes,  P.  & Brush,  T.  (1994).  Improving  staff  development  in  rural  communitiesusing  distance  education  and 
communication  technology.  Rural  Special  Education  Quarterly.  13.  19-24. 

Koehler,  M.  & Baxter,  J.  (1997).  Leadership  through  Collaboration:  Alternatives  to  the  Hierarchy.  Larchmont,  NY:  Eye  on 
Education  Inc. 

Langer,  J.A.  & Applebee,  A.N.  (1986).  Reading  and  writing  instruction:  Toward  a theory  of  teaching  and  learning.  In  E.  Rothkof 
Eds.).  Review  of  Research  in  Education.  13.  171-194. 

Oliver,  R.  & Reeves,  T.C.  (1996).  Dimensions  of  effective  interactive  learning  with  telematics  for  distance  education.  ETR&D. 
44.  45-56. 

Polloway,  E.A.,  & Patton  J.R.  (1997).  Strategies  for  teaching  learners  with  special  needs.  Upper  Saddle  River, NJ:  Prentice  Hall. 

Riddle,  J.F.  (1994).  Factors  contributing  to  achievement  and  course  satisfaction  of  distance  students.  Greeley,  CO:University  of 
Northern  Colorado.  Unpublished  doctoral  dissertation. 

Slavin,  R.  (1990).  Cooperative  learning:  Theory,  research,  and  practice.  Englewood  Cliffs,  NJ:Prentice  Hall. 

Wagner,  E.  D.  & McCombs,  B.L.  (1995).  Learner  centered  psychological  principles  in  practice:  Designs  for  distance  education. 
Educational  Technology.  March-April. 32-35. 

Winn,  W.  (1993).  Instructional  design  and  situated  learning:  Paradox  or  partnership?  Educational  Technology,  March. 


A Web-based  study  of  students’  attitudes  towards  the  Web 


Thao  Le 

School  of  Education 
University  of  Tasmania 
Australia 
T,Le@utas, edu.au 

Quynh  Le 

University  Department  of  Rural  Health,  Tasmania 
University  of  Tasmania 
Australia 

Qu  ynh . Le  @u  tas  .edu.au 


Abstract:  While  experts  in  educational  multimedia  generally  welcome  the  Web  in  education,  it  is 
important  to  take  into  account  the  views  of  learners  about  the  role  of  the  Web  in  teaching  and 
learning,  as  they  are  the  primary  participants  in  the  Web-based  educational  experience.  This 
paper  discusses  a Web-based  study  on  learners’  views  of  the  Web.  The  discussion  starts  with  an 
examination  of  the  emergence  of  the  Web  in  education  and  some  key  features,  which  represent  a 
shift  to  a new  educational  paradigm.  However,  what  Web  specialists'  value  may  not  be  necessarily 
accepted  by  learners.  Therefore,  it  is  important  to  examine  learners’  view  in  relation  to  the  Web 
impact.  The  second  part  of  the  paper  deals  with  a study  using  a Web-based  research  approach  to 
investigate  the  attitudes  of  university  students  towards  the  role  of  the  Web  in  teaching  and 
learning. 


Introduction 

The  emergence  of  the  Web  in  education  has  generated  a great  deal  of  interest  among  theorists  as  well 
practitioners.  For  many  Web  enthusiasts,  the  Web  has  challenged  the  traditional  strict  curriculum  control  approach  in 
which  learning  is  at  the  mercy  of  teaching.  The  Web  can  be  used  to  foster  three  significant  aspects  of  learning: 
independent  learning,  creative  learning,  and  flexible  learning.  While  it  is  generally  accepted  that  the  Web  is  an  important 
facility  for  teaching  and  learning  in  a modem  world,  its  role  in  education  can  be  questioned  from  different  fronts.  From  the 
critical  theory  perspective,  for  instance,  the  Web-based  teaching  may  easily  commit  the  same  ideological  ‘crime’  as  other 
conventional  teaching  methods:  cultural  insensitivity,  misempowerment,  mistreatment  of  learners  as  virtual  beings. 

It  is  important  to  take  into  account  the  views  of  learners  about  the  role  of  the  Web  in  teaching  and  learning,  as 
they  are  the  primary  participants  in  the  Web-based  educational  experience.  This  paper  discusses  a qualitative  study,  which 
investigated  how  learners  valued  the  Web  educationally.  The  paper  will  start  with  an  examination  of  the  emergence  of  the 
Web  in  education  and  some  key  features,  which  represent  a shift  of  educational  paradigm.  The  follow-up  discussion  will 
present  a study  using  a Web-based  research  approach  to  investigate  the  attitudes  of  students  towards  the  role  of  the 
Internet,  mainly  the  Web,  in  teaching  and  learning. 

The  Emergence  of  Web-based  Teaching 

The  Web  has  become  a world  phenomenon.  It  has  permeated  many  aspects  of  the  modem  world,  particularly  in 
business  and  education.  The  10th  World  Conference  on  Educational  Multimedia  and  Hypermedia  held  in  Freiburg, 
Germany  in  June  1998  attracted  over  1000  participants  from  46  countries,  with  a total  of  485  papers  and  130  posters.  This 
is  one  of  many  Web-related  seminars  and  conferences  around  the  world  in  the  same  year.  The  cliche  ‘www  dot’  has 
appeared  constantly  in  many  TV  ads  and  public  announcements.  The  Web  has  created  its  own  culture  with  its  own 
language  and  its  own  people  with  their  discourse  community.  In  the  educational  context,  the  Web  has  provided  a new 
platform,  which  stimulates  a new  spirit  and  practice  of  teaching  and  learning.  In  the  context  of  multimedia,  as  Fisher 
(1994)  points  out,  the  real  appeal  is  in  more  than  the  ability  to  merge  video  and  computer  technology  in  a single 
application,  although  that  has  provided  the  excitement  that  drives  multimedia  research  and  product  development.  The 
hypertext  aspect  of  multimedia  provides  the  underlying  informational  motive  force  that  stands  to  revolutionise  the 
presentation  of  information.  Hyperlinks,  automatic  searches,  and  the  ability  to  connect  pieces  of  information  in  a network 
or  web  of  knowledge  offer  new  capabilities  to  information  developers  using  multimedia,  but  the  added  complexity  offers 
new  challenges  as  well  (Fisher  1994). 
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The  Web  with  a New  Educational  Paradigm 


Roberts  (1997)  describes  a paradigm  shift  caused  by  the  introduction  of  the  Internet  in  education.  He  says: 

In  the  field  of  education,  it  is  argued  that  we  are  undergoing  a ‘paradigm  shift’;  the  rejection  of  one  set  of  values 
and  ideas  about  education  and  the  adoption  of  a new  set  with  regards  to  what  constitute  effective  pedagogy.  This 
paradigm  shift  is  said  to  be  occurring  world-wide  but  faster  in  some  parts  than  others  depending  on  the  availability 
of  resources,  existing  infrastructure  and  the  stage  of  development  reached. 

(Roberts,  1997,  p.380) 

The  emergence  of  the  Web  in  education  can  be  viewed  in  three  phases.  Broadly  speaking,  a teacher  or  learner 
who  becomes  a member  of  a Web  discourse  conununity  tends  to  go  through  three  stages,  which  can  be  described  in  terms 
of  the  relationship  between  the  Web  and  learning:  learning  about  the  Web,  learning  the  Web  and  learning  via  the  Web, 

• Learning  about  the  Web:  This  is  the  initial  stage  of  getting  to  know  what  the  Web  is  all  about,  A decade  ago,  the 
World  Wide  Web  was  not  in  existence,  A few  years  later,  the  Web  metaphorically  became  a street  directory  for 
commuters  on  the  superhighway  embedded  in  a complex  network  of  electronic  interaction  like  a spider  web. 

• Learning  the  Web:  The  Web  is  not  a terrifying  land  where  novices  are  tortured.  It  can  be  used  and  enjoyed  by 
computer  experts  as  well  as  lay  people.  Web-  oriented  devices  are  produced  constantly  to  ensure  user-friendliness. 
Learning  the  Web  can  range  from  learning  complex  tasks  with  the  use  of  Java  technology  to  create  powerful  ‘web- 
ware’  to  learning  how  to  navigate  on  the  superhighway  simply  by  ‘pointing  and  clicking’.  In  other  words,  the  Web 
can  be  a great  challenge  for  Web-based  software  developers  as  well  as  a relaxing  tool  for  Internet  travellers. 

• Learning  via  the  Web:  This  is  the  most  crucial  stage  of  Web  orientation.  The  Web  provides  a special  key  to 
education:  Teaching  and  learning  can  take  place  in  a virtual  reality.  The  traditional  face-to-face  teaching  in  a 
classroom  is  no  longer  the  only  way  of  teaching.  Learners  are  no  longer  confined  to  their  seats  in  front  of  a teacher. 
They  do  not  have  to  wait  for  the  time  when  a library  is  open  to  search  for  information. 

The  Web  has  become  a challenge  to  traditional  face-to-face  teaching.  Its  emergence  in  educational  theory  and 
practice  has  been  marked  by  the  following  Web  phenomena:  Web-based  learner,  virtual  class,  and  courseware. 

Web-based  Learners 

In  the  current  context  of  teaching  and  learning,  lecture  in  its  tradition  sense  does  not  meet  the  demand  of 
learners  as  lecture  can  only  function  in  a very  limited  context.  Lectures  are  traditionally  the  kernel  of  university  education. 
This  traditional  form  implies  that  everybody  who  wants  to  attend  has  to  be  in  the  lecture  room  at  lecture  time.  If  the  target 
group  for  the  lecture  is  distributed  over  a larger  area,  attending  lecture  can  often  be  time-consuming  or  even  impossible 
(Anderl  & Vogel  1998).  In  a narrow  traditional  approach  to  teaching,  students  are  normally  treated  as  passive  learners. 
Their  minds  are  treated  as  empty  vessels  to  be  filled  with  knowledge.  Teachers  are  knowledge  transmitters  and  are 
therefore  the  primary  resource  for  learning.  Learners  are  taught  how  to  learn.  This  notion  of  learning  has  been  challenged 
on  the  grounds  that  learning  is  not  to  receive  knowledge  but  to  make  sense  of  knowledge  and  to  promote  in  a learner  an 
independent  mind,  which  can  inform,  reflect,  and  even  challenge  the  conventional  knowledge  and  wisdom.  The  Web  is 
claimed  to  be  one  of  the  most  powerful  tools  for  providing  teachers  and  learners  with  necessary  conditions  for  independent 
and  interactive  learning.  It  provides  an  educational  discourse  in  which  learners  can  interact  widely  with  other  members  of 
a learning  community  at  the  same  time  learners  are  in  control  of  their  own  learning.  Their  interaction  for  learning  can  be 
immediate,  prompt,  widely  shared  and  resource-supportive  and  this  may  not  be  possible  in  a traditional  mode  of  teaching 
in  which  teachers  and  students  are  heavily  constrained  by  the  physical  condition  of  a classroom. 

Virtual  Class 

The  concept  ‘virtual  reality’  has  generated  generic  concepts  such  as  virtual  course,  virtual  university,  virtual 
conference  etc.,  which  have  become  realities  in  current  academic  activities  around  the  world.  Since  the  world  has  become 
a global  village  in  which  villagers  communicate  quite  conveniently  via  the  Internet,  education  is  no  longer  confined  to  a 
group  of  local  students  coming  from  a community  in  which  a university  or  school  is  located.  Universities  have  offered 
courses  for  students  of  diversified  geographical  and  cultural  backgrounds.  These  courses  were  originally  offered  in  an  off- 
campus  mode  in  which  students  received  course  packages  via  mail  such  as  course  materials,  textbooks  and  video  tapes  and 
they  participated  in  telephone-based  discussions,  Aoki,  Fasse  and  Stowe  (1998)  pointed  out  that  virtual  universities  are 
identical  to  traditional  universities  in  the  traditional  sense.  The  key  difference  is  that  virtual  universities  do  not  require 
students  to  commute  to  campus  and  to  physically  attend  classes.  The  rapid  development  of  multimedia  and  hypermedia  has 
provided  universities  with  information  technology  which  has  transformed  the  traditional  off-campus  teaching  into  virtual 
teaching  in  which  interaction  is  not  only  much  quicker  but  there  is  great  improvement  of  the  quantity  and  quality  of  course 
delivery. 
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Courseware 

The  concept  ‘courseware’  designates  the  method  of  course  delivery  via  the  Internet.  A virtual  class  needs 
courseware,  which  guides  and  facilitates  students’  learning.  Aedo,  Diaz  and  Montero  (1998)  argue  that  ‘a  flexible  CAL  system 
should  provide  teachers  with  mechanisms  to  adapt  the  courseware  to  their  own  methods  as  well  as  their  students’  needs’.  A 
courseware  may  include  traditional  reading  materials.  However,  the  fundamental  difference  is  that  it  liberates  learners  from  the 
traditional  linear  mode  of  learning.  Fisher  (1994)  pointed  out  that  computers  have  given  users  a power  not  widely  understood 
till  now;  the  power  to  break  the  enforced  linearity  of  information,  if  not  of  time.  The  computer’s  ability  to  search  far  more 
quickly  than  any  human  can  do  makes  possible  an  almost  seamless  presentation  of  information  that  can  come  from  sources  that 
are  not  only  diverse,  but  that  are  invisible  - and  ultimately  of  no  concern  - to  the  user.  There  is  no  need  to  organise  vast 
quantities  of  information  in  a strictly  linear  manner.  The  problem  now  is  learning  to  think  in  such  a way  that  this  power  can  be 
put  to  use  (Fisher  1994).  However,  there  is  still  a danger  that  the  old  transmission  model  of  one-way  instructional  delivery  will 
be  repeated,  ignoring  the  importance  of  students’  interaction  and  leaving  students  autonomous  and  isolated.  Teachers  can  use 
the  Web  to  place  their  course  materials  for  students’  retrieval,  but  still  the  Web  may  be  a primitive  media  for  creating  an 
interactive  learning  environment  (Aoki  et  al.  1998). 

Learners’  Evaluative  View  of  the  Web 

One  of  the  big  shifts  in  educational  theory  and  practice  is  the  focus  on  learning  rather  than  teaching.  This  shift  has 
been  reflected  by  the  recent  emergence  of  innovative  concepts  such  as  learner-centred,  independent  learning,  process-oriented, 
negotiated -curriculum  etc.  The  great  attention  given  to  constructivism  in  educational  multimedia  in  the  past  decade  has 
reinforced  the  premise  that  learners  are  the  key  players,  not  passive  receivers,  in  the  educational  process.  Teaching 
methodology,  learning  environment  and  teaching  resources  are  critically  considered  to  ensure  that  the  new  paradigm  is  not  just 
‘old  wine  in  new  bottle’.  However,  curriculum  innovation  cannot  meaningfully  take  place  unless  learners  are  allowed  to  play  a 
part  in  the  innovation  process.  The  obvious  step  to  be  taken  is  to  seek  the  views  of  learners  on  teaching  and  learning  as  they  are 
‘real  experiencers’,  not  virtual  beings.  Their  feelings  and  thoughts  on  teaching  and  learning  should  be  the  basis  for  curriculum 
innovation.  In  the  context  of  Web-based  education,  the  logical  step  is  to  conduct  a study  to  find  out  learners’  perception  of  the 
Web  and  its  role  in  education. 

Web-based  Discussion  Board 

A big  advantage  of  the  qualitative  over  quantitative  approach  is  that  learners  were  encouraged  to  express  as  much  as 
possible  and  as  freely  as  possible  their  views  on  the  use  of  the  Web  in  teaching  and  learning.  To  achieve  this,  a number  of 
discussions  were  conducted  using  the  Web  as  a free  and  convenient  forum  for  learners  to  voice  their  own  opinions.  A Web- 
based  discussion  board  was  created  with  the  following  features  to  ensure  confidentiality  and  effectiveness. 

• Participants  were  not  required  to  identify  their  names  and  email  addresses. 

• Pseudonyms  were  encouraged  to  use  to  promote  interpersonal  communication.  For  example,  addressing  the  other 
participants  personally  as  ‘Jim’,  ‘Puzzled’,  ‘ T&T’  etc  instead  of  ‘the  previous  writer’,  ‘the  first  person  in  the  discussion’. 

• Informal  discussion  was  the  nature  of  interaction.  Participants  were  encouraged  to  write  down  what  came  into  their  heads. 
They  did  not  have  to  be  concerned  about  grammar  and  spelling. 

Discussion  Format  and  Data  Gathering 

The  data  gathering  was  conducted  as  a normal  learning  activity  in  a university  context..  Undergraduate  and 
postgraduate  students  were  invited  to  participate  via  the  Discussion  Board.  Invitations  were  informally  extended  to  students 
through  their  teachers  in  lecture  times,  notices  on  notice  boards,  and  in  public  places  such  as  canteens  and  sport  events. 
Education  students  were  particularly  targeted  as  they  were  familiar  with  Web-based  teaching  and  they  were  introduced  to  a 
courseware  on  communication  as  a component  of  their  course. 

There  were  three  discussion  topics  covering  four  different  aspects  relating  to  the  theme  ‘ the  Web  and  its  role  in 
education’.  Each  topic  started  with  a discussion  statement,  which  aimed  at  stimulating  responses  from  learners.  The  three 
discussion  statements  were: 

Topic  1:  The  role  of  the  Web  in  education 

The  Web  has  been  introduced  to  teaching  and  learning.  Some  people  are  very  excited  at  the  prospect  that  the  Web 
will  not  be  just  another  added  tool  to  education  but  also  present  a significant  alternative  to  the  traditional  face-to-face 
teaching  and  learning;  while  others  are  either  unenthusiastic  or  sceptical  about  its  educational  significance.  What  do 
you  think?  Please  share  your  views  on  this  issue? 

Topic  2:  Critical  Educational  Web 

Critical  theory  stresses  the  importance  of  critical  examination  of  the  nature  of  the  discourse  or  the  context  in  which 
teaching  and  learning  take  place.  Such  critical  examination  may  reveal  problems  in  education  such  as  ideological 
imposition,  misempowerment,  imperialism,  cultural  insensitivity  etc.  Some  people  argue  that  the  Internet  or  the  Web 
can  cause  such  problems.  What  do  you  think?  Please  share. 

Topic  3:  Learners  in  courseware  development 
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Web“based  teaching  in  the  form  of  courseware  has  been  introduced  to  tertiary  education.  Courseware  is  often 
constructed  jointly  by  subject  specialists  and  Web  specialists  to  teach  an  on-line  course  on  the  Web.  Some  people 
argue  that  learners  should  be  involved  in  the  development  and  evaluation  of  Web-based  courseware  while  others 
argue  that  learners’  involvement  is  not  necessary  or  impractical  and  we  should  have  good  faith  in  experts.  What  do 
you  think? 

Data  Analysis  and  Results 

The  use  of  the  discussion  board  in  this  study  was  to  identify  the  meanings  and  concepts  underlying  learners’  attitudes 
toward  the  use  of  the  Web  in  education.  Therefore  the  data  collected  from  the  discussion  board  were  analysed  in  terms  of 
conceptual  issues  which  could  be  grouped  under  three  headings:  positive  aspects,  negative  aspects  and  suggestions. 

Positive  Aspects 

• The  Web  as  a resource 

• Non-threatening  condition 

• Choice  and  independence 

• Interactivity  with  people 

• Planning  for  the  Web 

Negative  Aspects 

• No  virtual  learners 

• Ideological  bias  loaded 

• Imperialism/  commercialism 

• Unnatural/false  pretending 

Suggestions 

• Experts  & learners’  contribution  to  courseware 

• Learner  as  partners 

Discussion  Based  on  the  Conceptual  Issues 

One  of  the  advantages  of  a qualitative,  approach  is  that  the  focus  is  uncovering  the  meanings  (concern,  inspiration, 
mind-puzzling  etc)  which  could  not  be  captured  easily  in  a quantitative  study.  These  meanings  are  not  necessarily  in  the 
imagination  of  the  researcher  when  the  research  is  planned  and  carried  out.  The  following  conceptual  issues  reflect  strongly  the 
meanings  underlying  the  attitudes  of  the  respondents. 

Positive  Aspects 

Learners  viewed  the  Web  as  a powerful  resource,  an  inspiring  teacher  and  an  innovative  device,  which  gives  learners 
motivation,  responsibility  and  independence  in  learning.  The  Web  provides  a dynamic  discourse,  which  could  not  be  replaced 
by  the  traditional  face-to-face  teaching.  Learners  can  travel  virtually  in  a world  of  knowledge  and  interaction.  This  positive 
evaluation  could  be  demonstrated  in  the  following  selected  responses  from  different  learners: 

- The  Web  is  a friendly  forum.  It  is  real  to  me,  not  artificial  at  all,  no  way! 

- 1 feel  more  relaxed  to  discuss  via  the  Web,  It 's  fun.  You  don  7 feel  as  if  someone  would  laugh  at  you.  You  have  more 
time  for  thinking  properly. 

- On  the  Internet,  I have  the  choice  of  selecting  which  issues  to  discuss  and  whom  to  discuss  with.  This  is  very 
important  to  me.  I could  not  easily  do  this  in  a face-to-face  situation.  If  you  don  7 think  this  is  real  interaction,  what 
else  then!!!! 

- We  can  exchange  our  views  on  various  issues  and  aspects  of  life.  We  can  chat  on  many  things,  which  are  of  interest  to 
us.  You  don't  have  to  'talk'  if  you  don't  feel  like  it.  We  talk  about  travel,  cooking,  politics,  literature,  science.... If  you 
think  this  is  not  learning,  what  is  it  then? 

Negative  aspects 

Some  learners  were  not  quite  enthusiastic  or  optimistic  about  the  role  of  the  Web  in  education  while  others  accepted 
the  contribution  of  the  Web  in  education  but  also  pointed  out  some  potential  problems.  Attention  was  given  to  the  area  of 
‘imperialist  Web’  and  its  ideology-loaded  mission.  This  reflected  their  concern  about  the  whole  discourse  of  education  to  which 
the  Web  is  attached. 
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- In  face-tO‘face  discussion,  we  can  interact,  I mean  'real  ’ interact  together.  I enjoyed  real  group  discussion.  On  the 
Web,  all  the  personal  things  are  not  there  any  more.  Of  course,  I still  can  do  it  on  the  Web,  if  I have  to... but  still  it  is 
not  real  interaction,  is  it? 

- Everything  we  do,  think,  act  reflect  our  culture,  our  attitudes,  and  cultural  and  social  backgrounds.  How  could  we 
leave  all  these  behind  when  we  travel  on  the  Internet.  Thus,  teaching  and  learning  on  the  Web  cannot  be  immuned  from 
cultural  and  social  interference.  We  may  think  that  we  are  neutral.  I don  V think  so. 

- When  students  come  from  different  countries,  different  cultures,  there  must  be  some  misunderstanding, 
misinterpretation,  and  all  these  can  create  mistrust,  confusion  and  perhaps  conflict. 

- The  Internet  is  a product  of  a commercial  world.  It  is  commercially  motivated.  The  main  aim  of  commerce  is  to  make 
money.  It  is  interesting  to  see  how  education  with  the  use  of  the  Internet  can  be  free  from  commercial  influences  and 
controls?  We  need  to  look  beyond  the  surface  level  to  see  how  imperialism,  in  a new  form,  enters  other  human  cultures. 
I think  the  Internet  reflects  the  ideology  of  its  masters.  Just  look  at  hardware  and  software!  Where  do  they  come  from? 

Suggestions 

Apart  from  many  responses,  which  reflect  some  concern  as  well  as  admiration  for  the  Web  and  its  role  in  education, 
learners  also  offered  some  ideas  and  reservations,  which  should  be  taken  into  consideration  by  Web  instructional  designers  and 
teachers.  Issues  raised  in  their  responses  include  the  importance  of  partnership  in  Web-based  courseware  development, 
learning-centred  approach  of  Web-based  teaching,  and  learners’  feedback. 

- Navigation  is  an  important  feature  and  device.  Some  navigation  on  the  Web  are  so  well  thought  out  that  users  can 
expand  their  learning  and  curiosity  while  travelling  on  the  Www  due  to  excellent  design.  I have  come  across  many  Web 
sites,  which  are  technically  very  complicated,  but  educationally  they  are  hopeless 

- I don  7 think  that  we  should  allow  experts  to  control  our  destiny.  We  should  take  an  active  part  in  it. 

-'Learners  are  partners'?  As  partners,  they  must  be  involved  partially  or  wholly.  Of  course,  not  everyone  wants  to  be 
involved.  It  must  be  on  a voluntary  basis.  Forced  feedback  leads  to  superficial  results. 

- I do  well  in  subjects  when  lecturers  give  me  flexibility,  take  interest  in  what  I think.  My  feedback  is  always  sought  and 
valued.  With  this  approach,  I can  learn  so  much. 

The  Web  and  Face-to-face  Teaching 

The  Web  has  won  the  hearts  of  many  university  students  who  are  trained  to  be  teachers.  It  is  clear  from  their 
messages  that  they  have  made  use  of  the  Web  in  the  course  of  their  learning  at  the  university.  The  word  ’however’  was 
constantly  in  their  consciousness  when  they  discussed  about  the  role  of  the  Web  in  teaching  and  learning.  In  other  words,  there 
was  some  reservation  about  the  idea  that  the  Web  can  take  over  face-to-face  teaching  as  seen  in  the  following  two  responses 
from  students. 

• The  first  response  emphasises  the  significance  of  social  interaction  which  the  Web  could  not  adequately  provide. 

The  availability  of  the  Web  as  a classroom  tool  is  exciting!  It  is  not  only  a way  for  students  to  be  able  to  access  up-to- 
date  information,  but  for  teachers  as  well.  The  Web  is  loaded  with  teaching  ideas  and  lesson  plans  on  almost  any  topic 
you  can  think  of  and  there  are  of  course  the  'wonderous'  mailing  lists  we  all  learned  about  last  year  which  provide  the 
opportunity  for  contact  with  fellow  professionals.  However,  while  an  undeniably  useful  tool,  the  Web  for  me  is  not 
really  an  'absolute'  alternative  for  traditional  face-to-face  teaching.  Today's  society  demands  that  the  mastering  of  all 
types  of  personal  interaction  be  included  as  part  of  a school's  curriculum.  But  with  the  way  technology  is  heading,  the 
scary  thing  is  that  tomorrow's  society  may  not...? 

• The  second  response  emphasises  the  access  problem  facing  students  and  teachers. 

I don't  think  the  Internet  will  be  used  to  replace  teachers,  but  they  will  become  more  important,  especially  when  trynng 
to  get  up-to-date  information.  I also  think  that  even  though  the  web  has  a lot  of  things  to  offer  teachers  and  students 
alike,  it  also  requires  a fair  bit  of  time  to  access  and  find  the  relevant  information  you  need.  Some  teachers  don't  have 
that  time  with  all  the  other  requirements  they  have  and  this  might  be  another  reason  why  they  hesitate  to  use  the 
Internet  in  their  classrooms. 

The  following  response  raised  the  problems  of  Web  control  and  Web  abuse  in  the  class. 

Although  the  web  is  an  exciting  new  introduction  into  classrooms,  I also  believe  that  in  some  cases  many  students  use  it 
for  their  own  entertainment  purpose.  This  is  fine  if  not  in  school  and  class  time.  But  some  of  the  material  that  these  kids 
are  getting  hold  of  is  just  not  suitable.  The  teachers  and  parents  of  these  children  should  make  sure  that  they  have 
suitable  blocking  benefits  such  as  net  nanny  or  similar  programs  so  that  these  sites  are  not  available  to  younger 
children. 

Conclusion 

The  Web  has  been  introduced  to  education  and  its  role  has  been  subject  to  critical  discussion  by  educational 
multimedia  experts  as  clearly  indicated  by  various  papers  presented  at  this  conference  and  other  similar  conferences  in  the 
world.  It  is  important  for  us  to  share  our  exciting  experiences  brought  to  us  by  educational  multimedia.  We  may  expect  that  the 
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superhighway  will  bring  more  educational  innovations  to  the  world.  However,  the  most  important  thing  for  us  to  remember  is 
that  the  journey  on  this  educational  superhighway  should  include  learners  as  our  travelling  partners.  As  one  of  the  learners  in 
this  study  has  inspired  us  with  the  following  statement:  ‘Virtual  teaching  is  a fascinating  growing  area  and  I think  we  should 
support  it.  Support  does  not  mean  to  learners  are  like  obedient  sheep  which  know  nothing  and  are  at  the  mercy  of  their  master. 
They  should  be  consulted  if  learning  takes  place  meaningfully.’ 
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Abstract:  The  Web,  today,  is  a firmly  established  (virtual)  reality.  A few  years  after  its 
impressive  break  through,  from  limited  professional  circles  to  everybody’s  working  and 
social  life,  the  Web  constitutes  an  additional  space  for  people  to  communicate,  work,  trade  or 
spend  leisure  time.  And  increasingly,  too,  a place  to  learn.  We  have  currently  reached  a 
stage  at  which  a mapping  of  Web-based  learning  environments  (WBLE)  is  required  to 
answer  crucial  questions  at  the  content,  teaching,  learning  and  communication  levels.  To 
relate  to  these  questions  we  developed  a Taxonomy  of  WBLE’s,  implemented  it  for  the  study 
of  about  500  educational  Websites,  and  elaborated  on  practical  implications  of  the  study’s 
results.  The  overall  picture  we  have  unveiled  may  sound  deceptive,  and  can  be  summarized 
as  ‘'one  step  ahead  for  the  technology,  two  steps  back  for  the  pedagogy”.  But  a more 
thoughtful  consideration  of  the  results  suggest  directions  for  the  further  development  of  novel 
Web-based  educational  models. 


The  Web,  today,  is  a firmly  established  (virtual)  reality.  A few  years  after  its  impressive  break  through, 
from  limited  professional  circles  to  everybody’s  working  and  social  life,  the  Web  constitutes  an  additional 
space  for  people  to  communicate,  work,  trade  or  spend  leisure  time.  And  increasingly,  too,  a place  to  learn 
(Berenfeld,  1996). 

Within  the  global  network,  already  an  unknown  quantity  of  educational  Websites  of  various  kinds  has 
been  developed,  and  their  number  is  daily  growing.  These  Web-based  learning  environments  (WBLE)  reflect 
educators’  attempts  to  wrap  together  Web  technology  features  (e.g.,  information  manipulation, 
communication,  and  creation  tools)  according  to  their  educational  and  pedagogical  beliefs  in  pursue  of 
learning  goals.  The  sites  differ  from  each  other  for  instance  in  terms  of  the  identity  of  their  originators  (e.g., 
teachers,  students,  development  centers,  research  centers)  and  their  goals  and  motivations,  in  regard  to  subject 
matter,  focus  (e.g.,  communication,  information  retrieval),  pedagogical  approach,  or  learning  activities.  An 
obvious  result  of  this  variability  is  the  uneven  educational  value  and  quality  that  characterizes  the  growing 
aggregate  of  educational  sites. 

Given  the  continuous  increase  in  quantity  and  diversification  in  quality,  and  the  high  level  of 
expectations  in  the  educational  community  regarding  the  educational  potential  of  the  Web  technology,  we  have 
reached  a stage  at  which  an  appropriate  mapping  of  the  WBLE  landscape  is  required.  Key  questions  to  be 
answered  in  the  mapping  process  are: 

- What  characterizes  educational  Websites  at  the  content,  teaching,  learning  and  communication  levels? 

(Detailed  mapping  of  relevant  features) 

- What  is  the  current  overall  picture  of  the  educational  Websites  field  regarding  key  teaching  and  learning 

issues,  vis-a-vis  the  educators'  expectations?  (Overall  analysis  and  evaluation  of  trends  and  solutions) 
What  can  be  concluded  from  the  current  state  of  affairs  as  a basis  for  the  further  development  and 

implementation  of  educational  Websites?  (Practical  implications  and  conclusions) 

To  answer  these  questions  we  developed  a mapping  or  classification  scheme,  the  Taxonomy  of  WBLE 
(Nachmias,  Mioduser,  Oren  & Lahav,  in  press),  implemented  it  for  the  study  of  about  500  educational 
Websites,  and  elaborated  on  practical  implications  of  the  study’s  results.  In  the  following  sections,  we  will 
briefly  describe  our  classification  scheme,  we  will  present  the  study  and  its  findings,  and  finally  we  will 
discuss  key  issues  resulting  from  the  observed  picture  and  suggest  directions  for  further  development  of 
WBLE. 


Taxonomy  of  WBLE 
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For  this  study  we  developed  a classification  scheme  or  taxonomy  of  educational  Websites  aimed  to 
reflect  the  developers*  educational  philosophies  as  well  as  their  actual  manifestations,  by  revealing  how 
different  functionalities  are  configured,  the  knowledge  is  structured  and  represented,  and  communication 
features  are  implemented.  Our  taxonomy  characterizes  an  educational  Website  by  about  100  variables 
regarding  four  main  dimensions:  basic  descriptive  information  (e.g.,  site  ID,  updating,  population); 
pedagogical  and  educational  considerations  (e.g.,  instructional  model,  interaction,  cognitive  processes); 
knowledge  attributes  (e.g.,  representational  structure,  navigation  tools);  and  communication  features  (e.g., 
types  of  teleleaming,  communication  means).  For  a more  detailed  presentation  of  the  taxonomy's  rationale, 
background  and  description  refer  to  Nachmias,  Mioduser,  Oren  & Lahav  (in  press). 


Method 

Five  evaluators  acted  as  research  assistants.  All  five  are  students  in  the  graduate  program  of 
Communication  and  Computers  in  Education  in  Tel-Aviv  University  School  of  Education,  have  scientific 
background,  and  are  currently  science  educators.  First,  each  evaluator  was  assigned  to  find  about  100 
educational  Websites  meeting  all  following  three  criteria:  (a)  It  was  deliberately  developed  for  educational 
purposes;  (b)  Its  topic  is  clearly  defined  (in  this  study  the  area  of  science,  technology  and  mathematics 
teaching  was  selected);  and  (c)  it  is  clearly  identifiable  as  a unit  as  opposed  to  “mega-sites”,  links-repositories, 
or  general-access-sites  to  conglomerates  of  projects  or  web  pages.  In  this  process  524  sites  were  initially 
selected,  and  at  the  end  436  Websites  were  included  our  final  sample.  Sample  selection  took  place  in  March 
1998. 

Following  this  stage,  each  one  of  the  five  evaluators  received  about  90  Websites,  randomly  selected 
from  our  list,  to  be  characterized  according  to  the  Taxonomy  of  WBLE.  All  the  evaluation  process  was  carried 
during  April  and  May  1998.  By  June  15th  1998,  the  database  of  436  Websites  by  all  taxonomy  variables  was 
completed.  In  order  to  assess  the  validity  of  the  database,  a sample  of  about  25%  of  the  Websites  was  analyzed 
once  again  by  a different  evaluator. 

A detailed  presentation  of  the  results  is  beyond  the  scope  of  this  paper  (see  Mioduser,  Nachmias,  Lahav, 
& Oren,  1988).  In  the  following  we  present  and  discuss  a selected  sample  of  findings. 


Results  and  Discussion 

The  transition  of  the  Web  technology  from  its  early  rudimental  stages  to  the  current  “everyone-can-do- 
it”  stage,  generated  high  expectations  among  educators.  These  expectations  relate  to  the  Web’s  potential 
impact  on  educational  processes  in  three  main  domains  fostering  (a)  the  raise  of  new  pedagogical  forms 
emerging  out  of  unique  features  of  the  technology  (a  “Webagogy”?);  (b)  the  development  of  improved 
information-organization,  representation,  and  handling  capabilities;  and  (c)  the  enhancement  of 
communication  processes  among  students  and  teachers  and  support  for  collaborative  learning.  Our  aim  in  this 
study  was  to  assess  the  extent  to  which  educational  Websites,  these  sites  deliberately  developed  for  educational 
purposes  realize  the  potential  and  fulfill  the  expectations. 

Pedagogical  Characteristics  of  WBLE’s 

Current  pedagogical  approaches  support  learning  processes  that  include  the  student  active  involvement 
in  the  construction  of  knowledge,  her  interaction  with  peers  and  experts,  the  adaptation  of  instruction  to 
individual  needs,  and  new  ways  to  assess  the  students’  state  of  knowledge  and  learning.  Our  expectation  was 
that  the  development  process  of  educational  Websites  would  reflect  these  approaches.  Moreover,  given  the 
innovative  character  of  the  technology,  it  could  be  expected  that  even  new  pedagogical  forms  based  on  the 
unique  features  of  the  technology  would  arise. 

Notwithstanding,  the  results  indicate  that  this  is  not  the  situation  (see  Table  1).  Only  28.2%  of  the  sites 
include  inquiry-based  activities,  and  more  than  three-quarters  were  highly  structured  placing  the  control  over 
the  learning  process  mainly  on  the  computer  side.  Most  sites  elicit  cognitive  processes  such  as  retrieving 
information  (52.5%)  or  rote  learning  (42%),  fewer  focus  on  analysis  and  inference  processes  (32.6%)  and  even 
less  on  problem-solving  and  decision-making  (5%).  Only  a few  sites  include  student-modeling  and  adaptation 
mechanisms.  In  addition,  and  considering  the  fact  that  network  technologies  appear  to  be  ideal  milieu  for  the 
implementation  of  collaborative  work,  it  is  highly  deceptive  to  find  that  only  2.8%  of  the  sites  support  any 
form  of  collaborative  learning.  These  results  conclusively  show  that  the  pedagogical  approaches  favored  by 
educators  and  researchers  for  the  development  of  valuable  learning  environments  are  still  far  from  being 
implemented  in  most  educational  Websites. 
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The  gap  is  even  more  evident  if  we  take  in  account  the  accomplishments  reached  within  the  digital- 
technology-in-education  field  in  its  previous  form,  namely  (non-networked)  instructional  software.  Regarding 
interaction  types,  we  found  that  most  sites  include  browsing  (76.4%)  or  simple  forms  of  interaction  (42.4%), 
and  few  sites  offer  complex  (3%)  or  even  on-line  (6.4%)  activities.  Few  sites  include  any  form  of  feedback, 
either  automatic  (16.3%)  or  human  (5.5%).  Most  sites  offer  resources  and  means  related  to  information 
handling  (65%).  Only  few  offer  the  student  online  tools  (12.8%)  or  resources  external  to  the  site  itself  such  as 
resources  in  other  sites  (3 1 %)  or  experts  (8.7%). 


Yes 

No 

Instructional 

configuration 

Individualized  instruction 

407  (93.3%) 

29  (6.7%) 

Classroom  collaborative  learning 

54(12.4%) 

382  (87.6%) 

Web  collaborative  learning 

12(2.8%) 

424  (97%) 

Instructional 

model 

Directed 

330  (75.7%) 

106  (24.3%) 

Inquiry-based 

123  (28.2%) 

313  (71.8%) 

Instructional 

means 

Information-base 

283  (64.9%) 

153  (35.1%) 

Tools 

56(12.8%) 

380  (87.2%) 

Structured  activity 

211  (48.4%) 

225  (51.6%) 

Open-ended  activity 

43  (9.9%) 

393  (90.1%) 

Virtual  environment 

30  (6.9%) 

406  (93.1%) 

Student  modeling/adaptive  mechanism 

0 (0%) 

427  (97.9%) 

Interaction 

type 

Browsing 

333  (76.4%) 

103  (23.6%) 

Multiple  choice  question 

137  (31.4%) 

299  (68.6%) 

Simple  activity 

185  (42.4%) 

251  (57.6%) 

Complex  activity 

13(3.0%) 

423  (97.0%) 

On-line  tool 

28  (6.4%) 

408  (93.6%) 

Expert  consultation 

58  (13.3%) 

378  (86.7%) 

Cognitive 

process 

Information  retrieval 

229  (52.5%) 

207  (47.5) 

Memorizing 

183  (42.0%) 

253  (58.0%) 

Information  analysis  and  inferencing 

142  (32.6%) 

294  (67.4%) 

Problem  solving  and  decision  making 

22  (5.0%) 

414(95.0%) 

Creation  and  invention 

20  (4.6%) 

416(95.4%) 

Locus  of 
control 

Student  controlled 

377  (86.5%) 

59(13.5%) 

Software  environment  controlled 

77  (17.7%) 

359  (82.3%) 

Mixed  initiative 

26  (6.0%) 

410(94.0%) 

Feedback 

Automatic 

71  (16.3%) 

365  (83.7%) 

Human  asynchronous 

17(3.9%) 

419(96.1%) 

Human  synchronous 

7(1.6%) 

429  (98.4%) 

Help 

functions 

Technical  help 

91  (20.9%) 

345  (79.1%) 

Contextualized  content-help 

152  (34.9%) 

284  (65.1%) 

Didactic  help 

73  (16.7%) 

363  (83.3%) 

Learning 

resources 

Within  Website  resources 

363  (83.3%) 

73  (16.7%) 

Linked  WWW  resources 

135  (31.0%) 

301  (69.0%) 

Additional  external  resources 

93  (21.3%) 

343  (78.7%) 

External  resources  only 

4 (0.9%) 

432  (99.1%) 

Real  time  data  collection 

6(1.4%) 

430  (98.6%) 

Ask  an  expert 

38  (8.7%) 

398  (91.3%) 

Ask  a peer 

17(3.9%) 

419(96.1%) 

Evaluation 

Standardized  tests 

29  (6.7%) 

407  (93.3%) 

Alternative  evaluation 

7 (1.6%) 

429  (98.4%) 

Table  1:  Websites  analysis  for  the  pedagogical  dimension 

Pre-Web  (digital)  educational  materials  present  fascinating  examples  of  the  multiple  ways  educators 
succeeded  in  harnessing  the  new  technologies  to  educational  needs  and  goals  (e.g.,  constructivist 
environments,  intelligent  tutoring  systems,  sophisticated  multimedia  learning  environments).  Against  this 
rich  background,  and  looking  at  pedagogical  qualities  and  resources,  the  vast  majority  of  educational  sites 
prove  to  be  the  unripe  fruits  of  the  promising  but  still  immature  Web  technology. 
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Information  Representation  and  Handling 

In  cultural  terms,  in  our  digital  times  the  visual-world  (e.g.,  still  images,  icons,  video-clips,  animated 
graphics,  movies)  has  reentered  the  scene  in  stronger  presence  and  linguistic  status  and  meaning  than  in  its 
pre-Gutenberg  incarnation.  High-level  and  sophisticated  integrated-media  is  perhaps  one  of  the  defining 
characteristics  of  state-of-the-art  Websites  today.  But  again  our  results  showed  that  educational  Websites 
march  behind  (it  should  be  noted  that  we  are  not  looking  after  educators’  technical  -and  even  pyrotechnical- 
use  of  imaging  technology,  but  after  their  use  of  visual  languages).  The  vast  majority  of  sites  are  still  heavily 
based  on  text  (93%  of  the  sites  include  more  than  one  text  field  in  all  its  pages).  About  58%  of  the  sites 
include  at  least  one  image  per  page;  most  sites  do  not  include  interactive  images  (96. 1 %),  animated  images 
(81.9%),  or  sound  (see  Table  2). 


Not  at 
all 

At  list  one 
in  the  site 

50%  of 
pages  in  the 
site 

One  per 
page 

More  than 
one 

Text 

2 

(0.5%) 

0 

(0%) 

3 

(0.7%) 

24 

(5.5%) 

407 

(93.3%) 

Image 

63 

(14.4%) 

64 

(14.7%) 

55 

(12.6%) 

117 

(26.8%) 

137 

(31.4%) 

Interactive  image 

419 

(96.1%) 

9 

(2.1%) 

1 

(0.2%) 

1 

(0.2%) 

5 

(1.1%) 

Animation 

357 

(81.9%) 

35 

(8.0%) 

18 

(4.1%) 

18 

(4.1%) 

8 

(1.8%) 

Sound 

426 

(97.7%) 

6 

(1.4%) 

1 

(0.2%) 

2 

(0.5%) 

1 

(0.2%) 

Real-time 

updating 

431 

(98.9%) 

0 

(0%) 

3 

(0.7%) 

1 

(0.2%) 

1 

(0.2%) 

Table  2:  Representational  means  in  educational  Websites 

Regarding  structure  and  organization  of  knowledge,  the  Web  is  the  realization  of  the  hypertext  (or 
hypermedia)  model.  Non-linear  structure,  complex  linkage  within  and  between  information  units,  and 
appropriate  navigation  and  search  tools  are  defining  features  of  this  model.  Our  results  reveal  only  a shallow 
presence  of  these  features  in  the  evaluated  Websites  (Table  3).  Only  about  half  of  the  sites  included  within- 
the-site  linkage  to  a reasonable  extent  (more  than  one  link  per  page),  and  about  1 1%  of  the  sites  refer  to  other 
sites  (external  linkage)  at  the  same  extent. 


Not  at 
all 

At  list 
one 

in  the  site 

50%  of 
pages  in 
the  site 

One  per 
page 

More 

than 

one 

Within  the  site 

116 

(26.6%) 

32 

(7.3%) 

20 

(4.6%) 

50 

(11.5%) 

218 

(50.0%) 

Links  to  external  sites 

253 

(58.0%) 

68 

(15.6%) 

37 

(8.5%) 

32 

(7.3%) 

46 

(10.6%) 

LINKS  TO: 

Yes  (N  of  sites  %) 

No  (N  of  sites  %) 

external  databases 

74(17.0%) 

362  (83.0%) 

external  tools 

12(2.8%) 

424  (97.2%) 

external  activities 

42  (9.6%) 

394  (90.4%) 

virtual  reality  devices 

8(1.8%) 

428  (98.2%) 

human  communication 

25  (5.7%) 

411  (94.3%) 

Table  3:  Link  structure  in  educational  Websites 
Communications 
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The  results  of  this  study  show  that  limited  communication  resources  were  used  in  most  Websites  (Table 
4).  The  most  (and  almost  sole)  resource  present  in  the  sites  is  electronic  mail  (about  65%  of  the  sites).  Other 
tools  such  as  discussion  groups,  chat,  or  any  form  of  distant  work  (e.g.,  tele-manipulation,  tele-creation)  were 
found  only  in  a few  sites.  Moreover,  features  aimed  to  support  working  groups  or  learning  communities  were 
not  found  in  any  of  the  evaluated  sites.  The  gap  between  expectations  and  actual  implementation  in  the 
communications  domain  is  even  more  evident  than  in  the  previously  discussed  domains.  The  main  reason  for 
that  is  that  the  technological  resources  do  exist  and  are  being  successfully  implemented  in  other  areas  of 
people’s  life  (e.g.,  work,  professional  training,  banking,  shopping).  In  addition,  transactions  among  humans 
and  between  humans  and  information  resources  are  quintessential  to  education,  and  it  is  not  hard  to  conceive 
endless  forms  of  support  that  communication  technology  could  offer  for  these  processes.  As  for  today’s  reality, 
this  support  is  not  yet  a function  in  most  educational  Websites. 


Yes 

No 

(N  of  sites,  %) 

(N  of  sites,  %) 

Synchronic  activities 

17(3.9%) 

419(96.1%) 

Communication  means 

e-mail 

283  ('64.9%) 

153  (35.1%) 

Discussion  group  without  mediator 

15  (3.4%) 

421  (96.6%) 

Discussion  group  with  mediator 

10(2.3%) 

426  (97.7%) 

Chat 

8(1.8%) 

428  (98.2%) 

Moo/mud 

0 (0%) 

436(100.0%) 

Video  conference 

0 (0%) 

436  (100.0%) 

Tele-manipulation 

1 (0.2%) 

435  (99.8%) 

Tele-creation 

7(1.6%) 

429  (98.4%) 

Table  4:  Use  of  communication  resources  in  Websites 


Conclusions 

One  step  ahead  for  the  technology,  two  steps  back  for  the  pedagogy.  One  can  depict  that  way  the  usual 
loops  affecting  the  educators’  assimilation  process  of  new  technologies  for  the  last  decades.  As  experienced 
educators  we  hold  substantial  models  regarding  the  varied  facets  of  our  practice  (e.g.,  how  to  build  a lesson 
plan,  to  assess  a learner’s  performance  or  behavior,  to  develop  a learning  unit).  These  models  are  usually  tied 
to  the  (technological)  resources  at  hand,  and  they  affect  each  other  mutually.  It  seems  reasonable  to  assume 
that  when  facing  the  assimilation  of  a new  technology  we  use  these  models  as  input  to  the  process.  The  result 
is  usually  a transition  period  at  which  we  replicate  the  known  models  by  means  of  the  new  technology.  While 
first  assimilating  the  computer  technology  developers  replicated  the  programmed  instruction  paradigm  by 
means  of  the  new  technology,  initially  in  the  form  of  electronic  worksheets  and  booklets  then  evolving  in  time 
into  sophisticated  drill  and  practice  and  structured  tutoring  software  (Venezky  & Osin,  1991).  Our  claim  is 
that  this  study  reveals  a similar  transitional  phenomenon  regarding  the  vast  majority  of  educational  Websites. 
Most  sites’  main  component  is  the  information-base,  built  upon  the  hypermedia-CD  model  (even  a feature 
with  can  be  claimed  as  unique  to  the  Web,  as  the  linkage  to  external  sites,  is  today  included  in  many  hybrid 
CD-Web  products).  As  for  interactivity  features  based  on  the  implementation  of  new  technological  resources 
(e.g.,  forms,  Java  applets,  Shockwave),  most  online  activities  resemble  the  automatic-feedback  (behaviorist- 
like)  transactions  of  classic  CAI  (e.g.,  multiple-choice,  select-correct-part,  assemble-correct-configuration). 

In  light  of  these  results  one  can  adopt  the  skeptics’  perspective  and  argue  that  Web  technology  has  little 
to  offer  to  education.  But  one  can  also  adopt  a more  thoughtful  perspective,  reflecting  on  the  character  of  this 
transition  stage  and  generating  new  possible  models  and  trends  based  on  substantial  educational  needs. 

The  authors  are  part  of  the  community  of  educators  dealing  with  the  problematic  involved  in  the 
assimilation  of  Web  technology  to  education  (Mioduser  & Oren,  1998;  Nachmias,  Mioduser,  Oren  & Lahav,  in 
press).  Although  in  this  study  we  have  referred  to  the  educational  Websites  population  in  quantitative  terms, 
we  are  aware  that  a number  of  fascinating  examples  of  high  pedagogical-quality  sites  do  exist  in  the  Web.  But 
our  main  purpose  in  this  study,  rather  to  focus  on  the  exceptions,  was  to  map  and  learn  what  most  existing 
sites  and  current  trends  as  they  are  delivered  to  cyberspace  have  to  offer  to  educators  and  learners.  Based  on 
this  and  similar  studies,  the  next  steps  should  focus  on  the  research  and  development  of  novel  Web-based 
educational  models  (Windschite,  1 998)  and  on  the  implementation  of  a revised  configuration  of  the 
technology-assimilation  evolutionary  loop:  two  steps  ahead  for  the  pedagogy/technology,  one  step  back  for 
reflection  and  mindful  planning  of  subsequent  steps. 
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Abstract:  Many  Higher  Education  institutions  are  responding  to  external  pressures  by  developing 
an  on-line  presence.  Coincidentally  changes  in  pedagogy  are  demanding  the  kinds  of  functionality 
offered  by  on-line  learning  delivery  and  support  systems.  Creation  of  a virtual  university  is  not  a 
simple  transformation  from  old  methods  into  new  media  and  for  traditional  universities  there  is  a 
high  degree  of  risk  in  developing  an  off  campus,  on-line  operation.  De  Montfort  University’s 
Electronic  Campus  is  a hybrid  approach  which  aims  to  re-purpose  and  integrate  existing  teaching 
with  new  on-line  methods  as  far  as  possible  in  order  to  achieve  maximum  flexibility  of  access  for 
existing  on-campus  students.  Given  limitations  on  resource  availability,  existing  resources  need  to 
be  re-engineered  to  bring  them  into  line  with  the  chosen  venture.  This  entails  obtaining  the  support 
of  staff  at  all  levels  through  a conscious  programme  of  culture  change,  which  is  described  here. 


All  Change 

There  is  a growing  belief  that  learning  is  a key  to  economic  success  and  competitiveness,  whether  at  the  level  of  a 
company,  an  individual,  or  the  state.  Knowledge  is  rapidly  outdated  in  a fast  changing  world,  employees  need  to 
enhance  their  attractiveness  through  periodic  self  development  and  employers  need  to  update  he  skills  of  their 
workforce  in  order  to  remain  competitive  (BCC  1998)  (loM  1998).  Notwithstanding  local  labour  shortages  in  the 
United  Kingdom,  there  is  a developing  surplus  of  people  combined  with  a declining  number  of  jobs  overall  (DfEE 
1998a). 

The  appearance  of  corporate  universities  such  as  Unipart  'U’  Oittp://w ww.unipart.co.uk/97report/ugc9 7.htm)  and  the 
British  Aerospace  Virtual  University  (http://www.bae.co.uk)  are  manifestations  of  these  concerns  and  of  a growing 
sense  that  industry  needs  are  not  being  adequately  met  by  traditional  education  and  training  providers.  Recent  UK 
initiatives  such  as  the  proposed  University  for  Industry  (DfEE  1998b)  and  Lifelong  Learning  reflect  similar  concerns 
at  Government  level. 

At  the  same  time  change  is  being  driven  from  within  as  traditional  models  for  mass  higher  education  based  on 
behavioural  theories  give  way  to  more  learner  centred  and  resource  based  philosophies  (Hardy  1997).  Learner 
centred  approaches  seek  to  accommodate  individual  learner  needs  and  acknowledge  that  learners  may  have  other 
calls  on  their  time.  This  leads  to  more  flexible  delivery  and  support  strategies  which  allow  learners  to  shift  the  time, 
place  and  pace  of  learning.  Rigid  classroom  timetables  and  enrolment  schedules  become  more  relaxed,  allowing  for 
multiple  delivery  of  the  same  curricula  at  different  places  at  different  times.  The  role  of  the  teacher  shifts  away  from 
that  of  expert  instructor  to  learning  facilitator.  Involvement  of  the  learner  in  the  development  of  the  learning 
experience  necessarily  requires  the  'teacher  expert'  to  relinquish  control  and  allow  others  (learners)  to  contribute 
(Laurillard  1 993).  The  rapid  development  of  digital,  networked  multimedia  technology  such  as  the  Internet,  email 
and  computer  based  and  video  conferencing  assist  with  this  process  and  at  the  same  time  open  up  opportunities  for 
learners  to  seek  out  and  utilise  learning  resources  offered  by  other  parties  (Bacsich  1998). 
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Competition,  Globalisation  and  Commoditisation 

In  the  UK,  per  capita  spending  on  Higher  Education  students  declined  by  20  per  cent  between  1991/2  and  1996/7 
(THES  1998),  stimulating  greater  competition  between  universities  for  resources.  Competition  is  not  just  local.  UK 
Higher  Education  providers  are  facing  increased  competition  in  traditional  overseas  markets  as  well,  where  countries 
such  as  the  USA  and  Australia  are  vigorously  recruiting  students  whom  might  have  otherwise  studied  in  the  UK. 
Many  overseas  governments  are  seeking  to  expand  and  enhance  their  local  provision  instead  of  sending  such  large 
proportions  of  undergraduates  to  study  abroad,  e.g.  India,  Singapore,  Hong  Kong,  Indonesia,  South  Africa.  From 
the  UK  perspective  the  recent  appearance  of  Virtual',  'on  line'  universities  such  as  Western  Governors’  University 
(www.wgu.edu)  and  Colorado  University's  CU  Online  (www.cuonline.edu).  with  their  potential  to  deliver  globally, 
are  cause  for  concern.  Commercial  on-line  organisations  such  as  Phoenix  University  (www.uophx.edu).  or  ZD 
University  (www.zdu.com)  are  further  indications  of  the  possible  shape  of  things  to  come. 


Re-inventing  the  University 

All  of  the  above  factors  are  driving  Universities  to  reconsider  their  roles  and  methods.  What  is  clear  is  that  new 
ways  have  to  be  found  of  carrying  out  at  least  some  of  the  functions  of  traditional  universities  and  that  overall  new 
ways  of  doing  things  will  have  to  cost  less  than  traditional  methods.  In  the  absence  of  significant  sums  of  new 
money  to  effect  change,  the  task  facing  universities  today  is  how  best  to  re-engineer  the  resources  already  at  their 
disposal  to  deliver  and  support  the  changes  needed. 

Many  traditional  Higher  Education  institutions  are  seeking  to  create  some  kind  of  on-line  presence,  Forbes 
magazine  in  the  US  has  published  a guide  to  the  'top  20  Cyber-Us’,  compiled  by  Ebeling  and  Bistayi  (1997),  Philips 
and  Yager  (1998)  have  published  a guide  to  virtual  university  study  opportunities  which  profiles  195  accredited 
universities  offering  over  1000  distance  learning  courses.  On  the  face  of  it,  becoming  a virtual  university  appears  to 
be  straightforward.  Bacsich  (1998)  has  observed  that  the  following  methodology  is  generally  accepted: 

• Lecturing  maps  to  videoconferencing 

• Tutoring  maps  to  computer  conferencing 

• Books  map  to  CD-ROM  and  the  Internet 

However,  when  one  takes  a closer  look  at  the  reality  of  virtual  university  provision  to  date  it  is  interesting  to  note 
how  much  of  it  depends  on  older  technologies  including  books,  video,  and  residential  periods  on  campus  as  well  as 
videoconferencing  (Ebeling  and  Bistayi  (1997),  Philips  and  Yager  (1998).  There  appear  to  be  a number  of  good 
reasons  why  established,  traditional  Higher  Education  providers  should  hold  back  from  large-scale  conversion  to  on- 
line delivery.  These  include:  the  high  costs  of  courseware  development;  the  longer  development  timescale  and 
relative  inflexibility  of  resource  based  learning  compared  with  face  to  face  classes;  shortages  of  appropriate  ICT 
infrastructure;  staff  concerns  about  the  impact  of  new  technologies  on  their  own  Jobs  and  on  the  quality  of  the 
student  learning  experience;  and  management  concerns  about  the  possible  consequences  of  failure  if  large  scale 
investments  do  not  deliver  the  required  competitive  advantage.  Also,  traditional  universities  have  strengths  that  are, 
by  definition,  missing  from  cyberspace:  physical  locations  that  provide  a respite  from  the  everyday  demands  of 
home  and  work;  facilities  and  equipment  that  are  not  readily  simulated  in  virtual  space;  people  with  whom  to 
socialise,  enjoy  physical  contact,  chance  encounters,  etc. 

In  seeking  to  retain  the  benefits  of  a traditional  university,  minimise  the  risks  of  a virtual  enterprise,  while  obtaining 
some  of  the  advantages  of  on-line  delivery,  De  Montfort  University  has  opted  for  a hybrid  model.  The  Electronic 
Campus  is  an  Internet  based  addition  to  the  face  to  face  delivery  and  support  systems  of  the  university  that  is 
intended  to  enhance  flexibility  of  access  to  learning  on  campus.  In  an  era  of  lifelong  learning,  increasing  numbers  of 
individuals  are  likely  to  need  to  access  educational  opportunities  for  irregular  periods,  fitting  in  with  domestic  and 
employment  commitments.  Even  traditional  full-time  students  increasingly  have  to  fit  their  studies  around  paid 
work  in  order  to  finance  their  education.  Fixed  attendance  at  lectures,  limited  opening  hours  for  buildings, 
communication  via  notice  boards,  face  to  face  meetings,  seminars  and  tutorials  all  create  barriers  to  flexibility  of 
access.  Restricting  the  Electronic  Campus  to  on-campus  students  may  appear  at  first  to  be  paradoxical.  However 
the  rationale  for  this  decision  follows  logically  from  the  earlier  discussion  regarding  the  risks  of  going  virtual.  On 
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campus  access  ensures  that  the  benefits  of  a traditional  style  university  are  retained,  with  the  addition  of  some  of  the 
benefits  of  on-line  access. 

The  University  has  a threefold  strategy  for  achieving  this  goal,  involving  a combination  of  in-house  development, 
importation  of  resources  developed  elsewhere  and  collaborative  development  ventures  with  other  organisations. 
While  the  University  has  aspirations  to  produce  as  much  of  its  own  material  as  possible,  it  is  recognised  that  on  our 
own  we  are  unlikely  to  be  able  to  achieve  a significant  shift  in  favour  of  telematic  methods  within  a reasonable 
timescale.  More  rapid  transition  can  be  achieved  by  leveraging  the  resources  of  other  organisations,  either  by 
importing  their  materials  or  through  collective,  collaborative  effort  to  jointly  produce  new  materials.  However,  in 
order  to  establish  a basis  for  collaboration,  potential  partners  have  to  have  something  to  share.  So  in  the  first 
instance  the  emphasis  has  been  on  the  first  two  strands  of  the  strategy:  in  house  development  and  importing. 


Progress  to  date 

In  the  first  22  months  over  30  different  Electronic  Campus  development  projects  have  been  established,  ranging 
across  all  Faculties  and  at  all  levels  including  further  education,  undergraduate,  postgraduate  and  continuing 
professional  development.  Some  of  the  projects  are  complete  courses,  some  are  full  modules  and  others  are  parts  of 
modules.  Students  started  to  study  the  first  of  these  modules  during  the  first  semester  of  1998  and  at  the  time  of 
writing  the  total  number  of  students  studying  on-line  is  over  1500.  In  most  cases  projects  have  been  based  on 
resources  already  developed  in  the  university  and  in  areas  most  likely  to  have  maximum  impact  on  students  and 
staff  The  learning  activities  modelled  include  lectures,  seminars,  tutorials,  practical  assignments  and  cognitive 
assessments.  Materials  produced  include  pdf  documents,  straight  HTML  pages,  web  interfaced  databases,  including 
image  databanks,  and  on-line  assessment  exercises,  plus  on-line  computer  conferencing,  videoconferencing,  FAQs, 
e-mail  and  synchronous  'chaf . Projects  underway  will  generate  video  and  audio  files  as  well. 

In  addition  to  these  in-house  projects,  the  University  has  been  actively  embedding  into  the  curriculum  learning 
resources  which  have  been  developed  at  other  institutions  under  the  auspices  of  the  UK  Teaching  and  Learning 
Technology  Programme  (Scott  et  al  1 998). 

In  setting  up  the  Electronic  Campus  initiative,  care  has  been  taken  to  address  key  issues  identified  as  crucial  to 
success  in  earlier  studies  (Brown  1 997),  in  particular: 

• Future  proofing 

• Staff  attitudes 

• Champions 

• Learning  support 


Future  proofing 

Standards  offer  the  potential  of  savings  through  reduced  staff  development  costs  and  bulk  purchase  discounts.  They 
can  also  reduce  the  learning  curve  for  inexperienced  development  teams.  Off-set  against  this  is  the  risk  of 
committing  too  early  to  inappropriate  tools.  The  common  core  of  all  the  Electronic  Campus  projects  is  HTML 
compliant  content,  enabling  material  to  be  accessed  from  a variety  of  different  platforms  via  the  Internet.  HTML 
was  adopted  as  the  standard  delivery  environment  in  order  to  future  proof  the  investment  as  far  as  possible. 
Variations  such  as  Java,  XML,  DHTML,  etc.  have  been  avoided  for  the  time  being  to  ensure  that  material  does  not 
acquire  obsolescent  legacy  status  as  technologies  change  in  the  medium  term  future. 


Staff  attitudes 

Awareness  of  the  Electronic  Campus  has  been  raised  through  separate  meetings  with  all  Deans  of  Faculties, 
presentations  to  senior  staff  down  to  the  level  of  Heads  of  Departments,  development  of  an  Electronic  Campus  web 
site,  publication  of  articles  in  the  university  in-house  newsletter  (13  so  far)  and  through  direct  liaison  with  Faculty 
members  via  dedicated  central  staff  with  appropriate  skills  and  experience  (the  Learning  Development  Managers 
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described  below).  The  Web  site  (http://dld.mk.dmu.ac.uk/ec/frame/frameset.htm)  provides  access  to  news  about 
Electronic  Campus  developments,  virtual  university  developments  elsewhere,  relevant  professional  associations, 
funding  opportunities,  tools  and  learning  resources,  staff  development  opportunities  and  to  newsgroups  concerned 
with  on-line  educational  developments. 

Within  a university  context  the  opportunities  for  direct  financial  reward  are  much  more  constrained  than  in  industry, 
however  it  has  been  possible  to  put  in  place  a number  of  incentives.  Pump  priming  funds  have  been  made  available 
centrally  for  Faculties  to  bid  against.  This  money  can  be  used  to  buy  equipment,  software,  travel,  learning 
resources,  staff  development  and  staff  time.  In  practice  78%  of  the  project  money  allocated  to  Faculties  has  been 
used  to  buy  out  staff  time  to  release  staff  from  other  duties,  particularly  teaching.  In  addition,  the  University 
performance  related  pay  scheme  and  the  University  promotion  system  have  been  examined  to  identify  ways  of 
explicitly  encouraging  and  supporting  the  development  of  teaching  excellence.  New  categories  of  teacher  status 
have  been  introduced  in  anticipation  of  the  accreditation  system  to  be  introduced  by  the  Higher  Education  Funding 
Council  for  England  Institute  for  Learning  and  Teaching. 

Care  has  been  taken  to  ensure  that  involvement  in  Electronic  Campus  activities  is  as  easy  as  possible.  A proforma 
set  of  questions  guides  Faculty  staff  through  the  selection  criteria  applied  to  proposals  for  funding.  Each  Faculty  has 
been  allocated  a specific,  individual.  Learning  Development  Manager,  to  work  with  them  to: 

• Develop  appropriate  teaching,  learning  and  assessment  strategies. 

• Develop  teaching  and  learning  proposals  for  internal  and  external  funding. 

• Identify  and  obtain  resource  based  learning  materials  produced  elsewhere. 

• Develop  and  implement  learning  support  strategies  and  systems. 

• Identify  and  meet  staff  development  needs  in  relation  to  implementation  of  the  Electronic  Campus. 

Learning  Development  Managers  (LDMs)  are  members  of  a central  curriculum  design  and  production  team  with 
educational  technology  and  curriculum  development  and  delivery  expertise. 

As  far  as  possible,  on-line  materials  have  been  developed  by  Faculty  staff  themselves  using  simple,  readily  available 
tools  such  as  Microsoft  Office  and  Adobe  Acrobat.  Microsoft  Office  products  were  selected  as  general  development 
tools  for  their  familiarity,  ubiquity  and  simplicity  of  use,  compared  with  professional  authoring  tools  and 
programming  languages.  Given  the  aim  of  ensuring  that  as  much  of  the  development  as  possible  could  be 
undertaken  by  Faculty  teaching  staff,  it  was  important  to  avoid  creating  barriers  such  as  the  requirement  to  learn 
specialised  software  packages.  Adobe  Acrobat  was  adopted  as  a means  of  readily  converting  large  quantities  of 
existing  text  based  documents  to  HTML  readable  form.  Where  more  sophisticated  products  have  been  required,  the 
LDMs  have  based  between  faculty  colleagues  and  central  support  teams  to  provide  services  using  more  advanced, 
professional  tools.  LDMs  provide  ongoing  assistance  with  learning  materials  design  and  production  and  a link  back 
to  the  central  production  resources  (graphic  design,  courseware  authoring,  programming,  desk  top  publishing,  audio, 
video  and  photography,  digitising,  print,  etc.). 

Staff  development  has  been  identified  as  another  important  enabler.  Electronic  Campus  staff  development  is  a 
combination  of  formal  training  events  and  informal  problem  based  learning.  For  example  a programme  of  formal  IT 
staff  training  courses  has  been  restructured  to  provide  basic  technical  authoring  skills  using  Microsoft  Office 
products  and  more  advanced  Internet  training  including  HTML  authoring,  web  site  development,  Internet 
communications  and  information  search  techniques.  Project  teams  are  also  offered  short  courses  on  the  use  of 
specific  software  applications  used  within  the  Electronic  Campus.  Informal  development  takes  place  through 
interaction  with  the  LDM,  beginning  with  the  project  proposal  proforma  which  encourages  teaching  staff  to  consider 
learning  objectives,  pre-requisites,  relationship  with  other  courses,  assessment  strategies,  teaching  methods,  learner 
characteristics  and  requirements. 


Champions 

A vital  ingredient  of  change  seems  to  be  the  presence  of  enthusiasts  in  key  positions  within  the  organisation  with  the 
authority,  charisma  and  skills  to  both  push  and  facilitate  the  change  process  (Brown  1 997).  The  Electronic  Campus 
initiative  is  led  by  a Pro  Vice  Chancellor,  supported  by  a Steering  Group,  made  up  of  key  university  senior  staff 
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including  the  Director  of  Learning  Development.  Reporting  to  this  group  is  an  Operational  Group  of  managers  from 
key  operational  areas  (Library,  Information  Services,  Educational  Technology,  Learning  and  Teaching,  Digital 
Library),  charged  with  identification  of  implementation  issues  and  solutions  in  response  to  steering  group  policies 
and  strategies,  A link  between  the  Operational  Group  and  the  Faculties  is  provided  by  the  LDMs  and  their  Faculty 
counterparts:  Teaching  and  Learning  Co-ordinators,  Successful  project  bids  result  in  the  formation  of  Project  Teams 
within  faculties,  supported  by  the  Faculty  LDM,  There  is  therefore  an  unbroken  chain  of  champions  from  the  Vice 
Chancellor  down  through  the  organisation  to  the  grass  roots  of  course  design  and  delivery. 


Learning  support 

The  quality  and  availability  of  support  for  learners  appears  to  be  a vital  ingredient  for  successful  independent 
learning  (Laurillard  1993).  For  this  reason  emphasis  has  been  given  to  the  development  of  on-line  student  support 
systems  using  email,  computer  and  videoconferencing.  First  Class  conferencing  software  is  the  standard  for 
computer  mediated  communications  within  the  Electronic  Campus.  It  has  been  in  use  within  the  University  for  four 
years,  supporting  teaching  and  learning  on  a number  of  courses.  The  University  now  operates  six  professional  level 
videoconferencing  suites  across  its  main  campuses  and  after  some  experimentation  with  different  systems  has 
standardised  on  Picturetel  equipment.  In  addition  Microsoft  Netmeeting  is  being  trialled  on  a number  of  Electronic 
Campus  modules  to  evaluate  its  potential  for  remote  class  rooms  utilising  its  desk  top  video  and  audio  conferencing 
features,  combined  with  application  sharing.  Web  CT  is  being  trialled  on  two  projects  to  evaluate  its  effectiveness 
as  a fully  integrated  on-line  course  development,  delivery  and  support  environment,  including  conferencing. 


What  works 

While  it  is  too  early  to  make  definitive  judgements  about  the  effectiveness  of  the  Electronic  Campus,  it  is  possible  at 

this  stage  to  comment  on  the  planning  implementation  process  in  term  of  what  has  worked: 

1 . Pump  priming  the  innovation  to  make  it  easy  for  Faculty  staff  to  get  involved  and  for  resource  managers  to 
sanction  their  involvement.  Also  to  make  available  additional  central  resources  beneficial  to  the  innovation, 
e.g.  Software  site  licences  for  software  packages  adopted  as  standards. 

2.  Maintaining  central  control  over  project  budgets  has  enabled  close  monitoring  of  expenditure  against  agreed 
schedules  and  deliverables. 

3.  Faculty  based  projects  and  development  teams  ensures  local  relevance  and  ownership  and  hence  commitment 
and  enthusiasm. 

4.  Establishing  identified  individuals  to  work  with  specific  Faculties  in  an  account  management  type  role  has 
developed  stronger  links  between  the  central  support  services  and  Faculties  and  encouraged  a culture  of 
innovation  and  creativity. 

5.  Focussing  on  HTML  as  the  primary  medium  for  delivery  and  support  has  helped  to  simplify  the  decision 
making  and  development  process.  The  choice  of  medium  sets  priorities  for  infrastructure  investment,  software 
tools  and  staff  development  and  defines  useful  parameters  for  functionality. 

6.  Concentrating  on  unsophisticated,  readily  available,  development  tools,  such  as  Microsoft  products,  ensures  that 
all  staff  have  easy  access  to  them  and  where  necessary,  training  courses  can  be  made  available  easily  and 
cheaply. 

7.  Adoption  of  a formal  process  of  bidding  for  identified  resource  has  made  the  activities  of  the  central  teams  more 
visible,  accountable  and  legitimised  them  to  a degree  not  previously  possible. 
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Conclusions 


Re-inventing  the  university  as  a hybrid  face-to-face/on-line  institution  is  more  than  just  a technical  challenge.  It 
requires  a culture  change  which  needs  to  be  led  from  the  top,  through  an  unbroken  chain  of  champions  down 
through  the  organisation,  including  the  resource  managers  at  middle  management  level.  Key  factors  to  pay  attention 
to  are  staff  attitudes,  future  proofing  and  student  learning  support.  It  also  needs  to  be  able  to  draw  on  and  re-purpose 
resources  already  available  within  the  organisation.  Because  of  the  large,  distributed,  nature  of  De  Montfort 
University  it  has  long  had  a need  to  research  and  exploit  the  benefits  of  networking  and  resource  based  learning.  In 
the  past,  small  scale,  bottom  up,  experiments  in  videoconferencing.  Computer  Mediated  Communications,  computer 
marked  assignments,  video,  print,  CD  ROM  and  Internet  delivered  course  materials  have  tended  to  have  only  a 
minor,  temporary  impact  on  the  organisation  because  of  their  inability  to  succeed  in  re-engineering  sufficient 
resources  to  achieve  lasting  institution-wide  change.  Other  parts  of  the  organisation  have  tended  to  resist  changes 
not  central  to  their  own  goals.  At  the  time  of  writing  (October  1998),  the  Electronic  campus  initiative  appears  to  be 
successful  so  far  in  converting  these  unrelated  initiatives  into  a coherent,  management  led,  university  wide 
programme.  During  the  next  twelve  months  the  challenge  will  be  to  embed  the  Electronic  Campus  activity  further 
into  the  normal  processes  of  the  university. 
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Abstract:  A project  is  described  in  which  a virtual  environment  is  combined  with  an  intelligent 
tutoring  system  for  troubleshooting.  The  virtual  environment  enables  the  student  to  perform 
troubleshooting  activities  in  a realistic  job-situation.  The  intelligent  tutoring  system  takes  care  of 
the  learning  process,  and  offers  structured  guidance  and  feedback  throughout  the  troubleshooting 
process.  In  doing  so,  a system  has  been  realized  that  helps  to  bridge  the  gap  between  vocational 
training  which  is  theoretical  and  general  in  nature  and  the  knowledge  and  skills  required  in  actual 
job  situations.  It  also  helps  in  raising  the  attractiveness  of  training. 


1.  Introduction 

There  exists  a widely  known  gap  between  the  contents  of  vocational  training  and  the  knowledge  and  skills  needed  in 
actual  job  situations.  This  gap  shows  itself  in  a number  of  ways.  The  knowledge  acquired  during  vocational  training 
is  often  rather  general  and  theoretical  in  nature,  leaving  the  transfer  to  the  real-world  practice  to  practice  periods. 
Students  often  complain  about  the  fact  that  they  see  only  a minor  relationship  between  what  is  trained  in  schools  and 
what  is  needed  in  the  actual  job  context.  Employers,  on  the  other  hand,  have  a problem  in  hiring  rightly  trained 
personnell,  and  have  to  put  in  a lot  of  effort  in  preparing  new  employees  for  their  jobs.  Another  problem,  especially 
with  respect  to  vocational  training  for  the  technical  subjects,  is  the  mismatch  between  the  number  of  students  in 
training,  and  the  number  of  technicians  needed  in  the  job  market.  There  is  much  more  demand  for  technicians  than 
the  schools  can  actually  deliver. 

Why  is  it  that  an  insufficient  number  of  students,  men  and  women,  choose  for  technical  vocational  training,  while  on 
the  other  hand  there  are  plenty  of  possibilities  on  the  job  market  afterwards?  What  can  be  done  in  terms  of  training 
to  raise  interest  in  the  technical  subjects  and  to  increase  the  attractiveness  of  education?  One  of  the  answers  to  this 
question  may  be  found  in  establishing  a strong  relationship  between  what  is  trained  in  schools  and  what  is  needed  in 
actual  job  contexts.  Multi-media  programs  in  which  a strong  link  is  provided  between  realistic  job  situations  and  the 
knowledge  and  skill  to  be  acquired  in  training  environments,  could  be  a strong  means  to  bridge  this  gap,  and  thus  to 
enhance  the  attractiveness  of  training.  The  program  Attractive  Vocational  Training  , partly  sponsored  by  the  dutch 
government,  aims  at  exactly  reaching  this  goal:  establishing  a stronger  link  between  training  and  job  contexts, 
thereby  making  training  more  attractive,  and  thus  providing  a strong  motivator  for  upcoming  students  to  choose  for 
vocational  training. 
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The  Troubleshooter  project,  partly  sponsored  by  the  above  mentioned  program  Attractive  Vocational  Training,  is  an 
example  of  a multi-media  virtual  environment,  to  be  used  in  vocational  training,  that  is  able  to  train  students  a 
complex  cognitive  skill,  troubleshooting,  in  a realistic  situation.  It  combines  a virtual  job  environment  with  an 
intelligent  tutoring  system,  that  guides  the  student  through  the  various  parts  of  the  problem  solving  process,  and 
depending  on  the  progress  made  in  learning  acts  more  as  a formal  coach  or  as  an  advisor.  The  multi-media  virtual 
environment  , enables  the  student  to  move  around  freely  in  a particular  simulated  building,  and  act  as  a realistic 
troubleshooter  in  a realistic  environment  doing  realistic  jobs,  including  having  conversations  with  other  (simulated) 
people  in  this  environment,  using  appropriate  documentation,  telecommunication  facilities,  tools  etc.  This  has 
resulted  in  an  attractive  training  program  in  which  a realistic  job-environment  is  provided  while  on  the  other  hand 
strongly  promoting  a structured  acquisition  of  troubleshooting  skills. 

This  article  will  describe  the  background  of  the  Troubleshooter  in  some  detail.  In  paragraph  two,  the  method  for 
training  of  troubleshooting  is  described.  Paragraph  three  describes  features  of  the  virtual  environment,  while 
paragraph  four  will  go  into  some  detail  about  the  interactions  between  the  intelligent  tutoring  system  and  the  virtual 
environment.  Finally,  in  paragraph  five  some  conclusions  are  drawn. 


2.  Training  of  Troubleshooting:  A Structured  Approach 

Structured  Troubleshooting:  The  Method 

The  TNO  Human  Factors  Research  Institute,  on  behalf  of  the  Royal  Netherlands  Navy,  has  carried  out  a study  into 
the  possibilities  of  improving  the  troubleshooting  performance  of  technicians  by  means  of  innovating  the  training 
practices.  The  project  started  with  a cognitive  task  analysis  of  troubleshooting  on  board  of  naval  frigates.  As  part  of 
this  analysis  we  investigated  how  well  technicians  were  able  to  solve  problems  that  they  were  confronted  with, 
coupled  with  an  investigating  into  the  problems  at  a cognitive  level  that  they  have  in  terms  of  troubleshooting 
performance.  This  analysis  showed  problems  on  at  least  two  levels:  1,  Beginning  technicians  lack  a systematic 
approach  to  troubleshooting,  resulting  in  a lack  of  goal-directed  troubleshooting,  and  2.  They  lack  a functional 
understanding  of  the  installations  that  they  have  to  maintain  (Schraagen  & Schaafstal,  1 996).  These  conclusions  are 
remarkably  similar  to  results  of  Schaafstal  (1993),  obtained  in  a rather  different  domain:  troubleshooting  in  a 
papermaking  installation.  These  findings  have  resulted  in  a number  of  recommendations  for  the  training  of 
troubleshooting  of  which  the  most  important  ones  are: 

The  training  of  a structured  approach  to  troubleshooting  should  be  embedded  in  regular  training  courses,  in  such  a 
way  that  this  structured  approach  becomes  a ’’second  nature"  of  technicians,  and  is  fully  integrated  in  training 
courses  on  maintenance  of  particular  installations.  For  reasons  of  transfer,  it  should  not  be  the  case  that  there  is  a 
separate  course  on  structured  troubleshooting,  apart  from  the  'regular'  maintenance  courses. 

What  is  meant  by  this  structured  approach  ? Cognitive  task  analyses  of  expert  troubleshooters  in  various  domains 
reveal  highly  similar  strategies  in  troubleshooting  across  domains.  This  strategy  consists  of  four  steps:  1.  Formulate 
problem  description.  2.  Generate  causes.  3.  Test.  4,  Repair  and  Evaluate. 

Ad  1 . Formulate  problem  description 

The  process  of  troubleshooting  is  started  when  the  technician  observes  abnormal  system  behaviour  or  receives  a 
notification  of  abnormal  system  behaviour.  Besides  recognising  abnormal  behaviour,  the  technician  also  needs  to 
identity  correct  phenomena.  This  is  an  important  task  because  determining  correct  phenomena  can  limit  the  number 
of  possible  causes.  By  determining  what  phenomena  are  still  normal,  the  technician  eliminates  certain  parts  that  do 
not  need  to  be  investigated  further. 

Ad  2.  Generate  causes 

Generating  possible  causes  is  a task  that  can  hardly  be  subdivided  further.  A distinction  can  be  made  between 
familiar  and  unfamiliar  problems.  In  case  of  familiar  problems,  causes  are  generated  mainly  by  a process  of 
recognition.  A particular  combination  of  normal  and  abnormal  phenomena  functions  as  a cue  for  the  retrieval  of  a 
particular  cause  from  memory.  For  instance,  the  technician  notes  that  the  TeleType Writer  does  not  print 
'application'.  Based  on  his  experience,  he  knows  that  this  symptom  is  most  often  caused  by  a faulty  DCL  TTW.  A 
technician  who  encounters  this  phenomenon  for  the  first  time  needs  to  use  reasoning  skills  and  schemata  in  order  to 
find  out  whether  the  phenomenon  is  caused  by  the  DCL  TTW  or  the  TTW  itself 

Ad  3.  Test 
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When  possible  causes  have  been  generated,  they  need  to  be  tested.  The  technician  needs  to  choose  the  right  testing 
method  and  the  right  testing  equipment.  By  'testing  equipment'  we  mean  all  tools  and  equipment  that  can  provide 
clarity  to  a technician  concerning  the  status  of  the  system.  The  next  task  is  performing  the  test.  This  amounts  to  the 
correct  setting  up  and  operating  of  testing  means  and  the  correct  reading  of  the  outcome  of  the  test.  After  this,  the 
outcome  of  the  test  needs  to  be  interpreted.  This  implies  that  the  technician  needs  to  formulate  an  expectation  before 
performing  the  test  and  subsequently  relates  the  outcome  of  the  test  to  this  expectation.  If  a technician  expects,  on 
the  basis  of  the  proper  functioning  of  the  system,  a low  level  (OV)  and  he  measures  and  observes  a high  level  (5V), 
then  the  cause  that  was  tested  needs  to  be  accepted,  provided  all  boundary  conditions  for  the  proper  execution  of  the 
test  have  been  fulfilled.  If,  however,  on  the  basis  of  the  outcome  of  a test,  a possible  cause  is  rejected,  the  next 
possible  cause  needs  to  be  tested.  When  all  possible  causes  have  been  rejected,  the  technician  needs  to  back  up  to 
the  task  of  generating  causes,  since  he  has  probably  overlooked  something.  When  the  correct  cause  has  been 
identified,  it  needs  to  be  evaluated. 

Ad  4.  Evaluate 

If  a technician  thinks  he  has  identified  the  correct  cause,  this  cause  needs  to  be  eliminated.  The  goal  of 
troubleshooting  is  to  return  the  system  to  its  normal  state  of  working,  which  asks  for  a thorough  check  for  system 
functioning. 

The  knowledge  of  installations  should  be  taught  at  a functional  level,  not  at  a component  level. 

A structured  approach  to  troubleshooting  should  not  be  trained  out  of  the  context  of  a particular  system,  but,  to 
enhance  transfer,  should  always  be  tuned  towards  a particular  system.  What  are  the  normal  characteristics  of  a 
particular  system,  e.g.  an  overheadprojector. 

The  method  is  supported  by  the  use  of  a fault  isolation  form,  that  is  filled  in  during  troubleshooting  sessions.  This 
forms  keeps  the  students  on  track,  prevents  them  from  forgetting  important  results  gained  sofar,  and  is  therefore  an 
important  aid  in  preventing  working  memory  overload. 

The  results  of  this  method,  more  thoroughly  described  elsewhere  (Schaafstal  and  Schraagen,  1998)  show  an 
enormous  improvement  in  performance.  Subjects  trained  in  the  new  method  solve  twice  as  many  problems  as 
before,  and  take  less  time  for  troubleshooting.  Apart  from  this,  there  is  a significant  improvement  in  the  quality  of 
reasoning  and  the  quality  of  system  knowledge 

The  Pedagogical  Approach  in  The  Troubleshooter 

The  method  described  above  functions  as  the  main  guide  for  troubleshooting  in  the  troubleshooting  system.  The 
implemented  fault  isolation  form,  which  has  to  be  filled  in  while  troubleshooting,  is  an  important  tool  in  determining 
the  progress  a student  makes  in  troubleshooting,  and  explicating  the  underlying  level  of  knowledge  and  skill, 
including  bugs  and  misconceptions.  The  evaluation  strategy  follows  from  an  overlay  model,  in  which  an  ideal  and 
expert  solution  is  compared  to  the  students  solution.  This  does  not  imply  that  only  one  solution  is  allowed  at  any 
point:  in  fact,  if  there  are  more  good  moves  possible  at  a particular  point  in  the  troubleshooting  process,  they  are  all 
allowed. 

Students  can  choose  between  a strongly  guided  method  or  a free  method  of  solving  problems.  In  the  guided  manner, 
the  student  is  very  strictly  evaluated  in  his  use  of  the  fault  isolation  form  and  is  given  appropriate  feedback  after 
finishing  each  stage  of  the  problem  solving  process.  If  he  makes  really  bad  mistakes,  he  receives  feedback  right 
away.  At  the  end  of  a case,  an  overall  feedback  is  given  to  evaluate  the  students  performance  in  troubleshooting. 

The  troubleshooter  contains  over  fifty  problems  with  various  variations,  and  with  a varying  level  of  difficulty.  A 
preset  selection  can  be  made  for  particular  types  of  problems  (e.g.  only  problems  with  the  overheadprojector),  and 
for  problems  with  a certain  level  of  difficulty.  This  ensures  that  the  chances  of  students  getting  bored,  because  they 
encounter  the  same  problems  repeatedly,  are  low. 

Since  the  training  of  troubleshooting  is  embedded  in  a realistic  environment,  not  only  troubleshooting  in  a strict, 
technical  sense  can  be  trained,  but  attention  can  also  be  given  to  training  and  emphasizing  the  right  attitude  of 
student-troubleshooters  towards  end-users  and  training  them  in  communicating  (e.g.  asking  the  right  questions  when 
setting  up  a problem  description). 
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3.  The  Virtual  Environment  in  The  Troubleshooter 


The  virtual  environment  in  The  Troubleshooter  was  modelled  after  a real  building,  housing  an  educational  institute. 
It  consists  of  an  enormous  amount  of  photographs,  which,  taken  together,  make  up  a realistic  environment, 
containing  of  classrooms,  corridors,  elevators,  staircases,  offices  etc.  The  possible  objects  for  troubleshooting  in  this 
building  are  coffeemakers  in  various  locations,  lighting,  overheadprojectors  in  the  various  classrooms,  and  a stereo- 
set. Of  all  the  technical  systems,  installation  documentation  is  available,  which  can  be  taken  with  you  on  your  way 
to  a particular  problem.  There  are  maps  available  of  the  layout  of  the  building,  test  equipment  is  available,  and 
communication  with  other  people,  e.g.  end-users  complaining  about  malfunctions  is  possible  through  simulated 
conversation. 


4.  The  Interaction  between  the  Virtual  Environment  and  the  Tutoring  System 

A session  with  The  Troubleshooter  (after  having  completed  a sign-up  procedure)  starts  with  some  notice  that  there  is 
some  system  malfunctioning  in  the  building.  The  task  of  the  chief  technician/maintainer  (i.e.  the  student)  of  the 
building  is  to  take  action  and  provide  adequate  solutions  and  repairs,  with  the  right  priority.  To  accomplish  this  task, 
he  may  ask  questions  to  the  complaining  person  through  the  phone,  write  down  notices,  go  to  the  piece  of 
equipment,  open  it,  take  measurements,  undertake  repairs.  While  collecting  information  leading  to  a problem 
description,  making  decisions  with  respect  to  the  likelihood  of  certain  causes,  while  collecting  measurement 
information,  he  fills  in  his  fault  isolation  form.  As  soon  as  the  troubleshooting  session  starts,  the  behavior  of  the 
student  is  monitored  in  the  background  by  the  tutoring  system.  Only  if  the  student  makes  really  bad  mistakes  during 
troubleshooting  (as  described  above),  or  after  having  finished  a stage  in  troubleshooting,  he  receives  an  qualitative 
evaluation  of  his  behavior:  what  went  well,  what  went  wrong,  has  the  student  performed  sufficiently  good  to 
progress  to  the  next  phase  of  the  problem.  After  finishing  the  problem  the  student  receives  an  end-evaluation:  what 
went  well,  and  what  could  be  improved  in  his  reasoning  throughout  the  troubleshooting  session. 


5.  Discussion  and  Conclusions 

We  believe  that  the  combination  of  a virtual  environment  and  an  intelligent  tutoring  system  is  a powerful  tool  in 
training  a complex  cognitive  skill  such  as  troubleshooting.  It  enables  training  in  a realistic  job-context,  not  only 
concentrating  on  the  technical  skills  that  are  needed  in  a particular  domain,  but  also  emphasizing  other  important 
skills  in  job-environments,  such  as  communication  skills.  In  doing  so,  we  believe  that  multi-media  environments  are 
a powerful  tool  in  enhancing  the  transfer  from  vocational  training  to  job-related  expertise  and  in  increasing  the 
attractiveness  of  training. 

Of  particular  importance  in  the  project  was  the  approach  to  troubleshooting  that  we  implemented:  it  provides  a firm 
grip  on  the  students  learning  process,  and  enables  intelligent  and  informative  feedback,  thus  maximizing  the 
possibilities  for  students  to  learn. 
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Abstract:  This  study  reports  preliminary  findings  about  critical  factors  that  promote  and  facilitate 
collaboration,  cooperation  and  communication  between  members  of  design  teams  who  create 
educational  software  for  classroom  use.  Factors  that  discourage  or  contribute  negatively  to  group 
interaction  and  production  were  also  considered.  Methods  employed  included  pre-  and  post-team 
surveys,  peer  reviews,  participant  interviews  and  journals,  and  observations.  Participants  were 
graduate  students  in  the  Instructional  Technology  program  in  the  College  of  Education.  Although 
multimedia  design  teams  in  commercial  settings  have  been  the  focus  of  many  articles  in  business 
publications,  there  is  little  research-based  literature  about  teams  composed  of  educators  developing 
courseware  for  classroom  settings.  In  order  for  multimedia  design  teams  to  work  together 
effectively,  members  must  possess  effective  and  well-developed  skills  of  collaboration,  cooperation 
and  communication.  Understanding  how  these  skills  may  be  developed  and  nurtured  plays  a critical 
part  in  courseware  development  and  may  facilitate  a more  effective  model  of  instruction. 

In  the  business  arena,  multimedia  design  teams  have  been  used  extensively  over  the  past  decade  to  develop  software  for 
commercial  purposes.  In  educational  settings,  however,  most  courseware  has  been  developed  by  individuals  who,  for  the  most 
part,  do  not  have  the  wide  range  of  skills  needed  to  produce  a quality  product.  By  working  in  multimedia  design  teams,  educators 
can  combine  their  skills  to  produce  both  a worthwhile  and  well-designed  piece  of  software  expressly  for  their  educational  setting. 

This  study  reports  preliminary  findings  about  critical  factors  that  promote  and  facilitate  collaboration,  cooperation  and 
communication  between  members  of  design  teams  who  create  educational  software  for  classroom  use.  Factors  that  discourage  or 
contribute  negatively  to  group  interaction  and  production  were  also  considered.  Methods  employed  included  pre-  and  post- team 
surveys,  peer  reviews,  participant  interviews  and  journals,  and  observations.  Participants  were  graduate  students  in  the 
Instructional  Technology  program  in  the  College  of  Education.  The  students  were  enrolled  in  two  sections  of  an  instructional 
technology  course,  CUIN  7327,  the  Design  and  Development  of  Instructional  Packages  during  spring  semesters,  1996  and  1997. 
In  1998,  this  course  was  renamed  to  more  closely  match  course  objectives  and  divided  into  two  courses  which  are  taken  in  a fall- 
spring sequence,  CUTN  7327,  The  Collaborative  Design  of  Multimedia,  and  CUTN  7357,  The  Collaborative  Development  of 
Multimedia.  Data  was  also  gathered  from  the  fall,  1998  section  of  this  two  sequence  course.  The  students  who  enrolled  in  these 
classes  were  from  a variety  of  backgrounds,  but  most  had  teaching  experience  in  either  K-12  classrooms  or  in  higher  education. 

This  project  fits  into  a larger  research  focus  on  educational  multimedia  courseware  development  and  authoring 
competencies  for  educators.  Although  multimedia  design  teams  in  commercial  settings  have  been  the  focus  of  many  recent 
articles  in  business  publications,  there  is  little  research-based  literature  about  teams  composed  of  educators  developing 
courseware  for  classroom,  not  commercial,  settings.  In  order  for  multimedia  design  teams  to  work  together  effectively,  members 
must  possess  effective  and  well-developed  skills  of  collaboration,  cooperation  and  communication.  Understanding  how  these 
skills  may  be  developed  and  nurtured  plays  a critical  part  in  courseware  development  and  may  facilitate  a more  effective  model 
of  instruction. 

Educational  Multimedia 

Using  the  computer  to  control  and  deliver  a full  range  of  audiovisual  media  offers  educators  a way  to  regain  the  power  and 
strength  of  audio,  video,  animations  and  interactive  simulations,  and  to  build  those  features  into  educational  materials.  These 
materials  actively  engage  students  by  giving  them  the  best  possible  presentation  and  explanations  with  whatever  media  are  the 
most  suitable  to  the  task.  Interactive  multimedia  implies  multiple  forms  of  communications  media,  controlled,  coordinated  and 
integrated  by  the  computer.  The  computer  allows  the  individual  user  to  interact  with  and  control  the  flow  of  information.  Instead 
of  reading  the  information  in  a conventional  linear  way,  the  user  is  encouraged  to  pursue  links  from  one  topic  to  another 
following  a uniquely  personal  trail  through  the  information.  The  user  becomes  an  active  participant  in  the  flow  of  the  narrative, 
actively  choosing  material,  and  in  some  cases,  modifying  the  content  itself  to  suit  his  or  her  needs  and  interests. 
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In  the  past  few  years  there  has  been  a dramatic  growth  in  the  interest  and  possibilities  of  using  multimedia  in  educational 
settings  (Ferretti,  1993).  However,  Wiburg  (1995)  noted  that  although  there  are  thousands  of  articles  about  multimedia,  there  is 
very  little  research.  In  addition,  Wiburg  continued,  much  of  the  research  that  does  exist  about  multimedia  has  been  conducted  in 
other  areas  and  is  peripheral  to  the  concerns  of  most  educators. 

Effective  Use  of  Technology 

There  are  many  viewpoints  on  the  competencies  that  educators  need  to  effectively  use  computer  technology  in  their 
classrooms.  Teacher  education  organizations  have  proposed  that  educators  should  have,  "The  ability  to  integrate  instructional 
technology  into  the  classroom  to  facilitate  interdisciplinary  teaching  and  learning,  supplement  instructional  strategies,  design 
instructional  materials,  and  enhance  hands-on  experiences  and  problem  solving"  (National  Council  for  the  Association  of 
Teacher  Education,  1 992)  and  "demonstrate  knowledge  of  uses  of  multimedia,  hypermedia  and  telecommunications  to  support 
instruction"  (International  Society  for  Technology  in  Education,  1992).  D'Ignazio  (1993)  stated  that  multimedia  has  a great 
potential  for  educators  at  all  grade  levels  by  bringing  even  "the  driest  areas  of  curriculum  to  life"  and  by  turning  the  classroom  in 
"an  exciting  studio-like  area"  (p.  490). 

Authoring  Courseware 

Lockard,  Abrams,  and  Many  (1997)  listed  several  reasons  why  teachers  may  find  it  advantageous  to  develop  their  own 
software.  First,  teachers  may  not  find  appropriate  software.  Educational  software  companies  must  develop  software  that  meets 
the  needs  of  many  students  in  order  to  be  cost-effective.  Second,  budgetary  considerations  may  prevent  teachers  from  purchasing 
a commercial  software  package.  Third,  educational  software  is  not  designed  with  specific  students  and  individual  needs  in  mind. 
Finally,  teachers  may  have  a personal  preference  for  creating  their  own  materials  including  the  natural  desire  of  experienced 
teachers  to  produce  their  own  educational  materials  of  all  kinds. 

Teachers  are  logical  developers  of  courseware  for  their  classrooms.  They  have  a better  knowledge  of  the  content  than  do 
commercial  software  designers,  and  they  bring  to  the  authoring  environment  their  knowledge  of  good  pedagogy  and  teaching 
skills.  Teachers  also  have  the  concrete  experience  of  actually  teaching  the  content.  They  have  learned  by  experience  what 
strategies  work  and  which  ones  must  be  reinforced  in  multiple  ways.  Teachers  understand  the  needs  of  their  audience  better  than 
anyone  else.  Many  teachers  may  actually  prefer  to  develop  their  own  materials  just  as  they  do  with  other  classroom  materials 
such  as  handouts  and  worksheets.  Kearsley  (1986)  suggested  that  teachers  should  author  their  own  programs  in  order  to  become 
better  software  evaluators.  He  stated  that  the  ability  to  distinguish  a good  software  program  from  another  is  one  of  the  critical 
skills  needed  by  software  designers.  He  continued  by  adding  that  evaluation  skills  and  design  skills  are  mirror  images  of  one 
another.  Finally,  teachers  are  the  ultimate  users  of  the  software. 

As  early  as  1984,  researchers  have  cited  the  need  of  some  teachers  for  producing  their  own  software  and  have  contended 
that  software  development  is  a valid  topic  for  teachers  to  study  at  the  graduate  level  (Young,  1984;  Monahan,  1987;  Overbaugh, 
1994).  In  addition.  Turner  and  Dipinto  (1992)  stated  that  hypermedia  gave  students  new  insights  into  writing.  Using  hypermedia 
caused  students  to  view  information  as  more  than  large  streams  of  text,  but  more  as  chunks  of  information  that  were  linked. 
Turner  and  Dipinto  reported  that  the  study  indicated  that  "the  time  students  invest  in  learning  to  use  software  and  hardware  not 
only  gives  them  a powerful  new  medium  of  communication  but  may  also  give  them  new  insights  into  organizing  and 
synthesizing  information"  (p.  198). 

Multimedia  Design  Teams 

In  spite  of  these  compelling  reasons  for  educators  to  author  courseware  for  their  unique  classroom  environment,  it  is  not 
reasonable  to  expect  that  educators  will  be  able  to  produce  professional  looking  courseware  taking  into  consideration  both  the 
limitations  of  time  and  production  capabilities.  The  range  of  skills  needed  to  produce  a multimedia  project  include  a detailed 
knowledge  of  computers,  information  processing,  graphic  design,  sound  and  video  editing,  an  authoring  language,  and 
instructional  design.  It  is  practical,  therefore,  to  view  the  development  of  educational  courseware  much  like  the  development  of 
commercial  software  utilizing  the  concept  of  multimedia  design  teams  with  specialists  in  the  various  production  fields. 

A design  team  usually  consists  of  several  individuals  with  specific  talents  who  contribute  in  some  way  to  the  development 
of  a shared  goal.  Martin  (1995)  stated:  "A  team  refers  to  a small,  tightly  knit  team  of  people  intensely  focused  on  meeting  a 
specified  objective  that  all  members  of  the  team  are  committed  to  accomplishing  together."  (p.65)"  In  a successful  and  effective 
design  team,  each  member  feels  a sense  of  trust,  commitment,  and  responsibility  for  other  members  and  a shared  vision  for  team 
goals  and  objectives. 

Multimedia  design  teams  are,  for  the  most  part,  organized  around  a project  manager,  an  instructional  designer,  a graphic 
artist,  an  audio  and  video  producer,  a content  writer  and  a programmer.  Working  with  a client,  the  team  develops  learning 
materials  that  may  include  some  form  of  computer-assisted  software  program,  a manual  or  guide  and  a set  of  usually  print-based 
teacher  materials. 

One  of  the  most  important  factors  in  a team  approach  to  courseware  design  is  the  interaction  and  communication  among 
team  members  (Ally,  1 985).  Team  members  are  required  to  work  cooperatively  and  must  learn  to  function  as  a unit.  In  spite  of 
several  different  areas  of  specialization  and  different  levels  of  interaction  with  one  another,  team  members  must  be  able  to 
communicate  ideas  and  meanings  effectively. 

In  the  business  field,  design  teams  may  spend  several  sessions  learning  to  work  collaboratively.  Unlike  the  highly  structured 
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procedures  of  cooperative  learning,  a collaborative  learning  environment  empowers  the  learners  to  actively  seek  knowledge. 
Teams  become  more  autonomous  as  they  learn  to  work,  interact  and  nurture  each  other  while  working  toward  the  team  goal. 
Bouton  and  Garth  (1983)  cited  the  fact  that  through  a collaborative  learning  situation,  the  learners  are  transformed  from 
spectators  to  being  involved  in  more  complex  interactions  in  an  active  learning  process. 

This  study  investigated  some  critical  factors  that  enhance  collaboration,  communication  and  cooperation  among  team 
members  developing  educational  courseware.  This  research  has  implications  that  will  impact  both  students  and  faculty  in  the 
design  and  development  of  educational  multimedia  courseware.  Current  trends  in  educational  research  indicate  that  the  delivery 
of  multimedia  materials  will  increase  over  the  next  decade. 

Research  Design 
Objectives 

Goals  for  this  study  included: 

1.  determining  those  behaviors  which  promote  group  cooperation  and  those  behaviors  which  contribute  in  a negative 
manner  to  teamwork  and  interactions 

2.  soliciting  from  participants  ways  that  positive  behaviors  may  be  learned  and  developed 

3.  examining  solutions  to  problems  which  occur  between  members  of  a team 

4.  creating  guidelines  and  suggestions  for  promoting  positive  team  interaction 

5.  suggesting  what  role  previous  team  experiences  may  have  on  group  communication  and  behavior 

Procedures 

Procedures  included  a detailed  literature  review  concerning  effective  team  qualities  in  multimedia  design  teams  including 
those  used  in  the  business  field,  pre-  and  post-team  surveys,  peer  reviews,  participant  interviews  and  journals,  and  observations. 
Participants  included  42  students  in  two  graduate  instructional  technology  courses,  CUIN  7327,  the  Design  and  Development  of 
Instructional  Packages  during  spring  semester,  1996  and  1997,  and  18  students  in  CUIN  7327,  The  Collaborative  Design  of 
Multimedia  (renamed  and  redesigned  fall,  1998). 

The  research  plan  began  with  an  initial  assessment  of  computer  skills,  multimedia  skills  and  knowledge  of  team  strategies 
based  on  two  surveys  completed  by  each  group  of  students  the  first  night  of  class.  The  Computer  Skills  Assessment  asked 
students  to  rate  their  skills  in  such  areas  as  proficiency  in  specific  types  of  software  programs  and  the  ability  to  use  hardware 
peripherals  such  as  scanners  and  digital  cameras.  The  Multimedia  Skills  Assessment  asked  students  to  rate  their  skills  in  areas 
such  as  the  use  of  specific  authoring  programs  such  as  Macromedia  Authorware  and  Director. 

In  addition,  this  survey  asked  students  to  rate  their  multimedia  project  experience  in  such  areas  as  designing  navigational 
structures,  building  a prototype  and  producing  video. 

Design  Teams 

Based  on  this  survey  and  student  interviews,  each  class  was  divided  into  design  teams  consisting  of  the  following  members: 
a project  manager,  a graphic  designer,  a videographer,  a programmer  and  a content  writer.  During  the  fall  1996  semester,  four 
teams  of  five  members  each  worked  with  four  different  faculty  members  to  develop  a multimedia  project  according  to  that 
client's  specifications  over  the  course  of  the  fourteen-week  semester.  During  the  fall  1997  semester,  four  teams  worked  on 
different  sections  of  a large  project  for  one  client.  During  the  fall,  1998  semester,  six  teams  worked  with  two  different  clients  to 
design  educational  software  for  an  undergraduate  history  class  in  the  college  of  Humanities,  Fine  Arts  and  Communications  and 
an  undergraduate  human  resources  class  in  the  College  of  Hotel  and  Restaurant  Management.  The  purpose  of  this  project  was  to 
provide  students  with  an  introductory,  but  realistic,  experience  in  developing  a piece  of  software  in  a team  environment  for  a 
specific  group  of  users. 

Design  Process 

Students  were  assigned  to  different  teams  based  on  the  skills  analysis  they  completed  on  the  first  night  of  class,  the 
instructor's  knowledge  of  each  individual's  background,  and  student  preference.  For  the  most  part,  the  graduate  students  in  this 
class  had  completed  both  the  introductory  instructional  design  course  and  overview  of  multimedia  course. 

The  next  few  class  sessions  were  spent  on  discussing  team  roles  and  developing  competencies  that  would  allow  students  to 
complete  their  part  in  the  projects.  Teams  also  met  with  their  client  weekly  to  discuss  and  develop  preliminary  analysis  and  goals 
for  the  project.  Students  soon  realized  that  working  in  a team  and  trying  to  develop  a piece  of  software  for  someone  else  is  a lot 
of  work. 

Communication  began  to  develop  both  in  class  team  meetings  and  through  email.  One  student  noted:  "I  remember  all  of  us 
being  polite  to  each  other  at  first.  However,  we  had  a lot  of  creative  work  to  accomplish  and  'polite'  wasn't  exactly  what  I had  in 
mind  from  this  team.  But  as  soon  as  we  got  to  know  each  other,  our  meetings  became  more  open  and  lively."  Another  member  of 
the  team  reflected,"  I found  these  discussions,  sometimes  heated  ones,  to  be  thought  provoking  and  interesting.  Ideas  were  put 
forth,  discussed  and  then  tested  for  consensus.  We  tossed  a lot  of  ideas  back  and  forth  while  we  were  trying  to  visualize  how  this 
product  should  look  and  what  was  needed  to  make  it  a reality." 

In  addition  to  communication  skills,  one  student  noted  in  his  Journal,  "I've  noticed  that  listening  skills  are  especially 
important  on  a project  team  ...  there  are  times  in  our  groups  where  people  don't  stop  to  listen  to  each  other's  ideas  and  it  really 
slows  progress  as  we  discuss  (debate)  a single  item  over  and  over.  It's  much  easier  when  everyone  takes  a turn  speaking,  others 
listen,  the  issue  is  out  on  the  table  and  a decision  is  made."Some  problems  were  encountered  when  quiet  students  were  reluctant 
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to  voice  their  opinions:  "I  believe  other  (students)  have  been  frustrated  or  have  kept  quiet  because  of  the  vocal  nature  of  a few 
members  of  the  class." 

Teams  also  foundered  a bit  at  first  trying  to  develop  ideas  and  models  for  their  projects.  Members  soon  realized  that 
decisions  should  be  made  by  the  team  and  not  by  individuals.  It  took  most  of  the  teams  several  weeks  of  meetings  both  as  a team 
and  with  the  client  to  develop  a plan  and  direction  for  the  projects.  Some  team  members  noted  that  success  was  related  to  early 
decision  making,  "We  had  to  'start  from  scratch'  a few  times,  but  those  times  were  well  justified  - and  were  team  decisions.  It 
took  us  about  two  weeks  before  we  settled  on  a direction.  And  we  did  not  make  a major  change  after  that  point." 

Some  teams  spent  several  sessions  establishing  team  rules  for  discussion;  this  usually  facilitated  decision  making  and 
communication.  Although  University  of  Houston  graduate  students  usually  work  full  time  and  often  commute  as  much  as  an  hour 
to  come  to  class  in  the  evenings,  team  members  met  many  meeting  requirements  by  using  email. 

Many  students  felt  apprehensive  about  time  considerations,  working  in  a team  situation  and  being  evaluated  by  their  peers 
rather  than  just  the  instructor,  "I  fear  that  the  amount  of  time  needed  for  production  will  be  very  demanding.  The  team  process 
and  work  is  worth  the  course  alone!  I'm  learning  tremendous  lessons.  It  is  also  a very  positive  experience  with  each  problem, 
group  interaction,  individual  interaction  building  a team  spirit.  I like  this!"  A student  remarked,  "I  think  everyone  is  'nervous' 
because  the  expectations  of  the  finished  product  is  high  &shy;  everyone  wants  to  create  something  really  good." 

Student  Resources 

Several  different  types  of  resources  were  available  for  student  learning.  These  included  the  textbooks,  I Sing  the  Body 
Electronic  by  Fred  Moody  and  Multimedia  Making  It  Work  by  Tay  Vaughn.  In  addition  Web  pages  for  each  class  were 
developed  and  used  as  means  for  sharing  information  and  progress  on  the  projects. 
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Figure  1.:  Web  Site  for  CUIN  7327  for  fall,  1998  semester 

Several  students  remarked  on  the  Web  pages  in  their  journals  noting,  "The  web  pages  have  some  interesting  links  and  the 
material  is  organized  in  a logical  and  consistent  fashion...  The  pages  are  a good  way  to  stay  connected  to  the  class  when  at  work 
or  at  home  - important  in  creating  a 'community*  because  of  job  or  personal  schedules." 

To  access  the  pages  used  during  fall  and  spring,  1998:  http://www.coe.uh.edu/courses/cuin7327 

Hypergroups 

In  addition  to  face-to-face  communication.  Hypergroups,  a web-based,  hypertextual  interface  for  the  automated  E-mail 
distribution  lists  was  used  to  facilitate  discussion  among  teams  and  clients.  There  were  several  discusison  areas  created.  In 
addition  to  a common  discussion  area  for  all  students,  there  were  private,  password  protected,  areas  for  individual  teams. 

Project  Presentations 

At  the  end  of  each  semester,  team  members  shared  their  final  projects  in  a special  evening  presentation.  Although  the 
content  and  quality  of  each  project  differed  greatly,  the  goals  of  the  class  were  clearly  met  - students  experienced  working  in  an 
authentic  learning  situation.  Several  teams  exceeded  expectations  in  functioning  as  a cohesive  unit  noting,  "We  functioned  as  a 
team  and  not  a collection  of  individuals.  Our  group  rose  to  the  highest  definition  of  a team." 

Preliminary  Findings 

In  each  semester,  a mid-semester  and  final  survey  on  Project  Team  Attitude  was  completed  by  each  member  anonymously. 
This  survey  has  been  adapted  from  closely  related  surveys  used  in  business  (Systems  One,  1996).  In  addition,  students 
participated  in  both  mid-semester  and  final  peer  reviews.  Peer  reviews  asked  students  to  rate  such  competencies  as  "Quality  of 
Assignments,"  "Responsibility  to  Team,"  "Contributions  to  Team,"  "Attitude,"  "Teamwork,"  and  "Overall  Team  Member"  for 
each  member  of  the  team  and  to  also  rate  themselves.  Comments  at  this  time  included,  "My  group  is  very  eager  and  positive 
&shy;  we  feel  a sense  of 'group.'  I still  think  we're  showing  our  best  selves." 

Direct  observation  of  team  meetings  and  interviews  with  team  members  were  also  conducted  by  the  researcher  at  critical 
milestones  for  courseware  development  over  the  course  of  the  semester.  In  addition,  team  members  were  also  required  to  keep 
journals  of  their  thoughts  and  feelings  during  team  meetings  and  other  development  opportunities. 
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Figure  2:  Hypergroups  Menu  for  Project  Teams 


Midterm  Evaluations 

Participants  used  a likert  scale  to  rate  their  perceptions  of  their  team  health  and  well-being.  Using  a scale  ranging  from  1 
strongly  agree  to  5 strongly  disagree,  participants  responded  to  such  statements  as:  The  goals  of  the  team  are  clearly  understood 
by  team  members.  Team  meetings  are  productive.  I feel  that  I have  been  an  important  part  of  this  team  so  far. 

The  highest  disagreement  among  all  teams  was  shown  on  statements  such  as,  "The  team  is  using  their  meeting  time 
effectively."  and  " The  team  is  learning  to  effectively  address  and  resolve  problems  within  the  team."  The  most  agreement  among 
all  teams  occurred  on  such  statements  as,  "The  team  is  developing  effective  strategies  to  meet  objectives,"  and  "Team  members 
are  making  efforts  to  get  jobs  completed."  The  ratings  ranged  from  2.65  to  1.85.  These  collective  results,  not  individual  ratings, 
were  shared  with  the  participants  and  strategies  for  facilitating  better  interaction  and  problem  solving  were  addressed  in  a group 
discussion.  At  this  stage  in  the  project,  there  was  a steep  learning  curve  and  many  problems  were  being  worked  out,  both  with  the 
clients  and  between  team  members.  The  overall  attitude  of  the  project  teams  ranged  from  4 responses  of  excellent,  6 responses  of 
very  good,  4 good,  and  4 fair. 

Final  Evaluations 

Similar  evaluations  were  completed  at  the  end  of  each  semester.  As  in  the  midterm  evaluations,  one  of  the  highest  areas  of 
dissatisfaction  among  all  teams  continued  to  be,  "The  team  is  learning  to  effectively  address  and  resolve  problems  within  the 
team."  The  most  agreement  among  all  teams  occurred  on  such  statements  as,  I feel  that  I have  been  an  important  member  of  this 
team  so  far,"  "Team  members  are  making  efforts  to  get  jobs  completed,"  and  "Team  members  have  been  generally  positive  in 
their  outlook  regarding  the  project."  The  ratings  ranged  from  2.09  to  1 .52. 

Peer  Reviews 

Peer  reviews  asked  students  to  rate  such  competencies  as  "Quality  of  Assignments,"  "Responsibility  to  Team," 
"Contributions  to  Team,"  "Attitude,"  "Teamwork,"  and  "Overall  Team  Member"  for  each  member  of  the  team  and  to  also  rate 
themselves.  Consistently  across  all  teams,  team  members  rated  their  other  members  above  average  on  most  categories.  Peer 
reviews  in  all  three  semesters  of  the  course  have  proven  very  effective  in  addressing  grading  issues. 

1998  Redesign  of  the  Course 

In  the  fall  semester  of  1 998,  the  course  was  redesigned  and  divided  in  two  parts  which  students  take  in  sequence.  Students 
had  noted  on  their  evaluations  that  the  time  was  not  sufficient  to  both  design  and  develop  the  piece  of  software  in  one  semester. 
Students  now  design  the  software  during  the  fall  semester.  They  develop  it  using  Macromedia's  Authorware  and  perfonn  a simple 
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evaiution  during  the  spring  semester.  This  redesign  has  been  very  effective  from  both  the  students’  and  teacher’s  points  of  view. 
Because  of  the  additional  time,  students  have  had  more  opportunities  to  both  present  and  share  informally  knowledge  of  the 
instructional  design  process  and  authoring  strategies.  In  addition,  one  client  has  been  more  involved  in  the  process,  learning  the 
software  and  participating  in  both  the  design  and  development  process  every  class.  It  is  also  interesting  to  note  that  team  roles 
were  not  assigned  during  this  semesterso  that  each  team  could  develop  the  most  effective  structure  for  that  particular  group. 

Discussion 

Although  many  components  of  this  research  are  still  being  evaluated  and  researched,  it  seems  from  a preliminary 
investigation  that  the  areas  of  communication,  collaboration  and  cooperation  in  a team  environment  are  rich  in  experiences  and 
nuances.  Successful  teams,  as  demonstrated  in  both  project  development  and  peer  reviews,  were  able  to  work  together  and 
resolve  problems  quickly  without  any  single  person  assuming  a dominant  or  overpowering  role.  Teams  that  experienced  some 
dissatisfaction  with  the  team  process  typically  did  not  form  a cohesive  group  or  work  together  to  solve  problems  that  arose. 
Although  a leadership  role  is  expected  of  the  project  manager,  successful  teams  clearly  developed  a process  for  equal 
participation  among  team  members.  Further  research  is  needed  to  explore  other  influences  such  as  time  constraints  and  different 
skill  levels  among  members. 
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Introduction 

As  we  enter  the  2U‘  century  and  embark  on  the  information  age,  many  institutions  and 
schools  are  turning  to  technology  to  enhance  their  programs  and  to  expand  their  horizons. 
Geography  is  no  longer  a barrier  for  preventing  people  from  accessing  information  and 
education  (Dixon,  1996).  Rapid  developments  in  teleconummication  technologies,  tightening 
budgets,  and  changes  in  student  demographics  have  stimulated  an  increasing  interest  in 
distance  education  in  all  educational  settings  (Honeyman  & Miller,  1993).  Through  the  use  of 
videoconferencing,  computers,  modems  and  the  Internet,  schools  are  able  to  deliver  courses 
and  degree  programs  to  students  in  distance  locations  without  requiring  them  to  set  foot  in  a 
traditional  classroom. 

Distance  education  is  an  emerging  technology  intended  to  deliver  both  resident  and 
remote  site  instruction  and  educators  who  use  distance  education  must  provide  educational 
experiences  that  will  equal  resident  education  in  terms  of  quality  and  quantity.  Both  resident 
and  distance  education  are  intended  to  provide  students  with  valid,  useful  information  that 
promotes  learning.  According  to  Swan  and  Brehmer  (1994),  distance  education  refers  to  “the 
simultaneous  delivery  of  instruction  from  a host  site  or  classroom  to  remote  site(s),  coupled 
with  real  time  live  audio  and  real  time  live  video  interaction  between  teacher  and  student(s)  - 
not  correspondence,  video,  or  internet  courses.”  Distance  education,  according  to  the  United 
States  Department  of  Agriculture,  is  a process  to  create  and  provide  access  to  learning  when 
the  source  of  information  and  the  learners  are  separated  by  time  and  distance,  or  both.  Rather 
it  is  the  process  of  designing  educational  experiences  that  best  suit  the  learner  whom  may  not 
be  in  a classroom  with  an  instructor  at  a specific  time.  Murphy  (1997)  defined  distance 
education  as  a premeditated  and  persistent  attempt  to  promote  learning  in  an  enviromhent  that 
includes  geographic,  temporal,  or  pedagogical  distance. 

Swan  (1995)  noted  that  advancements  in  communications  technology  have  dissolved 
some  of  the  major  distinguishable  characteristics  between  distance  education  and  traditional 
education.  According  to  Swan  and  Jackman  (1996),  strategies  of  teaching  at  a distance  and 
host  site  are  converging  because  traditional  teaching  strategies  are  being  abandoned  or 
modified  in  favor  of  a problem-based,  resource-based,  or  activity  based  approach  that  de- 
emphasizes  the  teacher  as  the  main  source  of  knowledge. 

The  increasing  availability  of  telecommunications  has  provided  vocational  or  applied 
education  faculty  with  unique  opportunities  to  plan  and  deliver  distance  education  courses  and 
programs.  Vocational  education  students  are  also  enrolling  in  more  distance  courses  and 
programs  due  to  availability,  time,  and  place.  However,  there  is  a lack  of  studies  that 
compare  student  achievement  by  students  receiving  instruction  via  distance  technology  versus 
students  receiving  the  same  instruction  through  the  traditional  resident,  host  site,  classroom 
setting. 

Purpose/Objectives 

The  purpose  of  the  study  was  to  ascertain  if  student’s  achievement  differences  existed  in 
courses  delivered  via  distance  education.  Specific  research  objectives  were: 
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1 . Describe  student’s  enrolled,  remote  site  and  host  site,  in  distance  education  courses  on 
selected  demographic  characteristics 

2.  Ascertain  if  differences  existed  between  remote  site  and  host  site  student’s 
achievement  based  on  grades/scores  (GPA)  obtained  by  grade  level. 

3.  Ascertain  if  differences  existed  between  remote  site  and  host  site  student’s 
achievement  (final  grade  received)  based  on  individual  course  success. 

Methodology 

Definitions 

Host  site:  The  school  where  the  instructing  teacher  is  residing  or  located  and  where  the 
course  originates  from  during  the  course  sessions.  Location  where  the  instructing  teacher  is 
physically  in  the  classroom  with  the  students. 

Remote  site:  The  receiving  classroom  where  the  students  are  physically  in  the  school  setting 
but  the  instructing  teacher  is  teaching  students  via  an  electronic  format.  Location  where  the 
instructing  teacher  is  not  physically  in  the  classroom  with  the  students 

Populations 

The  population  of  remote  site  and  host  site  schools  was  identified  from  an  alphabetical 
list  of  secondary  schools  utilizing  distance  education  technologies  supplied  by  the  State 
Department  of  Pubic  Instruction.  The  schools  were  all  located  within  one  Midwestern  State. 
Each  of  the  identified  schools  administration  were  asked  to  participate  in  the  study.  From  the 
total  list  of  schools  using  distance  education,  a total  population  of  schools  willing  to 
participate  were  identified,  N = 46  schools.  From  this  revised  list  of  schools,  a study  sample 
was  selected  using  appropriate  cluster  sampling  methods  outlined  by  Wiersma  (1995). 

As  each  school  was  selected,  all  courses/classes  being  offered  via  distance  education 
from  that  school  were  selected  for  this  study.  Each  student  in  the  study,  n = 623,  was  enrolled 
in  at  least  one  course  being  offered  via  distance  education.  To  retain  the  confidentiality  of  the 
student,  administrators  or  the  assigned  school  representative  was  asked  to  assign  an 
identification  number  to  each  student.  This  number  was  used  to  report  all  data  concerning 
that  student.  The  researcher  did  not  know  or  have  knowledge  of  any  student  name,  only  their 
assigned  number. 

Instrumentation  / Data  Collection 

The  study  instrument  was  completed  from  student’s  records  by  the  administration  or 
assigned  school  representative.  The  instrument  was  assessed  for  content  and  face  validity  by 
graduate  students,  teacher  educators,  and  state  supervisors  in  vocational  education.  This 
procedure  was  followed  because  more  than  one  person  in  a school  was  responsible  for 
providing  the  data  required  by  the  researcher.  Reliability  of  the  instrument  was  .89 
(Cronbach’s  alpha  coefficient).  They  were  asked  to  report  gender,  grade  level  of  student, 
period(s)  taking  distance  education  courses,  name(s)  of  specific  distance  education  course(s), 
location  of  student  (remote  or  host  site),  total  daily  assignment  scores,  exams  and/or  quiz 
scores,  and  final  exam  score. 

Data  Analysis 

Data  were  analyzed  using  the  Statistical  Package  for  the  Social  Sciences  (SPSS 
Version  6.1)  for  Windows.  All  tests  were  run  at  the  .05  alpha  level. 
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Results 

Objective  1:  Demographic  Characteristics 

Demographically,  students  in  the  study  were  predominately  located  at  remote  sites, 
68. 1%  (424),  and  3 1 .9%  at  host  sites  (199).  In  the  study,  there  were  245  male  students 
(39.3%)  and  378  female  students  (60.7%).  The  study  identified  ten  individual  courses  being 
offered  via  distance  education,  one  course  was  eliminated  from  the  study  because  no  results 
were  made  available  to  the  researcher.  Table  1 identifies  the  total  number  of  students  by 

grade  level,  9"  = 56  (9%),  lo"  = 126  (20.2%),  1 1"  = 161  (25.9%),  and  I2"  = 280  (44.9%). 
Table  1 Grade  Level  of  all  Students 

(P  Grade  Level 

300 

200 

100 

0 

As  shown  in  Table  2,  the  study  group  is  divided  into  groups  identified  by  specific 
course  name  and  by  location  receiving  the  course.  Frequencies  and  percentages  are  used  to 
identify  students  enrolled  in  distance  education  courses  all  sites  in  the  study. 


Table  2 Individual  Course  Enrollment  Frequencies  and  Percentages 

Course  Name 

All  Sites 

Host  Site 

Remote  Site 

N 

% 

N 

% 

N 

% 

Foreign  Language 

231 

37.1 

91 

39.4 

140 

60.6 

Ag  Business  Mgt. 

77 

12.4 

3 

3.9 

74 

96.1 

Vocational  Marketing 

21 

3.4 

14 

66.7 

7 

33.3 

Natural  Resources 

42 

6.8 

4 

9.5 

38 

90.5 

Math  - Calculus 

119 

19.1 

63 

52.9 

56 

47.1 

Chemistry 

70 

11.2 

9 

12.9 

61 

87.1 

Art 

14 

2.2 

5 

35.7 

9 

64.3 

Statistics 

14 

2.2 

6 

42.9 

8 

57.1 

Animal  Science 

35 

5.6 

4 

11.4 

31 

88.6 

Total 

623 

100 

199 

31.9 

424 

68.1 

Objective  2 

One  way  analysis  of  variance  was  used  to  test  if  differences  in  student  achievement 
existed  between  remote  and  host  site  students  based  on  mean  GPA.  No  significant  differences 
were  found. 

Table  3 identifies  the  mean  Grade  Point  Average  (GPA)  of  students  located  at  remote 
sites  and  at  host  sites.  The  grade  point  averages  of  students  enrolled  in  distance  education 
courses  at  both  the  remote  and  host  sites  were  very  similar. 
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Table  3 
Location 


Grade  Point  Average  According  to  Location  Receiving  Course 
N GPA SD 


Remote  Site 

424 

3.19 

.76 

Host  Site 

199 

3.14 

.84 

Total  All  Sites 

623 

3.18 

.78 

ANOVA  was  used  to  test  if  differences  in  student  achievement  existed  between  grade 

th 

levels  based  on  mean  GPA.  There  were  significant  differences  among  the  four  groups  (9 
grade,  lo'  grade,  1 1 grade,  and  12  grade).  The  analysis  of  the  data  yielded  an  F value  of 
2.84  with  a p of  .037.  Ninth  grade  students  earned  a GPA  significantly  higher  than  1 1 grade 

th  , 

students  and  9 grade  students  earned  a significantly  high  GPA  than  did  12  grade  students  as 
reported  in  Table  4. 

Table  4 Comparison  of  students  Grade  Point  Average  by  grade  level  and  by  site 

Grade  Level  / N GPA  SD  SE 


9 Host  Site 

2 

3.35 

.92 

.65 

9 Remote  Site 

54 

3.43 

.73 

.09 

9 Total 

56 

3.43 

.72 

.09 

GPA  / Grade  Comparison 

9, 10  E=  .095,  9, 11  E= 

.045*,  9, 12  E 

= .005* 

10  Host  Site 

70 

3.19 

.71 

.08 

10  * Remote  Site 

56 

3.25 

.58 

.07 

10*  Total 

126 

3.22 

.65 

.05 

GPA  / Grade  Comparison 

10, 11  E=  .712, 10, 12  E 

;=  .171 

1 1 * Host  Site 

71 

3.31 

.57 

.06 

1 1 * Remote  Site 

90 

3.08 

.79 

.08 

11  * Total 

161 

3.18 

.71 

.05 

GPA  / Grade  Comparison 

11, 12  E=  .297 

12*  Host  Site 

56 

2.87 

1.16 

.15 

12  * Remote  Site 

224 

3.16 

.78 

.05 

12*  Total 

280 

3.10 

.87 

.05 

Objective  3 

One  way  analysis  of  variance  was  used  to  test  if  differences  existed  between  remote 
site  students  GPA  and  host  site  students  GPA  by  individual  course.  There  were  no  significant 
differences  among  the  two  groups  (remote  site  and  host  site).  The  analysis  yielded  an  F value 
of  .5 1 with  a p = .47  as  reported  in  Table  5.  There  were  significant  differences  between  the 
groups  by  grade  level.  The  analysis  yielded  an  F value  of  12.23  with  a p = <.0001.  Analysis 
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of  data  of  student  achievement  (GPA)  by  remote  site  or  host  site  by  individual  course 
identified  no  significant  differences.  The  analysis  yielded  an  F value  of  .77  with  a g = .62  as 
reported  in  Table  5. 

Table  5 Analysis  of  Variance  for  Grade  Point  Average  on  Site  Location  and  Individual 

Course 

Source df SS MS F g 

Site  1 .262  .262  .511  .4749 

Class  8 50.194  6.274  12.231  <.0001 

Site*  Class  8 3.186  .398  .776  .6237 


When  comparing  student  GPA  by  course  we  see  differences  in  individual  courses.  All 
courses  GPA’s  were  identified  as  being  significantly  higher  than  GPA’s  in  Vocational 
Marketing.  Foreign  Languages  GPA’s  were  significantly  higher  then  GPA’s  in  Vocational 
Marketing.  Natural  Resources  GPA’s,  Chemistry  GPA’s,  and  Art  GPA’s  were  significantly 
higher  than  GPA’s  in  Foreign  Languages.  Natural  Resources  GPA’s  were  significantly  higher 
than  GPA’s  in  Ag  Business  Mgt.,  Math,  and  Animal  Science  as  reported  in  Table  6. 
Significant  differences  were  found  when  grouping  traditional  vocational  courses,  Ag  Business 
Mgt.,  Vocational  Marketing,  Natural  Resources,  and  Animal  Science,  together  and  comparing 
to  the  traditional  academic  in  student  achievement  as  measured  by  Grade  Point  Average. 
Students  in  traditional  academic  courses  (3.25  GPA)  had  a higher  GPA  than  did  students  in 
vocational  courses  (2.99  GPA).  The  analysis  yielded  an  F value  of  13.56  with  a p = .0003  as 
reported  in  Table  7. 

Table  6 Individual  Course  Success  fGPA')  by  Host  Site  and  Remote  Site 

Course  Name  Host  Site  Remote  Site 

N GPA  N GPA 


Foreign  Language 

91 

3.29 

140 

3.06 

Ag  Business  Mgt. 

3 

3.23 

74 

3.09 

Vocational  Marketing 

14 

1.70 

7 

2.00 

Natural  Resources 

4 

3.68 

38 

3.43 

Math  - Calculus 

63 

3.11 

56 

3.21 

Chemistry 

9 

3.57 

61 

3.58 

Art 

5 

4.00 

9 

4.00 

Statistics 

6 

3.50 

8 

3.50 

Animal  Science 

4 

3.43 

31 

2.88 

Table  7 Analysis  of  Variance  for  Grade  Point  Average  on  Academic  and  Vocational 

Courses 

Source df SS MS F p - 

Between  1 8.18  8.18  13.56  .0003 

Within  621  374.34  .60 


Conclusions 


1 . Students  enrolled  in  distance  education  courses  were  primarily  located  at  remote  sites. 
Remote  site  students  were  provided  the  opportunity  to  take  these  courses  because 
courses  were  being  offered  via  distance  education  technology.  Without  this 
opportunity,  most  of  these  students  (424)  would  not  have  been  able  to  enroll  in  these 
courses 

2.  Receiving  instruction  by  distance  education  resulted  in  no  differences  in  grade  point 
average  for  all  students  at  either  remote  site  or  host  site.  Students  in  9 grade  did 
have  higher  grade  point  averages  then  did  1 1 grade  and  12  students.  Twelfth  grade 

students  located  at  host  sites  had  a significantly  lower  GPA  than  12  grade  students 
located  at  remote  sites.  This  area  needs  further  analysis  to  determine  exact  reason  for 
this  occurrence. 

3 . Individual  courses  being  offered  via  distance  education  revealed  differences  in  student 
achievement.  Grouping  traditional  academic  courses  and  comparing  to  the  vocational 
courses  offerings  significant  differences  were  revealed. 

4.  The  results  of  the  study  offer  promise  that  distance  education  courses  do  provide 
opportunities  for  students  who  would  otherwise  not  be  able  to  take  these  courses. 
Additionally  student  success,  GPA,  was  above  average  (mean  GPA  whole  group  = 
3.18)  in  these  courses. 
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The  Web  was  first  envisaged  as  an  Information  Management  tool  that  helped  researcher  organise 
and  find  interesting  documents.  However  with  the  popularity  of  the  Internet  and  the  its  image  as  a 
Publication  tool  for  the  masses,  we  now  have  a Information  Mis-Management  Tool.  Increasingly 
people  are  using  it  for  searching  using  "Web  Portals".  Web  Surfers  are  morphing  into  Web 
Searchers.  XML  may  help  us  resurrecting  the  information  chaos  into  some  manageable  form.  This 
paper  will  briefly  describe  XML  and  its  potential  benefits  for  Educational  Delivery. 


1 Introduction 

The  Web  is  based  on  technology,  which  was  initially  invented  for  small  information  spaces  with  a small  number  of 
users.  HTTP,  URLs  and  HTML  are  excellent  solutions  to  a uniform  information  management  system.  With  the 
Internet  explosion  these  standards  have  been  stretched  to  their  limits. 

The  Web  has  evolved  rapidly,  so  rapidly  it  seems  more  like  a revolution  rather  then  an  evolution.  To  keep  up  with 
these  changes  the  Internet  authorities  have  had  a busy  time!  Each  of  the  original  Internet  technologies  has  been 
extended  in  numerous  ways.  HTML  has  seen  the  biggest  change,  initially  designed  for  static  text  only  hypertext 
pages,  now  incorporating  many  different  technologies  which  want  to  get  on  the  Internet  bandwagon. 

HTML  has  obvious  limitations  that  need  to  be  resolved.  These  limitations  include: 

• Unpredictable  appearance  of  HTML  documents  in  Web  browsers; 

• The  content  of  an  HTML  document  is  inseparable  from  the  presentation  tags; 

• Meta  information  in  HTML  is  restricted  to  document  wide  semantic  name/value  pairs; 

• The  majority  of  search  engines  index  only  the  textual  content  of  HTML  pages. 

This  paper  will  introduce  the  main  elements  of  Extensible  Markup  Language  (XML).  The  paper  will  outline  a case 
for  using  XML  within  Courseware  systems. 


2 Case  Study 

The  potential  of  XML  to  improve  the  delivery  of  course  material  is  now  illustrated  through  a short  case  study.  The 
course  under  consideration  is  delivered  by  the  authors  to  first  year  undergraduate  students  at  the  Business  School  of 
the  University  of  Strathclyde,  and  provides  an  introduction  to  Business  Computing. 

Many  of  the  generic  issues  relating  to  courses  supported  on  the  WWW  are  of  relevance  to  this  case  study,  including 
facilities  for  off-campus  access,  student  centred  learning,  standard  browser  access,  etc.  However,  there  are  a 
number  of  specific  issues  related  to  this  course  that  make  it  particularly  suited  to  web-based  support.  Firstly,  it  is  a 
rather  large  class,  by  UK  standards,  with  around  700  students.  There  is  a need  to  give  students,  particularly  in  the 
first  year,  some  sense  of  contact  with  teaching  staff  but  it  is  clearly  not  feasible  to  use  traditional  small  group 
meetings  due  to  the  time  and  labour  intensity  that  this  implies.  Secondly,  the  course  is  delivered  by  four  staff 


817 


members  from  two  different  departments  within  the  Faculty  (it  is  felt  that  as  this  class  is  for  all  Faculty  students  this 
is  a useftil  mechanism  and  indeed  proposals  are  in  place  to  increase  the  number  of  staff  involved).  In  this  context  it 
is  important  that  the  students  have  a ‘fbcal  point’  for  the  provision  of  course  information.  Finally,  students  entering 
this  class  come  with  a range  of  knowledge  and  skills  relating  to  the  use  of  computers  within  business.  There  is 
therefore  a need  to  tailor  material  in  such  a way  as  to  ensure  that  novices  are  not  over-awed  and  yet.  At  the  same 
time,  to  challenge  those  with  some  prior  knowledge  to  extend  and  in  particular  deepen  their  understanding  of  the 
subject. 

The  use  of  web-based  material  aids  in  addressing  a number  of  these  issues  for  example,  the  web  site  provides  a 
‘virtual’  focal  point  to  which  the  students  can  be  pointed  as  a source  of  initial  reference.  In  addition  to  finding  a 
course  overview,  copies  of  the  course  handbook  and  timetable  details,  information  can  also  be  found  for  classes  as 
they  take  place  each  week:  tutorial  material;  preparatory  reading;  useful  web  links  and  even  weekly  student 
feedback  can  be  checked  on  the  site.  When  the  ‘virtual’  solution  cannot  provide  adequate  support  the  site  points  the 
student  to  the  most  appropriate  staff  member  to  deal  with  their  query  and  encourages  e-mail  contact  in  the  first 
instance.  The  flexibility  that  the  WWW  provides  to  allow  the  four  staff  members  to  interact  with  the  students  and 
the  course  material,  as  well  as  with  each  other,  has  many  advantages  however  it  can  also  lead  to  problems. 

The  fact  that  no  one  person  has  overall  editorial  control  can  lead  to  conflicting  information  or  to  inconsistent 
presentation  over  a period  of  time.  Thus  there  is  a tension  between  the  benefits,  in  terms  of  flexibility  and 
immediacy,  which  the  WWW  offers  and  the  requirement  from  the  students  point  of  view  to  be  able  to  rely  on  the 
site  as  the  authoritative  source  of  course  information.  The  need  for  increased  control  on  the  consistency  and 
accuracy  of  information  has  been  partially  addressed  by  the  creation  of  course  ‘style  sheets’  but  this  rather  informal 
mechanism  does  not  provide  a complete  solution  and  the  potential  of  XML  to  assist  in  this  context  is  explored 
below. 

As  yet,  little  progress  has  been  made  in  addressing  the  widely  differing  levels  of  knowledge  students  have  on  entry 
to  the  course.  This  is  not  because  the  WWW  does  not  provide  suitable  mechanisms  through  which  diverse  sets  of 
material  can  be  delivered  but  rather  reflects  the  effort  involved  in  editing  and  maintaining  3-4  parallel  and  internally 
consistent  sets  of  material  for  the  varying  levels  of  student  expertise.  Once  again  it  is  the  authors’  view  that  the 
structuring  and  delivery  of  dynamically  changing  material  (i.e.  based  on  student  expertise  level)  can  be  handled 
much  more  adequately  using  an  XML-based  approach. 

3 XML 

XML  has  generated  a lot  of  attention  in  the  technical  press  recently  where  it  has  been  widely  referred  to  as  the 
replacement  to  HTML.  The  two  languages  are,  of  course,  related  and  share  a common  parent  in  the  Standard 
General  Markup  Language  (SGML,  ISO-8879).  SGML  is  a complex  standard  in  the  form  of  a meta-language  that 
allows  us  to  describe  how  a document  is  structured  by  means  of  a Document  Type  Definition  (DTD).  A DTD 
specifies  which  tags  a document  may  have,  what  they  are  composed  of  and  how  they  are  related  to  one  another  in 
the  structure  of  the  document  in  terms  of  sequence,  nesting  etc.  HTML,  for  instance,  is  an  SGML  DTD  defining 
documents  for  presentation  by  a web  browser  that  is  ‘hard- wired’  to  understand  the  HTML  DTD.  Any  users  of  the 
web  over  the  last  few  years  will  have  observed  how  HTML  has  been  continually  updated  and  expanded  to  increase 
its  flexibility  of  presentation,  its  ability  to  integrate  live  data  and  its  functionality  for  supporting  interaction. 
However,  these  extensions,  as  well  as  being  proprietary  and  non-standard,  have  been  unable  to  tackle  HTML’s 
inability  to  support  document  extension  or  inheritance  and  its  inclusion  of  document  styling  alongside  structure. 

Awareness  of  the  limitations  of  HTML  and  the  desire  to  avoid  the  proliferation  of  proprietary  HTML  extensions  led 
to  the  ratification  of  XML  by  the  World  Wide  Web  Consortium  (W3C)  in  February  of  1998.  In  terms  of  complexity, 
XML  lies  somewhere  between  SGML  and  HTML  and  is  aimed  at  providing  most  of  the  richness  of  the  SGML 
command  set  while  remaining  easy  to  learn,  implement  and  use.  It  has  been  described  as  a dialect  of  SGML  and  is 
itself  a meta-language.  Thus,  XML  allows  us  to  specify  not  only  a marked  up  document  but  also  to  specify  which 
DTD  it  conforms  to.  For  example,  in  the  case  of  web  documents,  there  is  always  an  implicit  association  with  the 
HTML  DTD  while  with  an  XML  document  it  is  usually  necessary  to  refer  to  the  associated  DTD  explicitly  in  the 
header  of  the  document. 


o 
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This  last  point  highlights  what  is  probably  the  most  important  departure  from  HTML  - that  XML  specifies  the 
structure  of  a document  via  its  accompanying  DTD.  Since  the  document  author  can  define  this  DTD,  any  customised 
tags  may  be  defined  and  used  within  the  document.  The  XML  markup  language  is  robust  enough  to  describe  a wide 
range  of  abstract  structures  in  the  form  of  a document  and  an  accompanying  DTD  describing  that  abstract  structure. 
XML  can  therefore  be  used  to  describe  data  objects,  structured  records,  and  many  other  types  of  structured  data.  The 
industry  has  been  quick  to  see  the  potential  of  this  new  standard  for  exchanging  and  sharing  not  only  documents  as 
is  currently  done  on  the  Internet  and  various  intranets,  but  also  to  share  a huge  variety  of  structured  data  using 
established  Internet  standards  such  as  HTTP  as  the  mediating  protocols. 

XML  appears  to  offer  a potentially  industry-wide,  customisable  format  with  the  robustness  and  flexibility  to  model  a 
huge  range  of  document  types.  It  seems  safe  to  suppose  that  support  for  XML  in  the  form  of  toolsets  and  application 
suites  will  be  forthcoming  from  an  industry  increasingly  accustomed  to  and  demanding  of  interoperability  between 
vendors’  products.  A long  list  of  "Who’s  \\hio"  of  software  companies-including  Microsoft,  Netscape,  Lotus,  IBM, 
and  Adobe  Systems-have  publicly  committed  to  supporting  the  XML  language.  ‘ 

The  three  most  important  parts  of  XML  are  the  Document  Type  Definition  (DTD);  the  Extensible  Style  Language 
(XSL);  and  the  Extensible  Link  Language  (XLL)  [Bradly  1998]. 

DTD  - The  Document  Type  Definition  specifies  the  logical  structure  of  a document.  It  defines  the  grammar  of  a 
document  and  enables  an  XML  parser  to  validate  a page's  use  of  its  tags.  The  DTD  defines  the  permitted  use  of 
each  tag  and  it's  associated  attributes.  The  DTD  consists  of  three  structures: 

• A description  of  the  nesting  of  hierarchies  for  document  structures  (i.e.  A Chapter  contains  Headings  and 
sections,  sections  contain  paragraphs  and  so  on.) 

• The  sequence  in  which  the  structures  are  permitted  (i.e.  A Title  is  before  an  Abstract  and  so  on) 

• Finally  the  DTD  describes  if  structures  are  optional  and  also  repeatable,  (i.e.  A Paragraph  must  have  at 
least  one  or  more  sentences). 

XSL  - Extensible  Style  Language  is  the  language  used  to  specify  style  sheets  for  XML  documents.  XSL  enables 
Web  browsers  to  change  the  presentation  of  a document.  This  is  similar  to  the  Cascading  Style  Sheets  (CSS)  used  in 
HTML  4.0. 

XLL  - Extensible  Link  Language  supports  today’s  Web  links  and  it  also  extends  links.  It  will  include  indirect  links, 
which  should  resolve  dead  links  and  other  hyper  link  problems.  XLL  is  currently  under  development.  XLL  will 
support  a number  of  link  mechanisms  [WWW3  XLL  98]. 


4 Internet  versioning  with  HTML 

For  a corporate  Internet  to  succeed  the  content  needs  to  be  up-to-date.  Most  companies  have  data  and  information 
stored  in  vendor  specific  file  formats,  such  as  Microsoft  Word,  Corel  Draw,  etc.  For  these  file  formats  to  be  stored 
on  the  corporate  Internet  requires  exporting  to  HTML  and  then  FTP  or  "Published"  to  a Web  server. 

HTML  is  a poor  presentation  language  with  little  or  no  structural  information.  Sadly  most  HTML  file  exporting 
tools  remove  important  semantic  information,  stored  in  vendor  specific  file  formats.  Vendors  are  producing  new 
software  tools  that  can  export  to  HTML  but  these  are  merely  add-on  features.  For  an  Internet  to  succeed  vendors 
must  create  software  that  use  a Web  file  format  as  the  primary  file  format.  In  time  software  vendors  will  become 
Web  centric,  until  then  we  have  a versioning  problem. 

IMAGE  NOT  AVAILABLE 

Figure  1.1  illustrates  the  current  situation  when  an  employee  saves  a Microsoft  Word  document  on  a Corporate 
Internet.  This  two  phased  saving  & publishing  to  the  Web  means  that  there  are  always  two  versions  of  the  document 
which  can  lead  to  versioning  control  problems.  HTML  files  are  secondary  file  stores,  much  like  a printed  document, 


For  more  information  on  XML  have  a look  at  http://www.xml.com/ 
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if  the  original  document  is  updated  then  the  HTML  document  must  also  be  changed.  For  an  Internet  to  succeed  then 
the  primary  file  store,  on  the  Web,  must  be  stored  for  company  access. 


5 XML  Benefits 

We  propose  storing  vendor  specific  file  format  in  XML,  The  software  vendor  can  add  all  the  extra  "proprietary" 
information  within  the  file  format  using  XML,  Then  this  XML  document  can  be  downloaded  and  interpreted  by  any 
XML  compliant  Web  viewer.  Thus  we  have  a vendor  neutral  viewing  facility,  allowing  users  to  have  thin-clients 
with  few  installed  programs. 

Figure  1 .2  shows  that  vendors  can  publish  the  DTD  for  a document  and  allow  many  users  to  view  the  information  in 
its  full  glory.  The  DTD  specifies  the  complete  semantic  of  the  document.  In  its  purest  form,  XML  will  allow 
software  vendors  to  create  complex  document  viewers. 

Another  advantage  of  XML  is  that  the  appearance  of  the  document  can  change  depending  on  the  media,  allowing  the 
document  creator  to  harness  all  features  of  each  medium.  An  author  must  customise  a document  for  a single  media 
and  hope  that  it  looks  well  in  different  media.  Potentially,  a single  XML  based  document  can  take  advantage  of  all 
the  features  of  a specific  printer  and  all  the  benefits  of  a Web  browser.  Also  XML  documents  can  be  customised  to  a 
number  of  users,  a manager  may  view  an  executive  summary  of  a project  report  while  the  project  team  views  the 
complete  report. 

Research  by  [Rath  98]  has  highlighted  that  XML  can  assist  in  the  Management  of  Content,  where  the  document 
content  can  be  maintained  by  editorial  support  systems.  Also  Metadata  can  be  incorporated  into  the  XML  documents 
so  that  the  documents  can  be  easily  indexed  and  searched.  Information  stored  in  a database  could  also  be  used  by 
XML  documents  by  enhancing  the  tag  definitions  for  Database  elements  such  as  table  names  and  fields. 

Data  delivered  in  XML  to  a browser  can  "give  Java  something  to  do".  This  would  mean  XML  could  convey  data 
and  data  types  to  a browser  in  a standardised  form.  In  the  past  migrating  data  between  systems  has  either  been 
impossible,  because  proprietary  software  actually  prevented  batch  upload  and  download,  or  it  has  been  a tedious 
process  of  reducing  complex  data  formats  to  flat  structures  like  comma-separated  variables.  In  this  process  there  are 
ample  opportunities  for  sophistication  in  the  data  structure  to  be  lost,  XML  promises  to  be  the  lingua  franca  for  data 
migration 


5.1  Standardised  DTDs  for  XML 

For  XML  to  succeed  the  domain  specific  DTDs  must  be  globally  accepted  within  a particular  domain.  There  is  little 
point  developing  numerous  DTDs  that  describe  Educational  Course  Material  as  we  could  then  have  incompatibility 
issues  between  the  different  Educational  DTDs,  For  XML  to  succeed  then  there  must  be  some  central  standards 
organisation  that  controls  the  evolution  of  one  single  DTD  for  a particular  Domain  such  as  Educational  DTDs.  In 
other  Domains  XML  DTDs  are  under  development,  these  include: 

• Mathematics  Markup  Language  {MML  used  for  displaying  Math  symbols  and  equations  etc  ) 

• Chemistry  Markup  Language  {CML  to  describe  molecules  and  their  structure  etc) 

• Electronic  Document  Interchange  (XML-EDI) 

• Channel  Definition  Format  (CDF  used  for  some  'Push’  technologies) 

• Synchronised  Multimedia  Integration  Language  {SMIL) 

• XML  Metadata  Interchange  {XMI  used  for  exchanging  software  models) 

We  propose  developing  a standard  DTD  for  courseware  systems.^ 

5.2  Developing  a Courseware  DTD  using  Visual  Rhetoric  and  existing  Learning  Models 

We  are  currently  developing  a generic  Courseware  XML  DTD  that  will  allow  us  to  create  re-usable  Courseware 
systems.  To  do  this  we  are  using  a Graphical  User  Interface  model  called  Visual  Rhetoric  to  define  how  course 


^ We  are  interested  in  developing  a Standard  XML  DTD  that  can  be  widely  used.  If  your  Organisations  is  interested  in  helping 
develop  this  Courseware  DTD  please  contact  us. 


content  should  be  presented.  We  are  also  hoping  to  improve  the  quality  of  the  information  delivery  by  using 
existing  learning  methods  to  develop  XML  DTD  tags. 

A document  can  be  interpreted  as  a visible  representation  of  a text  according  to  its  semantic  contents  [Southall  89]. 
The  use  of  different  fonts  and  typographical  styles,  as  well  as  the  introduction  of  spacing  and  pagination  rules  helps 
to  draw  attention  to  selected  parts  of  the  text  [Southall  92].  Thus  the  translation  into  graphical  terms  of  the  text 
rhetoric  which  results  from  both  the  logical  structure  of  the  text  and  its  pragmatic  component  can  be  called  visual 
rhetoric  [Landoni  97].  The  idea  is  to  define  those  parts  that  are  more  important  for  the  comprehension  of  the 
meaning  of  the  text. 

Visual  rhetoric  provides  the  reader  with  a graphical  mark  up  language  that  is  immediately  recognisable  on  the  basis 
of  previous  reading  activity.  Different  graphical  presentations  suggest  different  readings  and  affect  deeply  the 
interpretation  of  the  contents  of  the  same  text.  These  observations  lead  one  to  conclude  that  visual  rhetoric  is  a 
crucial  aspect  for  both  reading  and  browsing  a document.  We  will  be  adopting  this  approach  visual  rhetoric  when 
developing  the  XSL  and  XLL  that  will  define  the  presentational  aspects  of  the  DTD.  It  is  hoped  that  visual  rhetoric 
will  give  us  the  possibility  to  control/predict  the  way  student  interprets  and  interacts  with  the  Courseware 
documents. 

One  of  our  main  goals  is  to  improve  we  improve  the  effectiveness  of  a Courseware  system.  A DTD  that  uses 
'semantic  elements  to  exploit  knowledge  inside  the  information'  can  help  increase  the  quality  of  the  data  and  the 
service  of  online  information  [Rath  98].  At  present  we  are  identifying  teaching  models  that  encapsulate  most 
learning  methods  and  teaching  techniques.  We  will  then  create  XML  tag  definitions  that  correspond  to  each  element 
of  these  teaching  models.  It  is  a relatively  straightforward  task  to  translate  each  of  the  teaching  elements  into  DTD 
tag  definitions.  However  developing  a DTD  from  scratch  is  time  consuming,  and  requires  a great  deal  of  testing 
[Maler  96]. 


6 Commercial  acceptance  of  XML 

Many  software  vendors  are  investigating  XML  as  the  future  file  format  for  universal  viewing,  editing  and  saving. 
"XML  can  potentially  revolutionize  the  Web,"  said  Chris  Lilley,  from  the  World  Wide  Web  Consortium  (W3C). 
"It's  not  a magic  bullet  ...  but  as  a cornerstone  of  an  open  network  computing  environment  it's  very  important." 
Microsoft's  plans  a new  HTML-XML  hybrid  file  format  and  both  Lotus  and  Corel  plan  to  support  XML  in  their  next 
releases.  It  could  eventually  supply  the  basis  for  universal  file  formats  usable  by  all  suites  [PC  World  98]. 

Berners-Lee  [Montgomery  98]  is  optimistic  about  the  use  of  XML  markup  tags  outside  the  Web.  This  will  overcome 
the  problem  of  backward  incompatibility  between  document  formats.  The  W3C  is  talking  to  Microsoft  Corporation 
about  using  XML  as  a cross-platform  file  format  for  Word.  Berners-Lee  suggests  that  HTML  will  remain  reasonably 
static  at  Version  4.0,  as  XML  allow  vendor  to  extend  and  customise  tags  for  specific  purposes. 


7 Future  Work 

At  present  there  are  very  few  software  tools  that  support  XML.  To  us  XML  properly  within  a educational 
environment  we  need  software  that  can  support  XML:  author  a course;  publish  course  material;  capture  and  manage 
ongoing  student/instructor  interaction;  store  course  material  and  allow  easy  retrieval  methods  for  all  types  of  users; 
and  a user  interface/browsers  to  view  interact  with  the  material.  To  date  this  XML  support  is  limited  however  most 
software  vendors  have  identified  XML  as  a feature  which  will  be  added  to  forthcoming  products.  So  to  conclude 
when  it  comes  to  XML:  "Watch  this  (Name)Space!". 

References 

[Landoni,  M.  97]  The  Visual  Book  System:  A Study  of  the  Use  of  Visual  Rhetoric  in  the  Design  of 

Electronic  Books,  Glasgow:  Department  of  Information  Science  of  the  University  of  Strathclyde  (PhD  Thesis). 
[Southall,  R.  89]  Interfaces  between  the  Designer  and  the  Document,  in,  Andre,  J.,  Furuta,  R.  and  Quint, 

V.  {tds)Structured  documents.  Cambridge:  Cambridge  University  Press,  pp.  119-131. 


821 


[Southall,  R.  92]  Presentation  Rules  and  Rules  of  Composition  in  the  Formatting  of  Complex  Text,  in, 

Vanoirbeek,  C.  and  Coray,  G.  (eds.)  Electronic  Publishing  '92  - International  Conference  on  Electronic  Publishing, 
Document  Manipulation,  and  Typography,  Lausanne,  Switzerland  . Cambridge  University  Press,  pp.  275-290. 
[Bradly  1998]  The  XML  Companion,  N.  Bradley,  Addison  Wesley,  ISBN  0-201-342855. 

[PC  World  98]  File  Format  Frustrations:  New  Clouds  on  the  Horizon,  PC  World  July  1998, 

http://www.pcworld.eom/current_issue/article/0, 1 2 1 2,3929+7+0,00. html 

[Maler96].  Developing  SGML  DTDs,  Maler  and  Andaloussi,  1996,  Prenctine  Hall  ISBN  0-13- 

309881-8. 

[Montgomery  98]  Berners-Lee  Upbeat  on  XML,  Paul  Montgomery,  PC  Week  Australia,  04.15.98, 

http;//www.zdnet.com/pcweek/news/041 3/1 5ewww.html 

[XML-spec  98]  XML  Complete  Specification,  World  Wide  Web  Consortium, 

http://www.w3.org/pub/WWW/TR/WD-xml.html 

[WWW3  XLL  98]  XML  Linking  Language  (XLink),  World  Wide  Web  Consortium  Working  Draft  3-March- 

1998,  http://www.w3.org/TR/1998/WD-xlink-19980303 

[Rath  1998]  XML:  chance  and  challenge  for  online  information  providers,  Hans  Holger  Rath, 

Proceeding  of  the  Online  Conference  1998,  London. 

[Sonnenberg  96]  Design  of  a Reusable  IR  Framework,  Gabriele  Sonnenberg,  Hans-Peter  Frei, 

http://www.ubs.com 


822 


Totally  Integrated  Internet  Courses 


Richard  Dwight  Laws 
Independent  Study 
Brigham  Young  University 
United  States 
dwight_laws@byu.edu 


Abstract:  The  notion  of  a totally  integrated  course  is  ambitious,  particularly  when  that 
integration  goes  beyond  course  delivery  to  include  grading,  data  entry,  testing,  feedback,  and 
management  of  all  aspects  of  the  process.  Imagine,  if  you  will,  a totally  integrated  Internet 
course  (TIIC)  in  which  a student  can  find  the  appropriate  course,  investigate  it,  register  for  it, 
receive  all  necessary  materials,  submit  assignments,  get  grades  and  feedback,  take  the  final 
exam,  get  a course  grade,  and  receive  a transcript  in  the  mail  without  any  human  intervention 
(or  for  that  matter  without  the  university  even  knowing  that  the  student  had  been  there).  This 
paper  briefly  explains  how  totally  integrated  Internet  courses  are  delivered  and  managed  at  the 
Brigham  Young  University,  Continuing  Education,  Department  of  Independent  Study. 


Introduction 

Brigham  Young  University  (BYU)  is  moving  boldly  into  Internet  courses.  BYU  has  one  of  the  largest 
home  study  programs  in  the  nation,  with  over  500  courses  and  42,000  enrollments.  The  Department  of 
Independent  Study  manages  these  courses  and  plans  to  move  many  of  them  onto  the  Internet.  Currently,  BYU 
Independent  Study  offers  over  70  totally  integrated  Internet  courses  (TIIC).  There  are  problems  associated  with 
TIIC:  they  are  not  synchronous,  and  they  are  not  social.  However,  TIIC  have  major  strengths,  which  will  be  the 
subject  of  this  paper. 

Technology  has  given  faculty  new  tools.  However,  those  tools  can  quickly  become  burdensome  if  not 
properly  used  or  designed.  For  example,  the  most  prevalent  form  of  distance  courses  that  faculty  have 
attempted  is  the  use  of  e-mail  for  interaction  with  students  in  an  attempt  to  be  synchronous.  However,  faculty 
have  been  overwhelmed  with  demands  on  their  time  using  this  method  and  have,  for  the  most  part,  been  forced 
to  discontinue  using  the  tool  for  that  express  purpose.  Those  who  continue  to  use  e-mail  have  had  to  restrict 
enrollment  to  a manageable  few.  Michael  P.  Lambert  (1999)  says  that  “the  brand  of  distance  education  that 
traditional  educators  are  talking  about  for  the  most  part  is  “synchronous”  learning. . . .the  concept  of  ‘anytime’ 
learning  is  not  yet  ascendant.” 

Another  example  is  that  faculty  find  their  time  almost  consumed  when  they  try  to  do  their  own  course 
development  using  technology.  They  often  do  it  well  and  like  what  they  have  created,  but  it  just  takes  too  much 
time  to  create,  manage,  update,  and  trouble-shoot.  The  problem  is  of  nationwide  significance  and  needs  to  be 
addressed  in  order  for  technology  to  fully  benefit  faculty. 

TIIC  address  the  problem  of  “anywhere,  anytime”  learning  quite  well.  TIIC  accommodate  unlimited 
enrollment.  They  also  reduce  faculty  burden  and  at  the  same  time  generate  large  amounts  of  faculty  feedback 
asynchronously. 


The  Model 

The  system  that  drives  BYU  Independent  Study’s  TIIC  has  the  following  components: 

• Web  course  catalog 

• automatic  registration  and  credit  card  clearance 

• delivery  of  course  instruction 

• assignment  submission 

• computer  grading  of  all  submissions 

• faculty-designed  computer  feedback  for  all  missed  questions 

• testing 

• grade  posting 
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• personalized  Web  report  card  accessibility  for  students 

• and  transcript  distribution. 

Faculty  are  paid  up  front  for  the  development  of  the  course  and  then  receive  a royalty  per  enrollment 
on  the  back  end.  They  do  not  test  or  grade.  Faculty  involvement  and  time,  after  the  course  has  been  developed, 
is  virtually  zero.  Therefore,  there  is  no  faculty  burden  nor  enrollment  limit. 

As  assignments  come  in  and  are  graded,  the  data  is  stored  for  item  analysis  (ITEM AN).  With  a 
sufficient  number  of  the  same  assignment  from  multiple  students,  ITEMAN  then  analyzes  the  data  and 
determines  which  questions  are  weak.  These  questions  are  discarded  and  the  faculty  create  new  ones.  By  this 
process  the  courses  and  assessment  become  stronger  through  use. 

When  a student  comes  to  BYU  Independent  Study’s  Web  page  (http://coned.bvu.edu/isA  they  are 
given  the  options  of  exploring  all  aspects  of  the  TIIC.  They  can  read  through  the  course  catalog,  which  includes 
all  500  courses.  They  can  find  information  about  diplomas,  certificates,  and  how  to  succeed,  and  they  can  view 
frequently  asked  questions.  They  can  learn  about  other  free  online  services,  such  as  grade  checking,  exam 
requests,  and  questions  and  interaction  with  the  faculty  for  special  circumstances.  Finally,  they  can  explore  the 
listing  of  college,  high  school,  and  personal  enrichment  courses  available  to  them — both  paper  and  pencil  and 
Internet  courses. 

When  a student  picks  a course  to  inspect,  they  are  taken  to  a screen  that  gives  them  additional 
information  about  the  course,  including  an  audio  introduction  to  the  faculty,  instructions  about  navigating  the 
course,  course  features,  frequently  asked  questions,  and  course  specific  how-to-succeed  instructions.  They  are 
given  a sample  test  to  learn  how  testing  works  on  the  Internet.  They  are  shown  a table  of  contents  so  they  can 
know  if  the  course  teaches  what  they  need.  Finally,  they  are  given  the  opportunity  to  enroll.  If  the  student 
enrolls  using  a credit  card,  a password  and  a CD-ROM  are  immediately  mailed  to  them.  The  CD  is  used  in  a 
seamless  presentation  of  the  video  and  audio  part  of  the  TIIC  (this  will  be  modified  when  streaming  is 
sufficiently  robust  and  dependable).  The  student  is  then  able  to  open  the  main  part  of  the  course  and  proceed 
with  the  instruction  and  assignments. 

There  is  also  a powerful  tool,  located  on  the  home  page,  that  allows  faculty  access  to  a training 
segment  which  teaches  them  how  to  write  Internet  courses.  This  training  guide  was  developed  by  professional 
instructional  designers  to  help  faculty  move  through  the  standard  design  steps  for  instructional  delivery  at  a 
distance.  (Incidentally,  some  on-campus  faculty  have  discovered  this  faculty  guide  tool  and  are  using  it  as  a 
guide  for  developing  the  practicum  for  their  in-class  presentation.) 

When  the  student  has  enrolled  in  the  course  and  has  received  their  password,  they  can  open  the  course 
and  begin  studying.  When  they  are  ready  to  submit  an  assignment,  they  complete  the  questions  and  click  on  the 
Submit  button.  They  immediately  see  a report  of  their  grade,  followed  by  a list  of  ail  questions  they  missed. 
These  missed  questions  are  then  explained  by  feedback  from  the  faculty-designed  databank.  This  aspect  of 
TIIC  is  the  most  popular  element  with  the  students.  They  love  to  have  immediate,  and  complete,  feedback  for 
their  work — something  they  don’t  often  get  even  in  a classroom.  Many  students  who  have  enrolled  in  a TIIC 
have  given  the  reason  for  the  enrollment  as  being  the  feedback  statements  that  they  receive.  The  assignment 
grade  is  immediately  transferred  to  the  student’s  data  bank  and  entered  in  their  record.  The  student  doesn’t 
have  to  wait  for  answers  through  the  mail  and  is  therefore  able  to  continue  on  with  the  next  lesson  if  they 
choose.  Immediate  feedback  also  leads  to  higher  completion  rates  among  distance  education  students. 

After  all  assignments  have  been  graded  and  submitted,  the  student  must  find  a proctor.  The  TIIC 
system  will  either  forward  the  test  to  the  proctor  to  be  administered,  or  will  send  the  proctor  a code  word  which 
will  open  the  test  on  the  Internet.  In  the  case  of  the  Internet  version,  the  test  is  graded  online  and  entered 
immediately  into  the  student  record.  The  password  for  the  test  is  then  no  longer  valid.  The  TIIC  system 
calculates  the  student's  assignment  grades,  test  grades,  and  overall  course  grade,  and  then  sends  a transcript  by 
mail  to  the  student. 

None  of  the  aforementioned  steps  require  faculty  input  or  interaction;  therefore,  the  course  does  not 
add  to  faculty  load  and  can  accommodate  unlimited  enrollment. 
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StrengthsAVeaknesses 


Some  features  of  face-to-face  learning  (synchronous)  are  not  currently  available  with  technology.  The 
attitude  of  many  faculty  is  that  this  weakness  makes  distance  delivery  of  courses  unacceptable.  However,  face- 
to-face  instruction,  however  strong  it  may  be,  is  not  necessarily  a reality  even  in  the  classroom.  Consider  large 
classes  where  individuals  never  have  access  to  the  teacher.  Also  consider  shy  or  less-aggressive  students  who 
seldom  get  interaction  even  in  small  classes.  While  lack  of  synchronous  interaction  may  be  perceived  as  a 
weakness,  research  has  for  years  shown  that  distance  education  is  not  significantly  less  effective  than  on- 
campus  classroom  education.  Thomas  L.  Russell  compiled  248  research  reports,  summaries,  and  papers  dating 
from  1928  to  1996  into  a meta-analysis  which  he  called  the  “No  Significant  Difference”  phenomenon 
(http://teleeducation.nb.ca/nosignificantdifference/).  Michael  P.  Lambert,  executive  director  of  Distance 
Education  and  Training  Council,  in  Washington,  D.C.  (1999)  said,  “It  is  clear  that  distance  education  is  here  to 
stay.  . . . Within  the  next  three  years,  I predict  that  3 out  of  4 institutions  of  any  significant  size  will  be  offering 
distance  education  courses.”  However,  Lambert  is  well  aware  that  “the  faculty  is  the  single  biggest  asset  of  any 
university  and  is  at  the  same  time  the  single  biggest  stumbling  block  to  the  rapid  adoption  of  distance  learning 
techniques  on  a massive  scale.”  Lambert  says  that,  “some  faculties  fear  distance  learning,  seeing  it  as  a threat  to 
their  jobs,  to  academic  freedom  and  to  sound  learning.”  But  Lambert  predicts  that  “over  time,  most  faculty  will 
learn  to  love  distance  learning  and  come  to  see  it  as  a personally  liberating  and  empowering  force,  where  they 
will  become  tutors  and  facilitators.” 

The  TIIC  system  was  designed  to  accommodate  learning  “anywhere,  anytime.”  Its  weaknesses  are  the 
weaknesses  of  current  technology.  Its  strengths  are  the  strengths  of  current  technology.  Included  in  those 
strengths  is  the  ability  to  fashion  excellent,  educationally  strong  feedback  statements  to  continue  the  learning 
process  when  students  submit  assignments.  This  feedback  occurs  for  every  student,  even  if  there  are  1,000 
submissions.  Such  feedback  and  attention  would  be  impossible  for  a single  faculty  member,  or  even  a team  of 
graders. 

TIIC  allows  students  their  own  place,  their  own  space,  and  their  own  pace.  TIIC  is  not  subjective  nor 
bias  in  its  grading.  Currently,  assignments,  quizzes,  and  tests  are  well-designed  multiple-choice  and  other 
objective  forms.  It  is  anticipated  that  essay  will  soon  be  added  to  the  system.  It  is  also  anticipated  that 
computer  adaptive  testing  (CAT)  will  soon  become  a part  of  the  BYU  TIIC  system. 


Management 

While  TIIC  has  strong  educational  and  learning  features  it  also  has  some  distribution  and  management 
benefits.  The  system  has  reduced  mail  handling  by  400,000  pieces  per  year  even  as  enrollment  has  increased. 
Totally  integrated  Internet  courses  are  easy  to  update  and  keep  current  (unlike  printed  inventory  that  must  be 
used  before  reprinting).  TIIC  are  easily  formatted  and  changed  to  be  attractive  and  easily  navigated.  The  cost 
of  a TIIC  is  only  slightly  more  ($3,000  average)  than  developing  traditional  home  study  paper  and  pencil 
courses.  However,  TIIC  pay  themselves  back  as  much  as  three  times  faster.  The  following  graph  shows  the 
cost  comparisons  between  three  types  of  courses:  traditional  paper  and  pencil,  computer-graded  and  managed 
paper  and  pencil  (called  Speedback),  and  Internet  (TIIC). 


825 


The  following  graph  shows  the  comparison  of  time  a student  must  wait  to  get  feedback  from  an 
assignment  submission  or  test  with  traditional,  Speedback,  and  TIIC  courses. 
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Finally,  the  profit  comparisons  of  using  the  TIIC  as  compared  to  traditional  or  computer-managed 
courses  is  presented  below.  From  a management  or  operational  point  of  view,  the  difference  is  absolutely 
compelling.  Because  of  the  potential  for  education  for  profit,  we  see  many  corporations  rushing  to  the  totally 
integrated  Internet  courses  market. 


Conclusions 

Distance  educational  is  here  to  stay.  Internet  courses  are  here  to  stay.  Totally  integrated  Internet 
courses  can  provide  quality  education,  handle  unlimited  enrollments,  and  be  profitable.  The  TIIC  developed  at 
BYU  have  had  strong  enrollments  and  excellent  evaluative  feedback  from  students.  Although  the  TIIC  has 
distracters,  some  of  whom  are  very  determined,  the  model  has  attributes  and  features  that  are  very  compelling. 
However,  all  technology-based  instruction,  such  as  TIIC,  must  maintain  quality  education  as  the  foundation 
principal.  At  BYU,  TIIC  are  built  upon  a foundation  of  instructional  design  principles,  not  media  design 
principles. 
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Abstract:  This  paper  describes  an  activity  developed  at  the  Department  of  Electronic 
(Dipartimento  di  Elettronica)  - Politecnico  of  Torino  in  the  field  of  computer  based  training 
in  electronic  instrumentation  and  measurements,  where  the  author  was  a supervisor  for 
educational  process.  Several  modules  have  been  developed,  using  multimedia  technologies, 
to  assist  the  students  to  acquire  the  fundamentals  of  the  basic  instrumentation.  A client-server 
system  has  been  designed  in  order  to  allow  the  students  to  operate  on  a remote  laboratory  for 
experimental  training.  The  courseware  includes  lessons,  exercises,  and  a training  on  virtual 
instruments  which  emulate  actual  instruments.  The  students  can  also  carry  out  several  real 
laboratory  experiments  without  actually  being  in  the  laboratory,  by  using  a client-server 
structure  based  on  the  Internet.  The  students  have  shown  a favourable  acceptance  and  have 
satisfactorily  used  the  new  tools.  Demos  of  both  the  theoretical  and  practical  part  of  the 
courses  are  available  through  the  Internet. 

Introduction 

Education  in  fields  such  as  electronic  measurement  requires  students  to  gain  a reasonable  skill  in  using  various 
kinds  of  instrumentation  (Pisani  et  al.  1995).  Such  a skill  cannot  be  achieved  by  theoretical  lessons  only,  an 
intensive  laboratory  activity  is  also  always  required.  This  problem  exists  both  for  the  first  level  and  for 
qualifying  courses  (Sala  1999). 

Basic  instrumentation  teaching  is  required  for  first  level  courses  that  are  followed  by  a large  number  of  students. 
The  cost  of  basic  level  instruments  is  often  low,  but  large  classrooms  means  large  workbench  availability  and  a 
massive  and  qualified  assistance  that  is  not  easily  found. 

Qualifying  courses  are  followed  by  few  students  so  that  the  assistance  problem  is  reduced,  but  the  instrument 
cost  in  such  a case  is  often  rather  high,  thus  preventing  the  possibility  of  arranging  more  than  a few 
workbenches. 

Some  years  ago  the  Department  of  Electronics,  of  Politecnico  of  Torino,  was  involved  in  activating  traditional 
degree  and  diploma  courses  in  several  educational  structures,  throughout  the  Piedmont  region.  At  that  time 
requirements  stemming  from  a large  demand  for  education  on  experimental  subjects  and  laboratory  training  had 
to  be  met  with  limited  human  and  financial  resources. 

While  trying  to  find  a solution,  it  was  realised  that  most  of  the  time  the  students  spent  in  the  laboratories, 
especially  for  first  level  courses,  was  devoted  to  learning  the  operating  functions  and  the  use  of  the  same  basic 
instruments.  It  was  therefore  decided  to  invest  time  and  resources  to  develop  an  alternative  solution  to  the 
laboratory  replication  by  using  the  new  technologies  offered  by  computer  based  multimedia  courseware. 

The  goal  was  to  allow  the  students  to  carry  out  a pre-training  activity  outside  the  laboratory  and  possibly  at 
home;  each  student  could  thus  individually  adequate  the  learning  rate  to  his  own  capabilities.  After  this  pre- 
training phase,  students  who  enter  the  laboratory  require  reduced  assistance  and  less  time  to  complete  the 
training  activity. 

About  six  years  ago  several  tutorial  modules  were  developed  that  were  organized  like  lectures,  each  being  on  a 
particular  subject.  Each  module  contains  theoretical  and  functional  concepts  of  a basic  electronic  instrument  and 
allows  for  a simulated  instrument  exercising  by  providing  a series  of  instrument  simulators.  By  means  of  the 
realistic  control  panel  of  the  virtual  instrument  the  students  practice  under  several  operating  conditions  and 
situations.  The  pre-training  problem  is  now  becoming  more  important  since  the  Department  is  getting  involved 
in  an  education  experiment  named  “Nettuno”,  in  which  the  lectures  are  broadcast  on  TV  or  distributed  on 
videocassettes,  but  students  are  still  required  to  attend  laboratories  to  achieve  practical  experience  under  the 
guidance  of  an  instructor. 

New  modules  were  developed  to  tackle  both  the  demand  of  laboratories  distributed  throughout  the  territory  and 
requirements  of  the  distance  learning  courses. 
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In  addition,  it  was  decided  to  explore  the  possibility  of  making  a real  instrument  laboratory  available  for  use  at 
home.  This  work  is  devoted  to  reorganise  our  traditional  educational  approach,  based  on  lectures  and  successive 
laboratory  practice,  using  multimedia  tools  which  combine  two  different  learning  phases: 

• theoretical 

• practice 

in  a single  tutorial  stage  self-managed  by  the  student. 

Considerations 

The  actual  curriculum  of  the  engineering  school  include  courses  where  the  first  practice  electronic  laboratory 
came  before  the  first  lecturers  in  which  the  students  learn  the  theoretical  and  functional  aspects  of  the 
instrumentation  which  deeper  knowledge  comes  in  a successive  phase  respect  to  the  actual  student  requirements 
(Brofferio  1998). 

It  seemed  to  be  necessary  to  provide  the  first  course  students  by  multimedia  tools  which  allow  them  to  use  basic 
instrumentation  in  simple  physics  and  first  electronic  circuit  laboratories. 

If  these  tools  are  organized  with  different  study  in  depth  it  is  possible  to  satisfy  different  needs  of  different  kind 
of  students  (undergraduate  school  level  or  university  education).  On  the  basis  of  the  previous  consideration  we 
have  designed  several  didactic  modules  with  the  following  purposes: 

• to  expose  students  to  a comprehensive  range  of  electronic  instruments  and  basic  measurements  techniques; 

• to  allow  students  to  give  practice  on  the  particular  instrument  whose  front  panel  is  simulated  on  the  monitor 
and  the  behaviour  is  emulated  by  computer. 

The  learning  phase  can  be  managed  by  students  (e.g.,  as  homework)  without  involving  the  laboratory  structures 
(Pisani  et  al.  1995;  Sala  1999). 

Course  description 

This  multimedia  course  is  structured  on  a set  of  lessons  (modules)  each  containing  a particular  topic  or 
instrument  in  form  of  a self  - contained  multimedia  presentation. 

Users  may  reference  to  selected  parts  of  other  lessons  through  hyperlinks,  inserted  in  the  text  as  hot  words, 
mouse  sensitive  area  and  so  on. 

Differently  from  the  other  multimedia  applications  (e.g.,  traditional  CD-ROM)  our  case  is  oriented  to  defined 
objectives  of  knowledge  and  it  is  based  on  a student  model  with  a defined  knowledge  background,  so  it  was 
decided  to  guide  the  students  through  a path,  based  on  a structured  didactic  methodology,  to  reach  the  proposed 
educational  goal  (Bloom’s  taxonomy)  (Pisani  et  al.  1995;  Sala  1998). 

For  example  a student  of  a first  electric  circuits  course  should  be  learn  the  oscilloscope  or  meter  behaviour  and 
should  be  able  to  handle  them,  while  the  post-graduate  student  should  be  interest  more  deeply  to  the  internal 
circuitry  and  use  these  instruments  in  more  sophisticated  applications  also  involving  uncertainty  evaluation  and 
so  on  (Pisani  et  al.  1 995).  Each  module  is  divided  into  subjects  (pages)  at  the  same  hierarchical  level;  each  level 
contains  a subset  of  other  pages,  placed  at  a lower  levels. 

Hyperlinks  among  pages  have  been  studied  to  give  continuity  to  learning  trail  (Sala  1999). 

During  software  development,  we  put  a particular  care  to  make  “friendly”  the  user  interface  (Sala  1997;  Sala 
1999)  by  designing: 

• icons,  with  shape  and  image  recall  the  linking  function  (e.g.,  an  open  door  recalls  the  function  to  go  out  from 
the  application); 

• text-frame,  giving  brief  information; 

• hot  words,  which  open  one  a few  panel  of  explanations; 

• a map,  always  available  in  the  pages,  which  performs  the  “here  we  are”  function  and  makes  easy  the  navigating 
from  page  to  page  (the  navigation  inside  a hypertext  is  important)  (Calvi  1997); 

• visual  interface,  which  involves  the  choice  of  colours,  background,  bottom  shape,  type  of  font  and  so  forth 
(Sala  1998). 

Different  media  are  involved  e.g.: 

• animation  techniques,  which  are  an  efficient  learning  tool,  when  the  teaching  of  a subjects  would  be  difficult 
by  a written  description  alone  (e.  g.,  the  electron  beam  generation  and  control  into  a cathode  ray  tube); 

• digitalized  television  camera  images  (e.  g.,  to  zoom  an  instrument  inside); 

• audio  supports  to  stress  some  topic  in  a lesson  (Figure  1). 
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Figure  1 A page  dedicated  to  the  analog  oscilloscope 

Virtual  instruments  are  implemented  in  the  multimedia  packages  in  order  to  allow  simple  simulations  of  the  real 
instruments  during  the  self-training  phase.  For  example  when  learning  the  oscilloscope,  the  student  has  a virtual 
instrument  panel  available,  where  he  can  try  to  select  the  input  signals  and  can  evaluate  the  effects  of  the 
instrument  setting  on  the  display  (Figure  2). 


Figure  2 The  virtual  panel  of  the  analog  oscilloscope 
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Each  lecture  includes  several  tests  to  verify  the  level  of  the  acquired  knowledge  on  the  educational  objectives 
and  lesson  subjects. 

The  test  results  are  stored  into  a database  file,  which  can  be  processed  by  an  external  DBMS  (Data  Base 
Management  System)  program  with  the  possibility  to  extract  statistical  indicators  on  the  class  by  DBMS  query. 
Examinations  include: 

• puzzle,  where  students  must  rebuild  an  instrument  block  diagram  (e.g.,  oscilloscope  or  electronic  instrument) 
or  measurement  procedures. 

• laboratory  quizzes  by  multiple  choices; 

• simple  projects,  which  could  require  few  data  processing.  In  all  exercises,  the  data  are  randomly  selected  to 
always  generate  new  projects  entry  points; 

The  students  can  adapt  the  learning  rate  to  their  time  availability  and  cultural  background.  They  can  study  the 
theory  and  then  self-verify  their  knowledge  level  by  solving  practical  exercises  and  answering  a series  of 
questions.  The  student  only  need  an  inexpensive  personal  computer  with  a multimedia  extension,  that  is 
available  in  centralized  structures  of  the  university  or  possibly  at  home. 

At  present,  several  modules,  concerning  the  basic  laboratory  instrumentation  have  been  developed  using 
Toolbook^^^\  with  the  aid  of  several  thesis  student. 

The  subjects  are: 

• fundamentals  of  the  analog  oscilloscope,  which  includes  practical  exercises  performed  on  a virtual 
instrument  that  emulates  the  main  operating  functions; 

• fundamentals  of  the  analog  and  digital  voltmeters  and  their  operating  theory; 

• the  IEEE488  standard  interface  for  programmable  instrumentation. 


Experimental  training  using  hypermedia  facilities 

The  courseware  typology  explained  in  the  previous  section  well  suits  both  traditional  and  innovative  courses; 
however  a further  step  is  necessary  in  order  to  completely  adapt  it  to  the  requirements  of  courses  with  an 
intensive  experimental  activity.  Instrument  behaviour  knowledge  is  only  one  of  the  know-how  that  are  required 
to  carry  out  a measurement:  performing  a measurement  is  not  so  easy  as  it  requires  one  to  define  measurement 
procedures,  to  use  and  correctly  connect  different  instruments,  and  finally  to  process  the  experimental  data. 

This  knowledge  stock  is  traditionally  achieved  by  performing  practical  activities  in  different  subjects. 
Unfortunately,  such  an  approach,  which  requires  intensive  laboratory  attendance,  cannot  be  followed  in  distance 
learning  courses:  a further  step  is  necessary  to  break  the  time/distance  constraint. 

A possible  solution  could  be  attained  by  taking  advantage  of  modem  instrumentation,  which  is  equipped  with  a 
standard  computer  interface  and  is  remotely  programmable 

The  idea  is  to  design  a laboratory  that  is  simultaneously  and  remotely  accessible  to  several  students,  who 
concurrently  share  the  same  instrumentation,  but  not  necessarily  the  same  experiment.  The  instrumentation  and 
the  other  hardware  resources  are  accessible  in  a sort  of  time  sharing  process,  which  is  managed  by  a server  and 
transparent  to  the  user. 

The  virtual  laboratory  architecture  is  composed  of  two  kinds  of  subsystems:  the  measurement  Server  and  the 
measurement  clients.  In  detail: 

• The  measurement  Server  is  a computer  located  in  the  laboratory  and  directly  connected  to  the  programmable 
instrumentation.  The  Server  program,  that  runs  on  the  measurement  Server  accepts  measurement  requests 
from  several  clients,  dispatches  the  commands  to  the  instruments  and  sends  the  responses  back  to  the  clients. 

• The  measurement  clients  are  computers  running  a program  that  enables  the  students  to  manage  the 
experiments  via  suitable  virtual  panels  and  is  able  to  contact  the  Server  and  display  the  measurement  results. 

Many  solutions,  based  on  proprietary  or  standard  protocols,  can  be  designed  to  obtain  the  connection  between 
clients  and  the  Server.  The  standard  protocols  that  are  based  on  the  TCP/IP  protocol  suite  using  the  highest 
possible  protocol  layer.  Such  a choice  restricts  the  connection  flexibility,  but  makes  the  developed  system  easily 
portable  to  different  platforms  and  greatly  simplifies  the  program  development. 

The  connection  among  clients  and  Server  is  therefore  established  at  the  session  level  by  employing  the  session 
and  transport  services  which  are  provided  by  the  computer  operating  systems:  in  the  Windows  environment  the 
session  level  that  is  used  is  the  Dynamic  Data  Exchange  (DDE)  used  with  the  TCP  transport  (Figure  3). 

In  addition,  to  simplify  further  the  development  of  the  different  experiments,  an  application  layer  program  has 
been  designed  to  handle  the  client-server  communication.  Such  a program,  which  is  called  Measurement  Server 
Access  Facility  (MSAF),  is  composed  of  a Server  and  a client  part.  The  Server  part,  which  runs  on  the 
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measurement  Server,  implements  the  queuing  process  on  the  incoming  requests  and  keeps  track  of  the  different 
instrument  configurations,  that  are  required  by  the  clients.  The  client  part,  which  runs  on  each  client,  contains 
procedures  to  find  out  the  Server,  to  establish  the  connection,  to  post  the  measurement  request  and  to  inform  the 
client  program  about  the  expected  delay  before  the  measurement  is  actually  performed. 

By  using  the  services  provided  by  the  MSAF,  the  experiment  designer  is  only  requested  to  deal  with  the 
measurement  problem  without  being  bothered  by  the  underlining  communication  process. 

The  instruments  are  connected  to  the  Server  via  an  IEEE488.2  bus  while  the  device  under  test  is  managed  by  a 
driving  device  which  configures  the  experiment  set-up  using  a series  of  switches. 


Figure  3 Client  Server  layout. 

The  driving  device  uses  a multiplexed  address/data  scheme  that  allows  a theoretical  maximum  of  65k  switches  to 
be  used.  The  maximum  number  of  concurrent  experiments  obviously  depends  on  their  complexity  and  layout 
and  on  the  number  of  switches  that  are  actually  employed. 

The  experiment-specific  part  of  the  Measurement  Server  is  restricted  to  a parser  section  which  interprets  the 
client  requests  and  dispatches  them  to  the  instruments  in  a proper  format.  The  experiment-specific  part  of  the 
client  is  restricted  to  a virtual  panel  containing  the  instrument  controls  and  switch  set-up. 

The  Local  Server  is  a program  that  runs  on  the  client  computer,  and  which  allows  a simulated  version  of  the 
experiment  to  be  carried  out  in  the  absence  of  a network  connection  between  the  client  computer  and  the 
measurement  Server.  Such  a simulation  can  be  used  for  a “pre-training”  on  the  program  to  reduce  the  network 
traffic. 

All  the  software  has  been  developed  in  the  Visual  Basic  environment  with  the  exception  of  a minor  part 
written  in  Microsoft  The  software  is  released  with  a set  up  program  that  install  all  the  required 

components  in  the  client  computer. 

This  architecture  is  capable  of  servicing  about  twenty  clients  within  ten  seconds  which  is  quite  a reasonable 
waiting  time  and  is  suitable  for  many  different  experiments.  Up  to  now  several  experiments  regarding  both  the 
use  of  fundamental  instruments  (oscilloscopes,  DMM, ,..,)  and  the  basic  measurement  techniques  (for  example 
for  measuring  resistance,  power, ..)  have  been  designed. 

Experimental  results 

About  one  hundred  students  (selected  courses  both  in  the  “Nettuno”  project)  are  involved  in  the  experiment  and 
the  first  response  has  been  positive.  A the  moment  using  these  facilities  and  in  conventional  degree  courses 
The  testers  were  got  used  to  the  new  teaching  facility  quickly  and  with  great  interest.  In  a short  time  they  have 
been  able  to  discover  minor  problems  and  inconsistencies  in  the  proposed  material  and  suggest  valuable 
remedies.  This  feedback  is  quite  useful  both  for  teachers  and  students  and  allows  a continuous  quality 
improvement. 

The  effectiveness  of  the  learning  process  of  the  course  cannot  yet  be  stated,  since  these  innovations  were  only 
introduced  last  year,  nevertheless  the  students’  marks  in  the  first  tests  show  a positive  trend. 
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Conclusions 


Whether  it  is  accepted  or  not,  a fundamental  shift  is  occurring  in  education  as  a result  of  increasing  use  of 
computers.  Strange  (1995)  calls  the  shift  as  a cultural  revolution  in  teaching  and  learning.  The  profound  changes 
in  the  dissemination  of  information,  the  reorganization  of  information  to  include  multimedia,  and  the 
implications  of  what  it  is  to  be  a teacher  in  an  era  where  teaching  and  learning  increasingly  means  listening  and 
watching  as  well  as  reading  and  writing  have  not  been  absorbed.  The  extent  of  the  changes  due  to  this  cultural 
revolution  vary  depending  on  the  district,  school,  and  teacher  (Rath,  Rieck,  and  Wadsworth  1998). 

The  use  of  multimedia  solutions  in  degree  level  courses  has  proved  to  be  effective  in  several  different  fields. 
Courseware  with  important  laboratory  activities  can  also  be  tackled  by  adding  an  experimental  section  to  the 
conventional  theoretical  approach. 

Technical  teachers  can  use  these  multimedia  solutions  to  train  new  electronic  instrumentation  (Sala  1997). 
Integrating  technology  into  teacher  training  programs  is  important  because  it  appears  that  through  the  use  of 
technology,  teachers  change  the  way  they  present  instruction  (Stuhlmann  1998). 

Becker  (1992)  reported  that  teachers  who  had  more  experience  with  computers  were  more  inclined  to  integrate 
the  curriculum  across  subject  areas  and  use  computers  as  a tool  for  learning  (Stuhlmann  1998). 

The  solution  designed  by  the  author  for  electronic  measurement  courses  is  an  example  of  an  integrated  solution 
in  which  the  theoretical  part  is  made  available  through  a hypermedia  and  the  laboratory  part  can  be  carried  out 
by  accessing  real  instruments  through  a network  connection. 

Other  modules  are  in  aim  to  be  developed  concerning  the  impedance  meters,  the  spectrum  analyser  and  so  on. 
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Abstract:  A tempting  solution  to  the  problem  of  too  much  text  on  screen  is  to  present  some  of 
it  via  a spoken  commentary.  Questions  then  arise  as  to  what  is  the  best  balance  between 
displayed  and  spoken  text,  what  pedagogic  implications  accompany  the  use  of  speech,  how 
does  spoken  text  differ  from  written  text,  and  whether  different  types  of  user  prefer 
information  in  different  media.  This  study  addresses  these  questions  on  the  basis  of  tests 
carried  out  on  an  interactive  project,  and  a review  of  the  literature  of  educational  technology. 
It  shows  that  users  express  a clear  preference  for  a mix  of  media,  and  students  complete  study 
tasks  more  quickly  when  information  is  available  in  sound  and  text.  There  is  some  evidence 
that  preferences  for  one  medium  over  another  can  be  correlated  to  educational  background 
and  to  sex.  The  potential  value  of  further  research  in  these  areas  is  indicated. 


Introduction 

Research  on  the  effectiveness  of  different  media  within  multimedia  learning  materials  has  produced  no 
firm  evidence  on  the  question  of  whether  speech  or  text  is  more  effective  pedagogically,  but  users  show  a clear 
preference  for  a combination  of  text  and  speech  (Alessi,  1991;  Barron  and  Atkins,  1993;  Shih  and  Alessi,  1996). 
This  preference  was  shared  by  the  volunteers  who  tested  the  project  prepared  as  part  of  this  study  (Reilly  et  aL, 
1997). 

There  is  evidence  to  show  that  reading  large  amounts  of  continuous  text  from  computer  screens  is 
tiresome  (Dillon  et  al,  1988;  Gould  et  aL,  1987;  Gould  and  Grischkowsky,  1984),  and  that  tasks  linked  to 
reading  are  performed  less  efficiently  when  the  text  is  on  screen  (O’Hara  and  Sellen,  1997).  Open  University 
students  who  were  required  to  use  a CD-ROM-based  library  of  articles  as  part  of  a course  in  information 
technology  reported  a reluctance  to  read  text  from  screen  (Jones  et  aL,  1998).  These  findings  might  lead  one  to 
suppose  that  the  use  of  recorded  speech  to  reduce  the  density  of  screen  text  would  be  welcomed  by  students,  but 
the  evidence  from  software  trials  is  not  clear  cut 

As  interactive  media  tools  become  more  widely  used  in  teaching  there  is  a need  to  clarify  their  value  as 
learning  materials,  and  to  determine  guidelines  for  good  practice  in  their  design.  This  study  suggests  rules  of 
thumb  for  the  appropriate  use  of  recorded  speech  and  displayed  text  in  educational  multimedia. 


The  Renewable  Energy  Project 

In  1998  I was  part  of  a team  of  four  researchers  who  produced  an  example  of  interactive  learning 
material  based  on  an  Open  University  undergraduate  course  on  Renewable  Energy  (Reilly  et  aL,  1998).  Part  of 
the  project  presented  information  in  a mixture  of  screen  text  and  audio  voiceover.  The  project  was  tested  in  two 
phases  by  a total  of  eighteen  subjects. 


834 


Educational  Techniques  and  Technologies 

The  designers  of  educational  multimedia  attempt  to  create  products  that  adopt  a student-centred 
approach  to  learning.  Like  the  face-to-face  tutorial,  teaching  and  learning  tools  should  be  adaptive,  create  a 
discourse  between  teacher  and  learner,  encourage  reflection,  be  interactive  and  enable  the  student  to 
communicate  their  conception  of  what  they  have  learned. 

Where  these  characteristics  are  present,  the  teacher,  the  student  and  the  media  of  instruction  together 
form  a ‘conversational  framework’,  which  ‘serves  both  to  clarity  the  second-order  character  of  academic 
learning,  and  to  define  its  essential  components’  (Laurillard,  1993,  p.  102).  This  conversational  framework 
allows  the  student  to  reflect  upon  the  subject  taught,  and  convey  to  the  teacher  his  or  her  understanding.  The 
teacher  can  adapt  the  teaching  approach  in  the  light  of  the  student’s  description  of  their  understanding,  and  the 
student  can  amend  their  conception  in  the  light  of  the  teacher’s  response.  In  this  way  a discourse  is  established 
through  which  the  process  of  learning  moves  forward. 

No  artificial  presentation  can  entirely  substitute  for  a one-to-one  tutorial  dialogue  between  a teacher  and 
student.  But  the  opportunity  to  use  recorded  speech  in  the  context  of  an  interactive  presentation  can  bring 
something  of  the  quality  of  real  dialogue  to  a piece  of  tutorial  software. 

Our  project  testing,  and  the  results  of  research  cited  in  the  next  section,  show  that  users  respond 
favourably  to  teaching  software  that  includes  recorded  speech. 


Relative  Advantages  of  Text  and  Speech 

Alessi  (1991),  writing  in  the  context  of  the  instructional  use  of  interactive  video,  has  proposed  a number 
categories  for  classifying  the  relative  advantages  of  text  and  speech,  including:  (a)  realism,  (b)  dual  modality, 
(c)  learner  controllability,  and  (d)  learner  characteristics. 


Realism 


Realism  risks  overloading  students  with  excessive  detail.  However,  “The  use  of  human  voice  in  the 
delivery  of  instruction  appears  to  be  a case  where  increased  fidelity  does  not  correspond  to  increased  detail.” 
(Shih  and  Alessi,  1996:  p.  206) 


Dual  Modality 

Do  two  channels  of  instruction,  textual  and  auditory,  produce  a better  learning  outcome  than  one 
channel  alone? 

Barron  and  Atkins  observe  that  students  with  higher  verbal  skills  do  not  gain  significantly  from  the 
addition  of  audio,  but  that  students  with  lower  literacy  skills  may  register  better  performance  with  additional 
audio. 

Where  both  auditory  and  visual  media  are  used  in  an  instructional  product  there  is  some  evidence  to 
suggest  how  they  can  be  synchronised  to  best  effect.  Baggett  (1984)  conducted  a series  of  tests  in  which  students 
were  asked  to  study  the  components  of  an  assembly  kit.  The  parts  were  shown  using  a videotape  sequence,  but 
they  were  only  named  in  an  audio  voiceover.  Several  presentations  were  made  to  several  different  sets  of 
students.  Some  heard  the  commentary  slightly  ahead  of  the  visual  information,  some  saw  both  video  and  audio 
synchronously,  and  some  saw  presentations  in  which  the  audio  lagged  behind  the  video  by  up  to  21  seconds. 

The  results  showed  that  when  the  audio  preceded  the  video  information  much  of  the  audio  commentary 
was  forgotten.  The  best  recall  was  registered  by  the  groups  who  experienced  both  audio  and  video 
simultaneously  or  with  a 7 second  delay.  Delays  greater  than  7 seconds  led  to  success  rates  80%  worse  than  the 
best  scores. 
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Controllability 


A weakness  of  audio  instruction  is  that  it  is  transient.  Romiszowski  (1988)  reports  on  research  carried 
out  at  the  BBC  that  showed  that  only  28%  of  the  content  of  a five-minute  radio  programme  was  retained  by  the 
average  listener.  University  graduates  took  in  48%,  while  recent  school  leavers  retained  only  21%.  One  of  the 
advantages  of  text  is  that  it  can  be  rehearsed.  Text  can  also  be  read  at  the  reader’s  ideal  pace. 

For  audio  instruction  to  be  similarly  useable  it  must  be  controllable,  at  least  to  the  extent  that  users  can 
listen  to  short  sections  at  a time,  and  pause  and  replay  them  at  will.  This  was  borne  out  by  the  testing  we  carried 
out  on  our  project  (Reilly  et  ai,  1997).  Our  test  subjects  criticised  our  first  prototype  because  the  audio 
sequences  were  too  long,  and  could  only  be  replayed  from  the  beginning.  In  the  second  version  the  sound  files 
were  segmented,  and  each  segment  could  be  selected  and  played  separately. 


Learner  Characteristics 

As  noted  above,  variations  in  literacy  skills  will  have  an  effect  on  the  suitability  of  sound  or  text  as  the 
medium  of  instruction.  Other  learner  characteristics  may  also  play  a part  in  the  way  in  which  media  are 
perceived  by  users.  The  results  of  our  project  testing  (Reilly  et  al,  1997)  showed  a difference  between  men  and 
women  in  the  acceptability  of  spoken  commentary. 

Our  study  also  suggested  that  educational  background  may  have  an  influence  on  how  different  media 
are  perceived  by  users.  Of  eighteen  subjects  who  worked  through  our  project  over  the  course  of  two  rounds  of 
testing,  four  were  graduates,  one  was  an  undergraduate  and  thirteen  were  non-graduates.  Two  of  the  graduates 
and  the  undergraduate  expressed  a preference  for  reading  over  listening  to  a spoken  commentaiy.  All  the  rest 
reported  that  they  found  the  combination  of  text  and  commentary  helpful.  Age  variations  may  also  play  a part  in 
how  different  media  are  accepted  and  used. 


Balancing  the  Pros  and  Cons 

This  analysis  suggests  that  a choice  can  be  made  on  the  basis  of  the  nature  of  the  content,  the  literacy 
skills  of  the  target  users,  and  the  constraints  imposed  by  the  requirements  of  production  and  updating.  A simple 
rule  of  thumb  emerges:  the  more  abstract  the  content  and  the  more  literate  the  users,  the  more  text  one  would 
use;  the  more  concrete  the  content  and  the  less  literate  the  users,  the  more  speech  one  would  use 

Design  considerations  based  on  these  characteristics  alone  are  insufficient.  Before  this  rule  of  thumb 
can  be  used  with  any  confidence  it  is  necessary  to  examine  evidence  for  its  applicability. 


Effects  on  Learning 

When  Shih  and  Alessi  (1996)  offered  a group  of  test  subjects  a choice  of  text  only,  voice  only,  and  a 
combination  of  text  and  voice,  82%  showed  a strong  preference  for  the  combination  of  text  and  voice. 

Their  results  are  consistent  with  those  of  Barron  and  Atkins  (1993)  who  found  no  statistical  difference 
in  achievement  between  a text-based  tutorial  program  and  a combined  audio/text  program,  although  the 
combined  presentation  was  completed  by  subjects  significantly  more  quickly. 

This  observation,  that  a combination  of  media  leads  to  a quicker  assimilation  of  information,  bears  out 
the  results  of  a number  of  studies  cited  by  Bagui  (1998).  Kulik,  Bangert  and  Williams  (1983)  recorded  a 
substantial  cut  in  study  time  for  a group  of  secondary  school  students  who  followed  a computer-based  instruction 
session,  compared  to  another  group  who  were  taught  the  same  content  in  a conventional  classroom  lesson. 
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User  Preferences 


The  results  of  the  studies  referred  to  above  were  more  clear  cut  in  showing  the  preferences  of  users  for 
types  of  presentation.  When  my  colleagues  and  I tested  our  project  (Reilly  et  al  1997)  we  found  a similar 
marked  preference  for  combined  speech  and  sound.  Our  project  design  included  a series  of  audio/text  screen 
sequences  in  which  a spoken  commentary  was  summarised  by  bullet  points  that  appeared  sequentially  on  screen. 
In  some  sequences  the  bullet  points  were  verbatim  quotes  from  the  commentary.  In  others  they  were  paraphrases. 
In  all  cases  the  displayed  summary  appeared  on  screen  slightly  ahead  of  the  corresponding  commentary.  None  of 
the  test  subjects  commented  on  the  difference  between  the  verbatim  summaries  and  the  paraphrased  ones.  This 
difference  either  went  unnoticed,  or  was  not  considered  significant. 

The  sequence  of  text  first,  commentary  following  but  overlapping  was  noticed.  Half  the  subjects 
suggested  that  separating  the  text  and  speech  would  be  better  that  having  the  audio  running  while  the  text  was 
being  read,  ‘T  would  prefer  the  audio  sound  to  come  either  before  or  after  the  bullet  points  rather  than  together  as 
1 found  it  distracting,”  (Quote  from  test  subject  (Reilly  et  al.,  1997:  p,  3),) 

Dual  modality  in  this  context  was  not  seen  to  be  helpful.  The  subjects  clearly  preferred  to  experience  a 
single  channel  of  communication. 

Most  of  this  group  said  they  would  prefer  to  read  the  summary  before  they  heard  the  commentary  - 
although  one  subject  suggested  the  opposite  approach. 

Other  potential  variables  that  deserve  further  study  include  whether  personality  type,  sex  or  age  affect 
the  acceptance  of  different  media  types.  The  test  subject  who  said  he  would  prefer  text  only  commented  that  he 
wanted  greater  control  over  the  pacing  of  the  presentation. 

Control  was  a key  element  in  all  the  test  subjects’  responses.  The  spoken  commentary,  while  generally 
well  received,  was  perceived  to  be  difficult  to  select  and  rehearse.  These  difficulties  are  not  intrinsic  to  an  audio 
presentation,  but  were  a consequence  of  the  way  in  which  we  designed  our  use  of  audio.  In  our  revisions  of  the 
project  we  addressed  this  issue  by  dividing  the  audio  files  into  smaller  chunks  and  allowing  them  to  be 
individually  selected,  played,  paused  and  replayed. 


The  Combination  of  Text  and  Speech:  the  Value  of  Redundancy 

The  clear  preference  expressed  by  users  in  all  the  studies  cited  for  a combination  of  text  and  speech 
suggests  the  question:  ‘What  merit  is  there  in  presenting  the  same  information  in  two  forms?’  Donald  Norman 
(1993)  lists  some  advantages  for  redundant  visual  information.  He  describes  the  case  of  a business  presentation 
where  it  is  not  uncommon  for  the  speaker  to  use  slides  that  provide  no  new  information,  but  which  essentially 
repeat  what  is  being  said.  These  slides  offer  the  audience  a shared  workspace;  they  afford  co-operative  work; 
they  represent  a permanent  memory;  they  increase  available  memory;  they  aid  perceptual  processing;  and  they 
offer  an  alternative  channel  of  communication  for  those  people  who  prefer  auditory  information. 

Multimedia  presentations  extend  these  advantages.  Here  time  is  under  the  control  of  the  users,  not  the 
lecturer.  Control  is  also  in  the  hands  of  the  user  if  they  are  able  to  reselect  and  replay  information.  Perceptual 
processing  is  afforded  not  only  by  the  spatial  arrangement  of  ideas  on  a single  screen,  but  also  by  the 
accessibility  of  other  screens.  Individual  differences  can  be  catered  for,  not  only  in  offering  choices  between 
spoken  and  displayed  text,  but  also  by  offering  information  in  other  media  such  as  video  or  animations.  The 
wider  the  choice  of  media  types,  the  more  a wide  range  of  users  will  feel  accommodated  by  the  presentation. 

Greater  redundancy  provides  the  opportunity  for  users  to  tailor  a presentation  to  their  tastes.  But  the 
danger  of  excessive  redundancy  is  that  users  may  feel  uncomfortable  because  they  do  not  have  a feel  for  how 
much  content  they  should  access  to  gain  an  adequate  understanding.  The  initial  reaction  of  many  students  to  an 
Open  University  CD-ROM  that  contained  a library  of  reference  material  was  to  try  to  read  as  much  of  the 
content  as  possible  in  case  they  inadvertently  missed  some  important  detail  (Jones  et  al.,  1998),  Only  when  they 
had  become  skilled  in  carrying  out  focused  searches  of  the  material  were  they  reassured  about  how  much 
material  it  was  necessary  to  view. 

Successful  use  of  interactive  material  in  which  there  is  a significant  degree  of  redundancy  depends  upon 
users  acquiring  the  skills,  and  the  tools,  to  be  selective  and  to  evaluate  the  relevance  of  the  content  they  retrieve. 
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Spoken  and  Written  English 


An  important  aspect  of  the  design  of  a multimedia  product  that  uses  both  spoken  and  displayed  text  is  a 
recognition  of  the  differences  between  spoken  and  written  English.  The  two  forms  of  language  are  distinct. 
Written  English  conforms  to  rules  of  style  and  syntax  which  do  not  apply  to  spoken  English.  When  a speaker 
uses  the  style  of  written  English  he  appears  either  to  be  reading  or  reciting  a document,  or  else  he  comes  across 
to  his  listeners  as  stilted  and  excessively  formal.  These  characteristics  are  often  interpreted  as  insincerity  (Jay 
and  Jay,  1996). 

When  a reader  reads  a text,  they  can  pause  over  aspects  that  are  not  immediately  clear.  When  listening 
this  is  more  difficult.  Although  it  is  possible  to  provide  users  with  the  option  to  pause  an  audio  sequence,  if  your 
user  has  to  pause  frequently  they  will  find  your  product  tiresome  to  use.  There  are  certain  key  features  of  well 
prepared  spoken  English  that  help  to  avoid  the  need  to  pause. 

Abstract  nouns  should  be  avoided.  These  can  cause  listeners  to  struggle  with  the  speaker’s  meaning. 

Spoken  English  should  use  short  words  and  sentences,  and  should  avoid  abbreviations,  lots  of  numbers, 
and  unfamiliar  names  (Garrand,  1997).  Numbers  are  particularly  difficult  to  absorb  and  remember  from  speech. 

The  active  rather  than  the  passive  voice  should  be  used.  For  example,  ‘Solar  energy  provides  ten  per 
cent  of  Greece’s  winter  heating’,  is  better  than  ‘Ten  per  cent  of  Greece’s  winter  heating  is  provided  by  solar 
energy.’ 

Specialist  terminology  should  only  be  used  for  specialist  audiences,  or  when  its  use  is  specifically 
explained. 

Word  and  phrase  order  are  important.  Whereas  the  rules  of  written  style  suggest  that  one  should  avoid 
placing  parenthetical  phrases  after  the  main  verb,  this  style  is  difficult  for  listeners  to  follow.  Similarly,  with  a 
document  readers  can  jump  ahead  to  see  how  much  they  still  have  to  read.  Headings  and  paragraphs  can  provide 
signposting  and  orientation.  With  spoken  English  it  is  necessary  to  build  in  signposting  phrases,  such  as  “Well, 
that’s  the  basic  question,  so  now  we’ll  take  a more  detailed  look  at  viable  options.”  (Jay  and  Jay,  1996:  p.  50). 


Conclusions 

The  desire  to  create  a conversational  framework  or  dialogue  within  which  to  situate  learning  has 
implications  for  the  design  of  learning  tools.  Interactivity  helps  to  create  this  framework,  and  the  use  of  speech  in 
addition  to  text  adds  an  element  of  realism  to  the  media  mix. 

This  dual  modality  of  verbal  and  textual  communication  presents  advantages  in  giving  learners  an  aid  to 
processing  and  remembering  information.  It  offers  users  choice,  and  an  element  of  control  over  the  type  of 
learning  medium  and  the  pace  of  study.  Combining  speech  and  displayed  text  is  welcomed  by  users  and 
encourages  efficient  study. 

In  examining  learning  outcomes  it  will  be  necessary  to  classify  users  by  level  of  literacy,  previous 
educational  attainment,  age  and  sex.  The  evidence  from  our  project  testing  suggests  that  the  way  in  which 
information  in  different  media  is  received  may  depend  to  some  extent  on  these  variables.  The  issue  may  be 
complicated  by  age.  Sex  also  seems  to  be  a factor,  with  men  being  less  receptive  to  a spoken  commentary  than 
women. 

If  such  differences  prove  to  be  significant,  the  value  of  media  redundancy  will  become  clearer. 
Redundancy  provides  the  opportunity  for  users  to  tailor  a presentation  to  their  tastes,  and  enables  a piece  of 
software  to  cater  for  a range  of  users. 

Our  study  also  revealed  that  to  be  acceptable  a spoken  commentary  must  be  controllable.  It  should  be 
available  in  short  discrete  parts,  each  of  which  can  be  paused  and  replayed,  and  each  of  which  can  be  selected 
and  played  individually  (although  they  may  need  to  run  in  a particular  sequence  when  first  heard). 

While  media  redundancy  is  welcomed  by  users,  media  overlap  can  be  confusing.  Users  should  be  given 
the  opportunity  to  read  and  listen  separately,  and  not  be  forced  by  the  design  to  do  both  at  the  same  time. 

In  scripting  passages  of  recorded  speech,  the  scriptwriter  must  be  aware  of  the  difference  between 
spoken  and  written  language.  Spoken  language  follows  different  rules  of  syntax  from  those  of  written  language. 
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and  while  vocabulary  need  not  differ,  the  excessive  use  of  specialised  vocabulary  in  speech  runs  the  risk  of 
confusing  listeners,  even  when  they  are  familiar  with  it. 

Further  empirical  studies  and  more  widespread  experience  in  the  use  of  multimedia  in  higher  education 
will  improve  the  base  of  evidence  on  which  future  design  strategies  can  be  founded.  In  the  meantime,  the  study 
described  here  and  the  evidence  available  from  the  literature  suggest  that  presenting  information  in 
complementary  media,  with  careful  thought  given  to  the  balance,  synchronisation,  overlap  and  redundancy  of 
media  forms,  will  assist  learners  in  shaping  their  own  learning  strategies  and  give  them  greater  control  over  their 
learning. 
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Incorporating  the  Internet  in  the  Classroom 


By  John  F.  Bennett  Marketing  Professor,  Stephens  College,  Columbia,  MO.  USA 


Introduction 

Much  attention  has  been  devoted  in  recent  years  to  the  development  of  Internet  courses  that  are  targeted  to  distance 
learning  students.  This  follows  the  movement  towards  the  establishment  of  virtual  campuses  by  many  business 
schools  including  such  prestigious  institutions  as  Duke,  which  recently  graduated  its  first  class  in  its  Global  EMBA 
program,  and  MIT,  which  offered  its  first  credit  course  in  accounting  over  the  World  Wide  Web  in  fall  1997.  While 
this  trend  will  continue  to  grow  the  fact  remains  that  the  majority  of  business  courses  continue  to  be  offered  in  a 
traditional  classroom  setting.  However,  while  most  business  courses  are  being  offered  in  a traditional  classroom, 
they  are  certainly  not  being  offered  in  a conventional  manner  as  more  business  schools  have  exhorted  their  faculty  to 
incorporate  Internet  technologies  in  their  courses.  Unfortunately,  those  faculty  desiring  or  being  exhorted  to  move 
toward  technology-mediated  instruction  are  not  always  given  sufficient  guidance  on  how  this  should  be  done. 
Consequently,  many  respond  by  either  disdaining  technology  and  holding  steadfastly  to  the  status  quo  or,  worse, 
using  a haphazard  approach  in  incorporating  technology  in  their  teaching  without  any  real  understanding  of  its 
pedagogical  value.  The  purpose  of  this  paper  is  to  provide  direction  to  those  of  want  to  improve  the  quality  of  their 
on-campus  courses  by  incorporating  Internet  technology  in  their  teaching. 


Rationale  for  Using  Internet  Technologies  in  the  Classroom 

While  there  are  several  Internet  technologies  that  can  be  used  to  support  instruction,  the  ones  that  seem  to  be 
generating  the  greatest  interest  among  educators  are  email,  electronic  discussion  groups  and  the  World  Wide  Web. 
This  is  hardly  surprising  when  one  considers  the  many  benefits  these  tools  offer  the  classroom  teacher.  One  such 
benefit  is  improved  communication.  Electronic  discussion  groups,  for  example,  can  serve  to  enhance  and  extend 
interactions  between  students  and  between  students  and  their  instructors  (Karayan  and  Crowe,  1997).  Students  in  a 
class  can  share  an  insight  or  opinion  at  any  time  of  day  or  night  rather  than  having  to  wait  until  the  next  class 
meeting.  It  also  provides  a means  of  participation  for  those  students  who  are  reluctant  to  participate  in  traditional 
class  discussions  (Partee,  1996).  It  is  often  easier  for  these  students  to  discuss  values  and  personal  concerns  in 
writing  than  orally,  since  inadvertent  or  ambiguous  nonverbal  signals  are  not  so  dominant  (Chickering  and  Ehrmann, 
1996).  More  vocal  or  impetuous  students  can  also  benefit  from  the  increased  reflection  necessitated  by  putting 
thoughts  into  words  for  an  on-line  discussion. 

Using  email  and  electronic  discussion  groups  also  facilitate  such  activities  as  study  groups,  collaborative  learning, 
group  problem  solving  and  discussion  of  assignments.  Students  no  longer  have  to  worry  about  finding  time  on  their 
often  busy  schedules  to  have  a face  to  face  meeting  with  their  peers.  Increasing  opportunities  for  students  to  work 
together  increases  their  involvement  in  learning.  As  Chickering  and  Ehrman  (1996)  note,  being  able  to  work 
collaboratively  with  one’s  peers  improves  thinking  and  deepens  understanding.  It  also  allows  students  a more  real- 
world  experience  that  contributes  to  an  enhanced  educational  setting  (Hunter,  1990). 

Use  of  these  Internet  tools  can  also  encourage  active  learning.  Very  little  learning  occurs  when  students  simply 
listen  to  a teacher  and  memorize  a set  of  lecture  notes.  They  must  talk  about  what  they  are  learning,  write 
reflectively  about  it,  relate  it  to  some  past  experience,  and  apply  it  to  their  daily  lives  (Chickering  and  Ehrmann, 
1996).  In  other  words,  they  must  become  active  as  opposed  to  passive  learners.  If  structured  into  a class  in  a 
purposeful,  pedagogically-sound  manner,  listservs  can  improve  learning  (Everett  and  Ahem,  1994).  This  is 
supported  in  one  study  that  found  that  slightly  more  that  40%  of  students  indicated  that  their  learning  had  either 
increased  or  increased  a lot  from  using  e-mail  (Sandell,  Stewart,  and  Stewart,  1 996). 


840 


Requiring  students  to  communicate  via  e-mail  has  been  found  to  improve  their  ability  to  write  coherently 
(Karayan  and  Crowe,  1997).  In  the  process  of  contributing  to  the  on-line  discussion,  students  are  using  a mode  of 
expression  that  is  a form  of  writing:  composing  their  ideas  at  the  keyboard,  editing  their  comments,  and  then 
"publishing"  their  comments  to  the  list.  The  list  subscribers  then  provide  feedback,  commenting  on  the  quality  of  the 
message,  objecting  to  the  ideas  expressed,  asking  for  clarification,  and  so  on.  This  public  nature  of  the  writing  and 
public  exchange  of  ideas  causes  students  to  take  their  work  more  seriously.  They  make  a greater  effort  to  ensure  that 
their  messages  are  clearly  conveyed  and  that  their  claims  are  adequately  substantiated.  They  also  begin  to  take  pride 
in  writing  well,  especially  when  others  commend  them  on  what  they  have  written.  By  participating  in  an  public  e- 
mail  forum,  students  learn  how  to  think  through  ideas,  how  to  anticipate  and  respond  to  objections,  and  how  to 
express  themselves  with  clarity  and  power  all  of  which  improves  the  quality  of  the  responses  to  the  discussion 
(Karayan  and  Crowe,  1997). 

Expanding  the  boundaries  of  the  classroom  to  include  individuals  and  noted  scholars  from  other  parts  of  the  world 
is  made  possible  through  the  use  of  email  and  electronic  discussion  groups.  This  adds  an  intercultural  dimension  to  a 
class  that  might  otherwise  not  be  there  and  expands  the  diversity  of  thought  being  shared  in  class  discussions 
(Bailey,  1994). 

Use  of  these  Internet  technologies  also  makes  it  easier  to  appeal  to  a greater  number  of  learning  styles.  Visual 
learners,  for  example,  can  access  explanatory  diagrams  or  animations  to  enhance  meaning,  more  active  or  inquisitive 
learners  can  "read  around"  their  subject  by  means  of  hypertext  links  and  so  on.  In  addition,  the  different 
representations  of  information  (sound,  text,  video,  graphics,  etc.)  made  possible  through  Web  technology,  greatly 
increase  the  richness  and  appeal  of  the  learning  environment.  They  present  the  learner  with  a much  broader  coverage 
of  material  and  more  ways  to  encode  and  remember  that  information..  One  study  conducted  by  the  Center  for 
Innovations  in  Engineering  Education  at  Vanderbilt  University,  found  that  learners  using  an  electronic  laboratory 
simulator  on  their  own  time  in  lieu  of  physical  labs  were  able  to  solve  problems  on  physical  equipment  as  fast  or 
faster  than  those  who  took  physical  labs.  In  addition,  the  written  lab  tests  were  found  to  be  equivalent  between  the 
two  groups  (Campbell,  1998). 


Applying  Internet  Technologies  in  the  Classroom 

While  the  decision  about  whether  or  not  to  use  Internet  technology  in  the  classroom  can  be  difficult,  it  is  much 
easier  than  trying  to  decide  on  the  specific  Internet  technologies  to  use  and  how  they  should  be  applied  in  the 
classroom.  This  is  due  to  the  fact  that  teachers  do  not  all  subscribe  to  the  same  teaching  philosophy  or  style.  Rather, 
each  brings  to  the  classroom  his/her  own  unique  teaching  style  that  calls  for  a specific  set  of  instructional  methods 
and  technologies  - including  Internet  technologies  - that  would  not  be  appropriate  for  any  other  teaching  style. 
Therefore,  the  Internet  technologies  and  applications  cited  in  this  paper  are  meant  only  to  provide  the  reader  with  a 
greater  appreciation  and  understanding  of  the  pedagogical  value  of  the  Internet.  Whether  any  of  these  technologies 
or  their  applications  would  be  appropriate  for  the  reader  to  use  would,  again,  depend  on  his/her  preferred  teaching 
style. 

In  the  remainder  of  this  paper,  1 will  discuss  the  Internet  technologies  ! currently  use  in  my  undergraduate 
marketing  courses,  how  these  technologies  are  being  applied,  and  the  effectiveness  of  these  technologies  in 
enhancing  the  quality  of  my  courses. 

When  I became  convinced  that  the  Internet  had  the  potential  to  enhance  the  quality  of  my  marketing  courses  and 
began  to  incorporate  it  in  my  teaching,  I was  immediately  faced  with  a dilemma:  which  Internet  technologies  do  I 
use?  There  are  many  different  technologies  that  can  be  applied  in  the  classroom  which  is  both  an  advantage  and  a 
disadvantage.  It  is  an  advantage  because  it  gives  the  instructor  many  options  to  choose  from.  Everything  from  email 
to  chat  rooms  to  video  conferencing  over  the  Web.  However,  it  is  also  a disadvantage  in  that,  it  makes  the  task  of 
selecting  the  most  appropriate  technologies,  especially  for  someone  who  has  little  or  no  experience  using  these 
technologies  in  the  classroom,  a most  challenging  one.  To  facilitate  the  task  of  identifying  the  most  appropriate  set 
of  Internet  technologies  to  use  in  my  courses,  1 applied  the  following  evaluative  criteria:  1 . does  the  technology 
support  any  of  my  instructional  goals  and  objectives?  , 2.  is  it  technology  that  1 would  be  comfortable  using?,  and  3. 
are  there  sufficient  institutional  resources  to  support  the  technology?  After  considering  these  criteria,  1 decided  to 
use  three  Internet  technologies  in  my  courses:  email,  listservs,  and  the  World  Wide  Web.  These  were  the  Internet 
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technologies  I was  most  familiar  with  and  the  ones  that  I believed  would  best  support  my  instructional  goals  and 
objectives.  I was  also  confident  that  the  institution  could  provide  me  with  the  additional  resources  I would  need  to 
utilize  these  technologies  in  the  classroom. 

After  identifying  the  Internet  technologies  to  use  in  my  courses,  it  was  necessary  to  determine  how  they  should  be 
applied  to  meet  the  following  instructional  objectives:  1.  improve  communication  to,  from  and  among  students,  2. 
facilitate  access  to  course  materials,  2.  introduce  students  to  the  potential  of  the  Internet  as  an  information  resource 
and  encourage  them  to  use  it  as  a research  tool,  3.  increase  the  interaction  between  students  and  marketing 
practitioners,  4.  expand  coverage  of  topical  issues  in  marketing,  5.  appeal  to  a greater  variety  of  learning  styles,  and 
6.  facilitate  classroom  management.  The  specific  strategies  used  to  meet  these  instructional  objectives  are  detailed  in 
the  next  section. 


Improving  Communication 

In  order  to  improve  communication  to,  from  and  among  students  several  strategies  were  used.  During  the  first 
week  of  classes,  each  student  was  required  to  obtain  an  email  account  and  to  subscribe  to  the  class  list.  This  list  was 
used  throughout  the  semester  to  notify  students  of  scheduling  changes,  to  distribute  pertinent  news  reports  from  on- 
line publications,  to  post  a weekly  list  of  learning  objectives,  and  to  provide  any  needed  clarification  of  lecture  or 
reading  material.  Also,  on  a weekly  basis,  the  students  are  required  to  post  to  the  list  responses  to  questions  from 
assigned  readings  and  to  respond  to  at  least  two  of  the  postings  of  their  peers.  This  is  meant  to  augment  in-class 
discussions  and  provide  the  students  with  a communal  meeting  area  where  they  can  share  ideas,  ask  each  other 
questions,  seek  advice,  etc.  As  noted  earlier  in  this  paper,  providing  students  with  the  opportunity  to  participate  in 
on-line  discussions  provides  a means  of  participation  for  those  students  who  are  reluctant  to  participate  in  traditional 
class  discussions,  promotes  active  learning  and  improves  students’  ability  to  write  coherently. 

To  encourage  students  to  communicate  with  each  other  on-line,  a class  list  with  links  to  students'  email  addresses 
was  placed  on  each  course's  Web  site.  If  a student  wanted  to  send  an  email  message  to  another  student,  he/she  only 
had  to  access  the  Web  page  with  the  class  list  rather  than  search  for  that  student’s  email  address.  To  encourage 
students  to  communicate  with  me  on-line,  a link  to  my  email  address  was  provided  on  all  on-line  documents  like  the 
course  syllabus,  lecture  notes,  class  assignments,  and  review  exams.  For  those  students  who  wanted  to  communicate 
without  identifying  themselves  - most  likely  to  occur  when  the  student  has  a critical  comment  to  make  and  fears 
retribution  if  anonymity  isn't  guaranteed  - an  evaluation  form  was  made  available  on  each  course's  Web  site  that 
could  be  submitted  anonymously. 


To  Facilitate  Access  To  Course  Materials  And  Introduce  Students  To  The  Potential  Of  The  Internet  As  An 
Information  Resource  And  Encourage  Them  To  Use  It  As  A Research  Tool. 

For  each  course,  a Web  site  was  created  to  facilitate  access  to  all  course  materials  such  as  the  course  syllabus, 
class  assignments,  review  exams,  assignment  schedule,  Powerpoint  presentations  and  lecture  notes.  To  introduce 
students  to  the  potential  of  the  Internet  as  an  information  source  and  to  encourage  them  to  use  it  as  a research  tool, 
links  to  external  resources  were  provided  in  many  of  these  Web  documents.  For  example,  the  lecture  notes  have 
links  to  Web  sites  containing  additional  information  on  topics  covered  in  the  notes,  the  syllabus  has  links  to  the  on- 
line publications  used  in  the  course  such  as  Business  Week,  the  Wall  Street  Journal,  and  Advertising  Age,  and  the 
class  assignments  have  links  to  Web  sites  with  information  pertinent  to  the  assignments.  In  addition,  a directory  of 
marketing-related  Web  sites  is  maintained  on  each  course’s  Web  site  as  is  a directory  of  Web  sites  offering  guides 
designed  to  be  helpful  to  students.  These  would  include  guides  on  writing  a research  paper,  developing  a marketing 
plan,  creating  an  advertising  campaign,  giving  a formal  presentation,  etc.  Also,  students  are  encouraged  to  seek  out 
Web  sites  related  to  the  course  and  post  a link  to  the  site  on  the  class  list. 


To  Increase  Interaction  Between  Students  And  Marketing  Practitioners. 
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In  order  to  help  bridge  the  gap  between  the  classroom  and  the  "real"  world,  4-6  marketing  practitioners  with 
varied  types  of  expertise  are  invited  to  participate  in  on-line  discussions  with  my  students.  The  class  submits  a list  of 
5-7  questions  via  email  to  the  marketing  practitioner  who  then  posts  his/her  responses  to  the  class  list.  The  students 
are  then  given  up  to  a week  to  ask  the  marketing  practitioner  any  follow  up  questions  they  may  have.  The  postings 
of  the  marketing  practitioners  are  archived  and  used  in  future  classes. 


Expand  Coverage  Of  Topical  Issues  In  Marketing. 


To  keep  the  course  content  focused  on  topical  issues,  a summary  of  marketing  news  stories  from  on-line  journals, 
magazines,  and  newspapers  are  emailed  to  the  class  list  on  a weekly  basis.  Students  are  expected  to  read  these 
weekly  news  reports  and  be  prepared  to  discuss  them  in  class.  In  addition,  on  a biweekly  basis  students  are  required 
to  participate  in  an  on-line  discussion  of  a current  news  story  related  to  marketing  that  has  been  reported  in  the  Wall 
Street  Journal  (which  every  student  is  required  to  read  for  class).  Finally,  on  each  course’s  Web  site,  a 
comprehensive  directory  of  marketing- related  Web  sites  is  made  available  so  that  students  can  learn  about  current 
events  in  the  different  areas  of  marketing  studied  in  class.  To  encourage  students  to  use  this  directory,  a summary  of 
the  marketing  topics  discussed  at  each  Web  site  is  provided  as  is  a link  to  the  site.  Students  who  visit  these  sites  and 
share  with  the  class  what  they  read,  can  earn  extra  credit  points. 


Appeal  To  A Greater  Variety  Of  Learning  Styles 

To  appeal  to  the  variety  of  learning  styles  represented  in  a typical  class,  various  strategies  were  used.  For  visual 
learners,  explanatory  diagrams  were  made  available  on  the  course  Web  site.  The  diagrams  were  linked  to  the  topics 
in  the  on-line  class  notes  that  they  were  associated  with.  For  inquisitive  learners,  external  links  were  provided  in  the 
on-line  class  notes  and,  as  noted  earlier,  a directory  of  marketing-related  Web  sites  was  made  available  to  increase 
both  the  range  and  depth  of  information  available.  Finally,  the  on-line  notes  were  comprehensive  (average  length 
was  1 0 pages)  in  order  to  get  students,  especially  auditory  learners,  to  not  only  pay  greater  attention  to  what  was 
being  discussed  in  class  but  to  increase  their  degree  of  participation  in  class  discussions. 


Facilitate  Classroom  Management 

As  noted  earlier,  all  course  materials  - course  syllabus,  lecture  notes,  course  assignments,  assignment  schedule, 
and  review  exams  - are  made  available  on  the  course  Web  site.  Besides  providing  students  with  greater  and 
immediate  access  to  course  materials,  this  has  the  added  advantage  of  eliminating  the  time  and  expense  of  making 
and  distributing  hard  copies.  Other  time  saving  measures  include  the  requirement  that  all  homework  assignments  be 
submitted  electronically  so  that  they  can  be  graded  and  returned  via  email,  an  FAQ  section  on  the  course  Web  site 
with  answers  to  frequently  asked  questions  about  class  policies,  grading,  assignments,  etc.,  and  the  submission  of 
student  progress  reports  via  email. 


Student  Reactions 

Each  semester,  students  in  my  marketing  courses  are  surveyed  to  assess  their  reactions  to  using  the  Internet  in  the 
classroom.  A total  of  149  students  have  been  surveyed  over  a two  year  time  period  and  asked  to  comment  of  the  use 
of  electronic  discussion  group  assignments,  on-line  resources  such  as  lecture  notes  and  review  exams,  and  on-line 
discussions  with  marketing  practitioners.  A summary  of  the  survey  results  follows. 

Electronic  Discussion  Group  Assignments 

The  students  noted  several  benefits  and  problems  with  using  electronic  discussion  group  assignments.  The  most 
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frequently  cited  problem  with  using  EDG  assignments  was  inconvenient  access  to  computers  with  Internet 
connections,  Students  also  indicated  that  they  felt  overwhelmed  by  the  number  of  e-mail  messages  they  were 
receiving  each  week,  This  was  particularly  true  among  those  students  who  were  on  several  class  lists.  Some  students 
noted  that  there  was  too  little  discussion  of  the  pedagogical  benefits  of  electronic  discussion  groups.  Consequently, 
at  the  beginning  of  the  semester  they  failed  to  understand  why  in-class  discussions  were  being  supplemented  with  e- 
mail  discussions.  It  was  only  after  participating  in  the  electronic  discussions  for  a few  weeks  that  the  students  began 
to  understand  their  value.  Finally,  some  students  lacked  experience  using  e-mail  and  expressed  a fear  of  this 
technology.  These  students  noted  that  their  participation  in  e-mail  discussions  would  have  been  greater  if  they  had 
received  some  remedial  assistance  with  the  technology  being  used. 

One  benefit  mentioned  by  several  of  the  students  was  that  electronic  discussions  provided  a convenient  means  of 
interacting  with  peers  as  well  as  the  instructor.  They  appreciated  being  able  to  send  and  reply  to  an  e-mail  message 
at  a time  of  their  choosing.  Students  also  noted  that  it  was  easier  to  collaborate  with  their  peers  on  group  projects. 
The  frustrations  caused  by  trying  to  arrange  convenient  meeting  times  for  a group  of  students  to  physically  meet  and 
work  on  a project  are  largely  eliminated  with  electronic  discussions  groups, 

A few  of  the  students  commented  that  they  were  more  comfortable  with  written  rather  than  oral  discourse.  They 
stated  that  they  felt  more  willing  to  make  provocative  assertions  during  e-mail  discussions  and  to  challenge  the 
views  of  their  peers  and  the  instructor.  In  short,  they  felt  liberated  by  this  new  medium. 

Some  of  the  students  believed  that  they  had  a much  better  understanding  of  the  discussion  topics  as  a result  of 
being  required  to  read  and  respond  to  their  peers'  postings  to  the  list.  They  appreciated  the  fact  that  in  electronic 
discussions  they  can  take  more  time  and  reflect  longer  before  responding, 

On-Line  Resources 

Students  found  the  availability  of  lecture  notes,  Powerpoint  presentations,  and  review  tests  on  the  course  Web  site 
to  be  very  beneficial.  Many  students  commented  that  the  on-line  notes  and  Powerpoint  presentations  allowed  them 
to  focus  less  on  note  taking  and  more  on  participating  in  class  discussions,  and  improved  their  comprehension  of  the 
lecture/reading  material.  The  placement  of  links  to  external  resources  in  the  on-line  notes  was  very  much 
appreciated  by  the  students  because  they  made  it  convenient  for  them  to  gather  additional  information  on  the 
concepts  being  studied. 

The  on-line  review  tests  were  favorably  perceived  because  they  provided  students  an  opportunity  to  conveniently 
assess  their  comprehension  of  the  reading/lecture  material  at  any  time.  The  students  also  indicated  that  they  were 
encouraged  to  seek  assistance  by  being  provided  with  several  links  to  the  instructor's  email  address  on  each  on-line 
review  test.  Finally,  students  responded  that  they  found  the  on-line  review  tests  more  helpful  than  conventional 
review  tests  since  the  questions  were  linked  back  to  the  sections  of  class  notes  that  they  were  drawn  from.  This  made 
it  much  easier  for  the  students  to  find  detailed  explanations  for  the  answers  given, 

On-Line  Guest  "Speakers” 

The  opportunity  to  engage  in  on-line  discussions  with  marketing  practitioners  was  favorably  perceived  by  the 
students.  Many  of  the  students  indicated  that  their  interest  and  understanding  of  the  subject  matter  was  greatly 
enhanced  by  being  able  to  engage  in  an  on-line  discourse  with  marketers  about  a wide  variety  of  topics.  Students 
also  commented  that  these  on-line  discussions  gave  them  a much  greater  appreciation  of  the  increasingly  important 
role  that  Internet  technologies  now  assume  in  the  workplace.  This  was  reinforced  by  many  of  the  guest  speakers  who 
often  discussed  how  they  utilized  the  Internet  in  their  jobs.  Finally,  some  of  the  students  thought  the  on-line  guest 
speakers  were  much  more  informative  than  in-class  guest  speakers.  This  is  due  to  the  fact  that  each  guest  speaker 
had  agreed  to  hold  an  on-line  discussion  with  students  over  a period  of  several  days  rather  than  a single  class  period 
as  is  typically  the  case  when  a speaker  comes  to  the  classroom. 


Conclusions 

Based  on  student  feedback  and  my  own  observations  and  experiences  over  the  past  two  years,  I believe  the 
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instructional  objectives  established  for  the  Internet  technologies  I have  incorporated  in  my  courses  were  largely 
achieved.  For  others  considering  using  the  Internet  technologies  in  their  courses,  I offer  the  following 
recommendations: 

1)  Consider  only  those  technologies  able  to  support  the  teaching  and  learning  strategies  being  used.  Using 
technology  for  primarily  comestic  purposes  - perhaps  to  give  one  the  image  of  being  on  the  "cutting  edge"  - will 
almost  certainly  lead  to  a degradation  rather  than  a genuine  enhancement  of  the  quality  of  the  course. 

2)  Determine  the  availability  of  the  resources  and  support  services  needed  for  the  technology  being  considered. 
Be  especially  mindful  of  the  difficulty  students  may  have  getting  access  to  computers  connected  to  the  Internet. 
Ideally,  all  students  would  have  their  own  computers  that  would  provide  access  to  the  Internet.  However,  it  is  more 
realistic  to  assume  that  at  least  a few  students  will  be  wholly  dependent  on  the  institution's  computer  resources. 
Should  access  to  computers  on  campus  be  a problem  for  students,  the  use  of  Internet  technologies  may  need  to  be 
limited,  perhaps  significantly,  until  access  improves. 

3)  Assess  whether  there  are  differences  in  the  students'  levels  of  comfort  and  understanding  of  the  technology 
being  used.  If  differences  are  found,  provide  whatever  assistance  is  needed  to  eliminate  or  minimize  them. 
Otherwise,  some  of  the  students  may  become  frustrated  and  simply  refuse  to  use  the  technology. 

4)  Consider  making  computer  literacy  a prerequisite  for  the  course.  Doing  so  will  help  eliminate  the  need  for 
remedial  computer  training. 

5)  On  a regular  basis,  measure  the  extent  to  which  the  technologies  are  meeting  their  intended  objectives  and 
expect  to  have  to  make  several  changes  before  coming  up  with  the  optimal  mix  of  technologies  to  support  your  style 
of  teaching. 

6)  Make  sure  that  students  understand  the  pedagogical  benefits  of  the  Internet  technologies  being  used  in  the 
course.  Doing  so  can  lessen  the  resistance  students  may  have  toward  using  these  technologies. 

7)  Finally,  expect  the  unexpected.  Web  links  may  be  rendered  invalid,  disk  drives  may  crash,  pesky  viruses  may 
infect  your  network,  etc.  However,  despite  the  occasional  problems  they  may  cause,  Internet  technologies  have  the 
potential  to  greatly  enhance  the  quality  of  the  education  you  provide  your  students. 
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Abstract:  This  paper  will  report  on  the  development  and  implementation  of  the 
Millennium  Satellite  Project  (MSP)  in  five  secondary  schools  in  England.  It  will  show 
how  the  project  was  constructed,  the  roles  schools,  teachers  and  pupils  had  in  the 
development  of  teaching  material  and  learning  experiences.  The  pilot  stage  of  the 
research  is  considered  here  as  a case  study.  MSP  was  initially  constructed  to  motivate 
schools  to  use  the  Internet  in  science  teaching,  and  help  promote  one  particular  aspect 
of  the  Science  National  Curriculum  in  England  and  Wales;  that  of  Earth  and  Space.  It 
was  intended  that  pupils  would  be  able  to  access  data  from  the  satellite  through  a web- 
site on  the  Internet.  A major  objective  of  the  project  was  to  establish  the  potential  of 
using  satellite  science  within  the  National  Curriculum,  and  as  a means  of  motivating 
students  towards  Science.  The  project  is  a collaborative  venture  between  the  University 
of  Surrey,  Roehampton  Institute  and  MSCL. 

Introduction 

The  use  of  information  and  communication  technology  (ICT)  in  subject  teaching  in  England  and 
Wales  has  become  a statutory  aspect  of  a teacher’s  life.  Teachers  have  to  have  knowledge, 
understanding  and  skills  in  using  ICT  in  subject  teaching.  Within  this  remit  teachers  should 
encourage  pupils  to  become  familiar  with  ICT  and  positive  users  of  it,  ensure  that  all  pupils  have 
opportunities  to  use  ICT.  (adapted  from  Circular  4/98)  This  same  document  talks  of  ICT  being 
considered  as  more  than  just  another  teaching  tool.  Its  potential  for  improving  the  quality  and 
standards  of  pupils’  education  is  quoted  as  being  significant,  (pg.l,  Circular  4/98).  Against  this 
background  the  MSP  was  trialed  in  schools.  The  project  was  not  aimed  at  testing  or  examining  any  of 
the  above  issues.  They  serve  merely  as  a way  of  contextualising  the  problems  that  exist  in  many 
schools  trying  to  meet  the  demands  of  a National  Curriculum  and  ICT  programmes.  What  the  project 
has  brought  to  light  is  that  these  problems  are  real,  but  that  teachers  are  trying  to  deal  with  them. 

The  purpose  of  the  project  was  to  develop,  in  conjunction  with  teachers  and  schools;  suitable  teaching 
material  based  on  satellite  technology  and  communication.  The  material  had  to  be  relevant  to  the 
National  Curriculum  in  England  and  Wales,  and  allow  the  topic  of  “Satellite  Science”  to  be  taught  in 
a motivating  and  innovative  way.  The  lessons  were  originally  designed  for  Year  9 (age  13/14),  pupils, 
(attainment/assessment  levels  6 and  7 within  the  National  Curriculum  for  Science  ),  but  could  be 
adapted  to  suit  Year  7 (age  11/12)  and  Year  8 (age  12/13)  pupils,  dependent  on  the  approach  chosen 
by  the  teacher. 

The  concept  of  ‘Satellite  Science’,  was  introduced  to  the  pilot  schools  as  a series  of  lessons  based  on 
‘Earth’  and  ‘Space’.  The  first  part  of  the  lesson  entitled,  ‘Earth’  would  be  carried  out  in  the 
classroom  as  an  experiment,  while  the  second  part  of  the  lesson  entitled,  ‘Space’,  was  to  be  used  in 
conjunction  with  the  Internet  and  satellite  data.  The  aim  of  this  section  of  the  lesson  was  to  allow  the 
pupils  to  compare  data  gained  from  their  class  experiment  “Earth”  to  the  same  or  similar  types  of 
experiment  carried  out  in  “Space.”  The  data  would  be  accessed  via  the  Internet,  which  would  have  a 
Web-site  designated  to  the  project  containing  all  relevant  data  from  the  satellite.  The  teachers 
regarded  the  approach  as  a particularly  motivating  one  and  were  keen  to  participate  in  the  project. 
The  teacher’s  interest  stemmed  from  their  desire  to  enhance  their  classroom  teaching  through  this 
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innovative  approach  to  delivering  a difficult  part  of  the  Science  National  Curriculum. 

The  schools  that  participated  in  the  research  project  had  certain  commonalties;  these  were  felt  to 
apply  to  schools  throughout  the  country.  They  included  the  need  to  meet  the  requirements  of  the 
National  Curriculum,  had  a divers  spread  of  pupils,  both  in  age  and  ability,  had  access  to  computers, 
the  Internet  and  basic  scientific  equipment. 

The  pilot  schools 

Five  schools  were  selected  for  the  project.  These  schools  were  identified  through  a variety  of  criteria, 
namely:  motivation  and  enthusiasm  for  the  project,  Internet  capacity,  science  expertise  and  science 
teaching  facilities.  Within  these  broad  criteria  a cross-section  of  schools  were  chosen  from  urban  and 
rural  environments.  Three  schools  in  different  London  Local  Education  Authorities  and  two  from 
East  Kent  were  selected.  The  schools  reflected  a wide  spectrum  of  ability  and  range,  as  well  as 
diversity  of  school  type.  These  included  a boys  Grammar  school,  a comprehensive  school,  a 
community  school,  a High  school,  and  a ‘City’  technology  school.  The  school  rolls  were  all  greater 
than  700  pupils. 

The  pilot  teachers 

Within  each  school  a science  teacher  was  nominated  as  co-ordinator  of  the  school-based  project. 
Their  collaborative  role  included  assisting  with  the  development  of  the  teaching  material,  designing 
experiments  and  trialing  pre-developed  lesson  plans.  In  total  6 teachers  and  approximately  360  pupils 
took  part  in  the  initial  stage  of  the  project.  Each  teacher  in  their  respective  schools  taught  five  pre- 
designed and  developed  lesson  plans.  The  material  was  given  to  pupils  ranging  in  age  from  1 1 to  14 
over  a period  of  a month. 

Teacher  A was  working  in  the  boys  Grammar  school.  He  was  an  experienced  specialist  physics 
teacher,  who  used  the  trial  material  with  year  9 pupils,  aiming  at  the  higher  levels  (6  to  8)  within  the 
National  Curriculum  (NC)  assessment  levels.  He  considered  the  material  as  a stepping  stone  for 
further  extension  work,  by  exploring  the  concepts  of ‘satellite  technology’  and  its  implications  in  both 
a qualitative  and  quantitative  manner.  Teacher  B,  an  experienced  specialist  chemist  cross-curricular 
teacher  was  working  in  a community  school,  he  used  the  material  with  Year  7 groups,  extracting 
relevant  aspects  of  the  lessons  in  a qualitative  and  conceptual  manner.  Teacher  C,  an  experienced 
generalist  science  teacher,  teaching  in  a High  school,  followed  the  suggested  lesson  plans  with  a Year 
9 group  and  aimed  at  reaching  the  National  Curriculum  assessment  levels  4 to  5.  The  teacher  was 
also  looking  to  develop  the  existing  lesson  material  further,  with  particular  interest  in  developing 
material  that  would  be  of  value  to  investigative  science.  Teacher  D,  a young  specialist  biologist  cross- 
curricular science  teacher,  working  in  a comprehensive  school  with  low  ability  groups,  (NC  levels 
2/3/4),  followed  the  pre-designed  lesson  plans,  but  had  to  create  suitable  lower  level  exercises  and 
material  to  suit  the  groups.  Teacher  E a less  experienced  generalist  science  teacher,  was  working  in 
an  11  - 16  comprehensive  school  with  Year  9 (NC  levels  4/5)  groups,  followed  the  lessons  as 
suggested  in  the  pre-designed  teaching  pack. 

The  Trial  Period 

The  project  was  designed  to  be  in  three  phases.  This  paper  concentrates  on  Phase  1 the  trial  period. 
This  was  established  to  run  for  a month.  The  teachers  were  encouraged  to  use  the  pre-designed 
lessons  with  their  chosen  year  groups.  They  were  asked  to  collect  work  covered  by  the  pupils,  keep  a 
log  of  the  lessons  they  taught,  and  make  notes  on  any  changes  or  developments  made  to  the  pre- 
designed lessons.  The  research  officer  maintained  contact  throughout  the  trial  period  by  visiting  the 
schools  three  times.  During  this  period  information  was  exchanged,  difficulties  and  successes 
discussed,  monitored  and  recorded. 

At  each  visit  to  the  school  the  teachers  were  interviewed,  (all  of  which  were  taped  for  future 
transcription  and  analysis).  During  the  interviews  they  were  asked  to  comment  on  each  lesson. 
Specific  importance  was  given  to,  the  successes  and  failures  of  the  lesson,  the  pupils  likes  and 
dislikes  towards  the  lesson,  problems  encountered,  both  in  the  classroom  and  with  the  equipment, 
access  to  Information  Technology  (IT),  and  the  relevance  and  suitability  of  the  suggested  NC 
assessment  levels.  The  teachers  also  completed  a questionnaire  after  each  lesson.  This  was  to  gain 
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responses  to  the  lessons  under  specified  headings.  These  included:  the  practicalities  of  the  lesson 
plan,  teacher  reaction  to  the  lesson  plan,  pupil  reaction  and  the  methodology  used  by  the  teacher  to 
deliver  the  particular  lesson,  and  the  extent  to  which  differentiation  was  allowed  for. 

At  the  end  of  the  trial  period  a debriefing  meeting  was  held  to  discuss  the  following  points: 

1.  How  relevant  and  useful  were  the  pre-designed  lessons  in  terms  of  the  science  National 
Curriculum? 

2.  How  was  teaching  and  learning  affected  by  the  use  of  IT? 

3.  What  particular  problems  were  encountered  when  trying  to  teach  using  the  Internet  and 
specified  Web  pages? 

The  pilot  project  outcomes 

The  overall  project  had  two  distinctive  strands,  that  of  curriculum  development,  and  that  of  satellite 
communication  as  a new  tool  for  teaching  science.  For  the  purpose  of  this  paper  the  focus  of  the 
discussion  is  the  teachers  role  in  using  the  lesson  plans  and  developing  strategies  for  using  new 
technologies  in  the  classroom.  The  teachers  themselves  played  an  important  role  in  the  development 
of  the  lesson  plans. 

The  data  collected  and  initial  analysis  of  the  material  showed  two  significant  areas  for  science 
education  within  the  British  context,  these  included: 

• teaching  and  learning  using  ICT  in  science:  this  can  be  subdivided  into  three  issues; 

i.  problems  associated  with  whole  class  use  of  the  Internet 

ii.  the  implications  of  satellite  data  and  Web  navigation  can  have  on  learning; 

iii.  problems  associated  with  differentiated  skills  and  knowledge  levels 

• relevance  of  using  the  Internet  to  the  Science  national  curriculum; 

Teaching  and  learning  using  ICT  in  science. 

All  schools  within  the  project  possessed  Information  Technology  (IT)  capabilities  to  some  degree,  but 
accessibility  varied  greatly  between  the  schools.  One  of  the  schools  had  instant  and  permanent  access 
to  the  Web  directly  from  their  classroom.  Three  schools  needed  to  justify  their  need  for  the  use  of  the 
Web  and  bid  for  regular  access.  The  last  school  had  limited  access;  there  was  one  computer  network 
for  the  whole  school.  This  had  timetabled  time;  the  teacher  had  to  negotiate  use  of  the  network  for 
this  project.  Despite  this  diversity  of  hardware  access,  all  the  schools  were  keen  to  make  use  of  the  IT 
facilities  available  to  them.  All  the  teachers  saw  the  need  to  develop  their  own  pedagogical  skills  in 
relation  to  ICT,  as  well  as  the  possibilities  it  offered  for  the  development  of  pupil’s  use  of  ICT. 

The  teachers  acknowledged  that  the  concept  of  satellite  science  had  been  a significant  motivating 
factor  for  the  pupils,  both  interest  in  science  lessons  and  a will  to  actively  participate  within  these 
lessons  had  been  noted.  Teacher  E commented  that  “ these  lessons  are  much  more  exciting  when  you 
get  the  information  from  the  Internet  instead  of  giving  yet  another  worksheet.  Pulling  data  from  the 
Web  and  using  the  computer  to  help  develop  ideas  is  an  excellent  way  to  motivate  pupils  and  enhance 
their  learning.”  Teacher  A,  commented  that  “the  pupils  all  seemed  very  enthusiastic  about  getting 
involved  with  NASA,  the  Web  and  space  science”.  A general  comment  made  by  the  teachers  was  how 
mundane  experiments  such  as  hot  and  cold,  could  be  made  more  exciting  when  compared  to  similar 
situations  in  space. 

The  motivational  aspect  of  using  a Web-site  devoted  to  satellite  communication  is  without  question, 
however,  the  teachers  did  encounter  some  fundamental  problems  related  to  whole  class  teaching  and 
learning.  The  teacher  interviews  clearly  show  that  the  problems  encountered  fall  into  three  distinct 
areas,  that  of  classroom  management,  differentiated  learning,  and  hardware  failure/support.  The 
following  extract  taken  from  Teacher  C and  Teacher  B’s  interview  highlights  some  of  the  key  issues 
raised  by  the  teachers  generally  when  thinking  about  using  ‘satellite  science’  through  the  Web  as  a 
means  of  teaching  and  learning. 

Researcher:  Could  you  see  satellite  science  drive  science  lessons  forward? 

Teacher  C:  Yes,  very  much,  I got  very  enthusiastic  and  thought  let’s  plan  this  and  discovered  I 

didn’t  have  the  required  equipment  to  conduct  all  the  experiments  in  the  lesson  packs 
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Researcher: 
Teacher  C: 

Researcher: 
Teacher  C: 
Researcher: 
Teacher  C: 

Teacher  B: 


How  do  you  feel  teaching  with  the  Web? 

I think  it  would  be  brilliant,  but  the  only  problem  would  be  getting  access  to  it,  as  the 
rest  of  the  school  wants  access  too! 

Could  you  get  access  once  or  twice  a week? 

If  we  put  our  case  forward. 

Do  you  think  using  the  Web  is  important  to  science  teaching? 

Yes,  its  such  a media  topic  anyway,  kids  are  into  it  and  anything  which  gets  their 
enthusiasm  for  science  is  valuable. 

I’ve  got  access  to  a machine  in  my  room-but  the  Internet  was  not  functioning.  Access  to 
either  NASA  or  the  designated  Web-site  is  definitely  a motivational  factor  for  kids. 


Access,  availability  and  reliability  were  words  continually  used  by  the  teachers  as  reasons  for  not 
using  the  Web.  For  teachers  who  could  not  access  the  Web  from  their  classrooms,  found,  what  could 
be  termed  classroom  management  issues  as  key  to  a successful  lesson.  The  teachers  frequently 
referred  to  lesson  timing  and  planning,  minimising  time  wasting  in  taking  the  pupils  to  the  computer 
centre  or  network  area,  making  sure  pupils  had  logged  on  and  had  the  relevant  Web  pages.  Pupils 
also  required  clear  instructions  on  how  to  effectively  navigate  the  Web-site.  With  classes  of  30+  this 
was  not  easy.  Teacher  B was  most  concerned  with  this  aspect  stating  that:  “I  do  have  worries  in  a 
sense  access  is  unlimited,  you’ve  only  got  to  click  on  the  wrong  icon,  you  can  get  anywhere  and  it  is 
difficult  to  get  back  and  so  lose  focus.  As  I have  said  before,  access,  network  crashes,  lack  of  up  to 
speed  modems  can  all  be  demotivating  factors  for  pupils,  as  well  as  time  wasting  for  the  teacher.  It’s 
the  management  of  the  practicalities  that  are  worrying.” 


Pupils  in  most  schools  worked  in  pairs  or  triads,  depending  on  the  availability  of  machines.  Teacher 
B highlighted  what  all  the  teachers  thought  in  his  comment  that,  “Two  pupils  per  machine  for  8-10 
minutes  leads  to  problems  of  practicality,  BUT  a few  on  a machine  is  better  than  none.  Once  on  the 
Web-site  pupils  were  expected  to  find  the  data  sets  required  for  their  specified  experiment. 
Downloading  data,  saving  and  using  data  was  for  some  teachers  the  most  demanding  part  of  the 
Web-site  space  element  of  the  lessons.  All  commented  on  the  different  levels  of  skills  and  knowledge 
pupils  had  regarding  both  scientific  knowledge  and  ICT  skills.  Teacher  D suggested  that  time 
allowed  for  computer  use  in  her  school  was  insufficient  for  this  type  of  work.  She  stated  that  “I’ve 
been  teaching  about  satellites  and  this  work  consolidates  what  I’ve  been  doing.  Fits  nicely  to  the 
scheme,  BUT  I’ve  got  to  finish  this  unit.  We  need  at  least  45  minutes  on  the  computer,  this  isn’t 
always  possible  or  practical.”  Pressure  of  fulfilling  the  demands  of  the  national  curriculum 
requirements  for  science  take  precedence  over  extended  use  of  the  Internet  or  IT  alternatives”. 


The  second  point  raised  by  the  teachers  related  to  “differentiated  learning”  based  on  two  variables, 
one  being  the  pupils  ICT  capabilities  and  secondly  the  pupils  understanding  of  the  scientific  concepts 
being  explored.  On  the  one  hand  they  thought  that  the  ICT  component  was  a great  way  to  help 
learners  of  all  abilities,  as  pupils  could  work  at  their  own  pace.  Yet,  on  the  other  hand  they  reported 
that  the  differing  abilities  of  the  pupils,  both  scientifically  and  with  respect  to  ICT,  gave  the  teachers 
problems  in  managing  the  pupils’  learning,  and  directing  their  own  teaching  of  these  sessions.  What 
is  interesting  about  this  assertion  is  that  the  teachers  perceived  these  two  areas  as  classroom 
management  issues,  rather  than  issues  related  to  pedagogic  practice.  Although,  the  teachers  were 
concerned  about  levels  of  learning  in  both  science  knowledge  and  ICT  skills,  their  responses  were 
clearly  demonstrating  a concern  that  related  to  managing  the  learning  situation  as  opposed  to  how 
they  should  or  could  change  their  teaching  style  to  accommodate  a different  tool  for  learning.  In  this 
case  the  computer  and  the  Internet. 


Teacher  A was  clear  about  the  problems  that  could  occur  from  a combination  of  complex  scientific 
concepts,  ICT  skills  and  teacher  interaction  with  pupils.  He  suggested  that  “what  you  want  is  to 
stimulate  the  kids  with  this  {space  science) , and  this  will  stimulate  them.  You  will  want  to  be  able  to 
perpetuate  that  stimulation  and  I can  see  that  with  some  teachers  it  could  go  over  their  heads.  You 
will  get  pupils  going  down  to  the  Internet  and  finding  out  information  and  coming  back  and  saying, 
‘what  about  this  and  that’,  those  teachers  who  are  not  switched  on  to  space  and  satellites  would  have 
to  manage  the  situation  carefully.  Flexibility  is  the  key,  but  depends  on  the  teacher’s  approach” 


When  the  teachers  were  asked  about  teaching  with  IT,  and  to  comment  on  the  practicalities  of  using 
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ICT  skills  as  a means  to  improving  and  extending  individual  pupil’s  knowledge  base,  such  as  graph 
work,  the  following  responses  were  recorded  (all  comments  related  to  the  solar  cell  experiment,  fig 


1). 

Teacher  E: 


Researcher: 
Teacher  E: 

Teacher  D: 


Researcher: 
Teacher  D: 

Teacher  C: 

Researcher: 
Teacher  C: 


Researcher: 
Teacher  C: 


Teacher  A: 


Researcher: 
Teacher  A: 


Teacher  B: 

Researcher: 
Teacher  B: 


Following  this  experiment  the  pupils  have  to  plot  a graph,  both  for  the  classroom-based 
experiment  and  the  satellite  based  experiment.  It  didn’t  take  them  long  to  do  the 
practical  side  of  the  experiment.  Most  of  the  practical  was  taken  up  with  them  doing 
the  graph.  That  took  a good  15-20  minutes. 

Would  you  have  liked  to  try  doing  the  graphs  on  the  computer? 

No,  I think  that  would  have  confused  them.  They  can  do  graph  drawing  on  the 
computer,  but  it  takes  an  awful  lot  longer  than  if  they’d  done  it  by  hand. 

The  practical  was  fine,  but  I had  to  sketch  out  the  axes  for  them  (pupils),  as  that  takes  a 
long  time.  I gave  them  one  of  the  graphs  and  they  plotted  it  then  joined  up  the  dots. 
They  enjoyed  that  and  are  getting  more  skills,  which  they  need  throughout  the  science 
curriculum,  but  this  section  did  take  15-20  minutes. 

Would  you  have  liked  to  try  doing  the  graphs  on  the  computer? 

No,  they  would  find  it  too  difficult  and  it  would  take  up  much  more  time,  which  we  do 
not  have. 

The  kids  actually  had  a go  at  the  practical.  It  was  fine.  They  enjoyed  it  so  that  was 
good. 

Plotting  the  graph,  was  that  a successful  exercise? 

Yes,  but,  if  you  are  going  to  give  them  the  axes,  it  lowers  the  attainment  level.  If  you’re 
aiming  at  a higher  level,  then  they  (the  pupils)  should  think  about  it  themselves.  The 
concept  is  good. 

Would  you  have  liked  to  try  doing  the  graphs  on  the  computer? 

Well,  yes  and  no.  If  I had  unlimited  access  yes,  it  would  be  good  way  of  differentiating 
and  stretching  the  able  pupils.  No  for  the  less  able  as  they  would  struggle.  Time  would 
not  allow  for  it.  It  is  best  managed  as  a whole  class  activity  on  the  board.  This  way  also 
allows  me  to  quickly  see  who  can  or  can’t  do  the  graph. 

What  you  can  get  out  of  this  experiment  with  Year  9 kids  and  our  more  able  kids  is  that 
they  can  use  the  graph  to  calculate  a variety  of  variables.  The  very  able  were  looking  for 
patterns  of  results  and  so  on.  Our  year  9 are  just  about  into  Y=  MX+C,  into  a straight 
line  and  inverse  relationships  and  we  relate  that  back  to  maths. 

Would  you  have  liked  to  try  doing  the  graphs  on  the  computer? 

That’s  a difficult  one.  No  not  in  science  lesson  time.  I would  expect  that  type  of 
exercise  in  maths  lessons.  We  do  expect  computer-plotted  graphs  in  science,  but  the 
pupils  do  these  in  their  own  time. 

I did  this  with  Year  7.  My  worry  was  whether  we  could  graph  the  results.  Graphs  with 
year  7 are  always  useful. 

Would  you  have  liked  to  try  doing  the  graphs  on  the  computer? 

Not  with  Year  7,  possibly  with  year  9.  My  only  worry  is  staff  would  need  to  have  it  all 
mapped  out  beforehand,  access,  network  crashes,  etc.  don’t  bode  well  to  using  science 
time  for  such  activities. 


These  comments  clearly  demonstrate  the  problems  the  science  teacher  faced,  in  terms  of  differentiated 
teaching  and  learning  environments.  All  the  teachers  highlighted  the  issue  of  “time”  for  science.  The 
pressure  to  fulfil  the  requirements  of  the  National  Curriculum  is  clearly  stated.  It  is  also  clear  that  the 
teachers  felt  that  ICT  was  a motivating  issue  in  teaching  “satellite  science”,  but  some  of  the  associated 
problems,  such  as  drawing  graphs,  was  best  taught  by  the  teacher.  Direct  teacher  input  was  considered 
a more  effective  means  of  facilitating  individual  learning.  It  is  also  interesting  to  note  that  the  only 
references  to  pedagogic  practice  as  related  to  IT  surrounds  the  issue  of  time  management,  and  not 
what  the  new  technology  learning  tool  might  offer  the  teaching  environment.  This  view  is  not  new 
and  has  been  reported  in  many  academic  papers,  conferences  and  books  (McAteer,  E.  1998,  Wegrif,R. 
1998,  Gilliver,  R.S.  1998).  What  is  important  to  the  educational  context  is  that  despite  there  having 
been  great  leaps  in  both  hardware  and  software  teachers  use  of  new  technologies  is  constrained  and 
not  perceived  as  part  of  their  pedagogic  practice. 

Relevance  of  using  the  Internet  to  the  Science  National  Curriculum. 
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Use  of  the  Internet  and  science  education  was  considered  very  important  to  the  pupils  developing 
research  skills  and  meeting  the  ICT  components  of  the  National  Curriculum.  However,  teachers  were 
very  clear  about  the  implication  of  teaching/leaming  time  in  science,  and  the  way  the  Internet  use 
eroded  this  time.  Accessing  information  had  to  be  a definite  part  of  the  science  lesson,  as  with  the 
solar  cell,  and  data  being  accessed  directly  form  the  Web  to  create  graphs  of  light  intensity.  As 
Teacher  E commented  computer/Intemet  drove  the  lesson  along. 

Researcher:  Is  ‘satellite  science’  a useful  concept? 

Don’t  know! 

Has  it  a place  in  the  National  Curriculum? 

We  already  have  a National  Curriculum. 

Do  you  like  the  concept  of ‘satellite  science’? 

Yes,  but  this  topic  isn’t  entirely  relevant  in  the  National  Curriculum.  They  (pupils) 
won’t  be  asked  about  it  in  their  Standard  Attainment  tests  (SATs),  but  the  skills  that  it 
brings  out,  graph  skills  would  be. 

Do  you  like  the  concept  of ‘satellite  science’? 

Yes,  as  a motivating  aspect  to  science. 

Has  it  a place  in  the  National  Curriculum? 

Yes,  but  it  is  not  that  easy.  The  Curriculum  prescribes  what  we  teach  and  at  present, 
this  is  not  in  the  Curriculum,  but  I see  it  as  strength  that  pupils  can  access  NASH  and 
designated  Web-sites.  The  strength  of  the  Web  is  the  flexibility  of  movement.  A real 
timetable  of  constant  data,  a satellite  downloading  data  would  captivate  pupil’s 
imagination  for  this  reason  it  should  be  included. 


Teacher  E: 
Researcher, 
Teacher  E: 
Researcher, 
Teacher  D: 


Researcher: 
Teacher  B: 
Researcher: 
Teacher  B: 


Teacher  A and  C consolidate  the  above  views  that  the  Web  aspect  of  satellite  science  would  be  an 
excellent  way  for  pupils  to  investigate  and  download  their  own  information  and  experimental  data. 
They  went  on  to  suggest  that  ‘satellite  science’  would  add  a totally  different  dimension  to 
‘Investigative  Science’  and  Attainment  Target  1 of  the  National  Curriculum,  at  all  levels. 


Concluding  Comments 

This  paper  has  reported  on  the  first  phase  of  a multifaceted  research  project  that  is  looking  at  the 
concept  of  satellite  science  as  a means  of  motivating  students’  interest  in  science.  The  context  has 
been  the  National  Curriculum  for  Science  in  England  and  the  use  of  ICT.  What  is  important  to  the 
report  is  that  despite  the  will  to  use  new  technology  as  a form  of  pedagogic  practice,  teachers  feel 
external  pressures  constrain  their  use  of  these  technologies.  However,  pupils  were  motivated  and  did 
produce  innovative  and  constructive  work.  The  development  of  the  lessons  by  the  teachers  showed  the 
need  to  have  teachers  at  the  forefront  of  curriculum  development. 

The  second  phase  of  the  project  is  extending  the  number  of  participant  schools  and  teachers.  It  is  also 
going  to  look  at  other  areas  of  the  National  Curriculum  to  see  how  satellite  science  can  enhance  the 
quality  of  student  learning. 
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Abstract:  The  increasing  accessibility  of  the  Internet  as  a tool  for  the  delivery  of  educational 
content  and  interaction,  combined  with  its  importance  as  a source  of  information  for 
researchers  provided  the  rationale  for  the  project  described  in  this  paper.  The  paper  reports  on 
the  development  of  a web  site  to  support  the  teaching  of  research  skills  to  postgraduate 
students  in  art  and  design.  The  model  of  development  adopted  by  the  project  is  presented  and 
discussed,  with  a particular  focus  on  three  issues:  the  revision  of  existing  content  for  web- 
based  delivery;  the  development  of  a web-based  form  of  presentation;  and  the  consideration  of 
how  to  structure  students’  use  of  the  resource.  In  conclusion  the  paper  looks  at  the  piloting 
and  evaluation  of  the  site  and  areas  for  future  development. 


Introduction 

Postgraduate  education  in  art  and  design,  in  common  with  many  other  areas  of  postgraduate  study  is  undergoing 
a period  of  considerable  change.  Three  factors  are  of  particular  importance  to  the  resource  development  project 
discussed  in  this  paper  [1]: 

• the  increase  in  the  number  of  postgraduates  and  postgraduate  courses  (this  has  been  a general  trend 
affecting  most  subject  areas  in  higher  education) 

• the  question  of  what  constitutes  research  in  art  and  design,  and  what  therefore  might  be  considered  an 
appropriate  body  of  research  skills 

• the  significance  new  developments  such  as  the  Internet  for  teaching,  learning,  and  research 

Art  and  design,  like  many  other  areas  of  postgraduate  education,  has  experienced  a considerable  increase  in  the 
number  of  students  attending  taught  courses  and  registering  for  higher  degrees,  both  in  real  terms  and  in 
proportion  to  undergraduates.  The  recent  Review  of  Postgraduate  Education  in  the  United  Kingdom  gives  an 
indication  of  the  extent  of  this  change: 

“In  1979,  for  example,  there  were  100,900  postgraduate  students  in  the  UK 
compared  with  a total  student  population  of  787,000  (13%),  whereas  in  1994-5,  the 
comparable  proportion  was  21%  (315,400  and  1,538,600  respectively).”  (HEFCE 
1996) 

This  general  trend  towards  greater  student  numbers  at  postgraduate  level  has  been  accompanied  by  two  further 
developments.  Firstly,  there  has  been  a lively  international  debate  concerning  the  nature  of  research  in  practical 
and  creative  subjects  such  as  art  and  design  [2].  This  has  in  part  been  driven  by  the  increasing  number  of 
students  wanting  to  progress  to  doctoral  work  in  these  areas,  but  also  represents  some  interesting  debates  about 
the  nature  of  knowledge  and  enquiry  in  fields  of  creative  and  practical  endeavour.  Although  this  area  of 
discussion  is  beyond  the  scope  of  this  paper,  the  consequent  importance  accorded  to  the  teaching  and  learning 


[1]  The  development  of  the  research  training  web  site  forms  part  of  a broader  project  - the  Research  Training  Initiative  - 
initiated  in  1995  to  develop  and  publish  research  training  resources  for  postgraduates  in  art  and  design  (see  Newbury  1996). 
To  date  the  project  has  published  seven  printed  research  guides  (all  but  one  of  which  are  now  out  of  print)  and  a CD-Rom  of 
case  studies  of  masters  projects  in  jewellery  and  silversmithing  (Newbury  1997).  Further  information  about  the  project  can 
be  found  at  http://www.biad.uce.ac.uk/research/index.html. 

[2]  See  for  example  No  Guru,  No  conference  proceedings  (Strandman  1998),  Gray  1995,  UKCGE  1997. 
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of  research  skills  has  been  significant  for  this  project.  Secondly,  the  increasing  accessibility  of  the  Internet  is 
having  a significant  impact  on  the  way  in  which  staff  in  higher  education  think  about  teaching  and  learning  [3]. 

Although  electronic  resources  are  often  put  forward  as  an  efficient  solution  to  the  problems  raised  by  increased 
student  numbers  without  a comparable  increase  in  teaching  staff,  the  majority  of  work  to  date  has  been  in 
relation  to  undergraduate  (often  modular)  programmes.  A brief  review  of  resources  currently  available  suggests 
that  relatively  little  of  the  work  in  this  area  has  been  at  a postgraduate  level  [4].  The  emphasis  on  the  world 
wide  web  and  CD-ROMs  as  virtual  storehouses  of  knowledge  and  information  has  tended  to  favour  subjects 
with  large,  well  structured  and  relatively  stable  knowledge  bases,  rather  than  practice-based  subjects  such  as  art 
and  design  where  much  of  the  subject  knowledge  has  not  been  committed  to  textbooks.  Many  computer-based 
learning  packages  also  have  a very  directive  style  and  are  therefore  less  suited  to  the  more  open  investigative 
learning  appropriate  to  postgraduate  level  study.  This  is  perhaps  slightly  ironic  in  that  the  Internet  and  the 
world  wide  web  were  originally  developed  by  and  for  researchers.  As  Hobbs  and  Taylor  point  out: 

“Researchers  are  still  some  of  the  keenest  users  of  the  WWW  in  view  of  its  potential 
for  contact  and  collaboration,  for  disseminating  research  findings,  and  for  facilitating 
peer  review  of  the  outcomes  of  research.”  (Hobbs  & Taylor  1996) 

The  development  of  the  research  training  web  site  discussed  in  this  paper  is  a response  to  this  context.  An 
important  difference  between  undergraduate  and  postgraduate  study  is  that  the  postgraduate  student  must  learn 
to  find  and  manage  the  information  that  supports  his  or  her  own  learning.  Postgraduate  students  beginning 
research  for  their  dissertation  or  major  project  are  confronted  by  a complex  landscape  of  information  sources.  It 
is  interesting  that  what  can  be  considered  a disadvantage  of  the  world  wide  web  for  undergraduate  students  - 
the  lack  of  any  clear  narrative  which  structures  the  learning  process  (Laurillard  1995)  - has  advantages  for 
research  students  - the  direct  access  it  gives  to  sources,  databases,  other  researchers  and  even  research  subjects 
(Foster  1 994).  However,  it  was  novice  rather  than  experienced  researchers  that  were  the  focus  of  this  project. 
It  was  therefore  important  that  the  material  we  produced  supported  students’  independent  learning  and  research, 
as  they  began  to  deal  with  a complex  range  of  sources  from  which  to  retrieve  information.  Our  proposal 
therefore  was  to  take  as  a starting  point  material  already  available  on  the  collection  and  management  of 
information  for  research,  and  to  develop  from  this  a web-based  interactive  environment  which  would  support 
the  learning  of  a number  of  key  research  skills. 


A Model  of  Web  Resource  Development 

New  educational  technologies  have  strengthened  the  trend  towards  specialisation  in  the  development  of 
teaching  and  learning.  In  addition  to  subject  knowledge,  two  specialised  domains  of  knowledge  are  emerging. 
Firstly,  knowledge  of  teaching  and  learning  processes.  As  Laurillard  points  out,  innovative  teaching  methods, 
by  which  she  means  those  that  depart  significantly  from  the  narrative  or  telling  mode,  “require  more 
pedagogical  understanding  of  the  professional  teacher”  (Laurillard  1995,  p.l83).  Secondly,  specialised 
knowledge  in  the  design  (in  its  broadest  sense)  of  computer-based  teaching  and  learning  materials.  There  is  an 
emerging  literature  on  models  of  developing  electronic  resources,  which  attests  to  the  novel  and  still 
problematic  nature  of  this  kind  of  resource  development  (Borkowski  et  al.  1996;  D’Halluin  et  al.  1996;  Porritt 
1997).  Porritt,  for  example,  identifies  a further  degree  of  specialisation.  He  outlines  four  roles  - student;  tutor; 
author;  designer  - and  considers  the  different  models  of  interaction  and  overlap.  These  range  from  the  “tutor 
designer”  within  which  the  tutor  takes  on  the  additional  roles  of  author  and  designer,  to  “open  authoring”  within 
which  resource  development  is  carried  out  by  teams  made  up  of  specialist  tutors,  authors  and  designers. 
Although  these  specialist  areas  are  separable  in  theory  it  is  often  the  case  that  the  discussion  of  computer-based 


[3]  When  we  originally  surveyed  our  higher  degree  students  prior  to  developing  the  training  resources  they  indicated  that 
electronic  resources  were  among  the  least  flexible  and  desirable  forms  of  delivery.  Although  this  situation  has  changed, 
many  more  students  now  have  Internet  access,  its  appropriateness  needs  to  be  considered  on  a case  by  case  basis.  On  a 
larger  scale  the  question  of  the  accessibility  of  web-based  course  material  has  been  an  issue  for  the  Open  University  where 
the  development  of  innovative  distance  learning  materials  has  had  to  be  matched  with  its  policy  of  accessibility. 

[4]  There  are  of  course  some  exceptions  to  this  general  observation.  See  for  example  Jennings  1997  and  Lidstone  & Lucas 
1998, 
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learning  has  become  part  of  a broader  debate  around  innovations  in  teaching  and  learning.  In  some  instances 
computer-based  learning  developments  are  seen  as  a vehicle  for  introducing  changes  in  teaching  practice  that 
are  in  themselves  considered  desirable.  Underlying  this  is  the  view,  perhaps  not  always  justified,  that  there  is 
an  inherent  compatibility  between  computers  and  certain  philosophies  of  education. 

This  developing  context  has  given  rise  to  a range  of  approaches  to  the  use  of  the  Internet  and  multimedia 
technologies  for  teaching  and  learning,  from  small  local  developments  by  enthusiastic  teaching  staff  to  large 
scale  centralised  funding  [5].  The  development  of  this  project  drew  on  a number  of  extant  models  and  had  the 
following  five  features: 

• Specialised  content  development.  The  project  was  a response  to  developments  in  the  subject  area  as  well 
as  to  the  introduction  of  the  Internet  as  a tool  for  teaching  and  learning.  As  a result  we  had  produced  a 
range  of  new  material  on  research  in  art  and  design  during  the  three  years  of  the  project.  It  is  also  worth 
noting  that  the  print  format  we  adopted  - a series  of  research  guides  rather  than  a sequential  programme  or 
single  publication  - provided  a degree  of  flexibility  that  proved  useful  in  its  conversion  to  electronic  format. 

• Central  resourcing  at  a faculty  level.  A degree  of  central  resourcing  meant  that  the  work  was  not  additional 
burden  for  already  busy  teaching  staff  and  therefore  enabled  the  development  to  get  off  the  ground. 
However,  it  was  important  that  the  resource  retained  a subject  identity  given  the  relative  novelty  of 
research  study  in  art  and  design  (although  much  of  the  material  had  generic  applicability).  Unlike  a 
number  of  other  universities,  which  have  adopted  a university  wide  approach  to  research  training,  our 
initial  audience  was  specifically  art  and  design  students. 

• Core  team  responsible  for  the  resource  development.  Importantly,  however,  the  main  member  of  staff 
working  on  the  project  was  also  delivering  teaching  in  research  methods  on  a number  of  courses.  This 
enabled  the  resource  to  be  developed  in  a way  that  was  responsive  to  student  and  tutor  needs.  The  use  of 
an  intranet/intemet  based  resource  offered  an  opportunity  to  supplement  existing  classroom  teaching  with 
resources  that  students  could  access  at  times  when  it  was  most  needed,  principally  during  the  period  of 
independent  research  study. 

• Skills  development.  Although  it  is  possible  to  consider  teaching  staff  simply  as  “infomiation  owners” 
(Borkowski  et  al.  1996),  with  content  handed  over  to  a specialist  team  for  the  creation  of  web  pages,  this 
represents  only  a base  level  model  of  development.  In  this  project  the  member  of  staff  responsible  for 
content  development  was  also  responsible  for  the  overall  design  and  navigation  of  the  site.  This 
necessitated  learning  to  use  basic  web  page  software.  Although  the  final  screen  design  and  authoring  was 
not  done  this  way,  the  drafting  of  the  pages  and  site  architecture  in  web  format  made  it  possible  to  think 
through  important  issues  about  how  students  would  use  the  information.  It  also  highlighted  some  basic 
issues  such  as  the  need  to  carefully  structure  large  amounts  of  text  to  be  accessible  on  screen. 

• Professional  design  input.  The  final  stage  of  the  project  involved  working  with  a professional  web  design 
team.  Although  design  in  its  broadest  sense  was  set  largely  at  an  earlier  stage,  the  design  of  the  pages  on 
screen  has  the  potential  to  either  enhance  or  detract  from  the  overall  structure.  Given  that  the  project  was 
situated  within  a faculty  of  art  and  design  and  that  it  was  for  use  by  art  and  design  students,  it  was 
considered  important  that  the  finished  site  should  be  visual  attractive,  pleasant  to  use  and  functional.  This 
stage  enabled  us  to  make  use  of  some  features  of  advanced  web  page  authoring  which  would  have 
otherwise  been  beyond  the  skills  of  the  team. 

One  of  the  key  principles  that  emerged  from  the  model  of  development  used  in  this  project  was  the  importance 
of  a close  integration  between  the  different  stages  in  the  process,  from  content  generation  to  screen  design  to  the 
placing  of  material  within  the  curriculum. 


[5]  For  example,  in  the  United  Kingdom  the  Teaching  and  Learning  Technology  Programme  (TLTP)  has  invested  a total  of 
£37  million  since  1992  in  the  development  and  implementation  of  technology  based  learning.  Many  of  the  projects  has 
been  at  an  institutional  or  inter-institutional  level. 
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Figure  1:  Map  of  pilot  website  contents 


Contexts  for  Electronic  Learning 

One  of  the  dangers  of  a specialised  production  model  of  electronic  resource  development  is  the  failure  to  give 
sufficient  consideration  to  the  contexts  into  which  materials  are  being  introduced.  The  emphasis  is  placed  on 
the  knowledge  content  at  the  expense  of  the  processes  of  teaching  and  learning.  As  within  any  form  of  teaching 
there  is  a need  “to  change  information  exchanges  into  learning  activities”  (D’Halluin  et  al.  1996).  In  order  to 
avoid  the  pitfall  of  offering  students  content  without  an  appropriate  learning  context  or  framework,  we 
incorporated  two  additional  features  into  the  web  site:  a discussion  forum;  and  guided  learning  activities. 

The  discussion  forum  serves  three  purposes.  Firstly,  it  provides  for  student  interaction  both  within  and  across 
sites  (the  piloting  of  the  resource  is  taking  place  across  several  universities).  It  is  hoped  that  this  will  enable 
students  to  share  problems  and  solutions,  and  disseminate  useful  information.  Secondly,  a number  of  the 
learning  activities  invite  students  to  post  short  written  pieces  to  the  forum,  for  example  reviews  of  particularly 
good  or  bad  web  sites.  This  is  one  way  of  prompting  students  to  think  about  writing  for  a semi-public  audience. 
It  is  envisaged  that  this  may  be  appropriate  as  a form  of  course  work,  with  the  archived  messages  providing  an 
easy  means  of  assessing  students’  contributions.  Models  of  practice  in  this  area  already  exist,  though  largely  at 
undergraduate  level.  Thirdly,  electronic  discussion  lists  are  now  an  important  form  of  academic  research 
networking  (Berge  & Collins  1995).  It  is  important  therefore  that  research  students  are  introduced  to  this  form 
of  communication,  its  benefits  and  limitations.  Although  advances  in  Internet  technology  mean  that  video 
conferencing  is  becoming  increasingly  viable,  the  move  towards  the  “conversational  ideal”  (Jones  1995)  in 
electronic  communication  should  be  subject  to  scrutiny.  The  dynamics  of  asynchronous  email  communication 
have  some  useful  advantages  for  both  for  research,  and  for  teaching  and  learning.  For  example,  as  something  of 
a hybrid  between  written  and  spoken  forms  of  communication,  email  can  offer  the  opportunity  for  considered 
responses  without  introducing  the  formality  of  written  essays.  It  can  also  reduce  some  of  the  inhibitions  to 
participation  that  exist  in  face  to  face  encounters  (though  of  course  it  does  introduce  new  problems  of  its  own). 

The  learning  activities  on  the  site  have  been  designed  to  facilitate  students’  use  of  the  content  - there  are  cross 
links  between  activities  and  relevant  sections  of  the  electronic  research  guides  - and  also  to  help  students  to 
develop  strategies  for  gathering  information  relevant  to  their  research  topic.  The  site  has  been  constructed  to 
serve  a number  of  different  student  groups,  and  at  this  stage  is  a supplement  to  existing  teaching  rather  than  a 
compulsory  element.  This  will  obviously  have  a bearing  on  students’  use  of  the  material  and  is  something  we 
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have  had  to  consider  carefully.  As  a result  the  learning  activities  try  to  fit  with  tasks  that  students  will  need  to 
do  anyway  in  order  to  complete  an  independent  research  project.  It  is  important  that  the  activities  are  optional, 
as  they  will  not  be  equally  appropriate  to  all  student  projects.  Also  some  students’  prior  knowledge  may  mean 
that  completion  of  a particular  activity  is  unlikely  to  be  relevant.  For  example,  for  those  students  already 
familiar  with  the  use  of  email  discussion  lists  the  activity  related  to  this  may  be  redundant. 

These  features  are  intended  to  provide  a learning  context  that  supplements  the  traditional  class  or  seminar  group 
and  guides  students’  use  of  electronic  sources  and  the  Internet,  which  despite  claims  to  the  contrary  can  be  a 
sparse  and  isolating  learning  environment. 


Evaluation:  Issues  and  Methodology 

The  evaluation  of  web-based  resources  is  key  to  the  future  development  of  this  area  of  work.  The  number  of 
new  initiatives  in  web-based  learning  raise  issues  both  of  comparison  and  also  of  the  sharing  of  good  practice. 
The  evaluation  of  the  web  site  in  its  pilot  form  will  consider  several  aspects: 

• Navigation  and  interface  design  - the  ease  with  which  students  were  able  find  their  way  around  and  use  the 
materials. 

• Interaction  - the  quality  of  the  interaction  in  relation  to  the  materials  and  the  suitability  of  the  discussion 
forum  facility  for  this  purpose.  The  evaluation  will  also  consider  what  factors  may  affect  the  success  or 
failure  of  this  aspect  of  the  site. 

• Added  value  - what  the  web-based  materials  contributed  to  teaching  and  learning,  and  the  appropriateness 
of  using  the  web. 

• Context  - the  value  and  appropriateness  of  the  context  in  which  the  materials  were  introduced  and  the  way 
in  which  students’  use  of  the  materials  was  structured  (for  example  through  the  guided  learning  activities) 

Although  it  is  our  intention  to  monitor  the  use  of  the  web  pages  quantitatively,  we  are  principally  concerned  at 
this  stage  with  the  quality  of  the  interaction  with  the  materials  within  a relatively  small  user  group.  The 
methodological  emphasis  is  therefore  on  the  students’  experience  of  using  the  materials  and  tutors’  perceptions 
of  the  value  it  added  to  traditional  teaching  methods  in  this  area.  The  evaluation  methodology  will  therefore 
incorporate  the  following  elements: 

• Focus  group  discussion  with  small  groups  of  students 

• Collation  and  analysis  of  discussion  forum  interaction 

• Interviews  with  tutors 

• Expert  reviews 

Although  we  intend  the  material  to  support  students  during  a relatively  long  period  of  independent  research 
study,  in  order  to  get  some  timely  feedback  we  will  carry  out  the  piloting  over  one  15  week  semester  period. 
Several  small  groups  of  students  will  be  introduced  to  the  resource  at  the  beginning  of  the  semester.  They  will 
then  have  access  to  the  resource  throughout  the  period,  with  the  main  evaluation  work  taking  place  towards  the 
end.  The  pilot  groups  cover  both  taught  masters  and  research  students  across  a range  of  art,  design  and  media 
subjects.  This  will  give  some  variation  in  the  degree  to  which  students’  use  of  the  resource  will  be  guided,  and 
hopefully  provide  some  indicators  as  to  what  different  students  might  want  from  such  a resource.  In  addition  a 
small  number  of  expert  reviews  will  be  commissioned  in  order  to  subject  the  material  to  independent  scrutiny, 
and  give  some  measure  of  how  this  work  relates  to  the  current  ‘state  of  the  art’. 


Conclusion  and  Future  Directions 

The  web  site  is  at  a pilot  stage,  and  it  is  difficult  to  come  to  any  specific  conclusions  before  the  process  of 
piloting  and  evaluation  is  complete.  However,  it  is  worth  venturing  two  general  thoughts.  Firstly,  it  seems  that 
to  some  extent  the  project  is  representative  of  interim  work  in  this  area.  There  is  currently  a great  deal  of 
unevenness  in  postgraduate  students’  prior  knowledge  and  experience  of  the  internet.  Some  will  of  course  be 
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highly  proficient  and  experienced  users,  others  however  will  have  little  or  no  knowledge  and  experience  other 
than  that  gleaned  from  popular  representations.  However,  over  time  this  is  likely  to  change  dramatically  and 
resource  developers  will  need  to  respond  to  the  needs  of  a more  sophisticated  group  of  users.  A similar  point 
can  be  made  in  relation  to  staff  The  development  of  web-based  materials  is  new  to  most  academic  staff  outside 
of  computing  and  information  science.  It  therefore  makes  sense  to  provide  additional  support  in  this  area. 
However,  in  the  long  term  the  specialised  model  of  resource  production  this  implies  has  potential  dangers.  As  I 
have  argued,  students  and  academic  subjects  are  likely  to  be  served  better  where  there  is  a close  integration 
between  subject  knowledge  and  tools  for  teaching  and  learning.  It  seems  obvious  that  a subject  tutor  needs  to 
understand  how  the  lecture  or  the  seminar  works  in  order  to  teach.  It  should  be  no  less  obvious  that  they  need 
to  know  how  their  students  make  sense  of  electronically  based  resources.  Though  as  has  been  noted  this  does 
place  extra  demands,  both  intellectually  and  in  terms  of  time,  on  staff. 


Coda 

One  area  not  discussed  in  this  paper  but  important  to  the  philosophy  of  the  project  is  the  use  of  case  study 
material  as  resource  for  teaching  and  learning.  One  of  the  printed  research  guides  was  given  over  to  a series  of 
case  studies  of  completed  MPhil  and  PhD  projects  in  art  and  design.  An  important  area  for  the  future 
development  of  this  project  is  the  introduction  of  case  study  material  on  to  the  web  site.  The  use  of  web-based 
materials  offers  an  opportunity  and  a challenge,  to  develop  multimedia  forms  of  the  case  study  approach. 
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Abstract:  This  paper  outlines  six  basic  evaluation  designs.  For  each,  the  paper 
describes  the  type  of  technology  application  for  which  it  is  appropriate,  and  the 
questions  the  evaluation  design  can  answer,  and  summarizes  the  requirements, 
data  collection  and  analysis  procedures.  The  six  evaluation  designs  are: 

• Mastery-Based  Program  Effectiveness 

• Non-Mastery-Based  Program  Effectiveness 

• Affective  Outcomes 

• Case  Studies  / Qualitative  Assessments 

• Program  Comparison 

• Cost  Comparisons 

A more  lengthy  version  of  this  paper  includes  details  on  the  requirements  for 
successful  evaluation,  the  procedures  for  data  gathering,  and  the  procedures  for 
data  analysis,  and  guidelines  for  writing  the  evaluation  report.  It  is  available  at 
http://www.plato.com,  or  from  the  author. 


Introduction 

It  is  sound  educational  practice  to  evaluate  the  effectiveness  of  any  instructional  program,  including 
technology-based  implementations.  But  ifs  important  to  use  an  evaluation  design  that  is  appropriate  to  the 
way  in  which  you  are  using  the  technology,  and  to  your  needs  for  decision-making.  When  evaluations  are 
inconclusive,  ifs  often  the  case  that  an  inappropriate  evaluation  design  was  used,  and  the  desired  result 
could  not  have  emerged  from  the  evaluation. 

To  decide  what  kind  of  evaluation  to  do,  you  must  first  know  who  will  use  the  evaluation  results  and  what 
decisions  they  need  to  make.  You  can  then  combine  and  adapt  these  basic  designs  to  meet  your  program’s 
needs. 

Which  Model  Do  I Want? 

Here  is  a brief  summary  of  the  programs  to  which  the  six  models  often  apply.  For  a more  complete 
description,  refer  to  Foshay  (1994). 

Mastery-Based  Program  Effectiveness 

In  programs  suitable  for  this  model,  the  technology  provides  the  primary  means  of  initial  instruction,  and 
learners  are  managed  so  that  learners  begin  a learning  activity  only  when  they  are  ready  for  it,  and  they 
continue  to  study  the  activity  until  they  have  fiilly  mastered  it.  Thus,  in  contrast  to  a conventional 
instructor-led  classroom,  achievement  is  fixed  but  instructional  time  varies.  As  an  example,  mastery-based 
instruction  is  typical  of  GED  programs. 

Questions 

Use  this  design  when  you  want  to  find  out: 

• How  well  are  the  learners  progressing  due  to  the  technology? 

• What  is  the  longest,  shortest,  and  average  time  to  mastery? 

• What  proportion  of  learners  master  the  skills  taught  at  the  required  level? 
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• What  proportion  of  learners  do  not  complete  or  drop  out? 

If  the  technology  is  the  primary  instructional  mode  in  a mastery-based  program,  learning  outcomes  can  be 
attributed  mostly  to  the  technology. 

In  addition  to  these  questions,  people  sometimes  wish  to  know  about  long-term  retention  of  whafs  learned. 
Unfortunately,  research  on  long-term  retention  indicates  that  it  begins  as  a product  of  initial  level  of 
mastery  and  the  way  in  which  topics  are  taught,  but  that  by  far  the  biggest  single  factor  influencing  long- 
term retention  is  whether  the  learner  has  an  opportunity  to  use  whafs  been  learned.  Consequently,  there  is 
no  simple  way  to  evaluate  long-term  retention  due  to  any  type  of  instruction. 

Non-Mastery-Based  Program  Effectiveness 

In  a non-mastery  program,  instructor-led  classes  typically  are  the  primary  means  of  initial  instruction.  It  is 
more  for  the  technology  to  be  used  in  a complementary  or  supplementary  way.  Learners  are  managed  so 
that  instructional  time  (pacing)  is  fixed  and  achievement  varies. 

Questions 

Use  this  design  when  you  want  to  find  out: 

• How  well  are  the  learners  progressing  in  the  program  as  a whole? 

• What  is  the  longest,  shortest,  and  average  time  on  PLATO? 

• What  proportion  of  learners  master  skills  at  the  required  level? 

• What  proportion  of  learners  do  not  complete  or  drop  out? 

Affective  Outcomes 

In  any  evaluation,  if  s common  to  be  interested  in  how  learners  and  instructors  felt  about  their  use  of  the 
system,  their  confidence  and  their  role  in  the  instructional  process. 

Research  has  shown  that  affective  outcomes  are  largely  independent  of  actual  learning.  Thus,  asking 
learners  how  they  felt  about  the  value  of  a course  or  whether  they  achieved  its  objectives  is  not  an  adequate 
substitute  for  measuring  learning  using  either  of  the  two  "program  effectiveness"  evaluation  designs  above. 
Questions 

Use  this  evaluation  design  when  you  want  to  find  out  if  learners  using  the  technology  generally  feel... 

• good  about  their  experience? 

• unembarrassed  by  wrong  answers? 

• in  control  of  their  learning? 

• rushed? 

• the  system  is  relevant  to  their  needs? 

• the  system  accommodates  their  preferred  learning  style? 

• confident  of  their  ability  to  succeed? 

• more  goal-oriented? 

• motivated  to  use  the  system  effectively? 

And,  if  instructors  using  PLATO  generally  feel... 

• good  about  their  experience? 

• able  to  spend  more  time  1-on-l  with  learners? 

• in  control  of  the  instruction? 

• a generally  good  understanding  of  what  PLATO  teaches  (and  does  not)? 

• well  prepared  for  the  changes? 

• adequate  time  to  use  the  system  effectively? 

• adequately  trained  to  use  the  system  effectively? 

• motivated  to  use  the  system  effectively? 

You  can  answer  these  questions  at  the  end  of  the  program,  or  you  can  do  a "before  and  after"  comparison  in 
which  you  measure  expectations  and  current  attitudes  ("before")  against  actual  feelings  at  the  end  of  the 
program. 
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Case  Studies/Qualitative  Studies 

Decision-makers  often  have  difficulty  drawing  appropriate  conclusions  from  purely  numerical  evaluation 
studies.  Some  decision-makers  have  an  overt  distrust  for  quantitative  studies:  "figures  lie  and  liars  figure." 
In  addition,  evaluations  which  summarize  results  for  an  entire  group  require  considerable  effort  to  do,  and 
often  aren't  available  until  long  after  completion  of  the  program.  In  non-mastery-based  classrooms  where 
technology  is  not  the  only  instructional  element,  it  may  not  be  possible  to  use  a quantitative  design  to  study 
the  effects  of  the  technology.  Finally,  many  people  find  numerical  evaluations  too  abstract  to  say  whafs 
really  important.  This  is  often  the  case,  for  example,  with  collaborative  learning  programs. 

A case  study  (qualitative)  evaluation  is  an  alternative  which  addresses  these  criticisms.  In  general,  a 
qualitative  evaluation  can: 

• Serve  as  a useful  complement  to  a numerical  evaluation,  and  "put  a human  face  on 
the  numbers." 

• Carry  more  "face  validity"  than  a numerical  study,  and  thus  have  more  impact  with 
some  decision-makers. 

• Be  prepared  quickly  and  at  low  cost  if  the  number  of  learners  is  small. 

• Contain  "narrow  but  deep"  descriptions  of  one  or  more  experiences  with  the 
program,  rather  than  "wide  but  shallow"  generalizations  common  of  numerical 
evaluations. 

Among  the  limitations  of  qualitative  studies  are: 

• Observations  are  true  only  of  the  learners  interviewed  and  may  not  be  generally 
true. 

• Data  collection  is  less  objective  than  numerical  studies. 

• Including  the  learner  in  the  study  may  be  an  added  motivator  not  available  to  most 
learners.  This  can  lead  to  biased  conclusions. 

• Data  collection  and  interpretation  is  costly  unless  numbers  of  participants  is  very 
small. 

A qualitative  evaluation  can  be  used  instead  of  or  in  addition  to  a more  conventional  numerical  evaluation. 
Questions 

Use  this  design  when  you  want  to  ask: 

• What  are  the  needs  of  the  learner? 

• What  is  the  motivation  and  the  goals  of  the  learner? 

• What  is  the  learner's  pattern  of  technology  usage? 

• How  has  the  learner  integrated  the  technology  with  other  learning  experiences  to 
achieve  the  class'  learning  goals? 

• In  learning  from  the  technology,  what  facilitators  and  obstacles  were 
encountered? 

• How  has  the  technology  usage  affected  the  learner's  confidence? 

• What  have  been  the  learning  outcomes  for  that  learner? 

Program  Comparison 

There  is  over  25  years  of  research  showing  that  people  can  learn  from  computer-based  instruction,  and  that 
on  the  whole  CBl  implementations  achieve  greater  gains  in  less  time  than  large-group  classrooms. 

However,  it  is  still  true  that  decision  makers  often  want  to  know  if  using  technology  is  an  improvement 
over  whatever  is  being  done  now  in  their  program.  Unfortunately,  meaningful  evaluations  of  this  type 
require  carefully  controlled  "experimental"  designs  and  are  very  difficult  to  do  well.  There  are  a number  of 
reasons  for  this: 

• "Whatever  is  being  done  now"  is  often  imprecisely  defined  and  varies  widely 
from  instructor  to  instructor,  class  to  class  and  semester  to  semester. 

• It's  common  to  change  more  than  one  thing  in  a program  when  technology  is 
introduced,  so  it's  impossible  to  attribute  specific  changes  to  the  technology. 

• An  interpretable  "experiment"  has  requirements  which  are  incompatible  with 
many  program  structures. 
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Everyone's  performance  improves  for  a while  when  they  know  they're  "part  of 
an  experiment." 


Questions 

Use  this  design  when  you  want  to  ask: 

• What  learning  gains  occur  using  technology,  when  compared  to  our 
conventional  practice? 

• What  percentage  of  learners  achieve  a predetermined  mastery  level  using 
technology,  when  compared  to  our  conventional  practice? 

• What  is  the  longest,  shortest  and  average  time  to  reach  mastery  level  using  the 
technology,  when  compared  to  our  conventional  practice? 

• What  are  the  effects  of  using  technology  on  completion/dropout  rate,  when 
compared  to  our  conventional  practice? 

• Are  any  of  the  above  comparisons  large  enough  that  we  can  say  with  confidence 
that  they  are  not  due  to  chance  alone? 


Cost  Analyses 

Program  administrators  sometimes  want  to  cost-justify  use  of  technology  on  the  basis  of  reduced  (or 
avoided)  costs,  increased  enrollment  income,  or  increased  benefit  to  the  host  organization.  This  kind  of 
cost-effectiveness  justification  is  based  on  a comparison  of  the  results  of  conventional  and  technology- 
based  programs  arrayed  against  the  cost  of  each. 

Do  not  confuse  a cost-effectiveness  study  with  a simple  cost-cost  comparison,  in  which  the  costs  of  the 
conventional  and  technology-using  programs  are  compared  without  reference  to  the  effectiveness  of  the 
programs. 

Also,  do  not  confuse  cost-effectiveness  with  a cost-benefit  study.  A cost-benefit  evaluation  might  be  done 
for  a new  program  to  see  if  it  is  meeting  its  projected  costs  and  benefits.  To  do  a cost-benefit  study,  you 
would  use  the  "Mastery-Based  Program  Effectiveness"  or  "Non-Mastery-Based  Program  Effectiveness" 
evaluation  models,  and  collect  appropriate  cost  and  enrollment  income  data  as  well.  If  your  program  is 
justified  on  the  basis  of  benefits  to  a host  organization  or  the  general  community,  then  you  may  also  be  able 
to  gather  cost-avoidance  data  such  as  reduced  follow-on  training  time,  reduced  equipment  downtime, 
reduced  plant  or  process  startup,  and  so  on.  In  either  case,  there  is  no  comparison  in  a cost-benefit  study 
with  an  existing  program. 

To  do  a cost-effectiveness  study,  you  must  use  the  "Program  Comparison"  design.  Then  you  must  decide  if 
you  want  to  do  the  cost  analysis  on  the  basis  of  reduced  costs  or  increased  enrollments.  If  you  choose  to 
use  reduced  costs,  then  add  to  the  evaluation  a costing  model  for  each  treatment.  If  you  choose  to  use 
increased  enrollments,  then  add  to  the  evaluation  design  a comparison  of  enrollments  (or  retention)  in  each 
program. 

Evaluation  Procedures  Summary 

Often,  you'll  want  to  combine  the  evaluation  designs.  To  help,  the  tables  below  include  evaluation 
requirements,  data  gathering  and  data  analysis  procedures  for  the  six  evaluation  designs. 

Requirements 


Require- 

ments 

Mastery 
Prog.  Eff. 

Non- 
Mastery 
Prog.  Eff. 

Affective 

Outcomes 

Case/ 

Qualitative 

Program 

Compari- 

son 

Cost 

Compari- 

son 

Learners 
match  user 
profile 

X 

X 

X 

X 

X 

X 

Unlimited 

access 

X 

X 

Individual 

placement 

X 

X 
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Require- 

ments 

Mastery 
Prog.  Eff. 

Non- 
Mastery 
Prog.  Eff. 

Affective 

Outcomes 

Case/ 

Qualitative 

Program 

Compari- 

son 

Cost 

Compari- 

son 

Individual 

routing 

X 

X 

Work® 
own  pace 

X 

X 

"Place  out" 
with  module 
tests 

X 

X 

X 

X 

X 

Pass  module 
test  to  go  on 

X 

X 

X 

X 

X 

Offline  tests 
must  be 
aligned 

X 

X 

X 

Offline 
Learning 
activities 
aligned  & 
synchron- 
ized 

X 

X 

X 

Random. 

assignment 

X 

Parallel 

content 

X 

Post-test  in 
3 days  of 
completion 

X 

X 

X 

Implement- 
ation 
training, 
support  & 
monitoring 

X 

X 

X 

X 

X 

X 

No  contact 

between 

classes 

X 

Data  Gathering 


Require- 

ments 

Mastery 
Prog.  Eff. 

Non- 
Mastery 
Prog.  Eff. 

Affective 

Outcomes 

Case/ 

Qualitative 

Program 

Compari- 

son 

Cost 

Compari- 

son 

Pre-test  & 
prescription 

X 

X 

X 

X 

Monitor 

implemen- 

tation 

X 

X 

X 

X 

X 

X 

Mastery/ 

completion 

X 

X 

X 

X 

X 

Time 

X 

X 

X 

X 

Post- test  in 
3 days 

X 

X 

X 

X 

Question- 

naires 

X 
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Require- 

ments 

Mastery 
Prog.  Eff. 

Non- 
Mastery 
Prog.  Eff. 

Affective 

Outcomes 

Case/ 

Qualitative 

Program 

Compari- 

son 

Cost 

Compari- 

son 

Interviews/ 

focus 

groups 

before, 

during  & 

after 

X 

X 

Logs/ 

writing 

X 

Cost  data 

X 

Data  Analysis 


Require- 

ments 

Mastery 
Prog.  Eff. 

Non- 
Mastery 
Prog.  Eff. 

Affective 

Outcomes 

Case/ 

Qualitative 

Program 

Compari- 

son 

Cost 

Compari- 

son 

Convert 

pre/post 

scores 

X 

X 

X 

X 

Gain  scores 

X 

X 

X 

X 

Pre-post 
paired  bar 
graph 

X 

X 

X 

Gain  bar 
graph 

X 

X 

Mest 

X 

X 

X 

X 

Between 

groups 

comparisons 

X 

X 

Mastery/ 

completion 

% 

X 

X 

X 

X 

Time  on 
task 

X 

X 

X 

X 

Mastered 
vs.  non- 
mastered 
time 

X 

X 

X 

X 

Cost 

summary/ 

comparison 

X 

$ Benefit 
summary/ 
comparison 

X 

Non-$ 

benefit 

X 
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Abstract 

This  paper  reports  our  work  towards  the  development  of  an  authoring  system  for  automating  the  instructional 
development  of  adaptive  instruction  for  teaching  concepts.  The  core  of  our  work  is  the  design  of  a set  of 
instructional  events  for  concept  teaching,  Introduction,  Demonstration,  Practice,  and  Test,  and  the 
development  of  corresponding  instructional  event  shells.  An  instructional  event  shell  is  a piece  of  program 
code  that  executes  an  instructional  event.  The  focus  of  this  paper  is  a description  of  these  instructional  events 
and  their  shells  and  how  these  instructional  event  shells  can  be  used  to  author  computer-based  instruction  for 
concept  teaching.  Issues  related  content-based  and  event-based  authoring  will  also  be  discussed. 


Introduction 

With  the  rapid  advances  of  computer  network  and  multimedia  technologies,  the  use  of  interactive 
instructional  technologies  has  been  experiencing  a dramatic  increase  in  recent  years.  However,  development  of 
high  quality  computer-based  courseware  is  both  time  consuming  and  labor  intensive.  Therefore,  only  the  best 
funded  training  programs  can  afford  to  be  computer-based.  Many  organizations  that  are  attracted  to  the  concept 
of  computer-based  training  have  to  give  up  when  they  discover  the  cost  and  the  time  to  implement  computer- 
based  courseware. 

A traditional  computer-based  courseware  development  team  involves  subject  matter  experts,  instructional 
designers,  and  programmers.  The  subject  matter  expert  determines  what  to  teach  and  provides  subject  matter 
knowledge.  The  instructional  designer  together  with  the  subject  matter  expert  determines  how  to  teach  and 
designs  the  instructional  treatments  for  subject  matter  knowledge  to  be  taught.  The  programmer  generates 
computer  code  to  implement  the  instructional  treatments  for  the  delivery  of  the  instructional  material.  The 
development  processes  of  both  instructional  design  and  programming  are  very  long  and  labor  intensive  even  for 
some  simple  training  programs. 

To  reduce  the  time  and  cost  spent  in  programming,  authoring  systems  have  been  developed  and  used 
widely.  The  goal  of  existing  authoring  systems  is  to  eliminate  programming  in  the  courseware  development 
process  and  lift  the  expensive  burden  from  the  course  developer’s  shoulders.  Solving  the  programming  problem 
for  course  developers  with  authoring  systems  has  greatly  influenced  the  productivity  of  development  of 
computer-based  instruction,  and  the  number  of  interactive  instructional  software  products  has  increased 
geometrically. 

However,  the  current  conception  of  authoring  systems  has  not  solved  the  cost  and  time  problem  to  a 
sufficient  degree.  The  quality  of  many  current  instructional  products  still  appears  to  be  far  below  the  potential 
promised  by  the  current  computer  technology.  Even  though  the  expensive  programming  burden  has  been  lifted 
from  the  course  developer’s  shoulders,  there  remains  a equally  cost  and  unnecessarily  repetitive  instructional 
design  process  burden  which  still  rests  on  the  developer  (Merrill,  1985). 

Merrill  and  his  colleagues  (Merrill,  Li,  & Jones,  1991)  introduced  the  instructional  transaction  theory  that 
maintains  that  the  basic  building  block  of  instruction  should  be  the  instructional  transaction.  This  principle 
applies  particularly  to  the  highly  interactive  medium  of  computer-based  instruction,  in  which  the  most  powerful 
vehicle  for  learning  is  the  pattern  of  interactions  between  the  student  and  the  computer.  Based  on  the 
instructional  transaction  theory,  a simple  prototype  named  ID  Expert  (Merrill  & Li,  1 989)  was  developed.  ID 
Expert  was  intended  for  use  by  both  novice  subject  matter  experts  and  experienced  instructional  designers  and 
provides  instructional  design  and  templates  for  courseware  developers.  The  instructional  transaction  theory 
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provides  a theoretical  foundation  for  the  development  of  authoring  systems  that  automate  the  instructional 
design  of  computer-based  instruction  (Zhang,  et  al.,  1997).  Recently,  a few  commercial  authoring  systems  such 
as  ToolBook  and  Authorware  included  some  templates  or  widgets  that  speed  up  the  courseware  development 
process. 

This  paper  reports  our  work  towards  the  development  of  an  authoring  system  for  automating  the 
instructional  development  of  adaptive  courseware  for  teaching  concepts.  The  core  of  our  work  is  the  design  of  a 
set  of  instructional  events  for  concept  teaching.  Introduction,  Demonstration,  Practice,  and  Test^  (Merrill,  et  al., 
1 992)  and  the  development  of  corresponding  instructional  event  shells.  An  instructional  event  shell  is  a piece  of 
program  code  that  executes  an  instructional  event.  The  focus  of  this  paper  is  a description  of  these  instructional 
events  and  their  shells  and  how  these  instructional  event  shells  can  be  used  to  author  computer-based 
instruction.  The  ideas  introduced  in  this  paper  are  being  implemented  in  an  authoring  system  named 
ConceptAuthor. 


Course  Structure  And  Concept  Representation 

A concept  is  a set  of  specific  objects,  symbols,  or  events  which  are  grouped  together  on  the  basis  of 
shared  characteristics  and  which  can  be  referenced  by  a name.  Authoring  a course  using  ConceptAuthor  is  to 
create  a course  structure  and  provide  content  material  for  all  concepts  to  be  taught.  The  course  structure  and  the 
content  materials  of  concepts  are  stored  in  a database.  The  course  structure  is  separately  stored  from  the  content 
materials.  This  section  discusses  the  course  structure  and  concept  representation  implemented  in 
ConceptAuthor. 


Course  Structure 

A course  structure  is  a tree  of  course  units.  The  root  of  the  tree  is  the  course  and  the  leaves  are 
instructional  events.  A course  may  consist  of  a set  of  events  and  a set  of  lessons.  Instructional  events  of  a course 
may  be  a pre-test,  a post-test,  a course  introduction,  a course  summary,  and  a comprehensive  practice. 

A lesson  may  be  composed  of  a lesson  introduction,  a lesson  summary,  a comprehensive  practice,  a test, 
and  one  or  more  instructional  sessions.  An  instructional  session  may  consist  of  a set  of  instructional  events  and 
zero  or  more  other  instructional  sessions.  A session  may  teach  one  concept  or  a set  of  coordinate  concepts. 


Concept  Representation 

Each  concept  in  ConceptAuthor  is  stored  as  a frame  that  stores  the  information  about  the  concept. 
Information  stored  in  the  frame  may  include  the  concept  name,  the  concept  definition,  the  concept  context,  and 
a list  of  concept  characteristics,  lists  of  discriminans,  the  concept  instances,  and  the  problems.  The  concept 
name  and  definition  are  required  while  all  others  are  optional. 

The  concept  name  is  a string  and  a concept  may  have  more  than  one  name.  Most  likely,  a concept 
definition  is  a paragraph  of  text,  but  it  could  also  be  some  other  media  such  as  audio,  video,  and  images.  A 
concept  may  have  several  definitions.  Especially,  when  a concept  is  very  complex  or  disjunctive,  the  teacher 
may  want  to  teach  the  concept  in  several  steps.  First,  a simple  definition  is  taught,  and  then  this  simple 
definition  is  gradually  expanded  until  a complete  definition  is  taught.  The  concept  context  specifies  where, 
when,  and  how  a concept  is  used. 

A concept  characteristic  is  the  property  that  holds  for  all  examples  of  the  concept.  When  a concept  is 
fuzzy,  a concept  characteristic  may  only  hold  for  all  typical  examples.  A concept  characteristic  may  be  a 
property  or  a relation.  A concept  discriminant  is  a property  or  a relation  that  discriminates  the  concept  from  its 
coordinate  concepts. 

A concept  instance  is  either  an  example  or  a counter  example  of  the  concept.  Some  instances  are  more 
typical  (representative)  than  other  instances.  Some  instances  are  easier  to  understand  than  other  instances. 
Problems  are  used  for  practice  or  tests.  There  could  be  many  different  types  of  problems.  The  types  of  problem 
will  be  discussed  in  Section  3. 


A set  of  related  concepts  may  form  a concept  taxonomy.  In  this  taxonomy,  a concept  may  consist  of 
several  subordinate  concepts.  A subordinate  concept  may  be  a kind  of  its  superordinate  concept  or  a part  of  its 
superordinate  concept.  Coordinate  concepts  are  a set  of  concepts  in  which  each  member  shares  the 
characteristics  of  the  same  superordinate  concept  (Merrill,  1992).  The  course  developer  may  intentionally  group 
a few  concepts  for  some  instructional  purpose. 


Instructional  Events  for  Teaching  Concepts 

Concepts  may  be  taught  using  four  instructional  events:  Introduction,  Demonstration,  Practice,  and  Test 
(Merill,  et  al.,  1992).  These  four  instructional  events  may  be  used  repeatedly.  In  ConceptAuthor,  four 
corresponding  instructional  event  shells  are  being  implemented  to  execute  these  four  events. 


Introduction  Event 

An  introduction  event  introduces  to  a concept  to  the  student  by  presenting  the  concept  name,  the  concept 
definition,  and  the  concept  context.  In  addition,  this  event  also  shows  the  student  a representative  example  of 
the  concept.  All  materials  may  be  presented  using  different  media  such  as  text  (rich  text  format),  pictures,  still 
images,  animations,  audio,  and  videos.  All  materials  may  be  presented  in  one  or  more  pages.  Hyperlinks  are 
allowed. 

A set  of  coordinate  concepts  could  be  introduced  in  sequence  or  in  parallel.  If  concepts  are  introduced  in 
sequence,  each  introduction  event  only  presents  materials  of  one  concept.  An  introduction  event  presents 
materials  of  two  or  more  coordinate  concepts  if  they  are  introduced  in  parallel.  When  coordinate  concepts  are 
introduced  in  parallel,  the  differences  among  these  concepts  may  be  compared  and  presented. 

For  a complex  concept  or  a disjunctive  concept,  an  introduction  event  may  only  present  a part  of  the 
concept  definition.  Namely,  a concept  definition  may  be  presented  using  several  introduction  events. 


Demonstration  Event 

A demonstration  event  presents  the  student  a sequence  of  instances  (examples  and  counter  examples  of 
the  target  concept.)  Instances  presented  should  be  as  divergent  as  possible.  Namely,  they  should  cover  all 
aspects  of  the  target  concept  (examples  of  every  disjunct  and  examples  showing  every  characteristic  of  the 
target  concept.)  Easy  and  simple  instances  should  be  presented  before  difficult  ones.  Typical  instances  should 
be  presented  before  atypical  ones.  Near  miss  counter  examples  could  be  more  useful  than  other  counter 
examples  so  they  should  be  presented  more  often  than  other  counter  examples. 

Instances  may  be  presented  together  with  the  concept  definition.  Each  time  an  example  or  a counter 
example  is  presented  to  the  student,  an  explanation  of  the  example  or  counter  example  may  be  available  upon 
the  request  of  the  student.  At  any  time,  the  student  may  inform  the  event  how  well  he/she  understands  a 
presented  instance.  Presentation  of  instances  may  be  adjusted  according  to  the  student  feedback. 

Similarly  to  an  introduction,  instances  of  coordinate  concepts  may  be  demonstrated  in  sequence  or  in 
parallel.  When  demonstrated  in  sequence,  only  one  instance  is  demonstrated  each  time.  When  demonstrated  in 
parallel,  multiple  instances  of  different  concepts  are  demonstrated  simultaneously.  When  instances  of  different 
concepts  are  presented  in  parallel,  differences  among  the  instances  of  different  concepts  may  be  highlighted. 

For  a complex  concept  or  a disjunctive  concept,  a demonstration  event  may  only  present  a subset  of 
instances  of  a concept.  Namely,  all  concept  instances  may  be  presented  using  several  demonstration  events. 

Parameters  of  a demonstration  event  may  include  difficulty  level,  example  number,  counter  example 
number,  instance  presentation  order,  explanation  mode,  and  control  mode. 

Difficulty  level  is  the  average  difficulty  level  of  all  presented  instances.  Example  number  is  the  number 
of  examples  to  be  demonstrated.  Counter  example  number  is  the  number  of  counter  examples  to  be 
demonstrated.  Instance  presentation  order  is  the  order  in  which  instances  are  presented.  The  possible 
presentation  orders  include,  from  easy  ones  to  difficult  ones,  from  typical  ones  to  atypical  ones,  random, 
examples  before  counter  examples,  one  example  and  one  counter  example,  simultaneous  demonstration  of 
examples  and  counter  examples.  Explanation  mode  determines  if  an  explanation  of  each  instance  should  be 
provided  or  not  or  upon  the  request  of  the  student.  There  are  two  control  modes:  learner  control  and  system 


ERIC 


866 


control.  What  can  be  controlled  include  which  instances  to  be  demonstrated,  the  amount  of  instances  to  be 
demonstrated,  the  order  in  which  instances  are  demonstrated.  The  student  may  have  no  control  at  all,  partial 
control,  or  total  control.  If  the  student  has  total  control,  a list  of  all  instances  will  be  provided  for  the  student  to 
choose.  Learning  guidance  is  always  available  for  the  learner  control. 


Practice  Event 

A practice  event  provides  an  opportunity  for  the  student  to  apply  what  they  learned  in  the  introduction 
and  demonstration  events.  The  following  is  a list  of  different  types  of  problems. 

True/False: 

Instance  / is  an  example  of  concept  C. 

Instance  / is  a counter  example  of  concept  C. 

Properties  Pt,  P„  are  characteristics  of  concept  C. 

Properties  Pj,  P„  are  not  characteristics  of  concept  C. 

Properties  Pj,  P„  discriminate  concept  C/  from  concept  Q. 

Properties  Pj,  P„  do  not  discriminate  concept  C/  from  concept  C2. 

Instance  I is  an  example  of  concept  C because  of  Properties  Pj,  ...»  P„. 

Instance  I is  a counter  example  of  concept  C because  of  Properties  Pj,  Pn. 

Multiple  Choice: 

Which  of  the  following  concepts  does  instance  I belong  to? 

Which  of  the  following  instances  is  an  example  of  concept  C? 

Which  of  the  following  properties  is  a characteristic  of  concept  C? 

Which  of  the  following  properties  is  not  a characteristic  of  concept  Cl 

Which  of  the  following  Properties  discriminates  concept  C/  from  concept  C^. 

Which  of  the  following  Properties  does  not  discriminate  concept  C/  from  concept  C^. 

Instance  1 is  an  example  of  concept  C because  of  which  of  the  following  Properties. 

Instance  1 is  a counter  example  of  concept  C because  of  which  of  the  following  Properties. 

Matching: 

Given  n instances  and  n concepts,  match  one  instance  to  one  concept. 

Given  n properties  and  n concepts,  match  one  property  to  one  concept.  A property  matches  a concept  if  the 
property  is  a characteristic  of  the  concept. 

Fill-In-Blank: 

Given  an  instance,  name  the  concept  to  which  it  belongs. 

Given  a concept,  name  one  or  more  characteristics  of  the  concept. 

Problems  may  be  automatically  generated  by  the  system  or  provided  by  the  course  developer.  After  a 
question  is  answered,  the  system  may  ask  the  student  to  provide  an  explanation  for  his/her  answer.  This  may  be 
just  another  question  in  one  of  the  formats  specified  above.  Hints  for  each  problem  may  be  available.  Some 
examples  of  hints  are  concept  definition  with  important  characteristics  highlighted,  a similar  problem  with  its 
answer,  demonstration  of  a similar  instance,  and  hints  given  by  the  course  developer.  After  an  answer  is  given, 
feedback  will  be  provided.  If  the  answer  is  correct,  some  reinforcement  message  may  be  given.  If  the  answer  is 
not  correct,  feedback  may  include  the  correct  answer,  the  explanation  of  the  correct  answer,  why  the  answer 
given  by  the  student  is  not  correct,  and  reference  materials  relevant  to  the  problem. 

Problems  given  to  the  student  should  be  as  divergent  as  possible.  They  should  cover  all  aspects  of  the 
target  concept.  There  should  be  some  easy  and  simple  problems  as  well  as  difficult  ones.  Easy  and  simple 
problems  should  be  given  before  difficult  ones.  The  practice  event  is  adaptive  and  the  amount  of  problems  to  be 
given  and  which  problems  to  be  given,  and  the  difficulty  level  of  the  problems  are  dynamically  adjusted  based 
on  the  student  performance. 

Parameters  of  a practice  event  may  include:  difficulty  level,  amount  of  problems,  problem  type,  hint, 
feedback,  and  control  mode. 

Difficulty  level  is  the  average  difficulty  level  of  all  problems.  Amount  of  problems  is  the  number  of 
problems  to  be  presented.  Problem  type  is  the  percentage  of  problems  to  be  presented  for  each  question  type. 
Hint  is  if  hints  should  be  available  or  which  hint  should  be  available.  Feedback  is  if  feedback  should  be 
provided  or  how  much  should  be  provided.  There  are  two  control  modes:  learner  control  and  system  control. 
What  can  be  controlled  include  the  amount  of  practice  problems  and  the  types  of  problems.  The  student  may 
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have  no  control  at  all,  may  have  partial  control,  or  may  have  total  control.  If  the  student  has  total  control,  he/she 
determines  if  he/she  needs  more  practice  and  the  type  and  the  area  of  the  problems  he/she  wants  to  solve. 
Learning  guidance  should  always  be  available  for  learner  control.  An  exploration  event  could  be  developed 
using  the  demonstration  event  shell  with  learner  control. 


Test  Event 

The  purpose  of  a test  event  is  self-assessment  and  its  result  may  be  used  for  adaptive  instruction.  A test 
event  is  similar  to  a practice  event,  but  hints  and  feedback  are  not  provided  and  it  is  always  system  controlled.  A 
test  event  grades  every  problem  and  provides  a summary  of  the  test  result  for  the  student.  The  types  of  the 
problems  in  a test  event  are  the  same  as  those  described  in  section  3.3.  The  test  may  be  adaptive. 

Parameters  of  a test  event  may  include  difficulty  level,  amount  of  problems,  and  adaptation.  Difficulty 
level  is  the  average  difficulty  level  of  all  problems.  Amount  of  problems  is  the  number  of  problems  to  be 
included  in  the  test.  Adaptation  determines  if  the  test  is  adaptive  or  not. 


Authoring  A Course  Using  Instructional  Event  Shells 

Authoring  a course  using  ConceptAuthor  consists  of  the  following  three  phases. 

• Content  knowledge  acquisition 

• Course  structure  construction 

• Instructional  event  configuration 

Content  knowledge  acquisition  is  to  build  concept  taxonomies  and  enter  the  concept  name,  the  concept 
definition,  the  concept  context,  and  a list  of  concept  characteristics,  lists  of  discriminans,  the  concept  instances, 
and  the  problems  for  each  concept  in  the  taxonomies.  Course  structure  construction  is  to  create  the  tree  structure 
of  a course  and  define  all  course  units  including  instructional  events.  Instructional  event  configuration  is  to 
associate  instructional  event  with  the  content  knowledge  (concepts)  to  be  taught  and  set  values  of  all  parameters 
of  an  instructional  event.  ConceptAuthor  includes  a set  of  tools  to  assist  the  course  developer  in  all  these  three 
authoring  phases.  Part  of  the  course  structure  construction  phase  and  the  instructional  event  configuration  phase 
could  be  automated.  In  this  section,  we  will  not  discuss  the  authoring  processes  of  these  three  phases  and  how 
these  tools  work.  Instead,  we  will  concentrate  on  the  discussions  of  how  instructional  event  shells  can  be  used  to 
formulate  different  concept  teaching  strategies  (Merrill,  et  al.,  1992).  Many  different  concept  teaching  strategies 
may  be  created  using  the  four  instructional  events.  We  will  only  discuss  a few  of  them. 

A typical  and  conservative  strategy  is  the  sequential  strategy  that  teaches  one  concept  after  another.  The 
instructional  sequence  for  teaching  one  concept  is  introduction,  demonstration, practice,  and  test  (Merrill,  et  al., 
1992).  Coordinate  concepts  should  not  be  taught  one-by-one  and  should  be  taught  simultaneously  so  that 
similarities  and  differences  of  these  coordinate  concepts  may  be  compared.  One  strategy  for  teaching  coordinate 
concepts  is  the  sequential-simultaneous  strategy  that  introduces  and  demonstrates  all  coordinate  concepts  one- 
by-one  and  practice  and  test  on  all  coordinate  concepts  simultaneously. 

An  alternative  strategy  for  teaching  coordinate  concepts  is  the  simultaneous  strategy  in  which  all 
coordinate  concepts  are  introduced,  demonstrated,  practiced  , and  tested  simultaneously. 

The  last  strategy  discussed  in  this  section  is  the  discovery  strategy  that  is  very  different  from  previous 
introduced  strategies.  The  discovery  strategy  facilitates  learning  by  discovery.  It  guides  the  student  to  discover 
the  definition  of  the  target  concept  by  presenting  a sequence  of  examples  and  counter  examples  of  the  concept. 
The  order  in  which  these  examples  and  counter  examples  are  presented  should  be  well  designed.  After  one  or 
more  examples  and  counter  examples  are  presented,  a few  problems  may  be  asked  to  check  how  well  the 
student  learned  by  discovery.  And  then,  more  examples  and  counter  examples  may  be  demonstrated  again,  and 
a few  more  problems  are  asked.  This  discovery  process  repeats  until  the  student  discovers  the  concept  definition 
or  the  student  quits  this  process.  After  this  discovery  process,  the  concept  definition  is  presented  followed  by  a 
test.  Similarly,  the  discovery  strategy  can  be  sequential  and  simultaneous.  The  discovery  strategy  discussed 
above  is  sequential. 


868 


Conclusion 


In  this  paper,  we  reported  the  work  we  conducted  in  development  of  a concept  teaching  authoring 
system,  ConceptAuthor.  This  work  is  a part  of  a larger  project.  One  of  the  objectives  of  development  of 
ConceptAuthor  is  to  explore  the  methods  for  developing  authoring  systems  to  support  content-based  authoring. 
The  second  objective  of  this  work  is  to  investigate  methods  for  automating  instructional  design  of  adaptive 
instruction  using  instructional  event  shells.  ConceptAuthor  is  being  implemented  on  Windows  95  in  Visual  C++ 
and  Access. 
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Abstract:  The  deficiency  of  the  auditory  sense  in  the  hearing  impaired  raises  the  question  as  to  what  extent 
this  deficiency  affects  their  cognitive  development  and  intellectual  abilities.  In  studies  that  have  been  carried 
out  over  the  years,  many  theories  have  been  presented  on  the  cognitive  development  and  performance  of 
hearing  impaired.  The  purpose  of  this  study  was  to  discover  whether  the  practice  of  rotating  three-dimensional 
objects  with  Virtual  Reality  will  have  an  effect  on  the  flexible  thinking  in  hearing  impaired  children.  The 
study  was  carried  out  with  60  children,  of  which  44  were  hearing  impaired.  The  hearing  impaired  children 
were  distributed  into  two  groups:  the  experimental  group  and  the  control  group.  The  experimental  group 
played  virtual  3-D  “Tetris”  individually,  for  15  minutes,  once  a week  over  a period  of  three  months.  The 
control  group  played  2-D  “Tetris”  over  the  same  period  of  time.  In  addition,  16  children  with  normal  hearing 
took  part  in  the  study  as  a second  control  group  in  order  to  establish  whether  hearing  impaired  children  really 
are  at  a disadvantage  in  terms  of  their  flexible  thinking.  The  experimental  group  and  the  hearing  impaired 
control  group  were  evaluated  by  the  Torrance  sub-test  “Circles”  (1966),  before  and  after  the  experiment.  The 
results  clearly  indicate  that  practicing  3D  spatial  rotations  with  VR  significantly  improved  the  flexible 
thinking  in  the  experimental  group  as  opposed  to  the  hearing  impaired  control  group,  who  did  not  improve 
significantly.  Also,  before  the  experiment,  it  was  discovered  that  the  hearing  impaired  children  attained  lowers 
scores  in  flexible  thinking  than  the  children  with  normal  hearing.  After  the  experiment,  however,  the  results  of 
the  experimental  group  improved  to  the  extent  that  there  was  no  noticeable  difference  between  them  and  the 
control  group  of  children  with  normal  hearing. 


Introduction 

The  concept  of  “creativity”  is  regarded  and  defined  in  different  ways  by  different  researchers.  The  definitions  differ  from  each  other 
according  to  the  difference  in  their  approaches  to  creativity.  In  general,  one  can  divide  the  definitions  into  three  categories  (Nevo,  1997): 

1 . The  definitions  which  emphasize  the  creative  process. 

2.  The  definitions  which  deal  with  the  creative  person  and  his/her  personality.  These  definitions  focus  on  inter-personal  differences  such  as 
the  characteristics  of  the  creative  person  and  how  s/he  differs  from  the  “regular”  person. 

3.  The  definitions  which  deal  with  the  product.  They  emphasize  the  realization  of  the  potential  of  the  main  idea  and  the  extent  to  which  the 
final  product  differs  from  that  which  already  exists. 

The  approach  of  Guilford  (1967,  1970)  and  Nevo  (1997)  has  been  of  great  influence.  They  refer  to  factors  in  activating  and  factors  of 
character,  but  their  main  contribution  is  in  the  detailed  analysis  of  the  elements  found  in  the  process  of  creative  thinking.  According  to  their 
approach,  creative  thinking  is  a complex  and  diversified  phenomenon  and  it  involves  a number  of  capabilities  which  are  almost  independent  of 
each  other.  Guilford  distinguishes  between  a)  convergent  thinking  b)  divergent  thinking. 

Convergent  thinking  points  to  the  production  of  information  emphasizing  one  single  probable  and  best  result.  On  the  other  hand,  divergent 
thinking  flows  in  many  different  directions  and  is  connected  to  thinking  functions  in  which  there  is  not  just  one  correct  solution.  This  way  of 
thinking  is  also  known  as  creative  thinking.  Guilford  identified  four  factors  related  to  the  ability  to  think  creatively: 

• Fluency — the  ability  to  think  of  a number  of  possible  solutions  to  a given  need. 

• Flexibility— the  ability  to  change  approaches  and  points  of  view,  and  not  stick  to  only  one  approach. 

• Originality — the  ability  to  arrive  at  new  and  unconventional  solutions. 

• Elaboration- — the  ability  to  develop  and  improve  basic  ideas. 

Torrance  (1980)  points  out  that  most  young  children  are  creative,  but  that  this  ability  wanes  as  they  mature.  Schools  teach  with 
conventional  methods,  which  focus  on  precision,  speed  and  the  provision  of  correct  answers.  There  exist  specific  teaching  programs  which 
attempt  to  develop  creative  skills  using  various  activities  such  as:  brainstorming,  creative  drama,  activities  related  to  creative  art,  unconventional 
uses  of  daily  objects,  etc. 

This  study  focused  on  the  child’s  ability  to  think  flexibly  (the  second  aspect  of  creativity  described  by  Guilford  (1967,  1970)  above). 
Sternberg  & Powell  (1983)  define  flexible  thinking  as  the  ability  to  look  at  things  from  different  angles.  They  point  out  that  during  adolescence 
the  ability  of  the  child  to  think  flexibly  is  more  prominent.  This  flexibility  is  expressed  in  two  opposite  directions.  On  the  one  hand,  children 
exhibit  better  ability  to  think  consistently  and  adhere  to  methods  which  proved  effective  in  solving  problems.  On  the  other  hand,  when  necessary, 
they  are  capable  of  changing  their  work  methods  and  exchanging  them  for  more  successful  methods.  Flexible  thinking  is  one  of  the  most 
important  characteristics  of  intelligent  behavior. 

Guilford  (1967,  1970)  claims  that  flexible  thinking  is  the  ability  to  create  a flow  of  ideas  while  changing  direction  or  correcting 
information.  In  his  opinion,  there  are  two  types  of  flexibility  a)  spontaneous  flexibility — spontaneous  change  in  the  thinking  process  and  the 
transition  to  another,  and  b)  adaptive  flexibility — the  ability  to  adapt  to  changing  instructions.  The  component  of  flexibility  appears  to  Guilford 
to  be  related  to  the  ability  to  generalize  and  abstract. 

Researchers  studied  the  ability  of  hearing  impaired  children  to  think  flexibly  both  verbally  and  in  terms  of  shapes.  This  study  relates  solely 
to  non-verbal  ability. 
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Laughton  (1988)  compared  the  traditional  approach  of  teaching  art  to  teaching  programs  geared  to  developing  creative  ability.  He  studied 
28  hearing  impaired  children  between  the  ages  of  8-10,  who  took  part  in  one  of  the  two  programs  for  twelve  weeks.  The  children  were  tested  in 
the  Torrance  fomial  test  before  and  after  the  intervention.  It  was  found  that  there  was  a significant  improvement  in  flexibility  and  originality 
among  the  children  who  studied  according  to  the  new  program.  Laughton  (1988)  summarizes  and  claims  that  by  means  of  the  appropriate 
teaching  strategy  it  is  possible  to  develop  creative  aptitudes  with  hearing  impaired  children  and  to  help  them  to  become  less  concrete  and  rigid  in 
their  thinking. 

Saraev  & Koslov  (1993)  examined  100  deaf  children  and  164  hearing  children  between  the  ages  of  7-12.  One  of  their  findings  shows 
lesser  ability  in  creative  imagination  among  the  deaf,  and  rigidity  in  their  way  of  thinking. 

King  & Quigley  (1985)  also  claim  that  hearing  children  surpass  hearing  impaired  children  in  creative  ability. 

The  teaching  of  art  usually  receives  lower  priority  than  academic  subjects  in  the  teaching  programs  for  the  hearing  impaired.  Researchers 
have  found  that  focusing  more  on  strategies  for  creative  thinking  will  have  a favorable  effect  on: 

a)  abstract  thinking, 

b)  imagination, 

c)  the  attention  paid  to  various  details  of  patterns  whose  basis  is  visual  but  which  can  basically  serve  as  linguistic  behavior, 

d)  potential  emphasis  on  the  creation  of  vocational  opportunities, 

e)  more  efficient  functioning  in  situations  requiring  the  solving  of  problems,  etc,  (Laughton,  1988). 

Bunch  (1987)  also  claims  that  teachers  tend  to  concentrate  on  the  areas  in  which  the  hearing  impaired  experience  the  most  difficulty — the 
language,  communication  and  reading.  Studies  show  that  the  hearing  impaired  lack  the  aptitudes  required  for  artistic  development.  Bunch  claims 
this  is  not  due  to  lack  of  potential,  but  rather  the  lack  of  opportunities  to  develop  the  potential  required  for  these  aptitudes.  In  his  opinion,  if  these 
pupils  are  given  the  opportunity  to  develop  their  potential  and  the  teachers  are  provided  with  methods  of  encouraging  creative  thinking,  hearing 
impaired  children  will  progress  and  reach  the  level  of  hearing  children.  He  perceives  music,  dance,  drama  and  visual  art  as  creative  tools  which 
can  be  used  to  develop  language  and  cognitive  skills  such  as:  problem  solving,  abstract  thinking,  clarity  of  thought,  developing  imagination,  etc. 
Upon  examination  of  various  teaching  programs.  Bunch  found  three  basic  goals  for  the  teaching  of  art  to  hearing  impaired  children: 
strengthening  visual  awareness;  developing  pre-artistic  abilities;  developing  imagination. 

In  recent  years,  one  can  notice  an  active  approach  of  intervention  in  the  cognition  of  the  deaf,  in  an  effort  to  improve  their  intellectual 
functioning.  At  the  root  of  this  trend  is  the  belief  that  deaf  students  possess  the  same  range  of  intellectual  potential  as  hearing  pupils.  They  can 
reach  this  potential  if  their  environment,  their  instruction  and  the  tools  employed  are  appropriate  and  encourage  learning.  In  addition,  the 
researchers  leading  this  approach  point  out  the  great  importance  of  intervention  programs  for  the  improvement  of  the  cognitive  achievements  of 
the  hearing  impaired  (Gruler  & Richard,  1990;  Huberty  & Roller,  1984;  Martin,  1991). 

Many  other  researchers  note  the  close  link  between  creativity  and  imagination.  The  ability  to  imagine  and  transform  images  is  important 
in  many  areas,  including  creativity  (Kaufman,  1985;  Kosslyn,  1980;  Shepard  & Metzler,  1971).  Singer  (1966)  found  in  his  study,  that  children 
with  developed  imaginations  were  more  creative  than  children  with  poor  imaginative  ability.  That  is  to  say,  the  ability  to  imagine  different 
scenes  and  animals,  the  ability  to  deal  with  a wide  range  of  changes  and  imagined  transformations  and  the  ability  to  be  flexible  in  thought— all 
indicate  a tendency  toward  creativity. 

Studies  have  thus  proven  that  hearing  impaired  children  tend  to  be  more  concrete  and  rigid  in  their  thought  processes.  They  usually 
choose  one  familiar  means  of  solving  problems  and  use  it  to  deal  with  most  of  the  problems  that  they  encounter.  Tlie  purpose  of  this  study  was  to 
prove  that  it  is  possible  to  improve  the  flexibility  of  thinking  in  hearing  impaired  children  with  the  help  of  a cutting  edge  technology — Virtual 
Reality  (VR). 

Pantelidis  (1995)  defines  VR  as  an  interactive  multimedia  environment,  based  on  the  computer,  in  which  the  user  is  assimilated  into,  and 
becomes  an  active  participant  in  the  virtual  world.  This  technology  can  present  information  in  a three  dimensional  format  in  real  time  so  that  the 
user  becomes  an  active  participant  in  the  environment  which  communicates  interactively  without  the  use  of  words.  VR  makes  it  possible  to 
convert  the  abstract  thinking  into  more  concrete  by  providing  a perspective  on  processes  which  is  not  possible  in  the  real  world  (Darrow,  1 995; 
Durlach  & Mavor,  1995;  Osberg,  1995;  Pantelidis,  1995). 

Virtual  Reality  is  a new  thinking  tool  which  can  be  used  without  language,  and  it  might  become  an  appropriate  work  tool  for  hearing 
impaired  children. 


Subjects 

A pilot  study  was  carried  out,  in  order  to  validate  the  Virtual  Reality  game  as  a research  tool.  The  pilot  study  included  10  hearing 
impaired  children,  who  achieved  an  average  score  of  4.2  out  of  a possible  100  points. 

The  study  examined  44  hearing  impaired  children  between  the  ages  of  8-1 1 (average  age  9.3)  who  had  middling  to  severe  hearing  loss, 
with  an  average  hearing  loss  of  88.62  dB.  They  had  no  additional  problems.  The  children  studied  in  integrated  classes  in  the  only  two  schools 
under  Ministry  of  Education  supervision  in  the  central/  Tel  Aviv  school  district.  In  these  schools  hearing  impaired  children  study  in  small 
separate  classes,  but  take  part  in  general  school  activities  and  are  integrated  into  some  of  the  classes  together  with  hearing  children. 

In  addition,  we  chose  16  hearing  children  in  order  to  establish  whether  hearing  impaired  children  do  achieve  poorer  results  than  hearing 
children,  in  their  ability  to  think  flexibly.  The  ages  of  the  hearing  children  ranged  from  8-10  years  old  (average  age  8.8).  The  sample  included  a 
total  of  60  children  according  to  the  following  divisions: 

* 2 1 hearing  impaired  children  served  as  the  experimental  group. 

* 23  hearing  impaired  children  served  as  the  control  group. 

* 1 6 hearing  children  served  as  an  additional  control  group. 


Procedure 

Each  subject  in  the  experimental  group  played,  on  his  own,  a 3D  Tetris  game,  once  a week,  for  15  minutes,  over  a period  of  three  months. 
The  hearing  impaired  control  group  played,  during  the  same  time  period,  a 2D  Tetris  game.  The  control  group  of  the  hearing  children  did  not 
experience  any  intervention  at  all. 

The  experimental  group  and  the  hearing  impaired  control  group  were  tested  before  and  after  the  intervention  in  the  sub-test  “Circles”  - 
Torrance  (1966).  This  test  was  used  in  order  to  study  whether  practice  in  the  rotation  of  three  dimensional  objects,  which  requires  the  ability  to 
view  objects  from  different  angles,  will  have  an  influence  on  the  flexibility  of  thinking  in  the  subjects.  The  test  includes  verbal  and  non-verbal 
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tasks.  We  used  the  non-verbal  tasks  owing  to  the  verbal  insufficiency  of  the  subjects.  This  test  has  been  carried  out  many  times  over  the  years 
and  has  received  the  high  score  of  90  in  reliability  (Torrance,  1966). 

The  instructions  for  the  test  were  given  orally,  accompanied  by  the  use  of  sign  language,  in  order  to  ensure  that  all  the  children  understood 
them  fiilly.  The  hearing  children  were  tested  only  once. 


Virtual  Boy -Nintendo 

The  game  used  in  this  study  is  an  interactive  VR  game,  with  a distinctive  system  which  can  create  a dramatic  three-dimensional  world. 
Tlie  software  included  three  games  (Tetris,  Puzzle,  Center-Fill),  in  all  of  which  the  objective  was  to  carry  out  certain  demands  via  control  over 
three  dimensional  blocks:  the  subject  had  to  fill  a three  dimensional  block  with  various  shapes  made  up  of  smaller  blocks.  The  subject  had  to  put 
the  dropping  blocks  in  the  right  place,  and  accordingly,  accumulate  points.  In  order  to  accumulate  more  points,  the  user  must  act  accurately  and 
rapidly.  This  three  dimensional  VR  game  presents  to  the  user  many  ways  of  solving  the  problem.  Because  of  the  variety  and  quantity  of 
possibilities,  it  is  usually  not  possible  to  remember  exactly  how  the  problem  was  previously  solved,  and  the  user  solves  the  problem  placed 
before  him  differently  each  time  he  plays  the  game.  Thus  he  learns  that  there  are  many  and  varied  solutions. 

Figure  1:  Virtual  Boy- Nintendo  1995 


Results 

The  research  suggested  that  a clear  difference  would  be  found  between  the  experimental  group  of  hearing  impaired  children  and  the 
control  group  of  hearing  children  in  their  ability  to  think  flexibly  before  practicing  spatial  rotation,  by  means  of  the  VR  game.  After  the  practice, 
in  contrast,  the  ability  to  think  flexibly  improved  in  the  experimental  group  to  such  an  extent  that  no  clear  difference  was  found  between  this 
group  and  the  control  group  of  hearing  children.  That  is  to  say,  the  scores  of  the  hearing  impaired  children  in  the  experimental  group  were  similar 
to  tliose  of  the  hearing  children  in  this  test.  In  order  to  verify  this,  we  conducted  a one-way  analysis  of  variance. 

Table  1 presents  the  averages  in  the  measurement  of  flexibility  of  thinking  in  the  three  research  groups-the  experimental  group  of  hearing 
impaired,  the  control  group  of  hearing  impaired  and  the  control  group  of  hearing  children,  and  the  analysis  of  the  variations  among  the  groups. 

Table  number  1 : Results  of  the  one-way  variance  test  for  the  calculation  of  averages,  standard  deviation  and  variance  analysis  of  flexible 
thinking  by  research  groups  (experimental,  control  hearing  impaired  and  control  hearing)  and  by  time. 


Time 

HI  experimental 

HI  control 

Hearing 

Model  P,F 

Group  P 

control* 
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Before 

Average 

7.05 

5.91 

23.00 

F(2,57)=l  77.92 

P(l,2)=n.s. 

P<0.001 

P(l,3)<0.001 

P(2,3)<0.001 

Standard 

Deviation. 

2.85 

3.50 

2.37 

Sample  Size 

21 

23 

16 

After 

Average 

18.10 

5.96 

23.00 

F(2,57)=l  02.04 

P(l,2)<0.001 

P<0.001 

P(l,3)<0.001 

P(2,3)<0.001 

Standard 

Deviation. 

5.76 

3.47 

2.37 

Sample  Size 

21 

23 

16 

• HI=  Hearing  Impaired. 

* Comment:  the  control  group  of  hearing  children  took  the  tests  once  only;  that  is  to  say,  the  data  in  the  table  was  copied  from  “before”  to 
“after”. 

Figure  2 presents  the  average  scores  in  flexible  thinking  of  the  three  research  groups  - experimental,  control  hearing  impaired  and  control 
hearing,  before  intervention. 


Figure  2:  Averages  of  flexible  thinking  according  to  research  groups  before  intervention 
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Figure  # 3 shows  the  averages  in  flexible  thinking  measured  after  the  intervention. 

Figure  number  3:  Averages  in  flexible  thinking  according  to  research  groups  after  the  intervention. 


A look  at  Table  1 and  Figures  2 and  3 indicate  that  prior  to  the  practice  there  was  a considerable  gap  in  flexibility  of  thinking  between  the 
research  group  of  hearing  impaired  children  (both  the  experimental  group  and  the  control  group)  and  the  control  group  of  hearing  children.  The 
difference  favored  the  hearing  children.  No  considerable  difference  was  found  between  both  research  groups  of  hearing  impaired  children 
(experimental  and  control  groups).  In  contrast,  after  the  practice,  a clear  difference  was  found  between  the  experimental  hearing  impaired  group 
and  the  hearing  impaired  control  group  in  their  ability  to  think  flexibly,  favoring  the  experimental  group.  No  clear  difference  was  found  between 
the  experimental  group  and  the  control  group  of  hearing  children;  that  is  to  say,  the  children  in  the  experimental  group  improved  their 
achievements  significantly  and  reached  the  level  of  the  hearing  children  in  this  index. 


Discussion 

One  of  the  goals  in  the  education  of  hearing  impaired  in  Israel  and  throughout  the  world  is  the  emphasis  on  the  importance  of  nurturing 
thinking  in  these  children.  Studies  have  found  that  the  functioning  of  the  hearing  impaired  improved  after  appropriate  learning,  training  and 
practice  (Gruler  & Richard,  1990;  Martin,  1991).  In  addition,  the  existing  programs  of  intervention  do  not  exploit  the  vast  possibilities  of 
modem  technology,  especially  the  modem,  attractive  technology  - VR.  This  study  is  unique  due  to  the  use  it  makes  of  the  virtual  game,  which 
trains  in  spatial  rotation  as  a means  of  improving  flexibility  of  thinking  in  hearing  impaired  children.  To  the  best  of  our  knowledge,  this  is  the 
first  experiment  of  its  type  to  exploit  the  advantages  of  VR  technology,  and  of  this  game  specifically,  as  a means  of  intervention  for  nurturing 
the  cognitive  skills  of  the  hearing  impaired  population. 

This  study  found  a clear  difference  in  the  ability  to  think  flexibly  between  hearing  impaired  children  and  hearing  children  before 
practicing,  to  the  advantage  of  the  hearing  children.  This  finding  is  reinforced  in  previous  studies  which  found  that  hearing  impaired  children 
possess  lesser  ability  in  creative  imagination  and  have  a tendency  to  rigidity  in  their  thinking  (Saraev  & Koslov,  1993).  After  practice,  the 
children  in  the  experimental  group  improved  in  their  ability  to  think  flexibly  with  the  help  of  the  3D  VR  game,  and  no  marked  difference  was 
found  between  them  and  the  control  group  of  hearing  children.  In  contrast,  the  control  group  of  hearing  impaired  children  continued  to  score 
poorly  and  the  gap  between  them  and  the  hearing  children  did  not  narrow.  In  other  words,  the  hearing  impaired  children  who  played  with  the  VR 
game  reached  the  level  of  the  hearing  children.  This  finding  parallels  that  of  Bunch  (1987)  who  claimed  that  if  hearing  impaired  children  are 
afforded  opportunities  to  develop  their  potential  and  the  teachers  are  provided  with  methods  of  encouraging  creative  thinking  - hearing  impaired 
children  will  progress  and  will  reach  the  level  of  hearing  children. 

The  results  indicate  a clear  advantage  of  3 dimensional  VR  (received  by  the  experimental  group)  over  the  conventional  2 dimensional 
intervention  (received  by  the  hearing  impaired  control  group).  It  is  reasonable  to  assume  that  these  findings  were  obtained  due  to  the  differences 
between  the  two  types  of  practice.  The  children  in  both  groups  played,  during  the  same  period  of  time,  rotation  games  via  Tetris,  with  only  one 
difference  between  the  two  groups  - 3D  VR  versus  a 2D  game. 

A logical  explanation  of  these  findings  is  found  in  the  nature  of  VR  technology  and  in  earlier  studies.  This  technology  creates  a “pre- 
symbolic”  form  of  communication,  and  its  users  can  communicate  with  imaginary  worlds  without  the  use  of  words.  Thus,  a world  is  created 
charged  with  sights,  voices  and  sensations  which  surpass  syntax  and  language  (Passig,  1996).  The  hearing  impaired  children  who  used  this 
technology  were  able  to  realize  their  hidden  potential  without  linguistic  or  auditory  limitations.  VR  technology  does  not  limit  the  player  in  either 
the  manner  in  which  information  is  presented  or  in  his  movements,  and  the  user  is  able  to  immerse  himself  in  the  learning  environment 
(Pantelidis,  1995).  In  this  virtual  manner,  the  hearing  impaired  users  completely  immersed  themselves  in  the  game.  Tliey  felt  as  though  they 
themselves  were  moving  the  blocks,  searching  for  the  right  ones  and  rotating  them.  In  other  words,  the  abstract  became  less  vague  and  more 
concrete.  Various  studies  in  the  field  of  VR  also  found  that  this  immersion  serves  to  broaden  the  interface  with  the  senses  and  also  improves  the 
ability  to  understand  abstract  concepts  by  converting  them  to  more  concrete  ones  (Darrow,  1995;  Osberg,  1995).  Another  unique  quality  of  the 
technology  is  that  VR  causes  the  user  to  be  especially  active.  The  increased  liveliness  and  level  of  interactivity  causes  the  user  to  become  a part 
of  the  virtual  world.  This  tool  is  able  to  present  information  in  three  dimensions  and  in  real  time.  This  permits  the  user  to  be  an  active  participant 
in  the  environment  and  not  merely  a passive  observer  (Bricken  & Byrne,  1992;  Heim,  1992;  Osberg,  1995;  Powers  & Darrow,  1994).  Hearing 
impaired  children  require  more  active  involvement  in  the  learning  process  than  hearing  children  (Marzam,  1 998). 

Another  fitting  explanation  of  these  findings  is  the  fact  that  this  technological  tool  is  fun  and  motivates  the  user.  Studies  have  shown  that 
children  who  use  VR  enjoy  it  and  wish  to  continue  to  learn  more  by  using  it  (Bricken  & Byrne,  1992;  Talkmitt,  1996).  It  would  appear  that  the 
high  level  of  motivation  of  the  children  studied  led  to  their  continued  participation  in  the  intervention  program  and  their  success  in  it. 


Summary 

This  study  proved  that  a significant  improvement  in  flexible  thinking  took  place  in  hearing  impaired  children  due  to  the  use  of  a 3D 
virtual  reality  game.  Beyond  this  contribution,  important  in  itself,  the  biggest  contribution  is  the  advancement  of  this  field  to  the  point  where 
hearing  impaired  children  reached  the  level  of  hearing  children.  In  the  light  of  the  findings  of  this  study,  and  its  low  cost,  we  recommend  that 
schools  in  which  hearing  impaired  children  study,  purchase  this  hardware  and  software. 
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Abstract:  Web  based  technologies  and  efficient  techniques  on  the  LAN  and  the 
desktop,  can  be  combined  for  the  development  of  a single  modular  and  scalable 
educational  system,  capable  of  adopting  to  and  functioning  in  different  learning 
environments:  stand-alone  use,  use  in  a LAN  enhanced  classroom  and  distance 
learning  on  the  Web.  Such  a system  is  presented  in  this  paper.  The  design  of  the 
system  addresses  well-defined  educational  and  pedagogical  goals  and  attempts  to 
facilitate  different  computer  cognition  levels.  The  current  implementation 
demonstrates  the  use  of  the  system  with  educational  material  designed  especially  for 
teaching  Ancient  Greek  History  to  7*^  grade  students. 


1 Introduction 

ISTOPOLIS  is  a hypermedia  based  educational  system  initially  designed  for  courses  in  history,  culture 
and  related  fields.  Emphasis  has  been  placed  on  deployment  of  the  system  in  the  classroom.  The  design  of 
ISTOPOLIS  attempts  to  address  fundamental  educational  issues  and  goals,  including  transformation  of  available 
information  into  knowledge,  practicing  basic  skills  and  dealing  with  low  and  / or  highly  variable  computer 
cognition  level  on  the  part  of  the  students. 

The  system  provides  its  users  (educators,  students  and  content  providers)  with  a broad  set  of  services  over 
media-rich  educational  material,  in  a highly  configurable  manner.  The  material  used  may  cover  many  different 
knowledge  areas.  Educational  material  is  presented  to  the  end  user  in  the  form  of  hypermedia  documents.  Each 
document  covers  a small  subsection  of  the  target  knowledge  area  and  constitutes  an  information  item.  The 
collection  of  information  items  offered  to  the  user  follows  a concrete  data  organization  scheme.  Efficient  access 
to  the  document  base  is  facilitated  through  different  views  of  this  organization  scheme.  Each  view  is 
implemented  as  a highly  interactive  access  tool,  which  not  only  helps  the  user  locate  documents,  but  also  places 
the  documents  within  specific  contexts,  leading  to  a better  comprehension  of  the  material.  These  views  include  a 
thematic  categorization,  geographical  and  chronological  navigation  systems,  and  a simple  to  use  search  facility. 
It  follows  that  the  content  of  the  system  consists  of  a set  of  independent  and  multiply  structured  information 
items  that  users  can  access.  Additional  content  may  be  accessed  from  related  web  sites. 

Access  to  the  information  items  as  well  as  to  these  tools  may  be  provided  to  the  end-users  either  through  a 
standard  web  browser,  or  through  a specialized  client.  The  client,  apart  from  access  to  the  content,  enables  all 
users  to  compose  projects  (small  sets  of  hypermedia  documents  in  the  form  of  an  essay)  by  reusing  parts  of  the 
content.  Additionally,  it  enables  teachers  to  create  their  own  views  of  the  content  (in  the  form  of  guided  tours) 
and  teachers  and  students  alike  to  playback  these  tours.  Finally,  the  system  provides  teachers  with  a tool  for 
creating  and  grading  tests.  Each  test  can  incorporate  different  types  of  questions  and  can  be  completed  and 
submitted  on-line  by  the  students.  Access  to  the  system  is  controlled  through  a server-based  mechanism  that 
authenticates  users  and  assigns  them  their  respective  workspace. 

Many  of  the  services  built  in  the  client  and  the  server  are  by  now  well  established  services  and  are  very 
popular  for  accessing  Web  content.  Innovation  of  this  system  lies  in  the  way  that  these  services  interrelate  and 
integrate  into  a single  system,  with  a unified  front-end  for  the  end-user.  Porting  to  other  languages  is 
straightforward  and  an  English  version  of  the  system  is  now  under  development. 


2 Pedagogical  and  educational  issues  and  how  they  are  addressed 

There  is  a great  number  of  pedagogical  and  educational  issues  regarding  multimedia  educational 
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software.  In  this  section  we  place  emphasis  on  some  essential  requirements,  particularly  in  the  context  of  "human 
sciences"  and  junior  high  school  students.  We  also  discuss  ways  in  which  ISTOPOLIS  addresses  these  issues 
(Papatheodorou,  1998). 


2.1  Transforming  Information  into  Knowledge 

An  important  goal  in  developing  an  educational  application  is  transforming  information  into  knowledge. 
According  to  (Rohrs,  1984),  students  should  be  allowed  to  express  “their  creativity,  one  of  the  most  essential 
goals  in  education”,  while  the  goal  is  not  to  “turn  students  into  small  libraries  but  to  allow  them  to  take  active 
part  in  the  process  of  knowledge”  (Bruner,  1975).  It  is  also  well  established  that  notions  are  not  conquered  by 
students  through  definitions,  but  through  experience  and  “trial  and  error”  techniques  (Vygotsky,  1962). 

Addressing  this  goal  is  based  on  the  use  of  a set  of  information  retrieval  tools  (Access  Tools)  that  do  not 
only  enable  the  user  to  locate  information  (such  as  in  a search  system),  but,  at  the  same  time,  place  the 
information  within  complementary  contexts  (time,  place  and  subject).  The  Access  Tools  are  coupled  with  a 
facility  for  the  students  to  reuse  located  information,  combine  it  with  their  own  narration  and  personal  views  and 
transform  it  into  small  essays. 

2.2  Practicing  basic  skills 

Some  of  the  most  important  skills  that  students  should  practice  are  briefly  discussed  below,  together  with 
the  ways  that  ISTOPOLIS  attempts  to  accentuate  them.  Most  of  these  skills  are  of  particular  significance  in  the 
case  of  humanities  and  especially  history  (Curtis  (1994)  and  Culpin  (1994)).  Successful  practicing  depends  both 
on  system  qualities  and  on  the  way  the  software  itself  is  used  within  and  outside  the  classroom,  during  and  after 
normal  teaching  hours. 

Working  alone  and  within  a group.  It  is  essential  that  students  learn  to  "work  as  a team"  in  order  to  be 
"able  to  adapt  in  a continuously  changing  world"  (Rohrs,  1984).  ISTOPOLIS  allows  students  to  work  in  groups 
by  letting  teachers  assign  a unified  workspace  to  students  belonging  to  the  same  working  group.  In  this 
workspace,  projects  may  be  collectively  developed.  A student  may  also  use  the  system  individually,  either  in 
classroom  or  at  home,  since  the  system  can  also  operate  in  standalone  mode.  Moreover,  students  or  groups  may 
exchange  pages  or  bookmarks  on  pages  via  email. 

Identifying  reliable  information  and  distinguishing  between  facts  and  opinions,  evaluating  sources  and 
selecting  related  information  from  a variety  of  sources.  ISTOPOLIS  enables  students  to  develop  projects,  using 
information  available  locally  and  on  the  Web.  This  way,  it  offers  access  to  a great  variety  of  sources,  which 
deviates  from  the  common  single  textbook  approach.  Material  provided  with  the  application  is  developed  and 
tested  by  experts  and  can  be  a-priori  considered  reliable.  The  fact  that  this  information  follows  a common 
presentation  template  makes  it  visually  different  from  work  produced  by  other  students  as  well  as  from  other 
material  gathered  through  the  Web. 

Understanding  of  terms.  Terms  that  the  students  are  anticipated  to  be  unfamiliar  with  are  visually  marked 
on  the  screen.  Explanation  of  a term  is  given  as  a popup  activated  by  placing  the  mouse  pointer  over  the  term. 
Elaboration  of  more  significant  terms  is  provided  in  related  pages  within  the  educational  material. 

Locating  and  recalling  information.  ISTOPOLIS  places  the  content  within  specific  time  and  place 
contexts  and  provides  clear  thematic  categorizations,  thus  offering  considerable  assistance  in  locating  and 
recalling  interrelated  information.  Recalling  information  is  accomplished  through  the  same  tools  students  use  to 
locate  the  material. 

Describing,  narrating,  interpreting,  comparing,  drawing  conclusions  from  information  contained  in  texts 
and  expressing  opinion.  Through  authoring  a project,  students  are  given  the  chance  to  combine  various  tasks, 
such  as  composing  and  serializing  facts  presented  within  the  educational  material,  narrating  or  describing 
situations  and  facts  and  expressing  their  own  views.  Comparison  of  information  pieces  found  in  the  material  is 
made  easier  by  the  fact  that  these  pieces  may  be  very  easily  inserted  into  the  student’s  project. 


2.3  Computer  cognition  issues 

Some  important  issues  involve  the  computer  cognition  level  of  the  targeted  users.  One  issue  is  the  low 
level  of  familiarity  with  computers  for  the  majority  of  students,  especially  in  low-income  cases  and  rural  areas. 
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Thus,  an  important  requirement  is  that  the  system  is  easily  useable  by  inexperienced  users  as  well.  In  addition,  it 
should  help  users  to  acquire  and  practice  standard  computer  operation  skills  through  a widely  used  environment 
and  user  interface,  A second  issue  is  the  large  difference  in  computer  cognition  levels  among  students  in  the 
same  class.  The  problem  is  more  evident  in  urban  environments  where,  among  inexperienced  students,  there  are 
a few  with  strong  computer  experience.  The  system  should  attempt  to  provide  "equal  opportunity"  for  all. 

One  approach  that  ISTOPOLIS  uses  to  deal  with  these  issues  is  to  offer  different  modes  of  operation, 
corresponding  to  different  levels  of  computer  experience.  The  simplest  way  for  a student  to  use  the  system  is 
through  the  guided  tour  operation  mode  (sequential  navigation  through  a series  of  “pages”  selected  by  the 
teacher  from  the  educational  material  available).  In  a higher  mode,  the  system  provides  the  option  for  the  student 
to  customize  his/her  own  exploration  of  the  educational  material.  This  is  accomplished  through  hyperlinks 
embedded  in  the  content  and  deploying  the  Access  Tools  described  in  the  next  section.  In  this  mode,  the  student 
is  required  to  take  actions  that  are  normally  met  in  common  Web  browsing  activities.  In  a third  mode,  the  student 
may  reuse  portions  of  the  educational  material  and  compose  his  / her  own  hypermedia  documents.  This  process 
involves  basic  word  processing  skills  and  use  of  drag’  n’  drop  operations.  The  teacher  may  decide  on  the  use  of 
any  combination  of  these  modes,  according  to  the  level  of  the  classroom. 

Another  way  in  which  ISTOPOLIS  attempts  to  deal  with  these  issues  is  through  a simple-to-use, 
consistent  user  interface.  Browsing  and  authoring  are  implemented  by  integrating  a common  Web  browser  and  a 
word  processor.  Thus,  the  user  of  the  system  is  assisted  by  and  assists  training  in  such  basic  tools.  Access  to 
content  and  services  is  provided  through  a unified  workspace,  implemented  by  a specialized  client  designed  as  a 
single-document  interface  (SDI)  application.  The  user  is  not  required  to  wander  through  windows  or  switch 
between  different  applications  and  confusion  between  application  oriented  and  content  oriented  operations  is 
largely  eliminated. 

An  additional  issue  strongly  related  to  the  user's  experience  is  the  use  of  navigation  tools.  In  hypermedia- 
based  systems,  the  need  for  navigation-aiding  tools  complementary  to  following  embedded  hyperlinks  is 
practically  self-evident  (Christodoulou,  1998).  Several  tools  that  are  based  on  navigation  structures  have  been 
proposed  (Schwabe  1995).  Initial  findings  indicate  that  inexperienced  users  under-utilize  navigational  tools 
based  on  high  levels  of  abstraction  e.g.  concept-maps  (Zeiliger,  1997).  Tools  based  on  structures  and  models 
with  which  students  are  not  familiar  (e.g.  graphs)  would  increase  the  cognition  overhead  required  to  exploit  these 
tools.  Instead,  ISTOPOLIS  provides  access  to  multiply  structured  information  through  simple  navigation  tools 
(e.g.  maps),  consistent  with  the  students'  experience. 


2.4  Other  issues 

Teacher  related  issues.  ISTOPOLIS  supports  teachers  in  various  stages  of  the  educational  process,  such 
as  presentations  and  student  evaluation.  The  teacher  is  provided  with  the  ability  to  instruct  students  to  browse 
through  the  content  or  follow  a guided  tour.  ISTOPOLIS  enables  the  teacher  to  make  additional  content  available 
to  the  students  through  the  system,  by  authoring  and  publishing  his/her  own  projects.  Evaluation  of  students' 
performance  is  assisted  through  online  inspection  of  student  projects,  facilitation  of  test  development  and  online 
test  grading. 

On-line  / off-line  operation.  A user  may  autonomously  use  the  system  from  home  with  or  without  an 
Internet  connection  (stand-alone  or  on-line  operation). 

Updating  the  content.  In  the  current  implementation,  updates  of  the  content  and  of  basic  customization  of 
the  Access  Tools  are  supported.  This  includes  re-indexing  of  content  for  the  search  system,  changes  on  objects 
that  appear  on  maps,  addition  of  thematic  categories,  etc. 


3 Functional  speciflcation  of  the  system 

Users  of  ISTOPOLIS  may  be  classified  into  end-users  (teachers  and  students)  and  content  providers.  The 
system  may  be  deployed  in  the  classroom  LAN,  over  a wide  area  connection  with  a remote  server,  or  as  a stand- 
alone application.  ISTOPOLIS  offers  two  fundamental  services  and  a complementary  one  for  end-users,  which 
constitute  the  end-user  segment  of  the  system:  locating  educational  material;  reuse  of  material  for  creating 
projects;  and  creating  and  conducting  tests.  A back-end  subsystem,  ("preparation  segment"),  allows  the  content 
provider  to  enter  the  data  and  customize  the  functions  that  enable  end-users  to  locate  material. 
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The  system  is  based  on  educational  material  presented  to  the  user  in  the  form  of  hypermedia  documents, 
each  document  being  an  autonomous  information  item.  Apart  from  the  content  provided  with  the  system,  the 
user  may  access  HTML  pages  from  the  Web. 

The  easiest  way  for  students  to  access  information  is  through  a guided  tour  created  by  the  teacher.  In  this 
mode  of  operation  and  by  using  a small  set  of  buttons,  the  student  can  sequentially  navigate  through  a series  of 
content  pages  arranged  by  the  teacher.  The  teacher  can  locate  the  information  to  include  in  the  guided  tour  by 
using  the  Access  Tools  described  below  and  can  create  the  guided  tour  through  the  use  of  a pertinent  tool 
incorporated  in  his/her  workspace.  The  user  can  leave  the  guided  tour  and  browse  the  content  through  embedded 
hyperlinks  and  the  use  of  Access  Tools  and  return  to  the  flow  of  the  tour  by  clicking  a button. 

Access  Tools  presently  include  four  basic  tools  available  to  the  end  user  and  are  briefly  outlined  as 
follows.  All  information  items  are  associated  with  keywords  and  are  loosely  classified  according  to  three  basic 
classification  criteria:  time,  place  and  subject.  Data  directly  related  to  the  content  (such  as  title,  text,  images  and 
keywords)  as  well  as  fine  and  coarse-grained  classification  data  follow  a single,  simple  and  concrete  data 
organization  scheme.  This  makes  it  possible  to  present  three  different  views  of  the  content  as  main  branches  of  a 
subject  catalogue.  The  subject  catalogue  is  implemented  as  an  Interactive  Thematic  Catalogue  (ITC)  system 
described  in  (Styliaras,  1998).  For  the  chronological  and  geographical  classifications,  both  a coarse-grained 
(based  on  eras  and  greater  areas)  and  a fine-grained  (based  on  years  and  specific  map  coordinates) 
characterization  of  content  is  given.  Fine-grained  classifications  are  used  to  support  access  to  the  chronological 
and  geographical  views  through  visual  tools:  a timeline  and  a map-based  navigation  tool.  All  three  tools,  subject 
catalogue,  timeline  and  map-based  tool,  support  searching  or  filtering.  Map-based  navigation  provides  filtering 
based  on  time  and  subject  related  criteria.  Finally,  keywords  correlated  with  the  educational  material  are  used 
within  a search  tool  that  apart  from  searches  on  sets  of  keywords  allows  for  full  text  searches  on  the  material. 

The  specific  chronological,  thematic  and  geographic  context  of  each  information  item  is  exposed  to  the 
user  as  a hyperlink  to  the  corresponding  category.  Exposing  the  context  of  each  information  item  in  a typical 
fashion  helps  the  user  recall  the  information  at  a later  stage  (e.g.  when  authoring  a project).  Using  a hyperlink  to 
the  corresponding  categories  (and  tools)  enables  the  user  to  locate  relevant  information  within  specific  context. 
Customization  of  the  classifications  and  data  entry  of  the  actual  educational  material  is  achieved  by  the 
“content  provider”  through  a back-end  subsystem  (preparation  segment).  Data  entry  for  the  system  is  carried  out 
at  the  same  time  for  all  four  tools.  Update  of  a single  piece  of  information  automatically  updates  the  behavior  of 
the  Access  Tools.  It  should  be  stated  that  the  content  provider  need  not  be  confined  in  the  three  classifications 
chosen  for  the  current  implementation  of  the  system.  In  order  to  facilitate  use  in  different  knowledge  domains, 
creation  of  various  classifications  is  possible  for  the  content  provider  through  the  back-end  environment. 

Reuse  of  the  educational  material  is  achieved  within  the  specialized  client  through  storing  material  and 
incorporating  it  into  a project.  The  user  can  store  text  excerpts,  images,  bookmarks  and  hyperlinks.  Storing  a part 
of  information  is  implemented  through  a simple  process,  which  consists  of  selecting  the  information  item, 
dragging  it  and  dropping  it  onto  the  client's  inventory  area.  Bookmarks  are  stored  in  the  same  way,  by  selecting 
and  picking  up  a control  next  to  the  page’s  title.  All  items  in  the  inventory  area  are  represented  by  icons.  Users 
may  store  bookmarks  or  material  not  just  from  the  content  provided  with  the  system,  but  also  from  any  HTML 
page  on  the  Web,  exploiting  thus  a variety  of  sources. 

A project  is  organized  in  pages,  which  are  represented  by  icons  in  a page-index  area.  To  insert  a piece  of 
text  or  an  image,  the  user  selects  a page  in  editing  mode  (preview  of  pages  is  also  supported),  drag  the  icon  of 
the  text  or  image  and  drop  it  onto  the  page.  Users  may  create  hyperlinks  by  selecting  the  source  of  the  link  on  the 
page  being  edited  (text  or  image)  drag  a link  or  a bookmark  from  the  inventory  or  a page  of  their  own  from  the 
page  index  and  drop  it  on  a “link  creation  target  area”.  When  editing  a page,  users  can  write  their  own  text, 
modify  the  text  they  have  included  and,  of  course,  write  their  own  text.  To  keep  use  of  the  system  simple,  a small 
set  of  editing  functions  is  provided:  it  includes  character  and  paragraph  formatting  and  application  of  predefined 
styles  that  help  the  user  translate  contextual  information  (such  as  project  or  page  heading)  into  visual  (different 
but  concise  visual  formatting). 

A brief  overview  of  the  overall  architecture  and  system  building  blocks  is  presented  in  Figure  1 . Content 
and  access  tools,  guided  tours  (creation  and  playback),  project  authoring  and  publishing  and  completion  of  tests 
are  available  to  the  user  through  a specialized  client  that,  among  others,  incorporates  functionality  of  a web 
browser  and  an  HTML  Editor.  Separate  (for  the  current  implementation)  tools  facilitate  creation  of  tests  and 
support  the  content  preparation  and  Access  Tools  customization  process.  Content  and  access  tools  may  also  be 
available  to  the  user  through  a conventional  Web  browser.  In  the  current  implementation,  some  of  these 
functions  are  offered  only  for  Microsoft’s  Internet  Explorer. 

Within  the  client,  services  offered  are  grouped  into  operating  modes,  with  varying  levels  of  difficulty,  so 
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that  the  teacher  may  fine-tune  system  deployment.  Free  navigation  offers  use  of  the  Access  Tools,  bookmarking 
and  tests.  Guided  tours  offer  all  Free  navigation  services,  plus  a guided  tour  creation  tool  for  the  teacher  and  a 
guided  tour  playback  tool  for  the  students.  Project  authoring  offers  all  free  navigation  services,  plus  all 
operations  needed  for  authoring  projects. 

The  system  generally  follows  the  client-server  paradigm.  The  content  resides  on  the  server  side.  The  data 
organization  scheme  for  the  content  is  supported  by  an  RDBMS,  over  which  the  Preparation  Segment  of  the 
system  has  been  built.  The  biggest  part  of  functions  (including  the  Access  Tools)  is  also  implemented  on  the 
server  side.  In  order  to  offer  additional  services  to  the  teacher  (creation  of  guided  tours  and  tests)  the  system 
requires  that  the  user  be  authenticated.  This  process  authenticates  registered  users  of  the  application,  as  either 
teachers  or  students  (implying  different  functionality  of  the  client)  and  as  members  of  specific  working  groups  in 
the  case  of  students.  The  process  assigns  a working  area  in  which  users  store  their  work.  The  relative  location  of 
the  client  and  server  part  of  the  system  determines  whether  the  system  is  used  1)  over  a LAN,  2)  over  a remote 
connection  (on-line),  or  3)  as  a stand-alone  application  (off-line).  Special  care  has  been  taken  so  that  the  system 
may  be  easily  reconfigured  and  used  in  other  languages:  most  of  the  reconfiguration  needed  may  be  carried  out 
through  simple  changes  on  resource  files. 


Figure  1:  Architecture  of  ISTOPOLIS. 


4 Current  implementation  details  and  evaluation  results 

All  building  blocks,  demonstrated  in  Figure  1,  have  been  implemented  and  integrated  into  the  system, 
which  is  fully  operational.  The  current  implementation  has  focussed  mainly  on  the  system’s  operation  in  class. 
The  available  content  is  presently  targeting  7th  grade  students  and  covers  about  450  information  items  from 
Ancient  Greek  History  (specifically  the  Palaeolithic  period  to  1125  B.C.).  For  implementing  Web  browsing  and 
project-page  editing  functionalities,  Microsoft's  Internet  Explorer  and  Word  97  have  been  integrated  within  the 
system.  Architectural  and  implementation  details  may  be  found  in  (Papaterpos,  1999).  Figure  2 displays  how  the 
client  looks  like  in  project-creation  mode,  while  a user  is  locating  information  through  the  map-based  navigation 
tool.  A list  of  the  projects'  pages  appears  in  the  left  of  the  screen.  The  inventory,  where  chunks  of  text,  links  and 
images  are  collected,  appears  in  the  bottom. 

Currently,  the  system  is  undergoing  a systematic  on-site  evaluation  in  a set  of  Greek  High  schools.  It  is 
expected  that  the  on-site  evaluation  will  be  concluded  early  next  year  and  detailed  evaluation  statistics  will  be 
available  at  that  time.  Preliminary  in-house  testing  with  students  and  teachers  seems  to  validate  achievement  of 
most  of  the  requirements  set.  Students  relatively  unfamiliar  with  computers  easily  adjusted  to  the  gradually 
increasing  levels  of  difficulty  provided. 

From  the  testing  phase,  some  possible  enhancements  have  emerged.  For  example,  users  would  like  to  have  more 
material  from  different  sources.  Furthermore,  the  backend  should  be  made  available  to  teachers  in  order  to  have 
the  ability  to  add  more  material  (found  either  on-line  or  from  conventional  sources).  In  this  way,  there  will  be  no 
need  for  an  expert  content  provider  to  insert  the  data  into  the  system.  Concerning  the  guided  tours,  an  extra 
"presentation"  view  is  needed,  which  will  be  more  suitable  for  displaying  the  guided  tour  within  the  classroom. 
What's  more,  teachers  should  be  given  the  capability  to  annotate  pages  included  in  the  guided  tour.  Regarding 
the  subject  catalogue,  more  elaborate  filtering  could  be  applied  to  its  categories.  As  it  is  expected  that  groups  of 
students  will  use  the  system,  a better  collaboration  scheme  is  needed  for  students  working  within  the  group,  so 
that  it  won't  be  necessary  for  all  students  in  the  group  to  work  simultaneously. 
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Figure  2:  An  instance  of  ISTOPOLIS  in  action 
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Abstract;  This  paper  describes  an  interactive  multimedia  learning  environment  for  learning 
how  to  diagnose  fetal  abnormalities.  The  pedagogical  approach  adopted  in  the  design  of  the 
learning  environment  is  that  of  goal-based  scenarios.  The  paper  sets  out  the  theoretical 
underpinnings  of  this  approach  and  illustrates  the  learning  environment  by  means  of  a typical 
system  walk-through.  It  highlights  some  distinctive  features  of  the  system  as  well  as 
pedagogical  strengths  of  the  learning  environment.  A critique  of  the  learning  environment  is 
offered  with  respect  to  the  criteria  laid  down  for  the  design  of  goal-based  scenarios. 


Introduction 

There  is  considerable  evidence  today  indicating  that  the  knowledge  and  skills  of  experts  are  highly  situated  and 
case-based  (Kolodner  & Jona  1991).  This  is  especially  so  in  the  medical  domain  where  expertise  is  often 
characterized  as  being  data  driven  and  is  attained  through  exposure  to  a wide  range  of  cases  over  long  years  of 
practice. 

It  is  now  also  commonly  accepted,  especially  in  medical  education,  that  students  learn  more  effectively  by 
directly  participating  in  interactive  case-based  learning  scenarios  rather  than  by  passive  reading  and  observation 
(see,  for  example,  Barrows  1985).  Learning  by  doing  has  the  advantage  of  providing  for  deeper  engagement  in 
the  learning  process  and  superior  retention  and  transfer  of  knowledge.  Thus,  one  would  expect  medical  students 
to  learn  more  effectively  when  given  the  opportunity  to  acquire  clinical  diagnosis  and  problem  solving  skills 
using  the  'learning  by  doing'  approach  in  a context-rich  learning  environment.  In  addition,  experientially- 
grounded  learning  provides  a concrete  basis  for  subsequent  reflection  on  a student's  diagnosis  and  problem 
solving  actions.  Attempts  to  rethink  and  redesign  learning  experiences,  especially  technology-based  learning 
experiences,  occur  as  part  of  a broader  and  more  fundamental  shift  in  orientation  within  education  (e.g.  Schank 
1993/1994)  and  also  as  part  of  a shift  toward  constructivist  approaches  to  supporting  learning  (see,  for  example, 
Savery  & Duffy  1995). 

The  training  of  medical  general  practitioners  and  medical  students  in  the  management  of  fetal  abnormalities  has 
always  been  a difficult  matter  in  practice.  This  difficulty  stems  from  the  fact  that: 

a.  fetal  abnormalities  in  ultrasound  scans  of  pregnant  women  are  the  exception  rather  than  the  rule 

b.  the  repertoire  of  abnormalities  that  a student  has  the  opportunity  to  experience  firsthand  is  extremely 
limited 

c.  access  to  ultrasound  scanning  machines  and  training  time  on  such  machines  for  students  is  hard  to  come 
by  because  of  general  resource  scarcity 

To  overcome  the  above  difficulties,  we  have  embarked  on  the  development  of  a multimedia  goal-based  scenario 
for  learning  how  to  diagnose  and  manage  fetal  abnormalities. 


Theoretical  Background 
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A goal-based  scenario  is  a case-based  approach  to  learning.  The  approach  requires  instructors  to  design  situated 
and  authentic  problem  solving  activities  and  domain  contents.  Within  the  goal-based  learning  environment, 
students  are  set  specific  problem  solving  goals.  In  pursuit  of  these  goals,  they  perform  tasks  that  help  facilitate 
the  acquisition  of  the  target  skills.  In  so  doing,  they  also  learn  the  underlying  concepts  upon  which  the  target 
skills  are  based. 

The  goal-based  scenario  approach  to  the  design  of  learning  environments  requires  the  designer  to  define  a 
mission,  a mission  focus,  a cover  story,  and  appropriate  scenario  operations  (Schank,  Fano,  Bell,  & Jona 
1993/1994).  Briefly,  the  mission  refers  to  the  primary  goal  that  the  student  pursues  within  the  goal-based 
scenario.  The  mission  focus  refers  to  the  overall  organization  of  the  student’s  activities  within  the  goal-based 
scenario.  Mission  foci  may  be  of  different  orientations,  including  design,  discovery,  exploration,  and  control. 
The  cover  story  refers  to  the  premise,  designed  by  the  instructor,  under  which  the  mission  will  be  pursued. 
Scenario  operations  refer  to  the  specific  activities  that  the  student  performs  in  pursuit  of  a mission. 

There  are  seven  specific  criteria  that  instructional  designers  need  to  consider  when  designing  a goal-based 
scenario.  These  are: 

a.  thematic  coherence:  ensuring  that  the  process  of  achieving  the  goal  is  thematically  consistent  with  the 
goal  itself 

b.  realism  and  richness:  ensuring  that  the  learning  environment  amply  encompasses  authentic  tasks  in 
context-rich  settings 

c.  control  and  empowerment:  ensuring  that  students  are  able  to  take  control  and  feel  responsible  for  the 
completion  of  the  set  task 

d.  challenge  consistency:  ensuring  that  the  learning  environment  presents  a consistent  degree  of  challenge 
and  difficulty  over  the  entire  process  of  achieving  the  goal 

e.  responsiveness:  ensuring  that  time-critical  elements  of  the  environment  occur  in  realistic  simulated 
time,  and  providing  prompt  feedback 

f pedagogical  goal  support:  ensuring  that  the  learning  scenario  is  compatible  with  and  amply  supports  the 
acquisition  of  the  target  skills 

g.  pedagogical  goal  resources:  ensuring  that  materials  and  tools  needed  to  achieve  the  goal  are  available 
within  the  learning  environment 

In  practice,  the  extent  to  which  one  finds  it  easy  or  difficult  to  achieve  each  of  the  above  design  criteria  depends 
very  much  on  the  subject  domain  in  which  one  is  working. 


Project  Overview 

Our  research  project  has  two  main  parts.  The  first  part  focuses  on  helping  students  learn  how  to  diagnose  fetal 
abnormalities.  The  second  part  focuses  on  training  students  on  how  to  manage  and  counsel  patients  when 
abnormalities  are  discovered. 

The  first  part  of  the  project  makes  extensive  use  of  ultrasonography.  Ultrasound  images  of  the  unborn  fetus  are 
displayed  to  the  user,  typically  a trainee  gynecologist,  on  request.  These  images  can  either  be  digitized  videos  of 
ultrasound  investigations  or  they  can  be  still  image  "cuts"  (i.e.  cross-sections)  of  the  unborn  fetus  at  different 
body  locations  and  orientations.  Given  the  domain  of  diagnostic  ultrasonography,  we  have  made  extensive  use  of 
multimedia  and  created  a context-  and  media-rich  learning  environment  where  students  are  given  control  over 
what  they  wish  to  view  or  do  at  any  point  in  time.  When  the  user  has  reviewed  the  ultrasound  images  and  the 
patient’s  case  history  to  his  satisfaction,  he  is  requested  to  decide  whether  the  unborn  fetus  suffers  from  any 
abnormality,  and,  if  so,  identify  the  type  of  abnormality. 

Once  the  user  has  entered  a diagnosis,  the  system  makes  an  internal  check  to  verify  the  correctness  or  otherwise 
of  the  submitted  diagnosis.  If  the  submitted  diagnosis  is  incorrect,  the  system  commences  a sequence  of 
interaction  with  the  user.  This  sequence  is  designed  to  help  him  identify  the  cause  of  his  mistake. 

In  the  current  state  of  the  project’s  development,  the  rules  that  distinguish  different  abnormalities  related  to  the 
Central  Nervous  System  have  been  defined.  These  rules  are  stored  in  a database  that  the  system  refers  to  when 
trying  to  ascertain  the  correctness  or  otherwise  of  a student’s  response.  At  the  same  time,  a media  bank  of  images 
and  video  clips  has  been  created  for  use  as  presentation  stimuli.  Much  effort  has  been  expended  in  designing  and 
prototyping  the  interface  of  the  system,  with  a special  view  toward  task  support  and  learning  support. 


The  second  section  of  the  project  focuses  on  the  management  and  counseling  of  patients  when  a fetal 
abnormality  has  been  correctly  diagnosed.  Work  on  this  section  has  yet  to  commence. 


System  Description 

The  system  presents  users  with  the  ability  to  perform  "virtuar  ultrasound  investigations  of  women  who  are  24- 
weeks  pregnant.  This  is  a critical  stage  of  pregnancy  when  abnormalities  become  identifiable.  The  goal-based 
scenario  contains  many  different  cases.  Some  cases  have  single  abnormalities  while  others  contain  multiple,  but 
mutually  consistent,  abnormalities.  The  system  also  includes  normal  cases;  i.e,,  cases  with  no  abnormality.  When 
a user  first  logs  into  the  system,  he  is  presented  with  the  system  splash  screen.  On  dismissing  the  splash  screen, 
the  user  is  asked  to  log  in  and  to  enter  his  password.  He  can  then  choose: 

a,  to  be  presented  with  a randomly  selected  case  from  his  set  of  unsolved  cases 

b,  to  be  presented  with  a specific  case  based  on  the  patient’s  medical  history 

c,  to  be  presented  with  a particular  anatomical  system  to  investigate 

The  case  selected  by  the  user  is  then  presented  in  the  Diagnosis  screen. 

One  of  the  pedagogical  goal  resources  provided  by  the  learning  environment  is  the  facility  to  compare  the 
ultrasound  scan  presented  by  the  system  with  a "normal"  scan;  i.e.,  a scan  of  a fetus  with  no  abnormalities.  By 
clicking  on  the  middle  button  on  the  bottom-right  pane,  a separate  window  showing  a "normal"  scan  is  displayed 
(Fig.  1). 

The  provision  of  this  comparison  facility  is  pedagogically  driven.  The  comparison  is  very  useful  in  helping  users 
to  disambiguate  between  critical  features  on  the  scan  of  a normal  versus  an  abnormal  fetus.  Because  part  of  the 
diagnostic  skill  being  trained  here  is  perceptual  in  nature,  the  provision  of  the  contrasted  images  enhances  a 
user’s  ability  to  identify  and  distinguish  the  contrasting  conditions  of  a feature  being  examined.  In  addition  to  the 
normal  scan  facility,  the  system  also  provides  the  user  with  access  to  relevant  medical  terms  and  definitions  and 
a list  of  diagnosis  names. 


Figure  1 : The  Diagnosis  screen  with  a “normal”  scan  comparison 

As  shown  above,  the  user  is  able  to  explore  all  anatomical  regions  of  the  fetus  by  selecting  the  appropriate  tab 
along  the  top  of  the  window.  These  regions  include  the  Central  Nervous  System,  the  Abdomen  and  Pelvis,  the 
Limbs,  and  the  Chest.  Note  also  that  certain  regions  are  further  divided  into  sub-regions:  for  example,  the 
Central  Nervous  System  is  further  divided  into  the  Head  and  the  Spine,  as  reflected  in  the  sub-tabs  on  the  right 
hand  side.  After  the  user  has  explored  the  images  and  scans  of  his  choosing  to  his  satisfaction,  he  enters  his 
diagnosis  in  the  dialog  box  shown  in  the  bottom-left  of  the  window.  The  system  allows  the  user  to  either  type  in 
the  diagnosis  or  make  a selection  from  a pop-up  list.  We  have  provided  alternative  input  methods  so  as  to  avoid, 
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at  least,  initially,  users  experiencing  difficulties  over  the  correct  spelling  of  the  diagnoses.  Once  the  diagnosis  is 
made,  the  user  clicks  on  the  "Submit"  button. 

Suppose  that  the  user  enters  a correct  diagnosis:  hydrocephalus.  The  system  will  then  respond  as  shown  in  the 
following  screen  (Fig.  2). 


Figure  2:  System  response  to  a correct  diagnosis 

As  a separate  pedagogical  goal  resource,  the  learning  environment  provides  the  user  with  an  option  to  perform  a 
systematic  walk-through  of  critical  features  in  the  Central  Nervous  System  that  distinguish  between  a fetus  with 
hydrocephalus  and  a normal  fetus.  Walking  through  the  list  of  features  helps  students  to  imbibe  an  expert’s 
diagnostic  model  and  reinforces  the  user's  understanding  of  the  critical  disambiguating  features  of  the 
abnormality  in  question.  The  walk-through  is  depicted  below  (Fig.  3).  As  the  user  steps  through  the  list  of 
comparison  features,  he  is  requested  to  address  each  of  the  questions  posed  (e.g.  "Is  the  bony  calvarium 
present?")  in  the  context  of  comparative  images  of  a normal  fetus  and  a fetus  with  hydrocephalus.  The  provision 
of  these  images  grounds  the  user's  knowledge  in  the  perceptual  aspect  of  the  diagnostic  skill  being  nurtured.  As 
the  questions  are  systematically  addressed  and  answered,  the  Features  table  is  filled  out. 

Suppose,  however,  that  the  user  submits  an  incorrect  diagnosis:  acrania  (i.e.  the  cranium  of  the  fetus  is  missing). 
The  system  will  begin  to  interact  with  the  user  to  draw  his  attention  to  the  particular  features  of  the  expert’s 
schema  that  effectively  disambiguate  between  acrania  and  hydrocephalus,  the  correct  diagnosis.  The  system- 
user  interaction  is  driven  by  an  underlying  table  that  has  been  defined  for  all  features  of  interest  across  the 
different  anatomical  subsystems.  In  the  present  state  of  system  implementation,  the  user  is  advised  to  work  his 
way  through  the  walk-through  and  to  answer  the  question  posed  for  each  of  the  salient  features  that  comprise  the 
expert’s  schema.  The  next  figure  (Fig.  4)  shows  the  user  addressing  the  feature  related  to  the  presence  or  absence 
of  the  mid-line  echo  in  acrania  versus  hydrocephalus  in  the  ultrasound  image. 

To  further  enhance  the  salient  dimension  of  the  perceptual  element  that  the  system  is  attempting  to  direct  the 
user's  attention  to,  the  user  also  has  the  option  to  turn  on  a highlighting  feature  that  shows  him  the  portion  of  the 
image  to  which  he  should  be  directing  his  attention. 
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Figure  3:  Walk-through  for  diagnosis  of  the  Central  Nervous  System 


Figure  4:  Contrast  between  the  incorrect  and  correct  diagnoses 

It  should  be  noted  that  when  considering  the  above  comparison,  the  user  still  retains  the  option  of  overlaying  the 
image  of  a normal  scan  on  the  two  images  already  displayed,  as  a further  point  of  reference  for  making 
perceptual  comparisons.  As  shown  above,  the  user  addresses  the  question  "Is  there  a mid-line  echo?"  for  both  the 
incorrect  diagnosis  (acrania)  and  the  correct  diagnosis  (hydrocephalus)  and  submits  his  answer  to  the  system  for 
verification.  The  radio  button  style  of  interface  presents  the  user  with  a finite  set  of  possible  combinations  that  he 
can  select.  Hence,  it  is  not  ideal.  When  user  modeling  is  incorporated,  the  system  will  reason  interactively  with 
the  user  on  the  disambiguating  features  that  he  appears  not  to  have  grasped  or  discerned.  Notwithstanding,  the 
above  systematic  walk-through  is  still  a useful  component  of  the  system;  it  will  be  presented  to  the  user  when  the 
system  determines  that  the  user’s  understanding  is  so  weak  that  it  would  be  more  productive  to  place  him  in  a 
training-oriented  mode  of  learning. 

Assuming  that  the  user  makes  a correct  diagnosis  or  is  led  to  the  correct  diagnosis  by  the  system,  he  then  will 
have  the  option  to  proceed  to  the  next  section  of  the  learning  environment  that  addresses  the  management  and 
consultancy  of  the  patient  mother  or  to  return  to  the  beginning  of  the  system  and  select  a new  case  to  investigate. 


Discussion 
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The  system  that  has  been  described  above  is  work  in  progress.  The  interface  is  constantly  being  revised  and 
refined  as  we  endeavor  to  provide  the  system  with  a stronger  case-based  orientation.  We  are  also  working  on 
developing  the  system’s  user  model.  The  main  difficulty  here  is  that  our  medical  domain  consultants  do  not,  as 
yet,  possess  an  explicit  idea  of  how  they  themselves  handle  user  modeling  when  coaching  students  in  the 
performance  of  ultrasound  scanning  and  the  diagnosis  of  fetal  abnormalities.  In  light  of  this,  we  intend  to 
perform  several  empirical  investigations  of  instructor-student  interaction  to  flesh  out  the  relevant  attributes  that 
should  be  tracked  by  the  system  and  to  get  a better  sense  of  students'  learning  processes. 

We  have  reviewed  our  system  with  respect  to  the  goal-based  scenario  design  criteria  set  out  at  the  beginning  of 
this  paper.  We  are  satisfied  that  the  system  adequately  achieves  the  criteria  of  thematic  coherence,  control  and 
empowerment,  challenge  consistency,  and  pedagogical  goal  support.  As  for  realism  and  richness,  we  are  of  the 
opinion  that  there  is  a large  degree  of  realism  as  the  simulated  task  that  a user  is  confronted  with  is  highly 
authentic.  However,  there  is  some  degree  of  compromise  with  respect  to  richness  that  arises  from  the  ultrasound 
domain.  As  a contrast,  the  domain  of  upper  gastro-intestinal  bleeding  affords  far  richer  possibilities  in  terms  of 
the  simulation  that  the  system  could  be  made  to  support.  Such  a domain  supports  incorporating  the  dimension  of 
time  criticality  in  making  a diagnosis;  this  is  an  element  that  the  ultrasound  domain  inherently  does  not  support. 
We  have  certain  reservations  with  respect  to  the  responsiveness  of  the  system,  but  we  expect  this  to  improve 
when  the  user  modeling  component  of  the  system  is  in  place.  Finally,  with  respect  to  pedagogical  goal  resources, 
we  are  currently  working  on  supplegienting  the  current  system  with  additional  supporting  tools  and  information 
so  that  students  should  not  only  be  able  to  make  the  correct  diagnosis  of  a fetal  abnormality  but  also  be  able  to 
fully  understand  the  cause-effect  relations  underlying  the  abnormalities  they  discover.  In  this  way,  system  users 
or  trainee  gynecologists  will  be  better  placed  to  assess  the  management  and  consultancy  options  open  to  them 
when  they  proceed  to  the  second  part  of  the  goal-based  learning  environment. 


Conclusion 

In  this  paper,  we  have  described  a multimedia  learning  environment  for  learning  how  to  diagnose  fetal 
abnormalities.  The  system  was  designed  based  on  principles  derived  from  goal-based  scenarios.  It  consists  of 
two  parts:  the  diagnosis  part  and  the  patient  management  and  consultancy  part,  assuming  that  an  abnormality  has 
been  diagnosed.  To  date,  we  have  worked  only  on  the  first  part  of  the  project.  Most  of  the  media-based  element 
of  the  project  is  in  place.  This  work  has  not  been  easy  to  complete  and  the  amount  of  time  and  effort  needed  to 
sift  through  and  identify  suitable  video  footage  and  image  clips  are  enormous.  We  have  gone  to  considerable 
trouble  to  ensure  that  we  capture  the  best  quality  images  that  amply  show  the  critical  perceptual  features  of 
interest.  We  are  proceeding  to  direct  our  attention  to  the  user  modeling  aspect  of  the  work  so  that  the  system  will 
be  capable  of  handling  more  direct  and  dynamic  interaction  with  the  user.  The  walk-through  illustrated  in  the 
paper  will  only  be  executed  when  the  user  demonstrates  very  poor  competence  and  is  moved  into  a training- 
orinted  mode  of  learning. 

While  the  goal-based  scenario  that  we  are  creating  is  related  to  the  domain  of  ultrasonography,  the  system  can  be 
readily  customized  for  use  with  any  other  image-based  technologies  in  medicine;  eg.  MRI  and  CT-scans.  As  the 
system  was  developed  using  Macromedia  Director,  we  also  aim  to  have  a PC-Macintosh  cross-platform  product 
when  the  research  project  is  eventually  completed. 
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Abstract:  Both  representing  cases  as  fixed  scripts  as  well  as  the  development  of  initial  expert 
systems  as  a basis  for  computer  based  tutoring  systems  is  particularly  difficult  and  time  intensive  in 
a multi-institutional  development  project.  On  the  other  hand,  structured  systematic  knowledge  and 
a common  terminology  is  necessary  for  automatic  generation  of  information  and  quiz  pages  as  well 
as  automatic  feedback.  Therefore  we  provide  an  architecture  that  relates  tutoring  cases  to  general 
concept  representation  systems  containing  common  terminologies,  and  facilitates  an  incremental 
acquisition  of  systematic  knowledge. 


Introduction 

When  analyzing  the  problem  solving  methods  of  medics,  experts  combine  the  application  of  specialized  domain 
knowledge  with  a case-based  approach  to  find  the  right  diagnosis  and  therapy  (Patel  et  al.  1995).  Therefore,  the 
education  of  students  of  medicine  should  include  both  the  imparting  of  systematic  knowledge  together  with  case- 
oriented  learning.  This  applies  also  to  computer  based  training  systems. 

Tutoring  systems  range  from  a hard  coded  representation  of  cases  (Mayo  Clinic  1996)  to  applications  based 
on  expert  systems,  where  cases  form  an  instantiation  of  pre-coded  systematic  knowledge  (Schewe  et  al.  1996).  In  the 
former,  the  author  has  to  develop  fixed  scripts  of  the  tutoring  process  and  gets  no  knowledge-based  support  for 
authoring  and  tutoring.  The  latter  support  the  author  by  automatic  generation  of  quiz  and  information  pages  and 
make  an  automatic  feedback  in  the  tutoring  process  possible. 

If  clinical  institutions  of  different  domains  have  to  work  on  such  an  expert  system  component,  an  initial 
complete  development  is  difficult  and  time  expensive.  On  the  other  hand,  an  incremental  “case  by  case”  coding  by 
different  authors  requires  a common  terminology  and  a framework  identifying  and  relating  the  central,  medical 
concepts.  We  developed  an  architecture  that  contains  terminological  groups  from  different  concept  representation 
systems.  It  allows  the  construction  of  systematic  knowledge  by  linking  items  of  these  groups  together  when  authors 
build  tutoring  cases.  As  a result  of  this  process  we  obtain  a medical  knowledge  base  that  comprises  all  knowledge 
acquired  case  by  case.  Elements  of  this  knowledge  base  are  integrated  with  the  authoring  and  tutoring  system  into  a 
learning  environment.  The  basic  parts  of  both  systems  are  complemented  by  components  that  make  use  of  the 
knowledge  base  to  support  authoring  and  tutoring  processes  (Fig.  1). 
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Figure  1:  Architecture  of  the  learning  environment. 


Identification  of  Central  Concepts 

The  concepts  a medic  usually  uses  to  describe  a medical  case  can  be  divided  into  the  following  topics:  anatomy, 
phenomena  which  are  located  at  a specific  anatomy  (examples  for  the  term  ‘phenomenon’  are  ‘inflammation’, 
‘pain’...),  methods  the  medic  uses  to  look  for  or  to  validate  certain  phenomena  (examinations,  technical  examination, 
anamnesis)  , diagnoses,  and  therapies.  In  our  architecture,  we  reflect  the  medics’  perspective  by  representing  the 
according  terminological  groups  in  different  hierarchies  and  sets: 

• the  hierarchy  of  anatomic  structures  and  body  systems, 

• the  set  of  phenomena, 

• the  hierarchy  of  what  we  call  methods,  which  comprises  different  (technical  and  non-technical) 
examinations  and  the  anamnesis, 

• the  hierarchy  of  diagnoses  and 

• the  hierarchy  of  therapies. 

Contrary  to  the  elements  in  the  concept  hierarchies,  phenomena  are  structured  objects  consisting  of  a phenomenon 
name  and  a set  of  properties.  For  example,  some  of  the  properties  the  phenomenon  ‘pain’  could  have,  are  ‘intensity’, 
‘duration’  or  ‘quality’.  In  medical  technical  terminology,  there  exists  a large  range  of  systems  for  concept 
representation:  classifications,  nomenclatures  or  coding  schemes.  By  separating  the  different  hierarchies  mentioned 
above,  we  are  able  to  make  use  of  existing  terminology  systems  like  special  axes  or  parts  of  classifications, 
nomenclatures  or  special  coding  schemes.  As  an  example,  we  utilize  the  "anatomy"  axis  of  MeSH  (Medical  Subject 
Headings)  (WHO  1992)  and  the  ICD  (Systematic  Nomenclature  of  Medicine)  (NLM  1999)  diagnosis  hierarchy. 


Relating  the  Concepts 

The  information  inherent  in  medical  tutoring  cases  is  mainly  represented  by  links  between  concepts,  which  are  given 
as  hierarchy  items  and  phenomenon  instances.  For  example,  we  establish  links  that  relate  phenomena  at  a certain 
anatomic  structure  to  examination  methods  they  are  based  upon,  and  to  diagnoses  and  therapies  they  imply  (Fig.  2). 
Concrete  cases  constitute  instances  of  this  network. 
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Figure  2:  Integration  of  phenomena  into  concept  graphs. 

As  an  example,  a tutoring  case  may  describe  a patient  with  anginose  complaints.  Because  of  this,  the  medic  may 
order  an  electrocardiogram  examination,  which  reveals  a ST-segment  depression.  Taking  other  observed  facts  into 
consideration,  the  student’s  diagnosis  should  be  myocardial  infarction.  One  necessary  therapy  for  this  disease  is 
intensive  care.  When  the  author  describes  this  case,  several  links  are  established.  The  phenomenon  ‘ST-segment 
depression’  is  connected  with  the  anatomic  structure  ‘heart’.  Their  combination  is  linked  both  to  the  associated 
examination  ‘electrocardiogram’  and  the  diagnosis  ‘myocardial  infarction’.  Finally  the  diagnosis  ‘myocardial 
infarction’  is  linked  to  the  therapy  ‘intensive  care’. 

By  following  explicit  links  we  obtain  further  implicit  ones.  For  example,  a relation  between 
‘electrocardiogram’  and  ‘myocardial  infarction’  results  from  their  connections  to  the  given  anatomy  and 
phenomenon  combination. 


Supporting  the  Authoring  Process 

We  can  use  the  hierarchies  of  anatomy,  examination  methods,  diagnoses  and  therapies  to  support  the  author  in 
formulating  elements  of  tutoring  cases.  In  (Fig.  3)  an  example  of  a tree-control  is  shown  that  offers  a part  of  the 
MeSH  anatomy  structure  for  defining  the  location  of  a phenomenon.  This  can  be  done  by  selecting  the  appropriate 
concepts  from  the  tree,  which  are  inserted  into  the  structured  representation  of  the  phenomenon  on  the  right  hand 
side  of  the  depicted  window. 

When  building  a case  for  tutoring,  the  author  describes  a number  of  phenomena  observed  at  the  patient.  Furthermore 
he  is  able  to  relate  phenomena  at  a certain  location  to  one  or  more  differential  diagnoses.  Offering  to  him  links  from 
already  constructed  tutoring  cases  that  provide  differential  diagnoses  for  the  observed  phenomena  can  support  this 
process.  Structured  medical  case  knowledge  is  also  useful  for  building  quiz  pages  by  integrating  the  set  or  a subset 
of  those  differential  diagnoses  plus  some  other  irrelevant  ones  into  the  pages.  The  aim  of  these  quiz  pages  is  that  the 
student  specifies  correct  differential  diagnoses  at  different  stages  of  the  patient  information  gathering  process.  The 
final  diagnosis  is  put  together  with  other  differential  diagnoses,  for  which  not  sufficient  evidence  exists,  to  build  a 
quiz  page  to  find  the  final  diagnosis,  as  it  is  shown  in  (Fig.  4). 

Furthermore,  questions  about  any  direct  or  indirect  links  within  our  architecture  can  be  used  to  build  quiz  pages  that 
are  aimed  at  teaching  the  student  the  underlying  concepts  and  their  interrelations.  For  example,  general  questions 
like  "what  examinations  can  be  performed  on  the  specified  anatomies"  can  be  derived  from  the  linkage  of 
phenomena  with  examinations  and  anatomic  structures. 
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Figure  3:  MeSH  anatomy  structure  as  a tree-control  element. 


The  author  can  employ  links  for  building  information  pages  that  teach  the  student  the  according 
associations  of  medical  knowledge.  Former  experiences  with  tutoring  software  (Scheuerer  et  al.  1998)  showed  that 
relations  we  have  made  explicit  in  our  architecture  are  an  important  subject  matter  in  medical  teaching.  Information 
pages  using  text,  pictures,  animations  or  sounds  illustrate  both  medical  concepts  and  links  between  those  concepts. 
Using  concepts  or  concept  combinations  as  indexes  into  databases  of  multimedia  objects  supports  the  author  in 
creating  those  pages. 


Figure  4:  Sample  quiz  page  derived  from  differential  diagnoses  and  the  final  diagnosis  for  a phenomenon. 
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Supporting  the  Tutoring  Process 


The  links  mentioned  above  not  only  support  the  authoring  processes  but  also  the  automatic  generation  of  navigation 
aids,  online  information  windows,  and  feedback  in  the  tutoring  process. 

Menus  based  on  hierarchies  of  examination  procedures  can  be  used  to  provide  students  in  the  tutoring 
process  with  possible  choices  for  information  gathering  actions.  The  author  is  able  to  decide  if  the  entire  hierarchy 
or  sub-hierarchies  are  shown  to  the  student.  It  is  also  possible  to  focus  examination  methods  on  anatomic  structures 
by  using  acquired  links  between  them.  In  that  case,  a menu  primarily  showing  an  anatomic  structure  can  be 
constructed.  Every  sub-menu  has  an  additional  menu  item  that  pops  up  the  according  examination  methods.  This 
enables  the  student  to  select  only  those  examinations  that  can  be  applied  at  specified  anatomies.  For  example,  links 
from  the  anatomy  ‘thyroid’  to  examinations  like  ‘thyroid  function  test’,  ‘radioisotope  scanning’,  and  ‘sonography’ 
could  have  been  established  in  the  case  authoring  process.  Accordingly,  this  meaningful  collection  can  be  shown  to 
the  student  as  a selection  of  examination  methods  at  the  ‘thyroid’. 

Knowledge  formalized  within  our  architecture  can  be  used  for  an  automatic  feedback  generation  in  the 
tutoring  process.  We  allow  the  quantitative  or  qualitative  assessment  of  links  between  phenomenon-anatomy 
combinations  and  diagnoses,  as  it  is  done  for  example  in  the  D3  expert  system  shell  (Reinhardt  1997)  by  a number 
of  positive  and  negative  scores.  By  using  this  knowledge,  comments  and  corrections  that  refer  to  a diagnosis  chosen 
by  the  student  can  be  automatically  produced.  To  do  this,  the  system  searches  for  phenomena  given  in  a tutoring 
case  that  plead  for  or  rule  out  the  diagnoses  the  student  has  chosen,  according  to  the  positive  and  negative  links 
mentioned  above.  The  student  may  be  helped  by  this  function  at  every  stage  of  the  tutoring  process,  because  the 
described  feedback  methods  take  into  account,  what  the  student  already  knows  about  the  patient.  This  approach  can 
also  be  applied  at  links  between  diagnoses  and  therapies.  Additionally,  if  not  sufficient  information  about  the  patient 
was  gathered  by  examination  methods  to  confirm  or  exclude  a diagnosis,  the  links  mentioned  above  can  be  used  to 
give  hints  to  the  necessary  methods,  or  just  to  state  that  relevant  information  is  still  missing. 


Conclusion 

The  presented  architecture  allows  an  integration  of  different  concept  representation  systems  for  the  building  of 
tutoring  cases.  Additionally,  we  establish  links  between  these  concepts  to  formalize  systematic  knowledge  within 
medical  cases.  This  supports  the  use  of  a common  terminology  for  cooperating  authors  and  helps  to  avoid  redundant 
case  information.  Furthermore,  authoring  and  tutoring  processes  are  supported  by  mechanisms  for  generating  quiz 
and  information  pages  and  by  automatic  feedback  based  on  the  acquired  systematic  knowledge. 

The  presented  work  constitutes  a first  step  in  the  project  “Docs’ n’  Drugs  — Die  virtuelle  Poliklinik”.  In  this 
project,  a tutoring  system  will  be  developed  that  shall  represent  a virtual  clinic.  Multiple  clinical  institutions,  the 
University  of  Ulm,  the  University  of  Applied  Science  Ulm,  and  industrial  partners  are  involved  in  this  enterprise. 
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Abstract:  This  paper  describes  the  design,  development,  delivery  and  evaluation  of  a Web-Based 
Instruction  (WBI)  course  which  has  been  designed  to  support  a large  number  of  students  (approximately 
700  per  term).  The  paper  describes  the  course  philosophy,  purpose  and  design,  but  concentrates  on  the 
issues  that  must  be  addressed  when  designing  WBI  courses  for  large  groups  of  students. 


Background 

The  Faculty  of  Education  at  the  University  of  Alberta  requires  that  all  students  complete  a course  in  "computer 
applications."  Historically,  students  could  meet  this  requirement  by  taking  either:  a)  a traditional  face-to-face  (F2F) 
lecture/laboratory  course  offered  by  the  Department  of  Computing  Science  which  could  best  be  described  as  an  "Introduction  to 
Programming"  course,  or  b)  a traditional  (F2F)  lecture/laboratory  course  offered  by  the  Faculty  of  Education  which  consisted  of  a 
3-hour  "demonstration"  lecture  followed  by  a 3 -hour  "workbook"  laboratory  each  week. 

In  the  1997-98  academic  year  the  computing  course  requirement  for  education  students  was  decreased  from  3 credits  to  1.5 
credits,  causing  an  examination  and  reduction  of  the  content  of  the  course.  After  strong  arguments,  this  decision  was  reversed  for 
the  1998-1999  academic  year.  We  decided  that  rather  than  simply  re-instituting  the  course  used  prior  to  September  1997,  we 
should  completely  redesign  a 3-credit  course  for  delivery  in  September  1999. 

Faculty  of  the  Instructional  Technology  program  at  the  University  of  Alberta  have  been  developing  WBI  courses  for  a 
number  of  years  (e.g.,  Montgomerie  & Harapnuik,  1997;  Davies,  1998;  Davies  and  Carbonaro,  1998).  Because  of  our 
commitment  to  the  precepts  of  active  learning,  and  after  examining  the  costs  of  delivering  WBI  (Harapnuik,  Montgomerie  & 
Torgerson,  1998),  we  decided  to  design  the  course  to  be  delivered  through  Web-Based  Instruction  (WBI).  The  resulting  course, 
EDPY202:  Instructional  Applications  of  Technology  (http://www.quasar.ualberta.ca/edpv202)  is  publicly  available  on  the  Web. 

Philosophy  of  the  Course 


Tell  me  and  I will  forget. 

Show  me  and  I will  remember. 

Involve  me  and  I will  understand. 

(Variously  Attributed) 

The  philosophy  of  this  course  centers  on  the  theoretical  perspective  that  learning  is  a process  of  constructing  knowledge  as 
opposed  to  a process  of  recording  knowledge  (Harel  & Papert,  1991).  From  this  perspective  students  are  required  to  be  "active 
participants"  in  their  own  learning.  To  support  this  perspective,  students  need  to  be  placed  in  an  environment  whereby  they  have 
the  opportunity  to  take  ownership  of  their  own  learning.  We  have  attempted  to  create  such  an  environment  in  two  ways.  First  we 
have  designed  the  course  to  be  self-paced  with  varying  degrees  of  individualization.  This  approach  allows  students  a higher  level 
of  flexibility  with  respect  to  course  time  commitments  and  the  level  of  skills  and  knowledge  they  wish  to  acheive.  Course 
materials  can  be  studied  individually,  in  collaboration  with  peers,  or  in  collaboration  with  a more  knowledgeable  assistant. 
Second,  we  have  focused  learning  activities  on  software  tools  used  for  problem-solving  — "Mindtools"  (Jonassen,  1996).  This 
emphasizes  students  learning  "with"  the  computer  as  opposed  to  "from"  the  computer.  In  other  words,  computer-based  tools  and 
learning  environments  (e.g.  databases,  spreadsheets,  semantic  networks,  computer  conferencing,  hypermedia  construction,  or 
microworld  environments)  can  serve  as  cognitive  tools  or  extensions  of  the  mind.  With  the  use  of  such  "Mindtools,"  learners  can 
enter  an  intellectual  partnership  with  the  computer  in  order  to  access  and  interpret  information,  and  organize  personal  knowledge. 
A recent  article  (Connell,  1997)  concludes  that  "lA"  (Intelligence  Amplification)  is  preferred  over  "AI"  (Artificial  Intelligence). 
As  Connell  (1997)  states:  "In  AI,  ...  models  of  an  individual's  cognitive  processes  ...  serve  as  templates  to  construct  intelligent 
systems  capable  of  teacher  replacement  ...  In  lA,  the  roles  of  the  machine  and  user  are  reversed.  Instead  of  having  a system  in 
which  the  user  responds  to  the  machine,  you  have  a system  where  the  user  uses  a set  of  provided  software  construction  tools  to 
actively  construct  more  powerful,  personally  meaningful  representations  of  problem  spaces"  (p.  27). 

Traditional  educational  paradigms  must  be  shifted  somewhat  in  order  to  successfully  incorporate  technology  in  this  way.  For 
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technological  tools  to  become  student-centered  "Mindtools"  requires  a readiness  for  change.  For  example,  school  schedules,  the 
physical  setup  of  classes,  and  assessment  of  both  students  and  teachers  may  require  modification  to  accommodate  these  new  tools 
(Barron  & Orwig,  1995). 

We  have  made  many  such  adjustments  in  this  course.  First,  we  have  attempted  to  provide  students  with  a lot  of  scheduling 
flexibility.  We  impose  no  requirement  for  lecture  or  lab  attendance.  The  on-campus  computer  lab  facility  is  available  to  students 
during  a total  of  ten  to  twelve  three-hour  periods  during  the  week  (including  some  evening  times)  as  well  as  two  additional  four- 
hour  periods  on  the  weekend.  Subject  to  space  limitations,  students  are  free  to  use  the  lab  and  obtain  in-person  help  from  a 
teaching  assistant  during  any  of  these  time  periods.  Students  also  have  virtually  continuous  access  to  instructor  assistance  (or  peer 
collaboration)  via  Computer-Mediated  Communication  (CMC),  in  the  form  of  email  or  web-based  conferencing.  Most  students 
who  prefer  to  work  largely  at  a distance  make  extensive  use  of  CMC.  We  do  however,  impose  traditional  assignment  and  course 
completion  deadlines.  This  university  is  currently  not  prepared  to  allow  undergraduates  varying  periods  of  elapsed  times  in  which 
to  finish  courses. 

Second,  we  have  also  changed  our  assessment  methods  as  compared  to  many  traditional  courses.  In  this  course,  60%  of  a 
student’s  grade  is  based  on  a portfolio  of  term  work.  This  portfolio  is  a series  of  five  modules  which  students  must  submit 
electronically.  Each  module  consists  of  a computer  file  (or  group  of  files)  which  demonstrate  the  students  understanding  of  the 
various  course  topics.  The  content  of  their  submissions  is  largely  a matter  of  student  choice  and  creativity,  although  in  most 
cases,  students  must  indicate  how  their  project  relates  to  the  provincial  education  curriculum.  The  remaining  40%  of  a student's 
grade  is  based  on  a practical  exam  which  they  must  write  in-person  in  the  on-campus  computer  lab  facility.  This  lab  exam  is 
really  a mini-portfolio  of  small  projects  using  the  same  tools  the  students  have  learned  during  the  course.  The  purpose  of  this 
exam  is  to  obtain  an  objective  assessment  of  the  computer  literacy  level  of  each  individual  student,  in  comparison  with  peers. 
Students  in  this  course  receive  final  grades  on  the  same  traditional  9-point  scale  as  most  courses  at  this  university. 

Purpose  of  the  Course 

The  purpose  of  teaching  education  students  to  use  computers  is  so  that  they  will  be  prepared  to  integrate  the  use  of 
technology  into  the  curriculum.  This  introductory  course  aims  to  provide  undergraduate  education  students  with  the  basic  skills 
they  will  need  to  apply  information  technology  in  Kindergarten  - Grade  12  schools.  Although  this  course  is  primarily  a software 
tools  literacy  course,  it  does  provide  the  students  with  some  exposure  to  curriculum  integration.  Thus,  we  hope  that  these  soon-to- 
be  teachers  will  be  in  a position  to  assist  their  students  in  using  the  information  technology  tools  which  are  commonplace  in 
modern  society. 

At  the  same  time  we  were  designing  this  course,  the  Government  of  Alberta  published  Information  and  Communication 
Technology,  Kindergarten  to  Grade  12:  An  Interim  Program  of  Studies  (Alberta  Education,  1998)  which  will  form  the  basis  for 
the  provincial  program  of  studies  to  be  released  in  June  2000.  This  document  is  constructed  in  terms  of  "learner  outcomes"  and 
the  philosophy  is  stated  as: 

This  technology  curriculum  is  intended  to  provide  a broad  perspective  on  the  nature  of  technology  and  its 
impact  on  society.  Students  will  be  encouraged  to  grapple  with  the  complexities,  as  well  as  the  advantages 
and  disadvantages,  of  technologies  in  our  lives  and  workplaces.  It  is  not  intended  to  be  taught  as  a stand- 
alone course  but  rather  to  be  inllised  within  existing  courses.  Technology  is  best  learned  within  the  context  of 
applications.  Activities,  projects,  and  problems  that  replicate  real-life  situations  are  effective  resources  for 
learning  technology.  (Alberta  Education,  1998). 

The  objectives  for  EDPY202  are  directly  related  to  the  Alberta  Education  document  and  students  are  required  to  complete 
either  an  introductory  or  an  advanced  module  in  each  of  five  topic  areas: 

1.  Internet  Tools  (Email,  Web-based  conferencing,  Internet  searching,  FTP,  Telnet,  Simple  WWW  page  creation), 

2.  Digital  Media  Processing  (clip  art,  drawing,  painting,  scanning,  digital  photography,  digital  audio/video), 

3.  Multimedia/hypermedia  Presentations:  (PowerPoint  or  HyperStudio), 

4.  Spreadsheets  (e.g.  Excel),  and 

5.  Databases  (e.g.  FileMaker  Pro,  Access). 

Course  Design 

This  course  has  been  designed  to  be  self-paced  and  modularized  with  a focus  on  independent  study.  Three  optional  lectures 
are  given: 

1 . an  introductory  lecture  covering  the  philosophy  and  operation  of  the  course, 

2.  an  introductory  hands-on  workshop  concerning  basic  computer  operation  and  terminology,  and 

3.  a lecture  the  week  before  the  final  exam  (lab  practical)  which  explains  what  will  be  covered  and  the  exam  procedures. 
All  of  this  material  is  also  available  on  the  course  Web  page. 

An  introductory  and  an  advanced  module  for  each  of  the  five  topic  areas  have  been  prepared.  Students  must  complete  one 
module  from  each  of  the  five  areas.  All  modules  follow  the  same  structure  with  each  module  having  sections  on: 

1 . Just  the  Facts  (a  short  overview  of  what  needs  to  be  done) 

2.  Purpose 

3.  Prerequisites 
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4.  Objectives 

5.  Alberta  Learner  Outcomes  (explanation  and  direct  links  to  the  sections  in  the  "learner  outcomes"  document  which  are 
particularly  addressed  in  this  module), 

6.  Assignments,  and 

7.  Resources. 

A 3-hour/week  laboratory  period  is  assigned  for  each  student,  but  students  are  encouraged  to  work  at  home  rather  than  in  the 
laboratory  if  they  have  the  ability,  equipment  and  software.  This  has  two  major  results:  firstly,  many  students  appreciate  the 
opportunity  to  take  the  course  wherever  and  whenever  they  wish,  and  secondly,  the  teaching  assistants  in  the  lab  can  concentrate 
on  helping  those  students  who  really  need  help.  For  example,  students  who  are  very  novice  computer  users  find  it  very  difficult  to 
work  actively  and  independently,  especially  in  the  early  stages  of  the  course. 

Issues  for  Courses  with  Large  Enrollments 

Most  WBI  courses  are  designed  for  delivery  to  a relatively  small  class  of  students.  Our  own  courses  (Montgomerie  & 
Harapnuik,  1997;  Davies,  1998;  Davies  and  Carbonaro,  1998)  have  been  delivered  to  classes  as  large  as  80.  The  expected 
enrollment  for  EDPY202  is  approximately  700  students  per  term.  Given  this,  a number  of  issues  dealing  with  efficiency  of  both 
instruction  and  administration  needed  to  be  considered. 

Instructional  Issues 

The  course  home  page  was  designed  to  give  access  to  a list  of  navigation  headings  in  the  three  categories  of  Course 
Infomiation,  Modules,  and  Tutorials.  Figure  1 shows  the  drop  down  menus  for  the  navigation  categories. 

Tutorials  were  developed  for  many  of  the  specific  topics  in  the  course,  but  it  was  felt  that  it  would  be  a waste  of  time  to 
develop  new,  extensive  tutorials  on  the  Macintosh  and  Windows  operating  systems  and  on  commercial  productivity  tools  such  as 
Microsoft  Word,  Microsoft  Excel,  Microsoft  Access  and  FileMaker  Pro.  Our  website  has  a extensive  set  of  links  to  existing  web- 
based  tutorials.  Some  of  these  were  developed  at  our  university,  while  others  are  links  to  various  external  Internet  sources.  We 
also  reviewed  a number  of  commercial  tutorials  and  chose  the  MacAcademy  / Windows  Academy  series  of  CD-ROM-based 
instructional  programs  (http://www.macacademv.com/).  We  feel  that  these  programs  come  extremely  close  to  the  kind  of 
instruction  that  would  occur  in  a small  group  demonstration  with  the  additional  ability  for  the  student  to  stop  the  instruction, 
move  to  a different  place  in  the  instruction,  and  to  search  for  a specific  topic.  We  purchased  a number  of  sets  of  these  CD-ROMs 
and  placed  them  on  48-hour  reserve  in  the  library.  We  have  Macintosh  computers  in  our  laboratory,  and  we  negotiated  a license 
with  MacAcademy  / Windows  Academy  which  allows  us  to  put  images  of  the  Macintosh  instructional  CD-ROMs  on  a Macintosh 
G3  Server  and  to  allow  up  to  forty  concurrent  accesses.  This  has  turned  out  to  be  an  extremely  good  solution.  Students  can  access 
the  instruction  whenever  they  want  and  can  proceed  at  their  own  pace.  The  programs  they  are  learning  about  are  available  on  the 
same  workstation,  so  students  can  move  between  the  instruction  and  the  application  giving  them  the  ability  to  pause  the 
instruction  to  try  what  they  just  learned  on  the  real  application.  While  we  have  not  yet  permitted  it  (due  to  licensing  issues),  these 
instructional  programs  can  be  accessed  over  the  Internet,  and,  with  a relatively  high-speed  modem,  they  work  extremely  well. 


Administrative  Issues 

When  one  is  working  with  a class  of  20  students,  a number  of  administrative  tasks  are  handled  without  thought.  It  is 
necessary  to  automate  these  tasks  wherever  possible  when  one  is  planning  for  a class  of  700  students. 

Computing  Infrastructure 

The  delivery  of  a WBI  course  to  large  numbers  of  students,  requires  you  to  be  cognizant  of  various  computing  infrastructure 
issues.  The  Web  Server  must  be  capable  of  handling  a high  number  of  concurrent  hits  and  the  local  and  wide  area  networks  must 
be  prepared  for  a large  volume  of  traffic.  Our  course  website  (run  on  a Sun  Unix  system)  is  fairly  busy;  at  the  time  of  this  writing 
we  are  experiencing  thousands  of  hits  per  week  with  no  problems.  The  Windows  NT  server  which  handles  our  web-based 
conferencing  system  (WebBoard)  is  extremely  busy  with  several  courses  sharing  this  server.  We  have  had  to  increase  the 
memory  in  this  server  substantially  to  be  able  to  handle  the  demand. 


Identification  of  Students 

An  automated  procedure  was  written  to  merge  data  from  a number  of  different  central  university  administrative  files  into  a 
single  record  which  contains  the  data  necessary  for  the  course,  including:  Name,  Student  ID  number.  Computer  Identification 
(assigned  to  each  student  when  they  enroll  at  the  University  of  Alberta),  Course  Section  Identifier,  and  Laboratory  Section 
Identifier. 
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Student  Server  Storage  Space 

Students  need  to  have  a place  where  they  can  store  projects  they  are  working  on.  We  assign  each  student  space  on  a central 
server  which  they  can  access  either  over  AppleShare  or  via  the  File  Transfer  Protocol  (FTP).  An  automated  process  creates  the 
Server  Identification  (Login  ID)  for  each  student  which  is  the  same  as  their  University  Computer  Identification.  An  obvious 
scaling  issue  that  arises  in  this  area  is  how  much  disk  space  one  can  afford  to  allocate  to  students,  particularly  in  a course  such  as 
ours  in  which  students  are  expected  to  create  rather  large  multimedia  files.  We  provide  students  with  a very  modest  20  MB  of 
space  on  our  course  server  (in  addition  to  the  5 MB  they  receive  on  the  university's  general  server).  However,  when  multiplied  by 
several  hundred  students,  it  amounts  to  several  gigabytes  of  hard  drive  storage  that  we  must  budget  for  and  arrange  with  the 
university's  network  administration.  We  insist  that  students  compress  (e.g.  Stuffit  or  Zip)  assignment  files  to  help  conserve  disk 
space.  However,  most  students  do  eventually  exhaust  their  20  MB,  so  we  suggest  that  they  archive  older  assignments  on  other 
media  such  as  a 100MB  ZIP  disk. 

Web-Based  Conference  System 

We  feel  that  it  is  important  that  students  can  communicate  with  each  other,  with  the  laboratory  assistants  and  with  the 
instructors.  We  use  WebBoard  (O'Reilly  Software,  1999),  a computer  conferencing  system,  to  provide  for  this  communication. 
An  automated  process  creates  the  WebBoard  Identification  for  each  student  that  is  the  same  as  their  University  Computer 
Identification. 

There  are  various  scaling  issues  that  arise  when  utilizing  such  a conferencing  system  for  a large  group  of  students.  One  must 
be  prepared  for  a large  volume  of  messages.  We  have  found  it  very  useful  to  break  up  the  conferencing  system  into  more 
manageable  sub-conferences,  particularly  in  the  case  (such  as  in  one  of  our  course  modules)  where  student  conferencing 
messages  are  an  assignment  requirement.  Students  should  not  be  required  to  wade  through  hundreds  of  messages  at  a time.  In  our 
course,  we  have  also  set  up  conference  sub-topics  for  help  questions  related  to  specific  software  tools. 

Another  problem  is  that  some  students  do  not  naturally  know  how  to  provide  enough  details  when  asking  a question  on  a 
web-based  conferencing  system.  They  ask  a question  as  if  they  are  talking  to  someone  looking  over  their  shoulder.  Instructors 
have  to  reply  asking  for  more  information  - multiply  this  by  large  numbers  of  students  and  you  add  to  the  volume  of  messages. 
This  is  particularly  a problem  at  the  beginning  of  the  course,  but  as  the  students  get  used  to  this  mode  of  communication  they 
learn  to  describe  their  situation  more  accurately  the  first  time. 

Another  consideration  is  that  adequate,  paid  instructor  / teaching  assistant  time  must  be  allocated  to  monitoring  and 
answering  conference  messages  relatively  quickly,  including  evenings  and  weekends.  This  last  consideration  is  also  an  important 
factor  in  dealing  with  student  email  messages  in  a large  course.  Although  general  questions  are  raised  on  the  WebBoard,  student 
concerns  of  a more  personal  nature  are  handled  via  private  email.  In  a large  course  such  as  ours,  the  volume  of  messages  can 
obviously  become  very  high. 
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Figure  1:  Course  Navigation  Options 
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Students  in  our  course  access  four  individual  computer  systems  (i.e.  the  WebBoard,  the  Marks  Database,  the  Course  File 
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Server,  and  the  University's  general  purpose  server),  each  of  which  has  its  own  logon  ID  and  Password.  During  our  registration 
procedure  we  force  all  the  IDs  for  a user  to  be  the  same  as  that  already  set  for  access  to  the  general  university  server.  We  also  set 
the  password  for  the  three  systems  under  our  control  to  have  the  same  password.  On  the  Web  page  (and  in  the  first  laboratory 
period),  we  demonstrate  how  to  set  the  general  university  password  to  be  the  same  as  the  other  three.  Finally,  we  provide  a single 
Web  page  that  allows  student  to  change  all  of  their  passwords  at  once.  Students  very  quickly  forget  that  they  are  using  a number 
of  different  computers  with  different  ID/password  configurations  when  all  their  IDs  and  Passwords  are  the  same. 

Our  university  is  installing  a common  file  system:  Andrew  File  System  (Transarc,  1998);  and  common  authentication 
service:  Kerberos  (Neuman  & Ts'o,  1994).  Many  other  post-secondary  education  institutions  and  businesses  are  installing  similar 
systems.  At  this  time  our  Faculty  has  not  integrated  with  AFS  and  Kerberos.  In  some  ways  this  is  unfortunate  - it  would  have 
been  nice  not  to  have  to  "reinvent  the  wheel",  but  having  to  build  our  own  systems  did  give  us  a little  more  control. 

Submission  of  Assignments 

Students  need  to  be  able  to  submit  their  assignments  for  marking.  When  server  storage  space  is  set  up  for  the  student,  a 
"submit"  directory  is  set  up  within  that  space.  Students  are  told  to  place  their  assignments  in  that  directory  (either  through 
AppleShare  or  FTP)  when  it  is  ready  to  be  marked.  We  require  that  the  assignments  for  each  module  be  placed  in  a single 
compressed  archive.  At  the  time  the  assignment  is  due,  an  automated  procedure  scans  the  "submit"  directories  in  the  student  files, 
makes  a copy  of  any  new  archives  which  are  in  the  students'  "submit"  directories,  reassigns  ownership  of  these  copied  archives 
and  places  them  in  a folder  for  the  Markers  to  access.  This  procedure  also  sends  an  automated,  time  stamped  e-mail  message  to 
each  student  telling  them  which  (if  any)  files  were  copied  and  which  were  not  copied.  The  message  also  states  specifically  why  a 
file  was  not  copied  (e.g.,  an  earlier  copy  of  the  file  existed,  the  file  was  not  in  an  archive  (i.e.,  it  was  not  in  .sit  or  .zip  format)). 

Marking  Assignments/Exams 

A marks  database  was  built  using  a Web  enhanced  ACCESS  database.  This  is  a modification  of  the  system  reported  in 
Montgomerie,  Harapnuik  & Palmer  (1997).  Markers  and  staff  view  the  database  as  if  it  is  a spreadsheet  and  can  enter  and  change 
marks  and  comments  for  each  assignment  via  a web  browser.  We  provide  students  with  a web-based  query  which  allows  them  to 
see  only  their  own  marks  and  comments.  We  have  also  developed  several  customized  database  queries  that  help  to  ensure  the 
correctness  and  completeness  of  our  marking,  for  example,  by  pointing  out  students  who  have  not  received  a grade  for  a 
particular  module.  In  smaller  courses  this  is  usually  done  manually  with  little  effort,  but  in  a large  course,  ensuring  that  all  marks 
have  been  entered  is  a serious  concern.  The  course  administrator  also  has  the  ability  to  automatically  compute  the  final  course  9- 
point  grade  for  all  students  by  specifying  raw-score  cutoffs.  This  is  a huge  time  saver,  as  manual  entry  of  each  student's  final 
grade  would  take  several  hours.  The  database  also  has  a report  capability  that  produces  the  "official"  mark  sheets  in  the  form  that 
the  Office  of  the  Registrar  requires.  It  is  hoped  that  when  the  University’s  new  student  record  system  is  completed,  electronic 
transfer  of  marks  to  the  Registrar  system  will  be  allowed. 

Student  Evaluation  of  the  Course 

The  first  administration  of  the  course  was  during  September  - December,  1998.  Two  hundred  and  forty  students  enrolled  in 
12  lab  sections  were  asked  to  rate  their  technology  anxiety  and  personal  thoughts  about  using  computers.  These  students  were 
observed  working  in  the  lab.  Some  of  these  students  were  also  interviewed. 

Students  enrolled  in  the  Faculty  of  Education  are  required  to  take  either  this  course  or  a course  offered  by  the  Department  of 
Computing  Science;  therefore,  student  motivation  could  affect  the  ratings.  Students  did  not  know  the  identity  of  the  teaching 
assistants  at  the  time  of  registration,  hence  they  could  not  select  their  laboratory  section  based  on  teaching  assistant  reputation. 
Both  Elementary  and  Secondary  Education  students  were  enrolled  in  the  course.  Students  can  attend  any  lab  section  which  had 
space  available.  Each  lab  section  has  at  least  two  teaching  assistants  with  the  ratio  is  one  TA  for  each  15  students. 

Overall  students  had  a positive  perception  of  technology,  the  use  of  computers  and  the  course.  Students  liked  the  idea  of 
working  on  their  own  pace.  Students  felt  comfortable  knowing  that  they  could  always  find  a TA  in  any  lab  section  and  were  glad 
that  extra  Saturday  and  Sunday  lab  sections  were  offered.  Students  found  the  course  interesting  and  69%  of  the  students 
interviewed  believed  that  they  would  use  this  knowledge  in  classroom.  Although  students  reported  a high  the  level  of  anxiety  in 
the  beginning  of  the  course,  the  same  students  reported  that  this  anxiety  dropped  considerably  after  one  month  in  the  course. 

Students  working  at  home  reported  that  they  were  very  pleased  with  the  course  and  the  support  they  received  from  the 
course  instructor  and  teaching  assistants.  They  stressed  that  they  can  work  fast  since  all  the  material  they  need  is  on  the  Internet 
and  it  is  relevant  to  the  curriculum.  They  also  point  out  that  this  kind  of  course  saves  their  time  because  they  did  not  need  to  be  at 
the  university. 

However  there  is  always  some  room  for  improvement.  Novice  computer  users  scored  high  in  the  anxiety  scale  survey.  Most 
of  the  novice  students  reported  that  the  material  was  very  challenging  and  there  was  not  enough  time  to  study  the  material  on  the 
Internet  and  do  the  assignments.  They  also  reported  that  the  teaching  assistants  did  not  spend  enough  time  with  them.  This 
situation  was  also  observed  in  some  lab  sections. 

The  course  had  some  communications  problems.  Some  of  the  students  reported  not  being  able  to  contact  the  instructor  or  the 
TA  using  the  e-mail  or  the  WebBoard.  The  WebBoard  was  also  underutilized.  A large  number  of  students  stated  that  they  used 
the  WebBoard  only  to  do  the  required  assignment  and  they  never  looked  at  it  again  until  the  end  of  the  course.  Other  students 


898 


reported  that  they  did  not  get  good  answers  to  questions  which  were  e-mailed  to  the  TA.  On  the  other  hand,  it  was  observed  that 
some  students  can  not  state  their  questions  or  comments  clearly  when  they  are  writing  the  e-mail  messages.  Some  students 
reported  that  they  were  unable  to  get  any  assignment  marks  back  until  the  end  of  the  course  and  others  reported  that  assignments 
were  apparently  not  marked. 

The  course  has  a great  number  of  teaching  assistants  and  it  was  observed  that  different  TAs  gave  different  interpretations  of 
requirements.  It  was  observed  that  there  were  not  enough  TAs  in  some  lab  sections  to  provide  good  service  to  the  students.  In 
some  situations  it  was  observed  that  students  had  to  wait  more  than  fifteen  minutes  for  help.  Some  students  also  wanted  to  see  the 
course  instructors  in  the  lab  sections. 

These  criticisms  are  being  addressed  in  the  current  implementation  of  the  course. 

Conclusion 

This  paper  has  discussed  the  philosophy,  purpose  and  design  of  EDPY202:  Instructional  Applications  of  Technology  and 
provided  some  of  the  student  feedback.  The  course  is  offered  primarily  through  Web-Based  Instruction.  While  there  have  been 
some  “birthing  pains,”  and  we  expect  even  more  “growing  pains,”  we  are  very  pleased  with  the  results  so  far. 
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Abstract:  Quipunet,  a "virtual",  educational  organization,  with  the  use  of  Internet's  simplest  tools  has 
been  acquiring  valuable  experiences  since  the  latter  part  of  1995.  We  have  presented  virtual  seminars  in 
cooperation  with  UN-IDNDR  (United  Nations-  International  Decade  for  Natural  Disasters  Reduction); 
have  established  communication  networks  with  rural  areas  in  Peru,  allowing  us  to  help  the  victims  of 
disasters  ; have  used  same  networks  to  build  "bridges"  to  embrace  needy  schools;  and  have  sponsored 
virtual,  Writing  Contests  among  the  schools  of  Peru,  involving  students,  teachers,  schools  and  city 
officials. 


Introduction 

From  the  beginning  of  time  when  man  ventured  into  uncharted  territories,  with  no  one  to  guide  him,  with  no  rules  to 
follow,  he  had  to  improvise,  try  out,  make  mistakes,  and  try  again;  in  short,  he  had  to  Learn  by  Doing, 

The  Internet  was  to  us  that  uncharted  territory,  a new  frontier  that  offered  untold  wealth  of  information  and 
knowledge;  the  place  where  Quipunet  could  be  created,  http://www.quipu.net 

Quipunet  is  a grassroots,  "virtual",  non-profit  organization,  legally  incorporated  in  the  State  of  Washington  since 
November  of  1995.  Quipunet  is  made  up  of  Peruvian  volunteers  living  all  over  the  world,  who  have  formed  a 
cybernetic  link  to  help  our  country  of  birth. 

We  come  from  all  walks  of  life,  architects,  engineers,  professors,  students,  diplomats,  business  people,  and 
housewives.  We  reside  in  different  parts  of  the  world,  some  of  us,  computer  literate,  and  some,  unfamiliar  with  the 
Internet.  We  are,  truly,  a virtual  village. 

The  name,  Quipunet,  was  taken  from  the  Quechua  word  Quipus.  The  Quipus  were  used  by  the  Incas  as  a means  of 
maintaining  a statistical  inventory  of  their  Empire.  Long,  braided  strands  of  colored  yam  each  strand  had  a number 
of  varied  knots.  Using  a coded  system  where  each  color,  each  knot,  each  strand  represented  something,  the  Incas 
were  able  to  tally  and  record  their  resources.  Our  pages,  filled  with  information,  are,  our  electronic  Quipus. 

We  like  to  think  that  we  are  creating  a new  future  utilizing  modem  technology  -and  - ancient,  Inca  customs. 


Who  we  are 

What  we  have  learned  during  our  three  and  a half  short  years  of  our  existence  merits  the  telling.  We  have  been 
pioneering  the  Internet  with  no  one  funding  us. 

We  never  dreamt  that  our  small  group,  with  scant  means,  could  help  so  many,  from  so  far.  It  is  the  Internet  that  has 
given  us  these  new  tools,  to  learn  and  use;  it  is  the  Internet  that  has  been  our  "frontier"  where  everyday  we  find  more 
rich  lodes  to  mine.  Where  the  potential  to  really  make  a difference  is  open  to  anyone  no  matter  where  he  or  she 
lives.  All  anyone  needs  is  the  most  basic  knowledge  of  one  of  the  most  useful  tools  of  the  Internet;  the  electronic 
mail. 


Along  with  e-mail  comes  the  use  of  mailing  lists.  These  are  something  like  network  conversation  clubs,  or  special 
interest  groups.  San  Francisco  State  University  had  such  a list  for  Peruvians,  where  we  discussed  affairs  of  our 
country,  exchanged  recipes,  told  stories,  and  had  serious  debates.  It  was  during  one  of  these  debates  that  the  idea  to 
help  at  a distance  was  put  forth,  and  it  caught  the  imagination  of  us  all!  This  idea  turned  into  a project,  the  project 
into  an  organization.  A dream  was  bom!  It  was  very  emotional  seeing  the  messages  ’’count  me  in!”,  "count  me 
in!”.  What  started  with  only  a handful,  soon  grew  to  seventy,  then  eighty,  ninety  volunteers.  It  was  time  to 
organize  our  ideas  formally,  thus,  a committee  of  six  persons  was  formed:  Alberto  Delgado  in  Japan,  Herbert  Luna 
in  Brazil,  Fernando  Hermoza  in  Peru,  Ricardo  Corzo  in  Virginia,  USA,  Felipe  Polo  Wood  in  North  Carolina,  USA, 
and  Martha  Davies  in  Washington,  USA.  Six  persons  who  hardly  knew  each  other,  with  the  task  to  organize  this 
volunteer  group! 


The  First  Learning  Experience: 

The  first  few  days  we  all  had  excuses;  the  national  holidays,  vacations,  weekends,  etc.  Nobody  wanted  to  be  the 
"first".  I think  we  were  all  asking  ourselves:  "now,  what?"  Up  to  this  day,  none  of  us  can  recall  how  we  started,  but 
once  we  did  there  was  no  stopping  us.  After  186  e-mail  messages,  and  6 complete  corrections,  we  finished  the 
"draft"  of  our  By-laws,  Action  Plan,  Time  Schedule,  and  even  the  subscription  form!  All  done  in  three  and  a half 
weeks!  It  was  an  experience  we'll  never  forget!  The  experienced  "old  hands"  in  the  Internet,  as  well  as  the 
neophytes,  were  in  awe  at  seeing  the  documents  taking  shape  line  by  line,  day  by  day.  We  all  had  learned  how  to 
use  a virtual  table! 


The  Second  Learning  Experience: 

The  "draft"  was  presented  to  the  rest  of  the  members  and  Quipunet's  second  learning  experience  was  put  into 
motion.  Another  committee  was  formed  - it  was  their  turn  to  learn.  They  used  pre-registration  and  voting  "tables" 
to  make  it  easier  to  handle  and  to  avoid  a sea  of  e-mail.  Even  with  all  this  careful  planning  we  received  complains 
of  too  much  mail.  But  the  results  were  worth  all  the  trouble.  The  election  was  a totally  transparent  event! 

And  so,  Quipunet  was  bom,  in  November  of  1995,  with  the  following  mission: 

"To  Channel  resources,  materials,  and  knowledge  available  all  over  the  world  to  the  places  most  in  need  in  South 
America,  with  emphasis  on  Peru". 


The  Third  Learning  Experience: 

Our  first  official  incursion  on  this  new,  exciting  field  called  Internet,  was  planned  by  our  Vice-president,  Alberto 
Delgado,  who  at  that  time  was  living  in  Japan.  He  convinced  several  of  his  colleagues  at  work  and  from  the 
University  of  Kobe  to  contribute  information  gathered  after  the  terrible  earthquake  in  Kobe. 

In  January  of  1996.  We  presented:  "Kobe-Japan:  the  Earthquake  of 1995  and  its  Lessons**  - We  had  167  attendees 
from  16  countries.  We  were  delighted  to  learn  that  some  attendees  were  from  rural  areas.  The  presenters  were  top- 
notch  people  from  Japan  whose  experiences  were  still  fresh  in  their  minds. 


The  Fourth  Learning  Experience: 

Somebody  from  the  UN-IDNDR.  (United  Nations-  International  Decade  for  Natural  Disasters  Reduction)  took 
notice  of  this  seminar  and  asked  us  to  collaborate  with  them  on  their  first  virtual  seminar:  ** Solutions  for  Cities  at 
Risk**.  It  was  scheduled  for  September-October  1996.  http://www.quipu.net:  1 996 

That  event  attracted  458  attendees  from  58  countries.  Our  pages  registered  over  8,000  "hits”.  We  learned  about  the 
Master  plan  of  Istanbul  for  earthquake  hazards;  we  learned  of  the  vast  potential  of  the  Internet  to  link  individuals 
and  institutions  around  the  world;  we  learned  about  building  codes,  and  about  urban  protection  strategies. 
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On  September  -to-  October,  1997  in  cooperation  with  UN-IDNDR  we  presented:  "Floods  and  Droughts:  Issues  of 
the  2F^  Century",  http://www.quipu.net:  1 997 

This  seminar  was  a smashing  hit!  Presented  in  two  languages,  Spanish  and  English,  both  on  our  pages  and  on  two 
lists.  This  one  had  close  to  700  attendees  from  60  countries  and  our  pages  registered  over  9,000  hits.  This  seminar 
was  also  the  first  one  I was  involved  from  the  beginning  and  let  me  tell  you  how  impressed  I was. 

Picture  this:  From  Geneva,  the  word  goes  to  Alberto  Delgado,  our  vice-president  who  by  then  lived  in  Ecuador. 
Let's  have  the  seminar.  Alberto  gets  his  "team"  ready.  Three  Webmasters:  one  in  Australia,  one  in  Peru  and  one  in 
USA.  The  workplace:  San  Francisco  State  University,  where  our  server  resides,  and  where  we  had  two  more  team 
members.  Manager  for  the  Spanish  list  was  in  Taiwan;  and  manager  for  the  English  list,  Alberto,  in  Ecuador.  Two 
volunteers  in  Peru  took  care  of  the  publicity  in  Spanish.  That  year  we  had  eleven  translators,  five  in  Peru,  four  in  the 
States,  one  in  Spain  and  one  in  Taiwan. 

We  took  care  of  only  the  technical  part.  UN-IDNDR  took  care  of  the  presenters,  who  that  time  were  from  Australia, 
Burkina  Faso,  Argentina,  Spain,  USA,  Cambodia  and  many  other  countries.  Quite  an  undertaking... and  what  a good 
example  of  sharing  problems  and  solutions...  globally!  We  learned  about  the  community  response  to  the  Red  River 
flooding  in  Manitoba,  Canada;  the  risks  of  flooding  in  Australia;  the  risks  and  prevention  in  Valencia,  Spain...  we 
learned  this  and  more.  It  was  an  excellent  seminar  with  valuable,  first  hand  information. 

Last  year's  seminar:  "Prevention  Begins  with  Information,  http://www.quipu.net:  1 998  involved  the  media  as  a very 
important  partner  to  build  a culture  of  prevention.  Information  prior  to  disasters  is  crucial  to  the  saving  of  lives  and 
property.  The  media  has  to  help  educate  the  public  but  it  is  up  to  the  scientists  to  relay  clear  and  understandable 
messages.  We  had  several  representatives  of  the  International  media  and  also  representatives  of  different 
organizations  around  the  world. 

These  seminars  have  helped  developed  nations  as  well  as  developing  nations.  We  have  shared  wisdom  and  valued 
experiences  on  equal  footing.  Rich  technology  and  ingenuity  based  on  necessity,  make  good  partners. 

As  for  our  personal  learning.  Early  on  we  realized  that  we  had  to  keep  the  seminars  relatively  simple  in  order  to 
reach  the  rural  areas  where  the  cost  of  connectivity  is  so  high.  By  presenting  the  seminars  in  the  WWW  -page 
format,  without  the  frills  that  takes  so  long  to  load;  and  also  by  using  lists,  we  were  able  to  reach  people  that  have 
only  e-mail. 

The  Fifth  Learning  Experience: 

The  next  lessons  were  painful  to  learn,  because  it  was  through  tragedy  that  we  were  made  aware  of  the  Importance 
of  telecommunication  -and-  the  power  of  networking  during  disasters. 

The  visit  from  **E!  Nino'* 

On  February,  1998  El  Nino,  the  current  that  had  always  been  welcomed  as  the  giver  of  bounty  to  my  country,  that 
year  was  the  destroyer  of  life,  property,  and  hope,  bringing  along  heavy  rains  in  places  where  never  rained,  washing 
away  whole  towns  that  were  ill  prepared  to  resist  the  flooding. 

When  Edwin  San  Roman,  one  of  our  volunteers  who  lives  in  Lima,  Peru,  heard  of  the  devastation  caused  by 
flooding  in  Ica,  a city  3 hrs  away  from  Lima,  he  took  off  with  an  industrial  bread  making  machine  in  the  back  of  his 
pick-up.  With  the  help  of  the  Catholic  Church,  he  installed  the  machine  in  an  empty  school. 

He  sent  a message  to  our  list  asking  for  help.  I forwarded  his  message  and  urged  our  members:  "Let's  help  Edwin, 
to  help  Ica".  As  the  pledges  arrived  I would  post  the  amount  and  the  running  total  to  our  mailing  list.  Edwin,  in 
Lima,  reading  the  totals,  would  in  turn,  use  his  credit  to  buy  flour  and  yeast.  His  messages  kept  us  abreast  of  what 
was  happening.  He  made  each  of  us  participants.  Perhaps  not  in  body,  but  in  spirit.  We  were  there — helping 
make  30,000  rolls  of  bread  a day  to  feed  the  homeless,  hungry  people.  We  held  the  fort — until  the  big 
organizations  took  over.  According  to  the  bishop  of  Ica,  it  was  the  best  organized  team  effort!  And  all  this  was 
possible-  thanks  to  e-mail  and  the  mailing  lists! 
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The  next  story  is  one  with  a lesson  we  are  taking  to  heart. 

Another  volunteer,  Raul  Nakasone,  resident  of  the  USA,  spends  his  vacations  in  Guadalupe,  a small  town  in  Peru. 
Last  vacation  he  persuaded  Hugo  Cortez,  President  of  the  Board  of  Directors  of  the  "Public  Aid  Office"  of 
Guadalupe,  to  invest  in  a computer  for  the  town.  It  would  be  a "window  to  the  world"  he  told  them.  And  so  it  was. 
When  the  computer  was  installed  only  three  people  knew  how  to  use  it.  Before  Raul  left  he  taught  a few  more. 

Back  in  Olympia,  WA,  Raul  kept  in  contact  with  his  town  via  e-mail.  And  he  gave  them  instructions  on  how  to  be 
prepared  for  the  rains  that  were  supposed  to  arrive.  Of  course  nobody  was  prepared  for  the  magnitude  of  the  disaster 
that  was  to  hit  them.  The  heavy  rains  caused  mudslides  that  destroyed  the  roads.  The  town  was  isolated.... but  not 
abandoned.  Raul  wrote,  "Be  sure  to  boil  all  the  water  to  prevent  cholera".  The  people  followed  his  advice  and 
bought  two  wood-stoves  and  kept  them  going  day  and  night.  But  two  stoves  weren’t  enough  for  the  whole  town.  So 
Raul  asked  Quipunet  for  help  and  since  we  had  been  collecting  funds,  we  had  $1,000  to  send  to  Guadalupe.  Raul’s 
instruction-at-a-distance  went  like  this:  "Sra.  Juanita-  (she  is  the  president  of  the  association  of  mothers  in 

Guadalupe),  be  sure  to  buy  a notebook  and  keep  track  of  all  the  money,  who  is  getting  what.  Have  everybody  sign. 
You  and  your  committee  can  decide  what  your  most  important  needs  are."  One  computer  in  that  town  and  only  a 
handful  of  people  that  knew  how  to  use  it!  But  it  was  enough  to  help  them! 


The  Sixth  Learning  Experience: 

One  of  our  most  successful  projects  this  one  was  bom  at  one  of  the  lists  of  interest  called  SEDERUL  (Servicio  de 
Desarrollo  Rural).  We  had  wanted  to  promote  an  interchange  of  ideas  and  experiences  in  three  important  fields. 
Education,  Agriculture  and  Rural  Tourism.  Combining  these  three,  the  Work  Group  in  charge  of  this  project 
decided  to  have  a writing  contest  for  high  school  students  from  public  schools  in  Peru.  The  theme  they  choose  was 
"The  Natural  Resources  of  my  Land." 

Invitations  were  sent  to  1 30  schools,  and  to  involve  the  whole  community  prizes  were  offered  for  the  student,  the 
teacher,  the  school  and  the  community. 

First  prize  was  a computer  for  the  school,  and  $100  US  for  the  student  and  another  $100  for  the  teacher,  prizes  went 
all  the  way  down  to  $10. 

This  project,  like  all  the  other  projects  we  have  worked  at,  was  planned  and  executed  at-a  distance.  With  small 
groups  at  each  end  taking  care  of  the  details  like  diplomas,  prizes,  etc. 

It  was  an  outstanding  success  and  one  we  are  going  to  repeat  again  this  year! 

Other  projects  we  are  working  on: 

Embrace  a School: 

We  have  created  nine  work  groups  in  the  USA,  which  through  benefits  such  as  dinners,  dances,  and  musicals  are 
helping  needy  rural  schools.  Our  aim  is  to  provide  these  schools  with  one  computer  each.  We  believe  that 
computers  and  several  donated  educative  programs,  makes  a better,  long-term,  cost-efficient  investment  that  will  last 
for  a number  of  years.  We  want  to  provide  instruction  and  education  -at-a-distance  not  only  to  the  kids  but  also  to 
the  teachers,  opening  a window  to  the  world.  Educational  materials  are  almost  non-existent  in  many  of  the  rural 
schools. 

The  reports  from  an  inspection  trip  on  November  1998  were  very  encouraging.  All  the  projects  were  working  as 
planned;  the  results  better  than  expected.  The  following  are  the  best  examples: 

1-  Embracing  the  Little  Farmers,  in  Cusco,  Peru.  We  are  working  with  an  organization  formed  by  young  business 
owners,  giving  of  their  time  and  whose  aim  is  to  teach  native  kids  the  best  ways  to  farm  their  small  lots  of  land. 
Whatever  the  kids  can  produce,  and  if  of  good  quality,  it  is  bought  at  market  prices  by  the  business  owners.  This  is 
the  best,  well-planned,  full-cycle,  program  we  have  seen. 

2-  A Training  Center  for  Street  Kids.  These  are  kids  that  only  two  years  ago  were  stealing  in  the  streets.  With  our 
help,  they  are  being  trained  to  make  shoes.  The  self-esteem,  and  pride  while  showing  their  product,  was  beautiful  to 
see. 

3-  The  Women  Carpenters.  A whole  town  had  to  be  resettled  after  the  flooding  caused  by  El  Nino  that  left  them 
homeless.  Our  volunteers  who  visited  their  camp,  after  seeing  the  women  making  benches  from  apple  crates. 
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decided  to  pay  for  the  training  and  material  to  enable  them  to  help  themselves.  After  3 months  of  training,  the 
women  delivered  benches,  tables,  and  bookshelves  for  the  school.  They  also  learned  by  doing! 

My  Friend:  The  Book: 

We  collect  used  text  books  and  when  we  have  enough  boxes,  we  send  them  to  different  universities. 


What  have  we  learned? 

That  this  era  of  information  and  knowledge  will  drastically  change  our  world  as  we  know  it. 

That  we,  the  people  from  developing  countries,  have  to  use  more  imagination  and  creativity,  in  lieu  of  money  we 
don't  have.  But  it  can  be  done. 

That  we  must  use  simple  tools  to  reach  the  people  that  only  has  simple  technology. 

That  the  communication  networking  is  the  most  important  component  for  the  dissemination  of  information. 

That  the  Internet  is  open  to  all  the  world  to  be  mined  with  sophisticated  technology,  or  with  rudimentary  tools. 

That  we  all  have  valuable  information  to  exchange.  Traditional  knowledge,  in  exchange  for  up-to-date  information. 

That  there  is  room  for  us  all  in  this  new  frontier.  We  all  are  needed  to  fill  out  the  different  jobs  being  created. 

And  most  important  of  all,  that  we  have  a unique  chance  to  really  stop  the  devastation  of  the  world  by  channeling 
education  and  information  so  people  can  better  understand  and  take  care  of  their  quality  of  life,  preserve  their 
environment,  and  their  own  culture. 
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Abstract:  The  study  outlined  in  this  paper  was  part  of  a larger  study  that  examined  the  use  of  interactive 

multimedia  in  motivating  and  engaging  adult  learners.  This  paper  describes  the  use  of  the 
multimedia  microworld  to  explore  and  promote  the  use  of  self-regulated  learning,  and  in 
particular  examines  the  degree  to  which  higher  order  thinking  was  achieved  and  concludes  by 
suggesting  ways  in  which  these  learner  effects  could  be  achieved  to  a higher  degree. 


Introduction 

As  computers  become  more  and  more  integrated  into  higher  education  activities,  it  is  imperative  that  learning 
programs  be  designed  to  exploit  the  flexibility  that  resource-based,  self-paced  learning  can  provide.  In  order  to 
use  the  computer  as  a primary  learning  tool,  the  very  nature  of  learning  with  technology  has  to  be  rethought. 
Interactive  Multimedia  has,  to  date,  failed  to  fulfil  its  early  promise  of  making  a significant  difference  to  the  way 
in  which  learners  learn  (Russell,  1997).  One  of  the  reasons  for  this  may  be  that  multimedia  programs  are 
designed  for  a single  focal  point  teaching  through  a linear  sequential  process,  with  students  confined  to  a narrow 
learning  experience.  However,  with  a well  constructed  multimedia  program,  the  experience  of  the  student 
changes  to  one  of  cooperative,  contextual  learning  where  students  are  left  to  explore  the  learning  environment  in 
their  own  time,  at  their  own  pace  and  in  any  order  of  their  choosing.  This  form  of  learning  has  the  potential  to 
increase  the  motivation  and  engagement  of  the  students  (Stoney  & Oliver,  1997). 

This  paper  will  explore  the  notion  that  students  who  learn  in  an  applied  setting,  such  as  a microworld,  will 
experience  cognitive  engagement  and  motivation  through  the  relevance  of  the  material  to  the  students’  real  world 
or  macroworld. 

Cognitive  Engagement 

Cognitive  engagement  is  observable  when  the  learners  are  giving  sustained,  engaged  attention  to  a task  requiring 
mental  effort;  and  authentic,  useful  learning  is  produced  by  extended  engagement  in  optimally  complex  cognitive 
activities  (Como  & Mandinach,  1983).  Cognitive  engagement  and  motivation  are  inextricably  linked  together 
through  mental  representations,  monitoring,  and  evaluation  of  responses  and  strategic  thinking.  The  amount  of 
cognitive  effort  expended  is  an  appropriate  index  of  motivation  as  it  relies  on  the  learner  focusing  on  mastering 
the  learning  task  and  maintaining  a high  sense  of  personal  efficacy  (Shunk,  1989). 

The  highest  form  of  cognitive  engagement  is  self-regulated  learning  (Como  & Mandinach,  1 983),  where  learners 
plan  and  manage  their  own  learning  and  have  a high  degree  of  personal  control  and  autonomy.  This  is  critical  to 
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beginning  and  ongoing  motivation  through  specific  cognitive  activities,  and  assumes  that  the  students’  prior 
learning  will  act  in  concert  with  the  instruction  to  determine  the  types  of  cognitive  engagement  they  exhibit,  such 
as  attention  to  specific  information,  analysis  and  synthesis  of  information,  visualisation  and  ability  to  filter  out 
relevant  from  irrelevant  information.  Effective  cognitive  engagement  involves  selectivity,  task-specific  planning, 
drawing  on  previous  experience  and  transferring  new  knowledge  to  the  world  outside  the  classroom.  The  use  of 
interactive  multimedia  can  foster  and  develop  cognitive  engagement  through  its  ability  to  attract  and  hold 
students’  attention  and  focus.  The  very  attributes  that  provide  the  potential  to  develop  cognitive  engagement, 
however,  are  also  those  that  hold  strong  prospects  for  limiting  and  impeding  learning.  Research  into  programs 
and  applications  which  encourage  self-regulation  and  learner  control  frequently  reports  problems  associated  with 
learners’  inabilities  to  successfully  monitor  and  manage  their  learning,  and  to  remain  engaged  and  focussed  on 
the  learning  tasks.  Solutions  that  seek  to  enforce  control  or  direction  over  learners,  help  to  encourage  the  focus 
but  are  able  to  stifle  the  freedom  associated  with  the  forms  of  cognitive  processing  and  engagement  associated 
with  higher  order  learning. 

This  paper  describes  a study  in  which  a multimedia  program — developed  for  use  in  a university  course  in  the 
business  field — was  implemented  in  controlled  settings  that  enabled  an  exploration  of  the  higher  order  learning 
processes  of  learners.  The  program.  Principles  of  Financial  Investment,  was  designed  and  developed  to  support 
and  encourage  independent  and  self-regulated  learning.  Deliberate  aspects  of  the  design  included  elements  to 
motivate  and  engage  adult  learners  and  past  papers  have  explored  and  described: 

■ the  instructional  design  principles  guiding  program  development  (Stoney  & Oliver,  1 998); 

■ learner  response  to  the  various  design  elements,  (Stoney  & Oliver,  1 998);  and 

■ the  forms  of  motivation  and  engagement  among  adult  learners  supported  by  the  learning  environment 
(Stoney  & Oliver,  1998), 

Principles  of  Finance 


The  IMM  program  that  was  planned  to  replace  a conventional  course  in  business  finance,  was  carefully  designed 
to  incorporate  not  only  the  learning  sequences  needed  to  bring  about  the  cognitive  outcomes  but  also  to  include  a 
strong  affective  component.  The  microworld  consists  of  an  office  building  called  “Investment  House”.  Upon 
entering  the  building  students  are  asked  to  register  at  the  reception  desk  allowing  the  program  to  be  personalised. 

There  are  four  main  working/operating  areas;  the  stock  exchange,  a stock  broker,  the  Institute  of  Sound 
Investment  and  the  student’s  office.  The  nature  of  the  interactions  was  also  considered  to  maximise  engagement 
and  motivation,  so  a passive  click  and  read  type  program  was  avoided.  The  interaction  style  supported  the 
narrative  intention,  enhancing  the  participant’s  relationship  with  the  product  (Stoney  & Oliver,  1 997), 

The  Study 

The  multimedia  program  was  incorporated  into  the  course  of  study  for  students  studying  Finance,  Eight  of  the 
students  were  divided  into  dyads  for  the  purpose  of  this  study  and  used  the  program  in  a collaborative  fashion. 
The  students  worked  in  a self-regulated  fashion  to  complete  the  learning  activities  supported  by  the  program  in 
controlled  settings  that  enabled  the  researchers  to  observe  their  activities  and  to  record  their  communication  and 
interactions.  These  students  participated  in  an  in-depth  study  and  their  discourse  was  transcribed  and  analysed  to 
determine  the  forms  and  scope  of  cognitive  engagement  and  to  enable  exploration  of  the  higher  order  learning 
that  occurred. 

Findings 

The  resultant  conversations  revealed  several  levels  of  cognitive  engagement  and  activity  among  the  dyads.  To 
further  explore  and  analyse  the  instances  of  cognitive  engagement,  an  array  of  learning  activities  were 
categorised  and  identified  in  the  dyad  activities.  In  the  following  discussion,  we  have  separated  the  analysis  and 
discussion  of  the  forms  of  talk  and  cognitive  activity  into  sections  that  describe  lower  and  higher  order  thinking 
and  learning. 
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Lower  Order  Activity 


Lower  order  activity  can  be  identified  as  any  operational  tasks  requiring  more  than  mechanistic  activities, 
needing  almost  no  cognitive  engagement,  problem  solving  or  decision  making  (Stoney  & Oliver,  1998).  The 
students*  lower  order  activities  were  divided  into  three  main  sections — lower  order  talk,  information  seeking, 
procedural,  and  browsing.  Table  1 provides  an  overview  of  the  categories  by  which  the  lower  order  talk  and 
thinking  was  identified  (after  Herrington,  1997): 


Table  1.  Summary  chart  of  classiflcation  of  talk  by  pairs 


Category 

Sub> 

Category 

Definition 

Summary  of  student  talk 

Example  of  type 

Procedural 

Equipment 

Any  discussion  regarding 
the  hardware. 

These  comments  related  to  the 
computer  crashing. 

M:  Oh,  the  computer’s  stuck.  We’ll 
have  to  start  again. 

Software 

Any  discussion  regarding 
the  operation  of  the 
program  or  problems  with 
the  software. 

There  were  very  few  comments  of 
this  nature,  but  one  pair  could  see  the 
desktop  behind  the  program  which 
caused  one  of  them  some  difficulties. 

K;  Do  you  find  it  a bit  disconcerting 
with  all  this  behind  it? 

B;  It  is  a bit,  yeah 
K:  You’re  trying  to  read  it  and 
there’s  all  the  little  bits. 

Task 

Any  discussion  regarding 
the  requirements  of  the 
tasks. 

This  type  of  talk  related  to  the  ways  in 
which  the  students  decided  to  move 
through  the  program  and  the  order  of 
their  tasks. 

J:  What  do  you  reckon?  Do  you 
think  we  have  to  go  back  to  the 
Institute? 

Browsing 

Any  exchange  where  the 
students  were  talking 
about  looking  to  see  what 
was  available. 

There  were  very  few  incidences  of 
this  type  of  talk.  The  collaborating 
students  did  very  little  browsing. 

S;  Let’s  have  a quick  look  round 
first. 

M;  Good  idea,  see  what’s  here. 

Lower  Order 

Any  exchange  which  was 
of  a routine  nature,  such 
as  agreeing,  commenting 
without  applying  thought 
or  judgement. 

Talk  in  this  category  included  many 
routine  comments  about  the  task. 

G:  Tell  you  what  this  program  really 
brings  some  concepts  back. 

Infonnation 

Seeking 

Any  exchange  where  the 
students  were  actively 
seeking  infonnation,  but 
were  not  making 
judgements  about  it. 

Comments  in  this  category  were 
made  when  students  decided  to  look 
for  information  provided  to  help  them 
make  the  investment  decisions.  There 
were  comparatively  few  comments  in 
this  area  as  most  of  the  pairs  worked 
out  their  own  information. 

M:  Let’s  check  what  the  economic 
forecasts  have  to  say  about  all 
this. 

Off  task  activities  are  those  that  occur  when  the  students  lose  engagement  with  the  program.  The  off  task 
activities  noted  in  the  study  in  question,  were  caused  by  students  being  tired  or  listening  to  what  other  students 
were  doing.  The  technology  also  caused  some  distraction  from  the  program  when  it  crashed  or  failed  to  operate 
as  expected.  Table  2 shows  the  percentage  of  time  each  pair  spent  in  the  lower  order  activities  as  defined  above. 


Table  2.  Percentage  of  time  spent  in  lower  activities  by  each  pair 


Paiiptf: 

t-  ■ 

Procedural,  equipment, 
s/ware,  task 

Browsing 

Lower  order 
talk 

Information 

seeking 

Total  time  in  Lower 
order  activities 

, r 

10 

1 

18 

3 

32 

r 2 

22 

1 

16 

9 

48 

3 

15 

3 

25 

7 

50 

4 

10 

4 

25 

13 

52 

It  was  noted  from  the  in-depth  study  that  generally  the  students  spent  very  little  time  browsing,  instead  preferring 
to  talk  their  way  through  the  program  and  the  problems.  There  was  quite  a high  percentage  of  procedural  talk, 
probably  due  to  the  fact  that  the  program  was  a test  version  and  had  some  minor  glitches.  The  percentage  of  time 
spent  in  information  seeking  was  generally  low,  with  the  exception  of  pair  4,  who  spent  more  than  10%  of  their 
time  in  this  activity.  The  incidence  of  lower  order  talk  was  much  higher  than  expected,  and  this  could  be 
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attributed  to  the  fact  that  the  program  was  fairly  complex  and  therefore  students  spent  time  reading  aloud  to  each 
other,  discussing  routine  factors  such  as  where  to  go  next  and  making  comments  that  did  not  apply  thought  or 
judgement.  Although  there  were  many  instances  where  the  lower  order  talk  was  intermingled  with  the  higher 
order  talk,  generally  most  of  the  students  spent  the  initial  part  of  their  time  discussing  procedures  and  actively 
seeking  information  before  moving  on  to  higher  order  thinking. 

The  initial  high  percentage  of  lower  order  talk  by  students  using  the  program  was  expected,  as  the  students 
explored  and  experimented  with  the  structure  of  the  program,  and  this  dropped  away  towards  the  middle  and  end 
of  the  time  as  students  became  more  familiar  with  the  structure  and  content  of  the  program.  Although  the  amount 
of  time  spent  in  lower  order  activities  varied  between  the  dyad  groups,  they  spent  fairly  similar  amounts  of  time 
browsing.  The  main  differences  between  the  groups  were  information  seeking  (ranging  from  3%  to  13%), 
procedural  issues  (ranging  from  10%  to  22%),  and  lower  order  talk  (ranging  from  16%  to  25%).  The  types  of 
issues  which  contributed  to  the  lower  order  activities  were  problems  with  the  system;  for  example  when  one  of 
the  students  was  distracted  by  the  desktop  showing  behind  the  program,  or  when  the  system  crashed.  Some  of  the 
students  also  found  that  they  needed  to  be  directed  to  the  paper-based  instructions,  although  they  still  tended  to 
want  to  talk  through  their  navigational  problems  than  read  a manual.  Much  of  the  lower  order  activity  involved 
student  talk  which  was  of  a routine  nature,  where  students  were  agreeing,  reading  aloud,  and  commenting 
without  Judging.  A much  smaller  percentage  of  the  lower  order  activities  involved  information  seeking  where 
students  were  actively  looking  for  information  and  discussing  it  without  Judging  it.  Although  much  of  the  lower 
order  talk  was  necessary  in  terms  of  the  students  working  out  how  the  program  worked  and  commenting  on 
incidents  occurring  in  the  program,  some  of  the  occurrences  could  have  been  eliminated  by  ensuring  that  the 
program  was  more  stable,  and  by  including  more  problems  for  the  students  to  solve. 

Higher  order  thinking 

An  important  facet  of  developing  higher  order  thinking  skills  is  the  ability  to  reflect  on  the  learning  experience 
and  incorporate  new  knowledge  with  the  pre-existing  knowledge.  Although  the  importance  of  reflection  is  well 
documented  (eg  Laurillard,  1995;  Merriam,  1993),  it  is  rarely  supported  as  it  is  internalised  by  the  student  often 
without  the  instructor  being  aware  of  the  process  (Laurillard,  1995).  Teacher  oriented  classes  provide  the  least 
amount  of  time  for  reflection,  but  self-paced,  resource-based  instruction,  such  as  multimedia,  is  capable  of 
promoting  the  links  between  conceptual  and  experiential  learning.  This  can  be  achieved  by  building  in  a variety 
of  scenarios  that  offer  alternatives  to  students,  giving  them  choices  from  which  to  change  their  understanding  of 
a concept  whilst  being  able  to  relate  it  to  their  own  world. 

Traditional  teaching  often  employs  a lock-step  approach  to  the  acquisition  of  skills,  with  lower  order  skills  being 
taught  first.  There  is  a belief  that  lower  order  skills  are  prerequisites  to  higher  order  skills  and  that  mastery  of 
lower  order  skills  automatically  leads  to  higher  order  skills.  In  fact,  lower  order  and  higher  order  skills  can  be 
taught  concomitantly,  with  students  mastering  both  levels  as  they  apply  their  learning,  rather  than  learning  the 
skills,  practicing  and  then  applying  them.  This  prevents  skills  being  learned  in  isolation  and  students  then  having 
to  relearn  their  application  to  real  world  tasks.  The  students  are  active  in  the  learning  process,  applying  problem- 
solving in  context  which  in  turn  aids  in  the  acquisition  of  skills,  giving  the  students  a reason  to  learn  and  helping 
them  how  to  learn.  In  the  study  described  in  this  paper,  higher  order  thinking  was  classified  into  five  main 
categories:  planning/strategy;  uncertainty;  predicting/imposing  meaning;  multiple  perspectives;  and  coaching 
(Herrington,  1997).  Table  3.  gives  a breakdown  of  the  classification  of  the  categories. 

Table  3.  Summary  chart  of  classification  of  higher  order  talk  by  pairs 


Category 

Definition 

Example  of  student  talk 

Planning/ 

Strategy 

Decision  making  (Lewis  & Smith,  1993), 
considering  strategies  and  planning  events. 

“Yeah,  well  1 reckon  that  Castle  Mining  is  overvalued  so  why 
don't  we  check  out  what  the  broker  has  to  say  about  that, 
and  then  maybe  look  at  the  quarterly  reports  to  see  if  there’s 
some  sort  of  a trend?"  (Steve  talk). 

Uncertainty 

Any  statement  expressing  uncertainty.  Any  question 
clarifying  a point  (Ennis,  1993). 

"One  point  one,  where’s  the  12  come  from,  percent,  where’s 
the  16%  come  from?  Don’t  know?"  (Mark  talk) 

Predicting/ 

Imposing 

Making  a prediction,  deciding  what  to  believe,  solving 
non-routine  problems  (Lewis  & Smith,  1993),  making 

“Net  present  value  is  the  current  value  or  adjusted  value  on 
the  discounted  cash  flow  rates,  huh”  (Gary  talk) 
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meaning 

judgements  (Ennis,  1993).  Exploration  of  the  problem, 
using  inference,  induction  and  deduction  (Garrison, 
1992). 

Multiple 

perspectives 

Establishing  different  perspectives  (Duchastel,  1990). 
Grounding  in  the  real  world  (Garrison,  1992). 

1 know,  isn't  that  interesting.  Northern  Queensland,  Western 
Australia,  Cuba.  1 didn’t  know  Cuba  produced  gold,  1 don't 
know  what  to  do  with  the  market  at  the  moment  ‘cos  you 
wouldn’t  buy  gold  at  the  moment"  (Kelly  talk). 

Coaching 

Accessing  hints  help  in  the  form  of  support,  advice,  and 
explanations.  Students  teaching  each  other  the  material. 

“Yes  D is  probably  whatever  the  figure  that  they  say  starting 
6 years  from  now  but  its  only  5periods  working  here  but  it 
must  be  in  the  backtrack  of  the  5 periods. .’’(Brad  talk). 

Incidences  of  higher  order  thinking 


Although  three  of  the  pairs  spent  half  their  time  or  more,  in  higher  order  thinking,  it  was  felt  that  more  student 
time  should  have  been  spent  in  higher  order  thinking.  Part  of  the  reason  for  the  lower  than  expected  number  of 
incidences  could  have  been  the  fact  that  the  program  was  very  procedural ly  oriented  and  therefore  there  was  a 
high  incidence  of  lower  order  talk.  Analysis  of  the  time  spent  in  each  facet  of  higher  order  thinking  highlights  the 
fact  that  students  did  not  allocate  as  much  of  their  talk  time  to  grounding  their  new  knowledge  with  existing 
knowledge  as  had  been  expected.  One  pair  also  spent  very  little  time  in  planning  and  strategic  thinking,  and 
another  pair  did  not  coach  each  other  as  much  as  the  other  three  pairs. 


Table  4.  Percentage  of  time  spent  by  pairs  in  Higher  Order  talk 


Pair.'U 

NumbeiS 

Planning 

/Strategy 

Uncertainty 

Predicting/ 

Imposing 

Meaning 

Multiple 

Perspectives 

Coaching 

Total  time  spent  in 
HOT 

r X 

12 

8 

36 

8 

4 

68 

12 

4 

26 

5 

5 

52 

::f  3|  1 

19 

8 

21 

1 

1 

50 

4 

15 

21 

2 

6 

48 

Although  the  learners  were  crafting  solutions  to  problems,  leading  to  higher  order  talk,  building  more  problems 
into  the  program  would  increase  the  levels  of  planning  and  strategic  thinking. 

The  students’  talk  gradually  took  on  more  higher  order  aspects  as  students  became  absorbed  by  the  content  and 
began  planning  their  investment  strategies.  The  degrees  of  higher  order  talk  varied  between  the  four  dyads,  as  did 
the  incidents  that  triggered  it.  These  triggering  events  varied  from  relating  real  world  events  to  the  program’s 
events,  relating  new  information  back  to  old  learning  acquired  in  other  units,  judging  new  information  and 
projecting  outcomes  of  decisions.  The  incidence  of  coaching  also  occurred  to  various  degrees,  although  only  one 
pair  utilised  it  in  a very  limited  way. 

The  dyads  tended  to  have  fairly  different  examples  of  higher  order  talk,  with  Pair  4 doing  veiy  little 
planning/strategic  thinking  (2%)  and  Pair  3 doing  19%,  On  the  other  hand.  Pair  4 spent  quite  a lot  of  time  (15%) 
clarifying  statements  and  21%  of  their  time  predicting  and  imposing  meaning.  Pair  1 spent  the  longest  percentage 
of  time  on  prediction  (36%)  and  also  the  longest  percentage  of  time  on  establishing  different  perspectives  (8%). 
Pair  4 spent  the  most  time  accessing  the  on-line  support  and  help  (6%)  which  is  perhaps  reflected  in  their 
monetary  results  and  also  some  of  their  comments  about  their  lack  of  comfort  with  the  program  theoiy,  A 
comparison  of  the  time  spent  and  totals  profits  accumulated  with  the  degree  of  lower  and  higher  order  talk  show 
a clear  correlation.  Pair  1 who  had  the  highest  percentage  of  higher  order  activities  made  the  most  profit,  and  this 
correlation  holds  true  for  the  other  three  dyads.  It  also  holds  true  for  the  time  spent  in  the  program  with  the 
exception  of  Pair  2 who  spent  longer  in  the  program  than  Pair  1 but  made  less  money. 
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Table  5.  Comparison  of  lower  and  higher  order  activities  with  time  spent  and  total  profits  made 


Pair  Number 

Lower  Order 

Activities  (%) 

Higher  Order 

Activities  (%) 

Time  spent  in 
program  (minutes) 

Total  profit  ($) 

1 

32 

68 

103 

39,345 

2 

48 

52 

112 

35,239 

3 

50 

50 

101 

23,500 

4 

52 

48 

92 

22,080 

Summary  and  Conclusions 

It  is  clear  from  this  study  that  the  use  of  well  designed  interactive  microworlds  leads  to  learner  cognitive 
engagement  which  in  turn  promotes  greater  degrees  of  concentration  for  longer  periods  of  time.  Further,  if  well 
designed,  a program  will  drive  learners  towards  greater  levels  of  higher  order  thinking  which  in  turn  will  assist  in 
the  ready  attainment  of  implicit  goals  set  by  the  program  itself  and  explicit  goals  personally  set  by  the  learners. 
The  use  of  the  game  and  its  motivating  and  engaging  factors  created  an  environment  that  resulted  in  high  levels 
of  higher  order  thinking,  through  triggering  events.  The  program  provided  information  and  data  from  various 
perspectives  and  sources  and  facilitated  a real  connection  between  that  information  and  information  and  learning 
already  possessed  by  learners.  The  program’s  materials  encouraged  and  indeed  required,  learners  to  judge  and 
assess  the  credibility  of  potentially  conflicting  information,  and  to  develop  strategies  to  resolve  those  conflicts,  to 
clarity  issues,  to  solve  problems,  to  experiment,  to  think  strategically  and  critically,  to  sift  information,  to  predict 
and  impose  meaning  and  to  make  judgements  and  decisions  in  spite  of  apparent  contradictions. 

The  materials  within  the  program  were  written  to  be  understandable,  interesting  and  relevant,  not  isolated  from 
reality,  and  attempted  to  link  theory  to  practice  by  requiring  the  application  of  content.  In  other  words,  the 
program  attempted  to  focus  on  the  functional  context  of  acquired  knowledge  through  the  integration  of  skills  in 
an  authentic  and  realistic  manner  and  setting.  In  an  attempt  to  minimise  the  level  of  lower  order  thinking,  which 
requires  by  definition,  little  sustained  mental  effort,  the  program  was  designed  in  such  a way  as  to  reduce  the 
level  of  operational  mechanistic  tasks  and  allowed  easy  flow,  exploration  and  ready  access  to  information.  In 
order  to  further  reduce  the  levels  of  lower  order  thinking,  an  increase  in  problem  solving  could  be  encouraged  by 
providing  an  increased  opportunity  for  reflection,  providing  cognitive  scaffolds  to  move  learners  more  quickly  to 
levels  of  higher  order  thinking,  and  by  creating  cognitive  dissonance  in  both  teaching  materials  and  strategies. 
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Abstract 

WWWIC,  the  NDSU  Worldwide  Web  Instructional  Committee,  is  engaged  in  developing  a range  of 
Virtual  Environments  for  Education.  These  projects  span  a range  of  disciplines,  from  Earth  Science  to 
Anthropology,  and  from  Business  to  Biology.  However,  all  of  these  projects  share  a strategy,  a set  of 
assumptions,  an  approach  to  assessment,  and  an  emerging  tool  set,  which  allows  each  to  leverage 
from  the  insights  and  advances  of  the  others. 

Introduction 

The  NDSU  World  Wide  Web  Instructional  Committee  (WWWIC)  is  currently  engaged  in  several 
virtual/visual  development  projects:  three  are  NSF-supported,  the  Geology  Explorer  (Saini-Eidukat,  Schwert, 
and  Slator,  1998;  Slator  et  al.,  1998;  Slator,  Schwert,  Saini-Eidukat,  1999;  Schwert,  Slator,  Saini-Eidukat, 
1999),  the  Virtual  Cell  (McClean,  1998;  White,  McClean  and  Slator,  1999)  the  Visual  Computer  Program  and 
the  ProgrammingLand  MOO  (Hill  and  Slator,  1998  Slator  and  Hill,  1999).  These  have  shared  and  individual 
goals.  Shared  goals  include  the  mission  to  teach  Science  structure  and  process:  the  Scientific  Method, 
scientific  problem  solving,  deduction,  hypothesis  formation  and  testing,  and  experimental  design.  The 
individual  goals  are  to  teach  the  content  of  individual  scientific  disciplines:  Geoscience,  Cell  Biology, 
Computer  Science. 

In  addition,  WWWIC  is  applying  what  has  been  learned  in  Science  education  to  new  domains: 
history,  microeconomics,  and  anthropology.  Further,  WWWIC  has  active  research  projects  in  two  highly 
related  areas:  subjective  student  assessment  and  tools  for  building  virtual  educational  environments. 

The  WWWIC  program  for  designing  and  developing  educational  media  implements  a coherent 
strategy  for  all  of  its  efforts.  This  strategy  is  to  deploy  teaching  systems  that  share  critical  assumptions  and 
technologies  in  order  to  leverage  from  each  other's  efforts.  In  particular,  systems  are  designed  to  employ 
consistent  elements  across  disciplines  and,  as  a consequence,  foster  the  potential  for  intersecting  development 
plans  and  common  tools  for  that  development. 

Geology  Explorer 

Geology  Explorer  is  a virtual  world  where  learners  assume  the  role  of  a geologist  on  an  expedition  to 
explore  the  geology  of  a mythical  planet.  Learners  participate  in  field-oriented  expedition  planning,  sample 
collection,  and  "hands  on"  scientific  problem  solving.  The  Geology  Explorer  world  is  simulated  on  an  Object 
Oriented  Multiuser  Domain,  the  Xerox  PARC  LambdaMOO  (Curtis  1 992).  A text-based  version  of  Geology 
Explorer  was  tested  in  an  introductory  geology  class  during  the  Summer  1998.  Results  of  that  test  were  used 
to  prepare  for  a larger  test  in  the  same  geology  class  during  Fall  1998.  A graphical  user  interface  to  the 
Geology  Explorer  is  in  the  process  of  design. 

To  play  the  game,  students  are  transported  to  the  planet's  surface  and  acquire  a standard  set  of  field 
instruments.  Students  are  issued  an  "electronic  log  book"  to  record  their  findings  and,  most  importantly,  are 
assigned  a sequence  of  exploratory  goals.  These  goals  are  intended  to  motivate  the  students  to  view  their 
surroundings  with  a critical  eye,  as  a geologist  would.  Goals  are  assigned  from  a principled  set,  in  order  to 
leverage  the  role-based  elements  of  the  game.  The  students  make  their  field  observations,  conduct  small 
experiments,  take  note  of  the  environment,  and  generally  act  like  geologists  as  they  work  towards  their  goal 
of,  say,  locating  a kimberlite  deposit.  A scoring  system  has  been  developed,  so  students  can  compete  with 
each  other  and  with  themselves. 

The  Geology  Explorer  prototype  can  be  visited  at 

http://www.cs.ndsu.nodak.edu/~slator/html/PLANET/ 


911 


Virtual  Cell 

The  Virtual  Cell  (VCell)  is  an  interactive,  3-dimensional  visualization  of  a bio-environment.  VCell 
has  been  prototyped  using  the  Virtual  Reality  Modeling  Language  (VRML),  and  is  to  be  available  via  the 
Internet.  To  the  student,  the  Virtual  Cell  looks  like  an  enormous  navigable  space  populated  with  3D 
organelles.  In  this  environment,  experimental  goals  in  the  form  of  question-based  assignments  promote 
deductive  reasoning  and  problem-solving  in  an  authentic  visualized  context. 

The  initial  point  of  entry  for  the  Virtual  Cell  is  a VRML-based  laboratory.  Here  the  learner 
encounters  a scientific  mentor  and  receives  a specific  assignment.  In  this  laboratory,  the  student  performs 
simple  experiments  and  learns  the  basic  physical  and  chemical  features  of  the  cell  and  its  components.  More 
notably,  our  laboratory  procedures  are  crafted  such  that  they  necessitate  a voyage  into  the  Virtual  Cell  where 
experimental  Science  meets  virtual  reality.  As  the  project  progresses,  students  will  revisit  the  laboratory  to 
receive  more  assignments.  Periodically,  the  student  will  bring  cellular  samples  back  to  the  virtual  lab  for 
experimentation. 

The  implementation  of  the  Virtual  Cell  depends  on  coordinating  three  technologies:  1)  VRML 
visualization,  2)  Java  client  and  simulation  software,  and  3)  a text-based  MOO  server.  Students  use  a standard 
WWW  browser  to  launch  a Java  applet.  The  applet  provides  a connection  to  an  object-oriented,  multi-user 
domain  where  cellular  processes  are  simulated  and  multi-user  viewpoints  are  synchronized.  The  Java  applet 
also  controls  the  agent-based  implementation  of  organic  constituents,  and  launches  an  interface  to  the  VRML 
representation  of  the  Virtual  Cell,  allowing  the  student  to  explore  and  experiment  within  the  3D 
representation. 

The  Virtual  Cell  prototype  can  be  visited  at 

http://www.ndsu.nodak.edu/instruct/mcclean/vc/ 

Visual  Computer  Program  and  ProgrammingLand  Museum 

The  goal  of  the  Visual  Program  project  is  to  provide  an  environment  in  which  students  can  study  and 
learn  programming  techniques.  We  provide  tools  to  support  active  learning  using  visualizations  of  AI 
programs.  These  visualizations  include  animation,  fly-through  models  and  more  interactive  information 
models.  The  ProgrammingLand  Museum  implements  an  Exploratorium-style  museum  metaphor  to  create  a 
hyper-course  in  computer  programming  principles  aimed  at  structuring  the  curriculum  as  a tour  through  a 
virtual  museum.  Student  visitors  are  invited  to  participate  in  a self-paced  exploration  of  the  exhibit  space 
where  they  are  introduced  to  the  concepts  of  computer  programming,  are  given  demonstrations  of  these 
concepts  in  action,  and  are  encouraged  to  manipulate  the  interactive  exhibits  as  a way  of  experiencing  the 
principles  being  taught. 

ProgrammingLand  is  being  developed  on  the  Valley  City  State  University  (VCSU)  campus  as  a 
Virtual  Lecture  adjunct  to  introductory  programming  language  classes.  Students  peruse  the  exhibits  of  the 
museum,  reading  explanatory  text  that  is  displayed  when  they  enter  a room.  A topic  may  be  covered  in  one  or 
more  connected  rooms.  In  addition  to  the  displayed  text  there  are  a number  of  interactive  demonstration 
objects  in  the  museum  that  clarify  or  demonstrate  the  concepts.  One  such  object  is  a code  machine.  It  contains 
a short  portion  of  programming  language  code  and  can  perform  any  of  the  following  functions:  display  the 
code;  display  the  code  with  a line  by  line  explanation  of  the  purpose  or  syntax  of  each  line;  or  display  an 
execution  of  the  code  on  a line  by  line  basis.  The  goal  of  ProgrammingLand  is  to  facilitate  programming 
language  courses,  either  locally  or  at  a distance.  At  the  beginning  of  this  course  the  MOO  had  four  wings, 
each  incomplete.  One  of  these  was  an  introduction  to  using  a MOO,  each  of  the  other  three  dealt  with  the  one 
of  the  following  programming  languages:  C++,  Java  and  BASIC. 

The  ProgrammingLand  Prototype  can  be  visited  at 

http://www.cs.ndsu.nodak.edu/~slator/html/PLANET/wwwic-pland.html 


Blackwood 

WWWIC  is  in  the  process  of  designing  a virtual  environment  to  simulate  a 19th  Century  Western 
town.  We  will  populate  this  town  with  intelligent  software  agents  to  simulate  an  economic  environment 
representative  of  the  times.  This  spatially  oriented  virtual  environment,  will  borrow  freely  from  historical 
records  and  digital  images  from  archives  at  the  NDSU  Institute  for  Regional  Studies. 

The  educational  "game"  will  be  one  where  players  Join  the  simulation  and  accept  a retailer's  role  in 
the  virtual  environment.  Rather  than  everyone  vying  for  a portion  of  the  same  economic  market,  roles  will  be 
variable  and  specific.  For  example,  in  this  simulation  players  will  be  purveyors  of  dry  goods,  food  stuffs, 
blacksmithing  services,  mortuary  services,  saloons  and  gambling  establishments,  banks,  barber  shops, 
apothocaries,  messenger  services,  news  stands,  gunsmiths,  implement  dealers  and  so  forth.  Therefore,  players 
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will  only  directly  compete  against  other  players  with  similar  roles,  or  with  software  agents  in  the  same 
profession,  but  not  be  in  instant  competition  with  every  other  player  (Slator  and  Chaput,  1 986). 

In  addition,  the  environment  will  support  period-authentic  atmosphere  in  the  form  of  entertainments. 
For  example,  the  circus  might  come  to  town,  the  weekly  train  will  arrive  from  the  east,  a cattle  drive  will 
appear  on  the  scene,  preachers  and  circuit  judges  and  medicine  shows  will  pass  through,  and  the  occasional 
crime  will  be  reported. 

Virtual  Polynesia 

WWWIC  is  in  the  process  of  designing  an  immersive,  synthetic  environment  where  students,  in  the 
role  of  anthropologist  or  trader,  step  ashore  on  an  island  in  western  Polynesia,  in  the  south  Pacific,  near  the 
turn  of  the  19th  century.  That  island,  and  the  culture  encountered,  is  modeled  after  the  Samoan  islands  at  a 
time  a time  when  Samoan  culture  was  still  unaltered  by  Western  goods  and  ideas.  The  environment  will  focus 
on  a small  valley  and  surrounding  territory  which  represent  a microcosm  of  Samoan  society.  The 
anthropologist  is  able  to  observe  and  explore  the  traditional  society  as  it  had  developed  to  that  time.  He/she  is 
also  able  to  witness  the  contact  of  cultures  as  the  trader  enters  the  picture.  While  the  environment  will  be 
fictitious,  it  will  be  based  on  careful  attention  to  actual  Samoan  materials  and  cultural  traditions. 

Virtual  Polynesia  is  another  experiment  in  creating  virtual  worlds  where  more  than  one  role  is 
available  to  students.  In  this  scenario,  he  most  obvious  roles  are  trader  and  anthropologist.  The  trader  will 
visit  the  culture  looking  to  exchange  western  goods  for  items  in  Samoan  culture  that  would  have  value  in  the 
west.  The  anthropologist  will  look  to  the  discovery  of  cultural  artifacts  that  in  some  way  illuminate  our 
understanding  of  Samoan  society. 

The  WWWIC  Research  Strategy 

The  WWWIC  projects  are  designed  to  capitalize  on  the  affordances  provided  by  virtual 
environments.  For  example,  to 

• control  virtual  time  and  collapse  virtual  distance, 

• create  shared  spaces  that  are  physical  or  practical  impossibilities, 

• support  shared  experiences  for  participants  in  different  physical  locations, 

• implement  shared  agents  and  artifacts  according  to  specific  pedagogical  goals, 

• support  multi-user  collaborations  and  competitive  play. 

Specifically,  the  WWWIC  projects  each  design  with  the  following  over-arching  principles. 

Role-based:  Simulated  environments  enable  learners  to  assume  roles  in  particular  contexts  and  have 
meaningful,  authentic  experiences.  In  the  popular  culture,  this  approach  is  captured  in  the  John  Houseman 
adage,  "learning  not  the  law,  but  learning  to  think  like  a lawyer".  More  formally,  WWWIC  promotes  a 
learning  strategy  based  on  ancient  apprenticeship  where,  in  modem  terms,  the  student  progresses  by 
"modeling  the  expertise"  of  the  master.  Role-based  learning  is  leaming-by-doing,  but  not  the  mere  goal 
oriented  "doing"  of  a task.  Rather,  it  is  leaming-by-doing  within  the  structure  of  playing  a role  in  context. 
Instead  of  simply  teaching  goal-based  behavior  and  tactical  task-oriented  skills  and  methods,  the  role-based 
approach  communicates  a general,  strategic,  manner  of  practice  (McLuhan,  1964). 

Goal-oriented:  Goals  are  important,  but  within  the  context  of  roles.  It  is  through  goals  that  obstacles  leading 
to  problem  solving  are  encountered.  It  is  within  the  local  goal  framework  that  techniques  and  methods  are 
learned  and  rehearsed.  Practice  and  repetition  in  problem-solving  is  how  apprentices  learn  the  master's  craft. 
Goals  provide  problems  to  solve. 

Learn  by  Doing:  When  experiences  are  structured  and  arranged  such  that  playing  a role  in  the  environment 
illustrates  the  important  concepts  and  procedures  of  the  simulated  domain,  students  are  able  to  "learn  by 
doing"  (Dewey,  1900).  Experiences  are  the  best  teachers. 

Spatially  oriented:  WWWIC  simulations  are  spatially-oriented  to  leverage  off  the  natural  human  propensity 
to  towards  physically  plausible  context.  In  this  way,  simulations  promote  the  "willing  suspension  of  disbelief 
which  in  turn  reinforces  the  role-based  elements  of  the  environments. 

Immersive:  The  combination  of  role-based  scenarios  and  spatially  oriented  simulations  is  conducive  to  an 
immersive  atmosphere.  The  concept  of  immersion  has  long  been  shown  valuable  in  foreign  language  learning 
(where,  it  is  anecdotally  understood,  the  key  moment  arrives  when  the  learner  succeeds  in  reaching  the  point 
where  they  are  "thinking  in  X",  where  X is  French,  German,  Farsi,  or  whatever).  Immersion,  then,  is 
elemental  to  the  concept  of  role-  based  learning  where  it  is  the  strategic  thinking  of  the  master  the  apprentice 
eventually  learns  to  model 
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Exploratory*  Exploratory  simulation  means  enabling  students  to  pursue  their  own  interests.  This  approach, 
usually  referred  to  as  User-centered  design,  promotes  a pedagogical  environment  where  learners  are  self- 
directed  and  given  the  freedom  to  structure,  construct,  and  internalize  their  own  experience  (Duffy,  Lowyck, 
and  Jonassen,  1983;  Duffy  and  Jonassen,  1992). 

Game-like:  The  value  of  play  in  learning  can  hardly  be  over-stressed.  Students  quickly  tire  of  rigid  tutorial 
systems  designed  to  teach  at  any  cost  and  at  some  predetermined  pace  (Schank,  1991).  However,  since 
simulations  can  be  adaptive  and  responsive,  playing  a role  in  a simulation  can  be  fun.  Players  will  throw 
themselves  terrier-like  into- an  environment  if  it  feels  like  a game.  Insofar  as  possible,  educational  software 
should  be  engaging,  entertaining,  attractive,  interactive,  and  flexible:  in  short,  game-like  (Slator  and  Chaput, 
1986). 

Highly  Interactive:  One  major  challenge  for  science  educators  is  to  develop  educational  tools  and  methods 
that  deliver  the  principles  but  also  teach  important  content  material  in  a meaningful  way.  At  the  same  time, 
the  need  for  computer-  based  education  and  distance  learning  systems  has  become  increasingly  obvious,  while 
the  value  of  "active”  versus  "passive"  learning  has  become  increasingly  clear  (Reid,  1994). 

Multi-user/player:  One  challenge  is  to  craft  role-based,  goal-oriented  environments  that  promote 
collaboration  as  well  as  the  more  easily  conceived  competition.  The  answer  lies  in  designing  systems  where 
student/players  have  multiple  roles  to  choose  from,  and  to  carefully  construct  the  simulation  so  that  these 
roles  are  inherently  complementary.  WWWIC  educational  systems  are  uniformly  multi-user,  and  hosted  on 
the  MOO  architecture. 

Software  Agents:  Software  agents  are  implemented  to  exhibit  authentic  behavior(s)  of  the  following  types: 

• atmosphere  agents:  an  agent  that  simply  lends  to  the  local  color.  For  example,  in  an  urban 

simulation  there  might  be  a street  magician,  a street  vendor,  a beat  cop,  a street  sweeper, 
and  so  forth;  in  a museum  simulation  there  might  be  visitors  wandering  the  exhibits  or 
vendors  selling  popcorn;  on  a planet  perhaps  animals  roaming  the  desert. 

• infrastructure  agents:  an  agent  who  contributes  in  some  way  to  the  gameplay:  in  an  urban 

simulation  perhaps  a banker,  an  employee,  an  advertising  consultant,  and  so  forth;  in  a 
museum,  one  might  expect  a guide;  on  a planet,  another  kind  of  guide. 

• tutoring  agents:  an  agent  that  monitors  player  moves,  and  visits  players  to  give  them  advice  in  the 

form  of  expert  stories  and  cases,  or  in  some  other  way  assists  players  in  learning  to  play. 
These  will  represent  expertise  or  past  experiences  of  other  players. 

Unintrusive  Tutoring:  A key  feature  of  educational  media  is  the  ability  to  tutor  students.  In  WWWIC 
environments,  tutoring  is  done  through  unintrusive  but  proactive  software  agents.  Agents  monitor  student 
actions  and  "visit"  a student  when  the  need  arises.  Tutors  give  advice,  but  they  do  not  mandate  or  insist  on 
student  actions,  nor  do  they  block  or  prevent  student  actions. 

Intelligent  Software  Tutoring  Agents:  we  implement  three  different  approaches  to  intelligent  tutoring,  based 
on  the  knowledge  available  to  the  tutoring  agent. 

1.  Deductive  Tutors  provide  assistance  to  players  in  the  course  of  their  deductive  reasoning  within  the 

scientific  problem  solving  required  for  the  accomplishment  of  their  goals. 
Example:  intelligent  tutoring  agents  in  the  NDSU  Geology  Explorer,  which  work  from  knowledge  of 
the  rocks  and  minerals,  and  knowledge  of  the  "experiments"  needed  to  confirm  or  deny  the  identity 
of  a rock  or  mineral.  Three  opportunities  for  deductive  tutoring  present  themselves: 

1 . an  equipment  tutor  detects  when  a student  has  failed  to  acquire  equipment  necessary  to 

achieving  their  goals 

2.  an  exploration  tutor  detects  when  a student  has  overlooked  a goal  in  their  travels 

3.  a science  tutor  detects  when  a student  makes  a wrong  identification  and  why  (i.e.  what 

evidence  they  are  lacking);  or  when  a student  makes  a correct  identification,  but 
with  insufficient  evidence  (i.e.  a lucky  guess) 

2.  Case-based  Tutors  provide  assistance  to  players  by  presenting  them  with  examples  of  relevant  experience. 

This  is  accomplished  by 

• creating  a library  of  prototypical  cases  of  success  and  failure, 

• treating  the  student's  experience  as  though  it  were  a case 

• matching  the  student's  case  with  the  library  and  retrieving  the  most  similar,  relevant  case 

for  remediation 

3.  Rule  Based  Tutors  provide  assistance  by 

• encoding  a set  of  rules  about  the  domain 
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• monitoring  student  action  looking  for  one  of  these  rules  to  be  "broken" 

• "visiting"  the  student  to  present  an  expert  dialog,  or  a tutorial. 

Shared  Courseware  Tools:  Creating  virtual/visual  worlds  is  an  intensive  process  in  terms  of  pedagogical 
design,  knowledge  engineering,  and  software  development.  Having  gained  experience  in  the  hand-crafting  of 
these  systems,  WWWIC  is  now  in  the  process  of  designing  and  developing  an  integrated  library  of  software 
tools  to  substantially  streamline  the  development  of  future  worlds.  These  tools  primarily  support  simulation 
and  agent  building,  and  are  of  the  following  types: 

• Virtual  Abstraction  Tool:  Jia  (1998)  implements  a first  version  of  a graphical  tool  for  building  abstraction 

hierarchies  in  LambdaMOO.  This  tool  enables  the  creation,  deletion,  renaming,  and  recategorizing  of 
objects.  Tools  of  this  sort  enable  content  experts  to  visualize  the  structure  of  the  knowledgebase  and 
assist  with  creating  the  taxonomic  structures  for  representing  conceptual  knowledge. 

• Virtual  Entity  Tool:  We  are  implementing  a tool  to  employ  an  entity  template  system  with  a form-filling 

interface  to  enable  creation  of  multiple  instances  of  a category.  For  example,  we  will  define  a 
template  for  minerals  that  specifies  the  properties  indigenous  to  minerals,  and  ranges  of  values 
associated  with  each  property.  Then,  a content  specialist  will  create  new  minerals,  quartz, 
tourmaline,  talc,  etc.,  with  a graphical  form-filling  interface  where  values  such  as  color,  texture,  and 
hardness  can  be  quickly  and  easily  selected  from  menus.  This  tool  is  general  in  that  any  category  of 
entity  (animal,  mineral,  or  vegetable)  can  be  constructed  with  it. 

• Spatial  Environment  Tool:  We  are  implementing  a spatial  environment  tool  (i.e.  a virtual  map  building 

tool)  to  allow  environment  designers  to  graphically  create  and  manipulate  spaces  in  a virtual  world. 
By  using  a map-like  interface,  content  specialists  will  decide  on  the  specification  of  locations,  such 
as  geological  formations  and  placement  of  these  in  relation  to  each  other. 

• Integrated  Virtual  World  Building  Tool:  We  are  implementing  a master  tool  that  coordinates  and 

manages  the  process  of  building  virtual  worlds.  This  tool  supports  the  implementation  of  virtual 
worlds  from  the  ground  up,  by  giving  access  to  the  construction  tools,  and  a "surface"  view  of  the 
world  as  it  develops.  For  example,  content  specialists  building,  say,  virtual  space  for  paleontology, 
will  use  the  Virtual  Abstraction  Tool  to  create  the  hierarchy  of  concepts  related  to  fossils.  Then  the 
Virtual  Entity  Tool  will  be  used  to  create  an  inventory  of  fossils  in  different  categories.  Meanwhile, 
the  Spatial  Environment  Tool  will  be  used  to  create  canyons  and  mountains  where  the  fossils  will 
reside.  The  integrated  tools  set,  which  we  are  calling  GUMI  Suite  (Graphical  User-friendly  MOO 
Interface)  will  support  the  developer’s  exploration  of  the  virtual  world  as  they  develop  it.  At  the  same 
time,  the  integrated  tool  will  support  the  Deductive  Tutoring  Agent  Tool  (described  next),  since 
they  operate  on  the  same  objects. 

• Deductive  Tutoring  Agent  Tool:  We  are  implementing  a tool  to 

1 . provide  a menu  of  virtual  testing  equipment  and  the  range  of  values  each  produces  — from  this  a 

subject  matter  expert  can  choose  the  appropriate  instrument-value  pairs; 

2.  provide  a menu  of  substances  in  the  same  category,  to  serve  as  a template;  and 

3.  check  other  substances  to  insure  a unique  set  of  plausibly  sufficient  criteria  for  each. 

These  three  functions  will  insure  that  tutoring  is  supported  on  all  identification  tasks  and  will  have 
the  further  benefit  of  checking  for  consistency  of  artifacts  in  the  synthetic  world.  This  tool  will  be  integrated 
into  GUMl-Suite,  described  above. 

The  ultimate  aim  in  developing  software  tools  is  to  support  the  construction  of  synthetic 
environments  and  move  development  into  the  hands  of  content  specialists,  teachers,  and  curriculum 
developers,  rather  than  computer  programmers. 

Subjective  Assessment:  Developing  methods  for  the  assessment  of  student  learning  are  a central  element  of 
this  research  (Slator  et  al.,  1998).  Briefly,  the  assessment  goal  is  to  determine  the  benefit  to  students  derived 
from  their  "learn  by  doing"  experiences. 

Our  assessment  strategy  rejects  the  notion  of  standardized  multiple  choice  tests  as  an  adequate 
instrument  in  this  pedagogical  context.  While  there  are,  indeed,  facts  and  concepts  acquired  in  the  course  of 
exploration,  which  are  neatly  packageable  and  testable  with  objective  instruments,  the  effect  on  student 
learning  in  that  arena  will  not  be  significant,  nor  would  we  expect  it  to  be. 

Therefore,  our  assessment  protocol  is  a subjective  one  that  seeks  to  measure  how  student  thinking 
has  improved.  To  do  this,  players  are  given  a pre-game  narrative-based  survey  where  they  are  told  short 
problem  solving  stories  and  asked  to  record  their  impressions  and  any  questions  that  occur  to  them.  These 
surveys  are  analyzed  for  the  presence  of  what  could  be  considered  "important"  domain  or  problem-solving 
concepts  or  procedures. 
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Abstract 

A mixed  media  approach  to  distance  education  was  developed  which  combined  1)  traditional 
classroom  lectures  with  Interactive  Video  Network  (IVN)  multicasting,  2)  WWW-based  assignments, 
notes,  and  course  materials,  3)  WWW-based  quizzes,  4)  assignments  turned  in,  and  results  returned, 
using  E-mail,  and,  most  notably,  5)  a virtual  environment  modeled  on  the  Museum  metaphor,  where 
students  explored  singly  and  in  groups,  developed  virtual  machines  that  became  part  of  the  Museum’s 
’’collection”,  and  interacted  with  instructors  who  kept  ’’virtual”  office  hours.  The  course  was  attended 
by  students  from  two  campuses  70  miles  apart,  and  delivered  by  two  instructors,  one  from  a third 
campus,  60  miles  in  the  other  direction. 


Introduction 

Methods  for  managing  distance  education  continue  to  evolve.  Interactive  Video  Networks  (IVN)  are 
one  method  for  delivering  lecture  and  other  programming  to  remote  locations.  The  Worldwide  Web  provides 
the  ability  to  compose  attractive  text  and  image  presentations,  and  has  become  an  increasingly  vibrant  medium 
with  animation  and  even  video.  It  is  more  and  more  common  to  find  course  materials  online  in  the  form  of 
syllabi,  lecture  notes,  and  reference  materials.  Recently,  more  active  elements  have  appeared  in  the  form  of 
online  quizzes  and  simple  demonstrations.  These  two  approaches  supplement  the  time-honored  methods  of 
distance  education:  correspondence  courses  and  broadcast  media.  Most  recently,  research  into  the  development 
of  virtual  environments  for  education  have  started  to  have  a limited  but  highly  promising  impact. 

Approach 

There  are  many  challenges,  both  in  terms  of  developing  new  approaches  to  content  and  curriculum, 
and  in  delivering  that  content  to  remote  learners.  This  paper  describes  a curricular  experience  that  combined 
virtual  lecture  with  virtual  laboratory  to  produce  a virtual  course.  In  particular,  we  combined  a Virtual  Lecture, 
using  the  Interactive  Video  Network  (IVN),  with  a Virtual  Laboratory  and  Museum  of  Computer  Science,  the 
ProgrammingLand  MOO  (Hill  and  Slator,  1998),  to  deliver  both  lecture-based  and  hands  on  instruction  to 
students  in  remote  locations.  In  doing  so  we  pursued  a particular  theoretical  approach  to  this  new  pedagogy  - 
an  approach  that  stresses  the  importance  of  virtual  environments,  authentic  experiences,  and  active  learning. 
We  developed  a relatively  standard  IVN  course,  but  then  augmented  it  with  networked,  multi-player, 
simulation-based,  interactive  multi-media  - an  educational  environment  that  is  both  immersive  and  highly 
interactive  (Reid,  1 994). 

The  ProgrammingLand  Museum  implements  an  Exploratorium-style  museum  metaphor  to  create  a 
hyper-course  in  computer  programming  principles  aimed  at  structuring  the  curriculum  as  a tour  through  a 
virtual  museum.  Student  visitors  are  invited  to  participate  in  a self-paced  exploration  of  the  exhibit  space 
where  they  are  introduced  to  the  concepts  of  computer  programming,  are  given  demonstrations  of  these 
concepts  in  action,  and  are  encouraged  to  manipulate  the  interactive  exhibits  as  a way  of  experiencing  the 
principles  being  taught  (Duffy,  Lowyck,  Jonassen,  1983;  Duffy  and  Jonassen,  1992). 

Locality  and  Temporality 

Virtual  classrooms  and  virtual  laboratories  will  help  solve  many  of  the  problems  facing  the  modem 
university:  distance  learning  will  become  a reality,  learner  diversity  will  be  accommodated  (both  in  terms  of 
learning  styles  and  life  styles),  and  in  many  cases  the  curriculum  will  become  more  active,  more  role-based, 
more  self-paced,  and  more  "learn  by  doing"  than  "learn  by  listening"  (Schank,  1994). 
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It  is  not  hard  to  imagine  a day  when  the  curriculum  is  taught  in  both  real  and  virtual  laboratories  - or 
to  foresee  a time  when  students  will  take  virtual  field  trips  in  order  to  prepare  for  the  real  thing.  One  can  also 
imagine  a curriculum  that  combines  physical  classrooms  with  virtual  ones:  where  some  lecture  material  is 
delivered  same  time  and  same  place,  and  other  material,  employing  "time  shifting"  and  "place  shifting,"  is  not 
(see  Figure  1). 


Time/Place  Shifting 

Same  Time 

Different  Time 

Same  Place 

Classroom  Lectures 
Pros:  personal,  sometimes 
interactive. 

Cons:  inflexible  schedule,  social 
pressures,  mostly  passive 
learner  experience,  strictly 
local  audience 

Virtual  Environments 
Pros:  self-paced,  immersive, 

potential  for  active  learner 
experience,  programmable 
Cons:  high  development  costs 

Different  Place 

Live  Broadcast  (e.g.  IVN) 

Pros:  personal,  sometimes 

interactive,  recordable  for 
later  review,  potential  for 
wide  (broadcast)  audience 
Cons:  similar  to  lecture:  inflexible 
schedule,  social  pressures, 
mostly  passive  learner 
experience 

Recorded  or  Broadcast 
Programming 

Pros:  self-paced  (rewindable) 
Cons:  non-interactive,  passive 
learner  experience 

Figure  1:  Time  and  Place  Shifting 


Recursion 

From  the  first  day,  and  throughout  the  course  (COMP  372:  Comparative  Programming  Languages,  a 
required  upper  division  course),  we  attempted  to  stress  recursion  as  a theme.  This  served  two  purposes.  First, 
recursion  is  one  of  the  most  difficult  concepts  in  computer  science.  It  is  an  elegant  and  abstruse  mathematical 
construct  that  often  baffles  undergraduate  students.  Unlike  most  other  computing  concepts,  recursion  is 
difficult  to  teach  because  of  the  relatively  few  common  sense  day-to-day  activities  that  depend  on  it,  making  it 
difficult  to  draw  useful  analogies.  And  unlike  previous  generations  of  computer  science  curricula,  it  is 
common  in  these  enlightened  times  to  introduce  recursion  as  early  and  as  often  as  possible,  in  an  attempt  to 
reinforce  the  concept  until  it  becomes  familiar  and  workable. 

One  of  the  most  productive  moments  for  covering  recursion  in  the  curriculum  is  during  a course  on 
programming  languages.  In  this  context  program  execution  is  viewed  from  the  viewpoint  of  the  runtime 
environment  where  all  programs  are  managed  in  terms  of  memory  allocations  and  activation  records  on  a 
runtime  stack.  From  this  point  of  view,  recursion  is  not  a matter  of  infinitely  embedded  abstractions,  but  a 
more  concrete  matter  of  pushing  and  popping  activation  records  from  a stack.  The  stack  is  a powerful 
metaphor  in  computer  science,  and  provides  an  excellent  image  for  understanding  recursion,  but 
understanding  stacks  requires  understanding  more  fundamental  computing  concepts,  and  so  stacks  are  not 
immediately  introduced  to  beginning  students.  Ironically,  with  recursion  so  early  in  the  curriculum,  it  is  now 
possible  for  students  to  be  introduced  to  recursion  before  they  learn  about  stacks. 

Second,  in  the  interest  of  approaching  recursion  from  an  unusual  angle,  and  in  order  to  motivate  the 
programming  assignments  in  the  course,  we  decided  to  assign  projects  that  called  for  building  virtual 
machines  that  illustrated  programming  language  concepts,  as  a method  of  teaching  these  concepts  to  others.  In 
other  words,  the  students  were  asked  to  learn  about  concepts  in  order  to  build  artifacts  that  would  be  used  to 
teach  those  concepts.  The  virtual  course,  as  we  styled  it  (because  of  the  IVN,  WWW,  and  MOO  mix),  was 
about  studying  programming  languages  in  order  to  recursively  implement  museum  exhibits  in  order  to  teach 
about  programming  languages. 

Course  Components 

NDSU  COMP372:  Comparative  Programming  Languages,  was  offered  in  the  Summer  of  1998  during 
the  4-week  session,  and  was  comprised  of  the  following  elements. 
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IVN,  The  North  Dakota  Interactive  Video  Network 

The  North  Dakota  Interactive  Video  Network  (ND  IVN)  is  a two-way  interactive  telecommunications 
system  located  at  many  sites  throughout  the  state.  Any  combination  of  two  to  fourteen  sites  may  be  connected 
together  for  a single  event  and  several  events  may  occur  at  the  same  time.  Over  25  specially  equipped 
telecommunications  classrooms  and  conference  rooms  link  the  1 1 North  Dakota  State  University  System 
campuses,  the  state  capitol,  5 tribal  colleges  in  ND,  and  25  high  schools  in  the  state.  In  addition,  ND  IVN  has 
the  ability  to  connect  to  sites  world-wide.  ND  IVN  participants  can  hear  all  sites  at  all  times  but  see  only  one 
other  site.  The  Network  automatically  switches  the  video  to  the  site  that  is  currently  speaking.  For  the 
automatic  switching  to  occur,  a sound  must  last  about  two  seconds. 

An  IVN  room  is  designed  to  as  closely  resemble  a traditional  classroom  as  practical.  Each  room  had 
approximately  25  seats.  There  are  two  television  monitors,  one  displays  the  current  image  and  the  other  the 
image  being  transmitted  from  this  location.  Each  student  has  a microphone  on  their  table.  When  a student 
speaks,  then  the  image  from  that  location  is  broadcast  to  the  other  locations.  Thus  a reasonable  conversation 
can  be  carried  out;  however,  the  originating  site  can  not  tell  if  their  image  is  being  transmitted  or  not.  An 
instructor  can  also  transmit  computer  images  or  the  display  of  a paper  that  then  functions  like  a blackboard.  In 
this  particular  course,  there  were  four  such  sites:  two  on  the  NDSU  campus,  one  on  the  UND  campus  and  one 
on  the  VCSU  campus. 

WWW  Syllabus  and  Assignments 

All  pertinent  documents,  such  as  the  syllabus  and  assignments,  were  posted  on  web  pages.  In  most 
classes  this  is  a courtesy  to  students.  In  this  course  it  was  a requirement  since  none  of  the  locations  were  within 
50  miles  of  each  other. 

WWW  Exams 

The  WWW  test  system  was  developed  at  NDSU  for  multiple  choice  tests  on  the  web.  It  is  similar  to 
many  other  such  test-giving  systems.  A student  may  log  in  by  registering  their  name  and  obtaining  an  ID. 
They  may  then  take  a test  during  a particular  window  of  time,  which  was  usually  from  Friday  after  class  to 
Monday  before  class.  On  completion  of  the  test  they  were  immediately  given  their  resulting  score.  An 
instructor  could  gain  access  to  enter  a test,  modify  a test  or  obtain  class  scores.  Any  common  web  browser 
could  be  used. 

E-mail 

In  such  a situation  e-mail  becomes  a critical  communication  form,  since  distance  keeps  face  to  face 
conversations  at  a minimum.  E-mail  was  used  for  a variety  of  situations  in  this  course.  Assignments  that  were 
not  MOO  based  were  handed  in  through  e-mail,  with  the  program  and  other  documentation  as  an  attachment. 
The  time  stamp  of  the  e-mail  determined  whether  the  item  is  on-time  or  late.  MOO  assignments  were  handed 
in  by  e-mail  that  announced  its  completion  and  specified  the  object  numbers  of  the  finished  products.  Many  of 
the  office  visit  situations  were  also  handled  with  e-mail,  sometimes  more  easily  than  a real  visit;  questions 
could  be  answered  and  programs  could  be  examined.  For  example,  it  is  often  easier  to  attach  an  example 
program  to  an  e-mail  than  it  is  to  put  it  in  the  students  hands  in  a visit,  moreover  it  is  much  easier  for  them  to 
run  it  later.  The  time  delays  of  this  approach  leave  something  to  be  desired,  but  this  is  remedied  by  virtual 
office  hours  as  discussed  below. 

The  importance  of  E-mail  made  it  crucial  that  both  authors  processed  all  e-mail  several  times  a day. 
The  course  had  a very  short  time  duration,  just  four  weeks.  It  was  imperative  for  students  to  receive  quick 
response  to  e-mail;  and  three  exchanges  with  a single  student  in  a single  day  was  not  uncommon. 

The  ProgrammingLand  Musuem 

ProgrammingLand  is  a MOO  (Curtis,  1992)  being  developed  on  the  VCSU  campus  as  a Virtual 
Lecture  adjunct  to  introductory  programming  language  classes.  The  paradigm  employed  is  that  of  a museum 
where  students  examine  exhibits,  reading  the  explanatory  text  displayed  on  the  walls  of  each  room.  In  addition 
to  the  displayed  text  there  are  a number  of  interactive  demonstration  objects  in  the  museum  that  clarify  or 
demonstrate  the  concepts.  One  such  object  is  a code  machine  which  contains  a short  segment  of  programming 
language  code  and  can  display  the  code;  display  with  line  by  line  explanations;  or  display  a line  by  line 
execution  of  the  code.  ProgrammingLand  augments  programming  language  courses,  either  locally  or  at  a 
distance.  At  the  beginning  of  this  course  there  were  four  wings  under  construction.  One  of  these  was  an 
introduction  to  using  a MOO,  each  of  the  other  three  dealt  with  the  one  of  C++,  Java  or  BASIC. 
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This  approach  to  a MOO  has  much  in  common  with  many  web  oriented  approaches,  with  some 
differences  as  well.  A room  in  a MOO  corresponds  to  a single  web  page.  The  MOO  is  exclusively  text-based 
while  a web  page  can  augment  the  text  with  various  multimedia  objects.  However,  every  room  of  a MOO  is 
also  a chat  room  and  users  of  the  MOO  are  aware  of  the  presence  of  others  in  the  room  or  in  the  MOO,  unlike 
web  pages.  The  MOO  keeps  more  and  better  server  side  records  of  the  students  and  their  actions.  The  MOO 
server  code  actually  has  a web  interface,  but  using  it  loses  many  of  the  interactive  advantages  of  the  MOO. 

Three  assignments  in  COMP372  were  completely  MOO  based.  The  first  was  a trivial  exercise  in 
MOO  navigation.  They  were  to  find  an  object,  which  was  called  the  totem  pole.  When  they  found  it,  that  fact 
was  marked  on  their  MOO  user  ID.  The  purpose  of  this  was  so  that  the  students  could  explore  the  MOO  and 
see  how  it  was  arranged  and  how  exhibits  conveyed  the  desired  content.  In  the  second  assignment  each  student 
was  given  a LISP  function.  Students  had  to  build  an  instructional  suite  of  rooms  about  their  particular 
function.  They  were  to  reinforce  their  learning  by  teaching  others  in  the  context  of  the  MOO.  The  third  MOO 
assignment  was  to  create  an  interactive  machine  that  demonstrated  their  particular  LISP  function.  This  gave 
them  some  unique  insights  into  their  function,  but  also  the  object  oriented  script  of  the  MOO.  The  machine  in 
question  took  one  of  several  function  calls  and  results  and  verified  that  the  function  did  indeed  produce  the 
stated  output. 

Since  every  room  of  a MOO  is  a chat  room,  the  students  interact  with  each  other  and  the  instructors 
in  the  MOO.  This  suggested  the  concept  of  virtual  office  hours.  Each  instructor  would  guarantee  that  they 
would  be  in  the  MOO  at  a particular  time.  Any  student  with  a question  or  problem  could  then  go  and  have  a 
real  time  conversation  with  the  instructor.  If  that  conversation  involved  a MOO  project,  then  it  was  a simple 
matter  to  visit  the  room  and  give  what  ever  help  was  needed.  This  approach  is  not  quite  as  personal  as  a face 
to  face  meeting  and  it  does  tend  to  test  the  typing  skills  of  the  student,  but  is  a solution  to  the  distance  problem. 

Textbook 

The  textbook  was  Concepts  of  Programming  Languages  (3rd  Edition)  by  Robert  W,  Sebesta.  Published  by 
Addison  Wesley,  1996,  ISBN  0-8053-7133-8. 

Course  Details 

This  course  was  a typical  Junior  level  Programming  Languages  courses  for  Computer  Science  majors 
and  minors.  The  first  author  had  taught  the  course  in  previous  terms,  using  a MOO.  However,  a number  of 
wrinkles  were  requested  by  the  administration.  There  was  sorne  demand  for  the  course  on  another  campus  of 
the  NDUS  system,  so  it  was  requested  that  the  course  be  taught  over  the  Interactive  Video  Network  (IVN).  The 
course  was  to  be  taught  in  a four-week  session,  but  the  number  of  contact  hours  available  on  IVN  was  three 
hours  short  of  what  was  needed  for  a three  hour  course.  These  constraints  provoked  us  to  use  a strategy  not 
previously  used  by  either  of  us.  The  content  materials  would  come  from  lectures  delivered  by  IVN  and  the 
student's  offline  reading  of  the  textbook.  Tests  would  be  delivered  through  a World  Wide  Web  test  system 
developed  at  NDSU.  All  assignments  and  related  material,  such  as  the  syllabus,  would  be  available  on  a 
WWW  page.  Conventional  programming  assignments  were  done  on  a separate  minicomputer  using  a LISP 
interpreter  and  handed  in  via  email.  The  recursive  leam-by-teaching  approach  was  used,  by  having  students 
generate  lessons  inside  the  MOO. 

The  course  was  completed  over  the  course  of  four  weeks  (actually,  18  class  meetings),  and  was 
composed  of  the  elements  Just  listed.  Lectures  were  multicast  daily  by  the  first  author  from  an  instrumented 
IVN  classroom  on  the  NDSU  campus.  There  were  several  students  in  that  IVN  room,  several  more  in  an  IVN 
classroom  across  campus,  and  a handful  in  another  IVN  classroom  on  a campus  70  miles  to  the  north.  In 
addition,  the  second  author  participated  from  another  IVN  location  60  miles  to  the  west.  There  were  50 
students  altogether.  This  technology,  described  above,  is  relatively  solid  and  caused  few  problems  (there  were, 
however,  occasional  audio  anomalies,  where  sudden  bursts  of  static  or  other  noise  would  be  emitted,  and  there 
were  times  when  the  voice-activated  nature  of  the  two-way  audio  communication  was  awkward,  as  sometimes 
disembodied  voices  would  emanate  over  the  transmission  but  the  accompanying  video  would  never  arrive). 

The  course  syllabus  was  posted  at  a website  online,  and  reading  assignments  (both  from  the  text  and 
from  online  sources),  and  homework  assignments  were  posted  on  that  site  too.  In  addition,  the  details  of  the 
homework  assignments,  as  well  as  information  on  how  to  negotiate  the  pitfalls  of  electronically  submitting 
homework. 

Although  hosting  the  course  on  the  Interactive  Video  Network,  and  posting  the  syllabus  on  the 
Worldwide  Web  were  steps  in  the  virtual  direction,  the  technologies  of  the  Internet  were  employed  in  even 
more  interesting  ways  than  that.  In  particular,  exams  were  held  outside  of  class  and  administered  over  the 
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Web.  Tools  for  creating  these  online  quizzes  were  provided  by  the  NDSU  Multimedia  Center,  but  they  were 
experimental  and  proved  problematic.  Uploading  quizzes  was  painfully  time  consuming  and  fault-intolerant. 
Some  students  complained  about  not  being  able  to  re-take  a quiz  if  they  lost  their  connection,  and  there  were  a 
few  anomalies  in  grade  reporting  that  had  to  be  corrected  by  hand.  However,  the  technological  difficulties  with 
online  quizzes  were  not  extreme. 

The  pedagogical  difficulties  with  online  quizzes  were  a little  more  profound,  because  of  the  offline, 
self-paced  nature  of  the  Web.  In  this  class,  tests  were  posted  on  Friday  and  students  were  given  until  Monday 
evening  to  complete  them.  This  unstructured,  unproctored  protocol  meant  students  could  take  quizzes  at  their 
leisure  with  their  textbooks  open  on  their  lap.  As  a consequence  of  all  these  factors,  quizzes  were  painful  to 
implement  and  were  relatively  weak  indicators  of  student  progress.  Hence,  quiz  grades  were  quite  high  on 
average. 

There  was  no  attempt  made  to  conceal  that  this  approach  was  as  new  to  the  authors  as  to  the  students. 
It  was  perceived  that  they  responded  well  and  entered  into  the  adventure.  This  did  require  flexibility  when 
various  technical  problems  occurred.  In  the  end,  the  students  rated  the  class  highly  (see  below). 

Local  Context 

The  NDSU  World  Wide  Web  Instructional  Committee  (WWWIC)  is  currently  engaged  in  several 
virtual/visual  development  projects:  three  NSF-supported,  The  Geology  Explorer  (Slator  et  al.,  1998;  Saini- 
Eidukat,  Schwert  and  Slator  1999;  Slator,  Schwert,  Saini-Eidukat,  1999;  Schwert,  Slator,  Saini-Eidukat, 
1999),  The  Virtual  Cell  (White,  McClean,  Slator,  1999),  and  The  Visual  Computer  Program,  as  well  as  others. 
These  have  shared  and  individual  goals.  Shared  goals  include  the  mission  to  teach  Science  structure  and 
process:  the  Scientific  Method,  scientific  problem  solving,  deduction,  hypothesis  formation  and  testing,  and 
experimental  design.  The  individual  goals  are  to  teach  the  content  of  individual  scientific  disciplines: 
Geoscience,  Cell  Biology,  and  Computer  Science. 

These  projects  are  designed  to  capitalize  on  the  affordances  provided  by  virtual  environments.  For 
example,  to 

• control  virtual  time  and  collapse  virtual  distance, 

• create  shared  spaces  that  are  physical  or  practical  impossibilities, 

• support  shared  experiences  for  participants  in  different  physical  locations, 

• implement  shared  agents  and  artifacts  according  to  specific  pedagogical  goals, 

• support  multi-user  collaborations  and  competitive  play. 

Summary 

The  combination  of  IVN,  WWW,  and  MOO  proved  to  be  quite  effective  and  stronger  than  any  of 
these  alone.  The  use  of  the  MOO  greatly  enhanced  the  effectiveness  of  the  course  by  reducing  the  perceived 
separation  of  student  and  instructor.  Student  evaluations  of  the  course  were  quite  high,  with  92%  of  the 
students  responding  rating  the  quality  of  the  course  as  either  above  or  much  above  average.  Similarly,  88%  of 
the  students  believed  their  understanding  of  the  course  content  was  either  good  or  very  good. 

Future  Plans 

We  intend  to  conduct  a more  thorough  study  of  this  approach  when  the  course  is  offered  again  in  the 
first  'summer  session  of  1 999.  This  study  will  include  a survey  given  at  the  beginning  of  the  class  and  follow 
up  survey  at  the  end  that  attempts  to  capture  student  perceptions  of  this  form  of  distance  learning. 
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Abstract:  The  Virtual  University  is  an  Internet-based  learning  environment  that  includes  all  aspects 
of  a university,  developed  at  the  University  of  Hagen.  The  project  is  experimenting  with  and 
evaluating  different  forms  of  teaching  and  learning.  A new  form  of  teaching  is  the  virtual  seminar. 

Our  experiences  show  that  the  virtual  seminar  is  leading  towards  a major  quality  improvement,  in 
open  and  distance  education.  Learning  activities  are  changing,  the  alternation  of  receptive  and  active- 
expressive  activities  is  possible.  The  seminar  develops  into  a new  and  constant  cooperation  and 
communication  process. 

1.  Introduction 

As  life-long  learning  is  getting  more  and  more  essential  in  professional  life,  learning  via  Intra-and  Internet  will 
more  and  more  succeed,  mainly  with  the  increase  of  open  and  distant  learning  [MITT98a].  The  benefits  of 
distance  education,  time-  and  location  independence  and  the  advantages  of  the  Internet  (fast  information,  easy 
communication  and  co-operation  possibilities)  are  combined  in  the  project  virtual  university.  A prototype  with 
about  4.000  students  is  mnning  since  1 996  at  the  University  of  Hagen 

The  Virtual  University  [BM96]  of  the  University  Hagen  is  a virtual  university  system  that  integrates  all  functions 
of  a university  into  a complete,  homogeneous,  extensible  and  cost-effective  system  with  an  easy  to  use  and 
intuitive  user- interface.  The  main  menu  of  the  Virtual  University  [Fig.  1]  offers  the  functions: 


Fig.  1 : Home  page  of  the  Virtual  University,  http://vu.femuni-hagen.de 
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education  - for  participation  in  courses,  seminars,  practical  training  and  exercises;  news  - a campus  wide 
blackboard  containing  all  sorts  of  up-to-date  information  relevant  to  the  users  of  the  Virtual  University;  office  - 
the  component  including  administrative  functions;  research  - offering  access  to  all  research-related  activities  in 
the  university;  cafeteria  - a forum  for  social  contacts  between  students;  library  - offering  access  to  both 
traditional  and  digital  libraries;  information  - a menu  containing  general  information  about  the  university;  and 
shop  - offering  all  material  that  can  be  purchased  from  the  university. 

This  paper  aims  to  describe  the  different  phases  of  traditional  and  virtual  seminars  in  open  and  distance  teaching 
at  the  University  Hagen  in  correlation  with  the  use  of  different  communication  tools,  objectives,  and  learning 
activities.  The  major  breakthrough  is  attributed  to  the  multitude  of  communication  and  cooperation  facilities 
given  in  the  Virtual  University.  The  differences  and  the  results  of  these  differences  are  described  in  part  3 of  the 
paper,  the  fourth  part  gives  some  insights  our  experience  with  virtual  seminars.  In  the  summary  and  prospects 
(part  5)  recommendations  for  doing  successful  virtual  seminars  are  given. 

2.  Conventional  Seminars  and  Virtual  Seminars 

The  seminar  is  traditionally  an  important  teaching  method  in  higher  education,  also  at  the  University  of  Hagen, 
which  is  specialized  in  open  and  distance  education.  The  main  objective  of  the  seminar  as  teaching  form  is  to 
maintain  and  acquire  scientific  work  prospective  and  to  present  and  discuss  the  results  of  this  work.  So  far,  these 
objectives  of  a seminar  were  only  partly  fulfilled  by  the  conventional  seminar,  because  a continues 
communication-,  cooperation-  and  learning  process  is  hardly  possible. 

The  virtual  seminar  is  much  more  like  a seminar  in  presence  education.  Moreover  the  communication-, 
cooperation-  and  learning  process  is  much  more  structured  and  more  disciplined  than  in  presence  education. 

The  following  part  describes  the  different  phases  of  conventional  and  virtual  seminars  at  the  University  of  Hagen 
out  of  a student’s  perspective. 

2.1  Traditional  seminar  at  a distance  university 

The  conventional  way  to  take  part  in  a seminar  is  the  following: 

Students  get  their  topics  by  mail,  work  out  their  contribution,  travel  to  the  university  and  present  their 
contribution  in  the  presence  phase  (two  days).  There  are  only  three  seminar  phases  (introduction,  elaboration  and 
presence  phase),  with  nearly  no  cooperation  or  alternation  of  different  learning  activities.  This  way  causes 
different  problems: 

• students  have  rarely  contact  to  tutors,  students  are  working  alone  mostly 

• little  communication  between  the  students 

• discussing  questions  and  problems  is  time-consuming 

• students  can  not  participate  in  the  selection  of  topics 

• groupwork  is  nearly  impossible 

• nearly  no  cooperation  / arrangement  between  students  to  deal  with  overlaps  concerning  their  topics 

• there  is  less  alternation  in  the  students’  learning  activities 

• students  have  to  travel 

• the  presence  phase  is  completely  taken  up  with  presentations,  there  is  little  time  for  discussions 

2.2  Traditional  seminar  at  a campus  university: 

The  traditional  seminar  at  a campus  university  seems  to  be  a more  communicative  and  cooperative  way  to  reach 
a seminar’s  objectives.  But  in  it’s  structure  the  traditional  seminar  typically  is  hardly  better  than  the  distance 
seminar.  Normally,  students  do  not  know  each  other,  are  working  on  alone  their  contribution  with  little 
communication  with  other  students  or  the  tutor.  The  discussion  process  is  short  at  best  and  not  very  deep,  the 
students  usually  are  unaware  of  the  written  contribution  of  each  other. 

The  lively,  interesting,  well-structured  seminar  with  only  good  presentations  and  very  active,  cooperating  and 
communicating  participants  is  a rare  exceptional  case,  even  though  the  students  are  less  isolated  than  in 
traditional  distance  teaching  seminars. 

2.3  Virtual  seminar  at  a distance  university: 
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The  virtual  seminar  is  one  of  various  teaching  activities  at  the  Virtual  University,  The  Internet  services  used  for 
such  a seminar  are  email,  ftp,  news,  chat,  audio-  and  video  conferencing  [BERK97],  Services  were  individually 
combined  for  different  seminar  phases.  There  are  also  mixed  forms  between  virtual  and  conventional  seminars, 
e.  g.  only  certain  seminar  phases  are  assisted  by  Internet-based  tools.  In  the  following  only  pure  virtual  seminars 
are  considered. 

The  average  number  of  participants  is  about  1 5 students  preparing  lectures  (either  individually  or  in  co-operation 
with  other  participants),  supported  by  Internet  communication  services.  Furthermore  there  are  about  40-70 
students  taking  part  as  so  called  "passive"  participants.  These  passive  participants  are  having  the  possibility  to 
take  part  in  the  seminar’s  communication  process  but  are  not  producing  lectures. 

Since  the  beginning  of  the  Virtual  University  project  about  six  pure  virtual  seminars  have  taken  place  in  our 
computer  science  department.  The  way  to  execute  such  a virtual  seminar  is  as  follows: 


Seminar  phase 

Communication 

tools 

Objective 

Learning 

activities 

Co-operation 

Time 

Enrolment 

Mail,  Email 
(Mailinglist) 

Interesting  the  students 
for  a special  topic 

None 

None 

Two 
months 
before  the 
term  starts 

Orientation 

WWW,  News, 
Email, 

Decision  for  a specific 
theme  to  elaborate 
during  the  seminar  with 
student’s  participation 

Receptive 
and  active, 
expressive 

Discussion  about 
the  topics  with 
the  tutor  and 
other  students 

four  weeks 
before  the 
term  starts 

Introduction 

News,  WWW 

Presentation  of  the 
participants  and  the 
tutor,  first  introduction 
in  the  seminar’s  theme 

Active- 
expressive 
and  receptive 

With  the  tutor 
and  students 

At  the 
beginning 
of  the  term 

Preparation  of 
Groups 

News,  Email, 
telephone 

Building  of  seminar 
work  groups,  which  will 
work  out  one  topic 
together. 

Active- 

expressive 

With  the  tutor 
and  other 
students 

At  the 
beginning 
of  the  term 

Elaboration 

Recherche 

News,  Email, 

Knowledge  acquisition 
about  a special  topic  of 
a certain  theme 

Active- 

expressive 

With  the  tutor 
and  other 
students 

Structuring 

News,  WWW, 
Email 

Structuring  the 
information, 
presentation  in  the 
newsgroup  or  www 
Finding  out  overlaps  to 
other  topics 

Active- 

expressive 

Discussion  about 
the  structure  with 
the  tutor  and 
other  students 

Six  weeks 
after  the 
term  started 
(average 
number) 

Correction 

Email,  news 

Correction  and  revision 
of  the  reviewed 
structure 

Active- 

expressive 

With  the  tutor 
and  other 
students 

Two  weeks 
after 

Structuring 

Writing 

Mail 

Final  version  of  the 
written  contribution 

Active 

With  the  tutor 
and  other 
students 

Four  to  six 
weeks 
(average 
number) 

Finish 

o 

o 

-2 

PU 

WWW,  news. 
Chat,  Audio- 
and/or 

Videoconferenci 

ng 

Publishing  the 
contribution  in  the  web, 
discussion 

Active 

With  the  tutor 
and  other 
students 

Gradual, 
two  months 
before  the 
term  is 
ending  until 

Presence 

Personally,  or 

WWW 

Presentation  and 
discussion  of  lectures 

Active- 

expressive, 

receptive 

With  tutor  and 
other  students 

End  of  the 
seminar 

Table  1 : Seminar  phases  of  a virtual  seminar 
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To  summarize  the  advantages  of  virtual  seminars: 


• students  have  rather  good  contact  to  tutors  and  other  students 

• continuous  communication  between  the  students,  throughout  all  seminar  phases 

• discussing  questions  and  problems  is  time-  and  cost-effective 

• students  can  easily  participate  in  the  selection  of  topics 

• building  of  learning  and  working  groups  is  possible 

• it  is  possible  to  find  out  overlap  concerning  the  student’s  topics  and  to  provide  hyperlinks  to  each  others 
contribution 

• students  do  not  have  to  travel  at  all 

• in  the  case  the  presence  phase  is  taking  place,  the  time  could  be  used  for  discussions.  Furthermore,  the  time 
needed  to  getting  to  know  each  other  is  less  than  in 

• conventional  courses 

3.  Differences 

As  shown,  the  main  differences  are  in  the  use  of  communication  tools  and  in  the  learning  activities.  The  virtual 
seminar  is  much  more  structured  than  a conventional  seminar,  particularly  in  the  elaboration  phase,  which  is  one 
of  the  most  important  phases. 

Our  experience  show,  that  a well-structured  teaching  process  and  a continuing  guided  learning  process, 
supported  by  alternating  learning  activities,  are  the  best  guarantee  for  a major  quality  improvement.  The 
experiences  of  virtual  seminars  could  also  be  used  to  rise  quality  of  traditional  seminars  at  a presence  university. 

A short  comparative  overview: 

3.1  Seminar  Phases 

In  contrast  to  conventional  seminars,  the  virtual  seminar  is  enriched  by  the  phases  Introduction  and  Preparation 
of  Groups.  The  introduction  phase  has  the  objective 

to  introduce  the  participants  and  the  tutor  to  each  other,  mostly  in  the  seminar’s  newsgroup,  sometimes  in  the 
WWW  by  the  help  of  HTML-Pages. 

New  is  also  the  preparation  of  groups.  In  conventional  seminars  the  group  building  process  was  nearly 
impossible. 

Normally,  the  presence  phase  is  shortened  from  two  days  to  one,  which  reduces  the  costs  of  overnight  stays  for 
the  students,  but  it  is  also  possible  to  do  the  whole  presentation  and  discussion  phase  online  in  the  Web,  e.  g,  by 
the  use  of  text-chat,  audio-  and  videoconferencing. 

3.2  Communication 

The  conventional  seminar  uses  traditional  communication  tools,  like  mail,  telephone,  fax,  etc..  The 
communication  tools  in  a virtual  seminar  are  Internet  based.  These  communication  tools  enable  the  very  active 
communication,  co-operation  and  discussing  process.  For  the  first  time  synchronous  communication  with  several 
students  and  tutor  is  possible  in  a cost-effective  way. 

3.3  Objectives 

Some  new  objectives  are  given  in  the  virtual  seminar:  The  introduction  of  participants  and  tutor,  the  participation 
of  students  in  defining  seminar  topics.  Furthermore,  finding  out  overlaps  with  other  topics,  publishing  and 
discussing  the  contribution  in  the  web. 

For  the  first  time  the  classical  objectives  of  the  seminar  as  teaching  method,  to  learn  to  work  scientifically,  to 
present  and  to  discuss  the  results,  are  given  in  distance  education. 


3.4  Learning  activities 

It  is  evident  that  the  alternation  of  receptive,  active  and  expressive  learning  activities  is  much  better  given  in  the 
virtual  seminar  form.  Nearly  each  seminar  phase  contains  all  learning  activities,  in  contrast  to  the  conventional 
seminar  form,  which  comprises  only  two  phases  with  all  above  mentioned  learning  activities. 

3.5  Co-operation 

Co-operation  is  hardly  possible  in  the  conventional  seminar,  but  one  of  the  most  important  advantages  of  virtual 
seminars,  even  if  the  cooperation  process  is  guided  by  the  tutor. 

3.6  Time 

The  term  time  in  virtual  seminars  is  structured  in  greater  detail,  especially  in  the  elaboration  phase.  Giving 
feedback  to  the  students  is  fast  and  easily  possible,  for  the  students  the  possibility  to  give  feedback  to  each  other 
is  really  new.  The  visualization  of  the  working  process  in  the  elaboration  phase  is  an  important  step  to  break 
through  the  student’s  isolation.  Over-all  the  virtual  seminar  is  a time-effective  teaching  method. 

4.  Some  Experiences 

Since  the  Virtual  University  started  (in  winter  term  1996/1997)  about  six  completely  virtual  seminars  have  taken 
place. 

The  experiences  made  with  virtual  seminars  in  the  project  Virtual  University  are  throughout  positive.  A user’s 
survey  of  participants  of  the  virtual  university  in  spring  1 998  showed,  that  the  intensity  of  the  contacts  virtual 
seminars  is  much  better  than  in  conventional  seminars,  also  the  building  of  work  and  learning  groups. 
Furthermore,  most  of  the  students  were  of  the  opinion  that  the  discussion  quality  is  higher  than  in  conventional 
seminars.  The  majority  of  the  students  scored  the  offer  of  online-seminars  with  "very  useful"  [MITT98b]. 
Another  aspect  is  the  dropout  rate,  always  a problem  in  open  and  distance  teaching.  The  drop  out  rate  in  the 
virtual  seminars  was  significant  less  than  in  conventional  seminars. 

Robin  Mason  [MAS094]  describes  related  results  in  the  context  of  synchronous  communication. 

5.  Summary  and  Prospects 

The  virtual  seminar  allows  a well-structured  teaching  process  with  alternating  receptive  and  active-expressive 

learning  activities  which  promote  the  student’s  learning 

success. 

In  contrast  to  conventional  seminars  a continuing  learning  process  is  now  possible,  the  discussions,  taking  place 
in  the  Internet  are  much  more  structured  and  disciplined  than  in  the  conventional  seminars  presence  phase. 
Internet  based  communication  tools  enable  the  very  active  communication,  co-operation  and  discussing 
processes.  The  forming  of  groups  and  learning  together  are  new  and  highly  valuable  activities  for  the  distance 
students. 

By  working  out  the  seminar  contributions  in  HTML  it  is  for  the  first  time  possible  to  have  seminar  contributions 
as  a homogenous  hypertext,  to  avoid  overlapping  of  topics  and  learn  from  each  others  contribution. 

The  successive  publishing  of  the  contributions  enables  the  student  to  discuss  continuously  about  certain  topics, 
in  contrast  to  discuss  all  contributions  at  one  time.  Furthermore,  the  student  is  able  to  concentrate  on  the  main 
thesis  of  his/her  presentation,  because  of  the  most  others  know  his/her  work.  In  addition,  the  students  are  able  to 
learn  from  each  others  work. 
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Criteria  to  give  a successful  virtual  seminar 


Moderation  process 

Critical  for  the  success  of  a virtual  seminar  is  the  moderation  process.  It  is  necessary  to  look  regularly  in  the 
seminar  newsgroup,  to  read  daily  emails,  to  bundle  frequently  asked  questions  and  to  answer  quickly  to  the 
students’  questions.  In  the  beginning  of  the  seminar  many  messages  will  arrive,  but  then  their  number  will 
decline,  because  the  students  prefer  to  communicate  with  each  other,  using  the  newsgroup  and  other 
communication  tools. 

Structuring  and  Support 

As  in  conventional  seminars  it  is  helpful  for  the  students,  if  the  seminar  themes  are  clearly  structured  and  support 
is  given  for  the  literature  research  and  outline  of  the  contribution  (even  if  the  contribution  should  have  the 
HTML-format). 

Discussion  process 

The  system  with  active  and  passive  participants  is  useful  to  reach  the  "critical  mass"  of  students  to  have  a rich 
discussion  process. 

Communication  and  cooperation 

The  cooperation  process  should  be  guided  by  the  tutors  and  the  tutors  should  have  a look  at  the  students 
communication  styles  and  topics. 

Access  control 

In  the  Virtual  University  the  newsgroups  have  an  access  control  (like  the  Virtual  University  itself)  to  protect  the 
students  communication  process. 
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Abstract:  A Technology  Development  Centre  of  Finland  (Tekes)  funded  ProTest  project  has 
been  researching  and  developing  methods  and  systems  intended  for  improving  the  testing 
performed  within  embedded  systems  product  development.  The  Raahe  Institute  of  Computer 
Engineering  being  one  of  the  project  participants  constructed  and  demonstrated  the 
embedded  system  laboratory  environment,  which  can  be  used  as  part  of  everyday 
engineering  education  activities.  In  this  paper,  the  remotely  accessible  laboratory 
environment,  which  improves  the  possibilities  and  flexibility  of  engineering  education  in  the 
field  of  embedded  system  design  and  development,  is  presented.  In  addition  to  this,  the 
demonstration  of  the  environment  with  engineering  students,  and  the  corresponding  results, 
will  be  described  and  analysed. 


Introduction 

The  ever-increasing  need  of  the  electronics  industry  for  engineering  professionals,  and  the  extensive  growth  of 
markets  in  the  field,  require  new  and  more  efficient  methods  for  training  engineers.  In  order  to  keep  up  with 
the  rapid  development  pace,  educational  and  training  institutes  must  drastically  change  and  modify  their 
existing  laboratory  systems,  which  are  often  much  too  expensive  for  being  truly  cost-effective.  The  high-cost 
equipment  needed  for  modem  applications  require  efficient  usage  and  a high  utilisation  percentage  in  order  to 
guarantee  manageable  operation  costs.  Even  in  the  most  optimistic  situations,  the  expenses  will  probably  still 
be  too  high  for  smaller  training  institutes  [Paaso  and  Manninen  97]. 

Keeping  the  aforementioned  aspects  in  mind,  the  Technology  Development  Centre  of  Finland  (Tekes) 
funded  ProTest  project  has  been  researching  and  developing  methods  and  systems  intended  primarily  for 
improvements  in  product  development  testing.  The  Raahe  Institute  of  Computer  Engineering  (RATOL),  being 
one  of  the  project  participants  (together  with  Nokia,  Elektrobit,  and  TH  Engineering),  concentrated  on  the 
embedded  system  test  environment,  which  could  be  used  as  part  of  the  everyday  engineering  education 
activities.  With  close  co-operation  between  RATOL  and  the  Raahe  Laboratory  of  Oulu  University,  the 
experiences  from  the  IDEALS  project  (EU  Telematics  Applications  programme)  could  be  utilised  in  a more 
hardware-oriented  environment  [Manninen  and  Paaso  98,  Ideals  95]. 

Remote  access  or  tele-operating  systems  provide  new  possibilities  for  cost-effective  usage.  Tele-operating 
systems  enable  location-independent  control,  as  well  as  monitoring  and  result  analysing  of  the  laboratory  work 
and  system.  The  students  or  trainees  may,  thus,  access  and  operate  a fully  equipped  high-tech  laboratory, 
regardless  of  their  current  place  of  learning,  which  could  be  a computer  class  room,  work  place,  or  even  the 
trainee’s  home.  This  would  mean  less  travelling  and  a more  flexible  usage  schedule  for  the  students.  In 
addition,  the  remote  access  systems  provide  smaller  institutes  the  possibility  of  using  external  laboratory 
facilities  rented  from  other  organisations.  With  this  kind  of  arrangement,  the  high-end  equipment  would  have  a 
higher  utilisation  percentage,  which  in  turn  would  reduce  the  operational  costs  even  for  the  institutes  that  have 
acquired  the  systems  [Wunnava  and  Hoo  96,  Referowski  et  al  97]. 

The  current  lack  of  human  resources  in  the  field  forces  the  institutes  to  search  for  solutions  that  would 
require  the  least  possible  amount  of  personnel.  Modem  laboratory  facilities,  e.g.  micro-controller  development 
tools  and  emulators,  are  very  complex  and  usually  require  constant  monitoring  by  Laboratory  Engineers.  The 
laboratory  facility  centralisation  model  (Figure  1)  will  enable  integrated  support  and  services,  while  the  remote 
access  systems  will  provide  support-on-demand  even  for  the  remotest  locations. 
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Figure  1 . Laboratory  Facility  Centralisation  Model 


In  this  paper,  the  laboratory  environment,  which  improves  the  possibilities  and  flexibility  of  engineering 
education  in  the  field  of  embedded  system  design  and  development,  is  presented.  In  addition  to  this,  the 
demonstration  of  the  environment,  and  the  corresponding  results,  will  be  described  and  analysed. 


Laboratory  Environment 

The  design  and  development  of  the  remotely  accessible  laboratory  is  based  on  two  factors:  automation  and  tele- 
operation. Automation  enhances  the  laboratory  work  process  by  providing  learners  with  a set  of  tools  and 
procedures  they  can  use  in  order  to  avoid  long  and  tedious  routine  work.  The  tele-operation  system  acts  as  the 
remote  hands  and  eyes  of  the  learners,  who  might  be  physically  located  hundreds  of  kilometres  away  from  the 
actual  laboratory  environment. 

Both  automation  and  tele-operation  have  been  studied  within  various  research  projects  around  the  world. 
Automated  testing  is  only  a partial  solution  when  designing  the  remotely  accessible  laboratory.  The  practical 
work  conducted  with  a high  level  of  automation,  and  usually  as  batch-jobs,  prevents  the  learner  from  getting  a 
real  hands-on  experience  on  the  subject  [Knight  and  DeWeerth  96].  However,  the  technical  demands  of 
versatile  high-tech  tele-operation  systems  may  exceed  the  budget  and  expertise  of  most  educational  institutes. 
Even  if  the  system  itself  could  be  achievable,  the  networks  and  applicable  bandwidth  will  probably  create 
problems  when  attempting  to  transfer  high-quality  and  immersive  tele-presence  environments  to  end-users.  The 
optimal  solution  would  be,  first,  to  combine  some  of  the  features  from  both  of  the  factors  (automation  and  tele- 
operation), and  then  to  construct  supportive  solutions  to  enhance  the  quality  and  usability.  For  example,  virtual 
tele-operation  with  the  help  of  a 3D  model  of  the  remote  system  is  one  approach  to  the  problem  [Rastogi  et  al. 
96]. 

The  laboratory  environment  constructed  in  the  Raahe  Institute  of  Computer  Engineering  includes  a 
complete  and  remotely  accessible  development  and  testing  system,  which  is  automated  with  the  aid  of  scripting 
language.  The  modular  system  is  built  mostly  from  standard  components  so  that  it  is  easily  modifiable  based  on 
the  needs  of  various  laboratory  projects.  Figure  2 illustrates  the  layout  and  infrastructure  of  the  remotely 
accessible  laboratory  environment.  The  testing  facility  included  in  the  environment  is  developed  with 
educational  requirements  in  mind,  in  order  to  guarantee  the  easiness  of  concrete  usage  within  the  areas  of 
training  and  learning.  Furthermore,  the  system  is  integrated  in  a way  that  it  can  be  easily  moved  to  another 
location. 

The  testing  laboratory  environment  consists  of  the  system  constructed  specifically  for  verification  and 
demonstration.  The  central  components  of  the  system  are  the  unit  under  test,  robot,  video  camera,  and  the 
measurement  and  data  transfer  equipment  connected  to  the  control  computer.  In  this  case,  the  unit  under  test 
consists  of  a Lauterbach  emulator,  an  Intel  196  micro-controller,  and  a Bytronics  card  attached  to  the  controller. 

The  remotely  accessible  laboratory  environment  is  connected  to  the  outer  world  through  two  Ethernet  cables 
and  one  power  cord.  The  transmission  and  access  to  the  system  is  managed  through  the  controlling  computer, 
with  the  aid  of  remote  access  software. 


BEST  COPY  AVAILABLE 


930 


The  control  computer  contains  all  the  software  needed  to  operate  the  system  locally  and  remotely.  The 
Windows  NT  Server  4.0  (Terminal  Server  Edition)  acts  as  a platform  for  the  LabView  software,  which  in  turn 
manages  the  robot,  camera,  Input/Output  devices,  and  the  emulator  control.  The  remote  access  is  accomplished 
by  operating  the  control  computer  from  an  external  PC  containing  either  Terminal  Server  Client  or  Windows 
MetaFrame  software.  Figure  3 represents  the  desktop  view  of  an  external  PC,  which  is  remotely  accessing  the 
control  computer,  and  thus,  all  the  resources  manageable  through  that  computer.  The  main  user  interface 
consists  of  main  LabView  test  engine  control,  manual  robot  control,  and  the  video  camera  view  from  the 
laboratory  environment. 
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Figure  3.  The  Desktop  view  from  the  remote  control  and  monitoring  terminal. 


Verification  Demonstration  of  the  Environment 

In  addition  to  the  evaluation  with  end-users,  a wide  variety  of  heuristic  evaluations  from  experts  in  the  field 
were  used  to  identify  the  possible  flaws  of  the  system.  These  evaluations  were  combined  with  the  system 
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demonstration  used  to  verify  the  features  and  functionality  of  the  system.  Within  the  verification  demonstration 
of  the  remotely  accessible  laboratory  environment,  the  possibilities  to  use  and  control  the  environment  through 
tele-operation  have  been  widely  presented  and  tested.  These  possibilities  include,  for  example,  alarms  in 
exceptional  situations,  tele-operation  as  a recovery  procedure  (through  low  bandwidth  communication  network 
using  a GSM  cellular  phone,  modem  and  laptop  PC),  using  the  robot  for  mechanical  stimulus  and  as  a remote 
hand  (power  on,  keyboard  input,  etc,). 

The  tests  and  procedures  included  in  the  verification  demonstration  are  divided  into  four  scenarios,  which 
are  intended  to  model  real  usage  cases.  Each  demonstration  scenario  presents  a different  aspect  of  using  and 
accessing  the  laboratory  environment.  The  various  aspects  include  bandwidth-restricted  usage,  high  level  of 
automation,  and  high  level  of  manual  operations.  The  underlying  engine  with  the  accompanying  software  and 
hardware  is  the  same  throughout  the  demonstration  scenarios. 

Scenario  1 - High  level  of  automation:  Illustration  of  a situation  in  which  the  user  of  the  environment  is 
actually  operating  the  system  from  outside  the  laboratory  premises.  With  this  kind  of  arrangement,  the  end-user 
can,  for  example,  work  at  the  same  time  while  running  automated  testing  sequences.  The  basic  principle  in  this 
scenario  is  the  high  level  of  automation  within  the  laboratory  environment,  and  the  possibility  to  control  and 
monitor  the  system  outside  the  laboratory.  Within  the  demonstration,  the  connectivity  and  the  usability  of  the 
system  through  the  local  area  network  were  tested  and  evaluated. 

Scenario  2 - Tele-operation:  The  emphasis  of  scenario  2 is  on  remote  operations  and  on  the  control  of  the 
laboratory  environment  with  the  aid  of  various  supporting  devices.  The  scenario  is  an  illustration  of  a situation 
in  which  it  is  impossible  to  use  the  highly  automated  procedures,  and  thus,  it  is  compulsory  for  the  end-user  to 
have  enhanced  manual  control  of  the  system.  The  remote-controlled  robot  and  camera  act  as  the  hands  and  eyes 
of  the  user.  Figure  4a  illustrates  the  basic  structure  of  the  system  within  scenario  2.  The  demonstration  includes 
normal  batch  operations,  which  are  halted  by  random  exception  occurring  from  the  unit  under  test  or  from  the 
environment  itself  (e.g,  a power  off  situation). 

Scenario  3 - Mobile:  Demonstrates  the  most  advanced  remote  usage  situation  using  wireless  data  transfer 
and  place-independent  control  and  monitoring  possibilities  of  the  laboratory  environment.  The  case  illustrated 
in  scenario  3 includes  responding  to  the  stimulus  created  by  an  exception,  obtaining  information  from  the  status 
of  the  system,  and  executing  operations  to  recover  the  procedure  from  outside  the  laboratory.  The  interface 
between  the  end-user  and  laboratory  environment  consists  of  a GSM  mobile  phone,  a laptop  computer,  and 
GSM  connection.  Figure  4b  illustrates  Scenario  3.  The  demonstration  includes  responding  to  an  SMS  message 
transmitted  by  the  laboratory  environment  and  the  laptop-based  monitoring  and  controlling  of  the  system. 


Figure  4.  (a)  Remote  control  and  monitoring  with  the  aid  of  a camera  and  robot,  (b)  Remote  access  through 

wireless  communication  network  (GSM). 


Scenario  4 - Tele-teaching  through  remote  expert  services:  Illustration  of  a situation  in  which  it  is 
important  for  the  users  working  in  the  laboratory  environment  to  get  assistance  or  advice.  In  order  to  save  on 
travel  expenses,  the  system  presented  in  scenario  2 may  be  enhanced  with  audio  and/or  video  communication 
equipment,  and  thus,  provide  the  remote  expert  with  an  adequate  feeling  of  being  on  site.  A large  proportion  of 
the  system  features  will  thus  be  accessible  for  the  expert  who,  with  the  aid  of  the  communication  system,  will  be 
able  to  interact  with  the  system  as  well  as  with  the  users  working  within  the  laboratory.  Figure  5 illustrates  the 
demonstration  of  scenario  4. 
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Figure  5.  Tele-teaching  through  remote  access  and  video  conferencing. 

Experiences  with  Engineering  Students 

The  aforementioned  integrated  and  remotely  accessible  laboratory  environment  has  been  demonstrated  and 
tested  using  real  end-users  in  the  Raahe  Institute  of  Computer  Engineering  between  January  1999  and  March 
1999.  The  demonstration  included  a total  of  eleven  B.Sc.  students  (with  various  levels  of  previous  experience) 
who  completed  the  assigned  laboratory  work  as  part  of  their  compulsory  studies.  The  assignment  (30  to  60 
minutes  in  duration)  included  programming,  testing,  problem  solving,  and  analysing  of  the  micro  controller 
(Intel  196  with  Lauterbach  emulator).  The  objectives  of  the  task  were  set  in  a way  that  students  were  forced  to 
exploit  all  features  of  the  system  in  order  to  achieve  satisfactory  results. 

The  evaluation  consisted  of  scenarios  1 and  2,  with  scenario  4 downscaled  to  meet  the  available  resources 
(the  videoconferencing  was  changed  to  a mobile  phone  connection).  The  students  completed  their  assignments 
by  working  in  a remote  access  laboratory  with  no  possibility  for  "on  site"  operations.  The  actions  of  the  students 
were  monitored  in  their  work  place  as  well  as  within  the  laboratory.  Furthermore,  a set  of  questionnaires  was 
used  to  collect  feedback  from  the  users.  Figure  6 outlines  the  demonstration  arrangements. 


Figure  6.  Demonstration  and  evaluation  environment  of  the  remotely  accessible  laboratory. 


The  questionnaires  were  used  to  gather  information  about  the  expectations,  experiences,  and  opinions  of  the 
end-users.  The  questions  were  divided  into  the  following  categories:  user  background  (previous  experience), 
functionality  (system  performance),  and  comparison  (with  the  traditional  laboratory  work). 

The  analysis  of  the  questionnaires  indicates  positive  feedback  from  the  users.  From  scale  1 to  5 in  a total  of 
26  system  evaluation  questions,  the  lowest  average  score  was  3.1,  while  80  per  cent  of  the  questions  had  an 
average  above  3.6.  The  top  20  per  cent  of  the  questions  had  averages  from  4.5  to  4.9.  From  the  averages  and 
from  the  distribution  of  the  scores  within  individual  questions,  the  following  observations  can  be  made: 
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• The  system,  with  its  current  implementation,  does  not  prepare  the  students  for  practical  work  as  well  as 
was  required  (average  3.1),  although  it  may  be  argued  whether  the  result  is  actually  worse  or  better  than 
with  the  traditional  laboratory  arrangements.  Of  course,  the  lack  of  all  the  physical  elements  of  the 
normal  laboratory  might  move  the  student  away  from  the  real  world  usage. 

• The  completion  of  the  given  assignment  was,  in  the  students’  opinion,  as  feasible  to  accomplish  as  with 
the  traditional  environment  (average  4.0).  It  was  even  stated  that  the  connection  to  the  teachers  was 
easier  to  maintain  with  the  arrangement  described  above  (average  4.5). 

• Although  limited  in  terms  of  implementation  and  usability,  the  system  is  far  better  when  compared  with 
the  traditional  situation  with  the  resulting  need  to  travel  to  the  laboratory  facility  (average  4.3). 

• The  students  themselves  believe  in  the  system  and  were  motivated  to  exploit  its  potential  also  in  the 
future  (average  4.7). 

Conclusions 

The  great  demand  for  engineering  professionals  requires  new  methods  for  increasing  the  output  of  graduates 
from  universities  and  polytechnics.  One  proposed  solution  to  the  problem  is  the  integrated  and  remotely 
accessible  laboratory  environment,  which  enables  more  efficient  learning  and  practical  working,  even  in 
sparsely  populated  areas. 

The  laboratory  environment  constructed  in  the  Raahe  Institute  of  Computer  Engineering  has  shown  that  it 
is  possible  to  create  remotely  accessible  laboratories  with  current  standard  techniques  and  components. 
Furthermore,  the  verification  and  demonstration  of  the  environment  result  in  a large  number  of  possibilities 
corresponding  to  the  requirements  of  the  practical  learning  and  training. 

The  remotely  accessible  laboratory  can  be  constructed,  to  a certain  extent,  with  readymade  and  standard 
components.  However,  a large-scale  development  and  testing  environment  requires  more  software  and 
hardware  components,  which  in  turn  results  in  problems  within  the  system  integration.  Even  in  a small-scale 
realisation,  the  major  problems  encountered  were  caused  by  the  integration  of  various  software  components  and 
hardware  incompatibilities.  All  of  the  problems  can  be  solved  with  additional  work  and  further  development,  so 
the  solution  presented  is  realistic  even  for  smaller  educational  institutes. 

The  verification  indicates  that  the  end-user  of  a remotely  accessible  laboratory  environment  does  not 
necessarily  need  high-tech  futuristic  technology,  but  merely  safer,  but  applicable,  lower-end  tools  and  methods. 
For  example,  video  conferencing  with  high  quality  real-time  images  would  seem  to  be  less  favoured  than  the 
robust  and  somewhat  inferior  connection  and  interfaces  enabling  the  operating  in  a remote  environment. 

The  remotely  accessible  laboratory  environment  provides  more  flexibility  for  everyday  engineering  training, 
with  time  and  place  independent  access  possibilities.  Furthermore,  the  fact  that  the  environment  is  usable 
anywhere  from  the  network  enables  cost-effective  centralisation  procedures  and,  thus,  provides  an  opportunity 
for  smaller  organisations  to  exploit  the  features  of  a modem  and  high-tech  development  and  testing  laboratory. 
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Abstract:  In  the  education  of  embedded  software  engineers  there  is  a need  for  increasing 
the  graduate  output.  A substantial  amount  of  re-training  and  on-the-job  training  is  needed; 
thus,  there  should  be  plenty  of  flexibility  in  the  teaching  arrangements.  CBT  utilising 
telematic  facilities  is  the  most  relevant  form  of  technology  in  North em-Finland  and  the 
Oulu  area,  where  distances  are  long,  but  the  telecommunication  infrastructure  is  well- 
developed.  The  IDEALS  modular  training  system  (MTS)  and  methodology,  for  co- 
operative learning  and  authoring  utilising  CBT  courseware,  are  presented  in  this  paper. 
Concrete  results  show  the  cost-efficiency  of  the  methodology  and  system  in  the  CBT 
courseware  production. 


1.  Background 

The  IDEALS  project  of  the  EU  Telematics  Applications  programme  was  co-ordinated  by  Fraunhofer-IGD  in 
Darmstadt,  Germany.  There  were  thirteen  partners  from  Germany,  Greece,  Portugal,  Holland  and  Finland  in  the 
project,  which  ended  on  30  June  1998.  The  Raahe  Laboratory  of  the  University  of  Oulu,  RATOL,  was  one  of  the 
six  local  training  centres  (LTC)  in  the  project  learning  network,  and  had,  as  its  main  task,  to  co-operatively 
create  and  demonstrate  the  use  of  the  Computer  Based  Teaching  (CBT)  courseware  (CW)  on  the  topic 
’’Fundamentals  of  Computer  Graphics”  for  students  in  Institutes  of  Higher  Education  [Paaso  98,  IDEALS  95]. 

In  the  Raahe  Laboratory  of  the  University  of  Oulu,  the  ultimate  goal  for  the  CBT  research,  which  was  based 
on  the  IDEALS  methodology  and  system,  was  the  following:  A CBT-hased  methodology  developed  for 
embedded  software  engineering  education  will  enable: 

7.  to  develop,  by  utilising  the  telematic  multimedia,  the  desired  and pedagogically  functional  learning 
model,  which  is  based  on  the  learning-by-doing’  -philosophy  and  on  individual  student  profiles,  and 
which  originates  from  the  capabilities  of  graduates  required  by  industry 

2.  to  achieve,  as  a replacement  or  alternative  to  the  traditional  teacher-centric  education  model, 
significant  flexibility  and  efficiency  improvements  in  the  delivery  of  teaching 

3.  to  bring  a significant  cost-effectiveness  improvement  to  the  production  of  CBT  courseware  by  means 
of  CW  reuse. 

Essential  criteria  for  the  developed  learning  model  in  the  embedded  software  engineering  education  were  the 
support  for  learning,  regardless  of  time  and  place,  individual  learner  profiles,  learning  by  doing,  industrial  type 
teamwork  and  collaboration  (i.e.  concurrent  engineering  and  activation  to  interaction  between  students), 
interactivity  between  the  computer  and  the  student,  continuous  testing  of  the  knowledge  level  as  an  essential  part 
of  the  learning  process  [Manninen  and  Paaso  98a].  In  the  co-operative  telematic  authoring  part,  the  essential 
success  criteria  for  the  methodology  were  the  reduction  of  the  CBT  production  effort  and  the  easy  maintenance 
of  the  CBT  courseware.  These  criteria  were  validated  by  applying  the  developed  methodology  to  a representative 
course,  the  Graphical  User-interface  Programming  course,  in  the  embedded  software  area  [IDEALS  96]. 


2.  IDEALS  Learning  and  Authoring  Scenarios 

The  basic  architecture  of  the  IDEALS  Modular  Training  System  (MTS)  is  presented  in  Figure  1 . 
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Figure  1.  Overall  architecture  of  IDEALS-MTS. 


In  the  IDEALS  learning  scenario,  the  following  major  learning  sites  supported  can  be  identified: 

• remote  trainees  (individual  access  to  the  local  training  centre,  LTC) 

• trainees  at  training  service  provider  (TSP) 

• trainees  at  LTC. 

LTCs  and  TSPs  are  almost  identical  in  hardware  and  software.  The  only  difference  between  them  is  that 
LTCs  provide  management  functionality  for  distributed  courseware  and  administration  functionality  for  users. 
Each  LTC  is  the  owner  of  a distinct  subset  of  the  distributed  CW  domain.  It  may  also  provide  copies  of  CW 
owned  by  other  LTCs.  TSPs  provide  only  copies  of  CW  from  LTCs;  their  purpose  is  the  provision  of  computing 
power  and  CW  in  close  proximity  to  the  trainee's  working  place.  Each  LTC  and  TSP  provides  a CW  domain 
server  and  an  MTS  server.  All  LTCs  are  interconnected.  Over  these  trans-European  interconnections  the  LTCs 
share  their  CW  databases,  and,  therefore,  can  be  seen  as  one  virtual  LTC,  giving  access  to  the  whole  CW 
domain.  Learning  can  start  immediately  from  the  WWW-browser.  After  login,  the  trainee  can  start  a new  course, 
resume  a saved  course,  issue  a training-on-demand  request,  or  select  a learning  group.  After  working  on 
presentation,  exploration  and  test  units,  the  course  status  is  saved  and  the  learner  profile  updated.  Interactive 
learning  can  be  added  to  any  type  of  course  usage  at  any  time,  either  as  expert  consultation,  or  by  joining  a 
learning  group  [Paaso  et  al.  97]. 

An  additional  important  scenario  in  the  IDEALS  context  is  the  co-operative  authoring  of  the  courseware  and 
final  courses.  In  this  scenario,  authors  located  in  different  LTCs  are  preparing,  discussing,  and  finalising  the 
common  CW  domain,  using  different  telecommunication  capabilities,  such  as  audio  and  video  conferencing.  In 
order  to  create  a common  look  and  feel  for  the  common  CW,  IDEALS  style  guides  are  in  use;  however,  creation 
of  these  guides  first  presumes  “base”  CW  production.  Courseware  catalogues  consisting  of  the  CW  module 
profiling  information  are  available,  thus  enabling  the  multiple  use  of  CW. 


3.  Courseware  Characterisation 

The  target  group  for  the  implemented  CW  on  the  topic  Graphical  User-interface  Programming,  was, 
primarily,  students  in  universities  and  technical  colleges  (institutes  of  higher  education,  IHE),  and  also  personnel 
in  small  and  medium  sized  enterprises  (SME).  A more  detailed  target  group  includes  persons  who  already  are 
able  to  do  programming,  e.g.  in  C-H-;  so,  in  the  course  there  is  no  training  for  programming  basics.  The  course 
material  contents  handle  Windows-C  programming,  as  a subset  of  the  user-interface  programming  area.  An 
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emphasis  area  in  the  course  are  functional  parts,  which  activate  students  to  learn  by  themselves  (leaming-by- 
doing),  and  not  only  follow  the  course  passively.  Animations,  interactive  examples,  exercises  and  versatile 
hypertext  linking  support  the  theory  part  and  make  it  more  illustrative. 

Each  section  of  the  CW  consists  of  presentation,  exploration  and  test  parts,  whose  total  duration  and  division 
follow  the  guidelines  agreed  upon  in  the  IDEALS  consortium,  i.e.  for  presentation  30  per  cent  exploration  50  per 
cent  and  testing  20  per  cent: 

• In  the  presentation  part,  there  are  typically  text-picture-combination  presentation  pages  with  hot  words  (by 
clicking  them  sub-windows,  which  provide  users  with  some  specific  detail,  are  opened). 

• In  the  exploration  part,  the  'Step-by-step’  learning  principle  is  applied.  The  overall  learning  process  of 
programming  is  divided  into  the  following  four  steps: 

1 . Assignment  Viewing:  At  the  beginning  of  a new  programming  subject  to  be  learned,  learners  are 
introduced  to  the  final  goal,  i.e.  what  they  should  be  able  to  do  after  this,  in  order  to  obtain  an  in-depth 
and  clear  overview  of  the  assignment. 

2.  Virtual  Compilations:  The  virtual  compiler  simulates  the  features  of  the  real  compiler,  although  in  a 
much  restricted  scale.  This  provides  the  learner  with  the  opportunity  to  explore,  modify  and  test  some 
ready-made  software  components  without  causing  the  system  to  crash,  i.e.  in  Windows  programming 
to  change  the  windows  attributes,  etc. 

3.  Real  Compilations:  When  learners  have  achieved  sufficient  experience  in  the  virtual  environment,  they 
are  ready  for  trying  out  the  same  procedures  in  the  real  compiler  environment. 

4.  Test/Final  Project:  A major  feature  enabled  by  telematic  CBT  is  the  integration  of  teaching, 
development  tools  and  teamwork  into  one  learning  environment.  In  RATOL’s  CW  this  means  distant 
independent  collaboration  between  group  members  in  designing  and  developing  software  using 
concurrent  engineering. 

• In  the  testing  phase,  the  following  types  of  test  forms  are  used  at  present:  True-False,  Matching  Items,  Fill- 
In,  Single  Choice  from  Multiple  Items  and  Multiple  Choice  from  Multiple  Items  tests.  In  the  representative 
sample  of  Figure  2 from  RATOL’s  CW,  three  of  these  test  types  can  be  found  [Manninen  and  Paaso  98b]. 


Figure  2.  Sample  of  a test  form. 
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4.  Reuse  and  Multiple  Use  of  Courseware 


The  multiple  use  of  CW  means  the  use  of  the  same  CW  entities  by  many  users.  The  reuse  of  CW  means  the 
use  of  CW  production  patterns  (so-called  templates)  by  many  CW  producers  for  the  production  of  different  CW 
entities.  Basic  building  blocks,  BBBs,  hold  a central  role  for  the  multiple  use  of  CW  (especially  between 
different  LTCs),  and  for  the  understanding  of  extensional  and  itemisation  modularity  of  CW,  and  of  the 
procedure  of  designing  courses,  as  can  be  seen  in  Figure  3 (the  linkage  mechanisms  between  the  modules  of  the 
CW  shown)  (IDEALS  1996).  The  basic  building  blocks  are  the  components  utilised  from  the  common  CW 
database  by  a teacher,  in  order  to  configure  the  curriculum  of  a course  and  to  specify  the  course  flow  through  it. 
A basic  building  block  is  a composition  of  one  course  node,  several  learning  function  units  and  their  learning 
material  objects.  Their  purpose  is  to  represent  a certain  piece  of  knowledge  in  a CBT-based  format. 


3-level  Modularising 
Principle  of  IDEALS: 

1 . Course  Nodes:  course-flow  as  a 
flowchart  (branching  dependent 
e.g.  on  the  student  profile  and 
previous  tests) 

2.  Function  Units:  views  on  the 
student’s  screen 

3.  Learning  Material  Objects: 
learning  contents  as  various 
multimedia  presentations 


Figure  3.  Structure  of  a Basic  Building  Block. 


The  3-level  modularising  principle  of  the  CW,  with  self-descriptions,  virtual  references  and  user  profiling, 
are  the  strong  points  of  the  IDEALS  methodology.  They  enable  the  customisable  and  individual  course  flows. 
Basic  building  blocks  and  templates  in  the  common  CW  database,  listed  as  catalogues,  are  the  basis  for  multiple 
use  and  reuse  of  courseware. 


5.  Considerations  of  Authoring  Efforts 

The  official  evaluation  of  the  IDEALS  project  was  performed  during  spring  1998,  and  involved  the 
evaluation  of  both  authoring  and  learning  phases  in  the  IDEALS  demonstration  infrastructure.  Based  on  the 
results  relating  to  the  authoring  phase,  interesting  figures  may  be  presented.  In  Figure  4,  comparisons  of  the 
production  efforts  of  the  various  versions  of  RATOL’s  CW  are  presented  (all  three  phases  related  to  the  same 
CW,  i.e.  to  the  various  incremental  development  phases  of  Graphical  User- Interface  Programming,  GUIPRO\ 
including  also  the  case  where  multiple  use  of  40  per  cent  of  the  CW  is  utilised.  In  RATOL’s  16-hour  GUIPRO 
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course,  the  multiple  use,  thus,  means  producing  9.5  hours  of  own  CW  and  importing  6.5  hours  of  CW  from  other 
two  IDEALS  IHE  pilot  partners. 


Some  conclusions  relating  to  Figure  4 contents  can  be  drawn: 

• Utilising  of  templates  and  multiple  use  clearly  reduces  production  effort.  Larger  multiple  use  percentages 
would,  obviously,  bring  more,  nearly  linear,  reduction  in  production  efforts. 

• There  is  a distinct  learning  curve  in  the  CW  production  that  can  be  seen  as  the  development  continues.  The 
templates  also  improve  along  with  the  production.  Both  have  a clear  positive  effect  on  the  CW  production 
efforts. 

• The  production  effort  ratio,  even  with  multiple  use  (i.e.  40  per  cent  in  RATOL’s  case)  is  still  relatively  high, 
i.e.  50  hours  of  production  for  one  learning  hour.  However,  it  seems,  based  on  RATOL’s  experiences,  that, 
with  high-quality  interactive  CBT  CW,  the  required  CW  hours  for  individual  learning  could  drop  to  nearly 
half  of  the  required  contact  teaching  hours  for  the  same  learning  contents  in  traditional  teaching.  This  would 
indicate  that  the  real  production  effort  ratio,  for  the  same  learning  result,  could,  thus,  be  evaluated  with  this 
type  of  interactive  CW  to  be  without  multiple  use  at  about  45:1,  and  even  better  than  30:1  with  multiple  use 
of  40  per  cent.  However,  to  verify  this  conclusion,  more  extensive  evaluations  with  larger  amounts  of  CW 
and  students  are  required.  Naturally,  for  the  first  course  runs,  the  proportional  production  costs  are  high,  but 
after  that  the  CBT  course  can  be  very  cost-effectively  repeated  for  large  student  groups  [Paaso  98]. 


6.  Situation  and  Learning  Results 

RATOL  has  been  running  the  IDEALS-MTS  based  CBT  course.  Graphical  User  Interface  Programming, 
since  the  beginning  of  November  1997.  The  course  is  targeted  at  B.Sc.  (Eng.)  and  M.Sc.  (Eng.)  students 
(Engineer  and  Diploma  Engineer  students  correspondingly)  in  the  field  of  computer  engineering.  The  course  can 
also  be  accessed  by  visiting  users  also  through  the  internet  (Netscape,  version  3.0  or  later): 

• Address:  http://ltCTatol.Ji 

• User-id:  guest 

• Password:  guest. 
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From  the  first  two  course  runs  (Nov.  - Dec.  1997  and  March  - April  1998,  when  also  some  Nokia  test  users 
were  involved),  there  are  18  students  who  have  completed  the  whole  course  so  far  (exam  and  the  teamwork 
project  work);  this  is  about  half  of  those  who  signed  up  to  the  course.  It  is  to  be  noted  that  the  course  is  optional 
in  the  course  curriculum  of  the  students.  The  average  duration  of  studies  among  these  students  was 
approximately  three  weeks,  resulting  in  an  average  of  three  hours  of  study  per  week  per  student.  The  learners 
were  relatively  motivated  to  study  on  their  own  during  the  first  course  run.  There  were  no  major  activation 
procedures  on  the  part  of  the  lecturer  or  the  tutors.  In  the  second  course  run,  the  overall  activity  of  the  learners, 
and  knowledge  about  their  progress,  could  be  increased  by  means  of  milestones  and  rewarding  procedures,  i.e. 
some  didactic  tests  were  included;  they  have  a slight  effect  on  the  student’s  mark  received  from  the  course. 

When  considering  the  students  who  successfully  passed  the  course,  the  learning  results  are  positive, 
especially  after  considering  the  fact  that  they  have  been  able  to  implement  the  programming  project  and  pass  the 
test.  Even  more  interesting  is  to  know  what  is  the  teaching  and  tutoring  effort  during  the  course  runs.  It  is 
estimated  that  10  hours  of  the  highly  interactive  and  repeatable  GUIPRO  courseware  corresponds  to  about  30 
contact  teaching  lessons  (each  45  minutes,  in  total  22.5  hours).  The  start-up  lesson  and  tutoring  are  estimated  to 
require  approximately  seven  hours  in  total  per  course  run;  this  means  a saving  of  approximately  15  hours  in  the 
teaching  effort  for  this  learning  result. 

The  IDEALS  methodology  applied  to  embedded  software  engineering  education  functions  extremely  well, 
and,  in  fact,  the  methodology  would  seem  to  be  excellent.  However,  the  technical  status  of  the  IDEALS-MTS 
leaves  plenty  of  development  areas.  At  present,  the  instability  and  the  deficiencies  in  the  co-operative  authoring 
support,  in  particular,  are  the  most  challenging  technical  problems.  The  technology  available  almost  certainly 
will  change  and  develop.  The  next  version  of  IDEALS-MTS  will  also  be  an  improved  version. 


7.  Conclusions 

The  greatest  value  of  IDEALS,  for  the  future,  is  the  learning  network  producing  CW  in  the  same  modular 
(’standardised')  way,  and  this  is  the  optimum  time  to  expand  the  network  for  new  CW  authors  and  courses.  By 
means  of  co-operative  authoring,  remarkable  cost-effectiveness  improvements  in  the  costly  CBT  production  can 
be  achieved.  CBT  course  utilisation  may  be  at  least  a partial  solution  to  the  huge  need  of  increasing  the  graduate 
output  in  the  software  engineering  area.  Also,  for  the  students,  it  can  bring  an  increased  amount  of  flexibility  and 
financial  savings,  e.g.  by  decreasing  unnecessary  travelling,  in  the  learning  arrangements.  For  those  who  are 
fully  employed  during  their  studies,  it  might  even  be  the  only  possibility. 
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Abstract:  The  activities  of  our  institute  cover  various  fields  in  the  development  process  of  in- 
tegrated circuits  including  digital,  analogue  design  and  test. 

For  these  separate  fields  we  developed  an  interactive  learning  system.  It  is  based  on  the  capa- 
bilities of  the  hypertext  markup  language  (HTML)  and  uses  the  HTTP  internet  protocol  for  dis- 
tributing its  contents.  The  aim  was,  to  improve  the  quality  of  the  student’s  education  by  offering 
interactive  pages,  that  allows  on  the  one  hand  access  to  a knowledge  base  and  on  the  other  hand 
the  training  on  special  dynamic  examples.  The  usage  of  this  system  necessitated  amplification 
so  additional  tools  were  developed. 

The  original  VHDL  ( Very  high  speed  integrated  circuit  Hardware  Description  Language)  tuto- 
rial (www.vhdl-online.de)  presented  an  1 1 lesson  course  with  an  implemented  online  help  sys- 
tem and  a VHDL  glossary.  The  additional  features  are  a multiuser  editor  for  moderated 
teleworking  in  distributed  networks  with  graphical  enhancements  and  a hardware  interface  for 
programming  a field  programmable  gate  array  (FPGA)  with  a synthesized  VHDL  module  to 
start  an  online  test.  For  our  students  in  the  first  and  second  year  of  electrical  engineering,  we 
developed  tools  for  learning  nodal  analysis  and  displaying  the  numerical  results  as  a Bode-plot 
and  the  symbolical  results  as  a transfer  function. 


Motivation 

At  the  moment  students  use  our  VHDL  Online  system  to  get  information  about  the  hardware  description 
language  VHDL  and  the  design  flow  for  developing  digital  circuits.  It  is  an  important  task  of  our  institute  to  show 
students  the  different  possibilities  of  designing  integrated  circuits.  The  digital  design  methodology  is  only  one 


figure  1:  VHDL  Online  Server  Statistics 
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part  of  this  field  and  in  the  ftiture  the  need  of  mixed  signal  designs  (analogue  and  digital)  will  grow.  An  integrated 
learning  platform  for  the  different  fields  of  circuit  design  is  a good  starting  basis  for  the  students.  The  continuous 
usage  of  the  existing  learning  environment  confirms  our  strategy  in  teaching  and  is  highly  accepted  by  the  students. 
Fig.  1 shows  the  statistics  of  our  VHDL  Online  web  server.  For  this  reason  we  decided  to  expand  the  VHDL  Online 
system  with  additional  submodules  and  analogue  components.  The  learning  system  is  divided  into  several  sub- 
modules. 


The  VHDL-Online  Platform 

• VHDL  Tutorial:  It  is  structured  like  a lecture.  In  one  frame  the  students  see  the  slides  used  in  the  normal  lec- 


figure  2:  VHDL  Online  Workshop 

ture  with  the  major  difference  that  under  the  frame  the  slide  will  be  illustrated  as  plain  text.  The  contents  of  the 
text  is  the  same  as  the  tutor  tells  the  students.  At  the  end  of  a chapter  the  user  can  answer  multiple  choice  ques- 
tions related  to  the  contents.  If  the  answer  is  wrong,  links  will  appear  in  the  window,  where  the  user  gets  sup- 
port to  the  special  topic. 

• VHDL  Reference:  If  a designer  has  got  a VHDL  problem  (e.  g.  syntax  of  a procedure)  he  wants  to  get  the 
information  very  fast  instead  of  searching  for  the  answer  in  a lot  of  slides.  So  we  implemented  a search  func- 
tion in  this  module. 

• VHDL  Workshop:  In  the  next  step  the  students  apply  their  new  knowledge  to  a practical  training.  They  design 
a camera  control  system  using  VHDL.  During  the  separate  exercises  it  is  possible  to  get  information  about  a 
special  keyword  from  our  VHDL  Reference.  The  students  can  use  the  search  function  of  this  module.  The  aim 
of  this  workshop  is  writing  VHDL  Code,  simulating  and  synthesizing  it  with  professional  design  software 
(Synopsys)  from  any  internet  account  of  the  world  if  the  user  got  the  permission  from  our  institute.  At  the  end 
of  this  training  it  is  possible  to  generate  a netlist  of  the  circuit.  But  until  now  the  designer  has  not  been  able  to 
transfer  the  circuit  to  real  hardware.  We  needed  an  additional  hard-  and  software  module  to  program  a FPGA. 
The  students  can  verify  their  lesson  results  on  real  hardware.  These  new  features  are  described  later  in  this 
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paper. 

• VHDL  Chat:  The  first  intention  for  this  module  was  the  support  of  students  working  outside  our  office.  If  a 
problem  occurs  they  can  chat  with  other  users  to  exchange  their  experience.  But  we  received  negative  feed- 
back from  the  users  because  it  is  hard  to  describe  a problem  online  to  other  persons  who  are  not  able  to  see  the 
source  code.  The  reason  for  this  problem  is  the  strategy  of  the  IRC  (internet  relay  chat).  It  is  only  possible  to 
send  one  line  to  the  other  chat  members.  Another  point  is  that  not  every  user  works  at  the  same  time  and  join 
the  chat  group  to  answer  questions.  So  we  decided  to  develop  a multiuser  editor  where  other  people  (e.  g.  the 
tutor)  can  look  at  a source  code  and  make  comments  to  special  lines.  This  editor  is  described  later  in  the  paper. 

Fig.  2 shows  a collection  of  the  tutorial.  In  the  next  chapter  the  functionality  and  the  structure  of  the  new 
integrated  applications  are  described. 


Additional  Applications  for  VHDL  Online 


Modified  Nodal  Analysis  Environment 


Electrical  engineering  students  have  to  analyze  circuits  consisting  of  resistors,  capacitors,  inductors  etc.  and 
qualify  the  voltage  or  the  electric  current  of  a network.  To  do  this  they  calculate  the  transfer  function  using  modi- 
fied nodal  analysis.  For  a small  circuit  it  is  easy  to  manage  with  paper  and  pencil.  But  it  is  also  important  to  show 
the  students  in  a short  amount  of  time  how  the  circuit  behaves  if  the  parameters  of  an  electronic  component  are 
changed.  It  is  not  necessary  for  the  students  to  recalculate  the  same  circuit.  For  this  reason  our  institute  developed 
a learning  system  containing  (Fig.  3)  the  following  subcomponents: 

The  first  point  is  a knowledge  database  which  contains  background  information  about  the  topic  of  analyzing 


figure  3:  Modified  Nodal  Analysis 


electrical  circuits.  This  hypertext  based  module  uses  links  to  get  detailed  information  for  students  with  little  expe- 
rience. In  our  case  the  modified  nodal  analysis  will  be  described.  Because  of  the  modular  programming  structure 
of  the  whole  system  it  is  possible  to  exchange  this  current  method  with  another  one.  After  learning  the  basics  of 
the  modified  nodal  analysis  the  student  solves  exercises  to  apply  this  method  to  an  example.  This  circuit  can  be 
drawn  in  the  second  module.  It  is  a schematic  entry  tool  where  the  user  can  place  the  electronic  components  (power 
supply,  measurement  instruments,  resitstors  etc.),  assign  parameters  and  connect  the  modules.  This  program  has 
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got  all  edit  features  of  a professional  graphic  tool  (copy,  paste,  rotate,  move,  zoom  in,  zoom  out  etc.).  The  tool  also 
automatically  checks  whether  the  designed  schematic  is  correct  and  shows  the  error  location  in  case  of  a fault.  All 
components  and  parameters  of  a spice  netlist  are  supported  by  the  tool.  After  the  input  of  the  whole  circuit  either 
the  netlist  can  be  exported  to  another  professional  simulation  tool  (HSPICE)  or  the  next  system  module  will  be 
activated  to  calculate  the  transfer  function.  The  program  shows  the  elements  of  the  matrix  in  an  any  desired  way 
and  the  student  must  place  them  to  the  fitting  position  in  the  matrix.  Thus  the  user  does  not  solve  the  exercises  just 
by  clicking  buttons  but  rather  must  he  comprehend  the  things  he  learned  in  the  lessons.  To  analyze  the  circuit  it  is 
important  to  get  additional  information  like  magnitude  or  phase  of  the  transfer  function.  These  graphs  are  generated 
by  the  fourth  program  module  and  represented  in  a diagram.  Some  additional  buttons  alleviate  the  operating  of  the 
program. 


The  Multiuser  VHDL  Editor  for  Moderated  Teleworking 

To  improve  the  communication  among  the  students  a multiuser  editor  for  moderated  teleworking  in  distrib- 
uted networks  was  implemented.  The  editor  allows  a user  to  invite  other  users  to  an  open  document.  Several  user 
authorizations  can  be  set  by  the  owner  of  the  document  to  restrict  access.  These  authorizations  are  reading,  writing, 
drawing  and  copying.  If  there  is  a project  with  a common  database,  all  users  must  have  read  access  to  the  database, 
but  only  the  owner  of  the  database  has  write  access.  The  common  database  can  be  a specification  or  a document 


figure  4:  Multi  User  Editor  for  Moderated  Teleworking 


that  contains  global  variables  or  inter  module  connections.  For  group  work  or  for  interface  descriptions  only  the 
group  members  and  the  concerned  interface  groups  may  have  access  to  the  core  documents,  but  with  different  au- 
thorizations. The  tutor  or  superuser  can  see  all  documents.  So  group  work  can  be  established  over  a network.  Every 
user,  who  starts  the  editor  will  be  connected  to  a server  that  is  running  at  our  institute  providing  information  about 
work  in  progress  and  retrieving  all  the  documents  allowed  to  participate.  Once  connected  to  the  server,  all  docu- 
ments will  be  acknowledged,  but  only  those  documents  with  invited  co-users  are  broadcasted  to  the  mentioned  us- 
ers. An  easy  to  use  graphical  interface  was  implemented  to  support  this  feature.  Every  user  can  have  different  rights 


944 


BEST  COPY  AVAILABLE 


on  different  documents.  The  document  source  remains  on  the  computer  of  the  documents  owner,  even  if  it  is  a dis- 
tributed document.  Simultaneous  work  of  more  than  one  user  on  one  document  is  possible,  a scheduling  mecha- 
nism was  implemented.  User  identification  is  done  by  getting  a user  name  and  a machine  name  or  an  internet 
address.  A further  benefit  of  the  editor  is  an  intuitive  handling  of  the  editor  functions.  The  editor  behaves  like  a 
normal  text  editor,  so  the  user  needs  no  time  for  practicing.  Freely  movable  text  or  graphic  can  be  put  anywhere  in 
the  main  text  flow  and  can  be  anchored  there.  This  is  useful  for  commentating  issues  in  the  main  text  or  for  building 
exercises  with  gap  text.  Rudimentary  graphic  functions  are  also  implemented,  so  readability  of  the  text  improves 
by  laying  out  colored  structures.  Those  additional  features  have  no  effect  on  the  structural  appearance  of  the  main 
text  flow,  therefore  the  main  text  can  either  be  saved  separately  or  can  be  used  directly  for  compiling,  syntesizing, 
etc.  Fig.  4 shows  an  editor  screenshot  of  a VHDL  document  with  some  comment  text  (of  course  no  one  is  forced 
to  use  these  colors  or  comments). 

Due  to  platform  independence  and  best  compatibility  the  editor  was  developed  as  a java  application  running 
on  all  platforms  supporting  java  version  1.1.  The  concept  of  the  editor  allows  easy  updates  and  adaption  to  new 
requirements.  It  is  planned  to  write  a java  applet  version  for  using  the  editor  within  a HTML  document. 


The  FPGA  Programming  Environment 


Until  now,  VHDL  Online  provides  a tutorial  for  VHDL  that  starts  with  simple  VHDL  examples  and  ends 
up  in  the  synthesis  of  a complete  design.  Our  idea  now  is  to  extend  the  tutorial  so  that  students  can  test  their  own 
designs  on  a real  hardware  (Fig.  5).  To  extend  our  internet  services,  there  are  some  additional  requirements  to  the 
hardware.  Most  important,  the  new  hardware  should  be  available  within  a very  short  time.  We  started  a new  project 
to  examine  the  use  of  medium  size,  SRAM  based  FPGAs  as  hardware  resource  for  the  designs  from  our  tutorial. 
First,  we  changed  the  designflow  to  add  some  more  steps  that  are  required  for  FPGAs.  This  especially  is  the  map- 
ping from  the  result  of  the  synthesis  process  to  the  programfile  for  the  FPGA. 

The  total  design  flow  now  has  the  following  steps: 


Hardware 

Software 


Information 

Sourcefile 


figure  5:  FPGA  Designflow 


1 . Synthesis  of  the  VHDL  sources  to  a complete  design. 

2.  Design  Optimization  with  respect  to  the  target  hardware.  This  means  that  a special  synthesis  library  from  the 
FPGA  vendor  has  been  used  for  the  compilation  of  the  design. 

3.  Export  the  design  as  a netlist  (EDIF). 

4.  A special  software  from  the  FPGA  vendor  allows  the  mapping  of  the  EDIF  netlist  from  step  3 to  the  target 
hardware.  This  will  generate  a sourcefile  that  allows  the  programing  of  the  FPGA  (FPGA  programfile). 

5.  Download  the  FPGA  Programfile  to  the  hardware. 

6.  Provide  a suitable  user  interface  though  the  internet,  that  enables  the  user  to  interact  with  the  hardware. 
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Until  now  an  experimental  board  with  one  small  FPGA  has  been  developed.  Using  this  board  the  complete 
designflow  has  been  verified.  The  synthesis  step  has  been  modified  to  generate  the  required  EDIF  netlist.  We  also 
wrote  a small  utility  for  the  download  of  the  generated  FPGA  Programfile.  After  some  additional  changes  to  our 
design  and  our  hardware  we  successfully  implemented  our  design  on  the  FPGA  hardware.  As  a result  we  proved 
that  it  is  generally  possible  to  provide  tests  with  the  real  hardware  to  the  internet  community.  Now  our  activities 
concentrate  on  the  development  of  suitable  hard-  and  software  for  the  access  from  the  internet  to  our  FPGA.  We 
decided  to  build  a java  application  on  the  user  side.  This  seems  to  be  the  most  flexible  and  portable  way  to  provide 
a handy  interface  to  control  the  FPGA  hardware. 

The  complete  extension  for  our  VHDL  Online  tutorial  is  divided  into  the  following  parts 

1 . A medium  size  FPGA  is  used  as  reprogrammable  hardware  for  our  designs. 

2.  A small  digital  test  circuit  has  been  build  around  the  FPGA.  This  digital  test  circuit  allows  to  stimulate  and 
probe  all  the  pins  at  the  FPGA. 

3.  A small  8 Bit  micro  controller  is  responsible  for  the  communication  with  the  HTML  server  and  all  the  other 
low  level  programming  of  the  digital  circuit  and  the  FPGA. 

4.  A special  server  software  handles  the  communication  between  the  internet  and  the  micro  controller  on  our 
FPGA  board. 

5.  The  java  applet  on  the  user’s  internet  browser  can  now  communicate  with  the  server  software 


Conclusion 


To  improve  the  learning  performance  of  our  students,  we  developed  some  enhancements  to  the  VHDL  On- 
line tutorial.  In  this  paper  we  presented  tools  for  analogous  electrical  design  and  verification  by  modified  nodal 
analysis  that  are  fully  usable  via  the  internet.  With  these  tools  students  learn  about  simple  networks  containing 
sources,  controlled  sources  and  passive  electrical  elements.  The  other  aim  was  to  improve  the  communication  be- 
tween the  students  or  between  student  and  tutor  by  using  the  multiuser  editor.  So  learn  groups  or  designers  over 
the  network  are  possible.  The  third  application  is  real  hardware.  Students  learn  the  difficulties  of  programming  an 
existing  real  hardware  (the  FPGA)  instead  of  simulating  the  behavior  of  their  programs  on  the  computer.  This  ap- 
proach enables  the  students  to  find  some  clues  for  splitting  their  programs  into  FPGA  fitting  parts.  With  this  VHDL 
tutorial  the  student  can  understand  the  whole  VHDL  design  process  from  the  specification  to  existing  hardware.  In 
the  future  we  want  to  expand  the  developed  learning  environment  to  build  up  a system  where  students  get  infor- 
mation about  developing  integrated  circuits  (digital/analogue,  layout,  verification  etc.). 
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Abstract:  We  present  a model  and  system  for  the  cost-value  evaluation  of  teleteaching 
systems.  Basically,  the  model  considers  the  total  costs  of  a teleteaching  systems  and, 
additionally,  different  levels  of  benefits.  The  benefits  are  classified  as  either  objective,  e.g., 
transmitted  audio  and  video  quality,  or  subjective,  e.g.,  personal  feelings.  Two  cost-value 
ratios  are  computed  by,  first,  combining  the  total  costs  and  the  objective  benefits  and,  second, 
by  combining  this  result  with  the  subjective  benefits  value.  The  smaller  the  cost-value  ratio 
the  better  is  the  system.  The  model  is  implemented  by  a number  of  EXCEL®-worksheets. 


1 Introduction 

The  advanced  developments  of  communication  systems,  e.g.,  high-speed  networks  with  guaranteed  quality-of- 
service  as  in  ATM  (Asynchronous  Transfer  Mode)  networks  (H^del  & Huber  91;  de  Piycker  95),  and  audio- 
and  video  technology,  e.g.,  video  compression  schemes  such  as  JPEG  (Joint  Photographic  Expert  Group)  (ISO 
92),  have  yielded  very  powerful  digital  and  multimedia-oriented  information  and  communication  technologies 
(ICT)  (Fluckiger  95;  Steinmetz  & Nahrstedt  95).  These  new  technologies  are  influencing  most  areas  of  our  daily 
life  including  education  (Dutta-Roy  98).  In  distance  education,  these  technologies  open  the  door  to  very 
efficient  education  systems  (see,  e.g.,  the  publications  at  ED-MEDIA).  A matter  of  fact,  however,  is  that  these 
systems  are  not  for  free. 

In  this  paper  we  discuss  a model  and  system  for  the  cost-value  evaluation  of  teleteaching  systems.  We  define 
teleteaching  as  synchronous  and  interactive  distance  education:  synchronous  means  that  teachers  and  students 
meet  at  the  same  time  but  not  necessarily  at  the  same  place.  Interactive  means  that  all  participants  can  freely 
interact  to  post  questions  or  give  comments.  In  our  context,  teleteaching  takes  place  between  geographically 
distributed  classrooms.  As  a consequence  audio,  video  and  data  streams  have  to  be  transmitted  between 
classrooms.  In  support  of  this,  classrooms  are  equipped  with  devices  such  as  cameras,  microphones,  beamers, 
PCs,  coder-decoder  devices  and  communication  devices,  for  recording,  transmitting  and  displaying  multimedia 
data.  The  devices  mentioned  have  to  be  purchased,  installed,  maintained,  operated  etc.  Altogether,  the  costs  of  a 
teleteaching  system  are  serious. 

Our  work  on  a model  and  system  for  the  cost- value  evaluation  of  teleteaching  systems  was  motivated  by  the  fact 
that  since  2 /i  years  we  run  the  teieteaching  system  Teiepoiy  (Waiter  & Hanni  98),  but  its  accurate  costs  were 
unknown.  We  developed  a system  for  the  calculation  of  the  overall  costs  for  teieteaching  systems.  The  costs  can 
be  determined  in  total,  per  lecturing  hour  and  per  participating  student.  Furthermore,  different  teieteaching 
systems  can  be  compared  and  ranked  in  terms  of  their  respective  cost- value  ratios.  Basically,  for  comparing  and 
ranking  systems  the  model  considers  costs  per  hour  and,  in  addition,  different  levels  of  benefits.  The  benefits  are 
classified  as  either  objective,  e.g.,  audio  and  video  quality,  or  subjective,  e.g.,  personal  feelings. 

The  system  consists  of  a number  of  EXCEL®- worksheets.  The  worksheets  are  given  as  forms  to  be  filled  in. 
The  calculation  of  different  teieteaching  systems  is  possible  even  for  different  usage  scenarios  (in  terms  of 
number  of  classrooms,  hours  of  use  etc.). 

The  paper  is  structured  as  follows:  In  Section  2 we  give  a brief  description  of  a typical  teieteaching  scenario. 
The  diferent  types  of  costs  to  be  considered  are  discussed  in  Section  3.  Section  4 introduces  our  system  for  the 
evaluation  of  teieteaching  systems.  First,  we  explain  the  use  of  the  system  to  evaluate  a single  system.  Second, 
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we  show  with  a real  life  example  how  different  systems  can  be  compared.  In  Section  5,  we  conclude  with  a 
summary  and  an  outlook. 


2 Teleteaching  Scenario 

As  stated  in  the  introduction,  teleteaching  is  understood  as  an  interactive  and  synchronous  distance  education 
scenario.  More  specifically,  we  consider  a scenario  where  teleteaching  is  done  between  classrooms  only. 
Teachers  and  students  are  located  in  two  or  more  classrooms.  The  classroom  where  the  teacher  is  located  is 
referred  to  as  the  local  classroom  all  other  classrooms  are  remote.  Classrooms  may  be  at  universities  but  may 
also  be  at  private  organizations  and  business  companies  (Fig.  1). 


Local  Classroom 
at  University  A 


Remote  Classroom 
at  University  B 


Remote  Classroom 
at  University  C 


Remote  Classrooms 
at  Industry 


Figure  1:  Multipoint  Teleteaching  Scenario 

For  the  described  teleteaching  scenario,  the  following  multimedia  streams  are  exchanged  between  classrooms: 

• An  audio  and  a video  stream  of  the  teacher  which  is  permanently  transmitted  to  all  connected  sites. 

• An  audio  and  a video  of  the  students  in  the  local  classroom.  Note  that  the  audio  of  teacher  and  students 
are  mixed  into  one  audio  stream  which  is  sent  to  all  remote  sites. 

• A video  stream  of  a visual izer  (or  document  camera)  which  can  be  used  by  the  teacher  in  order  to 
present  material  which  is  not  available  electronically,  such  as  concrete  objects,  statistics  from  a present- 
day  Journal  etc. 

• A data  stream  is  transmitted  from  the  local  classroom  to  remote  classrooms.  This  data  stream  is  for 
teaching  aids  that  are  electronically  available.  A screen  or  application  sharing  software  is  used  for 
making  the  teaching  aids  visible  at  all  sites. 

• From  remote  sites,  an  audio  and  a video  stream  are  transmitted  to  all  other  sites. 

We  require  that  at  least  one  audio  and  one  video  stream  and,  additionally,  a data  stream  for  teaching  aids  can  be 
exchanged  between  classrooms.  Alternatively,  more  than  one  video  stream  can  be  sent  which,  however,  depends 
on  the  capabilities  of  the  system  used. 
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3 


Types  of  Costs 


In  a first  step,  we  made  a distinction  between  types  of  costs  that  are  independent  of  the  chosen  teleteaching 
system,  e.g.,  training  of  teachers,  preparation  of  electronic  teaching  aids,  general  infrastructure  in  classrooms, 
and  types  of  costs  directly  related  to  teleteaching.  The  latter  are  covered  by  our  model  and  are  discussed  below. 
We  have  adapted  an  approach  developed  in  (Gartner  98)  (for  the  total  costs  of  ownership  of  a PC). 

Equipment: 

• Audio:  microphones  for  teacher  and  students,  speaker  boxes,  audio  mixer. 

• Video:  cameras  for  teacher  and  students,  automatic  tracking  system  for  teacher  camera,  visualizer, 
LCD-beamers  for  displaying  video  (e.g.,  remote  classrooms)  and  computer  data  (e.g.,  teaching  aids). 

• Communication  infrastructure:  coder-decoder  (CODEC)  devices  for  converting  analog  signals  to  dig- 
ital ones  and  coding  for  transmission  over  a communications  network,  communication  equipment  such 
as  switches,  network  interfaces  for  switches. 

• Hardware  and  software:  PC  or  workstation  for  presentation  of  electronic  teaching  aids,  multimedia 
equipment  for  PCs  or  workstations,  network  interface  cards  for  PCs  or  workstations,  application  or 
screen  sharing  software  for  the  presentation  of  teachings  aids  locally  and  remotely,  presentation 
software  for  the  preparation  of  electronic  teaching  aids. 

Manpower: 

• Operator:  maintenance  of  the  teleteaching  system,  preparation  of  teleteaching  sessions,  installation  and 
deinstallation  of  the  teleteaching  system. 

• Teleteaching  assistance:  support  for  teachers  and  learners  while  a teleteaching  session  is  running. 
Operation: 

• Communication  interface:  basic  charge  or  connection  fee  for  access  to  a communications  network. 

• Communication  links:  time  and/or  volume  based  transmission  costs. 

The  costs  for  equipment  and  manpower  have  to  be  calculated  for  every  classroom.  This  also  applies  to  the  costs 
for  communication  interfaces.  The  costs  for  communication  links  depend  on  the  number  of  sites  interconnected, 
the  number  of  connections  that  are  used  and  the  kind  of  multipoint  communication  support  provided  by  the 
network.  If,  for  instance,  all  multimedia  streams  are  sent  from  one  site  to  all  other  sites  separately,  then  one  has 
to  set-up  one  connection  per  site.  If  multipoint  connections  are  supported  by  the  network,  then  duplication  and 
redirection  of  media  streams  is  done  in  the  network  itself  which  saves  communication  costs. 

It  turned  out  that  the  investments  in  audio  and  video  equipment  are  the  driving  cost  factors.  They  vary  and 
depend  on  whether  consumer  or  high-end  professional  audio  and  video  equipment  is  used  (some  figures  are 
given  in  Section  4.3).  These  costs  are  independent  of  the  underlying  communication  network  being  used.  If  a 
turnkey  teleteaching  system  is  purchased  the  overall  costs  are  fixed. 


4 Cost-Value  Evaluation  System 
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All  the  necessary  computations  described  below  are  programmed  into  the  worksheets  which  is  hidden  from  the 
user.  The  approach  and  the  environment  chosen  provide  the  flexibility  that  the  system  is  adaptable  and 
extensible. 


4.1  Total  Costs,  Objective  and  Subjective  Benefits 

In  a first  worksheet  the  costs  of  a teleteaching  system  according  to  the  types  of  costs  presented  in  the  previous 
section  are  collected.  From  this  worksheet  the  total  costs  of  a system  are  computed.  Furthermore,  various  other 
reference  figures  are  calculated  such  as  costs  per  hour  of  operation  and  costs  per  student.  We  use  the  costs  per 
hour  figure  in  our  cost-value  evaluation.  This  worksheet  has  to  be  filled  in  for  every  teleteaching  system. 

In  a second  worksheet  a value  for  the  objective  benefits  (of  a teleteaching  system)  is  computed.  Essentially,  we 
identified  a number  of  criteria  (referred  to  as  Service  Quality  in  Fig.  2)  which  allow  us  to  determine  the  overall 


performance  and  quality  of  a teleteaching  system.  The  criteria  include  audio  and  video  quality,  number  of  audio 
and  video  streams  that  can  be  transmitted  simultaneously,  administrative  and  operational  overhead  to  set-up  and 
run  a teleteaching  session  etc.  These  criteria  are  put  into  a check  list.  For  each  item  in  the  list,  a number  of 
options  are  pre-defined.  An  option  gives  a possible  instance  of  the  criterion.  Every  option  has  a value  defined  (1 
- 5).  The  value  reflects  the  option’s  importance,  i.e.,  counts  for  the  benefit  that  can  be  gained  if  this  option  is 
supported. 


Figure  2:  Cost- Value-Ratio  Evaluation  Process 

With  each  item  a second  value,  called  weight,  is  given  (in  the  range  of  1 - 5).  The  weight  represents  the 
importance  of  the  item  with  respect  to  all  others.  The  item‘s  value,  i.e.,  the  value  of  the  option  chosen,  is 
multiplied  with  the  item’s  weight  giving  a relative  weight  for  this  item  which  defines  the  relative  importance  of 
the  item  as  seen  by  the  user.  If  the  checklist  has  been  completed,  a single  value,  called  objective  benefits,  is 
computed  by  dividing  the  sum  of  all  relative  weights  by  the  sum  of  all  weights.  This  worksheet  has  to  be  filled 
in  once  and  will  remain  fixed  for  all  systems  to  be  evaluated. 

Similarly,  a third  worksheet  counts  for  subjective  preferences  of  users  (termed  Feelings  and  Perception  in  Fig. 
2),  i.e.,  how  they  have  perceived  a teleteaching  session  using  a specific  system.  The  form  of  the  worksheet  and 
how  it  is  processed  is  the  same  as  described  before.  At  the  end  of  the  procedure,  a single  value,  called  subjective 
benefits,  is  computed.  The  evaluation  of  this  worksheet  changes  for  different  systems. 


4.2  Evaluation  Process 

The  evaluation  process  is  done  in  three  steps  (Fig.  2).  In  a first  step,  all  worksheets  have  to  be  completed.  In 
addition  to  the  cost  summary,  a user  has  to  provide  figures  on  the  expected  usage  of  the  teleteaching  system: 
number  of  remote  classrooms,  number  of  lectures  per  week,  duration  of  each  lecture.  From  this  information  the 
costs  per  hour  are  computed.  For  an  application  with  real  world  data  for  two  different  teleteaching  systems  refer 
to  Tab.  1. 

In  the  second  step,  the  costs  per  hour  and  the  value  computed  from  the  worksheet  for  the  objective  benefits  are 
condensed  into  a single  cost-value  ratio  L This  is  done  as  follows:  The  costs  per  hour  value  is  divided  by  the 
objective  benefits  value.  The  lower  the  cost-value  ratio  the  better  is  the  system. 

If  the  computed  cost- value  ratio  1 is  almost  the  same  for  the  teleteaching  systems  evaluated,  then  the  better 
system  can  be  determined  by  applying  the  subjective  benefits  value  to  the  cost-value  ratio  1,  thus  giving  the 
cost-value  ratio  2. 
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4.3  Application  of  the  System 


As  an  application  of  the  cost-value  evaluation  system  we  compared  our  ATM-based  teleteaching  system 
Telepoly  and  an  ISDN-based  (Integrated  Services  Digital  Network)  teleteaching  system.  Telepoly  supports  the 
transmission  of  up  to  three  bidirectional  high-quality  video  and  audio  streams.  We  use  three  video  and  one  audio 
stream  for  which  we  make  an  ATM-reservation  of  38  MBit/s.  This  means  almost  12  MBit/s  for  the  transmission 
of  a Motion  JPEG  compressed  video  stream  and  2 MBit/s  for  a CD-quality  stereo  audio  stream.  Data 
transmission  for  teaching  aides  uses  the  Internet  and  is  not  included  in  the  following  cost  calculation. 

For  the  ISDN-based  videoconferencing  system  we  assume  that  three  basic  rate  interfaces  are  used  which  gives  a 
total  transmission  bandwidth  of  384  KB  it/s.  The  system  supports  the  bidirectional  transmission  of  one  video  and 
one  audio  stream  and,  additionally,  the  transmission  of  electronic  teaching  aids. 

The  cost-value  evaluation  of  the  systems  is  based  on  the  assumption  that  specific  audio  and  video  equipment  is 
purchased.  The  costs  for  audio  and  video  equipment  are  assumed  to  be  the  same  for  both  systems.  Tab.  1 
summarizes  the  cost  evaluation  for  the  systems  in  Swiss  Francs  (1  sFr  - 0.62  Euro  or  1 sFr  = 0.74  US$  at  time 
of  writing)  on  the  basis  of  an  amortization  period  for  the  equipment  of  three  years,  40  students  in  one  remote 
classroom,  six  hours  of  operation  per  day  which  gives  a total  of  840  hours  per  year  (assuming  28  weeks  of 
lectures  per  year). 

The  cost  evaluation  of  the  systems  shows  that  the  ISDN  system  has  an  advantage  over  the  Telepoly  system. 
However,  the  actual  transmissions  costs  in  MBit/s  are  less  for  ATM  (sFr.  37.63)  than  ISDN  (sFr.  90,00) 
although  the  reserved  bandwidth  is  much  (100  times)  higher  and  three  video  streams  are  transmitted 
simultaneously.  Note  that  transmission  of  one  video  stream  slightly  reduces  the  transmission  costs  only. 

The  second  and  third  step  in  the  cost-value  evaluation  of  the  teleteaching  systems  consists  in  applying  the 
objective  and  subjective  benefits  to  determine  the  cost- value  ratios  1 and  2,  respectively  (Tab.  2). 

The  ISDN  system  has  a better  cost-value  ratio  1.  Taking  into  account  the  subjective  benefits  then  the  ATM 
system  performs  slightly  better.  This  is  because  the  audio  and  video  quality  of  the  system  outperforms  the 
quality  that  can  be  achieved  with  an  ISDN  system.  Note  also  that  Telepoly  supports  transmission  of  up  to  three 
video  and  audio  streams  and  supports  multipoint  sessions  which  has  not  been  further  considered  in  the  above 
discussion. 


5 Conclusions 

We  have  presented  a model  and  system  for  the  cost-value  evaluation  of  teleteaching  systems.  The  model  was 
designed  with  two  main  objectives  in  mind.  First,  the  model  should  be  useful  in  order  to  determine  the  costs  of  a 
teleteaching  system.  For  this  we  defined  a comprehensive  list  of  types  of  costs.  Second,  the  model  should  be 
used  also  for  comparing  and  ranking  teleteaching  systems.  This  is  done  by  computing  two  cost-value  ratios.  For 
this,  objective  and  subjective  benefits  of  teleteaching  systems  are  ascertained  by  having  check  lists  to  be  filled 
in.  If  this  has  been  done,  then,  the  objective  benefits  are  considered  and  only  if  the  resulting  cost-value  ratio  1 is 
almost  the  same  then  the  subjective  benefits  value  is  used. 

The  system  has  been  implemented  as  a number  of  interrelated  Excel®-worksheets.  The  different  steps  in 
calculating  total  costs,  objective  and  subjective  benefits  and  cost- value  ratios  are  done  automatically.  The 
system  is  flexible  in  the  sense  that  changes  can  be  applied  whenener  necessary. 
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ATM  (38  M 
Total 

Bit/s)  [sFr.l 
per  hour 

ISDN  (384  1 
Total 

^bit/s)  [sFr] 
per  hour 

Equipment: 

Audio 

Video 

PC 

Control  (Touch  panel  etc.) 
Transmission  (CODEC,  switch 
etc. 

Total  costs  equipment/year 

143*507.- 

350*905.- 

8*000.- 

120*777.- 

61*988.- 

283*392.33 

271.90 

143*507.- 

350*905.- 

8*000.- 

120*777.- 

4’700.- 

209*296.33 

249.16 

Transmission: 

Basic  charge 
per  hour 
Bandwidth 

70*200.- 

83.57 

37.62 

1*917.- 

2.28 

90.00 

Manpower: 

In-  & Deinstallation/Guidance 
(h  a sFr.  100,—) 

X 1.5 

150.- 

X 1.33 

133.33 

Total  costs  per  hour 

543.09 

477.77 

Total  costs  per  student/hour 

13.58 

1 1 .86 

Table  1:  Costs  Figures  for  Telepoly-  and  ISDN-System 


Costs  per  hour 
Objective  benefits 


ATM 

sFr.  543.09 
4.24 


ISDN  system 
sFr.  474.77 
4.08 


Cost-value  ratio  1 
Subjective  benefits 


128.09 

4.51 


116.37 

4.05 


Cost-value  ratio  2 28.40  28.73 

(Assumptions:  840  hours/year,  1 remote  classroom) 


Table  2;  Cost-Value-Ratio  1 and  2 for  Telepoly  and  ISDN  system 


We  have  used  the  system  for  an  a posteriori  evaluation  of  our  own  teleteaching  system  Telepoly.  We  have  done 
a cost  analysis  to  find  out  the  actual  costs  of  a teleteaching  session.  Furthermore,  we  have  performed  an 
evaluation  of  our  system  against  an  ISDN-based  system.  Although  total  transmission  and  operation  costs  are 
higher  for  Telepoly  than  for  the  other,  the  cost- value  ratios  of  Telepoly  are  almost  equally  well.  The  cost-value 
evaluation  system  is  still  under  development.  In  particular,  the  information  on  the  objective  and  subjective 
benefits  needs  to  be  further  justified. 
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Abstract;  In  this  paper  the  object  oriented  design  is  combined  with  the  hypertext  mode 
to  achieve  an  efficient  system’s  structure  which  help  the  learner  to  navigate  freely  and 
stay  in  a certain  context.  Within  this  novel  framework,  tools  and  approaches  can  be 
shared  or  used  while  designing  educational  systems  for  different  topics  or  didactic 
methodologies.  Furthermore,  the  parallel  design  of  educational  systems  for  two  different 
topics,  history  and  physics,  using  the  combined  formation  is  presented.  We  have 
investigated  the  dependencies  of  objects  and  the  sharing  of  tools  used  during  the 
development  of  the  two  prototypes.  Both  systems  are  engaging,  expandable  and 
transferrable. 


Introduction 

The  user  interface  design  and  the  content’s  exploration  are  key  issues,  especially  in  educational 
applications.  The  interface  is  required  to  engage  the  user  and  offer  aesthetic  information  presentation.  The 
communication  and  interaction  for  the  content’s  exploration  should  lead  to  knowledge  aquisition.  Adaptation  to 
learner’s  selections  and  flexibility  for  different  scenarios’  execution  are  highly  recommended.  The 
disorientation  that  is  created  by  the  adaptation  and  the  flexibility,  should  be  eliminated  (Shneiderman  1992), 
(Metaxaki  et  al.  1991a),  (Kouroupetroglou  et  al.  1995). 

The  design,  implementation  and  content  development  in  multimedia  systems  are  quite  expensive.  Thus, 
resuability  of  the  content  and/or  a sucescfull  former  design  approach  is  a requirement.  In  some  applications,  the 
re-use  of  a piece  of  content  in  the  same  or  different  format  realizes  educational  aspects  or  didactic  procedures 
(Metaxaki  et  al.  1991b),  (Metaxaki  et  al.  1994). 

In  this  paper,  we  present  a novel  approach  to  meet  the  mentioned  needs.  The  structure  of  the  proposed 
system  is  a combination  of  the  hypertext  mode  and  the  object  oriented  architecture,  implemented  by  embedded 
objects  or  components.  There  can  be  a design  approach  includiiig  objecis  descriptions  which  can  be  enricned  by 
operation  functions  to  face  various  needs.  These  functions  can  be  interchanged  by  learner’s  selection  to  help 
him  navigate  freely  without  losing  the  context  of  the  exploration  (Rumbaugh  et  al.  1991),  (Fayad  and  Schmidt 
1997),  (Aedo  et  al.  1996),  (Wisnudel  1994). 

Two  applications  using  the  same  methodology  have  been  desogned  and  developed  to  explore  the 
dependence  and  the  transferability  of  the  object’s  description  and  presentation.  The  sharing  of  tools  was 
investigated,  too. 


Aspects  of  the  Combined  Methodology 

The  extensively  used  hypertext  mode  has  two  main  problems:  the  increasing  number  of  connections  over 
a topic  and  the  disorientation  these  connections  can  create  after  some  navigation  steps.  A novel  method  is 
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required  which  could  add  connections  of  data  and  tools  to  eliminate  this  disorientation.  The  object  oriented 
architecture  can  be  used  for  this  purpose. 

Our  decision  was  to  design,  implement  and  investigate  according  to  the  Component  Display  Theory 
(CDT)  defined  by  Merill  as:  “The  CDT  is  comprised  of  three  parts:  a two-dimensional  performance-content 
classification  system,  a taxonomy  of  presentation  forms,  and  a set  of  prescriptions  relating  the  classification 
system  to  the  presentation  forms.  CDT  assumes  that  all  instructional  presentations  are  comprised  of  a series  of 
discrete  displays  or  presentation  forms.  Any  presentation  can  be  described  as  a sequence  of  such  presentation 
forms,  together  with  the  interelationships  between  these  forms”  (Merill  1984),  (Merill  1983). 

A main  notion  of  the  embedded  component  (EC)  structure  is  the  independence  between  the  objects  used 
for  presentation  and  the  ones  used  for  the  information  presented.  This  allows  the  appearance  of  the  same 
information  in  multiple  locations  and  in  different  forms.  Moreover,  new  objects  can  be  created  in  succession, 
the  previously  constructed  ones  can  be  re-used  and  the  new  ones  are  easily  integrated  in  the  system.  An  object 
can  be  plain  or  enriched,  active  (sound,  video,  motion)  or  inactive  (plain  text).  Objects  can  be  concrete  or 
embedded  on  other  objects.  The  objects  can  be  interconnected  through  links  pre-existing  or  created  by  user 
queries.  Using  the  embedded  objects  we  can  succeed  in  the  integration  of  information,  the  integration  of  media, 
the  re-use  of  information  and  the  flexibility  of  the  presentation  of  information  on  a display  (Spitzer  1 997), 
(Baecker  et  al.  1995),  (De  Diana  et  al.  1994). 


The  Design  Approach 
System’s  Objects  and  Modules 
The  Objects 

The  objects  are  differentiated  as  data  objects  (dao)  and  display  objects  (dio).  The  dao  are  enfolding  the 
data,  the  implementation  of  the  possible  links,  and  the  execution  of  queries.  They  are  of  two  different  types,  the 
main  and  the  instance  data  objects  (mdao,  idao).  The  mdao  include  the  required  information  for  the  definition 
of  an  entity.  The  content  of  the  dao  can  be  a maximum  or  minimum  depending  on  the  topic  or  the  didactic 
procedure  i.e.  mdao  for  the  lesson  on  physics  include  the  maximum  content  for  a physical  phaenomenon  or  a 
minimum  to  be  further  combined.  The  ones  for  the  lesson  of  history  are  usually  of  a minimum  content  and  are 
combined  to  form  a historical  event.  Each  mdao  is  connected  with  one  or  more  idao,  containing  data  and 
format  information. 

The  dio  include  all  parameters  needed  for  the  formation  of  a presentation  on  the  display.  Their 
parameters  depend  mainly  on  the  type  and  format  of  information.  They  contain,  too,  the  interactivity  functions. 
The  execution  of  dio  performs  the  dao's  presentation.  We  emphasize  that  the  dio  do  not  define  the 
interconnections  of  this  presentation. 

We  can  say  that  the  system  communication  is  organized  in  two  layers.  The  presentation  layer  and  the 
data  layer.  The  user  interacts  with  the  presentation  layer.  At  this  layer  the  learner’s  requests  and  actions  are 
interpreted  and  transferred  to  the  data  layer.  The  data  layer  processes  the  interpreted  requests  and  a 
presentation  appears  on  the  display. 


The  Modules 

We  can  assume  that  the  system  consists  of  three  types  of  modules  (Fig.  1):  the  Objects  Module  (OM),  the 
Navigation  Tools  Module  (NTM)  and  the  Main  Pages  Module  (MPM).  The  Objects  Module  contains  all  the 
objects  of  the  system  we  have  already  described.  The  NTM  supports  the  selected  navigation  and  contains  all  the 
constantly  presented  elements  of  the  display  (chapters,  menus  etc).  The  MPM  includes  the  main  thematic  pages 
of  the  system.  These  pages  are  the  starting  point  for  the  user’s  exploration.  They  form  the  base  for  the 
orientation  and  the  structuring  of  user’s  reactions  and  the  selected  objects’  presentation.  The  pages  are 
intercommunicating. 

Both  the  hypertext  navigation  mode  and  the  embedded  object  architecture  co-exist  in  the  system.  The 
hypertext  based  interaction  is  realised  in  the  communication  between  the  MPM  and  the  NTM.  The  objects- 
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based  interaction  is  realised  in:  (a)  the  communication  inter-MPM,  between  the  MPM  and  the  OM,  (b)  the 
intercommunication  of  the  OM. 


Figure  1 : The  General  Design  Approach 

The  combination  of  the  two  modes  and  the  creation  of  the  modules  content  create  a frame  in  which 
various  didactic  procedures  can  be  realized.  The  system  can  be  adapted  to  the  learner’s  needs. 


Structure  of  the  System  and  Applications 

In  order  to  design,  structure  the  system  and  investigate  the  methodology  we  have  decided  to  prototype  in 
parallel  on  two  different  topics  with  which  we  had  already  a good  familiarity  (physics,  history).  The  two  topics 
concerning  their  implementation  were  differentiated  by  the  type  of  exploration  depth-width  and  the  format  of 
the  presented  information.  The  type  of  the  exercises,  too,  were  different.  The  base  of  reference  was  to  keep 
elements  of  an  electronic  hypermedia  book  (Laurel  et  al.  1990),  (Thuering  et  al.  1995),  (Mullin  1990), 
(Metaxaki  et  al.  1995). 

Both  systems  include  links  (for  direct  or  nonsequential  access),  embedded  objects,  navigation  bars  or 
menus,  wedges  (areas  offering  additional  information  by  direct  access),  floating  windows  (wedges  enriched  by 
functional  characteristics),  indexes  and  recapitulatory  nodes. 

In  Fig. 2 we  present  the  structure  of  the  applications.  The  structure  is  common  although  the  outlook  is 
different.  There  exist  on  the  display  the  constantly  shown  fields  and  the  changing  ones.  The  constantly  shown 
ones  (menu,  navigation  bar,  chapters  etc)  assist  the  user’s  orientation  in  a context.  The  user  selects  (links, 
buttons)  a sub-topic,  moves  directly  to  it  (main  pages)  and  starts  the  exploration. 

The  exploration  can  be  continued  in  depth  or  width  by  queries,  indexes  etc.  The  one  in  depth  will  occur 
by  selection  (links,  hotspot  areas  etc)  and  the  presentation  of  embedded  objects  in  wedges  or  floating  windows 
format.  The  other  in  width  will  occur  by  direct  returns  to  the  main  pages  and/or  the  extensive  formation  of 
interconnecting  embedded  objects.  Windows  or  wedges  can  be  opened  concurrently  or  in  sequence,  overlayed  or 
not. 


Discussion  and  Results 
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The  design  of  the  two  systems  in  parallel  was  proved  quite  succesful  and  spared  development  time.  The  topic  of 
the  nuclear  physics  needed  a large  amount  of  simulated  and  animated  information.  The  topic  of  history  was 
about  the  Emperor  Carolus  Magnus  and  needed  texts  and  the  reproduction  of  images.  Our  great  difficulty  was 
the  presentation  of  the  objects,  which  was  developed  in  a parametric  way  so  that  we  could  share  programming 
tools.  The  definition  of  the  changing  areas  and  the  floating  windows  outlook  as  well  as  the  connections  of  texts 
and  pictures  was  scheduled  in  our  system  for  the  pictorial  communication  research  (Metaxaki  et  al.  1996), 
(Kouroupetroglou  et  al.  1994).  We  can  conclude  that  the  information  content  was  developed  independently  but 
for  the  presentation  development  we  have  used  the  same  parametric  tools.  The  object  definitions  were  quite 
short  for  the  history  in  comparison  with  the  ones  for  physics  but  they  contained  more  data.  The  inteconnections 
between  objects  were  less  extensive  for  nuclear  physics.  This  means  that  the  objects  were  more  concrete.  The  in 
depth  exploration  of  the  topic  was  achievable  by  the  system  as  well  as  the  in  width  exploration  for  history. 


Figure  2:  The  common  structure  and  communication  for  both  prototypes 

We  work  further  using  the  same  methodology  and  transferring  tools  and  experience  to  the  topic  of 
human  sciences. 


Conclusions 

The  use  of  object  oriented  design  in  connection  with  the  hypermedia  mode  to  design  educational  systems 
is  a quite  successful  approach.  It  helps  the  learner  to  navigate  through  the  system  feeling  free  but  staying  in  the 
context.  The  learner  can  choose  his  own  way  to  explore  the  system.  The  teacher  or  tutor  can  realize  different 
scenarios  taken  advantage  of  the  system’s  flexibility.  The  system  can  be  explored  in  depth  or  width  or  both 
sequentially  depcndiiig  On  the  learning  process. 

The  implementation  of  the  approach  needs  expert  programmers  but  the  system  is  expandable  and 
includes  a large  amount  of  transferability.  When  the  system’s  content  is  existing  in  an  intra-net  server  this  type 
of  design  is  much  more  powerful. 
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Abstract:  This  paper  discusses  a goal-oriented  project  called  Distance  Learning  in  Multimedia 
Networks  (ETAKAMU)  <URL:  http://matwww.ee.tut.fi/kamu/>  that  is  a part  of  the  Finnish 
Multimedia  Programme  (FMP).  The  project  started  on  the  1st  of  February  1996  and  lasted  until 
January  1999.  The  project  combined  the  efforts  of  Finnish  telecommunication  companies,  content 
providers,  publishing  houses,  hardware  companies  and  educational  institutions  in  the  field  of 
distance  learning.  One  of  the  main  goals  of  the  project  was  to  develop  and  pilot  pedagogically 
appropriate  and  technically  functional  open  learning  environments.  In  this  paper  we  present  the 
main  results  of  the  project. 


Introduction 

The  Distance  Learning  in  Multimedia  Networks  Project  <URL:  http://matwww.ee.tut.fi/kamu/>  combined 
Finnish  technical  and  pedagogical  research  centres  and  institutions  in  the  field  of  distance  learning,  hardware  and 
software  producers,  publishing  houses  and  telecommunication  companies  in  an  extensive  joint  project.  In  1998  the 
budget  of  the  project  was  approximately  4,7  million  Finnish  marks  (900  000  USD).  The  project  was  funded  jointly 
by  the  Technology  Development  Centre  of  Finland  (TEKES)  and  by  32  Finnish  enterprises.  It  was  coordinated  by 
the  Hypermedia  Laboratory  of  the  Digital  Media  Institute  at  Tampere  University  of  Technology.  (Pohjolainen  & 
Ruokamo  1998.) 

The  aim  of  the  Distance  Learning  in  Multimedia  Networks  Project  was  to  research  and  develop 
pedagogically  appropriate  and  technically  functional  open  learning  environments  for  distance  learning  based  on 
computer  networks.  The  design  of  the  learning  environments  is  based  on  the  rapid  development  of  information 
technology,  especially  on  broadband  computer  networks  and  hypermedia-based  learning  materials.  The  learning 
environments  were  studied  and  developed  from  telecommunications  (hardware),  hypermedia  material  (courseware) 
(Nykanen  & Ala-Rantala  1998,  Pohjolainen,  Ala-Rantala,  Nykanen  & Ruokamo  1999)  and  pedagogical  points  of 
view  (Ruokamo  & Pohjolainen  1998). 

Distance  learning  using  multimedia  is  exploited  by  schools,  educational  institutions,  and  enterprises  for 
all  age.s  ranging  from  children  to  adult  learners.  Various  pilot  projects  and  teaching  experiments  was  set  up  in  ten 
pilot  areas  to  gain  feedback  from  the  methods  and  solutions  developed.  The  purpose  of  the  teaching  experiments 
was  to  test  both  the  technical  and  pedagogical  functionality  of  the  distance  learning  solutions  developed. 


Collaborating  Partners 

The  Distance  Learning  in  Multimedia  Networks  Project  was  divided  between  research  and  enterprise 
partners.  The  universities  were  the  research  partners  in  the  project.  The  Digital  Media  Institute  (DMI)  operating  at 
Tampere  University  of  Technology  was  the  project  coordinator  and  the  other  units  functioned  as  subcontractors  to 
it.  Research  partners  were 

• Tampere  University  of  Technology  / DMI  / Hypermedia  Laboratory, 

• Helsinki  University  of  Technology  / Centre  for  Educational  Technologies, 

• University  of  Tampere  / Hypermedia  Laboratory, 

• University  of  Tampere  / Institute  for  Extension  Studies, 


958 


• University  of  Jyvaskyla  / Information  Technology  Research  Institute, 

• University  of  Helsinki  / Lahti  Research  and  Training  Centre. 

The  project's  enterprise  partners  were  divided  into  steering  enterprises  and  follow-up  enterprises.  Steering 
enterprises  were  as  follows:  City  of  Jyvaskyla,  City  of  Tampere,  Data  Network  Institute,  Edita  Group,  Finnet/HPY 
Ltd.,  Helsinki  Institute  of  Physics,  Helsinki  Media  Company  Ltd.,  Jyvaskyla  Science  Park  Ltd.,  Lifelong  Learning 
Institute  Dipoli/Helsinki  University  of  Technology,  Nokia  Mobile  Phones,  Otava  Ltd.,  Sonera  Ltd.,  Sun 
Microsystems  Ltd.,  Tampere  Technology  Centre  Hermia,  WSOY  Ltd.,  and  Xenex  Ltd.  ETAKAMU’s  follow-up 
enterprises  were  AAC-Institute,  Centre  for  Technical  Training  AEL,  City  of  Nokia,  CSC  - The  Center  for 
Scientific  Computing,  Finnet/TPO  Ltd.,  Finnish  Association  of  Adult  Education  Centres  KTOL,  KVS-Institute, 
Mantta  Regional  Education  Center,  Promanet  Ltd.,  Promentor  Solutions  Ltd.,  STAKES  (National  Research  and 
Development  Centre  for  Welfare  and  Health),  Teleste  Educational  Ltd.,  The  Folk  High  School  for  Finns  in 
Sweden,  The  Subregion  of  South-East  Central  Finland,  Valmet  Ltd.,  and  WM-data  Kasanen. 


The  Project’s  Organization  and  Research  Activities 

The  organization  of  the  Distance  Learning  in  Multimedia  Networks  Project  was  divided  into  a steering 
committee,  a supporting  group,  research  groups  and  pilot  areas.  The  steering  committee  decided  on  general  plans 
and  coordinated  the  activities  between  industry  and  academia.  The  supporting  group  was  composed  of  all  the 
researchers  of  the  project  and  staff  from  research  units  that  were  responsible  for  the  project.  The  research  activities 
of  the  ETAKAMU  project  were  divided  into  two  parts,  i.e.  into  research  conducted  by  research  groups  whose 
objective  was  to  produce  knowledge  in  support  of  project  activities,  function  as  a support  for  pilots,  organize 
necessary  training,  and  analyze  experiences  gained  from  the  pilots.  The  three  research  groups  were  concentrating 
on:  development  of  a prototype  of  an  open  learning  environment,  development  of  criteria  for  technical  functionality 
and  structure  for  hypermedia-based  materials,  and  development  of  methods  for  distance  learning  and  pedagogical 
evaluation  criteria  for  open  learning  environments. 

In  addition  to  the  three  research  groups  several  pilots  were  set  up  in  various  areas  via  which  distance 
learning  solutions  in  multimedia  networks  were  carried  out  in  practice  and  feedback  on  the  performance  of 
technical  and  pedagogical  solutions  was  obtained.  The  participants  in  the  pilots  were  project  researchers  and  staff 
of  enterprises,  and  teachers  and  students  of  educational  institutions.  The  pilots  were  divided  into  the  following  pilot 
areas:  1.  Broadband  networks  and  technical  experiments,  2.  Natural  sciences,  3.  Language  learning,  4.  Education, 
5.  Communication,  6.  In-house  education  for  enterprises,  7.  History  and  civics,  8.  New  school  learning  materials 
(KUOMA),  9.  Finnish  children  abroad,  and  10.  Web  university  (WU)  (see  Rinta-Filppula  1998). 

During  the  three  years  project  expanded  in  many  ways.  Participants  included  pupils,  university  students, 
adult  learners  and  employees  of  enterprises  - the  idea  of  lifelong  learning  was  adopted.  Distance  learning  was  seen 
not  only  as  a matter  for  educational  institutions  but  also  enterprises  were  interested  in  utilizing  it.  Distance 
Learning  in  Multimedia  Networks  Project  grew  into  a significant  co-operative  effort  between  research  institutes, 
educational  institutions  and  enterprises. 

During  the  last  year  of  the  ETAKAMU  project  the  main  emphasis  was  on  analyzing  and  reporting  the  final 
results  of  the  project.  The  project’s  development  and  research  work  during  the  third  year  was  divided  as  follows: 

• Collecting  the  data  from  a number  of  teaching  experiments  that  were  arranged  during  spring  and  autumn 

1998. 

• Analyzing  of  the  results  were  done  using  pedagogical  and  technical  evaluation  criteria. 

• Development  of  the  prototype  of  an  open  learning  environment  has  continued  until  the  end  of  January 

1999. 

• Project’s  final  results  have  been  analyzed  and  will  be  published  as  a book  (Ruokamo  & Pohjolainen  1999) 

and  on  the  web. 

• In  January  1999  the  project  arranged  a closing  seminar  ’’Pedagogical  Methods  and  Technical  Solutions  for 

Distance  Learning  IV”. 
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Pedagogical  Research  Problems 


The  ETAKAMU  project  applied  seven  important  qualities  of  meaningful  learning  - based  on  the  list 
presented  by  Jonassen  (1995)  - as  guidelines  in  the  design,  implementation,  and  evaluation  of  the  open  learning 
environments  (see  Pohjolainen  & Ruokamo  1998;  Ruokamo  & Pohjolainen  1998,  1999).  On  the  basis  of  the 
qualities  of  meaningful  learning,  the  following  research  problems  for  the  pedagogical  evaluation  of  learning 
environments  were  set: 

1.  Does  the  learning  environment  support  learner-centered  learning? 

1.1.  Does  the  learning  environment  support  learners’  active  construction  of  new  knowledge  on  the  basis  of 
prior  knowledge  in  interaction  with  the  surrounding  reality  (Constructivity)? 

1.2.  Is  the  learner’s  role  active  and  are  the  learners  responsible  actors  in  the  learning  environment 
(Activity)? 

1.3.  Can  learners  work  together  and  construct  new  knowledge  with  one  another  while  utilizing  each  other’s 
skills  (Collaborativity)? 

1.4.  Has  the  learning  environment  taken  due  heed  of  learners’  goals  and  does  it  support  the  achievement  of 
their  knowledge  objectives  (Intentionality)? 

1.5.  Are  the  learning  tasks  located  in  meaningful,  real-world  tasks  from  the  perspective  of  the  learner  or  is 
there  simulation  of  this  through  certain  case-based  and  problem-based  real-life  examples  (Contextuality)? 

1.6.  Can  learners  transfer  what  they  have  learnt  from  that  situation  and  context  and  apply  that  knowledge  in 
other  situations?  Can  learners  utilize  formerly  learned  knowledge  and  skills  in  the  learning  of  new  things? 
(Transferability)? 

1.7.  Is  it  possible  for  learners  to  articulate  what  the  have  learned  and  reflect  on  the  thought  processes 
entailed  in  the  learning  process  and  the  decisions  taken  (Reflectivity)? 

The  foregoing  pedagogical  research  problems  served  as  the  basis  for  the  evaluation  framework  for  the 
pedagogical  evaluation  of  the  learning  environments.  This  is  intended  for  the  scrutiny  of  learning  environments 
and  learning  from  the  learner-centered  perspective,  with  consideration  for  the  perspectives  of  both  the  learner  and 
the  teacher. 


Main  Results 

Pilot  projects  in  various  fields  constituted  an  important  part  of  the  ETAKAMU  activity.  It  was  the  aim  of 
pilot  work  to  try  out  the  methods  developed  and  elicit  feedback  on  the  technical  and  pedagogical  functionality  of 
the  systems.  In  the  field  of  piloting  the  focus  was  on  the  further  development  of  the  learning  environments,  on  data 
collection  and  evaluation  work.  Pilot  studies  were  implemented  in  ten  fields  in  over  twenty  pilot  projects.  The 
following  presents  an  overview  of  the  main  results  from  certain  pilot  projects. 

A&O  Open  Learning  Environment.  The  A&O  learning  environment  is  based  on  a space  model  that  is 
divided  into  the  following  spaces:  1)  Office,  via  which  all  administrative  matters  related  to  studies  are  carried  out, 
2)  Study  for  students,  teachers  and  other  actors,  3)  Media  Center,  where  all  the  additional  and  reference  materials 
are  located  and  4)  Gallery  where  all  the  final  works  of  the  students  can  be  set  up  to  be  available  for  all  the  Internet 
users.  The  system  architecture  of  A&O  consists  of  data  management  facilities  including  data  acquisition  tools, 
tools  for  communication  and  collaboration,  course-dependent  cognitive  tools  and  authoring  tools  for  producing 
hypermedia  courseware.  Since  A&O  is  now  just  a research  and  development  version,  the  authors  have  not  yet  been 
able  to  test  the  A&O  learning  environment  in  teaching  experiments  and  user  trials.  These  will  be  arranged  in  the 
future,  (see  Pohjolainen,  Ruokamo  & Nykanen  1999.) 

Network  PC.  The  pilot  project  Java  in  distance  learning  was  implemented  in  the  DM  I Hypermedia 
Laboratory.  It  comprised  TUT  and  three  Tampere  schools  on  the  ATM  network.  Kaukajarvi  School,  Tampere 
Teacher  Training  School  and  Pyynikki  School  and  also  the  joint  network  computer  experimental  environment,  of 
Ylojarvi  Upper  Secondary  School.  JavaStation  was  used  to  carry  out  teaching  experiments  in  which  it  was  noted 
that  JavaStation  is  well-suited  to  use  in  educational  institutions,  (see  Hakkinen  in  Ruokamo  & Pohjolainen  1999.) 

Pythagoras.  The  Pythagoras  environment  for  statistics  <URL:  http://matriisi.ee.tut.fi/pythagoras/>  is 
targeted  at  pupils  of  the  secondary  and  upper  secondary  school.  Experiments  were  carried  out  in  numerous  schools 
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in  the  Tampere  and  Ylojarvi  areas.  The  outcomes  of  the  experiments  show  that  Pythagoras  is  viable  as  a learning 
environment  for  the  study  of  statistics.  The  questionnaire-tool  in  Pythagoras  was  felt  to  be  particularly  good  in  that 
it  enables  users  to  publish  their  own  questions  on  subjects  which  interest  them  on  the  web  and  also  to  collect 
answers  from  other  users  of  the  environment.  It  was  possible  for  users  to  utilise  the  data  they  had  themselves 
collected  for  subsequent  project  work,  (see  Maenpaa  in  Ruokamo  & Pohjolainen  1999.) 

New  learning  materials  for  schools  (The  KVOMA  Pilot),  The  KUOMA  learning  environment  and 
Environment  Net  <URL:  http://matwww.ee.tut,fi/ymparistoverkko>  were  developed  from  the  pilot  project  for  the 
study  of  environmental  subjects.  EnvironmentNet  was  tried  out  in  1998  in  Pyynikki  School  and  in  Ylojarvi  Upper 
Secondary  School.  The  experiences  gained  from  the  experiment  indicate  that  EnvironmentNet  provides  good 
opportunities  for  learner-centred  learning.  In  order  that  optimal  advantage  be  derived  from  these,  more  technical 
and  pedagogical  education  would  be  necessary  for  teachers  in  using  EnvironmentNet.  Pupils,  too,  would  require 
more  orientation  before  beginning  to  study  through  EnvironmentNet.  (Piiksi  & Muhonen  in  Ruokamo  & 
Pohjolainen  1999;  Piiksi  1999.)  On  the  same  KUOMA  pilot  project  the  learning  environment  for  elementary 
Russian  was  developed  called  Russian  on  the  Net  <URL:  http://matwww.ee.tut.fi/venaia>.  Russian  on  the  Net  was 
tried  out  in  the  Tampere  Teacher  Training  School  and  Ylojarvi  Upper  Secondary  School.  Results  indicate  that 
Russian  on  the  Net  brought  variety  and  interest  to  the  study  of  Russian,  for  teachers  and  pupils  alike.  Pupils 
reported  that  the  environment  was  useful  for  learning,  but  there  was  still  room  for  improvement.  (Hamalainen  & 
Muhonen  in  Ruokamo  & Pohjolainen  1999;  Hamalainen  1999.) 

Web  University  (WU),  On  ETAKAMU  the  international  pilot  Web  University  (WU)  was  developed.  Web 
University  pilot  <URL:  http://wwwcs.ccm.ch/>  operated  as  a virtual  university  through  which  it  was  possible  to 
participate  in  real  time  and  interactively  in  lectures  of  the  Academic  Training  Programme  by  CERN  and  in  CERN 
Colloquium  seminars.  International  interest,  and  the  further  development  and  expansion  of  the  pilot  have  shown 
that  transmitting  lectures  by  top  experts  is  appropriate  using  broadband  network-based,  videoconferencing 
technique,  and  multimedia  services.  Distance  learners  have  participated  for  a total  of  over  100  hours  in  two  years 
in  contact  teaching  realised  through  WU-mediated  CERN  contact  teaching.  Transmissions  were  in  real  time  and 
were  interactive.  However,  they  were  also  recorded  digitally  on  FUNET  multimedia  server  <URL: 
htto://mbase.funet.fi/>.  (Rinta-Filppula  1998;  Rinta-Filppula  & Penttila  in  Ruokamo  & Pohjolainen  1999.) 


Conclusions 

The  ETAKAMU  project  developed  many  different  learning  environments  for  different  target  groups  and 
learning  contents;  it  also  conducted  a considerable  number  of  teaching  experiments.  Most  of  the  learning 
environments  were  available  on  the  Internet.  They  utilized  WWW  hypermedia  properties,  the  opportunities 
provided  for  situations  and  simulations.  Communication  and  co-operation  tools  were  also  built  into  the 
environments,  likewise  cognitive  and  materials  production  tools.  The  activities  and  the  experimental  use  made  of 
the  learning  environments  were  successful  and  demonstrated  the  use  and  importance  of  technology  in  education.  It 
was  seen  that  technology  can  be  used  to  aid  the  construction  of  pedagogically  relevant  environments  and  tools 
which  support  learning. 

The  rapid  development  in  technology  has  been  accompanied  by  certain  problems.  With  the  rapid 
development  in  browsers,  plug-ins,  script  and  programming  languages  and  applications  the  computer  network 
environment  is  not  yet  stable  from  the  technical  point  of  view.  In  consequence  of  this  the  existing  learning 
environments  must  be  constantly  updated  so  that  their  operationality  may  be  guaranteed.  This  problem  may  cause 
less  trouble  in  the  future,  when  the  development  in  some  respects  reaches  a plateau  and  commercial  software 
dominates  the  markets. 

Although  hypermedia  enables  the  construction  and  adaptability  of  documents  there  are  as  yet  no 
systematic  models  of  the  structure  of  didactically  appropriate  learning  material  to  be  used  more  extensively. 
Standardization  of  the  document  structure  would  enable  the  use  of  content  in  different  environments  and  facilitate 
both  flexible  combination  of  contents  and  information  seeking.  Material  generated  by  both  materials  producers  and 
learners  constitutes  an  important  part  of  the  learning  environment.  Materials  production  would  increase 
considerably  if  producers  had  at  their  disposal  a user-friendly,  structured  tools  for  the  production  of  multimedia 
material. 

One  obstacle  to  the  expansion  of  the  use  of  learning  environments  is  the  problems  with  use.  Such 
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problems  are  manifest  in  difficulty  in  use  and  the  need  for  maintenance.  For  all  actors  a more  user-friendly  result 
could  be  achieved  through  better  user  connections  and  the  integration  of  all  necessary  tools  into  the  same  learning 
environment. 

In  distance  learning  achieving  objectives  entails  not  only  sophisticated  technology  but  also  appropriate 
pedagogy  and  contents  - these  must  serve  the  same  end  of  learner-centered  learning.  The  new  technology  also 
provides  scope  for  pedagogical  progress  and  for  the  realization  of  pedagogical  innovations.  Efficient  application  of 
the  existing  technological  options  requires  pedagogical  research.  An  optimally  efficient  and  meaningful  learning 
environment  commits  the  learners  to  active  construction  of  knowledge  instead  of  reproducing  this.  It  further 
commits  learners  to  discussion,  interaction,  co-operation  and  reflection  of  the  parties  in  the  learning  process  as 
opposed  to  the  one-sided  reception  of  knowledge  or  replication.  The  content  of  the  learning  environment  should 
engage  learners  in  meaningful  contexts  and  situations  in  which  it  is  possible  to  utilise  knowledge  and  skills  already 
acquired  and  to  transfer  these  to  further  situations. 

The  pedagogical  evaluation  framework  developed  in  the  ETAKAMU  project  proved  functional  in  the 
evaluation  of  learning  environments.  It  is  also  applicable  in  the  planning  and  implementation  of  learning 
environments  and  in  supporting  learning.  Of  the  seven  qualities  of  learner-centred  learning  it  was  constructivity, 
intentionality  and  reflectivity  which  were  found  to  be  best  realised  in  learning  environments  enabling  project  work. 
The  learning  environments  were  also  found  to  enable  contextuality  and  transferability.  Activity  and  collaborativity 
were  seen  to  pose  challenges  in  many  environments:  what  turned  out  to  be  problematic  was  how  to  get  learners 
motivated  and  committed  to  the  learning  task  and  also  to  make  active  use  each  other’s  skills  and  knowledge. 

The  technologies  available  today  enable  totally  new  learning  processes.  In  addition  to  the  participation  of 
traditional  actors  (teachers,  educational  institutions  etc.)  there  are  many  kinds  of  new  actors  and  producers  of 
services  such  as  teleoperators,  producers  of  technical  support  services,  producers  of  net  materials,  etc.  The  efficient 
and  pedagogically  sound  use  of  learning  environments  requires  overall  control  of  the  learning  process.  The 
dissemination  of  educational  technology  services  developed  in  pilots  and  scaling  them  for  more  widespread  use  is  a 
worldwide  problem.  The  efficient  dissemination  of  educational  technology,  contents  and  services  requires  that 
common  rules  of  play  be  agreed  on.  This  could  be  achieved  by  the  dissemination  of  best  practices  and  flexible 
standardising. 

The  efficient  use  of  information  technology  (IT)  in  teaching  in  Finland  is  still  at  the  experimental  and 
pilot  stages.  Good  experiments  have  been  conducted  but  they  need  support  in  order  to  be  realised.  The  risk  exists 
that  after  experimentation,  once  the  technical  and  pedagogical  support  personnel  have  departed,  the  environments 
used  during  the  experiment  will  not  continue  to  be  part  of  the  everyday  practice  of  the  educational  institution  or 
enterprise.  The  environments  used  in  the  experiments  were  neither  technically  nor  pedagogically  perfected 
products.  There  is  a need  for  more  user-friendly  technologies  to  support  learning,  likewise  appropriate  content  and 
more  widely  applicable  models.  Introducing  and  using  learning  environments  should  be  easy,  and  not  require  great 
effort.  Individual  experiments  should  progress  to  more  extensive  nation  wide  trials  and  action  models. 

At  schools  resources  should  be  invested  in  basic  and  further  education  for  teachers  so  that  they  acquire  the 
necessary  competence  to  use  take  pedagogical  and  technical  advantage  of  the  learning  environments  and  derive 
benefit  from  them.  Those  responsible  for  education  in  enterprises  should  also  receive  training  through  courses 
tailor  made  to  suit  their  needs.  At  the  present  time  the  exploitation  of  IT  in  educational  institutions  is  the 
responsibility  of  too  small  a number  of  active  teachers,  Maintenance  of  the  schools’  computer  classes  has  proved 
problematic  as  there  is  a discrepancy  between  the  time  needed  for  this  kind  of  work  and  the  remuneration.  Society 
should  contribute  adequately  to  improving  the  circumstances  under  which  the  new  technology  operates  so  that  it 
can  be  introduced  more  rapidly. 

The  major  developmental  trends  in  education  (see  Carley  & Dailey  1 998)  are:  1 ) sophisticated  technology 
will  play  a greater  role  in  education,  2)  the  role  of  commercialisation  will  increase,  3)  the  demands  of  lifelong 
learning  and  for  educational  services  will  be  greater  and  4)  there  will  be  redevelopment  and  reorganisation  of 
education  systems.  The  work  on  the  research,  development  and  evaluation  of  distance  learning  on  the  multimedia 
networks  has  served  to  promote  these  general  trends,  with  a particular  advance  in  the  combination  of  technology 
and  education.  The  increased  demand  for  varied  educational  services  has  come  strongly  to  the  fore.  With  the  help 
of  learning  environments  which  are  independent  of  time  and  place  it  is  possible  to  break  out  of  limitations  imposed 
by  the  classroom  and  individual  educational  institutions  an  offer  new  opportunities  for  training  for  companies  and 
adults.  Making  these  new  opportunities  a reality  requires  that  in  the  long  term  the  educational  systems  will  be 
reformed.  Due  to  the  nature  of  the  research  and  development  project  the  role  of  commercial  considerations  was  not 
accorded  so  much  attention  in  the  present  research.  Nevertheless  the  research  findings  of  the  ETAKAMU  project 
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give  good  cause  to  make  the  transition  from  the  phase  of  research  and  experimentation  to  product  development 
which  will  be  significant  for  international  markets. 

The  activity  of  the  project  on  distance  learning  in  multimedia  networks  served  as  proof  of  the  necessity  of 
co-operation  among  the  various  actors  in  the  development  of  educational  technology  and  its  exploitation.  Co- 
operation between  research  units,  educational  institutions  and  enterprises,  software  and  hardware  manufacturers, 
content  provider  and  teleoperators  has  been  fruitful  and  has  served  to  open  new  vistas.  This  valuable  experience 
serves  as  a good  point  of  departure  for  possible  initiation  of  national  and  international  projects  in  the  future. 
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Abstract;  This  paper  is  about  an  intelligent  multimedia  tutoring  system  of  the  passive  voice 
of  the  English  grammar.  The  system  may  be  used  to  present  theoretical  issues  about  the 
passive  voice  and  to  provide  exercises  that  the  student  may  solve.  While  the  student  types 
the  solution  to  an  exercise,  the  system  examines  whether  the  answer  has  been  correct  and  if 
not  it  attempts  to  diagnose  what  the  cause  of  the  mistake  has  been.  The  student’s  progress 
and  usual  mistakes  are  recorded  to  the  student  model.  This  kind  of  information  may  be  used 
in  the  individualised  error  diagnosis  of  the  student  in  subsequent  sessions. 

Key  words:  Intelligent  Tutoring  System,  multimedia,  student  modelling,  error  diagnosis, 
language  learning. 


Overview 


Students  who  learn  English  as  a second  language  often  experience  problems  in  learning  and 
consolidating  the  English  grammar.  The  passive  voice  is  an  important  topic  which  often  seems  difficult  to 
students  to  use  correctly.  The  problem  is  even  greater  in  cases  when  the  passive  voice  is  not  used  extensively  as 
a matter  of  style  in  the  first  language  of  students. 

A multimedia  tutor  of  the  passive  voice  can  be  very  useful  if  it  incorporates  intelligence  so  that  it  can 
help  the  student  in  correcting  his/her  mistakes  and  offer  an  individualised  explanation  for  the  cause  of  them. 
Many  researchers  point  out  (e.g.  McGraw,  1994)  that  the  integration  of  AI  technologies  and  hypermedia  may 
offer  better  support  to  students  than  any  of  these  alone.  The  multimedia  tutor  that  we  have  developed  has  been 
based  on  the  techniques  used  in  Intelligent  Tutoring  Systems  (ITSs)  (Wegner,  1987).  Our  system  focuses  on 
student  error  diagnosis  in  common  with  other  systems  in  the  domain  of  language  teaching  such  as  (Bos  and 
Van  De  Plassche,  1994),  (Heift,  1998)  and  (Kunichike  et  al.,  1998). 

In  particular,  the  passive  voice  tutor  constructs  a student  model  which  serves  as  a source  of 
information  that  can  be  used  for  the  interpretation  of  the  student’s  actions  and  possible  mistakes  in  solving 
exercises.  The  underlying  cause  of  a mistake  may  be  difficult  for  a tutoring  system  to  spot.  For  example, 
Hollnagel  (1991;  1993)  makes  an  important  distinction  between  the  underlying  cause  or  genotype  of  an  error 
and  the  observable  manifestation  or  phenotype  of  the  error. 

In  the  case  of  the  passive  tutor,  error  diagnosis  is  based  on  the  student  model  where  there  have  been 
recorded  previous  mistakes  that  the  particular  user  has  made  and  the  progress  that  s/he  has  made  since  previous 
sessions.  For  example,  if  a student  has  been  recorded  to  have  been  prone  to  spelling  mistakes,  then  the  system 
may  use  this  piece  of  information  for  the  interpretation  of  a mistake  by  favouring  the  cause  of  wrong  spelling 
rather  than  lack  of  grammar  knowledge  at  some  points  of  the  interaction. 


Architecture  of  the  Passive  Voice  Tutor 
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The  architecture  of  the  passive  voice  tutor  follows  the  main  line  of  Intelligent  Tutoring  Systems  (ITS) 
architectures.  Researchers  in  the  area  of  ITSs  (Hartley  and  Sleeman,  1973;  Burton  and  Brown  1976;  Wenger 
1987)  largely  agree  on  the  major  functional  components  constituting  an  ITS  architecture,  namely  the  Domain 
Knowledge,  the  Student  Modeller,  the  Advice  Generator  and  the  User  Interface, 

The  domain  knowledge  of  the  passive  voice  tutor  contains  knowledge  about  how  to  convert  a sentence 
from  active  to  passive  voice  and  vice  versa.  In  addition,  it  contains  knowledge  about  the  dynamic  construction 
of  new  sentences  that  can  be  used  as  exercises.  The  construction  of  new  sentences  is  done  using  several  lists  of 
words,  three  of  which  are  the  most  important.  One  for  possible  subjects,  one  for  possible  verbs  and  one  for 
possible  objects.  All  lists  are  semantically  related  among  them  so  that  sentences  that  do  not  make  sense  are  not 
allowed.  This  kind  of  knowledge  is  used  by  the  system  so  that  the  user  is  not  shown  the  same  exercises  while 
running  the  program.  The  same  kind  of  knowledge  can  also  be  used  by  the  student  or  teacher  for  the  creation  of 
new  exercises  that  can  be  answered  by  the  system  as  an  example  or  can  be  used  as  new  homework  for  the 
students. 

The  student  modeller  compares  the  solution  that  a student  has  given  to  that  of  the  system  and  if  these 
do  not  match  it  performs  error  diagnosis.  There  are  several  categories  of  error  that  may  be  recognised,  for 
example  spelling  mistakes,  accidental  slips,  mistakes  in  the  actual  conversion  of  passive  to  active  voice, 
mistakes  in  the  use  of  tenses,  etc.  The  student  modeller  is  also  responsible  for  recording  statistics  about  each 
student’s  performance.  For  example,  the  system  records  the  percentage  of  the  exercises  that  have  been  correctly 
answered,  the  percentage  of  spelling  mistakes  etc.  This  kind  of  information  is  also  used  in  error  diagnosis  to 
create  an  individualised  interpretation  of  a user’s  error. 

The  advice  generator  is  responsible  for  acting  in  the  most  appropriate  way  in  every  situation.  For 
example,  if  a student  has  made  a mistake  which  is  relevant  to  the  use  of  verb  tenses  then  the  advice  generator 
will  show  to  the  student  the  part  of  the  theory  that  deals  with  this  subject. 

Finally,  the  multimedia  user  interface  involves  animations,  sounds  and  a limited  form  of  natural 
language  so  that  it  can  attract  the  student’s  interest  in  the  subject. 


Overall  Performance 

The  overall  performance  of  the  passive  voice  tutor  may  be  summarised  in  the  following  functions 
which  may  be  selected  from  the  main  menu  of  the  system: 

• Theory:  At  this  point  the  student  may  read  the  theoretical  issues  that  are  relevant  to  passive  voice, 

• Solving  Exercises:  At  this  point  the  student  may  start  interacting  with  the  system  in  solving  exercises.  The 
student  is  given  four  types  of  exercise  to  select  from: 

1)  Multiple  choice  exercises. 

2)  Exercises  where  the  user  is  asked  to  rewrite  a sentence  using  passive  voice. 

3)  Exercises  where  the  user  is  asked  to  rewrite  a sentence  using  active  voice. 

4)  A puzzle. 

The  way  that  multiple  choice  exercises  have  been  constructed  is  based  on  an  empirical  study  that  has 
been  conducted  in  real  classrooms  and  through  interviews  with  students  and  teachers.  The  results  of  this 
particular  study  have  shown  what  the  most  common  students’  mistakes  are.  This  kind  of  information  has 
been  used  throughout  the  creation  of  the  erroneous  answers  that  have  been  incorporated  in  the  multiple 
choice  questions  development.  Furthermore,  the  above  results  have  been  used  for  the  analysis  of  possible 
students’  mistakes  in  the  design  of  the  diagnostic  component  of  the  student  modeller. 

In  the  case  of  exercises  where  the  student  is  asked  to  rewrite  a sentence  using  passive  voice,  the 
student  is  given  a sentence  in  the  active  voice  and  s/he  is  asked  to  retype  the  sentence  in  the  passive  voice. 
At  this  kind  of  exercise,  the  system  ignores  trivial  typographic  errors  such  as  the  absence  of  a fullstop  at  the 
end  of  the  sentence,  absence  of  any  space  between  words,  redundant  spaces  or  commas,  etc.  If  the  student 
has  made  an  error  other  than  a trivial  typographic  errors,  then  the  system  performs  error  diagnosis  taking 
into  account  information  that  has  been  collected  about  the  specific  user  in  previous  interactions  as  well  as 
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common  mistakes  that  have  been  classified  in  several  categories  of  error.  There  are  seven  types  of  errors 
that  may  be  recognised  by  the  system: 


1)  Accidental  slips 

For  example,  the  student  may  have  deleted  some  words  and  may  have  forgotten  to  complete  the 
sentence. 

2)  Spelling  mistakes 

A spelling  mistake  occurs  when  the  user  has  typed  the  expected  word  so  that  one  letter  is  redundant 
or  missing  or  two  neighbouring  letters  have  been  interchanged.  For  example,  the  student  has  typed 
“mather”  instead  of  “mother”. 

3)  Article  mistakes 

For  example,  the  user  may  have  used  “a”  instead  of  “an”. 

4)  Conversion  mistakes 

For  example,  the  user  may  have  not  converted  the  active  verb  “write”  to  the  past  participle  “written”. 

5)  Irregular  verb  mistakes 

The  user  may  have  used  the  usual  ending  “-ed”  to  create  the  past  participle  of  an  irregular  verb  (for 
example  s/he  may  have  typed  “buyed”  instead  of  “bought”). 

6)  Noun  mistakes 

For  example  the  user  may  have  typed  “book”  instead  of  “books”. 

7)  Verb  tense  mistakes 

For  example  the  user  may  have  typed  “been”  instead  of  “being”. 

In  some  cases  a mistake  of  the  user  may  be  attributed  to  more  than  one  categories  of  error.  For 
example,  if  a student  has  not  typed  the  word  “by”  in  the  converted  sentence,  this  could  be  either  accidental 
slip  or  a conversion  mistake.  In  this  case,  the  former  is  rather  superficial  whereas  the  latter  is  deeper  and 
shows  that  the  student  has  not  consolidated  the  knowledge  about  passive  voice.  In  cases  like  this  the  system 
takes  into  account  the  individual  features  of  the  user,  that  have  been  recorded  in  previous  interactions,  in 
order  to  formulate  the  kind  of  advice  to  give  to  the  student.  For  example,  if  the  particular  student  has  not 
previously  made  conversion  mistakes  but  has  made  carelessness  mistakes  then  the  system  favours 
carelessness  as  the  most  probable  cause  of  the  mistake. 

Examples  of  interactions  involving  students’  errors  are  the  following: 


First  Example 

System  exercise: 
Student’s  answer: 
System: 


My  sister  has  bought  this  book. 

This  book  has  been  bouht  by  my  sister. 

You  have  made  a spelling  mistake  in  the  past  participle.  You  should  have 
written  “bought”  instead  of  “bouht”. 

Please  try  again. 


Second  Example 

System  exercise: 
Student’s  answer: 
System: 


My  sister  has  bought  this  book. 

This  book  has  been  bought  my  sister. 

Neither  “by”  nor  “with”  was  found  in  your  sentence.  Please  notice  that  we 
use  one  of  these  words  in  passive  voice  for  the  agent. 

Please  read  again  the  theory  and  try  again. 


In  this  example  the  system  has  considered  the  error  as  a conversion  mistake.  Alternatively,  it  could 
have  considered  the  same  mistake  as  a carelessness  error.  However,  in  this  case  it  must  not  have  had 
sufficient  information  that  the  student  knows  how  to  use  “by”  or  “with”. 
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Third  Example 

My  sister  will  buy  this  book. 

My  sister  will  be  bought  by  a book. 

You  have  made  a serious  conversion  error.  Please  notice  that  in  passive 
voice  the  subject  and  object  should  exchange  places. 

Please  read  again  the  theory  and  try  again. 

In  the  case  of  exercises  where  the  user  is  asked  to  rewrite  a sentence  using  active  voice  the  function  of 
the  system  is  similar  to  that  of  exercises  that  were  explained  above.  The  only  difference  is  that  the  initial 
sentence  is  given  in  passive  voice  this  time. 

Finally,  the  type  of  exercise  called  “puzzle”  is  a kind  of  game  that  the  student  may  play,  so  that  s/he 
can  consolidate  knowledge  while  having  fun.  This  feature  is  quite  important  pedagogically  because  it 
stimulates  student’s  interest.  As  (Amory  et  al.,  1998)  point  out  while  students  learn  equally  from  playing 
games  and  traditional  practical  material,  they  find  playing  games  more  enjoyable  and  hence  they  attempt 
more  interactions. 

• Creating  Exercises/Examples:  At  this  point  the  student  is  shown  exercises  again,  only  this  time  s/he  is  the 
one  who  creates  the  exercises  and  the  system  solves  them  as  an  example.  The  student  may  create  sentences 
in  active  voice  by  selecting  subjects,  verbs  and  objects  from  three  lists  respectively.  Then  the  student  may 
select  articles  or  pronouns,  singular  or  plural  for  the  subject  and  object  as  well  as  tense  for  the  verb.  If  the 
created  sentence  does  not  make  sense  or  is  not  correct  then  the  system  gives  a message  to  the  user  to  let 
him/her  know  about  the  problem.  If  the  sentence  is  fine  then  the  system  rewrites  the  sentence  using  passive 
voice,  so  that  the  student  can  see  how  this  is  done. 

• Statistics:  The  student  is  allowed  to  see  the  statistics  about  his/her  performance,  which  have  been  recorded 
by  the  system.  In  addition,  s/he  may  ask  to  have  a printed  report.  The  student  can  see  the  statistics  about 
his/her  overall  performance  in  all  the  sessions  that  s/he  has  attempted  or  just  in  the  current  session.  S/he 
may  also  see  the  comparison  between  his/her  overall  performance  and  the  performance  in  the  current 
session  or  the  comparison  between  performances  in  the  current  session  and  the  previous  one. 

An  example  of  statistics  about  a user’s  overall  performance  is  the  following: 

Student  name:  John 
In  a total  of  25  exercises: 


Correct  sentences:  65.5% 

Sentences  having  spelling  mistakes:  12.5% 

Sentences  having  mistakes  due  to  carelessness:  25% 

Conversion  mistakes:  0% 


In  the  case  of  the  above  example  in  a subsequent  session  of  the  student  called  John,  the  system  will 
favour  the  cause  of  carelessness  for  a mistake  rather  than  lack  of  grammar  knowledge  in  cases  where  either 
of  them  could  be  considered. 


System  exercise: 
Student’s  answer: 
System: 


Conclusions 

The  passive  voice  tutor  is  a multimedia  system  that  combines  user-friendliness  with  intelligence  in 
interaction.  It  offers  individualised  advice  to  users  when  they  make  errors  after  it  has  performed  error 
diagnosis.  It  has  a mechanism  for  choosing  between  different  possible  causes  underlying  an  error  which  vary 
from  trivial  typographic  errors  to  deeper  misconceptions  concerning  the  main  topic  that  is  being  taught.  The 
design  of  the  diagnostic  component  has  been  based  on  an  empirical  study  that  has  been  held  in  real  classrooms 
and  has  shown  what  the  most  common  students’  mistakes  are. 
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Abstract:  This  paper  describes  a multimedia  tutoring  system  for  Algebra.  The  system 
is  called  “EasyMath”  and  it  incorporates  intelligence.  One  of  the  primary  aims  of 
EasyMath  is  its  usefulness  in  school  classrooms,  therefore  school  teachers  of 
mathematics  have  been  involved  throughout  the  life  cycle  of  EasyMath.  The  design  of 
EasyMath  has  been  based  on  the  results  of  an  empirical  study  that  was  conducted  at 
schools  and  the  resulting  product  was  evaluated  by  school-teachers  as  well  as  students. 
EasyMath  incorporates  techniques  from  Intelligent  Tutoring  Systems.  In  particular,  it 
takes  a reconstructive  approach  to  student  modelling  and  error  diagnosis,  which  uses  a 
generic  model  of  the  domain  for  the  reconstruction  of  correct  and  incorrect  solutions  to 
exercises.  EasyMath  provides  a facility  for  the  dynamic  construction  of  new  exercises 
for  students  on  which  it  can  perform  error  diagnosis  and  provide  appropriate  advice. 
EasyMath  has  been  evaluated  by  school  teachers  and  students.  The  results  of  the 
evaluation  show  that  the  multimedia  interface  provides  a user-friendly  interactive 
environment  which  seems  more  attractive  to  students  than  conventional  teaching 
methods.  In  addition,  EasyMath ’s  intelligence  guarantees  more  individualised  advice 
with  high  relevance  to  the  context  and  problems  of  the  students. 

Key  words:  Intelligent  Tutoring  Systems,  Multimedia,  Student  Modelling,  Error 
Diagnosis. 


Overview 

This  paper  describes  a multimedia  educational  system  for  Algebra.  The  system  is  called 
EasyMath.  One  of  the  main  aims  of  EasyMath  is  its  usefulness  in  the  environment  of  school  classrooms  in 
common  with  other  projects  such  as  (Alexandris  et  al.,  1998).  Therefore  school  teachers  have  been 
involved  in  the  phases  of  requirements  analysis  and  evaluation  of  EasyMath’s  life  cycle. 

In  particular,  during  the  requirements  analysis,  an  empirical  study  was  conducted  involving  4 
school  teachers  and  240  students  of  8 different  classes  of  the  same  grade.  The  results  of  the  empirical 
study  showed  what  the  most  common  misconceptions  of  students  have  been  concerning  Algebraic  powers. 
These  misconceptions  have  served  as  the  basis  for  the  design  of  the  student  modelling  component  of 
EasyMath.  The  empirical  study  is  described  in  more  detail  in  the  next  section. 

The  evaluation  phase  of  EasyMath  involved  4 school  teachers  who  were  different  from  the  ones 
involved  in  the  empirical  study.  EasyMath  was  also  evaluated  by  a total  of  180  students  of  6 different 
classes  of  the  same  grade.  The  evaluation  is  described  in  more  detail  in  a subsequent  section. 

EasyMath  incorporates  techniques  from  Intelligent  Tutoring  Systems  (Wenger,  1987).  It  follows 
the  main  line  of  an  ITS  architecture  which  consists  of  four  major  components,  namely  the  domain 
knowledge,  the  student  modeller,  the  advice  generator  and  the  user  interface.  ITSs  have  often  been 
criticized  that  in  spite  of  their  superiority  to  traditional  stand  up  training  they  may  miss  the  mark  in  terms 
of  task  reality,  feasibility  and  effectiveness  (McGraw,  1994).  However,  there  have  been  quite  a lot  of 
successful  evaluations  of  ITSs  (e.g.  Marks  & Greer,  1991)  and  reports  on  the  successful  use  in  classrooms 
of  ITSs  (e.g.  Koedinger  & Anderson,  1993). 

The  design  of  EasyMath  has  been  based  on  techniques  of  ITSs  that  can  be  useful  to  the 
improvement  of  the  quality  of  multimedia  educational  software  both  for  the  student  and  the  teacher.  In 
particular,  the  student  modelling  component  has  been  based  on  reconstructive  approaches  to  error 
diagnosis  which  have  been  used  in  many  ITSs  such  as  (Langley  et  al.,  1987;  Sleeman  et  al.,  1990;  Hope, 
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1 994).  These  systems  reconstruct  the  problem  solving  process  and  generate  mal-rules  from  hypothesized 
faulty  solution  paths  which  are  used  for  the  modelling  of  students’  misconceptions  or  procedural  bugs.  In 
the  case  of  EasyMath,  the  most  common  faulty  solution  procedures,  which  were  identified  in  students’ 
transcripts  during  the  empirical  study,  have  been  encoded  in  the  student  modelling  component. 

These  faulty  solution  procedures  are  used  for  two  purposes: 

1)  For  the  construction  of  new  multiple  choice  tests  where  the  wrong  answers  have  been  generated 
by  the  use  of  these  faulty  procedures. 

2)  For  diagnosing  what  the  cause  of  a student  error  may  have  been  while  s/he  is  solving  an 
exercise. 

Finally,  EasyMath’ s multimedia  interface  renders  the  system  quite  attractive  to  students.  The 
evaluation  results  have  shown  that  students  particularly  liked  an  educational  game  that  EasyMath 
incorporates  and  which  makes  full  use  of  multimedia  facilities  such  as  animations,  sounds,  etc. 


Empirical  Study 

EasyMath ’s  design  has  been  based,  to  a large  extent,  on  the  results  of  an  empirical  study  that 
involved  school-teachers  and  students.  These  results  were  particularly  useful  for  the  requirements  analysis 
of  EasyMath  as  they  showed  what  teachers  would  like  the  software  to  do  to  help  them  in  the  class.  In 
addition,  they  showed  what  particular  pieces  of  the  domain  knowledge  seem  difficult  to  students  to 
comprehend  fully  and  what  misconceptions  they  may  have  while  learning. 

Educational  software  should  be  used  to  supplement  human  teachers  and  not  substitute  them, 
therefore  human  teachers  should  be  involved  in  the  development  process.  Besides,  as  Mark  and  Greer 
(1993)  point  out,  the  accuracy  of  ITS  components  such  as  domain  knowledge  should  be  ensured  before  a 
system  is  completed  and  assumed  thereafter.  One  good  way  of  ensuring  this  accuracy  is  to  involve  human 
teachers  in  the  early  stages  of  the  development  process. 

The  empirical  study  involved  4 school-teachers.  Each  school-teacher  taught  2 different  classes  of 
the  same  grade.  Each  class  consisted  of  30  students.  All  students  of  every  class  were  involved  in  the 
empirical  study.  This  gave  a total  of  240  students  who  were  involved  in  the  empirical  study. 

All  students  were  given  the  same  tests  which  were  made  by  the  4 teachers  in  collaboration.  These 
tests  covered  all  the  material  taught  in  Algebraic  powers.  The  results  of  these  tests  were  analysed  so  that  as 
many  as  possible  errors  could  be  categorized.  As  a result  of  the  analysis,  the  most  common  categories  of 
error  were  identified  and  the  cause  of  them  was  diagnosed  by  the  human  teachers. 

For  example,  for  the  multiplication  of  powers,  the  following  most  common  categories  of  error 
were  identified: 

• If  both  powers  have  positive  bases  as  well  as  exponents,  the  user  may  multiply  instead  of  adding 
the  exponents.  For  example,  4^  * 4^  = 4*^  instead  of  4^. 

• In  the  case  when  both  powers  have  positive  bases  as  well  as  exponents,  the  user  may  multiply 
the  bases  instead  of  leaving  the  same  base.  For  example,  3"*  * 3^  = 9^  instead  of  3^. 

• When  both  powers  have  positive  bases  as  well  as  exponents,  the  user  may  multiply  the  bases 
instead  of  leaving  the  same  base  in  addition  to  multiplying  instead  of  adding  the  exponents.  For 
example,  2^  * 2^  = 4*^  instead  of  2^. 

• If  one  of  the  powers  has  a negative  exponent  and  the  other  has  a positive  exponent,  but  both  of 
them  have  the  same  bases,  the  student  may  subtract  instead  of  adding  the  exponents.  For 
example,  5^  * 5*^  = 5^  instead  of  5‘*. 

• The  user  may  make  a careless  mistake  either  in  entering  the  base  or  the  exponent  of  the  answer. 
For  example,  4^  * 4^  = 5^  instead  of  4^. 

• When  the  student  is  asked  to  multiply  a power  with  a number  that  can  be  converted  to  a power, 
s/he  may  make  mistakes  both  while  converting  the  number  into  a power  as  well  as  while 
multiplying  the  two  powers.  For  example  2^  * 8 = 16^  or  2^  * 8 = 2^  instead  of  2^. 

The  identified  most  common  categories  of  error,  their  causes  and  appropriate  advice  for  each 
category  were  encoded  to  EasyMath’ s domain  knowledge,  student  modeller  and  advice  generator. 
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Overall  design  of  EasyMath 


EasyMath  has  been  designed  to  be  used  by  students  and/or  teachers.  Students  may  use  EasyMath 
to  read  theoretical  lessons  about  Algebraic  powers  and  solve  exercises.  Teachers  can  use  EasyMath  as  an 
assisting  tool  for  the  creation  of  new  exercises,  which  can  serve  as  student  tests. 

The  various  ways  that  a student  and/or  a teacher  can  use  EasyMath  are  represented  as  different 
modes  of  function: 


• Lecturing  mode:  In  this  mode,  the  student  is  able  to  read  theoretical  issues  concerning  algebraic 
powers. 

• Solving  exercises  mode:  In  this  mode,  the  student  is  presented  several  exercises  to  solve.  EasyMath 
constructs  dynamically  new  exercises  of  different  levels  of  difficulty,  starting  from  the  first  level  to 
higher  levels. 

For  example,  in  the  multiplication  of  powers,  the  first  level  of  difficulty  comprises  exercises 
where  the  student  is  asked  to  convert  to  a power  a number  which  is  multiplied  by  itself  several 
times.  For  example,  3*3*3*3*3  = 3^.  The  number  which  is  multiplied  as  well  as  the  number 
of  factors  of  the  multiplication  are  generated  randomly  and  they  are  different  each  time  a student 
attempts  this  type  of  exercise. 

A higher  level  of  difficulty  includes  exercises  where  the  multiplication  factors  are  negative 
numbers.  The  different  sign  of  factors  renders  this  type  of  exercise  more  difficult  than  that  of  the 
first  level  and  so  on. 

For  every  type  of  exercise,  EasyMath  contains  knowledge  about  how  to  solve  it  correctly  and  how 
to  solve  it  in  several  faulty  ways  that  represent  the  most  common  errors  that  were  identified  in  the 
empirical  study  for  the  particular  type  of  exercise. 

If  a student  gives  an  answer  which  is  correct  then  EasyMath  congratulates  him/her  and  allows 
him/her  to  proceed  to  next  level  of  difficulty.  If  a student  gives  an  erroneous  answer  then  EasyMath 
tries  to  perform  error  diagnosis  by  generating  the  faulty  procedures  of  solution.  If  one  of  them  is 
found  to  match  the  student’s  answer  then  EasyMath  shows  to  the  user  the  appropriate  kind  of 
advice  and  records  the  type  of  error  identified  in  the  student’s  personal  record  so  that  it  can  have 
the  history  of  his/her  progress. 

The  way  that  EasyMath  performs  error  diagnosis  is  based  on  reconstructive  methods  of  student 
modelling.  For  example,  while  a student  converts  a number  into  a power,  s/he  may  make  the 
following  types  of  error: 

1.  Error  in  the  factorization  of  a number  to  prime  numbers.  This  error  can  occur  due  to  an 
erroneous  conduction  of  the  needed  divisions  as  well  as  a division  with  a number  that  is  not  the 
smallest  prime  number.  For  example,  while  factorizing  the  number  8 1 to  its  prime  numbers,  the 
student  may  divide  the  number  by  9,  which  is  not  a prime  number.  As  a consequence  of  this 
type  of  an  error,  the  base  of  the  power  that  is  given  as  an  answer  is  wrong,  9 instead  of  3 in  the 
example.  An  example  of  this  kind  is  shown  in  [Figure  la]  where  the  student’s  answer  is  9^ 
instead  of  3"^  which  is  the  correct  answer. 

2.  Correct  factorization  of  the  number  but  erroneous  transformation  to  a power.  For  example,  the 
student  after  factorizing  correctly  the  number  1 25  to  its  prime  numbers,  s/he  has  a 
misconception  in  identifying  which  of  the  numbers  is  the  base  and  which  is  the  exponent.  This 
may  result  in  a mistake  such  as  entering  a base  different  from  5 or  an  exponent  different  from  3. 
An  example  of  this  kind  is  shown  in  [Figure  lb]  where  the  student’s  answer  is  5^  instead  of  5^ 
which  is  the  correct  answer. 
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Figure  la:  One  case  of  erroneous  answer 


Figure  lb:  Another  case  of  erroneous  answer 


In  the  case  when  the  student’s  answer  is  of  the  type  that  is  illustrated  in  [Figure  la],  the  system’s 
answer  is:  "'There  has  been  an  error  in  the  factorization  procedure,  because  the  number  used  as  a 
base  is  not  a prime  number.^' 

In  the  case  when  the  student’s  answer  is  of  the  type  that  is  illustrated  in  [Figure  lb],  the  system’s 
answer  is:  "The  factorization  procedure  is  correctly  completed,  but  there  has  been  an  error  while 
constructing  the power.^' 

• Answering  multiple  choice  questions:  In  this  mode,  EasyMath  constructs  dynamically  a new  set 
of  multiple  choice  questions  which  are  presented  to  the  user.  The  construction  of  multiple  choice 
questions  is  based  on  domain  knowledge  and  the  student  model. 

Numbers  that  serve  as  bases  or  exponents  are  generated  randomly.  For  each  type  of  question  there 
is  one  procedure  that  solves  the  exercise  correctly  and  there  is  a number  of  faulty  procedures  that 
have  been  encoded  to  the  system,  which  represent  common  students’  mistakes.  The  result  of  the 
correct  procedure  is  used  as  the  correct  answer  in  the  multiple  choice.  Erroneous  answers  are 
generated  from  the  faulty  procedures  that  are  encoded  into  the  system.  For  each  question,  a number 
of  faulty  procedures  are  selected  randomly  from  the  whole  set  of  faulty  procedures. 

For  example,  when  the  system  issues  a question  concerning  the  multiplication  of  powers,  it 
performs  the  following  steps: 

1)  A number  within  the  range  [1,  9]  is  generated  randomly,  in  order  to  serve  as  the  base  of  the 
two  powers  to  be  multiplied. 

2)  Two  numbers  within  the  range  [1,9]  are  generated  randomly,  in  order  to  form  the  exponents 
of  the  two  powers. 

After  the  generation  of  the  question  that  will  be  shown  to  the  student,  the  system  chooses 
randomly  two  out  of  the  seven  possible  erroneous  procedures  and  based  on  them,  it  constructs  the 
two  wrong  answers  that  would  serve  as  alternative  answers  to  the  problem  that  the  user  has  to 
solve. 

An  example  of  a multiple  choice  question  is  the  following: 


6^  * 6'^  = ? 

a.  36'^ 

b.  6'^ 

c.  6-'^ 

In  this  case,  the  correct  answer  is  b.  However,  if  the  student  selects  a,  then  EasyMath  prompts 
him/her  with  the  message: 

''Identified  problem  in  the  multiplication  of  powers:  you  have  multiplied  the  bases  instead  of 
leaving  the  same  base. 

If  the  student  selects  c,  then  EasyMath  prompts  him/her  with  the  message: 

"Identified  problem  in  the  multiplication  of  powers:  you  have  multiplied  instead  of  adding  the 
exponents.  ” 

• Game  playing  mode:  In  this  mode,  the  student  is  given  a game  which  is  relevant  to  algebraic 
powers.  This  mode  is  quite  important  pedagogically.  As  (Amory  et  al.,  1998)  point  out,  games 
appear  to  motivate  students  intrinsically  and  represent  one  of  the  best  uses  of  multimedia  in 
education. 

• Assistance  in  the  creation  of  new  exercises:  In  this  mode,  the  teacher  can  profit  from  the  domain 
knowledge  and  student  modeller  to  create  new  multiple  choice  exercises  and  other  exercises 
concerning  algebraic  powers.  For  example,  the  teacher  types  in  a new  question  which  is  solved  by 
the  system  correctly  and  in  all  possible  faulty  ways  that  EasyMath  knows.  Then  the  teacher  is 
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presented  with  faulty  results  and  algorithms  so  that  s/he  can  select  the  erroneous  answers  for  the 
multiple  choice  question. 


Evaluation 

EasyMath  was  evaluated  by  4 school-teachers  and  a total  of  180  students.  The  4 teachers  who 
participated  in  the  evaluation  were  different  from  the  teachers  who  were  involved  in  the  empirical  study. 
They  were  asked  to  use  EasyMath  and  then  answer  a questionnaire.  Students  constituted  6 different  classes 
of  the  same  grade.  Each  class  consisted  of  30  students.  The  evaluation  by  students  was  done  in  two  phases. 
In  the  first  phase  1 20  students  of  3 different  classes  of  the  same  grade  were  asked  to  use  the  software  and 
then  answer  a questionnaire  which  included  questions  that  are  illustrated  in  [Table  1]. 


Question 

Agreement 

Lack  of 
preference 

Disagreement 

1. 

Do  you  think  that  using  EasyMath  makes  learning 
more  like  a game? 

95% 

1% 

4% 

2. 

Were  graphics  of  EasyMath  attractive? 

97% 

0% 

3% 

3. 

Were  the  colors  and  text  used  by  EasyMath 
convenient? 

90% 

3% 

7% 

4. 

Has  the  use  of  sounds  in  EasyMath  enhanced 
attention? 

92% 

2% 

6% 

5. 

Were  you  satisfied  with  the  responses  to  students’ 
errors? 

98% 

1% 

1% 

6. 

Do  you  believe  that  the  domain  of  Algebraic  powers 
was  fully  covered? 

66% 

15% 

19% 

7. 

Was  the  amount  of  different  exercises  enough? 

93% 

3% 

4% 

8. 

Were  you  pleased  by  the  way  student’s  progress  has 
been  recorded  and  used? 

89% 

3% 

8%  . 

9. 

Was  it  easy  to  get  familiar  to  using  EasyMath? 

94% 

4% 

2% 

10. 

Was  it  easy  to  trick  EasyMath? 

84% 

1% 

15% 

Table  1:  Questionnaire. 


EasyMath’s  quality  features  evaluated 

Corresponding  questions 
from  questionnaire 

Average  Score 

Friendliness  and  Attractiveness  of  the 
Graphical  User  Interface 

1,2,  3,  4,  9 

93.6% 

Completeness  of  the  Domain  Knowledge 

6,  10 

75% 

Correctness  of  the  interactive  student  solution 
checker 

8,  10 

86.5% 

Relevance  of  the  advice  provided  when  solving 
exercises 

5,  10 

91% 

Sufficiency  of  range  of  exercises  provided 

7 

93% 

Table  2:  Summary  of  results  of  evaluation. 


Answers  to  the  questions  of  the  questionnaire  (Table  1)  were  used  to  evaluate  certain  quality 
features  of  EasyMath  as  shown  in  Table  2.  In  this  table  the  first  column  illustrates  the  quality  features 
evaluated.  In  the  second  column  we  give  the  corresponding  questions  from  the  questionnaire  which  were 
used  for  the  evaluation  of  the  respective  quality  features.  Finally,  in  the  third  column  we  give  the  score 
which  has  resulted  from  the  average  of  the  provided  answers  to  the  corresponding  questions,  as  it  appears 
in  the  “Agreement”  column  of  Table  1.  For  example,  in  the  friendliness  and  attractiveness  of  the  GUI  we 
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take  the  average  of  the  answers  that  represent  the  agreement  percentage  for  questions  1,  2,  3,  4 and  9, 
namely  95%,  97%,  90%,  92%  and  94%  respectively.  The  resulted  average  score  is  93.6%. 

In  the  second  phase  120  students,  other  than  the  120  students  who  participated  in  the  first  phase, 
were  taught  half  of  the  syllabus  in  algebraic  powers  without  the  use  of  EasyMath  and  they  were  given  a 
written  test.  Then  they  used  EasyMath  while  being  taught  the  rest  of  the  syllabus.  In  the  end  they  were 
given  another  written  test  and  the  grades  of  their  first  and  second  test  were  compared.  46%  of  the  students 
obtained  a better  grade  in  the  second  test,  43%  obtained  the  same  grade  and  only  1 1%  obtained  a lower 
grade  in  the  second  test. 


Conclusions 

Multimedia  interfaces  when  combined  with  the  reasoning  capabilities  of  Intelligent  Tutoring 
Systems  can  provide  attractive,  user-friendly  environments  for  learning,  which  can  interact  with  the 
student  in  an  individualized  way.  This  kind  of  educational  software  can  be  useful  to  classrooms  if  it 
provides  assistance  to  teachers  and  helps  students  in  the  learning  process.  These  qualities  can  be  achieved 
to  a high  degree  if  school-teachers  participate  in  the  development  of  educational  software. 
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Abstract:  The  Intelligent  Essay  Assessor  (TEA)  is  a set  of  software  tools  for  scoring  the 
quality  of  essay  content.  The  TEA  uses  Latent  Semantic  Analysis  (LSA),  which  is  both  a 
computational  model  of  human  knowledge  representation  and  a method  for  extracting 
semantic  similarity  of  words  and  passages  trom  text.  Simulations  of  psycholinguistic 
phenomena  show  that  LSA  reflects  similarities  of  human  meaning  effectively.  To  assess 
essay  quality,  LSA  is  first  trained  on  domain-representative  text.  Then  student  essays  are 
characterized  by  LSA  representations  of  the  meaning  of  their  contained  words  and  compared 
with  essays  of  known  quality  on  degree  of  conceptual  relevance  and  amount  of  relevant 
content.  Over  many  diverse  topics,  the  lEA  scores  agreed  with  human  experts  as  accurately  as 
expert  scores  agreed  with  each  other.  Implications  are  discussed  for  incorporating  automatic 
essay  scoring  in  more  general  forms  of  educational  technology. 


Introduction 

While  writing  is  an  essential  part  of  the  educational  process,  many  instructors  find  it  difficult  to 
incorporate  large  numbers  of  writing  assignments  in  their  courses  due  to  the  effort  required  to  evaluate  them. 
However,  the  ability  to  convey  information  verbally  is  an  important  educational  achievement  in  its  own  right, 
and  one  that  is  not  sufficiently  well  assessed  by  other  kinds  of  tests.  In  addition,  essay-based  testing  is  thought 
to  encourage  a better  conceptual  understanding  of  the  material  on  the  part  of  students  and  to  reflect  a deeper, 
more  useful  level  of  knowledge  and  application  by  students.  Thus  grading  and  criticizing  written  products  is 
important  not  only  as  an  assessment  method,  but  also  as  a feedback  device  to  help  students  better  learn  both 
content  and  the  skills  of  thinking  and  writing.  Nevertheless,  essays  have  been  neglected  in  many  computer- 
based  assessment  applications  since  there  exist  few  techniques  to  score  essays  directly  by  computer.  In  this 
paper  we  describe  a method  for  performing  automated  essay  scoring  of  the  conceptual  content  of  essays. 
Based  on  a statistical  approach  to  analyzing  the  essays  and  content  information  from  the  domain,  the  technique 
can  provide  scores  that  prove  to  be  an  accurate  measure  of  the  quality  of  essays. 

The  text  analysis  underlying  the  essay  grading  schemes  is  based  on  Latent  Semantic  Analysis  (LSA). 
Detailed  treatments  of  LSA,  both  as  a theory  of  aspects  of  human  knowledge  acquisition  and  representation, 
and  as  a method  for  the  extraction  of  semantic  content  of  text  are  beyond  the  scope  of  this  article.  They  are  fully 
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presented  elsewhere  (Deerwester,  Dumais,  Furnas,  Landauer,  & Harshman,  1990;  Landauer  & Dumais,  1997; 
Landauer,  Foltz  & Laham,  1998),  as  are  a number  of  simulations  of  cognitive  and  psycholinguistic  phenomena 
that  show  that  LSA  captures  a great  deal  of  the  similarity  of  meanings  expressed  in  discourse  (Rehder, 
Schreiner,  Wolfe,  Laham,  Landauer,  & Kintsch,  1998  ; Wolfe,  Schreiner,  Rehder,  Laham,  Foltz,  Kintsch,  & 
Landauer,  1998), 

Based  on  a statistical  analysis  of  a large  amount  of  text  (typically  thousands  to  millions  of  words), 
LSA  derives  a high-dimensional  semantic  space  which  permits  comparisons  of  the  semantic  similarity  of  words 
and  passages.  Words  and  passages  are  represented  as  vectors  in  the  space  and  their  similarity  is  measured  by 
the  cosine  of  their  contained  angle  in  the  semantic  space.  The  LSA  measured  similarities  have  shown  to  closely 
mimic  human  judgments  of  meaning  similarity  and  human  performance  based  on  such  similarity  in  a variety  of 
ways.  For  example,  after  training  on  about  2,000  pages  of  English  text,  it  scored  as  well  as  average  test-takers 
on  the  synonym  portion  of  TOEFL — the  ETS  Test  of  English  as  a Foreign  Language  (Landauer  & Dumais, 
1997),  After  training  on  an  introductory  psychology  textbook,  it  achieved  passing  scores  on  two  different 
multiple-choice  exams  used  in  introductory  psychology  courses  (Landauer,  Foltz  & Laham,  in  preparation). 
This  similarity  comparison  made  by  LSA  is  the  basis  for  performing  automated  scoring  of  essays  through 
comparing  the  similarity  of  meaning  between  essays. 

Automated  scoring  with  LSA 

While  much  of  the  emphasis  on  evaluating  written  work  has  examined  mechanical  features,  such  as 
grammar,  spelling  and  punctuation,  there  are  other  factors  involved  in  writing  a good  essay.  For  example  at  an 
abstract  level,  one  can  distinguish  three  properties  of  a student  essay  that  are  desirable  to  assess;  the  correctness 
and  completeness  of  its  contained  conceptual  knowledge,  the  soundness  of  arguments  that  it  presents  in 
discussion  of  issues,  and  the  fluency,  elegance,  and  comprehensibility  of  its  writing.  Evaluation  of  superficial 
mechanical  and  syntactical  features  is  fairly  easy  to  separate  from  the  other  factors,  but  the  rest — content, 
argument,  comprehensibility,  and  aesthetic  style— are  likely  to  be  difficult  to  pull  apart  because  each  influences 
the  other,  if  only  because  each  depends  on  the  choice  of  words. 

Although  previous  attempts  to  develop  computational  techniques  for  scoring  essays  have  focused 
primarily  on  measures  of  style  (e,g,.  Page,  1994),  indices  of  content  have  remained  secondary,  indirect  and 
superficial.  In  contrast  to  earlier  approaches,  LSA  methods  concentrate  on  the  conceptual  content,  the 
knowledge  conveyed  in  an  essay,  rather  than  its  style,  or  even  its  syntax  or  argument  structure. 

To  assess  the  quality  of  essays,  LSA  is  first  trained  on  domain-representative  text.  Examples  of 
domain-representative  texts  include  textbooks,  articles  or  samples  of  writing  that  a student  would  encounter 
during  learning  in  that  domain.  Based  on  this  training,  LSA  derives  a high-dimensional  semantic  representation 
of  the  information  contained  in  the  domain.  Words  can  be  represented  as  vectors  in  this  semantic  space,  with 
the  semantic  similarity  between  words  characterized  by  the  cosine  of  the  angle  between  the  vectors  for  those 
words.  Similarly,  student  essays  can  be  characterized  by  LSA  vectors  based  on  the  combination  of  all  their 
words.  These  vectors  can  then  be  compared  with  vectors  for  essays  or  for  texts  of  known  content  quality.  The 
angle  between  the  two  vectors  represents  the  degree  to  which  the  two  essays  discuss  information  in  a similar 
manner.  For  example,  an  ungraded  essay  can  be  compared  to  essays  that  have  already  been  graded.  If  the  angle 
between  two  essays  is  small  then  those  essays  should  be  similar  in  content.  Thus,  the  semantic  or  conceptual 
content  of  two  essays  can  be  compared  and  a score  derived  based  on  their  similarity.  Note  that  two  essays  can 
be  considered  to  have  almost  identical  content,  even  if  they  contain  few  or  none  of  the  same  words,  as  long  as 
they  convey  the  same  meaning.  This  feature  of  the  comparison  illustrates  that  LSA  is  doing  more  than  just 
keyword  matching;  it  is  matching  based  on  the  conceptual  content  in  the  essays. 


Evaluating  the  effectiveness  of  automated  scoring 

Based  on  comparing  conceptual  content,  several  techniques  have  been  developed  for  assessing  essays. 
Details  of  these  techniques  have  been  published  elsewhere  and  summaries  of  particular  results  will  be  provided 
below.  One  technique  is  to  compare  essays  to  ones  that  have  been  previously  graded,  A score  for  each  essay  is 
determined  by  comparing  the  essay  against  all  previously  graded  essays.  The  scores  from  some  number  of 
those  pre-graded  essays  (typically  10)  that  are  most  similar  to  the  essay  are  weighted  by  their  cosine  similarity 
to  the  essay  and  used  determine  the  score  for  the  essay  This  technique  provides  a "holistic"  score  measuring 
the  overall  similarity  of  content  (Laham,  1997;  Landauer,  Laham,  Rehder,  & Schreiner,  1997),  The  score  is 
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holistic  in  that  the  grade  for  an  essay  is  determined  based  on  how  well  the  overall  meaning  of  an  essay  matches 
that  of  previously  graded  essays. 

The  holistic  method  has  been  tested  on  a large  number  of  essays  over  a diverse  set  of  topics.  The 
essays  have  ranged  in  grade  level,  including  middle  school,  high  school,  college  and  college  graduate  level 
essays.  The  topics  have  included  essays  from  classes  in  introductory  psychology,  biology,  history,  as  well  as 
essays  from  standardized  tests,  such  as  analyses  of  arguments,  and  analyses  of  issues  from  the  Educational 
Testing  Service  (ETS)  Graduate  Management  Achievement  Test  (GMAT).  For  each  of  these  sets  of  essays, 
LSA  is  first  trained  on  a set  of  texts  related  to  the  domain.  Then  the  content  of  each  of  the  new  essays  is 
compared  against  the  content  of  a set  of  pre-graded  essays  on  the  topic.  In  each  case,  the  essays  were  also 
graded  by  at  least  two  course  instructors  or  expert  graders,  for  example  professional  readers  from  Educational 
Testing  Service,  or  other  national  testing  organizations.  Across  the  datasets,  LSA’s  performance  produced 
reliabilities  within  the  range  of  their  comparable  inter-rater  reliabilities  and  within  the  generally  accepted 
guidelines  for  minimum  reliability  coefficients.  For  example,  in  a set  of  188  essays  written  on  the  functioning 
of  the  human  heart,  the  average  correlation  between  two  graders  was  0.83,  while  the  correlation  of  LSA’s  scores 
with  the  graders  was  0.80.  A summary  of  the  performance  of  LSA’s  scoring  compared  to  the  grader-to-grader 
performance  across  a diverse  set  of  1205  essays  on  12  topics  is  shown  in  Figure  1.  The  results  indicate  that 
LSA’s  reliability  in  scoring  is  equivalent  to  that  of  human  graders. 

In  a more  recent  study,  the  holistic  method  was  used  to  grade  two  additional  questions  from  the 
GMAT  standardized  test.  The  performance  was  compared  against  two  trained  ETS  graders.  For  one  question, 
a set  of  695  opinion  essays,  the  correlation  between  the  two  graders  was  0.86,  while  LSA’s  correlation  with  the 
ETS  grades  was  also  0.86.  For  the  second  question,  a set  of  668  analysis  of  argument  essays,  the  correlation 
between  the  two  graders  was  0.87,  while  LSA's  correlation  to  the  ETS  grades  was  0.86.  Thus,  LSA  was  able  to 
perform  at  the  same  reliability  levels  as  the  trained  ETS  graders. 
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Figure  1.  Summary  of  reliability  results  (N  = 1205  Essays  on  12  Diverse  Topics) 

While  the  holistic  technique  relies  on  comparing  essays  against  a set  of  pre-graded  essays,  other 
techniques  have  been  developed  that  also  effectively  characterize  the  quality  of  essays.  A second  technique  is 
to  compare  essays  to  an  ideal  essay,  or  "gold  standard"  (Wolfe  et  al.,  1998).  In  this  case,  a teacher  can  write  his 
or  her  ideal  essay  and  all  student  essays  are  then  judged  based  on  how  close  they  are  to  the  teacher’s  essay.  In 
two  final  techniques,  essays  can  be  compared  to  portions  of  the  original  text,  or  compared  to  sub-components 
of  texts  or  essays  (Foltz,  1996;  Foltz,  Britt  & Perfetti,  1996).  In  this  final  componential  approach,  individual 
sentences  from  a student's  essay  can  be  compared  against  a set  of  predetermined  subtopics.  This  permits  the 
determination  of  whether  an  essay  sufficiently  covers  those  subtopics.  LSA  derived  scores  based  on  the  degree 
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of  coverage  of  subtopics  in  essays  show  equivalent  correlations  with  human  graders  as  graders  correlate  with 
each  other 


Anomalous  essay  checking 

While  it  is  important  to  verify  the  effectiveness  of  computer-based  essay  grading,  it  is  also  important 
that  such  a grader  be  able  to  determine  if  it  can  not  grade  an  essay  reliably.  Thus,  a number  of  additional 
techniques  have  been  developed  that  can  detect  ’'anomalous"  essays.  If  an  essay  is  sufficiently  different  from 
the  essays  for  which  it  has  been  trained,  the  computer  can  flag  it  for  human  evaluation.  We  currently  flag  ones 
that  are  highly  creative,  off  topic,  or  violate  standard  formats  or  structures  for  essays.  In  addition,  the  computer 
can  determine  whether  any  essay  is  too  similar  to  other  essays  or  to  the  textbook.  The  program  is  thus  able  to 
detect  different  levels  of  plagiarism.  If  an  essay  is  detected  as  being  anomalous  for  any  reason,  the  essay  can  be 
automatically  forwarded  to  the  instructor  for  additional  evaluation. 

Experiences  in  the  classroom:  grading  and  feedback 

Over  the  past  two  years,  the  Intelligent  Essay  Assessor  has  been  used  in  a course  in  Psycholinguistics 
at  New  Mexico  State  University.  Designed  as  a web-based  application,  it  permits  students  to  submit  essays  on 
a particular  topic  from  their  web  browsers.  Within  about  20  seconds,  students  receive  feedback  with  an 
estimated  grade  for  their  essay  and  a set  of  questions  and  statements  of  additional  subtopics  that  are  missing 
from  their  essays.  Students  can  revise  their  essays  immediately  and  resubmit.  A demonstration  is  available  at: 
http://psvch.nmsu.edu/essay 

To  create  this  system,  LSA  was  trained  on  portion  (four  chapters)  of  the  psycholinguistics  textbook 
used  in  the  class.  The  holistic  grading  method  was  used  to  provide  an  overall  grade  for  any  essay.  In  this 
method,  each  essay  was  compared  against  40  essays  from  previous  years  that  had  been  graded  by  three  different 
graders.  To  verify  the  effectiveness  of  this  approach  for  providing  accurate  grades,  the  average  correlation 
among  the  three  human  graders  was  0.73  while  the  average  correlation  of  LSA’s  holistic  grade  to  the  individual 
graders  was  0.80.  To  provide  feedback  about  missing  information  in  each  essay,  individual  sentences  in  each 
essay  were  compared  against  sentences  that  corresponded  to  subtopics  of  the  essay  topic.  If  no  sentence  was 
found  that  matched  a subtopic  then  the  student  received  feedback  about  the  fact  that  their  essay  did  not  properly 
cover  that  subtopic. 

Students  were  permitted  to  use  the  system  independently  to  write  their  essay  and  were  encouraged  to 
revise  their  essays  and  resubmit  them  to  the  computer  as  many  times  as  they  wanted  until  they  were  satisfied 
with  their  grades.  The  average  grade  for  the  students’  first  essays  was  85  (out  of  1 00).  By  the  last  revision,  the 
average  grade  was  92.  Students’  essays  improved  from  revision  to  revision,  with  the  improvements  in  scores 
ranging  from  0 to  33  points  over  an  average  of  3 revisions.  An  additional  trial  is  underway  with  a similar 
system  in  a Boulder  Colorado  middle  school  in  which  students  summarize  texts  on  sources  of  energy. 
Preliminary  results  indicate  that  two-thirds  of  the  students  were  able  to  improve  their  summaries  based  on  the 
lEA's  feedback. 

In  both  the  undergraduate  and  middle  school  trials,  students  and  teachers  have  enjoyed  and  valued 
using  the  system.  A survey  of  usability  and  preferences  for/against  the  system  in  the  psycholinguistics  course 
showed  that  98  percent  of  the  students  indicated  that  they  would  definitely  or  probably  use  such  a system  if  it 
were  available  for  their  other  classes.  Overall,  the  results  show  that  the  lEA  is  successful  at  helping  students 
improve  the  quality  of  their  essays  through  providing  immediate  and  accurate  feedback. 


Implications  of  computer-based  essay  grading  for  education 

There  exist  a variety  of  applications  within  education  to  which  the  lEA  can  be  applied.  At  a minimal 
level,  the  lEA  can  be  used  as  a consistency  checker,  in  which  the  teacher  grades  the  essays  and  then  the  IE  A re- 
grades the  essays  and  indicates  discrepancies  between  the  two  grades.  Because  the  lEA  is  not  influenced  by 
fatigue,  deadlines,  or  biases,  it  can  provide  a consistent  and  objective  view  of  the  quality  of  the  essays.  The 
lEA  can  further  be  used  in  large  scale  standardized  testing  or  large  classes,  by  either  providing  consistency 
checks  or  serving  as  an  automatic  grader. 
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At  a more  interactive  level,  the  lEA  can  be  used  to  help  students  improve  their  writing  through 
assessing  and  commenting  on  their  essays.  By  providing  instantaneous  feedback  about  the  quality  of  their 
essays,  as  well  as  indications  of  information  missing  from  their  essays,  students  can  use  the  lEA  as  a tool  to 
practice  writing  content-based  essays.  Due  to  a lack  of  sufficient  teachers  and  aides  in  many  large  section 
courses,  writing  assignments  and  essay  exams  have  often  been  replaced  by  multiple  choice  questions.  The  lEA 
permits  students  to  receive  writing  practice  without  requiring  all  essays  to  be  evaluated  by  the  teachers. 
Because  the  lEA's  evaluations  are  immediate,  students  can  receive  feedback  and  make  multiple  revisions  over 
the  course  of  one  session.  This  overall  approach  is  consistent  with  the  goals  of  the  "Writing  across  the 
Curriculum"  approach  by  allowing  the  introduction  of  more  writing  assignments  in  courses  outside  of  English 
or  the  humanities.  Thus,  the  IE  A can  serve  to  improve  learning  through  writing. 

Finally,  the  lEA  can  be  integrated  with  other  software  tools  for  education.  In  most  software  for 
distance  education,  tools  for  administering  writing  assignments  are  often  neglected  in  favor  of  tools  for  creating 
and  grading  multiple  choice  exams.  The  lEA  permits  writing  to  be  a more  central  focus.  For  example,  in  web- 
based  training  systems,  the  lEA  can  be  incorporated  as  an  additional  module  which  would  permit  teachers  to 
add  writing  assignments  for  their  web  courses.  Essays  can  be  evaluated  on  a secure  server  and  scores  can  be 
returned  directly  either  to  the  students  or  to  the  teachers  along  with  the  essays.  Textbook  supplements  can 
similarly  use  the  IE  A for  automated  study  guides.  At  the  end  of  chapters,  students  can  be  asked  to  write  essay 
questions  addressing  topics  covered  in  the  chapter.  Based  on  an  analysis  of  their  essays,  the  software  can 
suggest  sections  of  the  textbook  that  the  students  needs  to  review  before  they  take  their  exams. 


Conclusions 

To  summarize  the  process  of  grading  with  the  lEA,  the  IE  A is  first  trained  on  some  domain 
representative  text  such  as  a textbook,  samples  of  writings,  or  even  a large  number  (300  or  more)  essays  on  a 
topic.  Second,  the  lEA  is  provided  with  one  or  more  pre-graded  essays  on  the  topic  to  be  graded.  Once  these 
pieces  of  information  are  processed,  the  lEA  is  able  to  provide  accurate  characterizations  of  the  quality  of 
essays  on  the  same  topic.  Domain  representative  texts  are  easily  available  in  electronic  form  and  essays  can 
now  be  easily  converted  or  directly  entered  by  students  in  electronic  form.  Thus,  it  is  highly  feasible  to  develop 
automated  essay  graders  for  a large  number  of  topics  across  many  domains.  Although  the  lEA  requires  a large 
amount  of  computer  processing  power,  it  can  be  run  from  a centralized  server,  allowing  students  to  access  it 
from  any  web  browser. 

The  Intelligent  Essay  Assessor  presents  a novel  technique  for  assessing  the  quality  of  content 
knowledge  expressed  in  essays.  It  permits  the  automatic  scoring  of  short  essays  that  would  be  used  in  any  kind 
of  content-based  courses.  Based  on  evaluations  of  the  lEA  over  a wide  range  of  essay  topics  and  levels,  the 
lEA  proves  as  reliable  at  evaluating  essay  quality  as  human  graders.  Its  scores  are  also  highly  consistent  and 
objective.  While  the  lEA  is  not  designed  for  assessing  creative  writing,  it  can  detect  whether  an  essay  is 
sufficiently  different  that  it  should  be  graded  by  the  instructor.  The  lEA  can  be  applied  for  both  distance 
education  and  for  training  in  the  classroom.  In  both  cases  it  may  be  used  for  assessing  students  content 
knowledge  as  well  as  providing  feedback  to  students  to  help  improve  their  learning.  Overall,  the  lEA  permits 
increasing  the  amount  of  writing  assignments  without  overly  increasing  the  grading  load  on  the  teacher. 
Because  writing  is  a critical  component  towards  helping  students  acquire  both  better  content  knowledge  and 
better  critical  thinking  skills,  the  lEA  can  serve  as  an  effective  tool  for  increasing  students'  exposure  to  writing. 
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Abstract:  Building  a visual  navigation  aid  system  for  WWW-based  hypertext  is 
difficult  due  to  the  implicitly  structured  nature  of  associative  hypertext.  The  task  can  be 
made  significantly  easier  by  identifying  and  explicitly  stating  the  latent  hierarchical  and 
pre-set  reading  order  of  hypertext  and  by  reducing  the  number  of  relations  presented.  A 
simple  formalism  and  a method  of  constructing  a visual  graph-based  navigation  tutoring 
system  for  WWW  is  presented.  The  work  is  demonstrated  in  practice  by  briefly 
introducing  a visual  navigation  system  prototype,  the  Navtutor  applet,  that  can  be  used 
with  existing  Web  learning  materials  with  little  extra  work. 


Introduction 

Today  World  Wide  Web  (WWW)  is  filled  with  information  presented  in  hypertext  format.  Thanks 
to  the  global  identification  and  retrieval  mechanisms  of  documents  and  the  standard  hypertext  markup 
language  (HTML),  WWW  potentially  offers  means  to  create  well-structured  global  hypermedia.  From  the 
authors'  point  of  view,  however,  to  date  the  syntax  of  HTML  has  not  really  supported  nor  been  used  for, 
e.g.,  explicitly  defining  document  categories  or  linking  semantics  needed  for  creating  navigation  tools 
visualizing  logical  contents.  This  has  led  developers  to  search  for  different  types  of  ad  hoc  solutions  for 
structuring  and  coding  their  WWW  material. 

Structured  hypertext  is  typically  captured  in  a shape  of  different-looking  WWW  sites  with 
(picture-based)  link  lists  and  overview  diagrams  similar  to  the  navigation  structures  proposed  in  the  early 
90's  (Nielsen,  1994).  Systematic  use  of  dynamic,  visual  graph-based  navigation  methods  has  been  little, 
most  of  the  educational  navigation  systems  rely  on  text-based  linking  and  image  map  techniques  (Turau, 
1998).  Due  to  the  enormous  amount  of  references,  visualizing  complex  associative  linking  is  generally 
considered  impractical  as  a navigation  metaphor.  Educational  material  is  still  at  largely  presented  using  the 
book  or  desktop  metaphors  which,  in  effect,  provide  little  basis  building  structural  mental  models  of  the 
fundamentally  non-linear  (structured)  hypertext.  The  straightforward  use  of  book  metaphor  with  linked 
keywords,  for  instance,  adds  little  to  using  well-written  conventional  paper  materials.  Similarly,  target- 
oriented  learning  is  very  difficult  if  it  is  simply  based  on  flat  associative  hypertext.  A dynamic,  map-based 
navigation  metaphor  should  help  users  building  concise  mental  models  to  construct  knowledge  of  the 
structure  and  relations  of  the  hypertext  at  hand  (Preece  et  al,  1 994). 

Motivation  for  us  finding  general  solutions  for  providing  students  visual  graph-based  navigation 
tools  arose  when  developing  Web-based  learning  material  for  undergraduate-level  mathematics  (see,  e.g., 
(Pohjolainen  et  al,  1996),  (Nykanen  & Pohjolainen,  1998)  and  (Luodeslampi  et  al,  1998)).  Simply 
producing  hierarchical,  associatively  linked  WWW  pages  seemed  to  fail  to  achieve  the  latent  potential  of 
using  computers  to  present  structured  learning  material.  We  felt  that  in  order  to  make  the  chosen  structure 
"live",  we  should  effectively  promote  the  use  of  the  fundamentally  non-linear  yet  logical  structure  of 
hypertext.  We  hoped  to  achieve  this  by  providing  students  maps  to  visualize  the  "expert's"  view  on  the 
structure  of  mathematics  (within  a given  course).  There  already  exists  case  studies  and  tools  of  visualizing 
navigation  and  searching  in  (educational)  contents  (see,  e.g.,  (Chavero  et  al,  1998)  and  (Akoulchina  & 
Ganascia,  1997))  but  our  goal  was  to  investigate  and  design  a general  WWW-based  navigation  system  to  be 
used  in  parallel  with  our  prototype  of  a Web-based  open  learning  environment  in  the  future  (Nykanen  & 
Ala-Rantala,  1997). 

During  the  process  of  structuring  our  learning  material,  we  learned  two  major  lessons.  First,  more 
detailed  structure  to  supporting  navigation  we  hoped  to  construct,  more  problems  we  encountered  in 
practice.  The  "goodness"  of  a definition  and  granularity  of  hypertext  depends  on  many  things  since 
structuring  material  can  be  based  on  a number  of  perspectives.  Historical,  purely  logical  or  pedagogical 
approaches  do  not  necessarily  yield  to  the  same  structure.  The  second  and  the  more  important  observation 
was,  however,  that  after  a guideline  general  enough  for  structuring  material  sufficient  for  our  purposes  was 


found,  it  was  rapidly  applicable  to  number  of  other  subjects,  besides  mathematics,  as  well.  To  us  the 
solution  was  to  identify  and  clarify  the  implicit  structure  of  (educational)  hypertext.  The  main  idea  is  to 
recognize  and  explicitly  state  the  logical  reading  order  of  "key"  pages  and  to  cluster  them  into  hierarchical 
groups. 

After  finding  a simple  syntactical  way  to  clarify  the  essential  structure  of  hypertext,  we  designed 
and  implemented  a simple  prototype  navigation  tutoring  system  to  exploit  and  test  our  ideas.  Our  software 
prototype,  the  Navtutor  applet,  was  designed  to  function  in  parallel  with  existing  Web  clients. 

The  purpose  of  this  paper  is  to  motivate  and  present  the  idea  of  constructing  a visual  graph-based 
navigation  tutoring  system  for  WWW,  and  to  suggest  means  to  do  this  both  in  theory  and  in  practice. 


Introducing  Partial  Order  and  Hierarchy  to  Hypertext 

Creating  a good  general  navigation  system  for  HTML-based  hypertext  is  very  difficult.  The  basic 
linking  method  using  associative  linking  is  extremely  general  by  definition  and  introduces  little  structure  to 
base  a navigation  support  system  on.  In  order  to  construct  a general  device  for  aiding  navigation,  an 
abstracted  reference  structure  of  hypertext  is  needed.  In  this  chapter  we  will  briefly  describe  the  syntax  of  a 
hierarchical  prerequisite  graph  (HPG)  that  is  used  to  model  the  hierarchical  and  ordered  structure  of  a 
given  hypertext. 


Implicit  Structure  of  (Educational)  Hypertext 

The  obvious  method  for  developing  syntax  for  practical  visualizing  of  hypertext  is  to  try  to  reduce 
the  number  of  relations  presented.  Motivation  for  this  is  found  from  the  contemporary  WWW  hypertext.  In 
practice  most  of  the  educational  WWW  sites  implicitly  add  two  types  of  additional  semantic  relations  to 
hypertext 

1)  by  grouping  individual  pages  of  hypertext  into  hierarchical  clusters,  and 

2)  by  introducing  a partial  reading  order  between  different  pages  of  hypertext. 

For  example,  most  WWW  sites  provide  users  a front  page  that  divides  information  into  various 
sections  each  of  which  consisting  of  a number  or  WWW  which  include  information  which  is  sensible  only 
when  read  in  some  appropriate  order.  Obviously  other  types  of  link  semantics  are  used  also,  including 
references,  notes,  examples,  etc. 

The  important  notice  is  that  the  seemingly  arbitrary  linking  does  not  essentially  reflect  the  logical 
structure  of  the  WWW  site.  From  the  "definition"  of  hypertext  we  know  that  the  structure  of  hypertext 
actually  predefines  the  conceptual  limits  of  the  associative  structure  of  hypertext  by  providing  users  links  to 
follow.  Similarly,  logical  reading  order  precedes  the  associative  structure  of  hypertext.  This  implies  that  the 
associative  structure  of  hypertext  itself  must  have  a backbone,  a core  structure,  which  lays  down  the  basic 
pre-set  logical  structure  of  the  hypertext.  After  all,  if  this  wasn't  true,  the  hypertext  would  actually  consist  of 
some  sort  of  an  associative  database  of  completely  independent  records  of  the  same  level  without  any 
specific  reading  order(s). 

Thus  the  associative  structure  of  hypertext  is  associative  only  from  the  viewpoint  of  the  users' 
limited  dynamic  control,  not  from  the  viewpoint  of  the  core  structure  of  hypertext  itself  The  core  structure 
of  hypertext  is  essentially  hierarchical  and  pre-set.  It  turns  out,  that  since  our  goal  is  to  introduce  a 
navigation  aid  for  (educational)  hypermedia,  this  observation  is  very  useful  to  us. 


Hierarchical  Prerequisite  Graph 

Technically  HPG  is  a semantic  network  consisting  of  vertices  corresponding  to  each  of  the  pages 
of  hypertext  and  directed  labeled  edges  defining  two  types  of  relations  between  vertices.  These  relations, 
partial  order  and  hierarchical  inclusion,  are  selected  according  to  the  desired  logical  structure  of  hypertext 
and  do  not  necessarily  include  all  the  associative  relations  between  the  pages  of  original  hypertext  nor 
consist  only  of  them.  Figure  1 presents  an  example  of  a HPG  consisting  of  nine  vertices  and  fifteen  edges  or 
relations  between  vertices. 
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Layer  0 


hierarchical  inclusion 
(reading)  order 


Figure  1:  An  example  of  a hierarchical  prerequisite  graph. 


The  example  should  be  interpreted  as  following:  the  referred  hypertext  consists  of  two  logical 
groups  of  pages,  H and  I (vertex  sets  {A,B,C}  and  {D,E,F,G}  on  layer  0)  that  have  a definite  prerequisite 
order  in  which  pages  are  assumed  to  be  read.  The  pages  H and  I are  also  assigned  a prerequisite  order  on 
layer  1 . In  practice  the  example  could  model,  e.g.,  a hypertext  of  two  ’’sections"  with  introductions,  each  of 
consisting  of  number  of  "topics’’. 

The  structure  of  an  HPG  is  straightforwardly  generalized  from  this  by  adding  more  hierarchical 
layers  (for  instance  in  the  previous  example  by  further  grouping  H and  1 to  some  group  J on  layer  2). 
Relations  of  HPG  are  constrained  as  following;  vertices  on  each  layer  form  a directed  acyclic  graph  (not 
necessarily  connected)  according  to  the  order  relations.  Each  vertex  on  a layer  i is  connected  to  exactly  one 
vertex  on  layer  i+1  or  to  nothing  ("top  layer")  according  to  the  hierarchical  inclusion  relations.  In  addition, 
each  layer  includes  only  the  minimum  number  of  edges  needed  to  present  the  order  relations  which  are 
assumed  to  be  transitive.  If  the  prerequisite  structure  is  consistent,  the  direction  of  associative  reference 
links  should  typically  be  against  the  direction  of  prerequisite  links  (order  relations).  Similarly  references  to 
concepts  of  higher  or  lower  levels  should  refer  to  different  layers  (hierarchical  relations)  of  the  HPG. 


Construction  of  Hierarchical  Prerequisite  Graphs 

From  the  formal  point  of  view  a HPG  can  be  described  as  a collection  of  directed  acylic  graphs 
G=(Go,  G|, ...,  Gi, ..  ,Gn)  where  each  Gj  corresponds  to  the  prerequisite  graph  at  layer  i (see  figure  1).  Each 
graph  Gi  consists  of  a set  of  vertices  V(Gj),  V(G)  = V(Go)  u V(Gi)  u ..  .u  V(Gn),  a set  of  edges  E(G|), 
E(G)  = E(Go)  u E(Gi)  u ...  u E(Gn),  and  a set  of  hierarchical  references  H(Gi)czV(Gi+i)  to  layer  i+l.  For 
Gn,  H(Gn)=0  because  the  vertices  on  layer  N have  no  hierarchical  parents.  Note  that  an  arbitrary  Gi  is  not 
assumed  to  be  connected.  This  implies  that  it  may  consist  of  several  components;  an  arbitrary  vertex 
aG  V(Gi)  may  not  be  accessible  from  vertex  bG  V(Gi). 

Prerequisite  graph  Gi  is  simple,  i.e.,  it  includes  no  parallel  arcs  nor  loops.  It  also  agreed  that 
prerequisite  semantics  define  no  directed  cycles.  From  the  definition  of  a HPG  it  also  directly  follows  that 
for  each  Gi  no  vertex  is  both  the  start-  and  endpoint  of  a directed  walk.  Graph  Gi  needs  not  necessarily  to  be 
a rooted  tree,  however,  since  it  may  include  more  than  one  root  vertices.  Figure  2a  presents  an  example  of  a 
simple  prerequisite  graph. 


Figure  2:  The  structure  of  a prerequisite  graph,  an  idea  of  a method  for  reducing  redundant 
semantic  information,  non-planar  K3  3,  and  clique  T3. 


Constructing  prerequisite  graphs  based  on  a given  directed  acyclic  graph  is  easy.  The  prerequisite 
(order)  relation  is  transitive,  which  provides  a straightforward  method  of  removing  semantically  redundant 
edges  (see  figure  2b).  Edges  are  removed  if  and  only  if  the  accessibility  of  vertices  does  not  change. 

Unfortunately,  from  the  viewpoint  of  visual  presentation  of  (even  small  portions  of)  prerequisite 
graphs,  the  given  prerequisite  graph  Gj  is  not  necessarily  planar  (like  the  example  in  the  figure  2a),  that  is, 
it's  sometimes  impossible  to  draw  an  (isomorphic)  2D  presentation  of  the  graph  with  no  crossing  edges. 
Figure  2c  demonstrates  this  with  a counter-example  of  a directed  graph  K3  3.  The  prerequisite  graph  has, 
however,  some  nice  properties  that  can  help  parsing  clear  visual  presentations  of  the  graph.  For  example,  it 
follows  from  the  definition  that  a prerequisite  graph  does  not  include  the  (directed)  clique  T3  (see  figure  2d) 
as  a sub-graph.  This  motivates  finding  appealing  local  isomorphic  representations  of  the  prerequisite  graph. 
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Formulating  Visualizations 

Our  goal  is  to  provide  a visual  navigation  aid  for  navigating  hypertext  that  can  be  modeled  using 
some  appropriate  HPG.  For  simplicity,  we  choose  to  visualize  the  given  HPG  as  one  layer  at  a time,  by 
dynamically  parsing  a visualization  graph  S of  a subgraph  of  the  Gj  selected. 

The  main  idea  to  visualize  the  structure  and  context  of  the  prerequisite  graph  is  to  use  direction 
metaphor,  our  intuitive  notion  of  physical  direction,  to  encode  the  semantic  information.  The  portion  of  the 
Gi  to  be  visualized  is  drawn  on  a (bound)  2D  surface  with  the  selected  vertex  in  the  middle.  Adjacent 
vertices  are  "grown”  in  north  and  south  directions,  according  to  the  prerequisite  relations  in  G,,  Figure  3a 
presents  an  example  of  a such  visualization. 


(a)  (b) 

Figure  3:  Example  of  visualizing  a subgraph  of  Gj  (selected  location  vertex  a is  marked  with  a 
dashed  circle)  and  illustration  of  the  direction  metaphor 

In  the  example  above,  the  semantic  interpretation  of  the  order  relation  is  clear.  Vertices 
prerequisite  (and  close)  to  the  selected  vertex  a are  positioned  south  and  vertices  accessed  from  a north 
from  a.  Because  S includes  no  cliques  T3,  crossing  edges  (if  any)  will  be  appear  in  S only  from  two  vertices 
away  from  a given  vertex. 

Direction  and  location  metaphors  try  to  provide  the  idea  of  direction  and  context  for  the  user  by 
dynamically  generating  a sequence  of  visualization  graphs  during  the  navigation.  Each  time  a new  location 
vertex  is  selected,  each  visualization  "points"  to  the  "center"  of  the  given  prerequisite  graph  (see  figure  3b), 
During  the  navigation,  the  view  "rotates"  around  the  "center"  according  to  the  prerequisite  relations.  The 
visualization  graph  also  naturally  provides  graphical  encoding  for  "closeness"  by  including  only  vertices 
"close"  to  the  selected  vertex  (in  the  sense  of  the  explicit  prerequisite  relations). 

However,  in  order  to  provide  user  a intuitive  method  for  navigating,  further  remarks  are  needed. 
Merely  creating  a balanced  2D  presentation  of  Gj  would  probably  just  confuse  the  user:  navigation  process 
is  typically  a sequence  of  selecting  adjacent  location  vertices  that  should  be  visualized  in  the  navigation 
process  as  moving  to  a certain  (abstract)  direction.  Presenting  user  a sequence  of  rapidly-changing 
visualizations  will  not  help  developing  a mental  (structural)  model  of  the  relations  of  the  graph  because  by 
doing  so  the  visualization  would  change  dramatically  during  each  navigation  step.  Navigation  systems 
ought  not  to  create  a set  of  independent  balanced  graphs  but  to  process  an  animated  sequence  of  dynamic 
graphs  instead. 

In  practice  this  suggests  that  the  visualization  should  be  parsed  dynamically  according  to  the  user's 
location  and  navigation  history  at  the  hypertext.  Formally  this  is  done  by  defining  a visualization  function 
vp  which,  according  to  the  user's  current  location  in  the  hypertext  and  some  parameters  P,  fixes  a subgraph 
of  Gj  to  be  visualized  as  a visualization  graph  S=vp(a),  When  user  navigates  in  the  hypertext,  visiting  nodes 
ai,  a2,  ...aj,  aM,  that  make  up  the  total  navigation  history,  the  process  generates  a sequence  of 
visualization  graphs  Si,  S2,  Sj,  ,,,,  Sm  to  be  show  to  the  user.  Now  each  Sj  is  a graphical  presentation  of 
selected  subgraph  of  some  Gi  with  fixed  relative  vertex  positions,  color  coding  etc,  set  in  P,  Typically  Sj 
would,  e,g,,  include  (provide  a one-to-one  mapping)  only  the  vertices  b^GGi  such  that  the  distance  from  b^ 
to  aj  is  less  than  some  fixed  value,  positioned  to  a plane  with  descriptive  labels,  according  to  some  balanced 
graph  drawing  algorithm  satisfying  the  idea  presented  in  the  figure  3a,  Note  that  the  vertices  of  different 
visualization  graphs  need  not  to  belong  to  the  same  layer. 

Each  transition  is  then  visualized  by  using  in-between  animation  frames  from  one  visualization 
graph  to  another.  Plane  transitions  along  the  "partial  order"  edges  are  visualized  by  moving  and 
transforming  Sj  to  Sj+i  by  moving,  hiding  and  showing  appropriate  vertices.  Vertical  transitions  along  the 
"hierarchical  inclusion"  edges  are  visualized  by  conventional  zoom  in  and  out  metaphors. 

This  scheme  can  be  further  developed  by  making  additions  to  the  visualization  process.  We 
connect  the  last  step  of  the  total  navigation  history  to  the  visualization  function  vp  by  exploiting  the  latest 
visualization  graph  Sj.i  (j>l)  which  makes  it  possible  to  locally  optimize  the  sequence  of  visualization 
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graphs  in  order  to  keep  the  sequence  of  visualization  graphs  as  smooth  as  possible  during  navigation.  This 
way  the  user  has  two  ftindamentally  alternative  options  for  visualizing  the  navigation  process: 

1)  the  user  can  choose  each  Sj=v/>(ai)  to  be  first  globally  optimized  and  then  presented  by  using 
animated  in-between  frames  from  Sj.i  (if  possible,  that  is,  j>l),  or 

2)  the  user  can  choose  each  Sj=v’/j(ai,Sj.i)  to  be  locally  optimized  to  match  Sj.i  as  closely  as  possible 
(if  possible,  that  is,  j>l)  and  then  presented  by  using  animated  in-between  frames  from  Sj.). 

The  two  options  introduce  two  different  ways  of  navigating.  The  former  navigation  method  is 
transition  invariant  always  producing  the  same  visualization  Sj=V/>(a)  no  matter  the  path  the  user  took  to  get 
to  a.  The  latter  navigation  method,  however,  is  transition  variant.  In  this  case,  the  path  the  user  takes  to  get 
to  the  vertex  a may  cause  visualization  Sj=v’/j(ai,Sj.])  to  look  different,  according  to  the  previous  vertex  aj^i 
in  the  total  navigation  history.  Note  that  both  types  of  visualizations  should  be  isomorphic  to  each  other;  the 
essential  difference  between  different  S’s  produced  is  the  calculated  vertex  positions  on  a plane. 

The  decision  between  transition  invariant  and  variant  navigation  modes  is  a trade-off  between 
smooth  in-between  animation  and  global  appearance  of  visualization  graphs.  The  former  provides  user 
intuitive  navigation  sequences,  but  can  sometimes  confuse  the  user  by  introducing  different  layouts  for 
visualizations  depending  on  the  navigation  history.  The  latter,  on  the  other  hand,  forces  the  visualizations 
always  to  look  similar,  but  can  sometimes  make  the  in-between  transitions  to  look  complicated. 


Demonstration:  Navtutor  Applet 

To  test  the  idea  of  exploiting  order  and  hierarchy  information  in  WWW  sites,  a prototype 
navigation  support  system,  Navtutor,  was  written  and  tested  with  our  existing  WWW-based  learning 
material  (see  figure  4).  Test  material  consists  of  our  experimental  learning  material  of  elementary  matrix 
algebra  (in  Finnish). 


irmi 


It* 

justly 


Figure  4:  Navtutor  window  on  top  of  a Netscape  browser  window. 

The  basic  idea  is  to  describe  a educational  WWW  site  in  the  terms  of  a HPG  description  file 
providing  structural  data  for  the  Navtutor  applet.  For  demonstration  purposes  we  designed  a small  "learning 
structure"  for  the  introduction  of  a matrix  algebra  course  by  defining  the  prerequisite  and  hierarchical  order 
in  which  we  thought  different  concepts  should  be  studied. 

The  basic  navigation  is  carried  out  with  a standard  Web  browser.  Browser  process  provides  user 
the  basic  navigation  functionality  essentially  by  fetching  and  presenting  HTML-encoded  documents  from 
the  WWW,  based  on  the  browser's  graphical  user  interface.  When  stumbled  across  a WWW  site  including 
appropriate  Navtutor  structural  information,  the  browser  process  loads  and  initializes  the  Navtutor  applet 
and  provides  it  information  about  the  browser  windowing  system  references  (in  order  to  display  pages  at 
request  and  notify  events)  and  instructs  what  hypertext  HPG  description  to  use. 

The  graphical  user  interface  of  the  Navtutor  applet  consists  of  two  parts:  map  view  and  control 
toolbars.  The  map  view  presents  a dynamic  visualization  of  the  selected  portion  of  the  HPG  describing  the 
core  structure  of  the  hypertext  (see  figure  3a).  When  user  selects  a vertex,  the  map  view  smoothly  animates 
to  the  selected  vertex  according  to  direction  metaphor  (see  figure  3b).  In  each  navigation  step  the 
visualization  is  dynamically  parsed  to  reflect  the  contextual  state  of  the  users  "location"  in  the  HPG  model. 
Vertical  navigation  from  HPG  layer  to  another  is  visualized  by  zooming  in  or  out  from  the  selected  vertex. 

In  addition  to  the  basic  navigation  functionality,  Navtutor  provides  also  basic  methods  for  more 
structural  navigation  by  using  explicit  navigation  modes.  Three  modes  of  navigation  are  supported:  explore 
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mode,  local  mode,  and  target  mode.  In  the  explore  mode,  the  visualization  graph  is  constructed  from  all  of 
the  vertices  (pages)  "close"  to  the  selected  node  by  using  simple  breadth-first  search.  In  the  local  mode, 
only  vertices  within  the  current  cluster  (hierarchical  group)  are  displayed,  and  finally,  in  the  target  mode, 
only  vertices  on  a path  to  a previously  selected  target  vertex,  are  displayed.  In  practice  this  means  that  users 
are  able  to  limit  the  information  present  in  the  map  view  to  meet  their  needs  when  using  different 
navigation  (and  learning)  strategies. 


Discussion 

Finding  a all-round  format  for  describing  educational  hypertext  structure  is  very  difficult.  Based 
on  our  experiences,  designing  structures  for  educational  material  is  hard,  but  producing  (or  converting) 
learning  material  within  the  given  format  can  be  equally  hard,  or  even  harder.  As  a rule  of  thumb  it  would 
seem,  that  instead  of  re-writing  existing  WWW  material,  using  relatively  light  additional  information 
structures  may  turn  out  to  be  a "good  enough"  solution  in  practice.  So  far  our  experiences  from  Navtutor 
strongly  support  this  point  of  view.  With  very  little  structuring,  most  of  our  educational  WWW  material 
should  benefit  from  the  single  navigation  support  system  implemented. 

Of  course,  one  should  never  apply  new  methods  blindly.  At  this  time  of  development,  we  are 
gathering  experiences,  based  of  which  our  ideas  are  later  revised  and  systems  further  developed.  One  very 
interesting  aspect  of  the  Navtutor  software  is  the  interface  with  our  software  platform  prototype  of  an  open 
learning  environment  (see  (Nykanen  & Ala-Rantala,  1997)). 

Finally  it  should  be  heavily  emphasized  that  we  are  not  implying  that  existing  navigation  systems 
should  be  replaced  with  Navtutor-like  applications  and  thrown  away,  but  rather  suggest  a parallel  and  a 
general  alternative  for  effective  and  context-based  navigation.  Empirical  research  supports  the  view  that 
map-based  navigation  aids  are  at  their  best  when  used  in  together  with,  e.g.,  book  metaphors,  not  alone 
(Leader  & Klein,  1996). 
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Abstract:  The  World  Wide  Web  (WWW)  has  been  heralded  as  a significant  opportunity  to 
deliver  successful  tele-educational  courseware  and  learning  experiences.  The  central  tenant  of 
the  WWW  being  that  everyone  (educationalists)  can  become  a publisher  of  educational 
material.  However,  although  possessing  technical  or  educational  skills,  WWW  user  interface 
design,  usability  and  good  pedagogical  layout  of  WWW  based  content,  are  skills  which  are 
frequently  weak  amongst  many  of  this  community.  This  paper  researches  and  proposes  an 
easy  to  use  ‘usability’  criteria  and  questionnaire,  which  facilitates  the  evaluation  of 
educational  WWW  courseware.  The  paper  describes  the  implementation  of  a WWW  based 
server  system  for  the  questionnaire.  The  paper  also  describes  how  the  system  was  trialled  on 
an  already  proven  WWW  courseware  system  and  still  provided  indications  where 
improvements  could  be  performed.  The  paper  concludes  with  an  assessment  of  the  benefits  of 
iteratively  utilising  such  a system  to  improve  WWW  courseware. 


1 Introduction 

The  last  three  years  has  seen  a rapid  increase  in  the  use  of  WWW  based  educational  tools,  courseware  and 
environments  (Wade  et  al.  98).  The  applicability  of  WWW  as  a medium  for  educational  experiences  has  been 
well  documented  as  it’s  ubiquity  of  presence,  it  rendering  capability  for  graphics,  text,  animation,  video  and 
audio,  and  it’s  potential  for  interactivity  (both  between  participant  and  educational  content  as  well  as  participant 
to  participant)  (Maurer  97),  (Turoff  95).  However,  it  is  also  well  accepted  that  educational  technology,  over  the 
last  thirty  years,  has  failed  to  achieve  the  revolutions  that  were  initially  forecast  (Bates  95).  There  have  been 
many  reasons  proposed  for  the  failures  of  various  educational  systems,  the  two  most  widely  reported  being  (i) 
the  lack  of  pedagogical  support  within  the  educational  systems  (ii)  the  difficulties  experienced  by  learners  in 
using  such  systems.  This  paper  addresses  this  second  most  common  cause  of  failure.  Usability  as  defined  by 
the  International  Standards  Organisation  (ISO)  is  the  effectiveness,  efficiency  and  satisfaction  with  which 
specified  users  can  achieve  specified  goals  in  particular  environments.  The  paper  focuses  on  evaluating  the 
users’  satisfaction  of  working  with  WWW  educational  systems  and  highlights  the  important  issues  when 
addressing  the  design  of  WWW  courseware. 


2 Usability  for  Educational  Systems 

This  section  identifies  the  users’  satisfaction  requirements  / user  interface  requirements  for  WWW 
courseware  which  are  derived  from  the  pedagogic  theory  imperatives. 


2.1  The  Learning  Environment 

A learning  environment  must  provide  the  learner  with  four  key  requirements  (Duchastel  97).  These  are 
information,  interactivity,  structure,  and  communication.  The  learning  environment  must  also  implement  key 
interface  design  concepts  (Jones  et  al.  95).  These  key  concepts  can  be  loosely  categorised  by  research  facilities, 
presentation  details,  integration  across  various  media,  appropriate  use  of  tools,  and  help  facilities. 


2.2  Layout  requirements  of  a Learning  Interface 

When  planning  the  layout  of  the  interface,  many  factors  will  require  consideration. 
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Engaging  the  User:  The  main  consideration  when  deciding  upon  the  layout  of  a screen  is  to  assist 
learners  in  focusing  their  attention  on  the  key  aspects  of  instructional  material  while  maintaining  interest  in  the 
overall  lesson  content  and  activities.  The  intent  is  to  draw  attention  to  the  relevant  lesson  content  by  adopting 
conventions  that  draw  proportionately  more  attention  to  the  selected  information  than  the  remainder  of  the 
lesson  (Hannafin  et  al.  89). 

HCI  Principles:  The  interface  of  the  instructional  material  must  adhere  to  the  HCI  principles  that  have 
emerged  through  the  years  (Macauley  95).  These  are:  the  interface  must  be  natural  for  the  user  to  use;  the 
interface  must  provide  user  support;  the  interface  must  be  consistent;  the  interface  must  be  non  redundant;  and 
the  interface  must  be  flexible. 

There  are  also  two  categories  of  design  rule:  standards  which  are  high  in  authority  and  limited  in 
application,  and  guidelines  which  are  lower  in  authority  and  more  general  in  application. 

Interface  Structure:  The  structure  of  the  interface  must  have  at  least  two  areas  for  the  instructional 
material  (Brooks  93).  These  are  a navigation  area  and  a presentation  area. 

Layout  of  the  instructional  material:  It  is  not  sufficient  to  place  the  information  from  a previous  source 
on  the  Web  and  expect  it  to  work  as  a presentation  (Brooks  93).  Many  suggestions  have  been  specified  over  the 
years  about  the  layout  of  instructional  material.  The  following  suggestions  could  be  considered.  The  provision 
of  a menu  in  some  guise,  which  is  available  at  all  times  (Rambally  et  al.  87).  The  location  of  material  of  the 
same  type  should  not  vary  from  one  screen  to  the  next  (Rovick  85).  There  should  be  consistency  in  the  use  of 
backgrounds,  fonts,  and  colour  (Brooks  93).  Any  special  effects  used  and  icons  should  be  tested  with  users  first 
to  check  their  appropriateness  (McFarland  95).  The  document  should  not  be  too  large  for  two  reasons.  It  will 
take  longer  to  download,  and  not  all  learners  may  like  scrolling  (Nielsen  97).  Scrolling  can  leave  a learner 
disoriented  (Berners-Lee  95). 


2.3  Presentation  of  Instructional  material  components 

The  components  that  are  required  to  produce  instructional  material  can  be  a combination  of  text, 
graphics,  video  images,  etc.  so  many  factors  require  considering.  Another  issue  of  concern  here  is  the  inclusion 
of  colour. 

Textual  Material:  When  deciding  on  how  to  present  text,  there  are  guidelines  that  need  to  be  adhered  to 
(Rambally  et  al.  87).  These  include:  placing  key  information  such  as  urgent  messages  and  instructions  in  a 
prominent  and  consistent  location;  standardising  the  terminology  used;  establishing  prompt-string  conventions 
(captions)  in  order  to  provide  a systematic,  predictable  visual  correlation  between  prompts  and  their 
corresponding  data  input  fields;  and  positioning  captions  in  a natural  and  consistent  physical  relationship  to  the 
corresponding  data  fields.  It  has  also  been  suggested  that  screens  are  easier  to  read  if  the  text  is  structured  in 
natural  eye  sequences,  such  as  from  top  to  bottom  (Hazen  85). 

When  writing  textual  educational  material,  guidelines  on  how  to  lay  it  out  should  be  adhered  to  (Bailey 
et  al.  91).  These  include:  distinctly  separating  paragraphs  on  the  screen  to  promote  cohesive  groupings;  left 
justifying  the  body  text  and  ensuring  that  it  is  right  ragged;  ensuring  that  hyphens  are  not  be  used  to  break 
words  at  the  end  of  a line;  using  sentence  case  rather  than  upper  case  as  it  speeds  up  the  reading  process;  using 
UPPER  CASE  to  draw  attention;  and  using  one  font  consistently,  with  point  size  greater  than  12. 

Non  textual  material:  Interactive  graphing,  windowing,  and  animation  differentiate  the  computer  as  a 
medium  from  most  other  media  (Hazen  85).  Tables  can  be  used  to  graphically  represent  complex  relationships 
(Schlegal  96).  Graphics  should  be  used  when  they  contribute  to  the  understanding  of  the  text  (McFarland  95). 
However,  considerations  of  the  size  of  the  graphic  should  be  made,  as  it  should  be  able  to  fit  into  the  graphical 
browser’s  window  in  order  to  provide  the  learner  with  the  whole  picture,  and  note  also  that  a large  graphic  can 
take  a long  time  to  download.  Slow  graphics  can  become  tiresome  and/or  interrupt  lesson  flow  (Hazen  85).  The 
key  point  with  non  textual  material  is  not  to  let  available  equipment  or  technology  determine  the  presentation  of 
the  instructional  material  (Brooks  93). 

Colour:  Colour  has  some  uses  when  presenting  material  (Rambally  et  al.  87).  These  include:  linking 
logically  related  data;  differentiating  between  required  and  optional  data;  highlighting  errors;  separating  various 
screen  areas  such  as  prompts  and  commands;  emphasising  key  points;  and  communicating  the  overall  structure. 
When  applying  colour,  the  following  guidelines  have  been  recommended  (Macauley  95):  use  colour  coding 
consistent  with  user  expectations;  similar  colours  should  denote  similar  meaning;  avoid  use  of  extreme  colour 
pairs  to  avoid  frequent  refocusing  and  visual  fatigue;  adults  may  need  higher  brightness  levels  to  distinguish 
colours;  colour  blind  users  must  be  considered,  as  they  may  not  be  able  to  distinguish  some  colour 
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combinations;  use  background  colours  in  large  blocks;  group  related  elements  by  using  a common  background 
colour;  use  bright  colour  for  emphasis  and  weaker  colours  for  background  areas;  and  brightness  and  saturation 
draw  attention.  In  relation  to  WBI,  there  have  been  recommendations  made  (Jones  et  al.  97).  These  include: 
selectable  areas  should  be  clearly  identified  by  a royal  blue  colour;  interactivity  is  apparent  by  changes  in  the 
cursor  as  the  cursor  is  moved  to  a hot  spot;  when  a selection  is  made,  the  HTML  standard  is  to  immediately 
change  to  a dark  red  selection;  and  to  indicate  progress  made,  accessed  links  become  a light  red  colour.  In 
contrast,  another  view  held  on  the  colour  of  visited  links  is  that  they  should  signal  the  fact  they  are  “exhausted” 
by  being  a darker  colour  than  non-visited  links  (Kirsanov  97). 


3 Evaluation  Criteria  and  Implementing  WWW  based  usability  evaluation  system. 

The  objective  of  the  evaluation  was  to  evaluate  whether  the  present  learning  environment’s  user  interface 
is  conducive  to  learning,  i.e.  measure  the  users’  satisfaction  in  terms  of  the  their  experience  with  the  system.  In 
order  to  carry  out  this  evaluation,  goals  had  to  be  specified  but  goals  on  their  own  are  not  measurable.  Therefore 
from  these  goals,  objectives  were  derived  (Mandel  97).  The  concentration  of  the  evaluation  was  on  subjective 
measures  rather  on  performance  measures  because  it  was  required  to  address  particular  aspects  of  the  system 
performance  and  thus  required  focusing  on  certain  tasks  (Whitefield  et  al.  91).  The  goals  and  resultant  criteria 
reflected  the  performance  levels  required  from  the  system  from  a learner  viewpoint.  These  included  the 
aesthetic  appeal,  ease  of  reading,  ease  of  navigation,  and  the  effort  required  reading  through  the  material  and 
navigating  through  the  system.  The  goals  are  shown  at  HREF  1,  and  resulting  evaluation  criteria  are  shown  at 
HREF  2. 

The  goals  and  resulting  criteria  produced  a long  list  of  items,  which  it  was  felt  would  be  too  detailed  for  a 
questionnaire.  It  was  decided  to  structure  the  questionnaire  into  five  sections,  one  for  each  of  the  HCI 
principles.  The  items  from  the  evaluation  criteria  were  examined  and  placed  into  one  of  the  sections.  Not  all 
evaluation  criteria  were  explicitly  included  in  the  questionnaire  as  some  items  were  combined  with  other  items 
or  were  deemed  to  have  been  redundant.  The  questionnaire  is  shown  at  HREF  3. 

The  questionnaire  was  placed  on  the  Web  so  that  the  user  could  fill  it  in  during  the  evaluation.  Therefore, 
the  design  of  the  questionnaire  involved  producing  five  web  pages,  one  for  each  HCI  principle.  There  was  a 
statement  given  for  each  question,  and  the  User  was  asked  to  respond  to  each  statement  by  choosing  a point 
from  1 to  5 which  determines  whether  they  Strongly  disagreed.  Disagreed,  Strongly  agreed,  Agreed  or  were 
Neutral  with  the  statement.  It  was  decided  to  read  in  the  questions  dynamically  when  the  HTML  file  was  being 
published  rather  than  specify  the  questions  in  the  HTML  file.  This  ensures  that  any  changes  that  are  to  be  made 
to  the  questions  only  require  a change  to  the  text  file  of  questions  rather  than  a change  to  the  HTML  file.  In 
order  to  provide  this  facility,  an  HTML  file  is  not  adequate,  so  a Common  Gateway  Interface  (CGI)  program 
written  in  PERL  was  used.  The  program  opens  and  reads  the  entire  contents  of  the  text  file  of  questionnaire 
statements  and  places  them  into  an  array.  For  each  element  of  the  array,  it  is  displayed  as  a statement  followed 
by  the  five  radio  buttons  representing  the  five  possible  responses.  When  all  elements  of  the  array  have  been 
displayed,  the  User  is  then  requested  to  Send  or  Clear  the  data.  The  second  PERL  program  elicits  the  responses 
and  stores  them  in  text  file  separated  by  tabs.  This  procedure  is  described  in  Fig  1 . 


Figure  1:  WWW  questionnaire  procedure 

1 . Browser  requests  CGI  program  from  web  server. 

2.  Web  server  recognises  CGI  request  and  initiates  the  CGI  program. 
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3. 


CGI  program  reads  questions  from  questionnaire  file  and  displays  the  questions,  and  requests  responses 
from  the  participant. 

4.  CGI  program  requests  access  to  another  CGI  Program  for  data  processing. 

5.  Web  server  recognises  CGI  request  and  initiates  the  CGI  program. 

6.  CGI  program  transfers  responses  to  Data  file. 

When  the  questionnaires  are  filled  in,  programs  that  can  be  run  on  the  Web  make  an  assessment  of  the 
responses.  In  order  to  produce  any  feedback/results  from  the  evaluation,  five  programs  were  designed  to 
produce  five  Web  pages  of  results.  In  each  program,  the  file  of  responses  for  the  respective  HCI  principle  is 
opened,  read  and  processed.  Processing  of  the  file  involves  accumulating  the  number  of  times  a response  value 
(1  ..5)  was  chosen  by  a participant  in  the  evaluation.  Then  for  each  question  pertaining  to  this  HCI  principle, 
the  question  is  displayed  and  the  accumulated  values  are  displayed  as  percentages  in  a table.  When  all 
questions  have  been  displayed,  the  overall  accumulated  values  for  each  possible  response  is  displayed.  Also,  an 
overall  assessment  is  made  which  determines  if  70%  or  more  Agree  or  Strongly  Agree  with  all  statements  then 
it  is  very  satisfactory.  If  50%  or  more  Agree  or  Strongly  Agree  with  all  statements  then  it  is  satisfactory.  If  70% 
or  more  Disagree  or  Strongly  Disagree  with  all  statements  then  it  is  very  unsatisfactory.  If  50%  or  more 
Disagree  or  Strongly  Disagree  with  all  statements  then  it  is  unsatisfactory.  This  procedure  is  described  in  Fig  2. 


Figure  2:  WWW  evaluation  results  procedure 


1 . Browser  requests  CGI  program  from  web  server. 

2.  Web  server  recognises  CGI  request  and  initiates  the  CGI  program. 

3.  CGI  program  reads  responses  from  Response  file,  processes  the  responses  and  produces  an  assessment  of 
results. 


4 Case  Study  - Improving  RDBMS  Courseware 

There  is  a WWW  based  self  learning  course,  which  has  been  designed  and  implemented  successfully  in 
Trinity  College,  Dublin.  A subsection  of  the  course  was  chosen  and  used  to  test  and  validate  this  evaluation. 
The  modules  chosen  were  the  modules  pertaining  to  the  Select  statement,  which  is  one  of  SQL’s  data 
manipulation  statements. 

The  participants  for  the  evaluation  were  fourteen  fourth  year  students  on  the  BSc  in  Commercial  Software 
Development,  in  Waterford  Institute  of  Technology. 

The  feedback  on  the  users’  satisfaction  of  the  RDBMS  course  produced  some  positive  and  negative 
comments. 

Naturalness:  The  questions  that  provided  negative  feedback  were  concerned  with  colour  and  scrolling. 
Examples  of  some  of  the  points  made  with  regards  to  colour  included  that  highlighted  words  and  link  items 
have  the  same  colour,  which  is  confusing;  and  that  words  are  highlighted  using  different  colours.  Scrolling  is  a 
facility  that  cannot  be  avoided  on  the  Web,  however  too  much  of  it  can  irritate  users.  The  participants  felt  that 
the  material  was  too  far  indented  and  so  the  user  had  to  scroll  across  the  screen  to  read  the  material  as  well  as 
scroll  downwards.  Another  point  made  was  that  spelling  errors  exist  and  some  words  are  joined  together  in  the 
instructional  material.  Positive  comments  were  also  given  which  included  that  the  instructional  material 
provided  was  very  clear  to  read  and  understand,  and  the  instructions  identified  and  described  each  icon 
sufficiently. 

User  Support:  The  first  question  about  error  messages  provided  some  negative  feedback  but  one  of 
reasons  for  this  was  there  were  no  errors  incurred  by  the  participants.  The  other  area  where  there  was  some 
negative  feedback  was  on  the  examples  used.  There  needs  to  be  more  examples  and  exercises  provided  for  the 
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User.  Other  points  noted  were  that  the  instructions  provided  were  clear  and  well  presented,  and  that  the 
Bookmarking  and  Notetaking  facility  was  a good  idea,  but  when  a bookmark  is  recorded,  some  feedback  should 
be  provided  to  let  the  User  know  that  it  is  actually  recorded.  Another  interesting  point  made  was  that  a forward 
button  should  be  included  as  well  as  a back  button.  The  main  reason  for  this  comment  was  that  the  participants 
felt  that  when  you  got  to  the  end  of  a module,  it  was  not  easy  to  know  where  to  go  next.  Some  used  the  back 
button  to  go  to  the  Roadmap  menu;  others  used  the  Main  menu  button  and  then  proceeded  to  the  Roadmap 
menu. 

Consistency:  The  responses  were  positive  except  for  the  question  on  error  messages.  The  same  reason  as 
above  can  be  concluded  for  this  response.  The  main  comments  provided  were  that  there  was  an  inconsistent  use 
of  sentence  case,  italics,  spacing  and  colour. 

Non  Redundancy  : The  responses  were  positive.  However,  the  main  concern  of  the  participants  was  the 
repetition  of  the  menu  throughout  the  instructional  material  as  well  as  providing  it  in  the  left  frame.  It  was  felt 
that  the  menu  provided  in  the  left  frame  would  be  sufficient  on  its  own. 

Flexibility:  Again  the  responses  were  positive.  The  comments  provided  included  the  reference  to 
proceeding  from  a module  to  the  roadmap,  which  it  was  felt  was  not  very  clear. 


4.1  Changes  indicated  and  implemented 

After  reviewing  the  questionnaire  results  and  interviewing  the  participants,  the  evaluation  identified  some 
changes  to  improve  the  users’  satisfaction  of  the  courseware,  which  were  implemented.  These  included  the 
following: 

1.  The  colour  of  links,  active  links  and  visited  links  were  changed  to  differentiate  them  from  text.  A consistent 
colour  was  chosen  used  to  highlight  words,  which  is  also  different  than  the  link  colours  used.  The 
background  colour  of  the  left  frame  is  different  to  the  colour  of  the  right  frame  so  it  provides  a contrast. 

2.  Scrolling  across  the  screen  was  eliminated. 

3.  More  examples  and  dynamic  exercises  are  included. 

4.  Spelling  errors  were  eliminated.  Consistency  in  the  use  of  italics,  sentence  case,  and  spacing  is  ensured. 

5.  The  provision  of  menu  items  with  the  instructional  material  was  eliminated,  as  it  is  confusing  for  the  User. 

6.  In  the  left  frame,  a link  back  to  the  Roadmap  menu  is  provided  to  allow  the  User  to  choose  their  next 
module. 

7.  The  left  frame  was  reduced  in  size  to  increase  the  size  of  instructional  material  presentation  frame. 

8.  A check  is  made  to  ensure  that  when  the  learner  attempts  to  bookmark  a page  that  it  is  not  bookmarked 
already,  and  when  a page  is  successfully  bookmarked  a message  is  displayed  informing  the  learner.  The 
information  displayed  in  the  Bookmark  window  informs  the  learner  of  the  Module  and  Page  titles  stored. 

9.  A status  bar  message  is  displayed  each  time  the  mouse  moves  over  a link  to  clarify  for  the  learner  what  the 
link  is  for. 

A second  trial  using  the  modified  courseware  was  conducted  recently  and  early  analysis  of  the  results 
indicate  an  improvement  of  all  five  sections. 

5 Conclusions  and  Summary 

The  use  of  such  an  evaluation  system  is  a vital  component  in  the  delivery  of  successful  WWW 
courseware,  and  may  be  as  important  as  courseware  authoring  tools  and  communication  tools.  The  criteria  and 
questionnaire  provide  an  explicit,  easy  to  use  method  of  evaluating  users’  satisfaction  for  educational  systems. 
The  WWW  based  implementation  provides  three  important  features: 

1 . It  greatly  reduces  the  overhead  in  processing  the  evaluation. 

2.  It  provides  semi  automatic  assessment  of  the  study  results.  E.g.  if  70%  or  more  Agree  or  Strongly  Agree 
with  all  statements  for  a particular  HCI  principle  then  the  adherence  to  that  principle  is  very  satisfactory. 

3.  It  promotes  and  encourages  learner  feedback  and  evaluation  as  it  greatly  reduces  their  effort  in  completing 
the  questionnaire.  The  system  also  provides  its  learners  with  the  results  of  their  efforts. 

The  overall  user  satisfaction  was  enhanced. 
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Abstract:  A continuous  improvement  team  is  a group  of  people  who  work  together  on  the  same 
project  and  are  committed  to  continuous  improvement  of  their  work  processes.  In  order  to  support 
continuous  improvement  teams  in  a virtual  corporation,  a comprehensive  system  is  needed  to 
support  integrated  collaborative  work  and  collaborative  learning  processes.  In  this  paper,  the 
SCOPE  system  is  presented,  which  provides  support  for  the  definition,  modification  and 
execution  of  session-based  collaborative  processes  for  continuous  improvement  teams. 


1 Introduction 

A virtual  corporation  is  a temporary  network  of  independent  operating  units  such  as  creative  designers, 
manufacturers,  suppliers,  customers,  and  other  experts  in  marketing  and  finance  linked  by  interactive  multimedia 
networks  to  share  skills,  production  facilities,  resources,  to  decrease  costs  and  to  increase  access  to  each  other's 
markets.  In  virtual  corporations,  the  definition  of  a task  is  continually  changing,  and  its  product  is  a so-called 
virtual  product  that  adapts  in  real  time  to  the  customer's  changing  needs  [Davidow  & Malone  93].  Excelling  in  a 
dynamic  business  environment  requires  more  common  understanding  and  agreement  among  team  members  than 
individuals'  expertise  and  experience  provides.  In  the  life  cycle  of  a virtual  corporation,  cross-functional  teams 
need  to  continually  expand  their  capacity  to  improve  their  work,  processes,  relationships  and  environment.  Such  a 
team,  called  a continuous  improvement  team  (CIT),  is  a group  of  people  who  work  together  on  the  same  project  or 
who  come  together  to  solve  a common  problem,  and  are  committed  to  continuously  improve  the  processes  on 
which  they  depend  to  get  their  work  done  [Hutchings  et  al.  93]. 

In  order  to  support  CITs  in  virtual  corporations,  a comprehensive  environment  is  needed  to  overcome  the 
barriers  of  physical  separation  and  to  support  integrated  collaborative  work  and  collaborative  learning  processes. 
Existing  learning  support  systems  are  designed  according  to  one  of  three  metaphors:  session,  process  and  place. 
Systems  that  rely  on  a sole  metaphor  provide  insufficient  support  for  continuous  improvement  processes 
[Hutchings  et  al.  93].  In  addition,  most  collaborative  learning  support  systems  are  separate  from  collaborative  work 
support  environments.  In  this  paper,  we  present  our  approach  to  support  an  integrated  collaborative  learning  and 
collaborative  work  process.  The  prototype  system,  session-based  collaborative  process-centered  environment 
(SCOPE)  integrates  process  support  technologies  into  a collaborative  support  environment.  A session  is  defined  as 
follows:  in  a period  of  time  activities  are  performed  by  multiple  actors  with  different  roles  in  a shared  information 
space  to  achieve  an  elementary  goal.  A session-based  collaborative  process  (SCP)  denotes  a process  that  consists 
of  a set  of  coordinated  sessions  [Miao  & Haake  98a].  SCOPE  provides  support  for  the  specification,  modification, 
and  execution  of  session-based,  integrated  collaborative  work  and  collaborative  learning. 

The  remainder  of  this  paper  is  organized  as  follows:  In  the  next  section,  characteristics  of  continuous 
improvement  processes  and  major  requirements  are  identified.  Section  3 presents  related  work.  Section  4 describes 
how  SCOPE  supports  continuous  improvement  processes.  Finally,  we  present  our  conclusions  and  future  work. 


2 Requirements  for  the  Support  of  Continuous  Improvement  Teams 

In  order  to  identify  the  characteristics  of  continuous  improvement  processes  and  the  major  requirements  of 
the  support  system,  a scenario  in  a virtual  corporation  is  developed,  which  is  based  on  a real  scenario  presented  in 
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[Hutchings  et  al.  93]. 


2.1  A Scenario 

Seven  members  of  a team  in  a virtual  software  corporation  are  responsible  for  designing  a software 
product.  The  team  consists  of  software  engineers  and  requirement  analysts  who  are  geographically  distributed.  The 
team  wishes  to  learn  how  to  perform  continuous  improvement.  The  team  and  software  process  consultants  jointly 
determine  what  is  needed  to  improve  the  team’s  capabilities.  Then,  training  consultants  design  a training  course  for 
the  team.  They  create  and  customize  necessary  materials.  Such  a course  takes  into  account  the  team's  total  work 
situation,  team’s  present  job  tasks  and  their  social  environment.  The  training  course  consists  of  a series  of  sessions. 
The  sessions  take  place  in  working  hours  and  are  conducted  in  the  same  environment  as  collaborative  work.  In  the 
first  lecture  session,  the  team  learns  sound  decision-making  strategies  and  principles.  After  the  lecture,  the  team 
practices  separately  in  two  groups  (function  design  group  and  interface  design  group)  and  sets  up  the  strategies  for 
how  they  would  like  to  make  group  decisions.  Then,  they  switch  to  their  collaborative  work  processes  and  follow 
the  strategies  to  make  decision  for  their  design.  During  work  processes,  the  data  collected  and  practice  results 
achieved  will  be  transferred  into  collaborative  learning  processes  as  parts  of  materials  of  the  following  sessions.  In 
the  succeeding  learning  session,  the  teacher  guides  the  team  to  analyze  and  evaluate  the  currently  used  decision- 
making strategies.  After  that,  the  team  will  learn  and  try  to  improve  their  decision-making  strategies  and  use  the 
improved  strategies  to  get  their  work  done.  That  is,  collaborative  learning  and  collaborative  work  in  virtual 
corporations  form  a process  in  which  the  team  as  a whole  continuously  reflects  upon  events,  gains  knowledge  about 
job  tasks  and  work  processes,  improves  product  quality  and  work  processes,  and  discovers  their  learning  needs  in 
the  workplace  - continuously. 

2.2  Characteristics  of  Continuous  Improvement  Processes 

Based  on  an  analysis  of  the  scenario  described  in  the  previous  sub-section  and  a reference  of  design 
principles  presented  in  [Hutchings  et  al.  93],  the  characteristics  of  continuous  improvement  processes  can  be 
identified. 

• Modularity  for  flexibility:  Information  and  materials  used  and  produced  in  learning  activities  or  working 
activities  are  documented  as  modules.  They  are  constructed  as  typed  and  self-contained  units  of  information  - 
artifacts.  Modularized,  structured  artifact  models  are  beneficial  for  the  team’s  common  understanding  and 
communication. 

• Team-based  collaborative  learning  and  collaborative  work  activities:  Interventions  are  delivered  in  team 
sessions,  and  emphasize  collaboration  and  shared  responsibility.  Sessions  are  designed  for  team  members  to 
gain  new  skills  and  knowledge  together.  In  addition,  team  members  collaboratively  perform  job  tasks  also  in 
sessions.  Team  members  may  adapt  or  improve  collaboration  strategies  to  conduct  sessions  according  to  their 
learning. 

• Phased  delivery:  Interventions  are  phased  in  their  delivery  over  time.  Intervals  between  learning  sessions 
provide  opportunities  for  the  team  to  apply  learning  in  work.  The  data  collected  and  test  results  achieved  in 
working  sessions  will  be  transferred  to  the  succeeding  learning  session.  Team  members  have  to  schedule  an 
integrated  collaborative  learning  and  collaborative  work  plan  to  help  communication,  understanding,  and 
coordination  among  team  members.  There  are  dependencies  among  sessions,  such  as  temporal  sequencing,  as 
well  as  among  participants  and  artifacts. 

• "Just-in-time"  effort:  Learning  and  its  application  occur  almost  simultaneously.  In  a virtual  corporation,  the 
external  environment  and  customer  demands  are  dynamic  and  changeable.  In  turn,  team’s  business  goals  and 
needs  related  to  a team’s  mission  should  be  adapted  to  the  changing  environment. 

2.3  Major  Requirements 

In  order  to  design  an  integrated  computer-supported  environment,  according  to  the  analysis  of 
characteristics,  the  major  requirements  are  identified  as  follows: 

(Rl)  Support  for  dealing  with  artifacts:  The  artifacts  are  modularized  and  structured,  and  should  be  adjusted  and 
tailored  dynamically  according  to  the  team’s  progress. 

(R2)  Support  for  session  activities:  The  system  should  support  team  members  with  different  roles  to  perform 
collaborative  activities  in  a shared  information  space.  In  addition,  the  system  should  provide  facilities  for  team 
members  to  define  and  modify  collaboration  protocols,  a computational  description  of  a collaboration  policy  or  co- 
decision strategy. 
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(R3)  Support  for  definition  and  execution  of  SCPs:  A SCP  model  is  a computational  description  of  a project  plan  or 
a curriculum.  The  system  should  enable  team  members  to  define  process  models.  It  is  desirable  that  process  models 
can  be  used  to  coordinate  group  work  and  learning  activities  automatically. 

(R4)  Support  for  dynamical  and  collaborative  change  of  process  models  by  teams:  The  system  should  support 
team  members  to  modify  SCPs  on  the  fly.  Instead  of  a single-user  process  modeling  tool,  the  modeling  tool  should 
be  able  to  be  used  jointly  by  teams. 

3 Related  Work 

In  this  section,  we  investigate  existing  systems  in  two  aspects,  system  development  and  integrated 
collaborative  work  and  collaborative  learning  support  environments. 

3.1  Development  Approaches  of  Learning  Systems 

Most  existing  learning  systems  or  environments  are  based  on  one  of  three  metaphors:  session,  process  and 

place. 

Session-based  systems  support  team  members  to  collaborate  within  single  sessions  or  meetings.  They 
provide  an  environment  for  sharing  information  and  performing  collaborative  activities  at  the  same  time.  These 
systems,  such  as  Microsoft  NetMeeting  [http://www.microsoft.com/netmeeting],  provide  support  for  dealing  with 
artifacts  (Rl)  and  for  session  activities  (R2),  but  have  no  support  for  the  remaining  requirements. 

Process-based  systems  provide  support  for  structuring  stages  according  to  the  content  structure  in  a 
overall  learning  process.  Learners  can  progress  at  different  speeds  in  the  learning  process.  These  systems  focus  on 
individual  learning.  Obviously,  these  systems  have  no  sufficient  facilities  to  support  session  activities  (R2). 

Place-based  systems  provide  virtual  places  to  hold  meetings  or  to  perform  activities  asynchronously.  All 
of  the  artifacts  produced  in  meetings  or  individually  are  left  in  the  places  to  be  reused  in  the  succeeding  activities. 
However,  the  sequencing  of  activities  performed  in  places  and  the  transferring  of  artifacts  has  to  be  done  manually. 
A lot  of  place-based  learning  systems  have  been  developed  recently,  such  as  Virtual  Places 
[http://www.virtualplaces.com],  CSILE  [Scardamalia  et  al.  94],  VITAL  [Pfister  et  al.  98].  These  systems  also 
provide  support  for  dealing  with  artifacts  (Rl)  and  for  session  activities  (R2).  However,  they  can  not  meet  the 
requirements  related  to  support  for  SCPs  (R3  and  R4). 

Mixed  approach:  Some  information  sharing  systems  (session-based  systems  and  place-based  systems)  add 
means  for  embedding  the  group's  overall  work  process  into  a general  collaborative  learning  environment.  For 
example,  CLARE  [Wan  & Johnson  94]  is  developed  based  on  a model  of  three-phase  collaborative  learning 
processes.  The  process  can  be  regarded  as  a SCP  to  some  extent.  However,  it  offers  no  support  for  dynamical  and 
collaborative  change  of  SCPs  (R4),  because  it  supports  a fixed,  domain-specific  process  model. 

3.2  Integrated  Collaborative  Work  and  Collaborative  Learning  Support  Environment 

Most  collaborative  learning  support  systems  can  not  support  learning  in  work  settings.  They  normally 
support  open-enrollment  styles  of  training  which  focus  on  improving  individuals.  Learners  may  successfully  gain 
knowledge  in  courses  and  then  they  will  apply  their  learning  at  different  places.  Such  courses  rarely  take  into 
account  learners’  present  job  tasks  and  their  social  environment.  Some  systems  can  be  used  to  support  both 
collaborative  learning  and  collaborative  work  separately,  because  they  only  provide  general  communication 
support.  However,  to  the  best  of  the  authors'  knowledge,  no  system  can  support  learners  to  define  and  modify  a 
computational  description  of  their  work  processes  as  learning  practices,  and  to  coordinate  their  collaborative  work 
by  using  the  computational  description  of  their  work  processes.  No  system  can  support  automatically  transferring 
the  data  collected  and  the  practice  results  achieved  in  work  processes  into  learning  processes  as  part  of  learning 
materials. 

4 The  SCOPE  System 

In  this  section  we  present  our  approach  to  support  CITs.  This  approach  can  be  characterized  by 

• using  hypermedia  to  represent  artifacts  in  a flexible  way, 

• providing  shared  hypermedia  workspaces  for  sessions, 

• using  hypertext  structures  to  specify  and  modify  organizational  structure,  collaboration  protocols,  and  SCPs, 
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• facilitating  the  execution  of  SCPs  and  collaboration  protocols. 

This  approach  has  been  tested  in  the  SCOPE  system  [Miao  & Haake  98a].  Next,  we  discuss  how  to 
support  the  specification  of  SCPs.  Then  support  for  continuous  improvement  processes  at  run  time  is  presented. 

4.1  Specification  of  SCPs 

In  this  sub-section,  we  present  our  approach  to  specify  SCPs.  We  discuss  how  to  specify  sessions  first  and 
then  present  how  to  specify  a SCP. 

4,  LI  Specification  of  a Session 

In  SCOPE,  a session  is  created  for  a CIT  to  learn  or  perform  job  tasks  collaboratively.  In  a session,  the 
common  information  space  is  potentially  accessible  by  all  team  members  involved  in  this  session.  Team  members 
can  perform  operations  synchronously  or  asynchronously  to  construct  shared  artifacts  together,  and  to  perform 
actions  to  change  the  state  of  collaboration.  It  is  important  to  notice  that  a CIT  consists  of  members  who  may  have 
different  privileges  and  responsibilities  to  deal  with  artifacts  and  to  coordinate  their  efforts.  To  define  a new 
session,  team  members  are  required  to  set  values  of  attributes  for  the  session.  Without  being  exhaustive  these 
attributes  include:  session  name,  session  type,  session  creator,  session  description,  collaborative  mode,  artifact 
models,  collaboration  protocol,  mapping  from  organizational  to  functional  roles,  scheduled  start  and  end  times,  etc. 
Some  attributes  used  in  specifying  a session  are  defined  by  using  specific  tools:  organizational  structure  definition 
tool,  artifact  definition  tool,  and  collaboration  protocol  definition  tool. 

Definition  of  members  and  teams:  Members  and  teams  are  defined  by  using  the  organizational  structure 
definition  tool.  With  this  tool,  a member,  an  organizational-role  or  an  organizational-unit  is  represented  as  a node 
with  a type,  a unique  name  and  a set  of  attributes  to  describe  properties.  The  typed  links  are  represented  as  direct 
arrows  between  suitable  nodes.  A description  of  an  organizational  structure  is  represented  as  a Directed  Acyclic 
Graph. 

Definition  of  the  artifact  model:  An  artifact  model  is  defined  in  terms  of  task-specific  typed  hypertext 
objects  (nodes  and  links)  together  with  operations  on  them  to  support  a certain  activity  [Streitz  et  al.  8 9]  [Wang  & 
Haake  97].  Artifacts  are  organized  into  these  node  and  link  types  according  to  their  structure,  function,  and 
behavior  properties.  An  artifact  may  be  hierarchically  structured,  and  the  structure  of  aggregated  artifacts  can  be 
defined  in  an  artifact  definition  tool,  which  is  a part  of  the  COWFISH  system  [Wang  & Haake  97]. 

Definition  of  collaboration  protocols:  A collaboration  protocol  is  described  like  a state-transition  diagram 
where  a node  may  contain  a sub  diagram  describing  the  state  represented  by  that  node  in  more  detail.  A state  node 
in  a diagram  represents  a state  within  the  collaboration  protocol.  A link  in  a diagram  represents  an  event.  The 
occurrence  of  an  event  causes  a transition  between  the  two  states  connected  by  the  link.  Each  state  of  the 
collaboration  is  associated  with  a set  of  behavior  rules.  A behavior  rule  specifies  which  roles  are  permitted  to 
perform  which  operation  on  what  type  of  artifacts.  A link  is  associated  with  a behavior  rule  or  a logical  expression. 
The  detailed  description  of  collaboration  protocols  is  presented  in  [Miao  & Haake  98b]. 

4,L2  Specification  of  SCPs 

Every  SCP  is  represented  by  its  description,  which  is  called  a process  model.  This  description  specifies  all 
sessions  embedded  in  the  process  and  the  relationships  among  these  sessions.  SCOPE  provides  a visual  process 
model  language  for  process  definition.  A process  model  is  described  as  a hypertext  document  consisting  of  nodes 
that  are  connected  via  links.  A process  can  be  decomposed  into  sub-processes.  The  components  and  the  structure  of 
a (sub)process  are  described  inside  the  (sub)process  node.  Session  nodes,  the  elements  of  processes,  are  specified  in 
the  way  described  above. 

The  process  models  are  specified  in  SCOPE  by  using  the  process  definition  tool  (see  Fig.  1).  The  tool 
shows  a window  title  bar  listing  the  name  of  the  overall  process  model  and  the  tool  type.  Below  that  is  a list  of 
icons  showing  the  current  users  of  the  tool,  followed  by  a button  bar  with  generic  functions  for  editing,  navigating 
hypertext  documents,  and  specific  functions  for  editing  process  models,  followed  by  a palette  of  components  of  the 
visual  process  model  language  (left  side)  and  the  content  pane  (right  side)  displaying  a process  model.  In  Fig.  1,  an 
example  of  a process  model  is  presented,  which  describes  the  collaborative  learning  process  explained  in  the 
scenario  (see  Chapter  2.1).  This  part  of  process  model  consists  of  a session  for  collaboratively  customizing 
materials,  a lecture,  two  practice  sessions,  and  a workshop.  They  are  connected  by  temporal  links,  such  as  “finish- 
trigger”  and  “AND-split”,  and  by  artifact  links  with  named  artifact  buffers.  It  is  important  to  note  that  the  artifact 
input/output  buffers  are  used  to  exchange  artifacts  across  (sub)processes.  For  example,  in  practice  sessions, 
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decision-making  strategies  are  defined  as  collaboration  protocols  that  will  be  transferred  to  the  collaborative  work 
process.  The  defined  collaboration  protocols  will  be  initiated  in  groups’  collaborative  work  sessions,  and  guide  and 
control  group  decision-making  processes  for  design.  The  feedback  from  the  collaborative  work  process  will  be 
imported  into  the  workshop  via  an  artifact-input  node.  After  a process  model  is  defined,  it  can  be  stored  in  a 
process  model  base  and  can  be  instantiated  and  reused.  Next,  we  will  discuss  the  run-time  features  of  SCOPE. 
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Figure  1:  An  example  of  a process  model 


4.2  Support  for  Continuous  Improvement  Processes  at  Run  Time 

Process  execution  is  concerned  with  the  enactment  of  a process  following  the  previously  defined  process 
model.  Now,  we  discuss  process  execution  from  the  users'  perspective. 

Team  members  can  select  a pre-defined  process  model  from  the  process  model  base  as  their  initial  process 
model.  If  no  suitable  process  model  exists,  they  can  define  a new  one.  When  they  start  to  execute  the  process 
model,  an  instance  of  this  process  model  is  created.  This  process  instance  will  progress  according  to  its  process 
model. 

Support  for  execution  of  session  activities'.  Each  team  member  has  a personal  information  panel.  In  this 
panel,  information  is  listed  about  processes  and  sessions  in  which  he/she  should  be  involved.  He/she  can  join  a 
session  by  selecting  it  from  the  session  list.  When  he/she  joins  a session,  the  top-level  of  the  shared  information 
space  of  the  session  is  displayed  in  the  session  browser  and  he/she  can  browse  through  the  artifacts  following  the 
recorded  dependencies.  In  this  browser  there  is  a palette  that  lists  the  artifact  types,  which  are  allowed  to  be  created 
in  the  current  state  according  to  the  rules  defined  in  the  collaboration  protocol  of  this  session.  The  shared  artifacts 
can  be  viewed  and  edited  simultaneously  by  team  members  working  in  the  session  depending  on  their  access  rights. 
When  a user  performs  an  operation  on  an  artifact,  the  system  will  check  whether  this  member  has  the  access  right 
to  execute  the  operation  according  to  rules  defined  at  this  state  of  collaboration.  If  the  check  is  successful,  the 
change  will  be  propagated  to  all  sites.  To  maintain  consistency  for  concurrent  changes  at  different  sites,  a fully 
replicated  concurrency  control  mechanism  is  provided  by  COAST  [Schuckmann  et  al.  96].  Transitions  from  one 
state  of  collaboration  to  another  may  be  carried  out  by  team  members  who  have  the  right  to  perform  actions. 
Allowed  actions  are  also  listed  in  a palette  in  the  session  browser.  Each  session  has  an  autonomous  agent  that 
monitors  the  status  of  collaboration  periodically  by  evaluating  the  transition  conditions.  If  a condition  is  met,  the 
system  changes  the  state  according  to  the  collaboration  protocol. 

Support  for  execution  of  SCPs\  Changes  of  the  state  of  collaboration  in  one  session  may  result  in  changes 
of  the  state  of  related  sessions  according  to  the  process  model.  In  the  case  of  two  sessions  connected  by  a “finish- 
trigger”  link,  once  the  source  session  is  finished,  the  state  of  the  destination  session  will  change.  In  SCPs,  artifacts 
will  be  automatically  transferred  between  sessions  according  to  the  process  model. 

Support  for  dynamical  change  of  SCPs:  Even  when  a process  instance  has  already  been  created,  parts  of 
the  process  model  can  be  modified.  That  is,  the  values  of  attributes  of  the  related  sessions  can  be  modified  and  the 
model  of  a sub-process  can  be  altered. 
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5 Conclusions  and  Future  Work 


In  this  paper,  we  described  the  major  requirements  for  supporting  ClTs  in  virtual  corporations.  Based  on 
the  concept  of  session-based  collaborative  processes,  we  developed  a modeling  methodology  and  an  environment, 
SCOPE.  The  system  can  support  definition  and  execution  of  session-based  collaborative  processes.  This  is  realized 
by  maintaining  shared  hypermedia  workspaces  with  the  aid  of  data  level  control  mechanisms,  such  as  access 
control  and  fully  replicated  concurrency  control,  and  user  level  control  mechanisms,  such  as  collaboration 
protocols  and  SCP  models.  In  addition,  the  SCOPE  system  provides  a high  level  of  flexibility  for  team  members  to 
adapt  artifact  models  and  to  modify  process  models  at  run  time.  By  using  the  facilities  provided  in  the  SCOPE, 
members  of  continuous  improvement  teams  can  define  and  execute  integrated  collaborative  work  and  collaborative 
learning  processes.  They  can  improve  their  work  processes  through  applying  their  learning  and  identify  learning 
issues  at  work  to  fit  the  changes  inside  and  outside  of  virtual  corporations. 

SCOPE  is  a prototype  system  of  our  ongoing  research  project  and  is  currently  being  tested  in  our  group. 
Most  of  our  work  to  date  has  been  focused  on  demonstrating  the  feasibility  of  implementing  and  using  the  system. 
We  will  develop  more  collaborative  tools  required  by  software  development  and  total  quality  management.  We  will 
further  test  and  evaluate  the  usefulness  of  the  system  in  more  real-world  settings. 
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Abstract;  There  are  many  reasons  why  an  adult  may  choose  to  undertake  a distance 
programme  and  in  recent  years  there  has  been  considerable  interest  in  off-campus  learning 
and  an  increase  in  the  availability  of  such  opportunities.  The  University  of  Greenwich 
(UK),  a long  time  provider  of  distance  learning  opportunities,  is  offering,  through  its  School 
of  Health  in  the  Faculty  of  Human  Sciences,  an  MSc.  in  Continuing  Professional 
Development  (CPD)  for  health  professionals.  Computer  Mediated  Communication  (CMC) 
is  at  the  core  of  the  curriculum  process  in  which  off-line,  asynchronous  conferencing 
ensures  participants  have  opportunities  for  critical  evaluation  and  reflection  before 
formulating  and  sending  considered  responses.  This  paper  considers  elements  of  the 
curriculum  design  in  order  to  reveal  the  extent  to  which  the  embedding  of  communications 
technology  has  provided  a unique  teaching  and  learning  experience.  It  will  demonstrate 
how  a multi-disciplinary  group  of  health  professionals  are  working  collaboratively  at  a 
distance,  through  computer  conferencing,  to  enhance  both  individual  and  professional 
development. 


Introduction 

The  use  of  communications  technology  clearly  has  enormous  potential  in  a number  of  educational  areas;  some 
of  these  are  already  being  realised  and  many  others  are  in  the  process  of  development.  Communications 
technologies  embrace  an  extremely  wide  range  of  resources,  from  simple  e-mail  to  sophisticated  multi-media 
applications.  In  this  paper  we  discuss  the  use  of  relatively  low-level  technology,  asynchronous  computer 
conferencing,  as  a delivery  and  support  tool  for  health  professionals  participating  in  a MSc.  in  Continuing 
Professional  Development  (CPD). 


The  Concept  of  CMC 

Computer  Mediated  Communication  (CMC)  is  not  a new  concept  nor  does  it  need  to  use  state  of  the  art 
technology.  Whilst  some  research  has  been  undertaken  into  the  medium,  much  still  remains  to  be  discovered 
and  understood  about  its  impact  on  teaching  and  learning.  Until  this  knowledge  is  available  it  will  not  be 
possible  to  exploit  the  technology  fully.  At  the  University  of  Greenwich  we  run  several  courses  using  CMC  and 
are  now  in  a position  to  begin  an  evaluation  of  its  potential.  In  particular  we  can  begin  an  analysis  of  its 
effectiveness  as  a curriculum  tool  in  the  delivery  of  the  MSc.  CPD  (Health). 

We  define  CMC  as,  ‘‘the  use  of  computers,  telephone  lines  and  specialist  software  to  facilitate  interaction 
between  students  and  tutors  irrespective  of  geographical  location  or  time  zone”  (Ryan,  1997).  It  is  this 
interaction  which  we  consider  one  of  the  key  aspects  of  CMC  because,  ‘‘where  opportunities  exist  for 
interaction  in  learning  situations  at  a distance,  it  helps  learners  to  make  leaps  in  their  understanding  that  are 
unlikely  to  occur  when  working  in  isolation  ".  (Owusu-Sekyere  & Branch,  1996) 
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Synchronous  Versus  Asynchronous  Systems 


There  is  no  doubt  that  modem,  synchronous  communications  systems  such  as  Video  Conferencing  and  Internet 
Relay  Chat  are  both  attractive  and  powerful  but  produce  technology-driven  environments  which  are  in  a 
constant  state  of  flux.  They  also  require  participants  to  be  available  at  the  same  time,  albeit  in  different  places, 
in  order  to  interact  and  collaborate.  Such  ‘real-time’  technologies  are  time-location  dependent  and  whilst  their 
appeal  can  not  be  denied  there  is  evidence  to  suggest  that  asynchronous  technologies  are  better  able  to  facilitate 
the  reflective  models  (Mason  & Kaye,  1990)  embraced  by  many  CPD  programmes. 

Computer  conferencing  which  uses  asynchronous  media  is  time-location  independent  and  provides  greater 
flexibility  and  student  autonomy  (Harasim,  1989).  This  asynchronicity  facilitates  a spirit  of  co-operative  enquiry 
(Heron,  1985)  leading  to  active  dialogue  through  which  participants  share  ideas  and  information  (Gundiy,  1992). 
The  resultant  peer-to-peer  collaborative  interactions  have  been  shown  to  increase  learner  satisfaction  (Kaye 

1992)  and  involvement  in  the  development  of  theories  of  professionalism  and  professional  practice  (Richardson, 

1993) .  Our  experience  indicates  that  this  asynchronous  CMC  environment  provides  real  opportunities  for 
collaborative  and  multi-disciplinary  discussion,  brainstorming,  problem  solving  and  networking,  which  we 
hope  to  demonstrate  in  this  paper. 

Many  adult  distance  learners  engaged  in  CPD  programmes  are  known  to  study  at  home  and  accessing  the 
Internet  would  normally  require  them  to  subscribe  to  a commercial  service.  To  benefit  fully  from  time-location 
independence,  students  should  not  be  required  to  work  synchronously  and  in  the  current  financial  climate  they 
should  not  have  to  bear  the  additional  costs  incurred  by  working  on-line  or  paying  subscriptions  to  Internet 
providers. 


The  Context  of  the  MSc.  in  CPD 

The  context  of  this  development  is  a MSc.  in  CPD  for  health  professionals.  Although  presently  being  delivered 
to  health  professionals  it  is  the  conviction  of  the  course  developers  that  the  model  itself  is  generic  and  has  the 
potential  to  be  transferred  to  other  professions  and  groups. 

Distance  learning  had  to  be  the  mode  of  delivery  for  this  MSc.  so  that  the  potential  participants,  identified 
through  market  research  and  enquiry,  would  be  able  to  access  the  appropriate  learning  opportunities  they 
needed.  It  was  clear  from  the  start  of  the  development  that  the  curriculum  model  selected  would  have  to  offer 
much  more  than  that  available  on  a traditional  paper-based  distance  learning  programme  where  even  the 
production  of  learning  materials  can  take  many  months  or  years.  This  would  be  inappropriate  for  a topic  as 
dynamic  and  diverse  as  CPD. 

The  development  team  holds  a strong  philosophy  that  professionals,  such  as  the  ones  for  whom  the  programme 
was  developed,  have  much  to  offer  their  fellow  students  from  their  previous  learning  and  experience.  Learning 
processes  selected  for  the  programme  would  have  to  acknowledge  and  make  the  most  of  this  knowledge,  "the 
richest  resources  for  learning  lie  in  the  adult  learners  themselves",  (Knowles,  1990).  Thus  it  was  decided  to 
utilise  a computer-mediated  communication  technology  which  would  facilitate  the  realisation  of  the  potential 
each  student  had  as  a learning  resource. 

Everyone  in  a multi-professional  group  of  students  would  be,  at  different  times,  colleague,  peer,  learner  and 
expert  and  it  was  considered  that  significant  deep-level  learning  could  result  from  the  discussion,  conversation, 
debate  and  argument  between  these  groups  which  CMC  would  facilitate.  For  the  students  to  see  themselves,  as 
important  and  crucial  resources  would  not  only  ensure  that  the  content  and  context  of  every  unit  would  be 
relevant  for  each  and  every  participant  but  also  that  the  group  process  that  would  be  fostered  would  result  in  an 
atmosphere  of  collaboration  and  co-operation.  We  think  we  are  achieving  this;  - one  student  recently  wrote  in  a 
conference  area  *\,,may  we  continue  to  bounce  our  ideas  off  each  other  like  light  and  reflect  images  in  a 
mirror  or  window  in  our  world  in  our  efforts  to  illuminate  the  subject  from  all  angles". 
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The  Concept  of  CPD 

The  issue  of  continuing  professional  development  is  of  major  importance  in  many  professions,  including  the 
health  professions  where  evidence  of  ongoing  participation  in  continuing  professional  development  (CPD) 
activities  is  likely  to  become  a requirement  for  re-registration  to  practice.  CPD  is  an  umbrella  term  for  a wide 
range  of  activities  relating  to  professional  competence,  some  practice  or  clinically  based  and  occurring  in  the 
workplace  and  others  more  academically  based  and  initiated  and  run  in  Higher  Education  Institutions  (HEIs). 
Methods  of  assessing  CPD  activity  are  also  diverse  and  it  is  clear  that  the  evaluation  of  the  relevance  and 
quality  of  an  individual’s  CPD  is  not  an  easy  task.  However,  that  CPD  should  occur  is  not  in  doubt  as 
maintenance  of  professional  competence  is  essential  for  the  protection  of  the  public.  Grant  (1994)  suggests  that 
professional  knowledge  has  a half-life  of  five  years  and  that  obsolescence  will  occur  if  no  new  knowledge  is 
assimilated  and  incorporated  into  practice  within  that  time. 

In  the  present  climate  of  economic  constraint,  in  the  UK  National  Health  Service  in  particular,  it  has  become 
increasingly  difficult  in  terms  of  both  finance  and  study  time  for  many  health  professionals  to  fulfil  their  CPD 
requirements.  We  are  in  a situation,  in  which  health  professionals  must  have  evidence  of  CPD  to  maintain  a 
licence  to  practice  but  at  a time  when  they  are  finding  it  more  and  more  difficult  to  participate  in  suitable  CPD 
activity.  It  is  in  this  context  that  the  MSc.  in  Continuing  Professional  Development  (Health)  (Collaborative 
Learning  through  Reflective  Practice  and  Computer  Mediated  Communication)  was  conceived  and  developed. 

The  programme  is  now  in  its  third  year  and  has  three  cohorts  of  students.  They  represent  almost  all  the  health 
professions  and  encompass  a wide  geographic  spectrum  including  Austria,  Canada,  Cyprus,  Malaysia,  Malta, 
New  Zealand,  South  Africa  and  the  UK. 

The  Choice  of  a CMC  System  - Why  Lotus  Notes? 

Having  decided  to  use  asynchronous,  text  based  computer  conferencing  on  the  basis  of  its  fitness  for  purpose 
(Laurillard,  1993)  the  software  chosen  was  the  groupware  product  Lotus  Notes.  Groupware  is  essentially  any 
computer  application  that  runs  over  a network,  and/or  is  accessed  by  direct  dial  means,  allowing  the  user  group 
to  increase  their  communication,  collaboration,  and  co-ordination.  Lotus  Notes  is  currently  the  only  software 
on  the  market  that  captures  these  processes  in  one  product,  with  the  potential  for  customisation. 

The  immediate  advantages  of  Notes  appeared  to  be  its  availability  on  multiple  platforms  in  both  server  and 
client  forms.  It  is  an  industry-standard  package  with  a massive  international  user-base  providing  sophisticated 
E-mail  and  conferencing  facilities.  We  felt  it  would  appeal  to  novice  users  of  communications  technologies  but 
with  infinite  possibilities  for  expansion  and  development.  It  was  supplied  with  a number  of  templates  that  were 
customisable  to  enable  the  development  of  a range  of  databases  to  support  conferencing  and  workflow 
applications.  Unlike  many  of  its  competitors.  Notes  allows  off-line  working  with  replicas  of  conferences  and 
mailboxes  using  a remote,  direct  dial  facility  with  the  option  of  networking  and  connectivity  via  TCP/IP. 

It  was  perhaps  this  apparent  ease  of  use,  the  attractiveness  of  the  interface  and  the  facility  to  work  completely 
off-line  that  appealed  most.  In  this  paper  it  is  not  our  intention  to  promote  a particular  brand  of  software  but  to 
examine  the  concept  and  impact  of  CMC  on  distance  learning.  What  we  have  experienced  whilst  using  our 
chosen  product  may  be  replicable  with  other  software. 

Unique  Features  of  CMC  Supported  Learning 

In  developing  a CMC  system  we  began  by  trying  to  reproduce  our  experiences  of  classroom-based  teaching  but 
soon  realised  that  computer  conferencing  impacts  on  the  dynamics  of  the  group  and  changes  the  nature  of  face- 
to-face  (F2F)  activities.  Additionally,  we  discovered  that  communications  technologies  provide  some  unique 
opportunities  for  interaction  which  should  be  exploited  (Anderson,  1996). 

Ongoing  evaluation  has  revealed  that  students  appreciate  the  opportunity  to  be  able  to  ask  for  and  receive 
almost  immediate  support  from  tutors.  They  welcome  the  interactive  and  collaborative  nature  of  the 
environment,  get  to  know  their  peers  and  tutors  as  recognisable  personalities  despite  the  absence  of  non-verbal 


cues.  They  comment  on  feeling  as  if  they  are  actually  attending  the  institution  (virtually)  and  make  reference  to 
being  part  of  a community.  This  twenty  four-hour  access  to  the  learning  community  and  the  automatic 
archiving  of  all  discussion,  a feature  of  the  software,  produces  two  highly  valued  resources.  These  two  features 
alone  establish  a novel  identity  for  CMC  supported  distance  education  in  that  neither  aspect  is  normally 
available  in  traditional  F2F  or  distance  models  of  learning.  (Ryan,  1997) 

As  indicated  by  the  literature,  we  found  the  asynchronous  and  archived  exchanges  between  participants  aid  the 
reflective  process  and  students  gain  considerably  from  access  to  a team  of  staff  who  are  able  to  provide 
different  perspectives  on  issues  and  problems.  Team-teaching,  long  since  abandoned  in  f2f  classrooms, 
becomes  a financially  viable  method  of  delivery  by  CMC.  This  virtual  environment  appears  to  provide  more 
opportunities  for  students  to  determine  the  pace,  direction  and  focus  of  discussions,  which  results  in  a more 
learner-centred  approach  to  the  curriculum.  Based  upon  what  has  been  learnt  we  have  now  developed  a 
customised  Notes  desktop  with  databases  that  combine  together  to  give  the  impression  of  a virtual  integrated 
learning  environment  (Ryan  and  Culwick,  1997).  Our  experience  of  using  CMC  has  raised  a number  of  questions 
and  identified  several  issues  for  curriculum  developers  and  learning  technologists.  Of  significance  is  the 
undoubted  impact  that  using  CMC  has  on  teaching  and  learning. 

Impact  on  Learning  and  Teaching 

The  themes  of  the  MSc.  in  CPD  (Health)  are  collaborative  learning,  reflective  practice,  critical  thinking  and 
research.  They  are  explored  in  each  of  the  five  units  of  the  programme,  which  are  followed  sequentially.  Each 
cohort  of  students  stay  together  throughout  the  programme  and  Accreditation  of  Prior  Learning  (APL)  is  not 
used  to  provide  exemptions  from  any  part  of  the  programme.  Although  this  might  indicate  a less  flexible 
approach,  it  is  an  essential  element  that  supports  our  philosophy.  The  responsibilities  inherent  in  being  a 
member  of  an  active  group  enhance  the  sharing  of  individual  experiences  and  perspectives  for  the  benefit  of 
the  group  as  a whole,  “...individual  learning  as  part  of  a group  process**  (Kaye,  1992).  The  flexibility  arises 
from  the  asynchronous  nature  of  the  programme,  giving  the  participants  opportunities  to  connect  to  the  system 
at  any  time  of  day  or  night  from  anywhere  in  the  world,  and  being  able  to  study  off-line,  without  the  pressures 
associated  with  on-line  working. 

Collaborative  Learning 

Professional  development  does  not  occur  in  isolation  but  through  collaborative  learning  and  working.  Within  a 
health  profession  environment  it  is  essential  that  participants  learn  about  each  other’s  professions  and  are 
helped  to  develop  inter-professional  working  practices.  One  student  described  how  this  approach  was 
appropriate  for  her,  “I  have  always  held  near  to  my  heart  the  potential  that  peer-to-peer  collaboration  has  for 
individual  and  multi-professional  CPD.  '* 

The  UK  Government  (1997)  has  clearly  articulated  its  intention  that  in  the  modem  National  Health  Service 
(NHS)  a foundation  for  a changing  health  care  system  must  be  formulated  in  which  the  barriers  between 
professions  are  removed.  This  allows  collaboration  with  other  agencies  concerned  with  the  health,  social  well 
being  and  economic  development  of  the  population  to  occur.  As  each  profession  strives  to  ensure  that  its 
practice  is  evidence-based,  cost-effective  and  can  be  clearly  shown  to  benefit  the  patients  or  clients  it  is 
important  that  professional  boundaries  are  identified  and  clarified  so  that  areas  of  overlap  are  redefined  and 
restructured.  In  the  multi-disciplinary,  collaborative  environment  of  the  MSc.  CPD  (Health)  the  ‘learning 
community’  which  is  formed  enables  participants  to  explore  their  paradigms  of  practice  as  they  share 
experiences  and  draw  on  communal  knowledge.  This  was  eloquently  summarised  by  one  student  commenting 
on  her  impression  of  a computer  conference,  ”...  discussion  is  available  to  the  whole  group  - and  we  can  build 
on  each  other's  contributions  in  the  conference  area  which  provides  opportunities  for  collaborative  decision 
making  through  negotiation  at  all  times.  ** 

The  multi-disciplinary  approach  enhances  a deeper  understanding  of  the  theoretical  and  practical  concepts  in 
the  context  of  the  diverse  social,  economic  and  political  health  environments  in  which  the  participants  work. 
From  an  educational  perspective  there  is  evidence  that  through  collaboration  significant  and  deep-level 
learning  occurs.  In  the  CMC  environment  discussion  between  peers,  colleagues  and  experts  is  a major  feature 


of  the  learning  process.  This  discussion  is  archived  and  available  for  re-examination  and  reconsideration,  and 
it  is  possible  to  see  in  the  interactions  how  the  participants  are  actively  searching  for  the  deeper  meanings  in 
the  debates  and  arguments  presented  by  their  colleagues.  Is  this  the  beginning  of  virtual  ethnography?  Perhaps 
this  is  a question  we  will  soon  be  able  to  set  about  answering. 


Reflective  Practice 

Reflective  practice,  described  by  Thorpe  (1993)  as,  key  process  in  professional  learning.  ” is  a 

fundamental  concept  within  the  programme.  This  is  in  line  with  current  moves  in  all  the  health  professions 
towards  evidence-based  health  care  that  advocates  reflection  on  practice  to  ensure  that  such  practice  does  not 
become  merely  routine.  That  this  is  happening  is  borne  out  by  a comment  from  one  student,  ''reflection  in/on 
action  since  beginning  the  MSc,  has  influenced  any  tendency  to  be  prescriptive,,,  ” Reflective  practice  is  a 
complex  process  of  learning  from  experience  and  involves  challenging  the  taken-for-granted  assumptions, 
values  and  norms  of  practice.  Participants  work  individually  applying  different  models  of  reflection  to  their 
own  practice  and  collaborate  in  the  computer  conferences  to  question  the  purposes  of  reflection  and  to  examine 
the  extent  to  which  reflection  can  shape,  improve  or  enhance  professional  practice.  They  achieve  deeper 
understanding  of  whether  or  not  reflection  can  lead  to  better  practice  and  whether  reflective  practitioners  are 
more  effective  in  practice.  Another  student  wrote,  "For  me  it  was  the  collaborative/shared  learning 
experiences  which  helped  me  to  at  least  begin  to  unravel  the  reflective  process  at  an  academic  level  as  distinct 
from  what  the  majority  of  us  would,  I suspect,  claim  to  have  already  been  doing  within  our  professional 
practice,  ” 


Critical  Thinking 

The  ability  to  think  critically  is  fundamental  to  work  at  Masters  level.  Students  need  to  be  able  to  interpret  and 
evaluate  sources  of  information,  not  only  text  and  research  findings  but  also  conversation,  discussion  and 
argument.  Critical  thinking  is  reasoning  in  which  problems  are  formulated,  assumptions  are  identified  and 
challenged,  alternatives  are  explored  and  facts  and  values  that  result  in  credible  actions  and  beliefs  are 
justified.  In  the  computer  conferences  the  students  are  encouraged  to  apply  their  critical  reasoning  skills  with  a 
positive  and  non-judgmental  attitude.  One  student  commented,  this  critical  thinking  and  analysing  of 
findings  is  vital  for  healthcare  professionals  to  enable  us  to  stake  out  our  own  professional  autonomy,  ” 


Research 

At  the  end  of  the  Masters  programme  the  students  produce  an  individual  piece  of  research  relevant  to  their 
own  practice.  The  evidence  we  are  collecting  suggests  that  their  collaborative  and  reflective  work  in  the 
conference  areas  enhances  their  acquisition  of  research  skills  and  adds  an  extra  dimension  to  their 
development  as  researchers.  One  student  wrote,  ‘*...  I thought  the  bit  on  qualitative  statistics  and  analysis  was 
well-led  and  therefore  it  inspired  more  collaboration,  " which  is  not  the  usual  sort  of  comment  heard  on  a 
research  methodology  unit!  This  (lengthy)  quote  from  another  student  encapsulates  what  we  hope  to  achieve, 
in  the  unusual  nature  of  CMC  classrooms,  it  is  fascinating  to  watch  the  evolution  of  students’  research 
ideas  from  first  thoughts  to  the  final  version  of  the  research  project.  It  is  unusual  because,  following  a 
traditional  degree  pathway,  students  and  tutors  would  not  have  intimate  insight  into  the  development  of  each 
of  our  research  projects.  In  effect  we  are  baring  our  souls  for  all  to  see  and  opening  ourselves  up  to 
constructive  criticism.  The  evolution  and  the  shaping  of  our  research  ideas  gives  us,  students  and  tutors,  an 
extremely  rich  source  of  information:  indeed  this  information  is  privileged,  would  students  and  tutors  on  other 
degree  pathways  have  this  opportunity  to  share  such  information?  I doubt  this  very  much,  " 


Conclusion 

In  this  paper  we  have  attempted  to  show  how  we  have  begun  to  harness  some  of  the  potential  of 
communications  technology  to  facilitate  the  continuing  professional  development  of  health  professionals 
undertaking  a Masters  programme.  We  have  demonstrated  the  use  of  relatively  low-level  technology  and 
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discussed  the  merits  and  advantages  of  this  approach.  We  have  not  dwelt  on  the  disadvantages.  We  are 
certainly  not  suggesting  that  there  are  no  disadvantages,  but  our  experience  so  far  indicates  that  these  are  far 
outweighed  by  the  advantages.  We  hope  that  we  have  been  able  to  show  the  potential  transferability  of  our 
chosen  model  to  other  professional  groups  and  welcome  the  discussion  this  paper  might  initiate. 
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Abstract:  Northern  Virginia  Community  College  (NVCC)’s  Extended  Learning 

Institute  (ELI)  is  currently  bringing  online  learning  ‘into  the  mainstream’  at  NVCC 
through  a large-scale  deployment  of  Web-based  courses  in  key  disciplines.  This 
diffusion  strategy  includes  developing  online  Associate  in  Science  (AAS)  degrees  in 
high  demand  (Information  Systems  Technology,  Business  Management)  or  of  special 
interest  (Public  Management),  and  also  on-campus  courses  with  ALN  components  at  all 
five  NVCC  campuses.  The  project  addresses  Northern  Virginia’s  needs  for  access  to 
“anywhere,  anytime”  educational  opportunities,  particularly  for  information  technology 
and  public  sector  workers.  Incorporating  asynchronous  learning  networks  (ALNs)  into 
courses  enables  utilization  of  rich  content  resources  and  levels  of  interaction  among 
learning  participants  comparable  to  classroom-based  courses.  The  project  thus  far  has 
enabled  significant  opportunities  for  innovative  and  effective  Web  course  design, 
including  use  of  several  Web-based  development  and  delivery  tools,  use  of  publisher- 
created  content  resources,  affordable  Web-based  transmission  of  mathematical 
expressions,  and  cross-course  content  linkage. 


Project  Background  and  Signiflcance 

ELI  as  a Distance  Education  Provider 

Northern  Virginia  Community  College  (NVCC)  is  the  2nd  largest  multi-campus  community  college  in  the 
United  States  and  the  largest  educational  institution  in  Virginia.  NVCC  currently  (Fall  1998  semester) 
enrolls  over  37,000  students.  As  NVCC’s  distance  learning  administrative  unit,  ELI  is  one  of  the  nation's 
oldest,  largest,  and  most  successful  distance  education  programs.  Since  1975,  ELI  has  enrolled  over 
160,000  students  in  its  courses;  ELI  currently  offers  over  140  courses  and  enrolls  over  10,000  students 
annually.  ELI  offers  entire  distance  education  degree  programs  in  General  Studies  and  Business 
Administration  and  has  also  developed  the  first  and  only  Associate  in  Science  (AS)  in  Engineering  degree 
program  available  in  the  United  States  through  home  study  distance  education. 

Since  it  primarily  serves  students  who  cannot  or  prefer  not  to  attend  regular  classes  on  campus,  ELI  has 
relied  largely  on  asynchronous,  or  time-independent,  forms  of  distance  education  delivery  such  as  print 
materials,  computer  conferencing,  voice  mail,  and  pre-recorded  telecourses  or  other  video  programs.  ELI 
students  highly  value  self-paced,  independent  study  because  of  its  flexibility  and  convenience.  However, 
most  of  ELI’s  learners  live  within  20  miles  of  one  of  NVCC's  five  campuses  and  can  handle  occasional 
campus  visits  for  exams,  laboratory  exercises,  or  other  course-related  activities.  Most  ELI  courses  also 
offer  continuous  enrollment,  enabling  students  to  register  on  any  day  of  the  year. 

Consistent  with  the  community  college  mission  to  provide  maximum  access  for  learners,  ELI  has 
generally  relied  on  proven,  reliable,  inexpensive,  and  easy-to-use  technologies  which  minimize  access 
barriers  (Sener  1997a).  Accordingly,  ELI’s  entry  into  development  of  Web-based  courses  has  been 


relatively  recent,  coinciding  with  a sufficiently  large  market  penetration  of  home  computers  and  Internet 
access. 

The  project’s  major  activities  address  the  following  needs:  the  need  for  accessible,  anywhere,  anytime 
learning,  for  information  technology  workers,  and  for  expanded  education  opportunities  related  to  public 
sector  employment  in  Northern  Virginia.  Different  strategies  are  being  used  to  address  each  of  these 
needs  based  on  their  particular  characteristics. 

Need  for  Accessible,  Anywhere,  Anytime  Learning 

This  project  addresses  the  problem  of  providing  accessible  educational  opportunities  to  the  growing  ’’new 
majority”  of  potential  students  who  seek  to  further  their  education  and  attain  degrees  but  have  significant 
obstacles  preventing  them  from  attending  traditionally-structured  on-campus  classes.  Community  college 
programs  in  the  U.S.  are  increasingly  attractive  to  adults  of  all  educational  attainment  levels  who  wish  to 
enhance  their  prospects  for  promotion,  find  new  jobs,  or  even  change  careers.  Community  college  students 
also  often  have  other  demands  on  their  time  such  as  work  and  family  responsibilities,  making  it  more 
difficult  for  them  to  attain  a degree  by  attending  regularly  scheduled  classes  on  campus.  These  factors  help 
explain  the  increasing  popularity  of  asynchronous  learning  networks  (ALNs)  and  other  forms  of 
accessible,  anywhere,  anytime  learning  delivery  systems.  ALNs  provide  access  to  remote  learning 
resources  at  the  learner’s  convenience,  not  dependent  on  synchronous,  real-time  communication.  This 
includes  ‘people  resources’  who  participate  in  the  learning  process  (e.g.,  faculty,  tutors,  peers,  guest 
experts)  and  the  wide  variety  of  possible  content  resources  (e.g.,  lectures  on  demand,  libraries,  computer 
simulations,  collaborative  work  products,  et  al.).  Assuming  that  learning  follows  from  supporting  access 
and  interaction,  ALNs  increase  opportunities  for  interaction  and  collaboration  among  learning 
participants  (Mayadas  1994). 

This  project  will  result  in  the  development  of  at  least  40  courses  which  utilize  ALN  in  a variety  of 
formats,  including  entirely  online,  Web-based  ALN  courses,  largely  online  courses  with  one  or  several 
face-to-face  meetings,  and  on-campus  courses  with  Web-based  ALN  components.  Half  of  these  courses 
will  enable  online  offering  of  AAS  degrees  in  Information  Systems  Technology  (1ST),  Business 
Management,  and  Public  Management;  the  remainder  of  the  courses  are  being  selected  for  development  in 
a wide  variety  of  disciplines  within  all  major  divisions  at  each  of  NVCC’s  five  campuses,  assuring  a 
widespread  diffusion  of  ‘model’  Web-based  courses  throughout  the  College.  As  the  largest  and  most  far- 
reaching  such  initiative  at  NVCC,  this  project  is  creating  a critical  mass  of  distance  education  and  on- 
campus  offerings  which  will  establish  Web-based  ALN  courses  as  a self-perpetuating  norm  at  the  College. 

Need  for  Information  Technology  Workers 

The  AAS  1ST  degree  program  is  designed  to  address  the  critical  shortage  of  information  technology  (IT) 
workers  in  Northern  Virginia  and  the  resulting  high  demand  for  1ST  courses  at  the  College.  The  demand 
for  workers  in  IT-related  fields  in  the  U.S.  has  grown  at  an  unprecedented  rate  during  the  past  decade, 
with  recent  estimates  ranging  as  high  as  346,000  unfilled  IT  jobs  in  the  U.S.  in  1998  (Information 
Technology  Association  of  America  1 999).  High  technology  firms  are  a major  component  of  the  economy 
of  the  Northern  Virginia  region  which  comprises  NVCC’s  service  area.  Northern  Virginia  now  has  the 
2nd  highest  concentration  of  high  technology  industries  in  the  U.S.,  ranking  only  behind  Silicon  Valley  in 
California.  Not  surprisingly,  the  nationwide  shortage  of  IT  workers  is  particularly  acute  in  Northern 
Virginia.  According  to  recent  estimates,  almost  23,000  high  tech  jobs  in  the  region  are  unfilled  (Stough 
and  Trice  1999)  and  112,000  additional  jobs  will  be  available  over  the  next  five  years  (Ryan  McGinn 
Samples  Research,  Inc.  1997). 

The  great  demand  in  Northern  Virginia  for  IT-related  education  and  training  opportunities  is  reflected  in 
the  demand  for  NVCC’s  1ST  courses.  The  1ST  program  is  designed  for  persons  who  need  knowledge  and 
skills  in  business  computer  information  systems  to  enter  or  advance  in  the  field,  or  for  persons  who  wish 
to  apply  1ST  to  their  current  fields.  1ST  is  one  of  the  largest  programs  at  the  College  with  over  2,200 


program-placed  students.  Accordingly,  the  1ST  and  related  courses  are  being  designed  to  accommodate 
moderate  to  high  enrollments,  as  this  program  is  expected  to  produce  at  least  1000  course  enrollments  per 
semester  when  development  is  completed  in  Spring  2000. 

Need  for  Education  Opportunities  Related  to  Public  Sector  Employment 

The  AAS  degree  program  in  Business  Management,  including  the  Public  Management  specialization, 
addresses  the  need  for  education  and  training  opportunities  relevant  to  public  sector  employment  in 
Northern  Virginia,  with  its  high  concentration  of  federal,  county,  and  state  government  employees. 
Although  federal  government  employment  in  the  region  has  not  grown  in  recent  years,  employment  at  the 
county  and  state  levels  has  grown.  There  is  a particular  need  for  education  and  training  for  historically 
underserved  populations  such  as  public  safety  and  support  employees.  These  populations  have  been 
overlooked  since  most  public  administration  programs  of  study  in  the  U.S.  focus  on  producing  Bachelor's 
and  Master's  level  graduates.  However,  community  colleges  are  highly  suitable  institutions  to  provide 
opportunities  for  these  populations  to  obtain  a successful  and  rewarding  education  experience  in  the  field 
of  public  administration  (Banas  and  Emory  1998). 

Although  the  Business  Management  degree  program  is  well  established,  the  Public  Administration 
specialization  is  a new  one  with  relatively  low  enrollments  thus  far.  Accordingly,  several  strategies  are 
being  employed  to  increase  program  enrollments.  These  include  designing  all  of  the  related  program 
courses  for  nationwide  availability;  making  versions  of  the  courses  available  in  a variety  of  formats, 
including  online,  compressed  video,  classroom,  and  on-site  delivery;  and  developing  customized 
certificate  programs  on  demand  for  employers  and  other  clients  with  a sufficient  cadre  of  employees  with 
specialized  needs.  When  completed,  the  Public  Administration  specialization  will  be  one  of  only  two 
such  Associate  degree  programs  available  in  the  U.S.  through  distance  education. 


Program  Implementation  Issues 

Any  large-scale  program  implementation  is  accompanied  by  a number  of  related  issues.  Some  of  the  key 
program  implementation  issues  in  this  project  include  ensuring  accessibility  and  affordability,  selecting 
software  and  technology,  promoting  learner  interactivity,  providing  adequate  support  services,  and 
ensuring  effective  program  evaluation.  Of  particular  interest  here  are  those  issues  related  to  software  and 
technology  implementation. 

Software  and  Technology  Implementation 

One  important  set  of  decisions  which  must  be  made  in  any  large-scale  implementation  of  Web-delivered 
courses  is  what  software  and  technology  to  use.  Such  decisions  include  the  following: 

Off-the-shelf  tools  or  make  your  own?  This  was  an  easy  decision  to  make  for  this  project:  NVCC/ELI 
does  not  have  the  resources  to  make  its  own  Web-based  instructional  development  or  delivery  products,  so 
commercial  software  products  are  being  used. 

Standardize  on  one  product  set  or  allow  many?  Although  there  are  many  advantages  to  requiring  faculty 
to  use  one  set  of  Web-based  development  and  delivery  products,  ELI  decided  to  support  many  such 
products  for  several  reasons: 

• In  our  opinion,  none  of  the  currently  available  commercial  products  are  so  superior  to  the  others  that 
they  merited  exclusive  selection. 

• Because  of  ELPs  “faculty-centered”  course  development  model  which  gives  faculty  primary 
responsibility  for  and  ‘psychological  ownership’  of  course  development  and  delivery,  it  makes  more 
sense  to  give  faculty  more  freedom  to  select  the  tools  they  will  use.  Given  the  current  state  of 
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evolution  of  Web-based  products,  any  selected  product  set  is  likely  to  dissatisfy  many  faculty  who  do 
not  like  that  particular  set  of  features. 

• Much  faculty  course  development  is  “textbook-driven,”  i.e.,  faculty  instructional  design  decisions 
determine  which  textbook  is  selected  for  the  course,  which  in  turn  drives  software  and  technology 
selection.  This  is  because  more  and  more  textbook  publishers  are  supplying  Web-based  software  and 
content  as  an  enhancement  to  their  textbooks.  Some  publishers  are  creating  their  own  Web  content 
and/or  integrating  external  Web  sites  into  their  textbook  content  available  in  CD-ROM  format. 
Many  other  publishers  are  also  bundling  Web  development  and  delivery  tools  with  their  textbooks, 
with  or  without  accompanying  content.  For  example,  McGraw-Hill  Learning  Architecture  uses 
TopClass^^  (McGraw-Hill  1999),  Course  Technology  uses  CyberClass^^  (Course  Technology  1999), 
etc. 

Consequently,  ELI  is  currently  using  a variety  Web-based  development,  delivery,  and  management 
software  tools,  including  Web  Course  in  a Box^^,  CyberClass,  and  Serf^^.  Other  faculty  are  choosing  to 
build  their  own  Web  pages  using  many  of  the  currently  available  tools  such  as  Netscape  Composer^*^, 
Microsoft  Front  Page^^,  AOL  Press^*^,  et  al.  ELI  also  uses  Allaire  Forums^^  as  a Web-based 
conferencing  software  tool  for  faculty  who  wish  to  integrate  discussion  forums  into  their  courses. 


Strategies  for  Innovative  Web  Course  Design 

A common  criticism  of  many  Web-based  courses  is  that  they  are  little  more  than  “page-flipping”  courses 
containing  computer-based  information  with  little  use  of  the  Web’s  special  capabilities  for  enhancing 
instruction.  This  project  is  providing  the  opportunity  to  apply  several  effective  strategies  for  innovative 
Web  course  design. 

Rich  Content 

Incorporating  ALNs  into  course  design  provides  an  effective  framework  for  developing  effective  learning 
environments  because  it  requires  integration  of  substantial  content  resources  as  well  as  activities  which 
promote  interactivity  among  learning  participants  (Sener  1997b).  The  quantity  of  Web-based  content  of 
instructional  value  has  rapidly  increased,  as  have  ways  to  organize  and  use  this  content  effectively.  For 
instance,  the  textbook  to  be  used  in  ELI’s  Introduction  to  Speech  Communication  course  includes 
hundreds  of  content-related  Web  sites  which  is  directly  referenced  to  textbook  content  by  page  number; 
the  publisher’s  web  site  even  uses  software  which  “regenerates”  dead  Web  links  by  automatically 
identifying  a suitable  alternate  link  when  a referenced  Web  link  becomes  inactive  (Allyn  & Bacon  1999). 
Similarly,  there  are  numerous  non-commercial  sites  such  as  The  Electronic  Hallway  (University  of 
Washington  1999),  which  has  a repository  of  case  studies  suitable  for  use  in  course  assignments,  as  is 
done  in  ELI’s  public  administration  courses. 

Promoting  Interactivity  among  Learning  Participants 

In  designing  ELI’s  Web-based  courses,  the  standard  of  desired  level  and  quality  of  interactivity  among 
learning  participants  is  that  the  opportunities  for  interaction  are  equal  to  or  greater  than  those  available  in 
the  comparable  classroom  course.  In  some  cases,  this  is  a relatively  modest  standard  to  meet,  as  many 
classroom  courses  do  not  necessarily  generate  much  interaction  among  learners  or  between  learners  and 
other  learning  participants  (instructor,  tutor,  etc.).  Consequently,  some  courses  rely  more  on  optional 
interactivity,  i.e.,  available  upon  demand,  while  other  courses  integrate  mandatory  interactivity  into  their 
assignments  (e.g.,  collaborative  projects,  homework  assignment  review,  etc). 

Mathematics  on  the  Web 
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Enabling  two-way  electronic  transmission  of  graphics,  equations,  and  formulas  has  always  been  a 
particularly  vexing  problem.  ELI  previously  developed  a serviceable  means  of  accomplishing  this  by 
using  Expressionist  and  FirstClass™  (Sener  1997a),  enabling  electronic  transmission  of  mathematical 
expressions  using  a remote  server  accessible  via  local  telephone  service.  However,  this  solution  was  not 
technologically  feasible  for  Web-based  courses,  so  a new  solution  had  to  be  found. 

The  current  Mathematics  for  the  Liberal  Arts  I course  uses  Scientific  Notebook™  (SN)  as  the  means  of 
mathematical  communication  with  students.  The  instructor  and  students  use  the  software  to  transmit 
quizzes,  homework  problems,  and  general  questions  and  answers  to  each  other.  The  software  is  priced 
very  affordably  (currently  <$30U S/copy)  as  it  is  bundled  with  the  course  textbook.  SN  can  be  used  as  an 
equation  editor;  it  also  has  a lot  of  computational  power  as  is  it  incorporates  the  symbolic  engine  from 
Maple™,  and  it  has  one  of  the  easiest  user  interfaces  of  the  available  math  software  packages.  Since  it  is 
not  currently  possible  to  post  SN  files  directly  on  the  web,  the  instructor  uses  MSWord™  with 
MathType™,  the  full  version  of  the  Equation  Editor  that  comes  with  Word  to  post  Web  pages  with 
mathematical  content  (Goral  1999).  This  procedure  is  much  easier  and  quicker  to  download  than  the 
previous  common  practice  of  embedding  mathematical  expressions  within  graphics  on  Web  pages. 

Cross-Course  Content  Linkage 

One  particularly  exciting  set  of  possibilities  enabled  by  a large  array  of  Web-based  courses  is  linkage  of 
content  across  courses.  The  instructional  possibilities  of  cross-course  content  linkage  are  just  beginning 
to  be  explored;  one  strategy  which  ELI  is  using  is  to  link  “helper”  courses  to  other  academic  courses.  One 
of  ELFs  new  courses.  Library  Skills  for  Research  (LBR  105)  is  a one-credit  course  designed  to  teach 
students  basic  library  research  skills,  including  online  research  (Egan,  Delmore,  and  Wharff  1 999).  The 
course  includes  mandatory  participation  in  discussion  forums  which  are  monitored  by  librarians  at  each  of 
NVCC’s  five  campuses.  Other  Web-based  courses  with  a research  component  are  starting  to  use  the 
library  research  skills  course  as  a resource  by  including  a link  in  their  Web  pages  to  the  LBR  105  course 
for  students  who  need  additional  assistance  in  completing  their  research  assignments.  Students  can  use  the 
linked  LBR  105  content  as  a refresher  or  reference  resource,  while  students  who  need  more  extensive 
assistance  can  sign  up  for  the  course  and  receive  online  assistance  from  the  librarians. 


Project  Status 

Currently  (Spring  1999  semester),  seven  courses  are  being  offered  with  over  280  enrollments;  an 
additional  ten  courses  will  be  available  in  the  Summer  1 999  semester.  When  the  project  is  completed  in 
the  Spring  2000  semester,  the  complete  AAS  degree  programs  in  1ST,  Business  Management,  and  Public 
Management  will  be  available.  A complete  Associate  in  Arts  degree  in  Liberal  Arts  will  also  be  available 
starting  in  the  Fall  1999  semester. 

For  more  current  information  on  this  project,  visit  the  “Bringing  ALN  into  the  Mainstream  at  NVCC’ 
project  website  link  at  http://elisp.nv.cc.va.us/. 
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Abstract:  In  this  paper  we  describe  the  new  tools  for  synchronous  and  asynchronous  learning 
developed  within  the  context  of  the  TeleTeaching  Projects  at  the  University  of  Mannheim.  With 
our  tools,  students  are  enabled  to  follow  lectures  not  only  in  the  lecture  halls  of  the  connected 
universities  but  also  at  home.  The  innovative  processing  of  course  material  allows  the  production 
of  educational  documents  that  consider  the  requirements  of  synchronous  and  asynchronous 
learning  modes  out  of  a single  primary  document.  Course  material  is  provided  within  a Web-based 
training  unit  designed  specifically  to  support  asynchronous  student  learning. 


1.  Introduction 

In  the  Fall  of  1995  the  University  of  Mannheim  initialized  a joint  Teleteaching  project  with  several  partner 
universities  (Eckert  et  al.  97)  in  Germany.  Three  different  synchronous  Teleteaching  scenarios  were  implemented 
and  evaluated.  In  these  synchronous  learning  scenarios,  production  and  consumption  of  the  educational  material 
takes  place  at  the  same  time.  The  first  scenario  is  called  the  "Remote  Lecture  Room  Scenario”  (RLR).  Lectures  are 
transmitted  live  via  the  Internet  to  lecture  rooms  at  the  partner  universities.  Transmission  is  entirely  based  on  the 
Internet  protocols  so  that  students  from  anywhere  can  participate  easily.  The  second  scenario  we  explored  is  called 
the  "Remote  Interactive  Seminar”  (RJS).  In  contrast  to  the  RLR  scenario,  a seminar  is  much  more  interactive  since  a 
discussion  between  the  three  sides  usually  takes  place  after  each  talk.  The  third  scenario  is  called  "Interactive  Home- 
Learning”  (IHL).  Its  target  users  are  students  running  Linux  or  Windows  95/98  on  their  PC  at  home  having  an 
Internet  connection  via  ISDN.  This  scenario  is  the  most  widely  distributed  among  its  members.  The  Goal  of  the  IHL 
project  is  the  provision  and  interactive  usage  of  multimedia  teachware  within  the  bounds  of  academic  education  over 
digital  networks  in  order  to  gain  experience.  From  a pedagogical  point  of  view  information  about  the  acceptance  and 
learning  success  of  such  a scenario  should  be  retrieved. 

In  addition  to  the  synchronous  scenarios,  asynchronous  learning  is  supported.  In  the  asynchronous  scenario,  the 
student  is  provided  with  course  material  for  the  lectures  and  seminars.  This  material  can  be  reviewed  at  any  time  and 
at  the  speed  the  student  desires.  Especially  in  the  preparation  for  an  exam,  access  to  course  material  is  viable  for  the 
students.  Besides  the  material  used  in  the  lecture  (slides  etc.)  additional  documents  are  handed  to  the  students  in  the 
asynchronous  scenario.  E.g.  the  video  and  audio  recordings  of  the  lectures  provide  access  to  the  explanations  by  the 
professor  of  a specific  topic.  Further  educational  material  (texts,  animations  etc.)  may  provide  background 
information  on  selected  topics  of  the  lecture. 

Many  other  related  projects  in  the  area  of  distance  education  exist  (Bacher  et  al.  97)  (Baehring  et  al.  98)  (Schill  et  al. 
98)  (Fames  97)  (Maly  et  al.  97).  Howerver  in  contrast  to  the  other  projects,  our  approach  integrates  synchronous  and 
asynchronous  forms  of  learning.  The  main  features  are  the  automatic  production  of  educational  material  for  both 
forms  of  learning  from  a single  primary  document.  Furthermore  we  provide  students  access  to  the  synchrounous  and 
asynchronous  learning  scenarios  from  their  home. 

2.  Synchronous  Scenario 


Our  main  goal  in  the  synchronous  scenario  was  to  gain  experience  in  the  transmission  of  lectures  to  student 
households  via  ISDN.  Thus  an  infrastructure  had  to  be  established  and  software  components  designed  and 
implemented.  From  an  pedagogical  point  of  view  information  about  the  acceptance  and  the  learning  success  of  the 
scenario  had  to  be  obtained.  (Fig.  I)  depicts  the  synchronous  IHL  scenario.  Lectures  were  transmitted  via  high- 
bandwidth  ATM  connections  to  the  remote  lecture  rooms  and  simultaneously  to  the  student  PC  at  home  via  a low- 
bandwidth  ISDN  connection. 

2.1.  Implementation 

During  the  summer  semester  1998  we  evaluated  the  scenario.  Four  groups  of  participants  were  selected  and  equipped 
in  order  to  attend  to  our  teleteaching  lectures.  The  first  group  of  participants  attended  the  lectures  directly  in  the 
classroom  where  the  lecture  took  place.  The  second  group  of  participants  was  located  at  a partner  university, 
connected  via  ATM  links  with  a reserved  bandwidth.  The  reserved  link  allowed  the  transmission  of  high-quality 
video  of  the  lecturer.  The  third  group  consisted  of  participants  who  attended  the  lectures  individually  at  various 
universities  in  Germany,  connected  via  the  public  MBone,  i.e.  they  depended  on  the  best  effort  IP  protocol  for  packet 
delivery  without  bandwidth  reservation.  The  fourth  group  consisted  of  students  who  attended  the  lectures 
individually  at  home.  These  participants  were  connected  via  ISDN  and  used  multimedia  PCs  running  Linux  or 
Windows  95/98  or  NT.  Since  ISDN  provides  a reserved  bandwidth  of  64  kbit/s  or  128kbit/s  (if  two  channels  are 
combined)  the  students  could  depend  on  a relatively  reliable  connection,  although  the  bandwidth  available  is 
relatively  low. 


Figure  : The  Integrated  Home  Learning  - Teleteaching  Scenario 

It  proved  to  be  a complex  task  to  set  up  a multicast  connection  among  these  different  groups  of  participants.  (T  ig.  1.) 
depicts  our  solution:  The  reserved  ATM  connection  to  the  participants  of  the  second  group  was  provided  as  a 
Permanent  Virtual  Circuit  with  a peak  bitrate  of  3 MBit/s.  Upon  this  PVC  a LAN  emulation  was  set  up:  At  both  sites 
Edge  Devices  were  installed  which  acted  similar  to  remote  repeaters.  Ethernet  frames  were  transmitted  over  ATM 
connections.  Since  participating  hosts  could  be  connected  through  conventional  ethemet  this  solution  allowed  a 
flexible  integration  of  new  hosts  into  the  virtual  LAN.  A dedicated  router  equipped  with  two  Ethemet  interfaces 
acted  as  multicast  gateway  to  the  MBone  (router  A in  (Tig.  1 )1. 

The  simultaneous  transmission  of  multicast  streams  through  high-bandwidth  ATM  connections  and  low-bandwidth 
ISDN  connections  creates  two  technical  problems:  At  first,  the  transmission  of  high-quality  video  requires  a high 
bandwidth  of  at  least  600kbit/s  but  ISDN  provides  only  128kbit/s.  . Second,  broadcasting  networks  like  Ethemet 
inherently  allow  multicasting.  Since  ISDN  is  not  a broadcasting  channel,  a tunnel  must  be  set  up  dynamically  in 
order  to  transmit  multicast  data. 

The  Dial-In  Multicast  Gateway  allows  the  transmission  of  selected  teleteaching  lectures  sent  via  IP  multicast,  over 
dial-in  connections  such  as  ISDN.  It  sets  up  tunnels  dynamically;  thus  selected  multicast  sessions  can  be  transmitted 
through  channels  without  multicast  capabilities.  Furthermore  it  includes  a simple  filtering  mechanism  that  scales  the 
bandwidth  needs  of  transmitted  streams  according  to  given  priorities.  For  each  selected  media  stream  of  the  lecture 
the  desired  quality  of  service  parameters  can  be  set  interactively,  e.g.  a certain  share  of  the  available  bandwidth  can 
be  reserved.  In  order  to  allow  a graceful  scaling  of  video  we  integrated  a simple  scaling  mechanism  for  H.261  video 
streams  that  controls  the  temporal  resolution  of  the  video. 

In  principle  the  gateway  works  in  the  following  manner:  Users  connect  to  the  server  through  the  client  application. 
Once  a user  has  logged  in  the  client  will  receive  a list  of  lectures  available  for  transmission.  The  user  will  then 
choose  which  lecture  to  join.  A request  for  each  selected  media  streams  of  this  lecture  will  be  sent  to  the  server, 
which  then  will  transmit  and  rescale  the  media  stream  according  to  the  bandwidth  available,  A more  extensive 
description  of  the  Dial-In  Gateway  can  be  found  in  (Kuhmiinch  98). 
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2.2.  Evaluation  results 


During  the  summer  semester  1998  a lecture  on  Multimedia  Technology  by  Professor  Effelsberg  was  implemented 
with  the  technology  described  above.  The  remote  classroom  was  located  at  the  University  of  Freiburg.  A Web-based 
questionnaire  was  used  for  the  evaluation  of  the  synchronous  scenario.  Besides  sozio-demographic  information  it 
assessed  aspects  of  motivation,  the  technical,  organizational  and  didactical  quality  of  the  scenario  and  several  aspects 
of  the  subjective  bearing  and  perception  during  the  teleteaching  lessons.  The  questionnaire  was  similar  to  the 
standardized  questionnaire  HILVE  (Rindermann  et  al.  94).  Also  assessed  was  information  on  the  software  tools  for 
the  ISDN  users. 

A total  number  of  65  students  attended  the  lectures.  43  students  participated  directly  in  the  classroom,  12  students 
studied  at  home  and  10  students  in  the  remote  classroom,  connected  through  the  virtual  LAN.  Since  the  number  of 
participants  in  each  group  is  small  and  differs  widely  we  had  to  dispense  with  inference-statistical  evaluations.  The 
following  evaluations  therefore  are  purely  descriptive  but  we  refer  to  related  evaluations  of  other  projects  where 
possible  in  order  to  buttress  our  interpretation  of  the  evaluation  data. 

Comparisons  between  the  "remote  situation"  and  the  "local  situation"  showed  that  the  students  always  gave  a higher 
rating  to  the  "local  situation".  The  Home  Learning  students  as  well  as  those  in  the  remote  classroom  rated  perception 
in  a "normal"  face-to-face  lecture  higher.  The  reasons  mentioned  were  the  unfamiliarity  of  the  video  on  the  screen  or 
the  video  beamer  and  the  poor  audio  quality  combined  with  the  merely  sufficient  visual  quality  of  the  slides  in  the 
whiteboard.  Additionally  students  in  remote  situations  do  not  feel  themselves  to  be  perceived  by  the  lecturer  as  well 
as  they  would  have  been  in  the  local  situation.  They  also  reported  difficulties  concentrating  for  a full,  uninterrupted 
90-minute  lecture.  In  order  to  overcome  this  problem,  we  introduced  a 5 -minute  break  after  the  first  45  minutes. 
These  evaluation  results  coincide  with  those  in  (Eckert  et  al.  97). 

We  evaluated  how  the  students  assessed  the  quality  of  the  three  media  types  audio,  video  and  whiteboard  with 
respect  to  their  importance  in  the  learning  process.  Since  Home  Learning  students  and  those  in  the  remote  classroom 
received  the  lecture  in  differing  qualities  the  interpretation  of  their  results  are  discussed  separately.  (1.)  Home 
Learning  students:  The  evaluation  data  gives  reason  to  the  hypothesis  that  video  is  only  of  minor  importance  in 
computer  science  lectures,  although  the  received  quality  was  rated  as  merely  sufficient.  In  contrast  the  importance  of 
audio  and  whiteboard  was  rated  high.  While  the  whiteboard  quality  received  good  ratings  the  audio  again  was 
perceived  as  merely  sufficient.  (2.)  Students  in  the  remote  classroom:  Obviously  the  students  in  the  remote  classroom 
demanded  higher  quality  of  the  video.  They  rated  to  the  importance  of  video  average  and  the  importance  of 
whiteboard  and  audio  as  high.  While  the  video  and  whiteboard  quality  fulfilled  the  student  demand  the  audio  quality 
was  rated  as  needing  improvement. 

Specific  questions  designed  for  the  evaluation  of  the  Home  Learning  scenario  indicated  that  the  participants  were 
partly  motivated  by  technical  interest  in  ISDN  and  partly  by  personal  contact  to  the  department.  Additionally  some 
students  found  it  advantageous  that  they  did  not  have  to  commute  to  the  university.  Frequent  problems  with  the 
gateway  router  (router  A in  ( Fig.  1 ))  caused  the  reliability  of  the  connection  to  receive  only  an  average  ranking. 
Nevertheless  all  students  would  participate  in  the  experiment  again,  especially  if  higher  bandwidth  (provided  for 
example  over  ADSL)  were  available. 

An  important  finding  is  that  the  Home  Learning  scenario  is  generally  accepted  by  students  although  higher 
bandwidth  is  obviously  necessary.  Nevertheless  students  in  the  remote  classroom  as  well  as  the  home  learning 
students  reported  a less  personal  atmosphere.  This  result  coincides  with  other  evaluations  (Eckert  et  al.  97). 

3.  Asynchronous  Scenario 

3.1.  Objectives 

One  of  the  main  goals  of  the  asynchronous  scenario  is  to  provide  the  student  with  educational  material  that  can  be 
visited  selectively  depending  on  one’s  personal  learning  process.  The  material  usually  accompanies  a lecture  that  is 
held  in  the  synchronous  scenario.  In  contrast  to  the  synchronous  scenario,  where  content  is  pushed  towards  students, 
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the  asynchronous  scenario  enables  a student  to  selectively  pull  the  desired  material.  Due  to  this  fundamental 
conceptual  difference,  the  range  and  the  amount  of  material  presented  in  the  asynchronous  scenario  may  be  much 
greater  than  in  the  synchronous  scenario.  Additional  material  may  also  be  used  to  explore  certain  topics  of  the  lecture 
in  more  depth  or  to  address  different  needs  of  learners  by  providing  compact  explanations  for  fast  learners  and  full 
texts  for  slower  ones.  Since  students  interact  directly  with  the  material,  highly  interactive  material  like  animations 
and  simulations  can  be  handed  to  them. 

3.2.  Preparation  of  Course  Material 

The  lecture  material  of  the  course  on  Multimedia  Technology  given  in  the  summer  semester  of  1 998  was  presented 
not  only  during  the  synchronous  lectures  but  was  also  provided  for  asynchronous  learning.  Since  the  demand  of 
lecture  material  in  both  scenarios  differ,  a novel  authoring  process  was  developed  that  allows  efficient  production  of 
material  for  both.  We  also  set  up  a system  which  allows  the  material  to  be  captured  that  was  created  during  the 
synchronous  lecture. 

3.2.7  Authored  Material 

For  the  material  authored  by  hand  (e.g.  the  slides  of  a lecture)  we  found  it  to  be  a great  advantage  to  leave  the  choice 
of  the  authoring  tool  to  the  lecturer.  This  enabled  the  lecturer  to  efficiently  create  the  course  material  because  no  time 
was  needed  to  learn  the  operation  of  a new  tool.  The  lecturer  may  also  reuse  any  material  he  has  created  previously 
with  his  favorite  authoring  tool.  This  will  tremendously  reduce  the  amount  of  time  required  to  prepare  a lecture, 
especially  if  material  from  previous  semesters  is  used.  The  documents  produced  with  the  chosen  authoring  tool  are  in 
a tool-specific  document  format  (e.g.  Microsoft  Word  format)  which  is  usually  not  appropriate  for  the  presentation  of 
the  material.  We  call  this  document  format  a primary  format. 

As  mentioned  above,  the  lecture  material  in  the  synchronous  scenario  is  presented  using  the  MBone  whiteboard  wb 
(Jacobson  92)  which  is  able  to  process  documents  in  postscript  format.  Within  the  asynchronous  scenario,  the  course 
material  is  provided  on  the  teachware  Web  server.  Since  both  scenarios  must  be  considered  during  the  authoring 
process,  a conversion  from  the  primary  format  to  multiple  presentation  formats  is  required.  This  conversion  process 
should  be  executable  with  minimal  user  interaction.  In  particular,  it  should  not  be  necessary  to  edit  the  conversion 
results,  because  these  changes  would  be  lost  if  the  conversion  process  is  repeated. 

The  authoring  process  we  have  implemented  allows  to  produce  the  presentation  formats  postscript,  PDF  and  HTML 
out  of  a single  primary  format  document.  The  authoring  process  is  subdivided  into  three  steps.  In  the  first  step,  the 
document  is  edited  using  the  desired  authoring  tool  (in  our  case,  Microsoft  Word).  In  the  second  step,  the  document 
is  converted  to  the  presentation  format.  In  contrast  to  postscript  and  PDF,  the  conversion  to  HTML  proved  to  be  very 
difficult.  Our  experiments  with  HTML  export  filters  showed  that  the  resulting  HTML  documents  often  could  not  be 
post-processed  any  further.  Very  good  results  were  achieved  by  exporting  a document  to  the  RTF  format  and 
converting  the  RTF  document  to  HTML.  This  process  is  suitable  for  all  authoring  tools  that  are  able  to  export  RTF 
documents  and  therefore  does  not  restrict  the  flexibility  in  the  choice  of  the  authoring  tool. 

In  the  third  step  of  the  authoring  process,  the  HTML  document  resulting  from  step  two  is  post-processed  for  its 
actual  presentation.  Using  a Perl  script,  the  WBT  described  in  Section  3.3  is  automatically  generated.  To  accomplish 
this  task,  the  script  reads  a configuration  file  in  which  the  linking  structure  of  the  WBT  is  specified  and  inserts 
appropriate  links  into  the  HTML  documents.  The  configuration  file  can  be  edited  by  the  author  to  specify  further 
material  that  should  be  integrated  into  the  WBT. 

5.2.2  Captured  Material 

Course  material  can  be  gathered  during  the  live  lecture  by  recording  the  audio  and  video  data.  In  our  case,  we 
recorded  audio  and  video  using  the  MBone  VCR  recorder  (Holfelder  97). 

Besides  audio  and  video,  an  important  source  of  material  are  the  slides  that  are  presented  by  the  lecturer  using  the 
wb.  In  contrast  to  the  audio  and  video  data,  wb  data  cannot  be  recorded  with  a state-of-the-art  recorders  like  the 
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MBone  VCR,  But  even  if  the  data  of  the  wb  could  not  be  recorded  directly  in  the  past  semester,  it  proved  to  be  a 
good  source  of  information  for  assigning  parts  of  the  recorded  audio  and  video  to  the  corresponding  slides  of  the 
lecture,  The  recording  of  a lecture  comprises  the  full  90  minutes.  In  order  to  determine  the  parts  which  belong  to  a 
single  slide,  the  recording  must  be  indexed.  This  indexing  is  done  automatically  during  the  recording  of  the  lecture 
by  our  tool  listenwb.  This  tool  examines  the  data  stream  produced  by  the  wb.  It  records  the  time  at  which  a slide  is 
changed  in  the  wb.  This  point  in  time  is  converted  to  a timestamp  relative  to  the  start- time  of  the  video  and  audio 
recording  by  subtracting  an  offset.  At  the  end  of  a recording,  an  index  is  created  by  listenwb  which  contains  the 
timestamps  of  page  changes  relative  to  the  beginning  of  the  video  and  audio  recording.  This  index  is  the  foundation 
of  the  configuration  file  used  by  the  perl  script  in  the  generation  of  the  WBT, 

3.3.  Web-Based  Training  (WBT)  Unit 

The  complete  course  material  of  the  1998  Multimedia  Technology  course  was  combined  into  a WBT  (Effelsberg  98), 
This  WBT  integrates  all  the  material  (hand-authored  and  captured)  and  provides  access  through  the  common  user 
interface  of  a Web  browser  (FiR>  3 ). 


3.3.1  Applying  an  Instructional  Design 

A very  important  aspect  of  a WBT  is  the  appropriate  structuring  of  the  material  it  contains.  For  this  reason,  the 
instructional  design  of  the  WBT  (Fig.  2)  was  developed  in  cooperation  with  a research  group  at  the  Department  of 
Psychology  at  the  University  of  Heidelberg  under  the  direction  of  Professor  Reimann.  The  foundation  of  the  WBT 
consists  of  the  slides  of  the  lecture.  These  slides  are  linked  in  the  order  in  which  they  were  presented  during  the 
lecture.  Each  slide  contains  a forward  and  a backward  button  in  the  navigation  bar  on  top  of  the  screen  (Fig.  3), 
This  allows  a student  to  successively  walk  through  the  complete  course.  Additionally,  a self-expanding  index  enables 
a student  to  access  material  on  a specific  topic.  Each  slide  contains  a link  to  the  corresponding  part  of  the  video  and 
audio  recording  of  the  lecture.  Some  of  the  slides  contain  links  to  additional  material.  This  additional  material  is 
structured  in  multiple  layers  where  each  layer  contains  different  material  (full  text,  animations  and  exercises)  on  the 
same  topic  as  the  slide.  Thus,  a student  may  descend  through  different  layers  until  the  topic  has  been  completely 
understood.  Afterwards,  the  student  ascends  to  the  upmost  level  and  continues  with  the  next  slide.  Currently  we  have 
integrated  Java  animations  and  exercises  into  the  WBT;  the  integration  of  full  text  will  follow 

4.  Conclusions 

In  this  paper  we  summarized  our  synchronous  and  asynchronous  teaching  and  learning  scenarios.  We  described 
details  of  the  technical  implementation  and  presented  initial  evaluation  results  which  prove  that  teleteaching  in  its 
synchronous  mode  is  generally  accepted  by  the  students  although  they  expect  high  technical  and  organizational 
quality.  For  the  asynchronous  scenario,  we  presented  an  innovative  course  material  processing  scheme  that  enables 
the  production  of  educational  documents  considering  the  requirements  of  different  learning  modes  out  of  a single 
primary  document. 

Future  work  aims  to  add  interactive  media  such  as  VRML  or  Java  animations  and  simulations  as  new  educational 
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paradigms  to  our  asynchronous  and  synchronous  teleteaching  scenarios.  This  comprises  the  development  of  a general 
framework  for  the  transmission  and  recording  of  interactive  media  (Mauve  et  al.  98). 
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Abstract:  In  this  paper,  we  present  Project  Clio,  a Web  logging  system  that  permits  instructors, 
instructional  designers,  Web  designers,  and  course  web  architects  to  better  gamer  information 
that  helps  them  understand  how  students  use  course  webs,  and  therefore  how  to  improve  course 
webs.  In  particular,  the  system  allows  one  to  determine  the  path(s)  individual  users  follow 
through  a site,  the  time  they  spend  on  each  page,  and  their  simultaneous  use  of  multiple  pages. 

While  there  are  now  a number  of  systems  for  logging  Web  use,  ours  is  able  to  provide  more 
targeted  and  more  individualized  information  than  typical  systems. 

1.  Introduction 

Students  at  most  universities  now  have  access  to  the  World-Wide  Web,  putting  the  Web  in  a position  to  be  the 
common  medium  of  hypermedia  for  class  materials.  Hypermedia  has  many  advantages  over  a paper-based 
presentation  of  information.  Of  primary  importance  are  that  hypermedia  (1)  permits  the  easy  integration  of  media 
other  than  text  and  (2)  supports  a structure  quite  different  from  the  linear  presentation  of  textbooks.  Links  allow 
students  to  access  related  material  easily,  from  various  nodes  in  the  course  web. 

At  the  same  time,  this  new  technology  requires  analysis  of  its  use  and  usefulness.  Instructors  need  to  know  how 
students  actually  use  Web  materials  so  that  they  can  better  learn  how  the  Web  and  their  course  webs  affect  students’ 
understanding  of  information  and  their  performance  in  class.  Such  analysis  requires  careful  and  regular  gathering  of 
information  on  how  individual  students  use  course  webs. 

The  Web  was  not  originally  designed  to  provide  a great  deal  of  specific  information  on  how  readers  use  individual 
sites  and  pages.  The  log  files  generated  by  most  Web  servers,  such  as  the  NCSA  HTTPd  (NCSA  1997)  are 
inadequate  for  careful  analyses.  In  particular,  log  files  list  only  pages  accessed,  the  time  the  page  was  accessed,  and 
the  machine  that  issued  the  request,  but  not  the  individual  that  accessed  the  pages,  the  previous  and  next  pages  on 
that  individual’s  exploration  of  the  Web,  or  the  amount  of  time  the  person  spent  on  the  page.  Successful  acquisition 
of  more  detailed  data  requires  working  around  this  original  design  decision.  Recently,  new  technologies  such  as 
cookies,  Java,  and  Javascript  have  extended  the  opportunities  not  just  for  hypertext  on  the  web,  but  also  for  tracking 
and  analyzing  use  of  the  Web.  Using  these  technologies  it  is  now  possible  to  track  students  more  closely  as  they 
move  within  and  beyond  the  course  web.  In  particular,  we  can  log  the  path  of  each  particular  student  and  record  the 
time  each  student  spends  on  each  page. 

2.  Improving  Understanding  of  Site  Usage 

Why  analyze  course  webs?  The  primary  goal  of  course  web  analysis  is  to  help  teachers,  site  designers,  and 
instructional  designers  as  they  evaluate  the  use  and  usefulness  of  course  webs.  By  “evaluate  use”,  we  mean  an 
analysis  of  what  pages  students  use,  how  long  they  use  them,  how  they  get  to  them,  how  they  leave  them,  and  what 
other  pages  they  use  in  conjunction  with  those  pages.  By  “evaluate  usefulness”,  we  mean  a consideration  of  the 
effects  of  pages  and  use  patterns  on  learning,  based  not  just  on  the  preliminary  analysis,  but  also  with  additional 
infonnation  such  as  records  of  student  performance  or  targeted  interviews,  as  in  (Jones  et  al.  1996). 

There  are  many  potential  benefits  from  more  thorough  analysis  of  course  web  usage.  These  include  a better 
understanding  of  student  learning  patterns,  justification  for  the  time  and  effort  spent  developing  a site;  and  support 
for  improving  the  site.  While  such  benefits  can  also  be  gained  in  careful  study  of  commercial  sites,  the  focus  of  our 
tool  is  on  understanding  student  patterns. 
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Information  presented  with  hypermedia  does  not  need  to  appear  in  linear  format  as  in  a lecture  or  in  a text  book. 
This  change  can  have  a dramatic  effect  on  how  well  students  learn  and  retain  course  material.  An  analysis  system 
can  allow  instructors,  instructional  designers,  and  course  web  architects  to  correlate  viewing  information  on  the  Web 
with  related  test  questions  (Trochim  1996),  student  interviews  (Jones  et  al.  1996),  or  student  perceptions  of  their 
own  web  use  (Rebelsky  1998). 

With  more  detailed  usage  logs,  one  can  also  ask  questions  like 

• What  are  the  more  popular  parts  of  the  course  web,  and  are  those  parts  used  more  for  their  links  (short  time  on 
page)  or  content  (long  time  on  page)? 

• How  do  readers  reach  particular  pages?  Can  they  quickly  reach  the  pages  they  want? 

• How  do  better  and  worse  students  use  the  web?  Can  all  students  benefit  from  the  strategies  of  the  best  students? 

• Which  pages  do  students  seem  to  leave  active,  perhaps  as  a reference  in  a background  window,  while  visiting 
other  pages?  Are  there  ways  to  improve  the  use  of  reference  pages  in  such  situations? 

To  permit  these  sorts  of  analyses,  a logging  system  must  identify  students,  keep  track  of  the  paths  they  take  through 
the  course  web,  and  identify  the  amount  of  time  they  spend  on  individual  pages.  This  means  the  system  needs  to  log 
when  a reader  loads  a web  page,  how  they  found  the  page,  how  long  the  page  is  open,  and  which  link  (if  any)  is 
followed  next.  Because  the  web  permits  readers  to  open  multiple  “windows”  on  the  same  site,  we  also  need  to  be 
able  to  determine  when  they  have  several  pages  open  concurrently. 

3.  Architecture 

Project  Clio  is  based  on  a number  of  interconnected  systems,  each  of  which  provides  a different  role  in  tracking  or 
helping  analyze  student  use. 

• An  identification  system  (described  in  section  3.1)  identifies  individual  users  and  provides  this  information  to 
the  other  parts  of  the  system. 

• A page  transducer  acts  as  an  intermediary,  fetching  requested  pages  and  extending  them  so  that  (1)  they  can  be 
tracked  with  the  tracking  system,  (2)  links  from  the  page  use  the  transducer  for  subsequent  requests,  and  (3) 
additional  activities  can  be  performed  during  the  fetch. 

• A tracking  system  (described  in  section  3.2)  records  simple  events  which  are  then  used  to  develop  usage 
information.  This  system  includes: 

O an  event  logger  which  logs  individual  events  to  a file  for  later  analysis; 

O a request  analyzer  (server  side)  which  gamers  information  from  the  initial  page  request;  and 
O a page  observer  (client  side)  which  gamers  information  from  the  client,  such  as  whether  the  student  is  still 
on  the  page. 

• A synthesis  system  which  translates  the  “raw  events”  (such  as  “the  user  has  loaded  page  A" ...  the  user  is  still  on 
page  ...  the  user  has  left  page  X")  to  a more  usable  form  (such  as  “the  user  was  on  page  X for  five  minutes”). 

• An  analysis  system  which  permits  interactive  analysis  of  the  translated  usage  information  and  supports 
questions  like  those  given  above. 

In  this  paper,  we  focus  on  how  the  system  tracks  users  with  an  identification  system  and  a tracking  system. 

3.1.  The  identification  system 

The  identification  system  is  loosely  modeled  after  the  Unix  model  of  users  and  groups.  Each  user  entry  contains  a 
single  line  in  a file  that  contains  all  of  the  information  for  a user.  Each  entry  contains  the  account  name  of  the  user, 
an  encrypted  password,  the  groups  that  the  user  belongs  to,  the  real  name  the  real  name  of  the  user  (or  anonymous, 
for  users  who  do  not  wish  their  name  to  be  known),  and  an  email  address  for  contacting  the  user. 


We  use  a group  model  so  that  different  types  of  tracking  are  possible.  For  example,  one  might  want  to  compare 
different  classes  using  the  same  web  site  or  only  log  members  of  a particular  group.  The  group  model  is  also 
appropriate  for  related  uses  of  the  system,  such  as  an  annotation  system  which  takes  advantage  of  this  system 
(Luebke  and  Mason  1998)  (Luebke,  Mason,  Rebelsky  1998). 

When  users  first  access  pages  that  are  logged,  they  are  prompted  by  a form  that  asks  for  account  and  password. 
Once  the  user  has  logged  in,  a session  and  session  identification  number  are  created  for  that  user.  As  the  user 
continues  throughout  the  logged  pages  the  session  identification  number  is  carried  with  them  via  query  strings  and 
cookies.  Once  the  user  exits  the  logged  pages  or  leaves  the  browser  the  session  number  is  expired  and  they  must 
login  again  if  they  choose  to  return  to  logged  pages.  In  this  way,  only  registered  users  of  the  system  may  view  the 
logged  pages.  This  can  easily  be  circumvented  by  allowing  a guest  account.  In  the  future  it  should  also  be  possible 
to  restrict  certain  pages  for  certain  groups  or  to  restrict  logging  to  certain  Internet  domains  (so  that,  for  example, 
internal  users  might  be  logged  but  external  users  would  not  be). 

The  identification  system  is  not  only  essential  in  providing  access  control  to  pages.  It  is  also  essential  in  keeping 
track  of  users  as  they  move  around  the  logged  pages  with  the  tracking  system.  With  the  identification  system,  each 
time  a user  enters  a page,  both  the  user  name  and  session  number  can  be  immediately  identified.  These  pieces  of 
data  are  used  in  constructing  the  log  files  for  each  user. 

3.2.  The  tracking  system 

The  tracking/logging  system  is  used  to  record  events  for  each  user  as  they  move  around  the  logged  webpages.  To 
improve  the  accuracy  of  the  tracking,  tracking  events  are  gathered  in  a variety  of  ways.  The  tracking  system  includes 
a server,  a Java  applet.  Javascript  scripts,  and  several  CGI  scripts  written  in  Perl.  These  various  components  report 
events  to  the  server  for  each  session.  Selected  events  are  describe  in  Table  1. 


Event# 

Sent  by 

Event  type 

Data  sent 

00 

observer:  applet 

left  page 

page  left 

01 

request  analyzer 

loaded  page 

page  loaded;  previous  page 

02 

request  analyzer 

from  page 

last  page 

03 

observer:  Javascript 

Javascript  enabled 

current  page 

04 

observer:  applet 

Java  enabled 

current  page 

05 

observer:  applet 

stay  on  page 

current  page 

Table  1:  Events  recorded  by  the  tracking  system. 

As  the  user  browses  through  the  logged  pages  these  events  are  generated  and  sent  to  a log  server  which  writes  them 
to  a log  file.  Because  it  is  traditionally  difficult  to  determine  when  someone  leaves  a page  (e.g.,  if  the  user  switches 
to  a page  on  another  server,  the  current  server  gets  no  notice),  the  stay  on  page  event  helps  us  determine  the  amount 
of  time  on  the  page.  One  might  also  or  instead  record  the  time  when  the  user  leaves  the  page  (Davis  and  Jain  1998). 

Since  we  rely  on  Java  and  Javascript,  problems  may  occur  when  the  user  disables  them  in  the  browser.  This  is  the 
reason  we  record  events  3 and  4.  In  the  synthesis  system,  these  events  are  looked  for.  If  they  are  not  found  we  know 
that  Java  or  Javascript  was  not  functioning  properly  and  that  we  must  use  a different  (and  less  accurate)  mechanism 
for  evaluating  some  aspects  of  traversal.  Thus,  analysis  component  accounts  for  this  and  tries  to  determine  some 
synthesized  events  based  on  a wide  variety  of  other  events. 
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4.  Anonymous  Browsing 


The  question  of  anonymity  with  regard  to  the  World-Wide  Web  is  a common  question.  Is  it  acceptable  for  sites  to 
track  users  paths  through  the  web?  We  considered  this  issue  in  great  detail  while  designing  the  logging  system.  For 
example,  would  students  be  less  likely  to  use  web  pages  if  they  knew  their  use  was  being  logged,  or  would  they 
“simulate  browsing”  to  make  it  appear  that  they  were  using  the  system  more  than  they  were?  In  addition,  what 
should  be  done  about  non-student  readers  of  the  course  web?  In  some  cases,  such  readers  should  be  discouraged 
from  using  the  web,  but  in  many  cases,  use  by  non-students  is  certainly  acceptable.  How  can  one  make  it 
appropriately  accessible  to  such  users  without  hindering  their  own  attempts  to  benefit  from  the  materials. 

It  is  clear  that  careful  studies  and  interviews  need  to  be  done  to  determine  all  of  the  potential  effects.  As  a temporary 
stopgap,  we  decided  to  provide  two  mechanisms  that  help  alleviate  some  of  these  concerns.  One  is  a permissions 
system  incorporated  in  the  identification  system  that  allows  the  instructor  to  specify  which  domains  require 
accounts,  which  domains  do  not  require  accounts,  and  which  domains  should  be  denied  access.  A second 
mechanism  was  to  permit  the  instructor  to  specify  that  anonymous  accounts  are  acceptable.  Let  us  consider  some 
details  of  anonymous  accounts. 

There  are  a number  of  options  for  anonymous  browsing;  one  might  use  a single  “anonymous”  account,  one  might 
use  multiple  anonymous  accounts  (one  for  each  user),  or  one  might  allow  users  to  turn  off  logging.  In  the  case  of  a 
single  account,  the  administrator  might  distribute  the  password  to  all  users  with  privileges  to  browse  the  logged 
pages.  This  method,  however,  has  a distinct  disadvantage.  One  purpose  of  the  logging  system  is  to  determine  paths 
for  each  student  across  sessions  and  not  just  in  a single  session.  With  a group  guest  account  it  is  unknown  what 
student  is  reading  the  web  at  a particular  time.  In  addition,  some  students  might  use  the  guest  account  some  of  the 
time  and  their  own  account  at  other  times,  leading  to  unreliable  data  collection.  Turning  off  logging  is  similarly 
unacceptable. 

A better  system  for  allowing  anonymous  browsing  is  to  create  accounts  for  all  desired  users  and  let  the  account 
names,  real  names,  and  email  addresses  be  blank  or  pseudonyms.  Such  accounts  may  be  distributed  anonymously 
(e.g.,  with  slips  of  paper  in  a hat).  In  this  manner  we  can  create  distinct  paths  for  each  particular  pseudonym  and 
thus,  each  user.  Regular  analysis  can  be  performed  without  any  loss  of  accuracy.  This  is  the  preferred  way  to  handle 
anonymous  logging. 

For  many  purposes,  such  as  marketing  and  other  corporate  uses,  the  name  of  the  user  is  the  desired  information.  For 
our  logging  system,  however,  it  is  frequently  not  the  name,  but  the  the  path  taken  by  any  particular  user.  Hence  the 
actual  name  of  the  user  is  only  useful  for  record  keeping  and  is  not  needed  to  actually  perform  many  analyses. 
Names  are  useful  only  when  usage  and  success  in  learning  are  correlated  in  one  of  many  possible  analyses. 

5.  Related  Work 

There  are  a wide  variety  of  systems  for  Web  analysis.  For  example,  the  Internet  Product  Watch  listed  116  “Analysis 
products”  on  March  25,  1999  (Internet  Product  Watch  1999).  Of  these,  perhaps  half  are  intended  to  gather  and  report 
site  usage  in  some  form  or  another.  Many  provide  a wide  range  of  reports.  For  example,  Accrue  Insight  (Accrue 
Software,  Inc.  1998)  provides  approximately  fifty  different  types  of  reports.  Is  another  reporting  system  needed? 

Because  many  existing  systems  are  targeted  toward  commercial  webs,  the  answer  is  yes,  course  webs  require 
different  systems.  One  reason  is  simple:  most  instructors  (and  even  institutions)  cannot  afford  the  commercial 
products,  which  are  priced  toward  industry  and  not  towards  academia. 

More  importantly,  existing  products  tend  to  focus  on  group  data,  rather  than  on  individuals.  For  example,  consider 
the  eight  “visitor”  reports  provided  by  Accrue  Insight  (Accrue  Software,  Inc.  1998):  total  visits,  total  visitors,  new 
visitors,  registered  visitors,  unregistered  visitors,  visitor  types,  visitor  types,  and  average  pages  per  visit.  None  of 
these  reports  give  details  about  individual  visitors;  rather,  they  are  mass  statistics  of  number  of  “things”  per  “time 
unit”.  Similarly,  consider  the  sixteen  reports  generated  by  FlashStats  (Maximized  Software,  Inc.  1999):  summary. 
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top  URLs  requested,  top  referrers  to  your  site,  search  phrases,  most  common  browsers,  bad  URLs,  bad  referrers, 
user  domain  analysis,  types  of  domains,  daily  totals,  hits  per  day  of  week,  and  hits  per  hour.  Again,  these  are 
primarily  counts  that  reflect  compound  rather  than  individualized  data.  (FlashStats  is  limited  to  the  information  in 
standard  log  files;  Accrue  Insight  uses  a number  of  other  technologies,  including  cookies,  to  gather  additional  data. 
Neither  provides  information  on  single  users.) 

By  providing  information  on  paths  (including  entry  to  web  and  exit  from  web),  we  are  able  to  provide  course  web 
architects  with  the  smaller-scale  and  more  individualized  information  that  they  need.  In  fact,  while  some  commercial 
software  can  track  paths,  that  paths  are  used  only  as  a means  of  collecting  more  general  data,  such  as  the  first  and 
last  pages  readers  visit  at  the  site  (Page  1997), 

There  are  also  client-side  technologies  that  can  more  closely  observe  users  interactions  with  their  browsers.  We  have 
chosen  not  to  explore  such  technologies  because  (1)  they  require  more  effort  on  the  part  of  the  student;  (2)  they 
typically  assume  a one  user/computer  model  that  is  not  applicable  in  the  academic  setting;  and  (3)  they  are  more 
difficult  to  implement  and  maintain  in  the  typically  heterogeneous  academic  environment.  By  relying  on  server-side 
tracking  along  with  lightweight  client  technologies  (e,g.,  Java),  we  are  able  to  support  a variety  of  browsers  and 
computer  platforms. 

6.  Future  Work 

We  have  developed  working  identification  and  tracking  systems  that  have  been  been  used  for  simple  experiments 
during  Summer  1998  and  Fall  1998.  Development  of  a sophisticated  analysis  tool  for  using  the  logs  and  a more 
robust  tracking  system  is  underway.  When  completed  the  tracking  and  analysis  tools  will  provide  the  means  to  track 
users  and  answer  questions  about  student  usage,  such  as  “How  often  and  how  much  do  students  use  particular 
pages?”  and  “How  do  students  typically  reach  the  most  useful  pages?”.  With  the  answers  to  these  questions  one  can 
design  more  usable  and  more  useful  course  webs. 

One  issue  that  is  of  particular  concern  for  tracking  paths  through  WWW  sites  is  that  a student  may  have  multiple 
windows  open  on  the  same  site  at  the  same  time.  This  is  in  stark  contrast  to  the  “one  view”  hypertexts  that  are 
typically  analyzed,  such  as  in  (Jones  et  al.  1996).  We  plan  to  consider  a number  of  possible  static  and  dynamic 
mechanisms  for  visualizing  the  multiple  simultaneous  paths  students  take  through  course  webs. 

The  identification  and  tracking  systems  are  also  being  used  to  support  additional  facilities  for  course  webs.  The 
identification  system  has  been  used  to  support  a group  annotation  system  (Luebke  and  Mason  1998)  (Luebke, 
Mason,  and  Rebelsky  1 999)  and  will  be  used  to  support  a system  for  building  trails  from  both  local  and  non-local 
pages.  The  tracking  system  may  also  be  used  to  support  a form  of  adaptive  hypermedia  in  which  pages  can  be 
modified  based  on  prior  use. 
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Abstract:  Research  in  the  information  society  is  undergoing  radical  change.  The  new  challenges 
make  it  necessary  to  rethink  research  and  the  ways  of  educating  researchers  so  that  they  can  cope 
with  them.  We  show  that  research  has  to  be  considered  as  skilled  practice,  consisting  of 
numerous  component  skills  which  have  to  be  trained  so  that  future  knowledge  workers  can 
successfully  deal  with  unstable,  uncertain  and  complex  situations.  As  a practical  application  of 
this  argument,  we  outline  a project  for  an  integrated  Internet  service,  called  BASES  - Basic 
Support  for  Educational  Study  and  Research. 


1.  Training  researchers 

It  is  not  possible  and,  in  fact,  not  sensible  anymore  to  provide  learners  with  pre-given  answers  to  pre-defined 
generalized  situations.  What  they  rather  need  are  the  skills  for  investigating  themselves  the  complex,  new  and 
uncertain  situation  they  have  to  deal  with.  This  kind  of  work  is  what  is  usually  called  "research",  or,  using  a 
modem  term,  "knowledge  work".  This  demand  a radical  re-thinking  and  re-design  of  teaching  and  learning,  cf. 
e.g.  (Schon,  1987;  Schon,  1983). 

Knowledge  workers  typically  have  to  "consume"  and  "digest"  information  from  widely  varying,  dispersed  and 
uncategorized  sources.  They  are  being  changed  by  information  and  seek  to  change  others  by  it. 

Changing  context  is  characteristic  of  the  situations  knowledge  workers  have  to  deal  with.  Buckingham  Shum 
(Buckingham  Shum,  1997p.902)  calls  these  situations  "wicked  problems",  as  they  cannot  be  solved  by  known 
methods  and  pre-defined  procedures.  The  process  of  identification  and  definition  (or,  as  we  would  say: 
construction)  of  the  problem  is  itself  the  main  task  at  hand,  requiring  complex  judgment  and  negotiation  among 
the  stakeholders.  In  this  process  of  framing  the  problem,  the  goals,  constraints  and  possible  solutions  change 
permanently,  as  does  the  need  for  and  use  made  of  information. 

Research  as  a professional  activity  is,  in  our  understanding,  not  restricted  to  the  research  professionals  at 
universities  and  in  R&E  departments  in  industry,  but  rather  a basic  and  essential  skill  for  knowledge  workers  in 
the  information  society.  But  in  our  case  we  decided  to  specialize  on  educational  research. 

2,  Research  as  skilled  practice 

Research  work  is  neither  an  art  that  cannot  be  taught  or  learned,  nor  a set  of  rules  and  regulations  to  follow.  It  is  a 
skill,  or  better  a set  of  skills,  that  can  be  acquired  (at  least  to  a certain  degree)  and  whose  acquisition  can  be 
supported  by  training. 


The  mastery  of  a skill  is  not  a question  of  "all-or-none":  it  is  not  the  case  that  one  day  we  are  unskilled  in  a field 
and  the  next  day  we  are  experts.  There  certainly  are  steps  and  phases  in  the  acquisition  of  skills,  i.e.  in  the 
emergence  of  the  capacity.  Following  the  model  proposed  by  Dreyfus  and  Dreyfus  (Dreyfus,  Dreyfus,  1986),  who 
distinguished  five  steps  in  the  process  of  passing  from  novice  to  expert,  Baumgartner/Payr  (Baumgartner,  Payr, 
1993)  have  outlined  a three-dimensional  heuristic  model  of  teaching  and  learning  with  three  different  styles  of 
teaching.  These  different  teaching  modes  have  important  impacts  of  the  adequate  types  of  learning  interactions: 


FIGURE  NOT  AVAILABLE 

Figure  1 : Three-dimensional  heuristic  model  of  teaching  and  learning 

The  social  perspectives  and  implications  of  these  styles  of  teaching  are  substantially  different.  The  knowledge 
transfer  model  is  based  on  the  belief  that  there  exist  people  who  know  the  right  answer  to  a pre-defined  question 
(e.g.  the  teacher).  It  is  heir  to  a positivist  tradition  and  corresponds  to  a hierarchical  career  model  with  competition 
at  its  center. 


FIGURE  NOT  AVAILABLE 

Figure  2: 

The  "situated  learning"  model,  on  the  contrary,  subscribes  to  the  constructivist  viewpoint  where  the  coach 
collaborates  with  the  learner  to  cope  with  a complex  real  situation  for  which  neither  of  them  has  a ready  solution. 
At  the  beginning  of  their  concerted  action,  learners  legitimately  participate  only  peripherally  (Lave,  Wenger, 
1991).  During  the  learning  process,  their  responsibility  grows  continuously  until  the  learners  can  themselves  act  as 
coach  for  other,  new  learners. 

The  difference  between  commonsense  knowledge  and  scientific  knowledge  is  not  the  quality  of  knowledge  itself 
Both  types  of  knowledge  are  structured  experiences  that  one  has  to  develop,  acquire,  examine  and  apply  for  a 
viable  (successful)  action.  The  quality  of  knowledge  does  not  depend  on  the  difference  in  range  and  object  domain 
(Meehan,  1991).  Scientific  research  develops  (or  should  develop)  knowledge  in  a systematic  way  and  uses  a meta- 
language (second  order  language)  to  describe  the  ongoing  procedures  and  activities  (Laurillard,  1993). 

Whatever  skill  we  want  to  train,  we  need  a second  order  language  which  functions  as  a training  language.  The 
lack  of  a meta-language  in  scientific  thinking  is  the  biggest  problem  we  have  to  confront  in  training  research 
skills.  The  existing  meta-language  describes  mainly  the  products  of  research  but  not  the  activity,  the  process  itself. 
In  training  the  knowledge  worker,  we  find  ourselves  in  a position  where  we  can  only  judge  if  the  product  fulfills 
the  scientific  standards  but  not  the  skills  that  have  to  be  improved.  We  are  like  football  trainers  who  can  only 
comment  the  outcome  of  a game,  qualifying  a won  game  as  "good"  and  a lost  one  as  "bad",  without  being  able  to 
specify  what  was  good  or  bad,  what  should  be  improved  and  how  this  could  be  done.  A prerequisite  for  training 
research  is  therefore  to  develop  this  meta-language,  starting  with  identifying  relevant  actions  and  processes  that 
make  up  the  complex  practice  of  doing  research. 

3.  Training  research  skills  with  Internet  support  - BASES  project 

The  logical  consequence  of  this  change  for  learning  in  context  ("situated  learning"  (Chaiklin.,  Lave,  1993;  Lave, 
1991)  is  that  the  support  for  training  these  skills  itself  should  make  use  of  the  new  media  that  are  able  to  integrate 
information,  interaction  and  tools. 

We  follow  this  approach  with  the  BASES  (BAsic  Support  for  Educational  Study  and  Research)  project,  an 
Internet  service  for  supporting  and  training  research  skills.  It  is  the  hands-on  complement  to  a book  on  the  subject 
that  we  are  currently  finishing.  With  BASES  we  support  the  training  of  research  skills  in  three  different  ways: 
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a)  Cognitive  Apprenticeship  (Collins,  Brown,  Newman,  1990):  We  provide  different  kinds  and  qualities  of  guided 
practice: 

• general  information  and  resources  permanently  updated  and  verified.  These  include  links  to  libraries,  grants; 
reviewed  software  tools  (freeware  or  demo  versions)  for  research  work  (e.g.  mind-mapping  software, 
bibliography  packages);  sample  macros,  stylesheets,  forms  etc. 

• training  modules  for  research-relevant  Internet  techniques  (e.g.  searching  for  and  in  online  databases,  assessing 
the  quality  of  resources) 

• interactive  tele-learning  modules  with  "human  backing"  for  general  research  skills  like  research  design, 
writing,  argumentation,  referencing. 

h)  Anchored  instruction  (Vanderbilt,  1990):  Starting  with  common  authentical  problem  situations,  BASES 
offers  a wide  range  of  activities  to  tackle  them: 

• FAQ  collections 

• moderated  and  guided  discussions 

• online  interconnections  among  users 

• modular  tele-learning  courses  for  different  authentical  problems  (how  to  limit  the  hits  of  search  engines,  how 
to  quote  Internet  resources, ...) 

c)  Cognitive  flexibility  theory  (Spiro,  Jehng,  1990):  Learning  to  defend  different  points  of  view  and  to  take 
multiple  perspectives: 

• giving  different  examples  of  the  same  task  (e.g.  paraphrasing  a certain  quotation,  constructing  the  same 
argument  on  different  logical  bases  ...) 

• defending  different  points  of  view  (e.g.  presenting  the  arguments  of  an  author  and  of  his/her  critics, 
summarizing  advantages  and  disadvantages  of  a certain  procedure,  product ...) 

• choosing  different  methods  on  the  same  subject  (taking  a commonsense  approach  and  a scientific  approach, 
proposing  a pragmatic  and  a theoretical  solution,  seeing  a problem  in  short  and  long  term  development ...) 

We  call  this  integrated  site  for  research  support  a service,  not  a server,  because  we  acknowledge  that  it  can  only 
function  optimally  with  "human  backing".  Beside  the  need  for  constant  updating  and  for  dealing  with  requests, 
problems,  contributions  and  suggestions,  there  are  different  situations  where  support  is  necessary  and  new 
requirements  arise.  These  could  range  from  university  teachers  using  the  resources  as  complementary  material  for 
their  courses  to  students  who  need  a last-minute  brush-up  for  presenting  their  thesis. 

Making  resources  and  training  offers  for  research  skills  permanently  available  also  addresses  the  need  for 
learning  on  demand.  Skills  cannot  be  acquired  on  store  and  in  one  step,  but  require  permanent  training  and 
refinement.  With  the  help  of  BASES,  learners  can  train  their  research  skills  whenever  they  need  them  and  in 
unison  with  their  growing  competence  and  the  increasing  complexity  of  their  research  tasks. 

Contrary  to  the  book  that  we  have  been  working  on  with  little  financial  support,  the  design  and  implementation  of 
the  BASES  service  requires  more  personpower  than  we  have  available  at  the  moment.  The  project  is  currently 
(End  of  October  1998)  submitted  to  national  funding  program  by  the  Austrian  ministry  of  science.  We  are  now 
shortlisted  and  have  a final  hearing  in  the  middle  of  November  in  Vienna.  We  hope  to  start  with  the  end  of  the 
year.  A prototype  of  some  of  our  ideas  can  be  found  under 

http://www.uni-klu.ac.at/~pbaumgar/deutsch/medien/htm  1/2 1 .htm 


4.  Summary 

Interconnected  problems,  rapidly  changing  unstable  environments,  and  an  indeterminate  future  of  our  society 
challenge  our  traditional  education  system.  What  is  needed  is  not  only  factual  and  procedural  knowledge  for 
solving  pre-defined  problems  but  the  active  skill  of  framing  problems,  designing  desirable  changes  and  inventing 
ways  to  bring  these  changes  about. 
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Research  is  not  a gift  based  on  "talent",  "intuition"  or  "art"  but  a skilled  social  practice  that  can  be  learned  and 
trained.  What  we  need  is  a second  order  language  for  this  training  process  that  satisfies  the  following  criteria; 

• definition  and  isolation  of  different  kinds  of  research  skills  and  training  of  these  skills 

• development  of  a program  to  progressively  chunk  these  different  kinds  of  isolated  skills  to 

• acquire  a gestalt  view  of  the  research  process 

• integration  of  the  challenges  raised  by  new  technologies  in  a self-study  socio-technological  environment. 

We  propose  the  BASES  project  as  a small  step  in  this  direction. 
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Abstract:  Three  distributed  learning  projects  involving  various  applications  of 

telecommunications  technologies  are  briefly  discussed  in  this  paper.  The  projects  were 
conducted  with  Montana  State  University-Bozeman  as  the  administrative  and  fiscal  agent. 
Two  of  the  projects  were  designed  for  higher  education  and  both  involved  collaborative 
efforts.  The  third  project  was  designed  for  k-12  teachers  and  involved  peer  coaching  over 
1300  Montana  educators  to  use  the  Web  and  telecommunications  in  their  classes. 

Introduction 


An  article  by  Gibson  and  Gibson  in  Adult  Learning,  identified  several  lessons  learned  from  over  100 
years  of  experience  with  distance  learning.  Lesson  one  states  that  distance  learning  is  different  from  that  in 
the  traditional  classroom  setting  and  traditional  models  should  not  be  the  base  against  which  success  or  failure 
is  measured.  Lesson  two  stressed  training  for  faculty,  administrators,  and  support  staff.  Lack  of  training  is 
identified  as  one  of  the  biggest  failures  in  distance  learning.  The  third  lesson  is  that  leamers^^  needs  must  be 
continually  supported  during  the  learning  experience. 

Technology  used  effectively  has  the  potential  to  link  faculty  and  learners  in  order  to  overcome 
distance  and  to  enhance  sharing  of  institutional  resources  among  units  of  the  university  system.  Technologies 
can  be  used  to  reinvent  the  means  by  which  instruction  is  delivered  and  the  kinds  of  activities  in  which 
learners  engage.  Technology  can  assist  in  meeting  increasing  demands  from  potential  learners  who  cannot 
afford  to  leave  a job  and/or  family  to  pursue  an  advanced  degree  on  a college  campus. 

These  lessons  serve  as  a framework  to  identify  the  multi-faceted  issues  in  these  projects.  First,  there 
is  a need  is  to  restructure  and  move  faculty  away  from  thinking  that  the  traditional  classroom  model  will  work 
with  distance  learning.  Second,  it  is  necessary  is  to  provide  extensive  faculty  development  and  support  for 
faculty  so  courses  can  be  developed  or  adapted  for  appropriate  distance  learning  formats  and  new  teaching 
skills  acquired  and  tested.  Third,  is  the  need  to  develop  an  adequate  support  system  for  learners  who  are 
engaging  in  distance  learning  activities  and  to  enhance  leamer/faculty  interaction.  Finally,  the  impact  that 
distance  learning  has  on  issues  such  as  faculty  work  loads,  salary,  advising,  promotion/tenure  decisions, 
intellectual  property  concerns,  and  learner  support  must  be  determined. 

Materials  listed  in  the  References  and  Web  Resources  section  can  provide  readers  with  many 
additional  considerations  for  designing  distance  learning  courses  and  learning  opportunities.  There  is  much  to 
consider  in  addition  to  a course  itself.  Institutions,  faculty,  and  course  designers  must  deal  with  intellectual 
property  issues  (copyright,  patent,  and  trademarks)  as  well  as  technical  support,  library  resources,  faculty 
development,  and  student  support.  There  are  also  many  inter  and  intra  institutional  challenges  which  can 
make  or  break  a course.  In  addition,  there  may  be  concerns  from  faculty  and  institutions  about  faculty  rights, 
course  ownership,  and  the  effectiveness  of  distance  learning  courses.  Inter  and  intra  institutional  issues  which 
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must  be  considered  include  application  paperwork,  registration  fees  and  procedures,  ease  of  registration, 
financial  aid,  transferability  of  courses,  learner  support  and  access  to  materials  and  library  resources 

This  paper  will  briefly  discuss  three  telecommunications  distance  learning  training  projects  in  the 
College  of  Education,  Health,  and  Human  Development  at  Montana  State  University-Bozeman  and  share  some 
of  the  lessons  learned  as  well  as  some  of  the  challenges. 

Telecommunications  Faculty  Development  Project 

During  1993,  the  first  interactive  television  (compressed  video)  phase  of  the  Montana  Educational 
Telecommunications  Network  (METNET)  was  implemented  and  it  consisted  of  compressed  video  codec 
(interactive  television)  units  installed  in  four  locations.  This  technology  made  available  many  distance 
education  opportunities  for  delivery  of  course  work,  staff  development,  and  videoconferencing  which  has 
previously  been  difficult.  In  order  to  demonstrate  need  and  applications  for  the  video  network,  courses  and 
training  had  to  be  developed  and  delivered  in  order  to  convince  the  institutions  of  higher  education,  state 
offices,  and  the  legislature  that  funding  for  additional  sites  should  be  continued.  Officials  of  the  Office  of  the 
Commissioner  of  Higher  Education  and  the  Telecommunications  Division  of  the  State  Department  of 
Administration  were  anxious  to  see  demonstrations  of  educational  applications  between  and  among 
institutions.  They  also  wanted  to  see  demonstrations  of  the  integrated  use  of  the  technologies  with 
audioconferencing  and  Internet.  To  date,  twelve  sights  are  now  operational  and  another  video  network, 
Visionet,  has  been  installed  due  to  a grant  from  the  TIIAP  program.  It  isnow  possible  to  link  to  a medical 
telemedicine  network  and  another  video  network  installed  in  eastern  Montana.  Thus,  the  state  is  quite  well 
covered  with  interactive  television  although  communications  costs  are  still  relatively  high.  The  METNET  is 
very  heavily  booked  with  courses,  training,  and  meetings.  This  project  provided  some  early  input  and 
demonstration  of  effectiveness  which  has  been  shared  with  subsequent  efforts. 

Description  of  Project 

The  director  was  given  the  charge  to  design  and  deliver  courses  and  staff  development  for  faculty  and 
teachers  using  the  METNET  technologies  of  compressed  video  and  the  bulletin  board  system  during  the  1993- 
1995  academic  years  at  the  Montana  State  University.  Interested  faculty  were  solicited  and  mini  grants 
available  for  the  design  and  delivery  of  courses  or  staff  development  using  the  system.  Formative  and 
summative  evaluation  was  conducted  with  the  idea  of  refining  and  learning  as  much  as  possible  about 
instructional  design  for  distance  education  courses  and  faculty  development.  This  information  was  then 
applied  by  the  director  to  future  courses  and  shared  with  interested  College  faculty  in  the  System.  The  Office 
of  the  Commissioner  of  Higher  Education  had  established  a goal  of  using  telecommunications  and  distance 
education  to  facilitate  intra  institutional  courses.  The  Montana  State  University  (now  the  Montana  State 
University-Bozeman)  as  a land  grant  university  has  a strong  commitment  to  utilizing  the  METNET  distance 
learning  technologies  for  the  State  through  course  delivery  to  college  students,  K-12  students  and  lifelong 
learners  as  well  as  other  applications. 


Short  Term  Objectives 

1 . A faculty  distance  education  resource  person  and  possibly  a cadre  of  faculty  would  be  trained  to 
use  METNET  and  distance  learning  technologies  as  relevant  to  Montana.  A "train  the  trainers"  concept  was 
used  to  facilitate  program  development  and  applications. 

2.  The  METNET  technology  would  be  used  to  teach  use  of  the  system  over  the  system. 

3.  Faculty  from  different  higher  education  institutions  would  demonstrate  a team  teaching  approach 
to  a course  using  the  network  and  possibly  combining  technologies. 

Long  Term  Objectives 
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1 . The  METNET  would  become  ingrained  into  education  at  all  levels  in  Montana.  The  METNET 
interactive  television  system  and  bulletin  board  would  be  used  as  an  instructional  tool  and  conferencing  tool 
by  all  three  groups  of  users. 

2.  A well  trained  group  of  users  would  have  developed  a substantial  amount  of  program  applications 
materials  for  use  on  METNET. 

3.  The  METNET  would  provide  Montanans  with  opportunities  for  economic  development  and  career 
options  by  workshops  through  Business,  Extended  Studies,  degree  granting  and  certification  courses,  licensing 
programs  and  various  other  activities. 


Summary 

The  project  proved  to  be  very  successful.  The  director  designed  and  delivered  the  first  two  courses 
over  the  METNET  to  five  sites  including  an  onsite  class  at  the  Bozeman  campus.  At  that  time,  the  broadcast 
studio  was  primitive  and  presented  many  challenges  for  all  concerned.  The  instructor  had  to  run  and  manage 
the  equipment  while  also  teaching  a live  class  to  five  sites.  During  the  semester,  some  assistance  was  provided 
with  camera  support  and  the  experience  of  managing  the  environment  provided  useful  information  about 
teaching  a course  over  live  television  to  the  College  as  well  as  the  Division  of  Extended  Studies  for  Montana 
State  University.  It  also  provided  information  concerning  teaching  techniques  and  what  types  of  additional 
equipment  such  as  microphones,  graphics  camera,  monitors,  and  furniture  was  needed  as  well  as  information 
about  room  arrangement  and  camera  placement.  During  the  second  phase  of  the  project,  a course  taught  by 
instructors  at  two  Montana  institutions  of  higher  education  was  designed  and  delivered.  It  dealt  with 
telecommunications  and  was  an  experimental  graduate  course  in  teaching  telecommunications  using 
telecommunications.  It  used  the  interactive  television  network  to  deliver  a weekly  two-hour  evening  class  and 
the  campus  computer  networks  were  used  for  an  online  component  to  apply  class  content  and  complete 
assignments. 

Several  pieces  of  information  were  gathered  from  the  project.  Permission  slips  granting  use  of 
images  were  gathered  from  each  student  at  sites  where  the  course  was  delivered  and  students  were  informed 
that  each  class  would  be  videotaped  for  research  puq?oses.  Each  session  of  the  course  had  a brief  formative 
questionnaire  administered  which  asked  students  a few  questions  about  the  night^Es  class.  During  the  course 
of  the  semester,  instructors  experimented  with  varying  the  rates  of  transmission  from  quarter  Tl,  and  lower,  to 
full  Tl  which  at  the  time  provided  onear  broadcasto  quality  video.  An  interesting  finding  was  that  the 
students  tolerated  poorer  quality  video  with  no  difficulty.  However,  they  were  insistent  that  the  audio  quality 
be  excellent.  Students  were  also  sensitive  to  feeling  involved  in  the  class  so  that  the  instructors  had  to  make 
sure  that  each  class  and  student  had  a fair  share  of  air  time  and  that  we  learned  their  names.  It  was  also 
important  that  remote  students  be  sent  materials  ahead  of  time  as  they  did  not  like  to  see  the  site  with  the 
instructor  receive  materials  prior  to  them.  Another  interesting  finding  was  that  after  the  first  session  or  two, 
students  learned  to  ignore  the  technology.  At  one  point,  one  instructor  created  a virtual  computer  lab  using  an 
audio  conference  for  a remote  site  having  trouble  with  a particular  Internet  navigation  tool.  It  was  surprising 
to  find  that  the  instructoriEs  presence  by  audio  with  both  she  and  the  class  looking  at  the  same  computer 
screen  so  that  she  could  guide  them  through  use  of  the  tool  was  more  important  for  student  learning  than  using 
video  without  a hands  on  experience. 

Other  lessons  learned  from  the  project  emphasized  the  intense  early  planning  which  must  occur  to 
prepare  for  a interactive  television  class  delivered  to  several  sites.  Students  must  be  able  to  receive  course 
materials  in  a timely  fashion  and  they  must  have  a way  to  contact  faculty  by  phone  if  something  is  not 
operative  at  the  remote  site.  Having  a way  to  contact  remote  students  personally  is  important  using  traditional 
mailings  as  well  as  electronic  mail  is  needed.  Technical  assistance  for  faculty  and  students  was  found  to  be  an 
imperative.  We  also  found  that  having  a person  at  each  site  to  handle  the  equipment  and  trouble  shoot  was 
also  a necessity  when  funding  allows. 


Montana  University  System  Distributed  Learning  Pilot  Project 
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This  project  began  during  May,  1998  and  was  funded  by  a competitive,  incentive  grant  from  the 
Office  of  the  Commissioner  of  Higher  Education,  Montana  University  System.  The  project  involved  designing 
and  delivering  five  graduate  education  classes  to  be  delivered  through  computer  conferencing  during  the  1 998- 
1999  academic  year.  Institutions  involved  were:  College  of  Education,  Health,  and  Human  Development, 

Montana  State  University— Bozeman  (administrative  and  fiscal  agent);  College  of  Education  and  Human 
Services,  Montana  State  University— Billings;  Department  of  Education  and  Graduate  Programs;  Montana 
State  University— Northern;  School  of  Education,  University  of  Montana-Missoula. 

To  date,  three  courses  have  been  delivered  with  two  to  be  completed  in  May,  1999,  Evaluation  will 
be  completed  by  June,  1999. 

Objectives 


1 . A core  of  courses  in  the  graduate  Master^^s  Degree  in  Education  was  selected  to  be  used  as  the 
focus  for  distributed  learning  delivery  efforts  in  the  College  of  Education,  Health,  and  Human  Development  at 
Montana  State  University-Bozeman,  the  College  of  Education  and  Human  Services  at  Montana  State 
University-Billings,  the  Department  of  Education  and  Graduate  Studies  at  Montana  State  University-Northern, 
and  the  School  of  Education  at  the  University  of  Montana.  Software  used  was  the  First  Class  Intranet  Client 
housed  on  servers  at  the  METNET  and  MSULINK,  Bums  Telecommunication  Center  at  the  Montana  State 
University  -Bozeman. 

2.  Through  faculty  development  and  support,  it  was  hoped  that  faculty  in  these  programs  would 
collaboratively  explore  program  design  as  well  as  adapt,  design,  and  deliver  courses  suitable  for  delivery  to 
distant  learners. 

3.  Distant  learners  were  to  receive  support  regarding  access  to  faculty,  library  resources,  and 
information  based  on  delivery  of  courses  and  services.  Information  gained  and  shared  will  further  enhance 
and  improve  future  offerings. 

4.  Coordination  of  course  offerings  would  result  in  better  distribution  of  programs  in  Montana  and 
utilization  of  scarce  resources. 

5.  Distant  learners  and  faculty  would  benefit  from  a systematic,  focused  approach  to  course  design 
and  delivery. 

Twelve  faculty  from  the  four  institutions  were  selected  to  work  on  the  project.  A two  day  faculty 
development  workshop  was  held  at  the  Bozeman  site  in  May,  1998,  to  begin  work  on  the  courses.  An 
experienced,  outside  distance  learning  consultant  was  hired  to  work  with  the  faculty  in  the  workshop.  Faculty 
received  summer  stipends  to  design  and  deliver  the  five  courses  during  the  academic  year. 

Challenges  and  Summary 

This  project  has  provided  a lot  of  evidence  which  exposes  inter  and  intra  institutional  challenges  and 
obstacles  to  distributed  learning  efforts  which  need  resolution  before  projects  can  proceed  smoothly.  The  easy 
part  was  working  with  the  faculty  and  students.  Challenges  included  advertising,  policies  and  procedures 
involving  course  registration,  fees,  transferability  of  courses  and  monies,  and  compliance  with  letters  of 
agreement.  Students  using  certain  Internet  service  providers  had  difficulties  using  the  First  Class  Intranet 
Client  initially.  The  project  also  demonstrated  some  of  the  challenges  in  working  collaboratively  across 
institutional  lines.  Preliminary  data  from  students  shows  a high  degree  of  satisfaction  with  the  courses  and 
they  felt  that  there  was  no  lack  of  quality  or  rigor  in  the  classes.  Most  students  would  have  preferred  at  least 
one  or  two  face  to  face  meetings  which  could  have  been  accomplished  using  interactive  television  or  an  on  site 
session.  Final  evaluation  of  the  project  will  become  available  in  June  and  will  be  shared  during  the  live 
presentation  of  this  paper. 


US  WEST  Montana  Teacher  Network 
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Challenges  and  Summary 


Funded  by  a grant  from  US  WEST  and  developed  in  cooperation  with  the  Montana  Education 
Association,  an  affiliate  of  the  National  Education  Association,  the  Montana  Teacher  Network  is  currently 
providing  peer  coaching  in  telecommunications  training  and  Web  applications  to  over  1300  Montana 
Educators.  The  paper  in  the  ED-MEDIA  99  proceedings  on  session  2483  provides  information  on  the  project 
so  only  a brief  summary  is  included  in  this  paper.  The  project  involved  distribution  of  notebook  computers  to 
an  initial  132  teachers  representing  one  percent  of  the  Montana  teaching  force.  The  computers  will  remain 
with  the  teachers  as  long  as  they  are  teachers  in  Montana.  Participants  were  selected  through  an  application 
process  and  agreed  to  participate  in  an  on  site  summer  workshop,  an  online  training  course  to  be  held  during 
1997-1998,  and  then  to  execute  a training  plan  to  peer  coach  at  least  ten  colleagues  during  1998-1999  thus 
impacting  over  1300  Montana  educators.  The  online  course  was  conducted  using  First  Class  Intranet  Client 
software  housed  on  a server  at  the  Montana  State  University-Bozeman  Bums  Telecommunication  Center 
server.  The  second  year  of  the  project  used  the  same  software  housed  at  the  Montana  Office  of  Public 
Instruction  server  for  the  METNET. 

The  project  has  been  very  successful  and  reports  and  anecdotes  received  from  the  coaches  have  been 
positive.  They  definitely  see  growth  in  their  schools  in  using  the  Web  and  telecommunications.  However, 
there  have  been  challenges.  One  problem  has  been  that  some  school  districts  were  not  able  to  provide  the 
Internet  access  which  was  promised  to  participants  selected  for  the  project  and  this  made  it  difficult  for  the 
teachers  to  fulfill  their  coaching  requirement  in  a timely  fashion.  Four  teachers  were  unable  to  complete  the 
project  and  their  notebook  computers  were  recalled.  It  has  proved  difficult  to  keep  track  of  the  second  group  of 
teachers  receiving  the  coaching.  Having  the  ability  to  offer  a credit  option  for  the  second  wave  of  teachers 
would  have  been  helpful  but  this  proved  to  be  impossible.  Had  it  not  been  for  the  geographical  challenges 
presented  by  the  size  and  population  of  Montana,  it  would  have  been  helpful  to  convene  the  teachers  in  groups 
two  or  three  times  during  the  project.  Instead,  the  METNET  bulletin  board  was  used  as  the  primary  gathering 
place.  Plans  are  underway  to  continue  the  Montana  Teacher  Network  and  expand  it  to  a community  approach. 
Additional  funding  and  partnerships  to  take  the  project  forward  into  2000  are  being  sought. 


A Few  Ideas  to  Consider  When  Planning  Distributed  Learning  Projects 

1.  Conduct  a needs  assessment  to  determine  the  target  population  and  consider  the  needs  of  the  group 
including  level  of  technology  literacy. 

2.  Determine  who  are  the  stake  holders  in  the  project  and  their  agenda. 

3.  Decide  in  advance  how  to  distribute  instructional  materials  and  obtain  copyright  clearance  and  permissions 
when  needed. 

4.  Select  the  technologies  to  be  used  and  consider  a combined  approach 

5.  Consider  all  financing  aspects  and  any  hidden  costs. 

6.  Determine  whether  courses  will  be  in-load  or  taught  for  extra  pay. 

7.  Consider  all  possible  policies  and  procedures  which  might  impede  the  project. 

8.  Determine  how  faculty  and  remote  learners  be  supported. 

9.  Determine  how  the  course  and  effort  will  be  evaluated  for  feedback  as  well  as  promotion  and  tenure 
considerations. 
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Abstract:  To  establish  a new  studying  course  extensive  researches  have  to  be  done. 
Government  of  Slovenia  - Ministry  for  Education  and  Sport  sponsored  the  research  for  the 
new  studying  course  "Multimedia  for  Teachers".  Fundamental  research  was  conducted  to 
verify  positive  effect  to  the  learning  process.  Research  of  written  materials  and  know-how  of 
similar  projects  was  done  to  define  objective  of  the  project.  Later  the  project  plan  to 
implement  the  studying  course  was  made  and;  verification  of  the  project  plan  was  not  left  out. 
The  project  is  now  at  the  beginning  of  the  implementation.  First  populations  of  students  - 
future  teachers  are  experience  new  approach  to  the  teaching  by  learning  multimedia  with 
computers. 


Introduction: 


The  research  projects  to  find  potentials  of  the  multimedia  in  various  branches  of  enterprises  were  conduct  a long 
time  before  the  technology  was  available  at  acceptable  price.  In  the  90's  technology  becomes  available.  Now  is 
the  time  to  use  the  results  of  research  projects  once  conducted  and  research  of  the  things  that  were  not  apparent 
till  now. 

Government  of  Slovenia  - Ministry  for  Education  and  Sport  funded  the  project  to  improve  computer  technology 
at  primary  and  secondary  schools  named  RO  (Computer  Literacy).  As  the  schools  get  better  equipped  teacher 
was  educated  to  use  the  potential  of  equipment.  One  of  the  five  strategic  research  subprojects  in  project  RO  is 
"Multimedia  in  Education".  In  the  research  work  for  a project  "Multimedia  in  Education"  three  parties  are 
involved:  Republic  of  Slovenia  - Institution  for  Education  and  Sport,  Faculty  of  Computer  Science  in  Ljubljana 
and  Faculty  of  Education  in  Maribor. 

Using  statistical  analysis  of  the  gathered  data  it  becomes  clear  that  multimedia  in  the  teaching  process  increases 
effect  of  learning.  By  combining  the  entertainment  and  education  the  pupils  finds  fun  in  absorbing  the 
educational  material  and  the  time  necessary  to  remember  the  certain  information  was  significantly  shorter  in 
compared  with  the  traditional  educational  materials. 

We  were  not  satisfied  with  results  of  statistical  analysis.  In  the  search  for  artifacts  that  would  clearly  support  the 
statistical  analysis  we  started  fundamental  research  of  effects  of  multimedia  material  to  the  brain  activity.  It  was 
found  that  a larger  portion  of  the  brain  is  active  if  multimedia  educational  material  is  used  (Jausovec,  1998). 
Statistical  analysis  was  confirmed. 

The  new  step  was  taken  to  study  the  market  available  multimedia  supported  educational  materials.  The  majority 
of  these  materials  is  in  English  language  and  not  always  suitable  for  use  in  the  primary  and  secondary  level  of 
education.  There  are  doubts  if  these  materials  can  be  used  in  the  large  scale  even  at  the  university  level.  The 
conclusion  is  simple.  We  need  to  prepare  the  multimedia  supported  educational  material  in  such  a way  to  be 
easily  translated  to  any  language. 

As  a test  the  multimedia  supported  educational  material  for  the  physics  was  prepared.  The  users  very  well 
accepted  these  educational  materials.  Using  this  material  we  find  that  the  most  appropriate  media  for  distribution 
is  CD-ROM.  The  bitrate  of  current  communication  network  does  not  allow  Internet  distribution.  Materials  have 
large  amount  of  video  clips  and  can  be  use  on  LAN  but  is  almost  impossible  to  transmit  them  to  students' 
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computers  over  telephone  lines.  On  the  other  hand  we  are  also  members  of  European  project  COLOS  (see 
COLOS)  where  the  interactive  computer  materials  is  accessible  over  the  Internet. 

When  we  gain  experience  we  find  out  that  teachers  have  to  be  prepared  to  use  this  material  as  a valuable  support 
in  their  work.  This  is  the  goal  of  the  project  "Multimedia  for  teachers". 


Project  Goal: 

Curriculum: 

Subject:  MULTIMEDIA 

Time  span:  60  hr;  30  hr  lectures,  30  hr  practical  experiments 
Year:  2.  or.  3. 


Students  will: 

• become  aware  of  the  importance  of  mono/multimedia  system  in  education; 

• become  aware  where  and  how  they  can  use  mono/multimedia  systems  in  their  work; 

• get  qualified  and  motivated  for  creative  problem  solution  in  primary  and  secondary  schools  using 
mono/multimedia  systems; 

• get  qualified  for  use  of  mono/multimedia  system  during  their  studying  and  research  & development 
activities. 

• get  acquainted  with  methods  and  rules  for  making  of  simple  mono/multimedia  systems  in  their  classes 
and  basic  laws  of  special  didactics  for  that  field. 


1 . educational  technology  basics:  terms,  classification,  history,  mono/multimedia  in  education; 

2.  monomedia  systems:  technical  didactical  characteristics  of  visual  and  audio  monomedia  systems; 

3.  application  of  computer  in  education  and  educational  information  system:  cybernetics,  systems  theory, 
information  theory,  communication  theory,  information  systems; 

4.  multimedia  systems:  terms,  classification,  history,  technical  didactical  characteristics  of  multimedia 
elements/systems  and; 

5.  planning  the  appliance  of  mono/multimedia  system  in  education. 


Goals: 


Contents: 


Project  Plan: 


IMAGE  NOT  AVAILABLE 


Figure  1 : project  plan  - multimedia  for  teachers 


Description  of  the  Plan; 


In  the  "initial  survey"  we  gain  the  agreement  of  the  different  participants  of  the  project.  It  was  decided  to 
separate  multimedia  into  two  courses.  First  course  "Multimedia  1"  would  gives  students  initial  knowledge  to  the 
traditional  multimedia  systems  (radio,  CD  players,  cassette  recorders,  minidisks,  TV  studio  equipment,  VCR’s, 
camcorders,  presentation  techniques).  The  student  course  having  elements  of  multimedia  1 was  held  even  before. 
It  was  called  "Educational  technology".  Now  it  was  reorganized  and  renamed  into  multimedia  1.  The  second 
course  "Multimedia  2"  would  extend  students  knowledge  to  the  computer  technology.  Objective  for  the  project 
is  two  level  of  knowledge.  In  the  lowest  level  students  would  learn  what  the  new  technology  is  capable  to  do.  In 
the  highest  level  students  would  be  capable  to  fully  utilize  the  new  technology.  With  these  levels  we  provide 
individualization  and  differentiation. 

In  the  "requirement  analysis"  we  were  to  make  a rough  list  of  required  equipment,  hardware,  software  and 
educational  material  for  each  of  the  working  place.  Most  of  the  traditional  multimedia  systems  were  already 
presented  at  the  faculty  but  computers  were  not.  The  project  could  easily  be  jeopardized  if  we  could  not  gain 
enough  funds  to  equip  a new  computer  multimedia  room.  It  was  decided  that  no  previous  knowledge  would  be 
required  for  course  "Multimedia  1"  but  for  course  "Multimedia  2"  a knowledge  of  use  of  computers  operating 
system  (MS  Windows);  wordprocessor  (MS  Word);  presentation  program  (MS  PowerPoint)  and;  Internet 
browsers  would  be  required.  Education  material  for  the  "Multimedia  1 " would  be  prepared;  printed  and  put  to 
every  working  place  but  for  the  "Multimedia  2"  education  material  and  descriptions  for  every  working  place 
would  be  on  the  Internet.  We  were  capable  to  do  this  since  every  working  place  would  have  a computer 
connected  to  the  Internet. 

The  phase  "Multimedia  J"  was  divided  into  several  tasks.  The  first  task  "mmJ  - required  components"  was  to 
establish  the  working  places;  put  equipment  to  that  working  places  and;  check  how  things  work.  It  also  includes 
the  detailed  learning  of  tutors  how  to  use  every  single  equipment.  When  task  "mm  I - required  components" 
starts  we  could  start  second  task  "mml  - preparing  edu.  material".  Education  materials  were  consist  of  manuals; 
description  of  the  working  place  and  required  work  to  do  at  the  specific  working  place. 

"mml  - first  group"  and  "mml  - second  group"  are  tasks  where  Multimedia  1 becomes  operational.  429  students 
in  the  second  year  passed  this  course.  Informations  we  get  from  them  were  satisfying.  Most  of  the  students  really 
enjoyed  working  with  audio,  video  and  presentation  equipment. 

At  the  same  time  we  start  with  elements  of  Multimedia  2 - "Multimedia  2 small  scale  test".  The  test  group  was 
the  students  of  computer  and  mathematics  teachers’  course.  We  got  valuable  responses  from  them  what  can  we 
expect  from  other  students.  They  help  us  prepare  better  educational  material  and  establish  the  working  places. 

In  the  task  "mm2  - preparing  hardware"  not  only  computers  needs  to  become  operational.  A new  LAN  with 
UTP  technology  has  to  be  made  and  furnished  the  whole  room.  The  task  "mm2  - preparing  and  edu. 
material"  takes  a lot  of  time  because  preparing  web  pages  with  instructions  and  lectures  takes  a lot  of  effort. 
(Ambron,  S.,&  Hooper,  K.  1990),  (Bergman,  R.  E.,  Moore,  T.,  V.  1991),  (Prater,  H.,&  Paulissen,  D.  1994),  (Gerlic,  I. 
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1991),  (Jereb,  J.,&  Jug,  J.  1991),  (Tauber,  H.  1993),  (Wodaski,  R 1992).  Now  we  are  at  the  point  where  first  student 
group  are  learning  multimedia  with  the  computer. 


Description  of  Multimedia  course 

It  was  easy  to  establish  first  part  of  multimedia  course  "Multimedia  1".  We  have  many  years  of  experience  with 
similar  course  "Educational  technology".  Preparing  students  to  the  course  "Multimedia  2"  was  primary  objective 
of  course  "Multimedia  1".  Most  of  the  students  have  sufficient  level  of  knowledge  even  prior  attending  this 
course  and  most  of  them  just  love  working  with  the  latest  technology.  For  most  inquisitive  students  we  also  have 
video  studio  equipment.  Students  have  to  pass  through  every  working  place  except  video  studio. 

Multimedia  2 was  completely  new  course.  We  had  skilled  personnel  but  no  experiences  with  students  for  this 
course.  Preparation  of  working  places  and  working  instructions  required  paramount  attention.  Everything  was 
designed  to  be  easily  changed.  Now  we  could  increase  or  decrease  required  work  in  a matter  of  hours.  Since 
everything  is  accessible  on  the  Internet  and  students  can  prepare  themselves  before  they  actually  come  to  work 
with  equipment. 

We  will  explain  procedures  of  course  "Multimedia  2"  in  more  details.  The  course  "Multimedia  2"  has  nine 
working  places.  Attending  students  have  instant  help  from  assistant  and  technical  support  person.  Students  are 
encouraged  to  pass  through  as  many  working  places  as  they  can  during  their  course. 

Description  of  working  places: 

• three  are  for  processing  audio  signals  (CD,  tape,  radio,  voice,  Internet ...); 

• two  are  for  image  processing  (scanning,  digital  photography,  retouching,  OCR  ...); 

• one  is  for  animation  (GIF,  AVI,  MPEG  ..); 

• one  is  for  Internet  (searching  for  multimedia  web  sites,  educational  software,  downloading,  installing, 
writing  manuals  for  downloaded  programs, ..); 

• one  is  for  video  capturing  and; 

• the  last  one  is  for  presentation  (PowerPoint,  DVD  ...). 


One  of  the  greatest  problems  we  encounter  was  the  required  knowledge  of  using  a computer.  Students  should 
gain  required  knowledge  in  the  secondary  schools  but  in  reality  we  find  out  that  this  is  not  true.  Since  every 
student  have  free  access  to  faculties  computes  (Internet;  e-mail  account  and;  PPP)  we  assume  that  they  will  use 
these  services.  We  were  wrong.  Faculty  of  Education  has  a lot  of  different  student  programs.  We  have  to  teach 
them  all.  We  were  aware  that  students  from  natural  sciences  programs  would  easily  accept  computer  as  a 
working  tool  than  the  other  students.  In  order  to  solve  the  problem  we  have  to  remake  the  lessons  and  make  them 
in  such  a way  that  it  would  satisfy  diversities. 

We  also  have  a problem  in  the  system  itself  Special  didactics  course  (didactics  of  physics,  didactics  of 
geography  ...)  does  not  prepare  students  for  new  technologies.  Students  are  unaware  of  importance  of  multimedia 
educational  material.  To  overcome  this  problem  we  need  to  change  the  way  people  think. 

Among  other  problems  we  also  face  software  problem  when  we  find  out  that  Slovenian  Windows  are  not 
completely  compatible  with  English  version.  We  could  not  use  Volume  Control  to  change  any  settings  of  the 
sound  card.  Any  attempt  was  stopped  with  error  message  and  Volume  Control  termination.  We  solve  this 
problem  by  installing  English  version  of  Windows. 


Future  plan 

Our  future  plans  are: 

• making  multimedia  learning  material  to  be  used  by  students  and  teachers  in  primary  and  secondary 
schools; 

• establish  multimedia  advisory  service  for  primary  and  secondary  schools; 

• establish  standards  and  quality  control  for  multimedia  education  material; 

• lobbying  for  improving  national/university  computer  network  to  advanced  high  bandwidth  computer 


Problems 


network; 


• establish  multimedia  student  program; 

• establish  university  on  demand  TV; 

• establish  connections  with  other  universities  and  improve  students  and  lecturers  mobility  and; 

• we  will  be  open  minded. 


Conclusion 

In  today’s  modem  world  we  cannot  allow  time  to  pass  by.  Learning  must  become  more  effective  and  multimedia 
is  the  answer.  So  future  education  cannot  be  without  multimedia.  As  this  is  true  a lot  of  work  needs  to  be  done  to 
produce  enough  multimedia  educational  material  for  every  program  and  eveiy  course.  Constant  education  of 
teachers  and  students  is  essential  for  this  to  become  true.  Multimedia  education  material  and  high  bandwidth 
network  must  be  accessible  to  every  interested  individual. 
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Abstract:  How  do  we  create  an  innovative,  supportive  environment  for  teaching  and  learning  in  an  electronic 
environment  while  attending  to  copyright  and  intellectual  property  concerns  especially  considering  recent  changes 
made  to  copyright  law  in  the  United  States?  The  subject  of  intellectual  property  is  one  of  importance  to 
individuals  involved  in  all  aspects  of  Internet  and  Web  based  education  globally.  This  session  will  set  forth  some 
intellectual  property  concerns  and  issues  for  faculty  and  institutions  and  suggest  policy  and  content  and  audit 
points  for  consideration  and  discussion. 


Introduction 

The  electronic  environment  is  forcing  educators  at  all  levels  to  revisit  issues  concerning  intellectual 
property.  Quick  availability  of  information  and  data  electronically  has  changed  the  way  the  general  public  views 
information  since  it  is  in  seeming  less  endless  supply  on  anyone’s  desktop.  Thus,  faculty  and  students  alike  are 
faced  with  a figurative  smorgasbord  of  materials  in  varied  formats.  A dilemma  concerning  intellectual  property 
occurs  when  owners’  rights  collide  with  users’  rights  and  the  public  need  to  access  and  use  resources.  Thought 
provoking  papers  about  the  Internet  and  intellectual  property  are  becoming  available  on  the  World  Wide  Web. 
Esther  Dyson  has  written  a book,  Release  2.0,  and  several  articles  dealing  with  intellectual  property  on  the  Internet 
and  the  intellectual  value  of  property.  An  article  by  Vincent  J.  Roccia  presents  an  interesting  perspective  on 
copyright  law  in  the  United  States  and  possible  changes  or  clarifications  needed  perhaps  to  enhance  applicability  to 
the  Internet.  Current  copyright  laws  in  the  United  States  perhaps  do  not  quite  address  the  Internet  per  se  because 
it  represents  a challenge  to  existing  law  and  interpretations. 

This  paper  will  present  a brief  overview  of  several  topics.  The  first  is  ideas  and  concerns  as  to  why  faculty 
and  institutions  in  education,  particularly  higher  education,  have  good  reasons  to  ask  questions  concerning  “who 
owns  what”  in  a era  where  educational  opportunities  may  be  delivered  through  the  World  Wide  Web  and  various 
distributed  learning  systems.  Secondly,  it  will  present  issues  relevant  to  learner  support  and  student  ownership. 
Lastly  it  will  present  sources  of  information,  and  sample  policies,  as  well  as  ideas  about  ownership  of  intellectual 
property  which  institutions  might  consider.  A copyright  update  for  1 999  will  be  an  inclusion  of  this  brief  paper  on 
intellectual  property  concerns  and  issues.  These  issues  include  highlights  of  Digital  Millenium  Copyright  Act 
signed  into  law  in  October,  1 998,  new  provisions  for  library  preservation  copying,  and  distance  learning  guidelines 
currently  under  study  by  the  Register  of  Copyrights. 


Institutional  and  Faculty  Concerns 


Distribution  mechanisms,  format  of  materials,  and  traditional  models  of  ownership  are  bringing 
increasing  concerns  about  intellectual  property  development,  use,  and  application  to  a new  importance  in  academic 
conversations.  University  professors,  under  the  old  way  of  viewing  ownership,  were  the  single  owners  and  authors 
of  intellectual  property  with  a few  exceptions.  However,  in  today’s  environment,  ownership  may  be  shared  with 
several  individuals  or  organizations  as  new  technology  encourages  development  of  multimedia  products  using  a 
variety  of  formats  and  pieces  which  may  involve  multiple  layers  of  copyrighted  materials. 

The  dilemma  of  “who  owns  what”  is  compounded  when  one  considers  matters  such  as  how  much  support, 
use  of  facilities,  and  equipment  involvement  are  used  to  produce  a work  using  newer  electronic  technologies. 
Often  the  institution  has  invested  a great  deal  of  funds  in  a project  or  product  before  it  is  used  for  educational 
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purposes.  The  author  believes  that  basically  four  major  issues  concerning  intellectual  property  need  to  be 
addressed  by  faculty  and  institutions  through  dialogues,  policies,  and  communication: 

1.  Ownership  of  intellectual  property 

2.  Rights  to  use  intellectual  property 

3.  Procedural  issues  concerning  intellectual  property 

4.  Special  considerations  concerning  copyright 

In  most  cases,  an  “audit”  or  checklist  of  what  rights  need  to  be  acquired,  cleared,  or  considered  must  be 
developed  prior  to  design,  production,  and  delivery  of  a course,  for  example,  through  the  World  Wide  Web  or 
through  other  distance  learning  distribution  systems.  Institutional  personnel  should  think  through  what  rights  are 
needed  for  a course  and  what  all  future  possible  uses  and  distribution  mechanisms  of  course  content  might  be  prior 
to  release.  If  not  accomplished  prior  to  development,  the  entity  could  find  itself  having  to  retrace  steps  and 
renegotiate  or  acquire  additional  rights  at  more  cost.  Thus,  planning  ahead  with  a checklist  approach  could  save  a 
great  deal  of  money,  effort,  and  hassle  for  all  involved. 

Some  authors  and  organizations  have  made  information  available  which  can  encourage  and  guide 
discussions  about  intellectual  property  in  the  higher  education  environment.  Many  professional  associations  are 
discussing  the  issues  but  no  definitive  checklist  or  statement  exists  currently  since  ultimately  each  institution  must 
deal  with  the  volatile  topic  of  intellectual  property  and  issues  concerning  “who  owns  what”  at  the  state  or  local 
level.  A few  examples  of  resource  materials  are  included  at  the  end  of  this  paper.  An  interesting  document 
entitled  Ownership  of  New  Works  at  the  University:  Unbundling  of  Rights  and  the  Pursuit  of  Higher  Learning  by 
the  Consortium  for  Educational  Technology  for  University  Systems,  CETUS  (www.cetus.org),  suggests  several 
points  worth  considering  in  a discussion.  The  group  sets  forth  a viewpoint  that  simple,  individual  ownership  of  all 
rights  which  are  associated  with  copyright  may  not  now  be  the  most  desirable  avenue  as  it  may  stifle  creativity  and 
new  work  unduly.  Thus,  it  is  time  for  higher  education  to  revisit  ownership  of  intellectual  property  in  order  to 
avoid  contention,  place  the  focus  on  optimal  access  and  development  of  works,  and  reduce  the  emphasis  on 
economics  which  often  dominates  discussions  about  intellectual  property.  CETUS  sets  forth  the  “three  C 
approach,”  to  conversations  about  intellectual  property:  creative  initiative,  control  of  content,  and  compensation 
concerning  published  as  well  as  unpublished  works.  The  approach  is  quite  useful  as  it  may  be  less  intimidating 
and  antagonistic  than  beginning  with  the  economic  issues  and  may  apply  more  broadly  to  faculty  engaged  in 
producing  materials  but  not  necessarily  receiving  compensation  for  those  materials. 

The  CETUS  model  for  discussion,  the  “three  C approach”  presents  the  following  points.  The  first  point 
is  creative  initiative  and  poses  discussion  questions  such  as  “who  generated  the  idea  for  the  work,  whether 
published  or  unpublished,  and  who  created  the  work  and  fixed  it  in  a tangible  medium?  For  example,  a 
Department  Chairperson  might  encourage  faculty  to  publish  but  not  dictate  the  ideas  and  content.  The  entities  of 
initiator,  creator,  and  fixator  may  not  be  the  same.  The  second  point  for  discussion  deals  with  the  control  of 
content  as  to  who  controls  creation,  production,  specifications,  and  authority  for  acceptance.  The  degree  of  control 
is  something  which  might  be  negotiated.  The  third  point  is  compensation  and  other  support.  The  CETUS 
document  suggests  that  unless  the  two  were  extraordinary— above  what  faculty  are  normally  provided—  the  faculty 
would  most  likely  retain  ownership  of  intellectual  property  they  created.  Again,  there  are  many  areas  of 
negotiation  under  the  third  discussion  point. 

Another  way  of  approaching  ownership  of  intellectual  property  is  presented  in  a paper  by  Dan  L.  Burk,  an 
attorney  and  Associate  Professor  of  Law  at  Seton  Hall  University.  He  presents  a good  overview  of  models  for 
copyright  ownership  of  electronic  course  materials  between  faculty  and  their  sponsoring  institutions,  is  paper 
presents  detailed  discussion  of  advantages  and  disadvantages  of  three  sets  of  options  and  two  models  which  might 
be  considered  by  universities  and  colleges  drafting  documents  for  copyright. 

Bruwelheide  presents  in  her  writings  several  issues  concerning  intellectual  property  and  some  materials 
specifically  targeted  to  copyright,  telecourses,  and  Internet. 

In  summary  for  this  section,  it  is  important  for  universities,  colleges,  and  faculty  to  engage  in  dialogue 
concerning  intellectual  property  so  that  ownership  issues  are  clearly  defined  before  products  are  developed  by 
institutions,  faculty,  and  even  students.  There  is  plenty  of  room  for  negotiation  but  faculty  must  be  informed  about 
policies  and  participate  in  the  development.  If  not,  both  faculty  and  institutions  stand  to  lose  a great  deal. 
Creation  of  new  intellectual  property  is  very  important  and  the  climate  must  be  supportive  for  both  sides. 
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Student  and  Learner  Support  Issues  and  Concerns 


Students’  rights  to  ownership  of  intellectual  property  they  develop  while  students  or  student  workers  at  a 
college  or  university  need  to  be  considered  in  addition  to  rights  of  faculty.  In  some  instances,  universities  and 
colleges  lay  claim  to  all  work  produced  by  students.  This  issue  needs  to  be  addressed  by  institutions,  policies 
developed,  and  ways  found  to  inform  students  as  in  the  catalogs  for  the  institutions. 


Learner  support  issues  for  courses  delivered  via  distributed  learning  technologies  bring  new  concerns  to 
the  discussion  table.  Included  are  topics  such  as  electronic  reserves  for  libraries  and  housing  of  course  syllabi  and 
materials  on  a university  computer  network.  The  author  presents  questions  such  as  the  following; 

1.  What  rights  need  to  be  acquired  and  cleared  prior  to  posting  course  materials  including  syllabi  and 
readings  on  a closed  versus  an  open  network?  The  same  question  applies  to  courses  delivered  via  the  World  Wide 
Web  and  other  distributed  learning  networks  such  as  video  conferencing. 

2.  Is  an  online  class  treated  the  same  or  differently  than  a traditional  “face-to-face”  class  when  it  comes  to 
copyright  issues?  Many  institutions  are  treating  an  online  class  such  as  one  delivered  via  the  Web  or  through 
computer  conferencing  software  as  a closed  class  which  must  be  password  protected.  Thus,  only  students  actually 
enrolled  in  the  class  could  access  materials  with  a password  and  not  much  material  could  be  seen  by  non  enrolled 
individuals. 

3.  What  about  posting  student  work  to  a class  web  site?  It  would  seem  certainly  that  clearances  would 
have  to  be  obtained  from  the  students. 

4.  May  students  freely  use  materials  they  find  via  the  World  Wide  Web  for  class  projects?  Does  the 
format  make  a difference  or  are  formats  such  as  music,  video,  and  graphics  to  be  treated  differently  than  print 
materials? 

5.  May  library  personnel  scan  articles  for  online  class  reserve  systems?  If  so,  must  the  network  be  closed 
except  through  students  enrolled  in  a specific  class? 


Closing  Statement 


This  paper  has  set  forth  some  ideas  to  provoke  discussion  and  perhaps  policy  development  for  intellectual 
property  issues,  particularly  copyright,  which  need  to  be  addressed  by  institutions  of  higher  education  and  faculty 
in  order  to  maintain  the  flow  of  good  materials  in  a changing  electronic  environment.  Failure  to  discuss  and 
address  these  issues  may  result  in  a curbing  of  creativity  and  much  antagonism  on  the  parts  of  faculty  and  their 
institutions.  Many  models  and  points  for  discussion  exist  which  can  lead  to  successful  negotiation  of  ownership 
issues. 
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Resource  List 

Materials  dealing  with  intellectual  property  globally  are  abundant  and  quite  easily  located  via  the  World 
Wide  Web.  The  materials  selected  for  inclusion  in  this  list  are  by  no  means  comprehensive  and  the  list  is  not 
exhaustive.  Sites  were  selected  because  they  provide  unique  information  and/or  links  to  many  additional  resources 
on  a variety  of  intellectual  property  topics  issues,  and  perspectives.  Readers  may  follow  links  in  the  resources  to 
obtain  more  information.  While  the  primary  focus  is  on  USA  resources,  some  urls  are  provided  for  international 
perspectives  and  have  links  to  hundreds  of  sites.  Sources  are  current  as  of  mid- 1999. 


World  Wide  Web  Source  List 


Consortium  for  Educational  Technology  in  University  Systems:  CETUS  Discussion  Series 
http://www.cetus.org 

Copyright  Crash  Course,  University  of  Texas 
http://www.utsystem.edu/OGC/IntellectualProperty 

Copyright  Management  Center:  Indiana  University-Purdue 
http  ://www.  iupui  .edu/it/  copy  info/ 

Copyright  Office,  Library  of  Congress: 
http://lcweb.loc.gov 

Fair  Use  Center 
http://fairuse.stanford.edu 


Sample  Intellectual  Property  Policies  in  the  United  States 


Carnegie  Mellon: 

http://gollum.mac.cc.cmu.edu/univ_policy/documents/IntellProp.html 


Copyright  Resources  Online: 

http://www.library  .yale . edu : 80/~okerson/copyproj . html 

Copyright  Resources  Online  --  Policies 
http://www.library.yale. edu:80/~okerson/copyproj.html#ucopy 

Johns  Hopkins  Medical  School: 

http://cwis.welch.jhu.edu/policy/intellectuaLprop_guide/somJntpol2.html 
University  of  Massachusetts: 

http://woods.uml.edu/www/telecom/TADS/tadsfwc.html 


Library  Issues 

Electronic  Scholarly  Publication: 
http://www.arl.org/transform/esp/index.html 

Electronic  Reserves: 

http://www.research.umbc.edu/aok/reserve.html 
Faculty  Guidelines: 

http://www.lib.umich.edu/libhome/Reserves/faculty/faculty.html 

Liblicense:Licensing  Digital  Information: 
http://www.library.yale.edu/~Llicense/index.shtml 

Licensing: 

http://www.arl.org/scomm/licensing/licbooklet.html 


International  Intellectual  Property  Sources 

Digital  Future  Coalition 
www.dfc.org 

Electronic  Freedom  Foundation 
http://www.eff.org/pub/lntellectuaLproperty/ 

European  Commission  Legal  Advisory  Board  Intellectual  Property 
http://www2.echo.lu/legal/en/ipr/ipr.html 

Franklin  Pierce  Law  Center: 
http://www.fplc.edu/ipmall/pointbox/pb_intr.htm 

Intellectual  Property:  Copyright  and  Intellectual  Property 
http://www.nlc-bnc.ca/ifla/II/cpyright.htm 

Intellectual  Property  Law  World  Wide 
http : //  WWW . ip  w w . com/i  ndex . html 

Public  Policy  Issues  Related  to  Intellectual  Property 
http://www.essential.org/cpt/ip/ip.html 
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World  Intellectual  Property  Organisation  - WIPO 
http://www.wipo.int/eng/index.htm 

World  Trade  Organisation  - WTO 
http://www.unicc.org/wto/Welcome.html 

World  Wide  Web  Virtual  Library:  Law  : Intellectual  Property 
http://www.law.indiana.edu:80/law/v-lib/intellect.html 
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Abstract:  This  study  investigated  the  use  of  asynchronous  (mailing  lists)  and  synchronous  (chat  sessions) 
Internet'based  communication  and  its  impact  on  teachers’  attitude  toward  collaboration,  activity 
completion  rate  and  test  performance.  Although  in  this  study  it  was  found  that  attitudes  toward 
collaboration  did  not  affect  test  performance,  the  data  suggested  a relationship  between  attitudes  toward 
collaboration  and  use  of  Internet-based  communication.  Also,  it  was  found  that  use  of  Internet-based 
communication  increased  the  likelihood  of  completing  the  course  activities. 


OBJECTIVES 

Cognitive  models  of  learning  stress  that  learners  are  active  agents  in  constructing  their  own  learning  (Shaw, 
1995;  Biehler  & Snowman,  1986).  Asynchronous  and  synchronous  communication  provides  users  with 
opportunities  to  collaborate  across  distances  through  such  technologies  as  mailing  lists  and  chat  sessions, 
respectively.  It  was  hypothesized  that  these  modes  of  communication  provide  unique  features  conducive  to  the 
development  of  a collaborative  community  of  learners.  The  benefits  of  Internet-based  communication  and 
collaboration  are  not  reflected  solely  in  test  performance.  Behind  the  behavioral  change  is  an  attitudinal  change. 
Gain  in  test  performance  might  be  short  term;  change  in  attitude  could  facilitate  life-long  learning  (Yu,  1998). 
Therefore,  this  study  investigated  the  use  of  asynchronous  (mailing  lists)  and  synchronous  (chat  sessions)  Internet- 
based  communication  and  its  impact  on  teachers’  attitude  toward  collaboration,  activity  completion  rate  and  test 
performance. 


THEORETICAL  FRAMEWORK 

Several  theoretical  perspectives  suggest  that  peer  interaction  occurring  in  computer  networked  groups  may  be 
beneficial  to  learning.  These  perspectives  include  Vygotsky's  (1978)  and  Piaget's  (1963)  emphasis  on  social 
construction  of  knowledge  and  Bandura's  (1969)  interpretation  of  peer  modeling  (King,  1989).  From  a social 
cognition  point  of  view,  when  educators  interact  with  peers  they  are  faced  with  ideas,  explanations  and  information 
inconsistent  with  prior  knowledge  and  beliefs.  Confronting  these  inconsistencies  challenges  one's  current  point  of 
view;  resolving  such  cognitive  conflicts  results  in  cognitive  restructuring  (Bearison,  1982).  Group  members 
presumably  add  detail  to  existing  cognitive  schemas,  explore  and  correct  misunderstandings,  fill  in  gaps  in  their 
knowledge,  and/or  reorganize  their  knowledge  structures  (Bearison,  1982;  Piaget,  1963;  Vygotsky,  1978). 

Collaborative  learning  is  the  acquisition  by  individuals  of  knowledge,  skills  or  attitudes  as  a result  of  group 
interaction,  where  that  specific  learning  could  not  have  been  derived  individually  (Schrage,  1990).  Computer- 
mediated  collaboration  provides  a medium  for  educators  to  learn  to  engage  in  collaboration  with  other  educators  at 
a distance.  Alavi  (1994)  observes  that  those  involved  in  collaboration  can  monitor  individual  thinking,  opinions 
and  beliefs  and  provide  feedback  for  clarification  and  change. 

Internet-based  communication  provides  both  asynchronous  and  synchronous  environments  for  collaboration. 
Synchronous  communication  (such  as  chat)  allows  interaction  to  occur  at  the  same  time  among  learners,  as  in 
classroom  discussions:  but  learners  in  the  Internet  environment  use  a keyboard  to  type  messages.  Local  Internet 
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accounts  enable  learners  to  engage  in  collaborative  chat  sessions  with  distant  peers,  expanding  the  collaborative 
environment  from  a single  classroom  to  classrooms  around  the  world.  Asynchronous  communication  (mailing 
lists)  allows  interaction  to  occur  at  different  times  and  locations  between  two  or  more  learners.  Learners  need  not 
be  present  to  receive  information,  and  may  communicate  when  ready.  Learning  occurring  as  a result  of  group 
interaction  may  take  place  in  a virtual  community  and  may  be  adjusted  to  student  need,  schedule  and  educational 
goals.  Both  of  these  forms  of  Internet-based  communication  expand  learners'  access  to  information,  resources,  and 
collaboration,  and  may  increase  learners'  attitude  toward  collaboration  as  well  as  achievement  performance.  In  this 
study,  we  are  interested  in  both  forms  of  Internet-based  communication  as  they  relate  to  learner  attitude  toward 
collaboration  and  the  completion  rate  and  performance  on  Internet  skill  tests. 


HYPOTHESES 

Based  on  the  above  theoretical  framework,  the  following  hypotheses  were  formed: 

(a)  Use  of  Internet-based  communication  increases  teachers’  positive  attitude  toward  collaboration  on  the  Internet 
as  measured  by  the  Stages  of  Concern  Instrument:  Attitudes  Toward  the  Internet  {ATI\  Hall,  George,  & 
Rutherford,  1977;  Well  and  Anderson,  1997); 

(b)  Use  of  Internet-based  communication  increases  course  completion  rate; 

(c)  Use  of  Internet-based  communication  increases  test  performance  as  measured  by  criterion-referenced  Internet 
skills  test; 

(d)  The  more  positive  teachers’  attitudes  toward  collaboration,  the  higher  the  increase  in  teacher  performance  as 
measured  by  criterion-referenced  Internet  skills  test;  and 

(e)  The  more  positive  teachers’  attitudes  toward  collaboration,  the  higher  the  course  completion  rate. 


METHODS 

PARTICPANTS 

Participants  in  this  study  were  one  hundred  sixty-one  K-12  educators  residing  throughout  a southwestern  state. 
All  participants  took  part  in  an  interactive  web-based  course  designed  to  enhance  the  collaborative  teaching  and 
learning  communities  for  participating  educators  as  they  acquired  Internet  skills. 


TREATMENT 

The  treatment  was  an  interactive  web-based  course  designed  to  enhance  the  collaborative  teaching  and 
learning  communities  for  participating  educators  as  they  acquired  Internet  skills.  Of  particular  interest  were  new 
outcomes  made  possible  through  use  of  advanced  technologies  in  instruction.  Therefore  we  looked  at  the 
engagement  in  Internet-based  communication  and  increases  in  Internet  skills. 


PROCEDURE 

Participants  were  part  of  a large  collaborative  community  (the  course)  as  well  as  of  smaller  communities 
focused  on  establishing  professional  relationships  and  on  technical  and  course  help.  These  smaller  communities 
were  facilitated  by  a group  leader  trained  in  the  use  of  Internet  technologies  and  in  how  to  facilitate  Internet-based 
communication.  Participants  had  access  to  chats  and  were  members  of  two  course-related  mailing  lists,  (a)  course 
announcements,  and  (b)  a group  list  headed  by  the  group  leader.  Group  leaders  conducted  at  least  one  chat 
bimonthly  and  communicated  with  their  participants  via  group  mailing  lists  biweekly.  Participation  in  chats  was 
mandatory  for  group  leaders  and  strongly  encouraged  for  participants.  Participants  were  encouraged  to  join  other 
mailing  lists  within  the  course  setting  as  well  as  outside  of  the  course.  Archives  of  eleven  chat  sessions  and  eleven 
mailing  list  groups  were  used  in  this  investigation.  The  Internet  skills  that  participants  were  to  learn  focused  on 
searching  techniques  and  the  use  of  email.  Participants  completed  a pretest  assessing  their  Internet  skills  in  these 
areas.  Following  the  pretest,  participants  completed  tutorials  specific  to  these  skill  areas,  then  completed  a posttest. 
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DATA  COLLECTION 

Two  instruments  were  employed  in  this  study.  The  first  instrument  was  the  Stages  of  Concern  Instrument: 
Attitudes  Toward  the  Internet  {ATI;  Hall,  George,  & Rutherford,  1977).  This  scale  was  developed  to  measure  the 
evolving  attitudes  of  learners  exposed  to  an  innovation.  In  this  original  scale,  the  variable  word  “innovation”  may 
be  substituted  with  the  innovation  of  interest.  Therefore,  in  a study  regarding  learner  attitudes  toward  a new 
innovation,  specifically  the  Internet,  Wells  and  Anderson  (1997)  substituted  the  variable  “innovation”  with 
“Internet”  to  measure  attitudes  toward  the  Internet.  Reliability  of  the  instrument  was  reported  as  .91  (Reed,  1990), 
and  discussed  in  detail  by  Wells  and  Anderson  (1997).  The  second  instrument  was  a performance  test  over  the 
Internet  skills  learned  that  was  developed  by  our  research  team.  For  both  measurements,  a pretest  was 
administered  prior  to  a web-based  instruction  program.  Three  months  later  a posttest  was  given.  The  interval  was 
short  in  order  to  avoid  the  "maturation"  effect  in  which  subjects  improve  over  time  regardless  of  treatments  (Cook 
& Campbell,  1979).  All  participants  completed  the  pretest  and  95  finished  the  posttest.  One  hundred  and  fifty-nine 
participants  completed  the  ATI  survey  and  fifty-six  completed  the  post-survey. 

A subscale  of  ATI  was  used  to  measure  the  attitude  toward  collaboration  though  the  entire  survey  was 
administered  to  the  participants.  The  survey  was  composed  of  35  questions  and  could  be  divided  into  seven 
subscales,  which  indicated  seven  stages  of  concern  with  the  Internet.  The  seven  stages  were  awareness, 
informational,  personal,  management,  consequence,  collaboration,  and  refocusing  (Wells  & Anderson,  1997). 
Please  see  Table  1 for  the  items  in  the  collaboration  subscale. 


Question  5: 
Question  10: 
Question  18: 
Question  27: 
Question  29: 


I would  like  to  help  other  faculty  use  the  Internet. 

I would  like  to  work  with  present  fellow  workers  and  others  who  are  using  the  Internet. 

I would  like  to  familiarize  my  colleagues  about  the  Internet  as  I learn  more  about  it  and  with  it  more. 
I would  like  to  coordinate  my  efforts  in  learning  about  the  Internet  with  fellow  workers. 

I would  like  to  know  what  other  people  are  doing  in  relation  to  the  Internet. 


Table  1:  Collaborative  subscale  in  Stages  of  Concern  Instrument:  Attitude  Toward  Using  the  Internet 

Factor  analysis  confirmed  the  factor  structure  of  the  instrument.  Based  on  the  pre-survey  data,  the  survey  items 
could  be  grouped  as  eight  latent  variables  according  to  the  Kaiser  criterion  (eigenvalue  as  one  or  larger).  By 
examining  the  inflection  point  in  the  scree  plot,  seven  factors  could  extract  most  of  the  eigenvalues.  In  the  post- 
survey, Kaiser  criterion  also  suggested  eight  factors  while  the  scree  plot  implied  five  to  seven  factors. 

Based  upon  the  pre-survey  data,  the  estimated  reliability  of  the  entire  ATI  survey  in  terms  of  Cronbach 
coefficient  Alpha  was  .86  whereas  the  Cronbach  coefficient  Alpha  for  the  collaboration  subscale  was  .87.  In  the 
ATI  post-survey  data,  the  overall  Alpha  was  .89  while  the  collaboration  subscale  Alpha  was  .79. 

The  second  instrument  in  this  study  was  a criterion-reference  performance  test,  which  consisted  of  30  items. 
Cronbach  coefficient  Alpha  of  the  pretest  was  .45  whereas  that  of  the  posttest  was  .86.  The  low  Alpha  in  the 
pretest,  which  was  expected,  implied  that  participants  were  unfamiliar  with  the  subject  matter  and  random 
guessing  of  answers  led  to  inconsistency.  In  the  posttest  the  high  Alpha  denoted  that  the  response  pattern  was 
internally  consistent. 

Finally,  use  of  mailing  lists  and  use  of  Internet  chat  were  logged  as  a measure  of  engagement  in  Internet-based 
communication  as  a collaborative  tool.  Originally,  engagement  in  collaboration  was  defined  by  the  number  of 
words  in  email/chat  and  the  total  number  of  messages.  However,  these  distributions  were  highly  skewed  and  thus 
violated  the  assumptions  of  parametric  tests.  Therefore,  observations  were  re-coded  into  four  binary  variables:  (a) 
use  of  mailing  lists,  (b)  use  of  chat  sessions,  (c)  use  of  both  mailing  lists  and  chat  sessions,  and  (d)  use  of  neither 
mailing  lists  or  chat  sessions. 


RESULTS 

Prior  to  addressing  the  hypotheses,  two  one-tailed  dependent  t-tests  were  employed  to  examine  whether 
improvement  in  teacher  scores  on  the  Internet  skills  test  and  the  ATI  collaboration  subscale  occurred  after  the 
treatment.  The  dependent  t-test  for  the  difference  between  the  pretest  and  posttest  yielded  a significant  difference,  t 


(94)  = 16.33,  p = .0001.  Given  that  the  effect  size  was  .03,  the  alpha  level  was  .05,  and  the  sample  size  was  95,  the 
power  for  this  t-test  was  .90,  which  indicated  a high  probability  that  the  null  hypothesis  was  correctly  rejected. 

On  the  average  participants  had  a decrease  in  the  positive  attitude  toward  collaboration  after  the  treatment,  M 
= -0.67.  A dependent  t-test  showed  that  this  decrease  was  not  significant,  t (42)  = -0.88,  p = 0.38.  This  result  might 
be  due  to  the  lack  of  statistical  power.  Given  that  the  effect  size  was  .03,  the  alpha  level  was  .05,  and  the  sample 
size  was  43,  the  power  for  this  t-test  was  .61,  which  indicated  that  a true  significant  difference  might  be 
undetected. 

To  test  the  hypothesis  regarding  the  relationship  between  engagement  in  Internet-based  communication  and 
attitude  toward  collaboration  (hypothesis  a),  exploratory  data  analysis  instead  of  ANOVA  was  performed  to 
examine  the  relationships.  The  data  structure  did  not  conform  to  the  parametric  assumptions  of  ANOVA  such  as 
normality  and  homogeneity  of  variances,  and  this  sample  size  was  not  large  enough  to  run  a robust  ANOVA  test. 
In  the  exploratory  data  analysis,  it  was  found  that  users  of  chat  and  mail  tended  to  center  around  the  mean  of  the 
difference  between  collaboration  pre-survey  and  post-survey,  as  shown  in  Figure  1.  Participants  who  engaged  in 
chat  sessions  had  less  variability  on  the  change  score  of  ATI  (-7.5  to  1.5)  than  the  general  test  population. 
Likewise,  participants  who  posted  messages  on  mailing  lists  also  centered  around  the  mean  (-4.5  to  4.5). 
Therefore  this  might  indicate  that  participants  who  engaged  in  chat  sessions  or  mailing  lists  have  less  change  in 
their  attitude  toward  collaboration  than  those  in  the  general  test  population,  maintaining  scores  close  to  the  mean. 
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Figure  1:  Linked  observations  between  use  of  collaboration  and  changed  scores  in  attitude  toward  collaboration 

A Chi-square  test  of  goodness  of  fit  was  applied  to  investigate  whether  engagement  in  Internet-based 
communication  could  increase  the  likelihood  of  completing  the  entire  instructional  module  (hypothesis  b). 
Likelihood  ratio  Chi-square  was  used  because  it  is  related  to  log-linear  model  and  logistic  regression  model,  and 
completion  of  module  was  a dichotomous  variable.  The  Chi-square  analysis  yielded  a significant  result,  Likelihood 
ratio  = 8.19,  p = .042.  Because  one  of  the  cells  had  no  subjects  and  some  cell  sizes  were  very  uneven,  a Chi- 
square  test  might  not  be  valid.  To  rectify  this  shortcoming,  an  exact  test  was  performed  using  StatXact  (SPSS  Inc., 
1999).  Both  asymptotic  inference  and  exact  inference  yielded  significant  results.  In  the  former  the  one-sided  p- 
value  was  .02  whereas  in  the  latter  the  one-sided  p-value  was  .03.  These  results  indicate  that  engaging  in  Internet- 
based  communication  increases  the  likelihood  of  completing  Internet  activities. 

As  the  data  structure  did  not  conform  to  the  parametric  assumptions  of  ANOVA,  a one-tailed  two-sample 
independent  t-test  was  applied  to  examine  hypothesis  (c),  regarding  whether  engagement  in  Internet-based 
communication  could  lead  to  improvement  as  measured  by  the  difference  between  the  pretest  and  the  posttest. 
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Originally,  use  of  Internet-based  communication  was  classified  into  four  groups,  namely,  use  of  chat,  use  of  email, 
both,  and  none.  For  conducting  a t-test,  the  first  three  groups  were  collapsed  into  one  group.  As  a result,  only  two 
groups  remained,  namely,  use  of  collaboration  and  no  use  of  collaboration.  The  two-sample  t-test  did  not  yield  a 
significant  result,  t(28)  = .61,  p = .54.  Therefore,  use  of  collaborative  tools  did  not  appear  to  have  an  impact  on  test 
performance. 

Regression  analysis  was  adopted  to  test  hypothesis  (d),  which  was  concerned  with  the  relationship  between 
attitude  toward  collaboration  and  test  performance.  The  assumption  of  random  residual  was  checked  by  plotting 
predicted  scores  against  residuals.  The  regression  analysis  disconfirmed  use  of  Internet-based  communication  as  a 
predictor  to  Internet  skills  test  performance,  (42)  = .0005,  p = .88.  The  scatterplot  of  the  two  variables  did  not 
display  a pattern  (Figure  2). 
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Figure  2:  Scatterplot  of  attitude  toward  collaboration  against  posttest  scores. 

A logistic  regression  was  employed  to  examine  hypothesis  (e),  which  addressed  the  relationship  between 
activity  completion  and  the  attitude  toward  collaboration  in  the  Internet.  The  logistic  regression  analysis  did  not 
yield  a significant  result,  p = .60,  odds  ratio  = .98,  therefore  scores  on  attitude  toward  collaboration  did  not  predict 
course  completion  rate. 


DISCUSSION 

Collaboration  provides  a way  for  educators  to  form  a virtual  community  of  learners  in  which  members  can 
engage  in  individual  thinking,  share  opinions  and  beliefs  and  provide  one  other  with  feedback  for  professional  and 
personal  growth  and  change.  In  the  course  studied.  Internet-based  communication  allowed  participants  to  engage 
in  collaboration  with  others  across  the  state.  Although  the  Internet  provides  different  types  of  environments  for 
communication  and  collaboration  among  users,  their  effects  are  as  yet  unknown.  In  this  study,  it  was  hypothesized 
that  asynchronous  and  synchronous  Internet-based  communication  provides  unique  features  that  promote  the 
development  of  collaborative  communities.  In  addition,  this  environment  would  then  impact  participants’  attitudes 
toward  collaboration  and  performance  on  Internet  skill  tests. 

Although  in  this  study  it  was  found  that  attitudes  toward  collaboration  did  not  affect  test  performance,  the  data 
suggested  a relationship  between  attitudes  toward  collaboration  and  use  of  Internet-based  communication.  It  is 
encouraging  that  use  of  Internet-based  communication  increases  the  likelihood  of  completing  the  course  activities, 
though  no  link  by  instructional  theory  was  established  between  the  two  variables.  It  was  speculated  that  those  who 
used  chat  sessions  and  mailing  lists  were  motivated  by  their  peers,  or  self-motivated  by  their  own  engagement,  but 
no  measure  of  motivation  was  conducted  in  this  study  and  thus  the  issue  remains  inconclusive. 
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The  results  do  not  substantiate  the  remaining  hypotheses.  Therefore  it  may  be  concluded  that  in  this 
population,  attitudes  toward  collaboration  are  little  effected  by  engagement  in  an  Internet-based  course.  In 
addition,  using  Internet-based  communication  such  as  chat  sessions  and  mailing  lists  do  not  effect  participant 
performance.  Similarly,  participants’  attitudes  toward  collaboration  do  not  predict  performance  on  Internet  skill 
tests. 

Although  not  all  results  lead  to  positive  conclusions,  it  is  important  to  understand  the  aspects  of  Internet-based 
communication  that  may  contribute  to  the  development  of  collaboration  among  educators  at  a distance. 
Information  about  the  characteristics  of  Internet-based  communication  and  methods  of  implementing  effective 
collaboration  must  be  investigated  to  increase  educators'  effectiveness,  to  enhance  achievement  for  all  students  and 
to  promote  systematic  school  improvement.  This  study  provides  a step  toward  such  research. 

Collaboration  is  not  a context-free  concept:  its  implementation  is  tied  to  specific  types  of  communication 
provided  by  the  medium  and  the  specific  application  within  the  medium.  As  indicated  in  this  paper,  mailing  lists 
and  chat  sessions  represent  two  different  modes  of  communication.  Within  these  modes,  there  is  further 
classification  by  application  such  as  moderated  and  unmoderated  groups.  Findings  in  this  study  should  not  be  over- 
generalized and  different  research  studies  for  different  modes  and  applications  of  Internet-based  communication 
are  recommended. 

Due  to  the  sample  size  and  the  distributional  assumptions,  variables  in  this  study  were  examined  in  a pairwise 
manner.  This  research  project  serves  as  a preliminary  study  to  identify  relationships  among  variables.  As 
relationships  are  identified,  a coherent  model  specifying  the  inter-relationships  among  variables  can  be 
constructed.  To  achieve  this  goal,  structural  equation  modeling  with  a large  sample  size  will  be  employed. 
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Abstract:  This  paper  describes  the  process  of  converting  an  interactive  multimedia  training 
program  with  laser  disc  video  to  a program  incorporating  MPEG-1  compressed  video  on  CD- 
ROM.  An  overview  of  the  hardware  and  software  demonstrates  how  a conversion  can  be 
accomplished  by  taking  a standard,  well-equipped  personal  computer  and  adding  less  than 
$ 1 500  of  additional  hardware  and  software.  A detailed  of  process  outlines  how  a quality  video 
production  can  be  obtained  with  minimal  cost.  Video  capture  and  edit  techniques  are  outlined 
to  provide  a workable  template  for  developers  to  follow.  Working  at  the  “low  end”  of  “high 
tech”  can  produce  a very  acceptable  multimedia  product. 


Introduction 

The  original  interactive  multimedia  program,  Managing  Asthma,  was  developed  by  personnel  at  Carle 
Foundation  Hospital  and  Illinois  State  University.  It  was  used  to  train  staff  how  to  teach  asthmatic  patients  the 
effective  use  of  inhalers  and  peak  flow  meters  (Byers  and  Rhodes,  1998).  This  program,  authored  with 
Asymetrix  ToolBook  II  Instructor  6.0,  contained  179  computer  screens  and  a total  of  30  minutes  of  full-motion 
video  edited  into  43  separate  files.  The  video  was  displayed  with  an  overlay  board  directly  on  the 
microcomputer  monitor.  User  control  of  the  laser  disc  player  was  enabled  through  a custom-built  program 
integrated  into  ToolBook  II  Instructor. 

The  multimedia  system,  i.e.,  microcomputer,  touch-screen  monitor,  videodisc  player  and  speakers 
were  placed  in  a self-contained  wheeled  “traveling  case.”  This  traveling  case  was  placed  in  four  departments  at 
Carle  Foundation  Hospital  and  at  seven  of  the  associated  Carle  Clinics  in  Illinois.  The  system  was  available  in 
areas  such  as  lunchrooms,  break  areas  on  the  hospital  floor,  or  staff  lounges. 

During  program  development,  formative  reviews  were  carried  out  by  health  care  specialists  and 
designers.  These  reviews  helped  ensure  that  the  program  incorporated  principles  basic  to  the  design  of 
successful  multimedia  training  applications  (Palmer  and  Rhodes,  1997).  More  than  200  health  care  personnel 
who  used  the  program  completed  an  evaluation  form  and  selective  interviews  were  also  conducted.  The 
response  indicated  that  staff  had  a very  positive  experience  using  the  interactive  multimedia  system  and  find  it 
quite  helpful.  More  than  70%  of  those  surveyed  rated  the  program  as  “excellent,”  and  25%  rated  it  as  “good.” 

Although  the  program  itself  had  exceptional  ratings  by  users,  the  logistics  of  moving  the  traveling  case 
from  location  to  location  and  related  difficulties  of  system  maintenance  became  significant  problems  for  health 
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care  personnel.  While  additional  programs  on  asthma  management  and  other  disease  entities  would  be 
welcomed,  an  alternative  to  the  laser  disc  format  became  necessary.  Compressed  video  on  CD-ROM  would 
solve  many  of  the  problems  identified,  especially  those  of  portability,  system  availability  and  cost.  With  the 
proliferation  of  CD-ROM  technology,  program  distribution  would  not  be  limited  to  the  traveling  kiosk  or  only 
to  locations  with  a similar  multimedia  workstation.  Unlike  laser  discs,  CD-ROMs  could  be  duplicated  almost 
immediately  at  minimal  cost  as  additional  institutional  usage  dictated.  Other  features,  such  as  audio  over  text 
would  be  available.  A determining  factor,  however,  would  be  maintaining  the  quality  of  video  scenes  because 
this  was  a significant  positive  factor  for  users. 


Video  encoding 

As  the  first  step  in  the  conversion  process,  during  the  winter  of  1998  we  examined  demonstration 
versions  of  some  two  dozen  commercial  programs  with  CD-ROM  video.  The  results  were  not  encouraging, 
especially  for  playback  on  a Pentium  90mhz  microcomputer  with  a 24x  CD-ROM  player  comparable  to  many 
of  the  machines  to  be  found  in  the  heath  care  setting.  The  most  successful  of  the  program  in  terms  of  video 
quality  were  those  in  which  the  original  scenes  had  been  made  with  “artificial”  (blue-screen)  backgrounds  that 
reduced  the  amount  of  information  to  be  compressed.  Since  the  Managing  Asthma  video  had  been  filmed  in  a 
real  hospital  setting,  this  was  not  an  option  for  us. 

At  one  point  we  considered  “leapfrogging”  CD-ROM  delivery  and  moving  directly  to  a DVD  format. 
We  soon  found  that  an  initial  startup  and  certification  cost  of  $50,000,  conflicting  technical  standards,  lack  of 
development  hardware  and  software  in  the  university  setting  and  playback  units  in  the  healthcare  setting  made 
this  technology  impractical  for  us,  at  least  for  the  present. 

After  considerable  further  investigation,  we  located  a software  program  containing  compression 
algorithms  that  are  significantly  advanced  enough  to  support  required  video  quality  levels  on  CD-ROM.  This 
program  is  the  XingMPEG  Encoder  2.2  from  Xing  Technologies  [www.xingtech.com]  at  a cost  of  $249.00. 
The  XingMPEG  Encoder  will  compress  30  minutes  of  video  (.AVI  format)  into  MPGl  files  that  fit  onto  one 
CD-ROM.  Screen  image  resolutions  from  160  x 120  pixels  to  352  x 240  pixels  are  available. 

We  have  found  that  both  256  xl92  pixel  and  352  x 240  pixel  images  both  provide  more  than  adequate 
video  quality  for  interactive  multimedia  training  purposes.  Indeed,  by  scaling  we  enlarged  the  352  x 240  pixel 
images  to  approximately  600  x 400  pixels  without  noticeable  loss  of  video  quality.  This  conclusion  does  not 
come  from  a controlled  research  study,  but  from  observation  of  experienced  PC  users.  We  had  planned  such  a 
formal  study,  but  after  receiving  user  comments  such  as  “Thaf  s really  great,”  and  “1  didn’t  know  you  could  do 
that,”  we  simply  moved  ahead  with  development.  As  audio  and  video  are  synchronized  and  there  are  no 
obvious  computer  artifacts  with  Xing  files,  video  quality  is  not  an  issue  for  viewers. 

For  the  developer,  the  XingMPEG  Encoder  has  a batch-processing  feature  that  will  compress  a list  of 
.AVI  files  without  operator  attention.  The  task  list  feature  enables  one  to  reconstruct  individual  MPG  files 
should  changes  be  required.  XingMPEG  Encoder  supports  PAL,  NTSC,  and  FILM  formats,  with  a wide  range 
of  data  rates  from  128kbps  to  over  1200kbps.  Video  can  thus  be  customized  for  various  delivery  environments 
such  as  hard  drives,  local  area  networks,  or  CD-ROMs. 

The  interface  screen  for  XingMPEG  Encoder  is  very  intuitive,  displaying  the  New,  Edit,  Delete  and 
Encode  buttons  in  the  main  interface  window.  For  each  entry  the  operator  selects  the  video  format  desired  and 
identifies  the  source  file  and  destination  file.  This  feature  saves  time  and  minimizes  the  possibility  of  errors. 
The  compression  process  is  not  time-intensive;  for  example,  we  found  that  a 30  second  .AVI  file  could  be 
compressed  into  a .mpg  file  in  about  four  minutes.  Batch  processing  can  be  set  up  to  run  overnight  without 
operator  intervention.  In  this  conversion  project,  30  minutes  of  video  in  43  .AVI  files  spanning  4.2GB  of 
uncompressed  data  were  compressed  to  an  equal  number  of  .mpg  files  spanning  481MB  resulting  in  a 
compression  ratio  of  8.79  to  1 . As  a compact  disc  will  hold  650MB  of  data,  adequate  storage  for  program, 
graphics,  and  audio  files  was  still  available. 


Video  capture  and  editing 

Without  quality  digital  .AVI  files  captured  from  analog  video,  compression  to  MPGl  is  not  a 


1052 


profitable  venture.  We  have  found  that  the  Miro  DC30  Plus  capture  card  and  associated  software  from  Pinnacle 
Systems  [www.pinnaclesys.com]  at  a cost  of  $900  has  done  everything  that  we  require.  The  Miro  DC30  card 
will  capture  NTSC  (composite  and  S-Video)  and  PAL  analog  video  at  7MBps  with  minimal  frame  loss,  perhaps 
one  frame  for  video  minute  on  the  PC  employed.  The  software  for  Windows  95  comes  bundled  with  the  board; 
the  user  interface  is  simple  and  procedural.  Software  for  Windows  NT  can  be  downloaded  from  Miro;  the  user 
interface  is  more  sophisticated,  with  both  introductory  and  advanced  levels.  We  have  used  the  Miro  D30  card 
and  software  to  successfully  capture  motion  and  still  frame  video  from  S-Video,  Hi 8,  and  VHS  tape  decks  and, 
for  the  Managing  Asthma  program  specifically,  a Pioneer  laser  disc  player. 

The  capture  process  was  made  much  simpler  by  having  the  file  names  and  entry  and  exit  points  on  the 
laser  disc  defined  before  capture.  All  systems  used  the  “capture  to  memory”  feature  with  the  overflow  going  to 
a hard  disk.  When  capturing  video  from  laser  disc  or  S-Video  tape,  a thin  bar  of  unsynchronized  video 
appeared  across  the  bottom  of  the  capture  window.  This  problem  was  resolved  by  selecting  the  “TV  cropping” 
setting.  We  captured  NTSC  video,  quarter  screen,  with  square  pixels  at  a rate  of  29.97  frames  per  second. 
Once  the  process  was  mastered,  which  took  less  than  a day,  fewer  than  three  minutes  were  required  to  setup 
each  video  sequence  for  capture. 

Because  the  video  scenes  had  already  been  prepared  for  the  laser  disc  version  of  Managing  Asthma^ 
video  editing  was  only  a minor  part  of  the  conversion  process.  When  capturing  video,  we  began  about  a second 
before  and  ended  a second  after  the  designated  segment.  The  excess  frames  were  easily  cropped  during  editing. 
We  experimented  with  cropping  files  using  two  editing  programs,  Adobe  Premier  4.2  and  MGI  VideoWave. 
Both  were  satisfactory  for  our  purposes,  though  with  differences:  Premier  has  more  editing  features  than  Video 
Wave,  but  with  its  relatively  complicated  interface  is  more  difficult  to  use. 


PC  hardware  and  software 

The  personal  computer  used  in  the  conversion  process  was  a Dell  300MHz  Pentium  II  machine  with 
10GB  of  HIDE  storage  and  128MB  of  memory.  SCSI  hard  drives  would  have  been  preferable  to  the  HIDE 
drives  in  efforts  to  minimize  frame  loss  during  video  capture,  but  the  latter  were  adequate  for  performance  in 
this  project.  The  hard  disk  space  required  for  video  capture  depends  directly  upon  the  amount  of  video  to  be 
captured.  The  .AVI  files  required  approximately  I MB  per  second  of  video  captured.  Digital  .AVI  files  are 
about  10  times  the  size  of  compressed  .mpg  files,  and  space  for  both  is  needed  during  the  compression  process. 
In  the  configuration  we  used  in  the  capture  process,  the  resulting  .AVI  files  were  uncompressed. 

We  were  able  to  store  the  43  video  files  of  Managing  Asthma  in  both  formats  with  space  to  spare.  A 
Gateway  200MHz  Pentium  Pro  processor  in  another  machine  also  proved  to  be  satisfactory  for  video  capture 
and  compression,  but  a Gateway  lOOMHz  Pentium  did  not.  There  was  a significant  loss  of  frames  in  the 
capture  process  when  a Gateway  lOOMHz  PC  with  64MB  of  memory  was  used. 

The  compact  disc  recorder  (CD-R)  in  the  PC  was  the  latest  Sony  model  with  bundled  Adaptec  Direct 
CD  software.  We  upgraded  the  software  to  Adaptec  Easy-CD  Pro  95.  There  were  a number  of  problems  with 
this  hard  ware/soft  ware  combination.  We  could,  with  enough  time  and  effort,  make  an  acceptable  CD-ROM 
containing  video  and  ToolBook  II  Instructor  program  files,  but  eventually  removed  the  Sony  from  our 
development  system.  We  settled  on  a Hewlett-Packard  4020i  CD-R  and  Adaptec  Easy  Creator  Deluxe  [SI 00]. 
Easy  CD  Creator  Deluxe  has  more  features  than  Easy-CD  Pro  95,  including  CD  duplication,  and  is  very 
reliable. 


Production 

The  procedures  for  capturing,  compressing,  and  editing  the  video  scenes  were  relatively  straight- 
forward, and  the  learning  curve  associated  with  hardware  and  software  described  above  was  not  particularly 
steep.  Adventures  at  the  low  end  of  high  tech  began  in  earnest  when  the  various  components  of  the 
development  system  were  combined  to  produce  the  new  CD-ROM  version  of  Managing  Asthma. 

ToolBook  11  Instructor  was  retained  for  authoring,  since  we  believed  that  using  it  would  minimize 
conversion  cost  and  conversion  time.  As  program  designers,  we  had  worked  with  versions  of  ToolBook  since 
Release  1 .53  and  were  familiar  with  its  idiosyncrasies,  or  so  we  thought.  Even  after  an  upgrade  from  version 
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6.0  to  version  6.1a,  there  was  a steep  learning  curve  for  incorporating  digital  video  in  an  MPEG  format  into 
Managing  Asthma.  It  was  necessary  to  make  significant  modifications  in  the  original  ToolBook  II  Instructor 
program  to  use  CD-ROM  rather  than  laser  disc  video. 

In  order  to  control  playback  of  the  video  segments,  the  ActiveMovie  Control  Object  extensions,  a 
feature  of  Window  95,  must  be  added  to  the  ToolBook  II  Instructor  environment.  These  extensions  enable  the 
developer  to  use  drag  and  drop  technology  to  insert,  position  and  size  the  window  in  which  the  video  is  to  be 
played.  The  next  step  is  to  set  the  ActiveMovie  properties.  Displaying  the  default  video  controls  (play,  pause, 
stop  rewind  and  fast  forward)  can  be  enabled  or  disabled  as  desired.  We  developed  a simplified  set  of 
customized  controls  for  video  control.  Nearly  50  parameters,  such  as  size,  window  placement,  auto-play  and 
auto-rewind,  are  configurable. 

In  ToolBook  II  Instructor,  the  properties  can  also  be  set  using  scripts.  The  ToolBook  II  Instructor 
search  and  replace  feature  makes  this  method  easier  to  use  when  making  a comprehensive  change  in  parameters 
such  as  the  video  display  window  size.  For  example,  the  sample  script  displayed  below  sets  the  display  area, 
file  to  be  played,  disables  auto  rewind,  plays  the  video  segment  and  stops  at  the  end  of  the  segment  with  the  last 
video  frame  displayed. 

set  bounds  of  ActiveMovie  ’’Video”  to  2000,700,8000,5000 

set  extMovieWindowSize  of  ActiveMovie  ’’Video”  of  page  ’’videointro”  to  3 

set  extFileName  of  ActiveMovie  ’’Video”  of  page  ’’videointro”  to  / vDrive&”:\asthmaMpg\astintro.mpg” 
set  extAutoRewind  of  ActiveMovie  "Video”  of  page  "videointro”  to  false 
get  extRunO  of  ActiveMovie  "Video" 

While  the  process  is  not  difficult  to  describe,  it  did  not  operate  very  smoothly  in  practice.  Asymetrix 
does  not  have  “native  supporf  ’ for  playing  .MPG  video  files;  that  is,  ActiveMovie  is  an  “add-in,”  not  part  of  the 
ToolBook  II  Instructor  environment.  ActiveMovie  is  not  documented  or  supported  by  Asymetrix  and 
minimally  documented  by  Microsoft.  ActiveMovie  properties  such  as  screen  size  and  video  file  name  that  were 
configured  with  object  control  extensions  would  not  remain  stable  in  ToolBook  II  Instructor.  We  found  it 
beneficial  to  save  the  file  after  every  change.  The  “drag  and  drop”  function  to  set  the  screen  size  was  not 
operational;  changes  were  made  by  adjusting  parameters  in  the  ToolBook  II  Instructor  script.  Some 
ActiveMovie  parameters  were  operational  in  the  script;  others  did  not  seem  to  be. 

Naming  conventions  for  .AVI  and  .MPG  files  were  limited  to  the  8.3  standard  because  of  ActiveMovie 
and  ToolBook  II  Instructor  constraints;  file  names  were  also  case-sensitive.  Thorough  documentation,  formerly 
often  an  afterthought,  became  a necessity.  Every  programming  session  was  indeed  an  adventure.  Eventually 
we  were  able  to  determine  the  optimal,  if  not  ideal,  relationship  between  setting  parameters  through 
ActiveMovie  object  control  and  ToolBook  II  Instructor  scripting. 


Packaging 

Once  the  program  was  converted  we  needed  to  package  the  application  in  a way  that  the  end  user  could 
easily  install  the  software.  The  AutoPackager  utilities  supplied  by  Asymetrix  provide  the  ability  to  distribute  a 
professional  looking  installation  program  for  a ToolBook  II  Instructor  application.  This  program  gives  the 
developer  the  option  to  select  the  target  distribution  media  of  CD-ROM,  floppy  disk  or  Internet  and  set  default 
configuration  selections  for  the  installation  of  the  program.  One  of  the  more  useful  options  is  the  ability  to  have 
the  installation  process  copy  the  assortment  of  ToolBook  files  to  the  hard  disk  and  allow  the  large  video  files  to 
remain  on  the  CD-ROM.  Unfortunately  this  option  does  not  support  add-ins  such  as  the  ActiveMovie 
extensions  we  used.  Video  files  thus  had  to  be  added  to  the  distribution  disc  separately.  We  did  use  the 
packaging  option  for  storing  the  audio  overlays  used  in  Managing  Asthma. 


Further  development 

We  have  decided  to  reduce  the  amount  of  adventure  in  further  development  by  shifting  to  Visual  Basic 
as  an  alternative  to  ToolBook  II  Instructor,  even  though  we  have  been  very  pleased  with  what  we  have  been 


\OS^ 


able  to  do  with  ToolBook  over  the  years.  Visual  Basic  will  provide  a more  stable  environment  for 
programming  in  general  and  ActiveMovie  specifically.  ToolBook  II  Instructor  does  not  have  support  in  the 
settings  in  which  we  work;  Visual  Basic  is  fully  supported.  Furthermore,  extensive  online  support,  user  groups 
and  many  third  party  add-in  modules  are  available  for  Visual  Basic.  Special  promotions  for  Visual  Basic  are 
often  to  be  found  in  academic  settings;  these  promotions  may  lower  the  cost  by  as  much  as  75%  from  advertised 
prices. 

The  CD-ROM  version  of  the  program  will  receive  the  same  type  of  use  in  “real  world”  settings  as  did 
the  laser  disc  version.  We  will  be  determining  whether  the  projected  savings  of  logistical  and  maintenance  time 
and  money  can  be  realized.  Of  most  importance  will  be  the  response  of  health  care  personnel  to  this  new 
delivery  system. 
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Abstract:  Struggles  over  educational  access  to  new  communications  technologies 

in  the  age  of  US  radio  and  television  provide  insights  into  current  struggles  of 
educational  access  to  the  “information  superhighway”.  Findings  from  an  historical 
analysis  of  these  earlier  periods  show  that  educational  access  to  new  communications 
technologies  is  often  constrained  or  marginalized  in  4 primary  ways:  1)  definitional 
distinctions  in  law  and  policy,  2)  high  costs  and  technical  operating  standards,  3) 
technical  ghettoization  in  low-power,  low-reach  technologies,  and  4)  mid-stream 
legal  and  policy  modifications  including  challenges  to  regulatory  jurisdiction. 
Similar  methods  of  constraint  are  identified  in  the  current  context  of  universal 
service  e-rate  discounts  for  K-12  access  to  the  NIL  It  is  suggested  that  educators 
should  be  excited  about  Nil  service  discounts,  but  wary  of  regulatory  constraints. 


Introduction^ 

The  information  age  is  upon  us  and  the  Clinton  Administration’s  1993  policy  mandate  to  connect  all 
schools  to  the  National  Information  Infrastructure  (Nil)  by  the  year  2000  has  placed  the  education  community  at 
center  (Information  Infrastructure  Task  Force  1993).  Such  connections  are  not  only  necessary  for  educational 
delivery  and  achievement,  it  is  claimed,  but  also  for  building  a publicly  accessible  advanced 
telecommunications  network  for  the  nation.  The  Telecommunications  Act  of  1996  is  the  legislation  recently 
enacted  to  usher  in  the  promise  of  this  new  age.  Its  universal  service  “e-  rate”  provisions  establish  discounts  and 
encourage  public/private  partnerships  so  that  K-12  schools,  public  libraries,  and  rural  health  care  systems  across 
the  nation  can  defray  the  costs  of  accessing  advanced  telecommunications  services. 

But  should  educators  trust  the  promise  of  the  universal  service  e-rate  as  a way  to  ensure  their  access  to  the 
Nil?  There  can  be  no  doubt  that  the  Nil’s  advanced  telecommunications  networks  and  information  systems 
have  much  to  offer  our  educational,  local,  national,  and  international  communities.  Likewise  there  is  precedent 
in  US  communications  history  for  tempering  market  forces  by  regulating  for  the  public  interest.  If  we  consider 
the  history  of  how  radio  and  television  were  introduced  and  regulated  for  educational  access  and  use,  can  we 
find  insight  into  the  current  situation? 

In  this  paper,  I report  findings  from  an  historical  analysis  of  struggles  between  1912  and  March  1999  over 
terms  and  efforts  to  regulatorily  ensure  educational  access  to  radio,  television,  and  networked  computer 
technologies.  By  submitting  policy  documents  and  related  academic  and  popular  articles  to  methods  of  textual 
analysis  and  genealogical  discourse  analysis  (Foucault,  1984),  I found  that  education  is  consistently  constrained 
from  being  afforded  secure  and  autonomous  access  when  new  communications  technologies  are  introduced. 
Although  access  is  supported  rhetorically,  financially,  and  regulatorily  -both  by  government  and  the  private 
sector  - it  is  limited  regulatorily  through  1)  definitional  distinctions  in  law  and  policy,  2)  high  costs  and 


* For  the  sake  of  space,  I have  not  included  references  to  legislation  or  regulatory  decisions  that  can  be  found  using  the  title 
and  date  references  provided  in  the  text. 


technical  operating  standards,  3)  technical  marginalization  in  low-power,  low-reach  technologies,  and  4)  mid- 
stream legal  and  policy  modifications  including  challenges  to  regulatory  authority  of  the  FCC. 

I argue  we  are  currently  seeing  these  kinds  of  constraints  in  the  case  of  universal  service  e-rate  provisions. 
Thus,  my  analysis  would  seem  to  suggest  that  the  education  community  should  both  embrace  and  be  cautious 
about  the  promise  of  the  NIL  For  while  advanced  communications  networks  offer  possible  solutions  for 
problems  relating  to  providing  K-12  education  to  a highly  diverse  national  population,  as  in  the  past,  they  also 
threaten  to  either  impose  significant  economic  and  organizational  hardship  on  schools  and  their  communities,  or 
saddle  them  with  unusable  or  marginal  technological  capabilities  and  unfulfilled  hopes. 

Limiting  Educational  Access  to  Communications:  The  Case  of  Radio 

New  technologies  create  the  need  for  new  regulations.  When  radio  was  introduced,  for  example,  US 
legislators  felt  it  necessary  to  regulate  it  so  that  dedicated  frequencies  could  be  set-aside  for  defense  uses.  In  the 
US’s  first  communications  regulation  — the  Communications  Act  of  1912  — a system  of  licensure  was 
established  that  allocated  particular  spectrum  frequencies  to  particular  licensees.  These  early  laws  and 
regulations  were  constitutive  in  education’s  marginalization  in  communications. 

In  the  first  place,  the  1912  Act,  defined  licensees  as  “persons,  companies,  corporations”,  and  successive 
amendments  defined  them  as  “individuals,  firms,  or  corporations”  (1927),  and  as  “persons,  corporations,  and 
amateur  stations”  (1934),  Nowhere  did  the  language  of  these  early  laws  deem  public  educational  institutions 
viable  licensees,  unless  tied  to  private  or  government  research  under  the  1912  provision  for  “conducting 
experiments  for  the  development  of  the  science  of  radio  communication”. 

Secondly,  The  1927  Communications  Act  established  costly  licensing  fees  and  required  that  stations 
operate  with  stronger  signals,  longer  hours,  facilities  built  to  code,  etc.  In  1929,  General  Order  40  re-allocated 
spectrum  and  placed  educational  stations  in  low-power,  low-reach  frequencies  which,  although  they  were  more 
expensive  to  operate,  were  more  difficult  to  receive.  These  legislative  and  technical  decisions  had  a chilling 
effect  on  educational  uses  of  new  radio  technologies.  Without  the  deep  pockets  or  the  technical  acumen  of  their 
commercial  counterparts,  educators  were  often  unable  to  meet  the  new,  significantly  more  taxing  financial  and 
technical  commitments  of  the  late  1 920s.  And,  because  their  stations  were  low-power  and  low-reach,  they  had 
trouble  developing  markets.  By  the  time  the  Communications  Act  of 1924  was  passed,  reports  like  the  following 
were  not  uncommon: 

Professor  Jerome  Davis  of  the  Yale  University  Divinity  School  presented... the  results 
achieved  by  the  present  radio  law.  When  it  went  into  effect  there  were  ninety-four  educational 
institutions  engaged  in  broadcasting;  today  difficulties  put  in  their  way  are  so  great  that  many, 
including  Columbia  University  and  the  Massachusetts  State  Department  of  Education,  have 
given  up  entirely,  while  many  of  the  others  are  rendered  almost  useless  by  the  policies  of  the 
government  (The  Nation  1934,  p,  201), 

Despite  these  developments,  there  was  some  financial  and  institutional  support  for  educational 
broadcasting.  For  example,  the  Office  of  Education  for  established  a Radio  Section  in  1 929  to  “initiate  and 
assist  with  research  studies  of  radio”  for  adult  education  (Saettler  1990,  p.213).  The  Payne  Fund  continued  its 
long-standing  support  of  lobbying  efforts  to  set-aside  dedicated  non-commercial  channels  by  coalitions  like  the 
National  Committee  on  Educational  Radio  (NCER)  (McChesney  1993).  President  Roosevelt  allocated  $75,000 
to  the  Office  of  Education  in  1935  to  develop  educational  programming  (Arafeh  1 998).  And,  radio  corporations 
offered  free  air  time  for  educational  broadcasting  as  part  of  their  “public  interest”  mandate  (Studebaker  1936)^. 

While  each  of  these  efforts  generously  contributed  to  the  cause  of  educational  radio  in  their  different  ways, 
commercial  stations  had  a particularly  strong  effect.  Not  only  were  their  lobbies  strong,  their  willingness  to 
broadcast  educational  programming  for  free  made  it  difficult  to  justify  the  more  costly  alternative  of  educators 
owning,  making,  and  running  their  own  stations  and  shows. 

By  1936,  then,  the  question  was  more  one  of  how  education  could  contribute  to  commercial  broadcasting’s 
“public  interest”  mandate  than  how  it  could  gain  dedicated  station  access  and  programming  control.  This  quote 
by  Commissioner  Studebaker  from  the  1 936  National  Conference  on  Educational  Broadcasting  is  instructive. 


^ US  Commissioner  of  Education  J.W.  Studebaker  (1936),  noted  that  “the  commercial  sale  value  of  the  [radio]  time  used  by 
the  Office  of  Education  during  the  last  six  months  has  been  estimated  at  several  hundred  thousand  dollars.  That  is  an 
impressive  amount  of  money... And  yet  these  [private  radio]  companies  provide  free  time  on  the  air  to  the  Office  of 
Education  and  to  other  educational  and  non-commercial  organizations”  (p.3). 
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particularly  in  light  of  current  efforts  at  public/private  partnerships  for  advancing  the  Nil: 

Suppose  we  state  the  problem  this  way:  How  can  public  enterprise  use  a utility  which  is 
privately  controlled?  Let  me  repeat  that:  How  can  public  institutions  (educational 
organizations)  use  the  publicly  owned  airwaves  which  are  controlled  by  private  enterprises 
under  federal  licenses?  Is  this  the  basic  question  that  confronts  us?...[T]o  gain  our  objectives, 
educational  broadcasting  requires  that  we  pool  our  knowledge  of  educational  purposes  and  of 
planned  instruction  with  the  practical  experience  of  broadcasters  schooled  in  the  technical 
complexities  of  radio... Let  us  - educators  and  broadcasters  - go  forward  together  (Studebaker 
1936,  pp.4-5). 

Educators  and  broadcasters  did  go  forward  - often  together  - but  push-me/pull-you  frequency  set-asides 
and  spectrum  re-allocations  kept  educators  off  balance.  The  25  AM  channels  reserved  by  the  FCC  for  in- 
school educational  use  were  later  transferred  to  FM  spectrum  where  limitations  in  bandwidth  left  only  5 low- 
power  channels  intact  (Sterling  and  Kitross  1990).  By  the  end  of  1941,  there  were  only  2 FM  education-run 
stations  which  grew  in  1945  to  6,  and  then  to  90  in  1952,  Frequency  relocation  to  the  88-108MHz  bandwidth, 
and  changes  in  FCC  guidelines  that  allowed  education-run  stations  to  broadcast  with  only  1 0 watts  of  power 
(5-10  mile  reach)  contributed  to  this  upswing  in  educational  radio  stations.  While  the  positive  impact  of  these 
regulatory  technical  changes  can  be  seen  by  the  dramatic  increase  in  stations  between  1941  and  1952, 
educational  aspirations  were  likewise  limited  by  low-power  stations  and  regulatory  uncertainty.  This  is  the 
fourth  method  of  educational  constraint  to  which  I want  to  draw  attention;  4)  mid-stream  modifications  and 
revisions  in  policy,  legislation,  or  administration  that  either  frustrate  or  erode  legitimate  access  and  positive 
activity. 

These  four  specific  regulatory  techniques  helped  marginalize  the  education  community’s  radio  use,  often 
resulting  in  reliance  on,  and  collaboration  with,  commercial  enterprise.  Federal  and  state  governments  were 
pleased  with  such  arrangements  because  it  relieved  them  of  funding  and  staffing  a more  robust  educational 
radio  sector.  As  well,  the  primary  burden  of  communications  infrastructure  development  was  placed  on 
private  industry  which  benefited  because  it  was  assured  control  of  the  medium  and  income  from  govemment- 
and  foundation-subsidized  educational  programming.  The  education  community,  however,  was  placed  in  a role 
of  dependence.  As  the  promise  of  television  loomed  on  the  horizon,  little  would  change. 

Limiting  Educational  Access  to  Communications:  The  Case  of  Educational  Television 

The  mid-40s  increase  in  educational  radio  channels  was  partly  due  to  regulatory  and  technical 
modifications  required  by  the  advent  of  television  in  which  education  got  a poor  start.  The  War  Production 
Board  imposed  a station  construction  freeze  between  1942  and  1945,  and  the  FCC  a debilitating  license  freeze 
in  1948  that  lasted  6 years.  Education  was  unable  to  get  its  foot  in  the  television  door  during  these  early  boom 
years. 

During  1950  and  1951  of  this  of  this  hiatus,  the  FCC  held  hearings  to  determine  the  new  regulatory  scheme 
for  both  radio  and  television  channel  assignments,  as  well  as  for  color  television  standards  and  educational 
television  set-asides  (Saettler  1968).  A combination  of  lobbying  efforts  by  the  Joint  Council  on  Educational 
Television  (JCET),  research  provided  by  the  National  Association  of  Educational  Broadcasters  (NAEB),  and 
the  active  participation  of  FCC  Commissioner  Frieda  Hennock  did  result  in  242  channels  (most  in  the  UHF 
band)  being  set-aside  in  1952  for  the  education  community.  Educational  broadcasters  were  encouraged  by  this 
show  of  FCC  support  as  well  as  the  generous  sponsorship  for  program  development  provided  by  the  Ford 
Foundation’s  Fund for  the  Advancement  of  Education  (TFAE)  (Brinson  1998,  DeVaney  1990,  Saettler  1990). 

As  in  the  case  of  radio,  however,  channels  were  located  in  low-power  UHF  frequencies  that  were  more 
expensive  to  operate  and  could  not  be  picked  up  by  all  television  receivers.  Four  years  after  UHF  ETV  channel 
WOSU  broadcasting  in  Columbus  Ohio,  for  example,  85%  of  the  sets  in  its  market  were  still  unable  to  receive 
its  signal  (Baughman  1985).  This  devastated  educational  broadcasters  who,  now  subject  to  more  stringent 
ratings  measurement  schemes,  were  hard-pressed  to  Justify  station  ownership  and  operation  for  such  small 
markets.  Even  with  the  success  of  the  1952  channel  reservations  then,  educational  broadcasters  once  again 
faced  structural  barriers  and  technical  marginalization  that  hindered  their  ability  to  provide  consistent,  wide- 
reaching,  and  receivable  educational  programming  to  the  schools  and  communities  in  their  particular  areas. 

Educators’  struggles  for  more  favorable  circumstances  took  a distinctly  different  turn  in  1962  when 
Congress  passed  the  ETV  Facilities  Act.  Partly  a response  to  an  increasingly  uncontrollable  commercial  sector, 
and  partly  an  attempt  to  bolster  the  goals  of  Johnson’s  Great  Society  program,  this  Act  initiated  federal  funding 


and  regulatory  support  for  public  broadcasting^.  As  a result  of  the  Act,  ETV  channels  increased  from  52  in  1961 
to  1 14  in  1966  and  then  to  252  in  1972  (Sterling  and  Kitross  1990)  - still  in  the  low-power  frequencies. 

Between  1962  and  1967,  educators  sought  programming  from  the  Ford  Foundation’s  National  Educational 
Television  Center  (NET)'^  and  settled  in  to  one  of  the  most  robust,  autonomous  educational  broadcasting 
moments  in  US  history.  Stations  were  in  place,  funding  was  available,  and  programming  was  on  the  rise.  When 
the  Johnson  Administration’s  appointed  Carnegie  Commission’s  recommendations  on  extending  the  ETV 
Facilities  Act  were  enacted  in  the  Public  Broadcasting  Act  in  1967,  however,  such  halcyon  days  were  over. 

This  Act  not  only  passed  control  and  funding  of  educational  broadcasting  to  a publicly-funded  but  semi- 
private Corporation  for  Public  Broadcasting  (CPB),  it  also  defmitionally  changed  the  educational  broadcasting 
charge  from  one  of  either  “educational”  or  “instructional”  programming  (i.e.  ETV  or  ITV)  to  one  of  “public” 
programming.  Within  this  definitional  shift,  ETV’s  instructional  mission  - which  included  distance  initiatives 
up  to  this  point  - were  excluded  from  public  broadcasting’s  mission.  “Public”  broadcasting  intoned  broadly 
“cultural”  programming.  Once  again,  definitional  requirements  resulted  in  a marginalization  that  not  only 
barred  educators  from  being  able  to  pursue  what  they  had  perceived  to  be  their  primary  program  content  domain 
but,  this  time,  also  resulted  in  a loss  of  their  stations  to  a semi-private  government  agency.  This  left  the  lower- 
power,  lower-reach  stations  - particularly  the  closed-circuit,  microwave-distributed  Instructional  Television 
Fixed  Service  (ITFS)  systems  -for  instructional  use.  In  the  best  cases,  municipalities  and  states  have  developed 
local  and  regional  instructional  networks  from  these  facilities.  However  and  again,  here  was  another  case  where 
education  was  being  defmitionally  and  technically  marginalized  while  simultaneously  being  kept  off-balance  by 
uncertainty  in  policy  and' legislative  determinations. 

Cable  broadband  technology  produced  a similar  scenario.  Early  days  of  its  regulation  held  educational 
promise  as  the  FCC  encouraged  cable  systems  to  provide  “public  access”  (i.e.  channels  designated  for 
autonomous  public,  educational,  and  government  use)  through  an  informal  recommendation  in  1969.  In  1972, 
the  FCC  mandated  that  cable  systems  serving  areas  with  3500  or  more  subscribers  provide  PEG  channels  and 
educators  found  yet  another  opportunity  to  utilize  new  communications  infrastructure  for  their  curricular  and 
pedagogical  purposes  (Engelman,  1 990). 

Unfortunately,  the  Supreme  Court  ruled  that  the  FCC  was  not  authorized  to  mandate  PEG  access  in  FCC  v. 
Midwest  Video  Corporation  in  1979.  This,  in  combination  with  the  Cable  Communications  Policy  Act  of 
I984's  statutory  location  of  jurisdiction  for  negotiating  PEG  set-asides  in  municipalities,  had  two  effects.  First, 
it  reduced  the  likelihood  of  establishing  PEG  channels  and,  second,  it  continued  the  trend  of  technical 
marginalization  by  virtue  of  PEG  stations  being  available  only  to  cable  subscribers  within  a small 
geographically-bound  system  (Arafeh  1992).  What  the  case  of  cable  shows,  however,  is  one  more  form  of  mid- 
stream modification  to  laws  and  regulation:  annulment  of  regulatory  determinations  through  formal  challenges 
to  the  jurisdiction  and  authority  of  the  regulating  body  - the  FCC. 

To  date,  schools  and  districts  have  had  varying  degrees  of  success  using  television  for  education.  This  is 
partly  because  they  have  had  trouble  gaining  access  to  production  equipment  and  facilities,  partly  because 
regulations  continue  to  change,  and  partly  because  they  do  not  have  the  financial  and  human  resources.  Some 
states  and  regions  have  been  able  to  develop  quite  robust  educational  networks  (i.e.  Iowa,  South  Carolina,  etc.). 
This  outcome  has  been  facilitated  in  more  recent  days  by  regulatory  initiatives  like  the  StarSchools  program.  It 
is  safe  to  say,  however,  that  educational  broadcasting  has,  and  continues  to  be,  difficult  for  schools  to  undertake 
- especially  elementary  and  secondary  institutions. 

Conclusion:  Lessons  from  the  Past  and  the  Promise  of  Universal  Service 

Combining  the  informational  and  computational  power  of  computers  with  the  fast  and  wide-ranging 
distributive  power  of  telecommunications  networks  presents  an  unprecedented  opportunity  for  commercial, 
interpersonal,  and  educational  interaction.  Or  does  it?  Do  these  limited  insights  on  the  past  help  provide 
perspective  on  the  present  and  future? 

It  would  seem  so,  for  while  networked  communications  hold  possibility  for  education,  so  did  radio  and 


^ The  Act  also  mandated  that  all  television  sets  be  manufactured  with  UHF  and  VHF  reception  capability. 

^ Originally  the  National  Educational  Television  and  Radio  Center,  which  was  founded  in  Ann  Arbor,  Michigan  in  1954.  A 
collective  that  developed  educational  programming,  it  changed  its  name  in  1963  to  the  National  Educational  Television 
Center  (NET)  and  received  funding  from  the  Ford  Foundation  (Sterling  and  Kitross  1 990). 
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television.  Once  again,  K-12  access  to  new  communications  technologies  is  controversial;  and  the  four 
regulatory  constraints  that  I have  outlined  above  remain  hard  at  work. 

The  universal  service  e-rate  program  has  been  very  successful  in  some  respects.  Over  $1.4  billion  dollars  in 
discounts  have  been  allocated  to  94%  of  the  schools  that  applied,  and  54%  and  48%  increase  respectively  in 
schools  and  classrooms  from  1994  to  1998  has  been  reported.  At  this  rate,  connecting  all  K-12  schools  to  the 
Nil  by  the  year  2000  may  be  an  attainable  goal,  that  already  seems  to  be  decreasing  the  positive  correlation  of 
school  connection  to  the  Nil  with  race  (Digital  Beat,  1 999). 

The  e-rate  program  is  also  being  contested,  however.  The  Telecommunications  Act  of  1996  has  deemed  K- 
12  schools  as  having  no  legitimate  interest  in  networked  telecommunications  other  than  as  service  end-users 
needing  service  discounts  through  an  e-rate  - a definitional  distinction  that  supports  the  historical  information 
provided  above.  And,  as  in  previous  communications  scenarios,  schools  may  be  marginalized,  or  even 
ghettoized,  by  their  less-than-state-of-the-art  technology  and,  ostensibly,  increasingly  low-technology 
connections.  Already  there  are  password-  and  technically-driven  “levels”  to  the  Internet  which  distinguish 
commercial  and  government  users  from  public  and  public  institutional  users  like  k-12  schools.  Legislation  like 
the  Next  Generation  Internet  Research  Act  of  1998  and  the  Technology  Administration  Act  of  1998  are  already 
trying  to  establish  the  I-NET,  a value-added  Internet,  which  again  threatens  to  create  and  justify  new 
technologies  and  capabilities  outside  of  K-12  education’s  reach.  Not  every  Nil  application  is  appropriate  for 
elementary  and  secondary  schools.  Yet,  as  our  conceptions  of  what  these  technologies  can  and  should  do  are 
formed  in  early  research,  development,  and  implementation  phases;  educational  and  public  aspirations  should 
not  necessarily  take  a back  seat.  As  my  research  shows,  limits  to  sophisticated  technology  has  been  particularly 
damaging  for  explorations  of  innovative  ways  education  can  contribute  to  uses  of  new  communications  and,  in 
doing  so,  contribute  to  the  public  and  national  interest. 

Lastly,  there  have  been  sufficient  mid-stream  modifications  to  e-rate  legislation  and  its  administration  to 
keep  educators  off-balance  and  unable  to  plan  for  their  futures.  Telecommunications  service  providers  have 
filed  suits  claiming  that  the  financial  obligation  is  overly  burdensome  and  that  cost  models  are  unreasonable 
(c.f  ALA  WON  1999b).  Industiy  has  also  chosen  to  “pass  through”  their  universal  service  costs  to  customers’, 
creating  public  stir  and  political  anxiety  (Atlanta  Journal  and  Constitution  1998).  Some  federal  legislators  have 
challenged  the  FCC’s  authority  to  create  a non-profit,  quasi-public  Schools  and  Libraries  Corporation  (SLC)  to 
administer  the  funds  (Simons  1998),  as  well  as  the  FCC’s  authority  to  pursue  social  policy  goals  through  its 
regulatory  activities  (Broadcasting  and  Cable  1999).  These  regulatory  destabilizations  have  created  a stunting 
period  of  uncertainty,  chaos  and,  at  times,  paralysis  as  administrative  jurisdiction  continues  to  change  and  the 
FCC  walks  on  egg-shells  in  order  to  keep  legislators  from  dismantling  it.  Most  recently,  there  has  been 
proposed  legislation  to  require  that  schools  receiving  e-rate  discounts  use  filtering  software^,  and  that  the  e-rate 
program  be  “killed”  altogether  Such  opposition  to  supporting  educational  access  to  networked 
communications  will  inevitably  have  devastating  effects. 

Should  educators  trust  the  promise  of  universal  service?  This  history  suggests  that  they  probably  should 
not.  And  this  is  bearing  out  as  the  universal  service  e-rate  program  - and  the  schools,  students,  and  communities 
it  is  supposed  to  benefit  - is  close  to  drowning  through  regulatory  measures  once  again.  There  is  the  potential 
that  this  will  be  just  another  unhappy  chapter  in  education’s  long  history  of  attempting  to  benefit  from,  and 
positively  use,  communications  technologies  in  the  course  of  its  work  of  educating  the  nation’s  children. 
However,  it  is  my  hope  that  the  insights  regarding  K-12  education’s  marginalization  relative  to  new 
communications  technologies  provided  in  this  paper  urges  education  advocates  to  keep  their  eyes  on  high-level 
policy,  legislative,  and  regulatory  activity  - particularly  when  new  technologies  emerge.  Only  in  this  way  will 
the  educational  community  - and  the  public-at-large  - gain  assured,  robust  access  to  new  communications 
technologies  now,  and  in  the  future. 
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Abstract:  The  positive  response  to  a tutorial  video  tape  developed  for  an  engineering  graphics 
class  ultimately  lead  to  the  development  of  a comprehensive  multimedia  instructional  CD- 
ROM/web  page.  The  project  in  its  current  state  consists  of  an  integrated  web  site  with  links  to 
hours  of  tutorial  movies,  lecture  presentations  of  all  class  lectures,  web-based  games  that  reinforce 
significant  course  topics,  and  a series  of  interactive  web-based  quizzes.  The  through  discussion  of 
the  specific  methods  and  tools  used  to  develop  each  component  of  the  project  is  presented 
including  a discussion  of  the  many  challenges  encountered.  A realistic  picture  is  painted  of  how 
the  project  both  positively  and  negatively  has  impacted  the  course  and  individual  students.  A 
detailed  estimation  of  the  overall  time  investment  made  in  the  development  of  the  project  provides 
a reference  for  those  considering  undertaking  a similar  project.  The  overall  outcome  of  the  project 
was  positive  and  has  led  to  an  expansion  of  the  project  to  include  the  offering  of  an  on-line  graphics 
course  and  the  development  of  similar  work  for  other  engineering  courses. 


Background 

Engineering  Graphics  is  an  introductory  engineering  course  that  teaches  the  fundamentals  of  graphical 
communication  and  how  to  use  a specific  computer  aided  design  drafting  (CADD)  software  package.  The  course  is 
often  the  first  engineering  course  that  students  take  and  many  base  their  decision  about  their  future  as  an  engineering 
student  on  their  experience  in  this  course.  A number  of  area  high  school  students  also  take  the  graphics  course  each 
semester.  A positive  experience  in  this  course  commits  students  to  the  engineering  program  and  motivates  them 
toward  completion  of  their  degree. 

Running  counter  to  the  objective  of  providing  a positive  experience  in  the  engineering  graphics  course  are 
several  realities  faced  by  all  academic  institutions.  Economics  dictate  high  student  to  teacher  ratios  even  in  a 
laboratoiy  setting.  Laboratory  assistants  do  not  have  the  same  level  of  training  or  motivation  as  the  professor. 
Students  enter  each  course  with  varying  backgrounds,  learning  abilities,  and  learning  styles.  Such  realities  often 
lead  to  a course  that  caters  to  the  average  student  leaving  slower  students  confused  and  brighter  students  bored. 

In  an  attempt  to  provide  a positive  experience  in  engineering  graphics  for  all  students  while  addressing  the 
issue  of  individual  student  needs  a video  taped  tutorial  session  was  developed.  The  tutorial  was  a compilation  of 
staged  individual  tutorial  sessions  that  addressed  the  most  common  problems  which  students  encountered  while 
completing  the  homework.  The  feedback  from  the  experiment  was  very  positive  and  required  only  a few  hours  of 
staged  tutoring.  One  criticism  of  the  video  tutoring,  however,  was  the  separation  of  the  tutoring  and  application.  To 
address  this  student  concern  a screen  capture  program  was  used  to  record  screen  activity  and  audio  for  the  staged 
tutorial  sessions.  The  reception  of  these  tutorials  which  could  be  viewed  in  the  computer  lab  and  run  simultaneously 
with  the  CADD  software  was  extremely  positive.  Encouraged  by  the  results,  the  tutorial  movies  became  the  first 
step  in  the  development  of  a comprehensive  multimedia  instructional  CD/ web  page.  The  project  to  date  includes 
hours  of  tutorial  movies,  lecture  presentations  of  all  class  lectures  for  the  semester,  web  based  games  to  reinforce 
significant  course  topics,  and  a series  of  interactive  web  based  quizzes  that  prepare  students  for  class  exams.  Similar 
multimedia  teaching  projects  have  been  developed  for  other  courses  and  disciplines  such  as  engineering  (Haugsjaa 
& Woolf,  1996,  Hill  et  al.,  1998,  Reed  & AQeh,  1998,  Suni  & Ross,  1997),  mathematics  (Antchev  et  al.,  1996),  and 
computer  science  (Marxhal  & Hurley,  1996).  The  engineering  graphics  project  has  been  successful  to  the  point 
where  students  rarely  seek  individual  tutoring  during  lab  and  almost  never  need  help  outside  of  lab.  The  CD/web 
page  accommodates  the  different  learning  styles  and  abilities  of  students  in  the  class  by  providing  a wide  range  of 


instructional  material  (Carver  et  al.,  1996,  Ellis,  1996).  What  was  only  partially  accomplished  in  the  lab  with  the 
constant  presence  of  an  instructor  and  lab  assistant  is  now  accomplished  with  the  CD  and  very  limited  lab  assistance. 
The  final  stage  of  development  will  be  to  include  a multimedia  presentation  of  the  lectures  in  the  project.  The  entire 
course,  lab  and  lecture,  could  then  be  offered  as  on  on-line  self-paced  course  with  hours  of  interactive  tutorial  and 
review  material  that  would  give  students  a positive  experience  in  engineering. 


Methods  and  Tools 

The  selection  of  tools  used  in  the  development  of  the  CD  was  initially  based  on  the  availability  of  and 
familiarity  with  particular  software  titles.  As  the  project  developed  and  certain  obstacles  were  encountered  a greater 
breadth  of  software  was  considered.  The  information  contained  on  the  CD  project  is  accessed  through  a series  of 
linked  HTML  pages  highlighted  with  animated  GIFS  created  in  AutoCAD  and  AVI  clips  obtained  with  a video 
capture  card.  The  lecture  presentations  are  accessed  through  links  to  Microsoft  PowerPoint  presentations  (Figure  1). 
The  laboratory  tutorial  material  is  presented  using  Lotus  Screen  Cam  movies.  The  games  and  quizzes  are  HTML 
files  that  use  simple  Java  Script  1.2  code.  The  information  was  created  and  compiled  on  a 200mHz  Pentium  and 
then  burned  onto  recordable  CDs  and  uploaded  to  a public  directory  on  a Unix  based  server.  The  entire  project 
requires  slightly  more  than  300MB,  the  Screen  Cam  movies  taking  the  bulk  of  the  disk  space.  The  development  of 
each  element  of  the  project  was  affected  by  the  capabilities  of  the  selected  hardware  and  software.  A prior 
understanding  of  the  limitations  and  capabilities  of  hardware  and  software  would  have  averted  many  of  the  obstacles 
encountered. 


Engineering  Graphics  Lecture  Notes 


1.  Introductjon 

2.  The  Design  IVocess 

3.  MCultiview  Sketches 

4.  Pjctorial  Sketches 

5.  Geometric  Construction 

6.  Multiview  Drawings 

7.  Kctoiials  and  Multiviews  Using  CAD 

8.  Auxiliary  Views 

9.  Section  Views 

10.  Dimensioning 

1 1.  Tolerancing 

12.  Three  Dimensional  Drawings 
.13.  Solid  Modeling 

14.  Graphic  sTVe  sent  ations 

15.  Animation 


Return  to  Engine ciing  Graphics  Home  Page 


Offset  Section 


♦ Tlie  internal  featm’es 
of  many  pait  can  not 
be  shown  using  a 
single  straight  cut  to 
create  the  sectioned 
drawing 

♦ An  offset  section  is 
used  for  sucli  paits 


Figure  1:  Web  Navigation  Page  for  Links  to  Lecture  Presentations 


Lecture  presentations 

The  original  lecture  presentations  were  prepared  using  PowerPoint.  The  lecture  material  presented  on  the 
CD  is  simply  a copy  of  the  lecture  presentations  used  in  class.  The  essential  capabilities  of  the  presentation  software 
for  this  project  are:  the  ability  to  efficiently  embed  large  high  quality  line  drawings  and  images  into  slides;  the 
availability  of  a free  viewer  for  viewing  the  presentation;  and  simple  but  significant  navigation  capabilities  in  the 
viewer.  Some  limitations  in  the  software  that  were  encountered  were  the  inability  to  embed  animated  GIFS  or 
MPEG  clips  into  the  presentation  and  the  absence  of  a web  browser  plug-in  for  earlier  versions  of  browsers.  These 
limitations  with  the  exception  of  animated  GIFS  have  been  addressed  with  the  latest  versions  of  software. 

The  presentation  software  allows  for  the  creation  of  HTML  pages  with  control  buttons  and  an  index  of  all 
slides  in  the  presentation.  However,  the  HTML  pages  scale  down  the  image  size  to  a selected  fixed  resolution  and 
use  a significant  portion  of  the  screen  for  slide  control.  In  addition,  the  HTML  pages  also  do  not  allow  for 
animation  of  slides.  Because  of  these  limitations  the  lectures  were  saved  in  the  original  presentation  format  and  with 
links  to  the  files  from  the  web  page.  This  requires  that  the  browser  is  set  up  to  launch  the  presentation  viewer 


program  based  on  the  MIME  type  if  the  file  is  on  a server  or  based  on  the  file  extension  if  the  file  is  on  the  CD.  An 
HTML  page  showing  how  to  setup  the  computer  and  the  web  browser  was  added  to  the  web  site  to  address  this 
issue. 


Tutorial  Movies 

The  tutorial  movies  were  created  with  Lotus  Smart  Suite  is  Lotus  Screen  Cam.  The  program  allows  for  real 
time  screen  capture  synchronized  with  audio.  This  type  of  screen  capture  program  allows  for  the  simple  creation  of 
tutorial  movies.  The  format  used  for  the  Engineering  Graphics  course  was  for  the  instructor  to  do  each  homework 
assignment  using  the  CAD  program  while  the  screen  capture  program  records  the  session.  The  accompanying  audio 
could  be  recorded  with  the  video  or  added  later.  The  finished  product  is  then  a set  of  narrated  tutorial  movies  that 
walk  the  student  through  each  assignment. 

The  usefulness  of  the  tutorial  movies  lies  in  the  fact  that  they  run  on  the  same  platform  as  the  program  that 
is  being  taught.  The  movie  can  be  paused  while  the  student  completes  a portion  of  the  assignment  and  then 
continued.  In  this  fashion  students  having  difficulty  with  the  homework  will  follow  the  tutorial  step  by  step  while 
more  advanced  students  will  watch  the  entire  movie  and  then  complete  the  assignment  on  their  own.  The  movies 
can  be  broken  into  major  segments  that  allow  the  user  to  jump  forward  or  back  to  the  start  of  each  segment.  The 
player  also  allows  the  user  to  fast  forward  through  each  segment.  The  movies  require  about  400-700kB/min 
(typically  60-70%  for  the  audio)  of  storage  for  a 600X800  screen  with  24  bit  color  and  8 bit  mono  sound  sampled  at 
1 IkHz  and  compressed  with  a 2: 1 compression  ratio.  The  size  of  a typical  10-minute  tutorial  would  be  about  4mB. 

There  are  several  notable  limitations  to  the  software  which  affect  the  use  and  creation  of  the  tutorial 
movies.  The  greatest  challenge  to  creating  the  tutorial  movies  is  to  create  a long  movie  that  does  not  contain  pauses 
or  errors  since  the  screen  capture  program  does  not  allow  for  editing  of  movies.  If  a long  pause  or  error  is  made 
during  the  recording  the  movie  must  be  redone.  Completing  a ten-minute  engineering  drawing  without  pauses  or 
errors  requires  planning  and  several  attempts.  One  way  to  address  this  is  to  record  the  tutorial  in  segments  saving 
each  as  a separate  movie.  The  segments  can  then  be  compiled  into  a single  tutorial  movie.  Therefore,  if  an  error  is 
made  in  the  last  segment,  only  the  last  portion  needs  to  be  re-recorded.  The  screen  capture  program  is  also  sensitive 
to  the  video  drivers  used.  When  using  the  CAD  program  a portion  of  the  cursor  often  disappears  and  occasionally 
the  program  would  occasionally  not  refresh  the  video  when  objects  were  added  or  erased.  The  use  of  some  drawing 
and  editing  commands  had  to  be  avoided  to  address  this  problem  which  was  restrictive. 

There  are  a two  drawbacks  with  the  program  related  to  playback.  First,  the  image  size  is  fixed  at  the  time 
of  recording  so  a low  level  of  resolution  must  be  used  to  accommodate  a variety  of  users.  Second,  the  player  does 
not  allow  the  user  to  rewind  the  video  except  to  the  start  of  each  segment.  This  can  be  frustrating  to  students  and  is 
their  primary  complaint  about  the  tutorials. 


Java  Script  Games  and  Quizzes 

A game  and  quiz  were  written  to  supplement  the  material  covered  in  each  lecture  and  associated  lab.  The 
quizzes  use  forms  with  radio  buttons  to  record  answers  to  True/False  and  multiple  choice  questions.  The  answers 
are  stored  in  an  array  and  compared  to  the  array  containing  the  correct  answers  when  the  user  selects  the  grade 
button.  The  number  of  questions  missed  are  displayed  using  an  alert  window.  The  students  can  then  change 
answers  until  they  receive  a perfect  score.  The  quizzes  give  instant  feedback  to  the  student,  are  interactive,  and  are 
very  simple  to  create.  Once  the  format  for  the  quiz  page  is  set  up  it  only  takes  a few  minutes  to  adapt  questions  for 
another  chapter.  The  JavaScript  quizzes  have  given  the  best  return  given  the  limited  investment  of  time. 

The  games  give  students  exposure  to  images  and  animations  that  were  created  in  the  CAD  program  that 
they  are  learning  to  use.  The  games  were  all  developed  using  JavaScript  code  to  manipulate  graphic  images  based 
on  user  input.  .The  simple  manipulation  of  images  based  on  user  input  allowed  for  the  development  of  a wide 
variety  of  games  which  each  effectively  teach  some  element  of  engineering  graphics.  The  first  and  simplest  game 
gives  an  palette  of  images  that  must  be  placed  on  a grid  in  the  proper  order  (as  in  a puzzle)  to  reveal  a computer 
generated  model  of  a utility  van  generated  in  CAD.  On  completion  of  the  puzzle  an  animated  GIF  is  displayed 
which  shows  the  vehicle  in  motion  and  a simulated  impact  with  an  automobile.  Another  game  allows  the  user  to 
manipulate  a 3-D  object  such  that  it  will  fit  correctly  into  a hole.  Two  other  games  use  image  maps  (Figure  2)  to 
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record  input  from  the  user  while  the  graphics  change  according  to  their  response.  With  the  development  of  5 
different  game  formats  a variety  of  other  teaching  games  could  easily  be  developed  using  the  same  formats. 

The  games  were  the  most  challenging  and  problematic  endeavor  on  the  project.  The  JavaScript  is  not  as 
straight  forward  as  the  quizzes  and  requires  more  programming  skill.  More  difficulties  were  also  encountered  with 
how  different  browsers  and  versions  of  browsers  handled  the  placement  of  graphics  images.  The  games  do  not  work 
with  Microsoft  Internet  Explorer  3.0  and  earlier  versions  or  with  early  versions  of  Netscape.  The  selection  and 
placement  of  images  with  the  mouse  causes  problems  if  the  user  tries  to  drag  the  image  into  place  or  moves  the 
mouse  while  pressing  the  select  button.  Feedback  from  some  students  that  the  games  don’t  work  reveals  that 
instructions  to  games  would  be  helpful.  The  level  of  success  achieved  for  someone  with  little  programming 
experience  and  no  JavaScript  experience  however  was  surprising.  It  can  be  assumed  that  an  experienced 
programmer  could  address  many  of  the  obstacles  encountered. 


Figure  2:  A JavaScript  Game  That  Uses  an  Image  Map  to  Aid  Students  With  Graphics  Visualization  Techniques 


How  the  Project  Has  Impacted  the  Course 

The  development  of  the  CD  has  impacted  the  course  in  several  significant  ways.  In  the  past  engineering 
graphics  was  taught  by  a number  of  faculty  and  the  lectures  were  presented  in  the  lab  to  relatively  small  class 
sections.  As  the  engineering  program  grew  it  became  necessary  to  combine  the  lab  sections  for  lecture  and  rely  on 
teaching  assistants  for  the  lab.  The  positive  impact  of  these  changes  relate  primarily  to  the  instructor.  By  combining 
sections  the  total  lecture  time  has  been  reduced  by  80%.  The  use  of  the  CD  in  the  lab  has  in  many  ways  eliminated 
the  need  for  the  instructor  in  the  lab.  On  the  negative  side  large  lectures  (70-80  students)  are  less  interactive  and 
more  impersonal.  The  laboratory  environment  has  also  become  less  interactive  and  more  impersonal  with  the 
exception  of  student  interaction  with  the  computer.  There  is  no  longer  a need  for  group  teaching  in  the  lab  as  each 
student  is  individually  tutored  on  the  computer.  Since  the  tutoring  uses  audio  the  students  wear  headphones  while  in 
lab,  which  reduces  interaction  among  students.  The  lab  has  transformed  to  become  an  environment  of  individual 
learning.  A semester  group  design  project  has  helped  to  reintroduce  interpersonal  communication  in  the  lab. 

As  the  students  are  provided  with  full  presentations  of  the  lecture  notes,  more  material  can  be  covered 
during  the  lecture  time.  Students  don’t  need  to  spend  as  much  time  copying  information  of  the  lecture  slides  since 
they  are  all  available  to  them  on  the  computer.  On  the  negative  side  some  student  forgo  the  lecture  for  an  individual 
reading  of  the  lecture  notes.  Rather  than  fight  this  trend,  work  toward  adding  audio  and  video  clips  to  the  lecture 
presentations  is  in  progress  giving  the  student  complete  access  to  all  the  lecture  material  on  their  own  schedule. 
Some  of  the  same  dynamics  occur  in  the  lab  where  students  work  on  their  own  time  rather  than  during  the  scheduled 
lab  time.  Quizzes  given  each  lab  period  motivate  students  to  attend  the  lab  where  they  feel  some  accountability  for 
turning  in  homework  and  keeping  caught  up  with  assignments. 

The  development  of  the  CD  has  also  impacted  the  future  of  the  course.  As  stated  previously  the  project  has 
made  it  possible  for  the  course  to  be  offered  on-line.  This  will  especially  be  helpful  for  area  high  school  students 
who  are  taking  the  course  to  have  full  access  to  the  course  material  from  their  individual  campuses.  Although  an  on- 
line course  offers  more  flexibility  for  students  it  restricts  the  flexibility  of  the  course  material.  Significant  changes 
to  the  course  that  require  redevelopment  of  the  material  are  avoided  because  of  the  investment  of  time  involved. 
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How  the  Project  Has  Impacted  the  Students 


The  greatest  positive  impact  that  the  project  has  had  on  students  is  they  now  use  their  time  more  efficiently. 
Labs  are  no  longer  filled  with  students  with  hands  raised  waiting  for  an  instructor  to  respond.  Instead,  students  work 
quietly  and  independently  on  their  homework  using  the  tutorial  movies  to  answer  their  questions.  Bright  students 
use  the  tutorials  as  a backup  when  they  encounter  a difficult  problem  and  quickly  complete  the  assignments.  Slower 
students  follow  the  tutorials  step  by  step  and  rarely  need  to  ask  for  assistance.  Both,  however,  learn  the  material  and 
complete  the  homework  assignments.  Students  who  don’t  finish  the  work  during  lab  can  return  any  time  to 
complete  the  assignments  while  still  receiving  instruction.  The  number  of  students  who  don’t  complete  the 
homework  or  complete  the  assignments  with  errors  has  dropped  noticeably.  This  has  led  to  a slight  improvement  in 
the  performance  on  exams. 

The  greatest  negative  impact  that  the  project  has  had  on  students  is  that  they  feel  less  connected  to  the 
instructor  and  the  engineering  program.  Contact  with  the  instructor  is  primarily  during  the  large  lecture  sessions, 
which  meet  weekly  for  50  minutes.  One  consequence  of  this  lack  of  connection  is  that  unmotivated  students  are  not 
personally  encouraged  and  eventually  drop  the  course.  Many  students  need  personal  encouragement  from  an 
instructor  who  knows  their  name  and  periodically  checks  on  their  performance  to  succeed  in  a course.  This  will 
likely  be  more  pronounced  with  an  on-line  course.  One  of  the  motivating  factors  for  offering  the  course  to  area  high 
school  students  is  do  draw  future  students  into  an  engineering  program  that  they  feel  connected  to.  This  aspect  of 
the  project  works  counter  to  that  goal. 

There  are  a number  of  other  notable  effects  of  the  project  on  the  students.  A student  who  falls  behind 
because  of  uncontrolled  circumstances  is  much  more  likely  to  catch  up  and  perform  well  in  the  course.  In  the  past 
this  required  a significant  effort  from  the  instructor  and  special  meeting  times  had  to  be  scheduled  with  the  student. 
The  visualization  tools  that  are  developed  by  the  student  as  they  interact  with  the  game  pages  are  invaluable  to  the 
student  and  difficult  to  teach  using  other  methods.  As  the  students  visualize  objects  repeatedly  in  the  games  with 
positive  and  negative  feedback  they  exercise  a developing  skill  that  would  otherwise  be  tedious  if  done  using  a 
printed  worksheet.  By  the  end  of  the  course  the  students  have  been  exposed  to  a breadth  of  examples  using 
interactive  graphics  to  teach  and  communicate.  The  students  seem  motivated  and  intrigued  by  use  of  engineering 
graphics  as  a powerful  communication  tool  whereas  in  the  past  they  had  a much  narrower  focus  for  the  application 
of  engineering  graphics. 


The  Cost  of  Development 

The  development  of  an  interactive  multimedia  teaching  tool  requires  a significant  investment  of  time. 
Assuming  that  the  developer  is  familiar  with  the  tools  needed  to  create  the  project,  which  is  significant  assumption, 
the  time  invested  in  a project  of  this  magnitude  would  be  on  the  order  of  several  hundred  hours.  Such  an  endeavor 
requires  an  extreme  commitment  of  resources  and  energy  far  above  those  required  for  the  development  of  a course 
taught  using  a traditional  approach.  The  costs  and  benefits  need  to  be  carefully  assessed  before  undertaking  the 
development  of  a project  of  this  magnitude  (Turner,  1997,  Ward  & Newlands,  1998).  The  62  tutorial  movies  on  the 
CD  were  each  recorded  several  times  and  cover  several  hours  of  instruction.  The  planning  and  recording  of  the 
tutorials  required  more  than  120  hours.  Putting  the  lecture  presentations  into  a computer  presentation  format 
required  approximately  4-5  hours  per  lecture.  The  bulk  of  the  time  was  used  to  generate  graphics  used  in  the 
presentations.  Future  development  of  the  lectures  into  multimedia  presentations  with  audio  and  video  is  expected  to 
take  8-10  hours  per  lecture  with  the  exception  of  the  first  lecture.  The  first  multimedia  lecture  presentation  has  been 
completed  and  required  more  than  40  hours  to  complete.  The  games  each  took  5-8  hours  to  design,  write  the 
JavaScript  code,  and  generate  the  graphics.  The  quizzes  could  each  be  developed  in  less  than  an  hour.  Setting  up 
the  overall  structure  of  the  web  site  and  linking  the  information  is  a relatively  simple  task. 

Reliance  on  previous  experience  can  greatly  reduce  the  required  investment  of  time.  The  first  project  has 
been  duplicated  in  a veiy  limited  way  and  applied  to  another  engineering  course  (Measurements  and 
Instrumentation).  In  less  than  30  hours  a significant  portion  of  the  structure  of  the  original  project  has  been 
recreated  for  the  new  course.  Handwritten  lecture  notes  were  scanned  and  saved  as  PDF  files  that  are  readable  with 
a browser  plug-in.  Old  exams  and  homework  solutions  have  also  been  added  to  the  site  to  give  many  of  the  benefits 
of  the  interactive  quizzes  for  a fraction  of  the  invested  time. 
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Conclusions 


The  overall  effect  of  a major  change  in  pedagogy  on  the  average  student  has  been  positive.  The  average 
student  has  adapted  to  the  use  of  the  multimedia  instructional  approach  quite  well  with  minor  improvements  in 
overall  performance  and  depth  of  knowledge  gained  in  the  course.  The  most  notable  changes  however,  are 
associated  with  those  who  fall  outside  the  range  of  average.  Advanced  students  are  stimulated  by  the  self-paced 
nature  of  the  course  and  stay  motivated  throughout  the  semester.  Slower  students  can  work  at  their  own  pace  and 
finish  a course  that  they  might  otherwise  drop.  There  does  appear,  however,  to  be  a group  of  students  from  the  full 
intellectual  spectra  who  do  not  perform  well  in  an  environment  with  there  is  little  interpersonal  connectivity  with  the 
instructor.  A small  section  using  traditional  instruction  could  be  added  to  address  this  problem. 

The  overall  impact  on  the  course  has  also  been  positive  with  a significant  increase  in  the  breadth  of 
material  covered  and  flexibility  of  delivery.  With  the  completion  of  the  multimedia  instructional  material  the  course 
can  be  easily  exported  off  campus  to  area  high  schools  students  and  to  those  who  wish  to  take  the  course  on-line  for 
convenience.  The  scheduling  flexibility  for  students  taking  an  on-line  course  will  likely  increase  enrollment  as  the 
course  is  an  introductory  level  course. 

The  cost  of  developing  the  project  was  significant.  Several  hundred  hours  were  required  to  bring  the 
project  to  its  current  level.  Justification  for  such  a tremendous  individual  effort  was  based  on  the  promise  of 
teaching  multiple  sections  of  the  course  over  several  semesters.  A scaled  down  version  of  the  project  has  since  been 
successfully  been  developed  for  another  engineering  course  with  a fraction  of  total  time  invested  indicating  that  the 
pedagogical  approach  can  be  successfully  adapted  to  other  courses. 
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ABSTRACT:  The  idea  of  using  animations  to  illustrate  dynamic  behaviors  of  computer  algorithms  is  over 
fifteen  years  old.  Over  a hundred  algorithm  animation  systems  have  been  built  since  then,  with  most 
developed  in  the  belief  that  the  animations  would  serve  as  effective  learning  aids  for  students.  However,  only 
recently  have  researchers  started  asking  the  question:  Do  algorithm  animations  really  help?  Unfortunately, 
results  of  experiments  driven  by  this  question  have  been  disappointing.  We  believe  that  previous  attempts  at 
using  animation  to  teach  algorithm  behavior  were  unsatisfactory  not  because  of  a flaw  with  animation  as  a 
technique,  but  because  of  the  approach  used  to  convey  the  animations.  In  this  paper,  we  present  a novel 
theoretical  framework  for  the  design  of  effective  algorithm  visualizations,  one  which  espouses  embedding 
interactive  analogies  and  animations  in  a context  and  knowledge  providing  hypermedia  environment.  We 
then  present  the  architecture  of  HalVis,  an  implemented  system  based  on  this  framework,  and  results  from 
five  empirical  studies  that  demonstrated  the  advantage  of  our  framework  over  traditional  means  of  instruction 
and  extant  algorithm  animations. 


Introduction 

Over  fifteen  years  ago,  with  the  unveiling  of  the  movie  Sorting  Out  Sorting  at  SIGGRAPH-81,  the  idea  of  using 
graphics  and  animation  to  illustrate  the  dynamic  behavior  and  functionality  of  computer  algorithms  was  bom.  Algorithm 
animation  appeared  to  hold  great  promise  as  an  instructional  aid,  with  many  dozens  having  been  built  such  as  Balsa  (Brown, 
1988),  Tango  (Stasko,  1990)  and  their  successors.  Most  were  developed  in  the  belief  that  algorithm  animations  would  serve 
as  effective  supplements  to  lectures  for  students  to  learn  about  algorithms.  This  belief  has  a strong  intuitive  basis.  Students 
have  always  had  a relatively  difficult  time  understanding  abstract  mathematical  notions,  especially  when  these  included 
dynamics  of  how  algorithms  manipulate  data,  so  concretizing  these  notions  graphically  and  animating  them  to  illustrate  the 
dynamics  ought  to  improve  learning. 

Though  research  on  designing  and  deploying  algorithm  animations  spans  over  15  years,  only  recently  have 
researchers  started  asking  the  question:  do  algorithm  animations  really  help?  Dozens  of  experiments  have  been  conducted  to 
test  whether  algorithm  animations  lead  to  improved  understanding  of  algorithms.  Hundhausen  (1996)  presents  an  excellent 
summary  of  the  results  of  29  empirical  studies  pertaining  to  the  comprehension  efficacy  of  algorithm  animations  and,  more 
generally,  software  visualization.  Unfortunately,  all  one  can  say  about  this  accumulated  evidence  is  that  the  results,  at  best, 
are  mixed.  While  animations  are  enthusiastically  received  by  the  students,  none  of  the  studies  has  proven  conclusively  that 
these  visual  presentations  actually  improve  learning  (Badre,  1991;  Byrne,  1993).  The  following  quote  succinctly  expresses 
the  frustration  felt  by  researchers  working  in  this  area:  "Unfortunately,  the  viability  of  algorithm  animations  as  instructional 
aids  remains  rooted  in  intuition.  No  substantive  empirical  evidence  has  ever  been  presented  to  support  these  claims"  (Stasko, 
1993).  Is  this  because  animation  is  an  ineffective  teaching  tool?  Intuition  tells  us  this  is  not  likely. 


Rethinking  Algorithm  Animations— From  Animations  To  Visualizations 

Research  presented  here  is  based  on  the  premise  that  animations  are  indeed  useful.  We  believe  that  previous 
attempts  at  using  animation  to  teach  algorithm  behavior  were  unsatisfactory  not  because  of  a flaw  with  animation  as  a 
technique,  but  because  of  the  approach  used  to  convey  the  animations.  Hence,  a rethinking  of  algorithm  animation  design  is 
required  in  order  to  harness  its  power  to  enhance  learning. 

Theoretical  foundations  of  our  approach  to  algorithm  visualization  are  rooted  in  several  areas:  (i)  a comprehension 
model  derived  from  research  on  how  humans  reason  using  static  diagrams  to  infer  the  dynamic  behavior  of  mechanical 
devices  and  its  application  to  the  design  of  hypermedia  machine  manuals  (Narayanan,  1998),  (ii)  recent  research  in  cognitive 
and  educational  domains  on  how  multimedia  and  hypermedia  can  improve  student  comprehension  (Crosby,  1995;  Daily, 
1 994;  Kehoe,  1 996),  and  (iii)  extant  research  on  the  design  and  evaluation  of  algorithm  animations  (e.g.,  Byrne,  1 996;  Kehoe, 
1996;  Lawrence,  1994;  Stasko,  1997;  see  Hundhausen,  1996  for  a comprehensive  overview).  Drawing  from  these  sources. 
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we  have  developed  a framework  for  the  design  of  algorithm  visualizations  called  HalVis  (Hypermedia  Algorithm 

Visualizations)  and  a corresponding  software  system.  The  key  idea  is  to  embed  algorithm  animations  at  various  levels  of 

abstraction  inside  a hypermedia  environment.  Distinctive  features  of  HalVis  and  their  rationale  are  described  below. 

• Learning  Objectives:  Algorithm  visualization  design  starts  with  a top-down  process  in  which  explicit  learning  objectives 
(e.g.,  understand  the  central  metaphor  such  as  that  of  values  ’’bubbling  up”  in  BubbleSort,  understand  core  operations  of 
the  algorithm  such  as  swapping  data  values  during  sorting  or  pivot  picking  in  Quicksort,  etc.)  are  used  to  divide, 
describe  and  deliver  information.  The  approach  is  learner-centered  (’’what  do  students  need  to  know”)  rather  than 
technology-centered  (’’what  can  we  show”).  In  HalVis,  the  focus  is  not  on  the  animations  but  rather  on  the  orchestration 
of  various  hypermedia  elements  to  meet  the  educational  objectives,  leading  to  effective  visualization. 

• Scaffolding:  The  learning  process  begins  using  real  world  analogies  (e.g.,  animated  playing  cards  that  illustrate  sorting 
and  merging  in  MergeSort)  intended  to  provide  a basic  conceptual  understanding  of  the  essential  elements  of  the 
algorithm.  These  analogies  serve  the  purpose  of  “scaffolding”  (Hmelo,  1 996)  and  are  a natural  prelude  to  subsequent 
learning.  Douglas  (1995)  and  Stasko  (1997)  have  shown  that  when  students  are  asked  to  design  algorithm  animations, 
they  employ  real-world  metaphors  such  as  a football  game  or  flying  spacecraft.  In  HalVis,  the  analogy  introduces  the 
algorithm,  and  then  the  gap  between  the  analogy,  the  abstract  components  of  the  algorithm,  and  the  concrete  graphical 
representations  used  in  other  modules  to  depict  the  algorithm’s  behavior  is  bridged.  Bridging  is  intended  to  facilitate 
comprehension  of  algorithm  animations  by  allowing  the  learner  to  internally  connect  the  analogy  and  the  animation.  All 
this  occurs  prior  to  showing  any  algorithm  animation.  We  believe  analogies  set  the  stage  for  subsequent  detailed 
learning,  help  improve  long-term  retention,  and  lead  to  better  visualization. 

• Chunking:  Given  the  big  picture,  learning  can  proceed  to  the  details.  Unfortunately,  current  algorithm  animation 
systems  present  the  detailed  dynamics  of  the  algorithm  as  a one-shot,  stand-alone  show  that  is  entertaining  to  watch  but 
tends  to  obscure  the  very  details  the  student  needs  to  learn.  Research  on  cognitive  media  types  (Recker,  1995)  points  to 
the  importance  of  using  different  types  of  information  (definitions,  examples,  etc.)  in  teaching  algorithmic  concepts. 
Kehoe  (1996)  have  also  pointed  out  the  importance  of  multiple  representations  and  media,  concluding  that  text  is 
important  for  precision,  pseudocode  is  useful  for  conveying  steps  of  the  algorithm,  and  animations  are  good  for  depicting 
operational  behavior.  HalVis  uses  a technique  called  ’’animation  chunking”  in  which  animations  are  broken  up  into 
meaningful  bite-sized  pieces.  Each  chunk  is  presented  in  synchrony  with  other  representations  (e.g.,  feedback  in  a text 
window,  pseudocode  with  relevant  lines  highlighted,  aural  explanations)  of  the  specific  actions  taking  place.  These  are 
embedded  within  a hypermedia  structure  that  provides  contextual  knowledge  using  hyperlinks  to  additional  explanations 
using  text,  audio  and  pictures.  Combining,  synchronizing,  and  presenting  information  in  multiple  media  in  discrete 
chunks  is  more  than  animation;  it  is  visualization. 

• Granularity:  Three  distinct  kinds  of  animations  provide  views  of  algorithm  behavior  at  different  levels  of  granularity. 
One  is  an  animated  real-world  analogy  intended  to  function  as  a scaffold  and  a bridge  by  conveying  the  essence  of  the 
algorithm.  The  second  is  a detailed  micro-level  animation  that  focuses  on  specific  algorithmic  operations.  The  third  is  a 
broader  macro-level  animation  that  shows  the  algorithm’s  behavior  on  large  data  sets.  The  detailed  animation  is  chunked 
at  multiple  levels  (e.g.,  at  the  level  of  a single  instruction  execution,  at  the  level  of  a ’’pass”  for  algorithms  that  make 
multiple  passes  over  data).  Each  chunk  is  presented  in  tandem  with  pseudocode  highlighting  and  textual/aural 
explanations.  The  learner  can  elect  to  see  the  animation  at  any  of  the  available  chunking  levels.  The  micro-animation  is 
presented  before  the  macro-animation,  both  of  which  are  seen  by  the  learner  only  after  the  animated  analogy.  This 
sequential  design  is  intended  to  address  a particular  difficulty  uncovered  by  an  earlier  study  that  stated,  ”a  paradoxical 
problem  with  the  animation  is  that  it  shows  the  big  picture  or  emergent  qualities  that  might  be  appreciated  only  by  those 
that  already  understand  the  algorithm  at  the  mechanical  level”  (Byrne,  1996). 

• Rich  Interactivity:  Most  educators  agree  increased  student  interaction  improves  learning.  Experiments  conducted  by 
Lawrence  (1994)  indicate  a learning  advantage  for  students  who  had  active  involvement  in  the  creation  of  input  values  to 
algorithm  animations.  In  contrast,  research  in  different  domains  indicates  that  passively  watching  an  animation  may  not 
facilitate  learning  (Rappin,  1997;  Stasko,  1993).  In  HalVis,  all  animations  allow  active  intervention  by  the  learner. 
Besides  the  usual  replay  and  speed  controls,  HalVis  allows  and  even  encourages  students  to  alter  the  data  and  rerun  the 
animations  to  see  how  the  algorithm  performs  on  data  sets  of  their  own  choosing.  Students  can  also  make  predictions 
about  performance  to  compare  with  actual  results.  Increasing  the  level  of  interaction  leads  to  better  visualization. 

• Critical  Thinking:  A potential  problem  with  multimedia  presentations  is  often  stated  as  ’’hands-on,  mind-off,”  where  the 
interactions  engage  the  user  so  much  that  analytical  thought  about  the  content  of  the  presentation  is  inhibited.  HalVis 
employs  two  kinds  of  probes  to  avoid  this  syndrome.  One,  called  ’’ticklers,”  are  questions  that  pop  up  in  random  order, 
but  always  in  an  appropriate  context,  and  can  be  dismissed  by  the  user.  These  are  questions  that  require  considerable 
mental  effort  in  the  form  of  self-explanations  (which  have  been  shown  to  promote  comprehension  (Chi,  1989),  but  the 
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system  makes  no  effort  to  enforce  answering  them.  The  other  kind,  which  we  call  "articulation  points"  involves 
questions  the  user  must  answer  before  proceeding,  and  the  system  provides  immediate  feedback  about  the  correctness  of 
the  answer.  These  questions  are  tied  to  learning  objectives  set  out  by  the  designer  and  help  students  self-diagnose  their 
learning  and  allow  them  to  reflect  on  what  they  know,  what  they  don’t,  and  where  to  find  relevant  information. 


Hal  Vis  Architecture 

We  have  created  a prototype  system,  called  HalVis,  to  test  the  components  of  the  framework  described  above. 
HalVis  is  implemented  using  Asymmetrix  Toolbook,  and  as  of  this  writing,  contains  visualizations  of  four  sorting  algorithms 
and  one  graph  algorithm.  Each  visualization  in  HalVis  consists  of  the  modules  shown  in  Figure  1 and  described  below: 


Figure  1.  HalVis  Architecture 


• Conceptual  View:  This  module  introduces  a specific  algorithm  in  very  general  terms  using  a real  world  analogy.  For 
instance,  BubbleSort  is  introduced  using  a flask  of  water  with  bubbles  that  rise  to  the  surface  according  to  their  size  and 
the  Shortest  Path  algorithm  uses  a simple  game  that  allows  the  student  to  observe  a map  of  airline  fares  through  various 
cities  and  select  the  route  with  the  cheapest  fare  to  illustrate  the  basic  working  of  the  shortest  path  algorithm.  This 
module  uses  animations,  text  and  audio  to  provide  the  student  with  a general  overview,  a visual  example  to  aid  long-term 
retention,  and  sufficient  bridging  information  to  help  learners  proceed  from  the  visual  elements  in  the  analogy  to  the  data 
structures  and  algorithm  operations  in  later  modules. 


• Detailed  View:  This  module  contains  two  screens.  One  consists  of  a detailed  textual  description  of  the  algorithm 
alongside  a pseudocode  representation  of  it.  Embedded  in  the  text  are  hyperlinks  to  related  information  in  the 
Fundamentals  module.  The  second  screen,  shown  in  Figure  1,  contains  four  windows  that  depict  various  aspects  of  the 
algorithm's  behavior.  The  Execution  Animation  window  shows  how  steps  of  the  algorithm  modify  data  structures  using 
smooth  animation.  The  animation  is  chunked  at  multiple  levels  of  granularity  corresponding  to  meaningful  units  of  the 
algorithm's  behavior,  and  can  be  altered  by  the  learner.  At  the  lowest  level,  the  animation  displays  the  execution  of  an 
individual  statement,  pausing  for  the  learner’s  signal  to  proceed.  The  next  level  corresponds  to  a logical  operation,  like 
completion  of  a single  pass  in  a loop.  At  the  highest  level,  the  animation  proceeds  to  completion  without  pausing.  The 
Execution  Status  Message  window  provides  comments  and  textual  feedback  to  the  student  about  key  events  and  actions 
during  execution.  This  is  also  available  as  an  audio  commentary.  The  Pseudocode  window  shows  the  steps  involved  in 
the  algorithm,  which  are  highlighted  synchronously  with  the  animation.  Finally,  the  Execution  Variables  window 
contains  a scoreboard-like  panorama  of  the  variables  involved  in  the  algorithm  and  their  changing  values.  Before 
launching  the  animation,  students  can  change  the  data  input  to  the  algorithm  as  well  as  the  speed  and  granularity  of 
animation  and  feedback.  Execution  of  each  step  of  the  algorithm  affects  the  display  in  the  four  windows  simultaneously. 
The  thumbnail  in  Figure  1 shows  a seven  vertex  graph  used  in  the  Shortest  Path  algorithm.  HalVis  intentionally  limits 
the  number  of  data  items  in  the  Execution  Animation  window  to  focus  attention  on  the  micro-behavior  of  the  algorithm. 

• Populated  View:  This  module  is  intended  to  provide  students  with  an  animated  view  of  the  algorithm  on  large  data  sets 
to  make  its  macro-behavior  explicit.  Many  of  the  details  presented  in  the  detailed  view  are  obscured  to  enhance  the 
student’s  focus  on  the  algorithm’s  performance.  For  example,  the  pseudocode  is  not  shown,  nor  are  variables. 
Animations  in  this  module  are  similar  to  those  in  extant  systems,  but  there  are  two  novel  features.  One  is  a panel  of 
counters  that  show  pertinent  performance-oriented  information  such  as  number  of  comparisons,  swaps,  recursive  calls, 
and  so  on.  Another  is  a facility  for  the  student  to  make  a prediction  about  different  parameters  of  algorithm  performance 
and  then  compare  those  against  the  actual  performance  when  the  animation  is  running.  When  the  learner  launches  the 
animation,  the  system  prompts  for  predictions,  creates  a random  initial  configuration,  and  proceeds  to  move  the  graphical 
representations  accompanied  by  audio  explanations  and  cues. 

• Questions  & Fundamentals:  The  Questions  module  presents  the  student  with  a combination  of  multiple  choice,  true- 
false,  and  algorithm  debugging  (by  reordering  the  steps)  questions  to  measure  competency  and  comprehension,  and  the 
system  provides  immediate  feedback  on  student  responses.  HalVis  also  pops  up  context-sensitive  tickler  questions  to 
help  focus  student  attention  on  key  aspects  of  the  algorithm.  The  Fundamentals  module  contains  information  about  basic 
building  block  topics  common  to  virtually  all  algorithms.  Examples  include  Comparing  & Swapping  Data,  Looping 
Operation,  Recursion,  and  so  on.  Generally,  this  module  is  accessible  only  through  hyperlinks  from  other  modules,  so 
that  the  basic  information  is  presented  on  demand  (in  response  to  a learner  request  in  the  form  of  clicking  on  a hyperlink) 
and  in  context  (of  algorithm-specific  information  within  which  the  hyperlink  is  embedded). 


Experiments  With  Hypermedia  Algorithm  Visualizations 

We  conducted  a series  of  five  experiments  to  evaluate  and  substantiate  the  effectiveness  of  hypermedia 
visualizations  of  algorithms,  specifically  to  validate  our  hypothesis  that  visualizations  will  be  more  effective  than  traditional 
teaching  methods  for  students  learning  about  algorithms.  In  each  of  the  experiments,  subjects  took  a pretest  to  measure  prior 
knowledge,  learned  the  algorithm  using  HalVis  or  a traditional  method,  then  took  a posttest  that  measured  knowledge 
improvement.  Students  were  tested  on  their  ability  to  recognize  and  reorder  pseudocode  descriptions  of  algorithms,  mentally 
simulate  algorithmic  operations,  and  predict  resulting  data  structure  changes.  In  these  studies,  we  did  not  differentiate 
between  visual  and  verbal  learners  since  HalVis  contains  rich  textual  and  visual  presentations  to  support  both  kinds  of  learner 
dispositions. 

Table  1 lists  information  about  these  experiments,  showing  the  student  level,  the  materials  used  by  the  two 
experimental  groups,  the  algorithm(s)  studied,  and  the  statistical  significance  of  results.  The  first  two  experiments  were 
described  in  detail  in  (Hansen,  1998).  Experiment  I showed  that  hypermedia  visualizations  are  more  effective  than  textbooks 
for  learning  about  algorithms.  This  result  was  successfully  replicated  in  Experiment  II  using  more  advanced  students. 
Experiment  III  compared  learning  from  HalVis  to  learning  from  a compilation  of  the  best  algorithm  descriptions  (extracted 
from  19  textbooks  published  between  1974  and  1997)  followed  by  problem  solving  activities.  While  the  posttest  averages 
favored  the  HalVis  group,  the  difference  is  not  statistically  significant.  This  experiment  indicated  that  interactive 
visualization  can  be  as  effective  as  learning  from  carefully  crafted  text,  when  that  textual  learning  is  followed  by  problem 
solving. 
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Experimen 

t 

Student 

Level 

Comparison  Groups 
(average  posttest  scores) 

Algorithm(s) 

Studied 

Statistical 

Significance 

I 

Sophomore 

HalVis 

(74%) 

Text 

(43%) 

MergeSort 

F(l,27)=10.9 

p<0.015 

II 

Junior 

HalVis 

(63%>) 

Text 

(44%) 

MergeSort 

Quicksort 

F(l,21)=4.9 

p<0.038 

III 

Sophomore 

HalVis 

(61%) 

Text  + Exercises 
(57%) 

BubbleSort 

SelectionSort 

F(l,24)=0.3 

p<0.57 

IV 

Sophomore 

HalVis 

(70%) 

Lecture 

(44%) 

SelectionSort 

MergeSort 

F(l,19)=5.3 

p<0.033 

V 

Junior 

HalVis 

(89%) 

Text  + Animation 
(71%) 

ShortestPath 

F(l,37)=12.8 

p<0.001 

Table  1.  Summary  of  Five  Experiments  Conducted  in  This  Research 


Experiment  IV  was  designed  to  compare  and  evaluate  how  algorithm  visualization  (AV)  and  conventional 
classroom  lectures  interact  and  contribute  to  student  learning.  Our  hypothesis  was  that  AV  alone  would  assist  learning  more 
than  a lecture.  If  so,  it  follows  that  AV  used  in  conjunction  with  lecture  should  assist  learning  even  more.  We  used  a 100 
minute  live  classroom  lecture  by  a computer  science  faculty  known  for  his  teaching  ability,  delivered  in  two  consecutive 
class  sessions.  Participants  were  second  year  computer  science  students  at  Auburn  University,  divided  into  two  matching 
groups.  One  group  used  HalVis  before  attending  the  lecture  (called  the  visualization-lecture  (VL)  group)  and  the  other  group 
used  HalVis  after  the  lecture  (called  the  lecture-visualization  (LV)  group).  Having  both  groups  attend  the  same  lecture 
eliminated  variations  that  can  occur  in  different  sessions,  while  allowing  interactions  with  the  teacher.  A first  posttest  was 
given  between  the  two  phases,  and  a second  posttest  was  given  after  both  phases.  The  first  posttest  results  indicated  that  the 
VL  group  learned  significantly  more  than  the  LV  group;  following  the  session  with  HalVis,  the  VL  group's  average  score  was 
70%  compared  to  44%  for  the  LV  group.  However,  the  second  posttest  results  showed  that  after  receiving  both  the  lecture 
and  the  visualization,  both  groups  performed  at  the  same  level  (72%).  These  results  suggest  that  visualizations  can  have  a 
significantly  higher  learning  impact  than  a lecture,  and  that  a combination  of  both  is  even  better. 

Experiment  V compared  the  effectiveness  of  learning  from  HalVis  to  learning  from  an  algorithm  animation  typical 
of  extant  research  on  this  topic.  One  of  the  most  mature,  widely  reported,  and  publicly  available  algorithm  animation 
platforms  is  the  Tango  software  suite  developed  by  Stasko  (1997).  The  Tango  software  distribution  contains  a library  of 
animated  algorithms,  including  eight  animations  of  the  Shortest  Path  algorithm.  Of  these  eight,  we  selected  one  that  appeared 
to  be  the  most  complete,  easiest  to  understand,  and  which  most  closely  matched  the  features  of  the  HalVis  system,  as  a 
representative  animation.  One  group  of  students  interacted  with  HalVis  and  the  other  group  interacted  with  the  Tango 
animation.  The  second  group  was  also  provided  with  a textual  description  of  the  algorithm,  to  which  the  HalVis  group  did 
not  have  access.  On  the  posttest,  the  HalVis  group  averaged  89%  while  the  Tango  group  averaged  71%,  indicating  that  a 
hypermedia  algorithm  visualization  is  more  effective  than  an  algorithm  animation. 


Conclusion 

Collectively,  these  experiments  provide  initial  statistical  validity  to  the  hypothesis  that  hypermedia  algorithm 
visualizations  can  be  significantly  more  effective  than  traditional  teaching  methods  considered  in  isolation.  More  details  of 
these  experiments  are  available  in  (Hansen,  1998).  There  are  a number  of  possible  reasons  for  this  effectiveness.  First,  in 
comparison  with  previous  animation  systems  that  only  presented  animations  with  some  textual  feedback,  HalVis  allows  the 
student  to  learn  incrementally  by  starting  from  a real  world  analogy  and  transitioning  to  the  algorithm  itself  Second,  the 
hypermedia  structure  allows  a student  access  to  fundamental  building  blocks  of  algorithmic  knowledge  in-context  and  on- 
demand.  Third,  the  top-down  learning  objective  based  design  approach  and  the  hypermedia  structure  have  allowed  us  to 
divide  information  into  manageable  pieces  and  present  each  chunk  using  the  most  appropriate  media.  Fourth,  rather  than 
providing  an  animation  all  in  one  piece  as  has  been  the  typical  approach,  HalVis  presents  three  kinds  of  animations  in  an 
analogical -detailed-abstract  sequence.  Fifth,  animations  are  delivered  at  multiple  levels  of  granularity.  This  animation 
chunking  makes  it  easier  for  students  to  pause,  repeat,  reflect,  or  access  other  relevant  information  through  hyperlinks  while 
watching  animations.  Furthermore,  animation  chunks  are  presented  in  synchrony  with  other  representations.  These  two 
unique  features,  we  believe,  result  in  the  dynamic  information  being  conveyed  better  in  context,  and  therefore  in  a more 
comprehensible  fashion.  Sixth,  animations  allow  both  active  intervention  and  performance  prediction  by  learners,  both  of 
which  can  improve  comprehension.  Seventh,  we  believe  that  questions  that  pop  up  in  context  in  different  parts  of  HalVis 
promote  reflection  and  self-explanation,  contributing  to  learning.  Future  studies  will  selectively  analyze  these  features  in 
order  to  identify  those  that  contribute  most  to  learning.  More  information  on  this  research  is  available  at  our  group's  web  site: 
http://www.eng.aubum.edu/cse/research/vi3rg/vi3rg.html. 
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Abstract 

The  last  two  decades  have  seen  a significant  increase  in  both  the  development  and  usage  of 
computer  based  learning  technology  in  tertiary  education.  Many  researchers  in  the  field  of 
education  have  begun  to  develop  an  understanding  of  student  learning  in  terms  of  learning 
strategies.  Such  constructs  represent  behaviours  and  thought  processes  adopted  by  the  student 
and  are  believed  to  mediate  learning.  At  present,  however,  little  is  known  of  the  impact  that 
computer  based  learning  technologies  have  on  these  strategies. 

This  paper  outlines  various  developments  in  our  understanding  of  student  learning  in  terms  of 
styles  and  strategies  that  have  occurred  over  the  latter  half  of  this  century.  The  paper  focuses 
on  a learning  strategies  framework  developed  in  the  United  States  and  its  initial  use  as  an 
evaluation  tool  for  courseware  in  a distance  education  setting  in  the  UK.  Preliminary  findings 
from  this  initial  study  designed  to  investigate  the  learning  strategies  users  adopt  when  using  a 
piece  of  courseware  are  also  described  along  with  plans  for  a future  large  scale  study. 


Introduction 

Increasingly,  educators  and  researchers  acknowledge  that  learning  strategies  comprise  some  of  the  active 
processes  that  need  to  be  examined  to  gain  a better  understanding  of  how  students  learn.  At  the  same  time 
educators  are  turning  to  software  developers  to  offer  computer  based  solutions  to  material  that  has  proved 
difficult  or  unsuccessful  in  the  traditional  educational  setting.  In  turn  software  developers  are  utilising  more 
sophisticated  forms  of  media  (Jonassen,  1988).  An  increasingly  popular  option,  especially  with  the  natural 
sciences,  is  the  computer  simulation.  Advances  in  both  hardware  and  software,  have  meant  increasingly 
sophisticated  simulations  becoming  common  computer-assisted  learning  (CAL)  tools.  Dating  back  to  the  late 
1970's,  simulations  have  provided  the  student  with  a meaningful  environment  within  which  they  can  interact 
with  physical  objects  or  scenarios  from  the  real  world  (Thurman,  1993). 

The  research  discussed  in  this  paper  examines  a resource  based  learning  tool,  "The  Human  Brain"  CD-ROM, 
which  makes  use  of  simulations.  The  research  aims  to  explore  this  tool  in  terms  of  students'  learning  strategies. 
This  form  of  evaluation,  where  students'  learning  strategies  are  the  focus  of  the  investigation,  has  almost 
exclusively  been  neglected  in  favour  of  one  that  determines  the  usability  or  effectiveness  of  the  software.  Yet, 
arguably,  notions  of  usability  and  effectiveness  are  dependent  on  the  learning  strategies  employed  by  students. 
The  evaluation  is  intended  to  provide  insight  into  the  success  and  limitation  of  the  software  in  terms  of  both 
usability  and  effectiveness.  The  research  has  also  aimed  to  examine  the  stability  of  student  learning  strategies 
over  the  duration  of  an  academic  course  and  explore  whether  learning  strategies  differ  as  a function  of  the 
learning  medium  students  use  on  the  course,  i.e.  computer  based  versus  non-computer  based. 

In  the  following  sections  the  notions  about  and  measurement  of  learning  strategies  will  be  described  with  a 
history  of  how  educational  researchers,  predominantly  in  the  States,  have  come  to  understand  student  learning  in 
terms  of  such  constructs.  Evaluation  at  the  UK  Open  University  (OU)  will  then  be  outlined  briefly, 


demonstrating  the  current  consideration  given  to  learning  processes  in  CAL  evaluation.  The  paper  will  then 
describe  a pilot  study  undertaken  in  1998  to  investigate  the  learning  strategies  of  student  learning  with  a 
computerised  resource  based  learning  tool. 


Background  Literature 

Learning  Strategies 

For  many  years  educational  researchers  have  sought  to  determine  and  understand  the  learning  processes  adopted 
by  students  in  tertiary  education.  Much  of  this  work  has  been  influenced  by  psychological  theory.  In  the  last  fifty 
years  or  so,  the  field  of  psychology  has  been  dominated  by  an  interest  in  cognitive  processes.  With  this  impetus 
and  also  an  increasing  understanding  of  personality  and  personality  typologies,  researchers  began  to  investigate 
whether  differences  existed  in  the  ways  individuals  approached  everyday  information  processing.  Differences 
were  found  which  indicated  a series  of  consistent  approaches  to  the  organisation  and  processing  of  information 
which  came  to  be  termed  cognitive  styles  (Tennant,  1988).  Attempts  have  been  made  to  examine  such 
differences  in  the  context  of  the  university  setting.  However,  according  to  Messick  (1981,  1984)  they  have 
generally  proved  unsuccessful  because  of  problems  associated  with  the  definition  and  measure  of  cognitive  style. 

By  the  late  1960's  researchers  began  to  acknowledge  that  learning  in  the  university  setting  differed  from 
everyday  learning  processes.  Work  carried  out  predominantly  in  Europe  during  the  1960's  and  1970's  proposed 
the  notion  of  student  learning  styles.  As  a construct,  learning  style  generally  referred  to  one  of  a finite  number  of 
learning  dispositions  that  students  adopted  in  their  pursuit  of  a degree  qualification.  It  also  incorporated  the 
notion  of  both  cognitive  and  motivational  processes  impacting  students'  learning  behaviour.  For  example,  a 
student  who  was  intent  on  gaining  a high  level  understanding  of  the  information  being  learnt  and  was  motivated 
to  do  so  by  a personal  interest  in  the  subject  matter  could  be  categorised  as  adopting  a deep  learning  style 
(Marton  and  Saljo,  1976).  In  contrast,  a student  who  was  preoccupied  with  meeting  the  assessment  aims  and 
requirements  of  the  faculty  and  whose  main  concern  was  to  learn  the  required  information  by  methods  of 
rehearsal  or  rote  memorisation  could  be  categorised  as  adopting  a surface  learning  style. 

The  term  ’’learning  style"  has  come  to  be  used  both  synonymously  with,  and  in  contrast  to,  the  terms  "learning 
approach"  and  "learning  orientation".  This  confusion  with  regards  to  terminology  is  associated  with  problems  of 
definition  regarding  what  behaviours  or  processes  the  construct  of  learning  style  actually  refers  to.  Terminology 
regarding  learning  style  and  definitions  of  the  construct  have  become  convoluted,  in  part,  because  of  a common 
aim  held  by  researchers  in  the  field  - to  increase  our  understanding  of  student  learning  by  determining  individual 
differences  in  the  manner  in  which  students  learn.  Confusion  has  arisen  because  a great  number  of  individual 
differences  have  been  found  to  exist  amongst  learners,  yet  researchers  report  their  findings  as  definitive  and 
finite.  Attempts  have  been  made  to  classify  the  various  findings  in  the  field,  but  even  these  systems  present  little 
agreement  over  the  classification  of  findings  from  over  thirty  years  of  work  (see  Murray- Harvey,  1 994). 

In  recent  years  there  has  also  been  an  increasing  amount  of  debate  about  the  benefit  of  learning  style  as  a 
measure  of  learning  behaviour  in  tertiary  education  (e.g.  Mitchell,  1994).  It  has  been  proposed  that  such  notions 
of  learning  preferences  and  habits  may  not  take  into  account  variation  in  the  learning  behaviours  and  processes 
that  occur  in  response  to  university  teaching  (McKeachie,  1998).  The  work  of  McKeachie  and  his  colleagues  at 
the  University  of  Michigan,  like  many  others,  represents  a move  away  from  the  classification  of  learners  towards 
the  classification  of  learning  according  to  a series  of  constructs  termed  learning  strategies  (Pintrich,  McKeachie 
and  Smith,  1989).  Pintrich  et  al  propose  a framework  of  learning  strategies  based  in  part  on  the  work  of 
Weinstein  et  al  (1987). 

Weinstein  and  Mayer  (1986)  provide  a definition  of  learning  strategies  as: 

"behaviours  and  thoughts  that  a learner  engages  in  during  learning  and  that  are  intended  to 
influence  the  learner’s  encoding  process... the  goal  of  any  such  strategy  is  to  affect  the  learner’s 
motivational  or  affective  state,  or  the  way  in  which  the  learner  selects,  acquires,  organises,  or 
integrates  new  knowledge"  (p315) 
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Learning  styles  provide  a classification  of  the  student,  for  instance,  as  in  the  examples  provided  earlier.  Learning 
strategies  present  a departure  from  this  method,  instead  learning  is  seen  as  comprising  a series  of  components  or 
constructs  that  relate  to  motivational  and  cognitive  aspects  of  learning  behaviour.  The  student  is  seen  as  using 
each  of  these  strategies  to  a varying  degree,  for  example  they  might  be  highly  extrinsically  motivated  and  make 
good  use  of  an  organisational  strategy.  There  is  no  categorisation  of  the  student,  merely  a profile  of  their  learning 
behaviour,  a profile  of  the  extent  to  which  they  use  a particular  set  of  strategies. 

In  attempting  to  measure  student  learning  strategies  Pintrich  et  al  (op  cit),  have  developed  the  Motivated 
Strategies  for  Learning  Questionnaire  (MSLQ).  The  MSLQ  is  based  upon  a theoretical  framework  comprising  a 
series  of  assumptions  about  learning  in  both  the  academic  and  real  world  which  assumes  the  use  of  a maxinum 
fifteen  strategies  (McKeachie,  1990). 

The  team  claim  that  students  construct  knowledge  on  the  basis  of  that  which  they  already  know.  It  is  also 
suggested  that  students*  ability  to  remember  and  utilise  what  they  have  been  taught  depends  on  the  levjel  to  which 
they  process  the  material  they  have  learnt.  Finally,  they  propose  that  human  beings  are  inherently  learning 
creatures  but  intrinsic  interest  can  be  weakened  by  a sense  of  hopelessness  about  one's  competence  as  a learner. 

Arguably,  the  MSLQ's  most  important  and  valuable  attribute  is  that  it  acknowledges  that  strategies  are  subject  to 
change  amongst  courses  and  sets  out  to  measure  strategies  specific  to  courses  and  course  components.  For  the 
purposes  of  the  planned  research,  this  is  an  extremely  valuable  attribute  of  the  MSLQ.  This  research  endeavours 
to  assess  the  impact  of  educational  software  on  students'  learning  strategies.  It  aims  to  examine  the  learning 
strategies  students  come  to  use  when  using  a resource  based  learning  tool  - "The  Human  Brain"  CD-ROM. 

The  framework  for  establishing  the  nature  of  these  strategies  has  been  informed  by  the  constructs  measured  by 
the  MSLQ.  Comparisons  and  distinctions  are  intended  to  then  be  drawn  between  learning  strategies  used  whilst 
working  with  the  CD-ROM  tool  and  other  teaching  materials,  predominantly  text-based,  on  the  course.  This 
comparison  may  appear  problematic  since  the  two  comprise  very  different  mediums,  but  the  comparison  being 
made  is  not  one  of  learning  outcome:  whether  there  is  a difference  in  the  quantity  or  quality  of  information  that 
is  learnt  between  computer  based  and  non-computer  based  materials.  Rather,  the  research  intends  to  ascertain 
whether  a different  or  similar  set  of  strategies  mediate  learning  with  computer-based  and  non-computer  based 
learning  materials.  This  is  important,  since  learning  strategies  vary  in  the  context  of  learning  domains  based 
upon  the  specific  learning  requirements  of  the  domain  and  previous  learning  experience.  To  that  end,  one  of  the 
aims  of  this  research  is  to  determine  whether  the  learning  processes  for  the  course  overall  feed  into  the  learning 
strategies  the  students  then  come  to  use  in  using  the  CD-ROM. 

The  design  of  "The  Human  Brain"  CD-ROM  was  prompted,  in  part,  by  the  difficulties  reported  by  past  students 
in  comprehending  the  material  in  traditional  text  or  two-dimensional  pictorial  format.  Generally,  where 
educational  software  has  been  made  available  it  forms  an  important  course  component,  yet  very  little  research 
has  examined  the  technology  in  terms  of  students'  learning  strategies.  At  the  Open  University  evaluation  studies 
have  embraced  the  study  of  users'  learning  processes  but  not  included  learning  strategies  in  their  assessments. 

Most  of  the  courseware  developed  at  the  UK  Open  University,  like  "The  Human  Brain"  CD-ROM  have  been 
produced  to  supplement  or  complement  various  science  courses  (see  Jones  et  al,  1996).  For  example.  The  Works 
Metallurgist  has  been  designed  to  help  students  with  problems  experienced  in  interpreting  phase  diagrams. 
Another  example,  the  Driven  Pendulum  assists  students  with  their  understanding  of  the  physics  of  motion. 
Increasingly  though.  Arts  based  subjects  are  also  drawing  on  computer  based  resource  learning  tools.  One 
example,  is  the  course:  Charles  Booth  and  Social  Investigation  in  the  Nineteenth  Century  which  introduces  the 
computer  as  a tool  for  historical  research.  It  allows  students  the  opportunity  to  use  and  construct  their  own 
historical  databases  from  a number  of  archive  resources  digitised  and  stored  on  CD-ROM. 

Evaluation  experience  from  such  courseware  and  other  projects  has  framed  the  development  of  an  evaluation 
framework.  Cl  AO!  (Context,  Interactions,  Attitudes  and  Outcomes),  by  the  Computers  and  Learning  Research 
Group  at  the  OU(Jones  et  al,  1996).  The  framework  is  intended  to  provide  the  evaluator  with  a variety  of 
methodological  tools  instead  of  a limiting  set  of  prescriptive  measures.  The  framework  has  been  used  to  evaluate 
courseware  on  a number  of  courses  taught  at  a distance  by  the  OU.  It  proposes  three  facets,  or  dimensions  that 
the  evaluation  process  should  come  to  incorporate  and  integrate:  context,  interactions  and  outcomes. 


At  present,  the  Cl  AO!  framework  proposes,  in  part,  that  the  process  of  evaluation  takes  into  account  various 
learning  behaviours  students  exhibit  when  using  courseware.  The  notion  of  various  learning  strategies  as 
proposed  by  Pintrich  et  al  would  provide  definition  and  more  importantly  a framework  for  understanding 
precisely  what  these  learning  processes  are. 


The  Pilot 

The  review  above  has  sought  to  identify  a need  for  viewing  students'  learning  in  terms  of  learning  strategies  and, 
with  the  growth  of  computer  based  resource  learning  tools,  to  examine  learning  with  these  tools  in  terms  of 
learning  strategies.  In  the  past,  student  learning  has  been  considered  in  terms  of  style,  initially  cognitive  style  and 
subsequently  learning  style.  These  conceptualisations  view  the  learner  as  a creature  of  habit  whose  learning 
behaviour  falls  into  a limited  set  of  categories,  e.g.  deep  approach,  surface  approach.  Increasingly,  though,  this 
classification  of  learners  is  being  replaced  by  an  examination  of  their  learning  behaviour  in  terms  of  learning 
strategies. 

The  notion  of  learning  strategies  accepts  that  learning  behaviour  is  subject  to  change  in  response  to  the  demands 
of  the  learning  situation,  with  the  characterising  feature  of  these  constructs  being  not  the  classification  of  the 
learner  but  of  their  behaviour.  Over  the  years  a series  of  learning  strategy  frameworks  have  been  developed.  For 
the  purposes  of  the  research  presented  in  this  paper  the  framework  developed  by  Pintrich  and  his  colleagues  has 
been  chosen  as  the  method  by  which  to  identify  the  learning  strategies  that  students  on  SD206:Biology:  Brain 
and  Behaviour  course  come  to  use.  The  framework  identifies  fifteen  learning  strategies  and  measurement  is 
made  of  the  extent  to  which  each  of  these  is  utilised  or  adopted  by  the  learner.  What  results  is  a learning  profile 
of  the  learner  as  opposed  to  the  characteristic  classification  of  the  learner  that  occurs  with  cognitive  style  and 
learning  style  measures.  This  profile  is  relevant  only  to  the  learning  situation  in  question,  i.e.  the  study  of  a 
particular  course  that  forms  part  of  the  degree  award,  since  learning  strategies  are  sensitive  to  learning  contexts 
and  different  courses  present  different  learning  contexts. 

The  use  of  computer  assisted  learning  (CAL)  tools  also  presents  a different  learning  context  and  the  purpose  of 
the  current  research  is  to  examine  the  impact  of  that  context  on  student  learning  processes  - their  learning 
strategies.  The  research  proposed  here  is  thus  framed  by  a series  of  issues  which  arise  from  the  literature 
examined  in  the  previous  sections.  Firstly,  much  of  the  work  on  learning  strategies  and  the  resulting  frameworks 
have  been  carried  out  in  the  US  on  students  studying  in  traditional  tertiary  education.  The  research  has  sought  to 
examine,  in  part,  how  comparable  those  findings  are  to  a British  sample  studying  at  a distance. 

There  has  also  been  little  work  on  the  stability  of  learning  strategies  over  a short  period  of  time.  The  constructs 
are  said  to  be  sensitive  to  context,  but  over  the  period  of  a course  there  may  be  other  factors,  for  example 
assignments  or  the  onset  of  assessment,  that  impact  the  strategies  that  students  come  to  use.  Thus,  the  research 
has  also  sought  to  examine  the  stability  of  students'  learning  strategies.  This  idea  of  context  impacting  upon 
learning  processes  leads  to  the  important  question  of  whether  students'  learning  strategies  change  as  a function  of 
the  medium  that  they  come  to  use  whilst  learning?  That  is,  would  the  option  of  using  a computer  based  learning 
resource  bring  about  changes  or  differences  in  the  strategies  students  use  when  learning  with  traditional  text, 
video  and  audio  material.  The  answer  to  this  question,  however,  is  dependent  on  knowing  what  learning 
strategies  students  employ  whilst  learning  with  a computer  assisted  learning  (CAL)  tool.  For  the  purposes  of  this 
study  a piece  of  courseware  developed  at  OU  has  been  chosen  as  the  piece  of  software  to  be  investigated  in 
terms  of  students'  learning  strategies.  The  pilot  is  intended  to  provide  an  understanding  of  the  learning  strategies 
used  by  students  undertaking  a course  at  the  OU  where  there  is  the  option  of  using  a CAL  tool  to  complement 
traditional  teaching  materials.  The  aims  of  the  intended  research  are  two-fold  and  comprise  two  research  phases. 
The  first  phase  serves  to  apply  the  MSLQ  to  a distance  education  setting  and  to  examine  the  stability  of  the 
learning  strategy  measures  over  a distance  of  approximately  two  months. 

Developed  in  America,  the  MSLQ  is  a self-report  instrument  designed  to  assess  college  students'  motivational 
orientations  and  their  use  of  different  learning  strategies  for  a specfic  college  course.  The  questionnaire 
comprises  81  items,  each  scored  on  a seven  point  Likert  (l=not  at  all  true  of  me  and  7=very  true  of  me). 
Motivation  is  measured  by  six  subscales:  intrinsic  goal  orientation,  extrinsic  goal  orientation,  task  value,  control 
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of  learning  beliefs,  self-efficacy  for  learning  and  performance,  and  test  anxiety.  Learning  strategies  are  measured 
by  nine  subscales:  rehearsal,  elaboration,  organisation,  critical  thinking,  metacognitive  self-regulation,  time  and 
study  environment  management,  effort  regulation,  peer  learning  and  help-seeking.  The  fifteen  subscales  are 
designed  to  be  modular,  that  is  measurement  of  all  or  specific  strategies  can  be  made  dependent  on  the  needs  of 
the  researcher  or  instructor  for  the  course  in  question. 

In  this  study  the  course  that  students  have  been  selected  from  is  a second  level  60  credit  course  run  at  the  OU  - 
SD206:  Biology:  brain  and  behaviour.  The  course  aims  to  teach  students  neurobiological  structure  and 
functioning  and  was  chosen  because  students  have  the  option  of  using  a computer  based  resource  learning  tool  - 
The  "Human  Brain"  CD-ROM.  The  CD-ROM  has  been  developed  in-house  at  the  OU  by  the  Biology 
Multimedia  Group.  Its  main  purpose  is  to  marry  together  the  teaching  of  neurobiological  functioning  and 
structure  in  a medium  that  is  able  to  present  detailed  information  in  a format  that  is  easy  to  use  and  affords 
understanding.  The  tool  is  anticipated  to  provide  approximately  30  hours  of  study  time.  It  makes  use  of  various 
features  including  narration,  text,  video,  animations.  Navigation  is  such  that  each  topic  is  organised  into  three 
levels  of  increasing  complexity.  The  topics  the  tool  covers  include:  general  aspects  of  the  central  nervous 
system,  the  cerebral  cortex,  the  spinal  cord,  vision,  hearing  and  speech.  The  tool  also  allows  the  student  to  assess 
their  knowledge  via  a series  of  multiple  choice  questions.  The  second  phase  aims  to  examine  the  learning 
strategies  students  adopt  when  using  the  CD-ROM  produced  for  SD206. 

In  June  of  1998  an  amended  version  of  the  MSLQ,  measuring  all  fifteen  subscales,  was  posted  to  a sample  of 
471  students  studying  the  SD206  course.  After  a period  of  two  months  those  that  responded  to  the  first 
questionnaire,  306  students,  were  sent  the  questionnaire  again.  From  this  second  posting  96  cases  have  been 
matched;  of  these,  approximately  half  are  using  the  CD-ROM.  This  exercise,  in  part,  aimed  to  explore  the 
learning  strategies  of  students  in  their  general  study  of  the  course.  Amendments  made  to  the  questionnaire 
included  changes  in  terminology  (e.g.  class,  instructor)  to  make  the  questions  applicable  to  students  studying  at  a 
distance.  Also  each  question  incorporated  the  course  code,  the  common  referencing  term  for  the  course  being 
studied.  It  was  intended  that  this  tactic  would  direct  the  responses  of  students  to  the  course  in  question  in  lieu  of 
the  cues  suggested  by  Pintrich  and  Garcia.  For  example: 

Original  Question  - In  a class  like  this,  I prefer  course  material  that  really  challenges  me  so 

that  / can  learn  new  things 

Revised  Question  - / prefer  course  material  for  SD206  that  really  challenges  me  so  that  / 

can  learn  new  things 

From  this  intial  phase  the  results  indicate  relatively  high  correlations  between  the  subscales  on  both 
questionnaires.  This  would  seem  to  indicate  a fair  degree  of  reliability  and  stability  in  the  subscale  measures.  In 
examining  the  subscale  scores,  findings  indicated  that  students  taking  the  course  are  highly  motivated  and 
making  good  use  of  learning  strategies  in  their  study.  In  many  respects  the  results  of  this  distance  education 
sample  mirror  the  results  of  students  studying  via  traditional  methods  in  American  colleges.  Also,  at  this  early 
stage,  those  using  the  CD-ROM  indicate  that  the  tool  is  easy  to  use  and  praise  its  development. 

The  second  phase  of  the  research  is  intended  to  build  on  the  intial  phase  and  its  findings  and  explore  the 
strategies  that  students  adopt  in  using  the  CD-ROM.  This  next  step  will  be  to  observe  and  interview  students 
using  the  tool  and  identify  the  strategies  they  come  to  use  based  on  the  framework  developed  by  Pintrich  et  al. 


Conclusion 

Our  understanding  of  student  learning  has  undergone  a series  of  changes,  from  initial  notions  of  cognitive  style 
and  learning  style  to  the  present  day  emphasis  on  examining  student  learning  strategies.  Measures  of  style  are 
still  common  today  but  the  extent  to  which  they  help  us  understand  the  complexity  of  student  learning  remains 
uncertain.  Learning  strategies  present  a move  away  from  the  classification  of  learners  towards  a classification  of 
learning.  Currently,  evaluation  practice  fails  to  adequately  understand  the  nature  of  the  user's  learning  processes. 
The  current  research  serves  to  remedy  this  by  implementing  a learning  strategies  framework  in  the  evaluation  of 
learning  processes  that  occur  whilst  learning  with  a computer.  It  is  intended  that  the  subsequent  research  findings 
will  inform  not  only  evaluation  practice  but  also  courseware  design. 
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Abstract:  There  is  a great  need  to  improve  the  accessibility  of  Web  sites  for  people  with 
disabilities.  This  paper  suggests  a five-step  process,  drawing  upon  our  experience  at  Educational 
Testing  Service  (ETS).  For  each  step,  we  offer  suggestions.  The  steps  are:  (1)  Initiate  a Web 
accessibility  project;  (2)  Identify  accessibility  problems  using  automated  tools  and  human 
evaluators;  (3)  Identify  possible  solutions,  such  as  providing  education  and  training,  making 
available  templates  and  examples,  utilizing  new  technologies,  and  forging  relationships  with 
outside  experts;  (4)  Select  and  implement  a solution;  (5)  Obtain  support  for  continuous 
improvement.  In  practice,  these  five  steps  may  be  somewhat  concurrent  and  may  be  repeated. 


Introduction 

Many  Web  sites  are  partially  or  completely  inaccessible  to  people  with  disabilities.  For  example,  many 
Web  pages  implement  graphics  and  audio  clips  in  ways  that  are  inaccessible  to  people  with  visual  and  hearing 
disabilities.  Also,  the  written  text  on  many  Web  sites  uses  language  that  is  unclear  or  unnecessarily  complex, 
making  it  inaccessible  to  some  people  who  have  language-related  disabilities,  such  as  cognitive  disabilities,  learning 
disabilities,  and  deafness. 

Fortunately,  Web  developers  can  make  their  sites  accessible  to  people  with  disabilities.  For  example,  pages 
with  images  can  be  made  accessible  to  people  who  are  blind  if  appropriate  text  alternatives  are  provided.  For 
instance,  if  a Web  page  uses  an  image  of  a flower,  the  Web  developer  can  insert  a text  description  of  the  flower  into 
the  “alt”  attribute  of  an  image  (IMG)  element.  The  Web  user  who  is  blind  can  then  use  speech  output  technology  to 
hear  the  text  alternative.  By  making  this  and  other  provisions,  Web  developers  can  ensure  that  Web  sites  are 
accessible  to  people  with  a variety  of  disabilities,  including  blindness,  deafness,  deaf-blindness,  and  many  other 
disabilities. 

However,  it  is  not  necessarily  easy  to  change  current  Web  development  practices  within  an  organization. 
Web  site  development  in  many  organizations  is  a complex  undertaking  involving  many  individuals  and  teams. 

Individuals  who  take  the  role  of  change  agent  need  to  be  aware  of  the  how  best  to  balance  the  technical, 
educational,  and  advocacy  aspects  of  the  role.  Such  individuals  can  benefit  from  learning  how  others  have  attempted 
to  address  the  challenge  of  improving  Web  site  accessibility  within  their  organizations. 

Background 

The  main  Web  site  of  Educational  Testing  Service  (ETS),  "ETS  Net"  (www.ets.org),  became  available  to 
the  public  in  May  1996.  The  site  includes  links  to  other  sites,  such  as  those  of  the  Graduate  Record  Examinations® 
(GRE®)  program,  the  Advanced  Placement  Program®  (AP®),  the  Graduate  Management  Admission  Test® 
(GMAT®)  program,  the  College  Board®,  and  others.  These  sites  provide  information  on  how  to  prepare  for  tests, 
how  to  register  for  them,  and  how  to  obtain  special  accommodations  (extra  time,  a sign  language  interpreter,  etc.). 
Other  features  of  ETS  sites  may  include  online  registration,  practice  tests,  and  financial  aid  information. 

Several  ETS  staff  expressed  an  interest  in  ensuring  that  our  Web  sites  were  accessible  to  all  users.  We  did 
not  possess,  individually  or  collectively,  all  the  expertise  needed  to  guide  the  corporation  in  achieving  this  goal.  We 
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determined  that  it  would  take  a specific  project  to  pull  together  the  expertise  and  resources  with  which  to  ensure  the 
accessibility  of  our  Web  sites. 

In  mid- 1996  we  sought  and  obtained  internal  corporate  support  for  the  ETS  Web  Access  Project  and  began 
organizing  our  team.  From  then  until  now  (March  1999)  we  have  carried  out  a number  of  activities.  On  the  basis  of 
our  experience  with  these  activities,  we  feel  that  we  can  offer  suggestions  that  may  be  useful  to  others  who  see 
similar  needs  in  their  organizations. 

Purpose 


The  purpose  of  this  paper  is  to  describe  a process  that  an  organization  can  follow  to  improve  the 
accessibility  of  its  Web  sites. 


Method 

The  method  for  describing  the  process  is  to  break  it  down  into  five  steps,  as  follows:  (1)  Initiate  a project; 
(2)  Identify  accessibility  problems;  (3)  Identify  possible  solutions;  (4)  Select  and  implement  a solution;  (5)  Obtain 
support  for  continuous  improvement. 

At  what  point  in  this  process  is  ETS’s  Web  Access  Project?  It  is  difficult  to  say.  Since  the  project  is  under 
consideration  for  its  third  year  of  funding  (from  ETS  Research),  one  would  judge  that  we  are  in  step  five.  The  fact 
that  we  are  beginning  to  implement  some  solutions  suggests  that  we  are  in  step  four.  Yet  in  other  areas  we  feel  that 
we  are  very  much  at  the  beginning  (step  one).  Thus  it  is  difficult  to  characterize  the  entire  project  as  being  at  a single 
point  in  the  process.  We  hope  that,  notwithstanding  these  ambiguities,  the  five-step  process  will  serve  as  a useful 
framework  for  organizing  our  suggestions. 


The  Process 

1.  Initiate  a Web  Accessibility  Project 

There  are  several  activities  that  may  help  an  organization  launch  a Web  accessibility  project. 

Communicate  the  Benefits 

Making  Web  sites  accessible  is  a practical  step  toward  providing  "equity"  and  "access"  in  the  delivery  of 
products  and  services.  Improving  accessibility  of  Web  sites  will  also  benefit  many  individuals,  beyond  those  who 
have  disabilities.  An  example  of  "side-benefits"  for  nondisabled  people  is  found  in  the  case  of  curbcuts  on 
sidewalks.  They  were  intended  primarily  to  benefit  individuals  with  wheelchairs,  but  they  also  benefit  parents 
pushing  strollers,  people  temporarily  on  crutches,  and  older  people  for  whom  stepping  on  or  off  a curb  is  difficult. 
Following  accessibility  guidelines  in  Web  sites  will  similarly  benefit  many  individuals  who  do  not  have  disabilities, 
including  individuals  with  slow  modems  (who  often  use  the  Web  with  the  graphics  turned  off),  text-only  Web 
browsers,  or  telephone-based  browsers  (which  use  Touch-Tone  signals  for  input  and  provide  audio  output).  The 
work  would  also  benefit  individuals  who  use  search  engines,  which  generally  rely  on  indexing  of  text  to  find  Web 
content.  Thus,  in  addition  to  just  being  the  right  thing  to  do,  improving  accessibility  of  Web  sites  can  increase  the 
base  of  clients  who  are  able  to  use  an  organization’s  products  and  services. 

Obtain  Organizational  Support 

Obtain  support  from  the  organization.  Our  project  has  received  internal  research  -area  funds  for  the  past 
two  years  and  will,  hopefully,  do  so  for  a third  year.  As  the  effort  continues,  we  hope  to  find  additional  sources. 

Ensure  Wide  Representation  on  the  Project  Team 

Ensure  that  the  project  team  includes  individuals  with  a variety  of  expertise  and  backgrounds.  The  Web 
Access  Project  team  has  included  members  from  different  areas  within  ETS.  Areas  of  expertise  represented  include 
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publishing  (paper  and  electronic),  systems  analysis,  instructional  design,  disability  access,  research  design,  and  other 
domains.  Individuals  with  disabilities  , such  as  those  with  blindness,  serve  as  team  members  and  consultants  on  the 
project. 

Review  Existing  Accessibility  Guidelines 

Review  existing  guidelines  for  Web  accessibility.  The  most  authoritative  document  on  the  subject  is  the 
“Web  Content  Accessibility  Guidelines,”  which  is  a proposed  recommendation  of  the  World  Wide  Web  Consortium 
(W3C)  (Chisolm,  Vanderheiden,  & Jacobs,  1999). 

2.  Identify  Accessibility  Problems 

Identify  and  document  accessibility  problems  according  to  a systematic  evaluation  methodology,  using 
both  human  evaluators  and  automated  tools.  Human  and  automated  methods  for  identifying  accessibility  problems 
complement  each  other.  Human  evaluators  are  more  expensive  to  use  but  can  help  researchers  identify  problems  that 
automated  tools  generally  cannot,  such  as  unclear  labeling  of  links,  unnecessarily  difficult  vocabulary  and  grammar, 
inconvenient  page  layout,  distracting  visual  stimuli,  ineffective  search  capabilities,  cumbersome  navigation,  and 
privacy  concerns.  Automated  tools,  on  the  other  hand,  are  able  to  identify  some  of  the  most  critical  problems  (e.g., 
lack  of  alternative  text  for  graphics)  very  quickly  and  at  a low  cost. 

Use  Human  Evaluators 

Have  human  evaluators  go  through  the  Web  site.  These  evaluators  should  include  (a)  individuals  with 
disabilities  and  (b)  experts  in  disability  access.  ETS  has  engaged  people  with  disabilities  to  evaluate  the  accessibility 
of  its  Web  pages.  An  evaluation  methodology  called  “heuristic  evaluation”  was  a useful  starting  point  in  our  work 
with  human  evaluators  (Hansen,  Forer,  & Jacequemin,  1998;  Levi  & Conrad,  1996;  Nielson,  1994). 

Use  Automated  Methods 

Use  automated  tools  for  validating  accessibility.  For  example,  "Bobby"  is  a software  tool  that  automatically 
evaluates  the  accessibility  of  Web  pages.  Bobby  has  features  that  make  it  an  essential  tool  for  Web  developers  who 
want  to  ensure  accessibility  of  their  Web  sites. 

One  can  begin  using  Bobby  by  going  to  the  site  for  the  Center  for  Applied  Special  Technology  (CAST), 
www.cast.org/bobby/.  (The  name  "Bobby"  appears  to  be  associated  with  the  site's  graphic  of  a British  police  officer, 
commonly  known  as  a bobby.)  The  user  types  in  the  URL  of  the  page  that  he  or  she  wants  to  analyze  and  then  clicks 
the  submit  button  (or  simply  presses  Enter).  Usually  within  half  a minute,  Bobby  analyzes  the  page  and  returns  a 
report  listing  accessibility  problems  and  showing  their  locations  on  the  page.  Bobby  is  especially  effective  at 
identifying  problems  that  hinder  the  effective  use  of  speech  output  technology  (e.g.,  "screen  readers")  by  people  who 
are  blind  or  have  other  language-related  disabilities. 

A downloadable  version  of  Bobby  has  some  special  advantages.  It  can  be  used  to  evaluate  accessibility  of 
Web  pages  that  are  behind  organizational  firewalls.  One  can  also  analyze  many  pages  at  a time  rather  than  one  at  a 
time.  ETS  staff  have  used  this  version  to  analyze  thousands  of  pages  (Hansen,  1998). 

3.  Identify  Possible  Solutions 

Identify  possible  solutions  or  elements  of  a solution  to  the  problems  that  have  been  identified.  Generally,  a 
workable  solution  will  involve  a combination  of  elements.  Depending  on  the  particular  set  of  problems  being 
addressed,  each  of  the  following  could  represent  either  a solution  or,  more  often,  an  element  of  a solution. 

Educate  and  Train  Staff 

Ensure  that  staff  who  work  with  Web  development  receive  training  that  acquaints  them  with  the  problems 
faced  by  individuals  with  disabilities  who  are  using  Web  sites.  Technical  staff  can  additionally  benefit  from  seeing 
actual  examples  of  HTML  code  that  address  these  problems.  One  source  in  which  to  refer  is  the  W3C/WAI  “Web 
Accessibility  Content  Guidelines”  (Chisolm,  Vanderheiden,  & Jacobs,  1999). 


Staff  who  are  members  of  a Web  accessibility  project  team  should  receive  significant  opportunities  as  well. 
Some  ETS  Web  Access  Project  staff  have  received  opportunities  to  both  provide  and  receive  instruction  on 
accessibility  issues.  They  have  gained  expertise  and  have  shared  their  knowledge  with  other  ETS  staff  through  talks, 
presentations,  and  consultations.  Some  project  team  members  have  also  presented  material  on  the  subject  at  national 
conferences  (Hansen,  Forer,  Jacquemin,  1998;  Hansen,  Katz,  & Forer,  1997;  Katz,  1997).  Thus,  project  staff  are 
gaining  expertise  that  will  allow  them  to  become  resources  to  the  organization  and  to  others. 

Make  Available  Templates  and  Examples 

Make  available  templates  and  examples  of  accessible  Web  site  components.  On  our  ETS  Web  Access 
Project  Web  site,  we  have  accessible  versions  of  Web  components,  such  as  practice  test  questions  that  provide  text 
descriptions  for  graphics  (e.g.,  charts,  tables,  graphs)  and  that  eliminate  certain  problems  caused  by  use  of  tables  and 
form  buttons.  We  have  implemented  one  version  of  an  accessible,  alternative  version  of  the  ETS  front  door 
(www.ets.org)  and  have  helped  develop  templates  that  will  be  used  in  many  other  pages. 

New  Technologies 

Conduct  ongoing  investigation  into  new  technologies  that  may  provide  important  accessibility  benefits. 
Among  the  most  important  are  HTML  4.0,  Cascading  Style  Sheets  (CSS),  SMIL,  Extensible  Markup  Language 
(XML),  Synchronized  Multimedia  Integration  Language  (SMIL),  and  Mathematical  Markup  Language  (MathML). 

Forge  Relationships  With  Outside  Experts 

Forge  relationships  with  outside  experts  and  their  organizations.  Many  organizations  wishing  to  improve 
Web  site  accessibility  will  benefit  from  hiring  outside  consultants.  Members  of  the  Web  Access  Project  worked  with 
a number  of  outside  experts. 

4.  Select  and  Implement  a Solution 

Select  and  implement  a solution  that  is  ambitious  yet  recognizes  limitations  of  time,  expertise,  and 
resources.  Develop  a strategy  for  making  a significant  difference,  but  realize  that  not  every  positive  change  can  take 
place  immediately.  For  example,  certain  lower-priority  problems  may  need  to  go  unfixed  for  a while  in  order  to 
allow  staff  to  focus  on  preventing  problems  in  new  Web  content.  Other  tradeoffs  may  involve  determining  how  to 
take  advantage  of  new  technologies  while  providing  adequate  support  for  users  of  older  technologies. 

Persistence  in  the  accessibility  effort  is  important.  The  design,  development,  and  maintenance  of  Web  sites 
involve  many  parts  of  an  organization,  and  change  may  be  slower  than  desired  or  expected.  Changes  to  some  sites 
may  be  frequent,  so  it  is  important  to  check  sites  frequently  for  accessibility. 

5.  Obtain  Support  for  Continuous  Improvement 

Obtain  support  for  continuous  improvement.  The  following  are  a few  ways  to  help  achieve  and  maintain 
continuous  improvement. 

Share  and  Celebrate  Successes 

In  a recent  article  that  appeared  last  year  our  ETS  corporate  newsletter  ("Project  aims  at  accessible 
websites"),  the  Web  Access  Project  team  members  shared  their  successes  and  discussed  areas  that  need 
improvement.  This  article  has  helped  raise  staff  consciousness  of  the  accessibility  issue  and  at  the  same  time 
credited  project  staff  for  their  efforts.  The  team  has  been  discussing  ways  of  publicly  recognizing  ETS  Web  sites 
that  achieve  a high  level  of  accessibility,  and  hopes  that  doing  so  will  help  make  striving  for  accessibility  something 
that  is  motivating  and  fun  rather  than  a burden. 

Conduct  Ongoing  Training  and  Education 

Conduct  ongoing  training  and  education  in  technology  and  accessibility  issues.  We  are  confident  that  with 
Web-related  technology  and  markets  changing  so  rapidly,  ETS  staff  will  need  to  receive  ongoing  training  and 
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education  to  ensure  improving  levels  of  accessibility  of  Web-based  products  and  services.  From  time  to  time,  if 
large-scale  changes  or  improvements  are  needed,  it  may  be  necessary  to  repeat  all  five  steps  of  the  cycle. 

A desirable  outcome  of  such  training  and  education  would  be  for  people  developing  products  and  services 
to  ask  themselves,  "How  would  a user  interact  with  the  system  if  he  or  she  had  a visual,  hearing,  learning,  physical, 
or  other  disability?"  After  answering  this  question,  developers  would  then  try  to  provide  alternative  media  or 
methods  to  allow  individuals  with  disabilities  to  obtain  information  from  a given  site. 

Involve  Individuals  With  Disabilities 

We  strongly  recommend  directly  involving  individuals  with  disabilities  in  all  aspects  of  Web  accessibility 
projects,  making  them  project  leaders  as  well  as  participants  or  evaluators.  Individuals  with  disabilities  encounter 
real  accessibility  problems  when  using  Web  sites.  While  many  of  the  problems  are  documented,  being  able  to 
witness  the  difficulties  or  to  interact  directly  with  individuals  who  have  faced  difficulties  will  add  motivation  and 
understanding  to  your  effort. 

Maintain  a Win-Win  Philosophy 

Avoid  developing  an  "us  versus  them"  mentality.  As  we  have  attempted  to  facilitate  change  in  the 
organization,  we  have  tried  to  avoid  placing  blame  for  our  current  Web  accessibility  problems.  The  technologies  are 
new,  the  standards  are  new,  and  we  are  only  beginning  to  understand  how  to  make  Web  sites  more  accessible.  We 
emphasize  the  fact  that  we  are  all  in  this  together,  that  we  are  all  learning,  and  that  by  working  together  we  can  make 
a positive  difference  in  the  lives  of  many  people.  We  take  the  approach  that  we  will  all  be  winners  if  our  Web  sites 
are  accessible.  Beyond  simply  doing  the  right  thing,  making  Web  sites  accessible  can  generate  good  will  and 
appreciation  from  customers  and  also  may  create  a competitive  advantage  when  dealing  with  the  increasing  number 
of  clients  and  potential  clients  who  value  accessibility  in  information  products  and  services. 

Develop  Consensus  to  Establish  Organizational  Policies 

Develop  consensus  among  relevant  constituencies  to  establish  an  organizational  Web  accessibility  policy. 
This  policy  should  define  minimum  requirements  for  accessibility  of  Web  sites.  ETS  project  staff  are  now  working 
with  the  corporate  Office  of  Disability  Policy  and  representatives  from  across  the  corporation  to  prepare  a corporate 
Web  accessibility  policy  document.  We  plan  to  continue  to  survey  the  accessibility  of  ETS’s  Web  sites  periodically 
with  both  human  and  automated  raters  in  order  to  track  our  progress  and  provide  feedback  to  Web  developers  and 
others.  We  believe  that,  with  proper  support  and  learning  opportunities,  the  programs  will  try  to  exceed  the 
minimum  standards  when  possible. 


Conclusion 

A highly  effective  Web  accessibility  project  will  find  that  elusive  balance  among  research,  development, 
and  communication.  Conducting  research  to  identify  problems  using  both  human  evaluators  and  automated  methods 
such  as  Bobby  has  been  enormously  helpful.  Development  of  accessible  Web  components  and  sites  (our  own 
accessible  Web  Access  Project  intranet  site)  has  been  important  because  it  helps  demonstrate  that  we  can  do  what 
we  are  saying  others  ought  to  do.  Communication  of  findings  and  results  is  an  important  challenge.  One  ought  not 
work  for  long  periods  without  presenting  results  that  can  be  understood  and  appreciated  by  individuals  outside  the 
project. 

Efforts  to  improve  the  accessibility  of  the  Web  for  people  with  disabilities  can  make  a difference  for 
millions  of  people  throughout  the  world.  Even  with  the  problems  that  individuals  with  disabilities  encounter  with 
our  Web  sites,  we  found  that  they  generally  loved  the  fact  that  they  had  ready  access  to  so  much  information.  Our 
current  efforts  will  help  consolidate  and  extend  the  benefits  that  Web  users  with  disabilities  have  been  receiving. 
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Abstract:  In  researching  educational  applications  of  virtual  reality,  it  is  now  possible  to 
place  several  students  within  a single  virtual  environment  (VE)  simultaneously.  This  raises 
questions  regarding  the  impact  of  collaboration  within  virtual  environments  on  overall 
learning  processes.  A preliminary  study  of  1 10  sixth  and  ninth  grade  students  was  conducted 
for  the  purpose  of  examining  peer  collaboration  within  a VE.  Students  worked  in  pairs  while 
investigating  the  concept  of  global  warming  within  a fully  immersive,  3-D,  virtual  reality 
based  model  of  the  city  of  Seattle,  Washington,  called  Global  Change  World  (GCW).  It  is 
concluded  that  most  students  thoroughly  enjoyed  and  valued  their  experience  with  GCW. 
Continuing  research  will  focus  more  closely  upon  the  impact  of  peer  collaboration  on 
preconceived  or  “naive”  scientific  concepts  as  well  as  the  possible  conceptual  change 
inspired  by  collaboration  within  GCW.  This  preliminary  study  has  successfully 
demonstrated  the  potential  for  GCW  to  facilitate  collaborative  learning  experiences. 


The  application  of  virtual  reality  (VR)  to  the  educational  process  is  becoming  more  commonplace,  with  large 
numbers  of  students  of  all  ages  experiencing  virtual  environments  (VEs)  in  educational  settings.  For  example, 
Youngblut  (1998)  identifies  over  forty  examples  of  VR  applications  that  are  specifically  designed  to  support 
learning.  These  learning  experiences  can  come  in  many  forms,  from  virtual  worlds  created  via  VRML  to  fully 
immersive  VEs  that  utilize  stereoscopic  headmounted  display  systems.  Moreover,  with  ever-increasing 
computational  processing  power,  the  rapid  growth  of  the  World  Wide  Web,  and  the  ongoing  construction  of  a 
digital  communications  infrastructure,  the  creation  of  distributed  fully  immersive,  multi-participant  VEs 
running  on  the  Internet  are  on  the  technological  horizon.  Given  the  growing  interest  in  distance  learning  and 
distributed  education  among  educators,  it  is  fair  to  assume  that  many  fully  immersive,  multi-participant, 
educational  VEs  will  come  on  line  as  the  technology  becomes  less  expensive  and  more  readily  available. 

Most  educational  applications  of  VR  are  designed  to  make  use  of  its  unique  features.  These  features  provide 
opportunities  for  students  to  gain  a greater  understanding  of  abstract  concepts  through  the  creation  of  visual 
metaphors  or  representations  and  the  ability  to  scale  and  manipulate  these  representations.  Working  with 
these  representations  allows  students  to  investigate  relationships  between  objects  in  VEs  which  are  unbounded 
by  distance,  time,  or  safety  concerns.  Designers  of  educational  VEs  believe  that  students  retain,  master  and 
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generalize  new  knowledge  better  when  given  the  opportunity  to  become  actively  involved  in  constructing  that 
knowledge  through  a coherent,  firsthand  interaction  with  knowledge  domain  representations  (Winn,  1993). 

As  it  becomes  possible  to  place  more  than  one  student  within  a VE,  questions  arise  regarding  the  potential 
impact  of  collaboration  on  learning.  The  purpose  of  this  paper  is  to  report  the  progress  of  a proof-of-concept 
project  which  is  being  conducted  by  members  of  the  University  of  Washington’s  College  of  Education  and  the 
Learning  Center  at  the  Human  Interface  Technology  Laboratory  (HITL)  at  the  University  of  Washington. 
Funded  by  a grant  from  the  UW  Royalty  Research  Fund,  this  project  seeks  to  investigate  the  nature  of  peer 
collaboration  within  VEs.  We  describe  an  investigation  that  looked  at  how  easily  school  children  adapted  to 
working  together  towards  the  ultimate  goal  of  developing  and  testing  hypotheses  regarding  environmental 
factors  that  affect  climate  change  in  a VE  designed  for  use  in  science  education. 


BACKGROUND 

Arriving  at  an  understanding  of  exactly  how  to  use  immersive  VR  to  support  the  teaching  and  learning  of 
abstract  concepts  continues  to  be  a challenging  and  elusive  goal  for  researchers  of  virtual  learning 
environments  (VLEs).  The  potential  for  VR  to  benefit  education  is  widely  recognized  and  a number  of  studies 
have  been  conducted  that  have  demonstrated  a capability  to  teach  content  using  VR  under  certain  prescribed 
conditions  (e.g.  Byrne,  1996;  Dede,  1995;  McLellan,  1996;  Osberg,  1997;  Winn,  1997;  Youngblut,  1998).  A 
significant  challenge,  however,  remains  in  fusing  of  the  affordances  of  VLEs  and  educational  methods  of 
exploiting  them  into  a demonstrable  theory  of  learning  for  VR. 

Zeltzer  (1992)  refers  to  the  three  attributes  of  autonomy,  presence,  and  interaction  in  describing  the 
affordances  VR  provides.  Autonomy  refers  to  the  notion  that  a virtual  environment  (VE)  is  to  some  extent 
capable  of  performing  its  own  actions,  independent  of  user  intervention.  An  autonomous  VE  follows  its  own 
path  to  goals  and  may  or  may  not  change  course  in  response  to  user  actions.  Presence  is  simply  the  experience 
the  user  has  of  being  in  a real  place  when  immersed  within  a VE.  Zeltzer  claims  that  for  presence  to  be  high, 
the  user  must  be  allowed  to  interact  with  the  VE  both  naturally  and  intuitively;  when  presence  is  high  the 
computer  interface  disappears.  Finally,  interaction  involves  the  ability  of  the  user  to  perform  actions  in  the  VE 
according  to  a logical  rationale.  Even  though  the  user  may  have  to  learn  how  to  function  appropriately  within 
the  environment,  the  laws  that  govern  the  VE  should  become  apparent  over  time,  allowing  for  a meaningful 
interactive  experience. 

Most  educational  applications  for  VR  are  designed  to  make  use  of  these  and  other  unique  features  and 
affordances.  Other  affordances  include:  1.  Allowing  students  to  gain  a greater  understanding  of  abstract 
concepts  through  the  creation  of  visual  metaphors,  2.  Allowing  students  to  directly  manipulate  and  scale 
virtual  objects  or  environments  for  clearer  understandings,  and  3.  Allowing  students  to  visit  places  and 
interact  with  events  that  distance,  time,  or  safety  concerns  would  normally  prohibit  (Winn,  1993,  Youngblut, 
1998).  Designers  of  VLEs  tend  to  believe  that  students  retain,  master  and  generalize  new  knowledge  better 
when  given  the  opportunity  to  become  actively  involved  in  constructing  that  knowledge  through  a coherent, 
firsthand  interaction  with  knowledge  domain  representations. 

Much  of  the  appeal  for  applying  VR  in  education  is  derived  from  the  observations  of  educational  theorists  like 
Bruner  (1986)  and  Piaget  (1929)  who  have  long  stressed  the  value  of  actualizing  learning  through  making  it 
more  real  for  students.  VR  technology  allows  for  the  creation  of  a VLEs  where  students  can  learn  by 
interacting  with  virtual  objects  similar  to  how  they  would  interact  with  real  objects.  Through  immersion  in  a 
VE,  students  become  a part  of  the  phenomena  that  surround  them.  Learning  is  facilitated  through  the 
construction  of  concepts  built  from  the  intuitions  that  arise  out  of  their  direct  experience  of  the  environment. 
More  recently,  research  (Clancey,  1993;  Bricken,  1990)  has  supported  the  notion  that  VLEs,  by  their  very 
nature,  increase  the  human  capacity  for  certain  types  of  learning  by  allowing  users  to  cross  the  boundary 
between  third  and  first  person  experience,  negating  the  need  for  a highly  abstract  symbol  system.  In 
traditional  education,  learning  the  symbol  system  of  a particular  knowledge  domain  is  often  a prerequisite  to 
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learning  its  content,  as  in  the  case  of  mathematics  or  music  (Winn,  1993).  The  problem  with  this  type  of 
learning  is  that  mastery  of  the  symbol  system  can  often  be  mistaken  for  mastery  of  the  content,  and  teaching 
may  end  well  before  students  make  the  link  between  the  two. 

First  person,  or  direct,  interaction  within  a VE  allows  students  to  construct  knowledge  out  of  their  own 
experience  without  relying  on  symbol  systems.  This  concept  of  knowledge  construction  among  learners  is 
more  generally  referred  to  as  constructivism  (Duffy  & Jonassen,  1992).  Self-constructed  knowledge  is  highly 
individualized  and  may  represent  an  improvement  over  similar  knowledge  learned  by  other  methods  because 
the  learners  shape  the  learning  experience  themselves.  In  other  words,  instead  of  relying  on  third-person 
instructor  or  text-based  accounts  of  how  things  occur  in  the  world,  students  immersed  within  VLEs  can 
directly  experience  and  interact  with  the  concepts,  principles,  rules,  and  procedures  found  in  the  domain. 
VLEs  designed  from  the  constructivist  approach  are  seen  by  some  as  having  great  potential  for  providing 
powerful  learning  experiences  (Bricken,  1990). 

Constructivism  has  entertained  a long  history  in  education  and  philosophy  and  is  representative  of  a wide 
diversity  of  views  that  may  be  summed  up  in  the  following  points:  1.  Learning  is  an  active  process  of 
constructing  rather  than  acquiring  knowledge,  and  2.  Instruction  is  a process  of  supporting  that  construction 
rather  than  communicating  knowledge  (Duffy  & Cunningham,  1 996).  While  constructivism  is  seen  to  imply 
that  there  is  a tangible  world,  it  argues  that  it  is  individuals  who  impose  meaning  on  that  world. 
Consequently,  there  can  be  many  meanings  or  perspectives  for  any  event  or  concept  and  the  goal  of  education 
can  no  longer  be  one  of  instilling  an  absolute,  correct  meaning  (Duffy  & Jonassen,  1992).  More  importantly, 
constructivism  brings  with  it  the  underlying  assumption  of  a learner-centered  approach  to  instruction. 
Constructivist  arguments  are  often  used  to  defend  the  design  and  implementation  of  VLEs. 

Traditionally,  most  VLEs  have  placed  a single  student  within  the  VE.  However,  as  the  technology  becomes 
available  educators  are  becoming  interested  in  investigating  the  potential  for  collaborative  learning  in  VEs. 
There  has  been  considerable  research  on  the  value  of  collaborative  learning.  O’Malley  (1995)  found  that 
much  of  the  research  on  collaborative  learning  has  evolved  from  the  works  of  Piaget  (1985)  and  Vygotsky 
(1978).  Crook  (1994),  for  example,  views  peer  collaboration  as  having  three  basic  cognitive  benefits; 
articulation,  conflict,  and  co-construction.  According  to  Crook,  peer  collaboration  forces  students  to  make 
their  ideas  explicit  and  public.  To  do  so,  they  need  to  learn  to  clearly  articulate  their  opinions,  predictions,  and 
interpretations.  Conflict  may  arise  when  students  disagree  in  regards  to  their  interpretations.  To  resolve  the 
conflict  engendered  by  collaboration,  they  must  justify  and  defend  their  positions  and  are  thus  forced  into 
reflection.  Piaget  (1985)  offered  a similar  view,  noting  that  socio-cognitive  conflict  often  arises  when  students 
holding  inadequate  or  differing  views  work  collaboratively.  As  these  differing  views  are  sorted  out,  students 
are  forced  to  reflect  upon  their  own  conceptions.  Crook’s  concept  of  co-construction  is  based  upon  Vygotsky’s 
(1978)  belief  that  learning  is  the  sharing  of  meaning  in  a social  context.  Students  collaboratively  co-construct 
shared  knowledge  and  understanding  by  building  upon  each  other’s  ideas.  Given  the  ability  of  VLEs  to 
support  multi-participant  activities,  it  is  easy  to  see  why  educators  are  very  interested  in  examining  the 
potential  for  using  them  to  support  collaborative  learning. 


GLOBAL  CHANGE  WORLD 

Global  Change  World  (GCW)  is  a VLE  that  was  designed  and  programmed  by  the  Learning  Center  of  the 
Human  Interface  Technology  Lab  (HITL)  on  the  University  of  Washington  campus.  The  GCW  environment  is 
created  by  two  networked  Hewlett-Packard  9000  workstations  running  DVISE  VR  software.  Hardware 
manufactured  by  Division  provides  the  physical  user-interface  and  consists  of  an  audio/visual  rendering  system 
connected  to  a headmounted  display  helmet  (HMD),  a navigation  and  control  wand,  and  a position  tracking 
system.  GCW  is  capable  of  supporting  peer  collaboration  by  allowing  more  than  one  student  to  be  immersed 
and  interactive  within  the  same  VE  at  the  same  time. 
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In  GCW,  student  pairs  enter  a virtual  model  of  Seattle  in  the  current  year.  They  are  able  to  navigate  their  way 
around  the  world  using  the  wand  while  viewing  the  world  through  the  stereoscopic  HMD.  Within  the  world, 
the  virtual  representation  of  each  of  the  participants  appears  to  the  other  as  a cartoonish  pair  of  large  eyes, 
spiral  ears,  a triangular-shaped  mouth,  and  a singular  cyber  hand  with  which  they  can  manipulate  objects.  In 
addition,  students  are  able  to  speak  to  and  hear  each  other  by  means  of  an  intercom  system  that  is  built  into  the 
HMD.  In  order  to  perform  tasks  within  GCW,  students  access  a virtual  tool  kit  that  allows  them  to  measure 
air  temperature,  amount  of  greenhouse  gasses  in  parts  per  million,  and  yearly  rainfall.  They  are  able  to  adjust 
such  variables  as  the  amount  of  green  plant  biomass  as  symbolized  by  trees,  the  number  of  factories,  and  the 
number  of  automobiles  present  in  the  world.  After  making  measurements  and  adjustments,  they  can  then  use 
a “time  portal”  to  go  to  a selected  year  in  the  future  and  repeat  their  measurements  in  order  to  determine  the 
impact  of  their  actions  on  the  climate  of  the  future. 

In  preliminary  studies,  GCW  has  been  taken  to  College  Place  Elementary  School  in  Edmonds,  Washington, 
and  Redmond  Junior  High  School  in  Redmond,  Washington  where  a total  of  1 10  students  have  participated  in 
collaborative  VLE  activities.  Through  these  visits  we  hoped  to  gather  information  concerning  the  educational 
effectiveness  of  employing  a multi-participant  VE  as  a science  education  tool.  Our  subjects  were  eighteen  1 1 
and  12  year-old  boys  and  girls  (9  female,  9 male)  from  a single  sixth-grade  class  and  ninety-two  14  and  15 
year-old  boys  and  girls  (47  female,  45  male)  from  three  ninth-grade  class  sections.  Differences  in  navigation 
and  general  task  performance  abilities  between  the  older  students  and  the  younger  students  did  not  appear  to 
be  substantial. 

Our  first  task  was  simply  to  familiarize  the  students  with  the  interface,  scientific  measuring  tasks,  and  basic 
nature  of  GCW.  The  goal  of  this  first  phase  of  the  GCW  peer  collaboration  investigation  was  to  observe  and 
record  the  student  interactions  with  each  other  in  both  world  navigation  and  in  performing  specific  tasks.  As 
noted  earlier,  the  HMDs  were  fitted  with  microphones  so  that  the  students  could  speak  to  each  other  from 
within  the  world.  This  allowed  the  students  to  communicate  with  each  other  in  their  normal  voice  and 
appeared  to  improve  communication,  facilitate  collaboration  and  enhance  feelings  of  presence.  Each  student 
pair  spent  an  average  of  20  minutes  in  GCW. 


RESULTS 

Upon  exiting  GCW,  students  were  given  a nine-question  survey  that  employed  a Likert- type  scale.  The 
questionnaire  asked  them  to  rate  the  quality  of  their  experiences  within  the  VE.  The  results  of  the  survey 
indicated  that  the  students  found  the  experience  to  be  highly  enjoyable  and  most  students  said  they  would  like 
to  repeat  the  experience.  About  5%  of  the  students  reported  malaise  (dizziness  or  disorientation).  A few 
students  reported  problems  with  using  the  wand  and  seeing  clearly  in  the  HMD.  Ratings  of  presence  were 
high  for  most  respondents. 

Clearly,  the  overwhelming  majority  of  students  thoroughly  enjoyed  their  experience  with  GCW  and  most  of 
the  students  felt  highly  immersed  within  the  VE.  It  was  apparent  that  peer  collaboration  played  a significant 
role  in  regards  to  the  level  of  student  engagement  within  the  VE.  The  ability  of  the  students  to  speak  clearly  to 
each  other  while  in  the  VE  seemed  to  greatly  facilitate  peer  collaboration.  Many  pairs  of  students  were  highly 
communicative.  As  they  navigated  through  the  world  or  performed  the  requested  tasks,  they  were  engaged  in 
near  constant  conversation  regarding  where  to  go,  what  to  do  next,  and  how  to  do  things. 

Observation  revealed  substantial  peer  collaboration  among  almost  all  pairings  of  students,  though  some  pairs 
needed  initial  prompting  and  encouragement  to  begin  conversing  with  their  partner.  Expert/expert  pairs  of 
females,  who  were  comfortable  with  the  interface,  were  the  most  vocally  communicative.  Novice/expert  and 
novice/novice  pairings  of  all  gender  combinations  were  somewhat  less  communicative.  Expert/expert  male 
pairings  appeared  to  involve  more  action,  such  as  racing,  but  still  incorporated  a significant  amount  of 
conversation,  as  well  as  the  inclusion  of  vocal  “sound-effects”  that  were  related  to  actions  being  made, 
particularly  among  the  younger  students. 
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Male/female  expert/expert  pairings,  as  well  as  male/female  expert/novice  parings,  resulted  in  what  appeared  to 
be  a more  self-reflective,  thinking  out  loud  style  of  conversation  with  the  experts  doing  far  more  talking. 
These  pairs  would  play  hide-and-seek,  in  the  case  of  experts,  or  simply  try  to  keep  track  of  each  other  by 
talking  about  what  they  were  seeing.  Most  groups  developed  strategies  for  finding  each  other  by  identifying 
prominent  landmarks  and  agreeing  to  meet  at  them.  Some  individuals  picked  up  and  carried  large  brightly 
colored  objects,  such  as  cars  or  boats,  and  waved  them  around  or  carried  them  along  with  them  so  the  other  at 
a distance  could  see  them. 

Though  students  can  fly  through  the  world  with  their  hand  controller,  many  of  them  were  physically  active  as 
well.  Many  students  were  observed  to  be  pointing  at  virtual  objects  with  their  free  hand  in  the  real  world 
while  talking  to  their  companion  in  the  virtual  world.  Several  students  were  quite  surprised  that  they  had 
walked  almost  halfway  across  the  room  when  we  removed  the  HMD;  they  had  assumed  that  they  stood  fairly 
still  while  they  were  flying  around  the  VE.  The  students  who  were  the  most  at  ease  with  the  interface  were  the 
most  animated  in  their  physical  movements  while  immersed  within  GCW. 

A number  of  students  found  GCW  to  be  easy  to  use  and  made  full  use  of  the  tools  provided.  These  subjects 
quickly  mastered  navigational  skills  and  the  use  of  the  environment  measuring  tools  that  allowed  them  to  view 
the  VE’s  current  temperature,  the  amount  of  greenhouse  gases  present,  and  the  amount  of  annual  rainfall. 
While  in  the  world,  they  were  instructed  on  the  use  of  the  wheels  controlling  the  quantity  of  the  VE  variables 
of  cars,  factories,  and  biomass.  These  students  flew  through  the  time  portal,  time  and  time  again,  in  efforts  to 
determine  what  the  impacts  of  their  changes  would  be  and  test  theories  regarding  what  changing  the  variables 
would  do  to  the  environment  over  time.  These  students  typically  rated  the  experience  highly  and  reported  a 
strong  sensation  of  immersion. 

Other  students,  however,  had  more  difficulty  mastering  the  use  of  the  environmental  tool  kit  and  in  trying  to 
come  up  with  theories  for  future  environmental  outcomes  based  upon  changes  made  to  the  variables.  Some 
dyads  never  got  to  the  point  of  being  comfortable  enough  with  the  interface  to  successfully  investigate  the 
environmental  research  elements  of  GCW.  Those  who  did  often  disagreed  on  the  expected  outcomes.  Time, 
as  well  as  the  scope  of  this  preliminary  study,  did  not  allow  us  to  perform  a further  investigation  into  the 
existence  or  nature  of  hypothesis  generation  and  testing  within  GCW.  We  do  know  that  the  students  were 
highly  motivated,  very  willing  to  work  in  pairs,  interested  in  exploring  virtual  space,  and  capable  of 
performing  investigative  tasks  within  GCW.  Further  research  that  focuses  more  closely  on  the  impact  of  peer 
collaboration  on  preconceived  or  “naive”  scientific  concepts  and  the  possible  conceptual  change  inspired  by 
collaboration  within  GCW  is  planned  and  data  collection  will  be  underway  throughout  1999. 


CONCLUDING  REMARKS 

While  the  potential  for  VR  to  facilitate  collaborative  learning  experiences  appears  to  be  great,  much  more 
research  is  needed  before  effective  collaborative  learning  strategies  can  be  developed.  It  is  anticipated  that 
these  strategies  will  vary,  depending  on  the  kind  of  educational  experience  desired  and  the  learning 
environment  employed.  However,  for  this  potential  to  be  realized,  designers  of  hardware,  software,  and 
instruction  must  make  sure  that  it  is  easy  for  multiple  participants  to  collaboratively  navigate  and  perform 
within  VEs.  This  requires  improvements  of  today's  input  devices  including  improvements  in  spatialized  audio 
systems,  less  cumbersome  HMDs,  simpler  wands,  and  the  eventual  introduction  of  haptic  (force  feedback) 
devices.  The  need  for  these  technical  advances  is  essential,  considering  that  the  networking  of  multi- 
participant, collaborative  virtual  environments  appears  to  represent  a significant  trend  for  future  applications 
of  VR  both  within  and  outside  of  the  educational  domain. 
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Abstract:  This  paper  reports  the  preliminary  findings  of  an  investigation  into  the  use  of 
internet  voice  technology  to  support  community  building  for  distance  language  learners.  It  is 
asserted  that  such  technology,  when  chosen  in  accord  with  the  pedagogical  aims  of  the 
learning  activity  designers  adds  value  to  the  overall  learning  experience.  It  empowers  and 
enables  such  learners,  realigns  the  tutor’s  role  in  the  direction  of  co-leamer  and  allows  both 
tutors  and  learners  to  participate  equally  in  the  learning  process. 


Introduction 

The  Open  University  is  the  UK’s  largest  modem  foreign  language  learning  provider  with  a current  enrolment  of 
approximately  8000  students,  all  of  whom  study  individually  at  home,  at  a distance  from  each  other  and  their 
tutors.  While  the  traditional  Open  University  methods  of  delivery  - print,  video,  audio  and  face-to-face  tutorials 
- are  used  by  the  majority  of  students,  alternative  methods  of  delivery  are  being  actively  investigated  for  those 
students  who  require  or  would  prefer  them  by  virtue  of  their  geographical  location  (they  may  be  in  an  area  which 
is  geographically  remote  from  face-to-face  provision)  or  circumstances  (they  may  be  unable  to  attend  face-to- 
face  tutorials  because  they  are,  for  example,  physically  disabled  in  some  way,  in  hospital  or  in  prison).  We 
describe  these  investigations,  outline  the  learning  environments  and  internet-based  tools  offered  to  learners  and 
tutors  and  examine  the  outcomes  of  the  initial  trials  in  terms  of  the  features  we  have  found  to  be  necessary  to 
support  and  maintain  an  online  community  of  students  who  are  located  at  a geographical  distance  from  each 
other. 


Background 

The  research  described  in  this  paper  comprises  part  of  a larger,  long-term  project  seeking  to  establish  a 
framework  for  the  use  of  networking  technologies  in  distance  language  learning  with  a particular  emphasis  on 
the  development  of  oral  - and,  as  a consequence  of  this,  aural  - skills  in  the  target  language.  During  the  lifetime 
of  the  larger  project,  technologies  from  telephone  conferencing  to  voice  conferencing  on  the  internet  have  been 
employed  in  order  to  provide  learners  with  increased  opportunities  for  speaking  and  listening  practice. 

Initially,  learning  activities  were  designed  for  use  with  telephone  conferencing  systems.  These  were  first 
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implemented  in  1995.  Groups  of  6-9  students  met  at  weekly  intervals  in  order  jointly  to  use  the  target  language 
to  participate  in  role-plays  or  other  pre-arranged  learning  activities  and  thus  to  practise  their  communication 
skills  in  tasks  requiring  collaborative  interaction.  This  framework  was  subsequently  widened  to  incorporate  both 
telephone  conferencing  and  e-mail  to  allow  students  to  prepare  the  broad  content  of  what  they  were  going  to  say 
(without  writing  a script),  to  refine  their  positions  for  the  role-plays,  and  to  negotiate  with  each  other  in  the  target 
foreign  language  about  how  to  address  the  fictitious  problem  which  had  been  posed  for  the  activity. 

Results  from  these  trials  [Hewer  & Kotter  forthcoming]  suggest  that  student  behaviour  is  modified  according  to 
the  tools  offered  by  the  learning  environment:  for  example,  it  was  found  that  learners  utilised  e-mail  to  plan  and 
rehearse  their  oral  contributions  in  a way  which  was  not  possible  to  them  when  only  telephone  conferencing 
without  e-mail  support  was  available.  Analysis  of  data  collected  during  the  course  of  that  research  has  also  led 
to  the  hypothesis  that  learner  performance  in  the  foreign  language  improves  when  learners  are  able  to  collaborate 
with  each  other  between  scheduled  events.  For  example,  students  used  e-mail  to  form  small  groups  whose 
members  supported  each  other  in  a way  that  went  well  beyond  what  a tutor  could  offer  - in  other  words,  students 
use  such  learning  environments  not  only  to  carry  out  the  required  task,  but  also  autonomously  to  form  self-help 
groups.  They  employed  e-mail  for  the  following  purposes:  to  enhance  their  knowledge  of  the  vocabulary  of  a 
certain  speech  register  and,  in  order  to  find  out  whether  other  students  were  able  to  make  sense  of  their  language 
output,  to  experiment  with  various  syntactic  and  lexical  forms  of  the  target  language  that  were,  as  yet,  beyond 
their  level  of  competence  (this  latter  behaviour  was  also  observable  in  the  synchronous  oral  exchanges).  In  this 
environment,  learners  were  able  to  receive  rapid  feedback  from  their  peers,  and  this  led  to  an  increased 
willingness  to  take  risks  in  the  foreign  language  and  so  to  develop  fluency  at  a much  quicker  pace  than  might  be 
expected  to  result  from  individual  studies  carried  out  in  isolation. 

A feature  which  is  notably  absent  from  most  distance  language  learning  settings  is  the  option  for  learners  to 
collaborate  with  each  other  outside  inflexible,  scheduled  meeting  times.  While  the  use  of  telephone 
conferencing  and  e-mail  addressed  this  problem  to  some  extent  by  encouraging  students  to  exchange  e-mail, 
there  was  still  no  affordable  way  for  them  to  participate  in  synchronous  interactions  which  were  flexible  as  to 
time,  frequency  and  the  number  and  composition  of  participating  groups.  With  the  recent  increased  robustness 
of  Internet  audio  technology,  however,  it  has  now  become  possible  to  offer  learners  this  improved  access  and 
flexibility,  and  it  was  as  a result  of  such  technological  developments  that  the  current  investigations  were 
implemented. 


Previous  uses  of  audio  applications 

A survey  of  the  literature  and  the  contents  of  listservs  concerning  the  pedagogic  uses  of  Internet-Audio,  suggests 
that  research  into  on-line  use  of  audio  applications  is  still  very  limited.  There  is  streamed  radio  (via  RealAudio) 
and  a small  number  of  homepages  from  which  the  visitor  can  download  short  audio  clips  (there  are  several  on- 
line dictionaries  that  incorporate  the  ability  to  allow  learners  to  listen  to  the  pronunciation  of  a given  entry),  one 
prominent  example  being  Randall  Davis’  Randall's  ESL  Cyber  Listening  Lab  where  it  is  possible  to  listen  to  pre- 
recorded “chunks”  of  language.  (This  site  also  includes  a feature  called  ’’Randall's  Voice  Mailbox"  which  "gives 
visitors  an  opportunity  to  ask  questions  via  e-mail  and  then  to  listen  to  their  responses  over  the  Internet" [Davis 
1998b],  In  other  words,  visitors  may  leave  e-mail  and  then  listen  to  the  pre-recorded  response  to  their  question 
once  this  has  been  uploaded  to  the  web  page. 

None  of  these  applications  uses  audio  in  a truly  interactive  form.  That  is  to  say,  none  employs  it  in  a two-way 
fashion.  Streamed  audio  is  very  much  uni-directional,  as  is  listening  to  sound  files  which  have  to  be  downloaded 
from  the  Internet.  The  only  exception  to  this  seems  to  lie  in  a case  study  reported  by  Elske  Heeren  in  1995.  This 
“...was  carried  out  to  investigate  small-group  collaborative  learning  through  real-time"[Heeren  1995].  As  a 
result  of  this  study,  Heeren  reports  that  "audio-conferencing  plus  a shared  workspace  led  to  equal  task 
performance  as  audio-conferencing  without  such  an  additional  tool"[Heeren  1995].  Even  this  investigation 
differs  from  the  research  reported  here  in  at  least  two  important  ways:  firstly,  whereas  its  focus  was  on  assessing 
the  technology's  potential  to  foster  decision-making,  the  current  investigation  deployed  the  technology  in  real 
language  learning  activities  and  secondly,  participants  in  Heeren's  investigation  were  students  of  philosophy 
rather  than  language  learners. 

In  the  light  of  the  learner  behaviours  observed  during  the  activities  using  telephone  conferencing  and  email  then, 
the  current  study  set  out  to  examine  whether  the  new.  Internet-based  learning  environment  would  support  and 
encourage  not  only  these  behaviours,  but  also  if  the  increased  flexibility  it  offered  would  modify  learner 
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behaviour  still  further,  encouraging  increased  collaboration  and  peer  support  and  thus  improving  the  learning 
experience  as  a whole. 


The  fluent’  project 

In  the  summer  of  1998,  volunteers  from  two  language  courses  - first  level  German  and  final  level  French  - were 
asked  to  participate  in  a series  of  training  activities,  followed  by  three  language  learning  activities.  The  entire 
project  lasted  from  October  1998  until  January  1999.  Learners  were  told  all  these  activities,  would  use  Internet 
audio  conferencing  software  and  would  give  them  voice  access  to  each  other  at  any  time  of  day  for  the  cost  of  a 
local  telephone  call,  regardless  of  their  geographical  location.  30  German  students  and  26  French  students 
volunteered  to  take  part  in  the  trials  after  receiving  detailed  information  further  to  their  initial  expression  of 
interest.  Of  these  volunteers,  thirty  students  participated  in  all  three  activities  while  approximately  three  per 
group  dropped  out  after  each  of  the  first  two  activities. 

The  learning  environment  which  was  developed  included  not  only  the  audio  client,  but  also  e-mail  and  a website 
which  contained  both  technical  and  pedagogical  information.  Learners  were  encouraged,  both  by  email  and 
during  their  audio  tutorials,  to  consult  the  website  to  obtain  information  about  how  to  use  the  audio  client  and 
about  the  content  of  each  activity.  The  former  information  was  provided  via  a Frequently  Asked  Questions  list, 
whilst  the  latter  was  posted  to  the  appropriate  webpage  on  a ‘need  to  know  basis’.  In  effect,  students  were  given 
information  about  each  activity  as  and  when  the  tutor  believed  this  to  be  appropriate.  The  web-page 
environment  allowed  dynamic  responses  to  student  requests  for  further  information  - whether  such  requests 
were  technically-based  or  activity-related. 

The  Internet  audio  conferencing  software  that  was  used  provided  very  low-bandwidth  transport  of  voice  using 
state-of-the-art  encoding.  While  there  are  subtle  artefacts  perceptible  to  the  attentive  listener,  the  voice  quality  is 
sufficient  for  the  task  of  providing  students  with  listening  and  speaking  practice  in  western  European  languages. 
The  technology  is  optimised  for  relatively  low-speed  exchange,  i.e.  using  V.34/bis  (28.8  or  33.6  kbps)  modems. 
It  works  with  de  facto  standard  sound  subsystems;  while  desktop  microphone  and  speakers  can  be  used, 
headphones  with  a powered  microphone  (widely  available  at  low  cost)  are  recommended.  The  encoding  and 
decoding  algorithms  are  simple  enough  to  permit  the  client  to  work  well  on  mid-range  computers  (a  lOOMhz  32- 
bit  processor  with  16Mb  of  memory  is  adequate).  The  server  software,  which  groups  users  in  virtual  ‘rooms’, 
can  run  on  affordable,  mid-range  computers  - we  have  been  using  a desktop  200MHz  Pentium  machine  with 
64Mbytes  of  memory.  The  server  must  be  able  to  manage  multiple  Internet  connections,  i.e.  have  an  industrial- 
grade  TCP/IP  facility  (supplied  with  most  operating  systems  designed  for  server  support).  The  server  for  this 
project  was  connected  to  a standard  campus  Ethernet  LAN  at  10Mbps. 

Students  were  able  to  load  and  configure  the  software  with  relative  ease.  Where  difficulties  did  arise,  they  could 
obtain  help  in  the  form  of  telephone  and  email  access  to  technical  specialists  on  the  research  team.  Those 
problems  which  occurred  were  in  two  major  areas:  some  students  - approximately  15%  of  the  overall  total  - 
dropped  out  of  the  trials  because  they  could  not  get  the  client  installed  or  running  on  their  machines,  but  the 
majority  experienced  few  problems.  Another  area  of  difficulty  concerned  latency  and  reliability  of  the  network 
connecting  students  to  the  server.  These  connections  were  provided  by  a number  of  commercial  ISPs  and  were 
not  under  the  research  project’s  control.  Thanks  to  expertise  provided  by  the  Open  University’s  Knowledge 
Media  Institute,  which  has  extensive  experience  in  telepresence  research  and  practice,  it  was  possible  to  inform 
students  of  the  source  and  extent  of  the  trouble  very  rapidly,  often  as  conditions  changed  “in  real  time.”  This 
ability  to  reassure  students  that  their  own  computers  were  not  necessarily  at  fault  helped  them  to  remain 
confident  and  engaged,  even  during  temporary  “rough  patches”  due  to  network  conditions.  However,  these 
difficulties  were  exceptional  - the  vast  majority  of  participants  experienced  no  problems  during  months  of 
activity. 


Findings 
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An  analysis  of  preliminary  results  suggests  that  the  type  of  audio-support  offered  to  students  during  the  course  of 
these  trials  is  what  many  distance  language  learners  who  are  unable  to  attend  face-to-face  tutorials  and/or  who 
want  to  learn  at  their  own  pace  have  been  looking  for.  Various  participants  confirmed  that  part  of  what  made  the 
project  so  attractive  to  them  was  that  they  did  “not  leave  home  and  travel  miles  for  a tutorial",  as  one  student  put 
it.  Another  major  asset  was  the  fact  that  the  servers  were  constantly  available,  or,  in  other  words,  that  students 
could  arrange  to  meet  either  on  a one-to-one  basis  and/or  in  group  whenever  they  pleased,  thus  supplementing 
the  scheduled  internet-audio  tutorials  and  occasionally  even  making  them  superfluous. 

One  very  important  outcome  of  this  study  was  that  it  appears  that  it  does  not  suffice  simply  to  provide  the 
software  and  to  leave  the  rest  to  the  learner.  In  fact,  the  first  few  weeks  of  the  project  required  researchers  to 
work  long  hours  to  ensure  that  both  the  technical  and  the  pedagogical  sides  of  the  project  were  well  developed. 
On  the  technical  side,  students  reacted  in  very  different  ways  to  the  learning  environment;  for  example,  one 
student  reported  needing  to  "overcome  still  high  level  of  nerves"  while  another  reported  exactly  the  opposite, 
claiming  to  feel  "a  little  more  relaxed  than  face  to  face  which  is  good".  The  project’s  supporting  website  had  to 
be  maintained  and  updated  on  a regular  basis  and  a weekly  surgery  was  provided  to  offer  technical  trouble- 
shooting advice  on  a one-to-one  basis.  From  the  pedagogical  point  of  view,  not  only  did  tutors  have  to  develop 
meaningful,  motivating  and  engaging  learning  activities,  but  it  was  also  necessaiy  to  lay  the  foundations  to 
encourage  learners  to  form  self-help  study  groups  - that  is,  to  develop  as  autonomous  learners, 

A key  figure  to  the  success  of  the  whole  project  was  the  tutor.  The  embarrassment  several  students  reported 
experiencing  when  they  sat  alone  by  their  computers,  finger  on  the  mouse-button,  listening  to  others'  voices  from 
the  headset,  made  it  crucial  for  tutors  to  provide  them  with  a fairly  high  degree  of  encouragement  or  “back 
channel  cues”,  such  as  uh,  yeah,  really,  etc,  or  the  equivalent  in  the  target  language.  As  the  participants’ 
familiarity  with  the  medium  grew,  they  became  more  confident  in  using  it  and  their  self-consciousness  waned.  It 
was  crucial,  though,  to  steer  them  through  these  initial  difficulties  and  also  to  allow  and  encourage  them  to  take 
their  time  in  composing  utterances  in  the  target  language  rather  than  to  exert  pressure  on  them  to  speak  in  order 
to  fill  the  sometimes  lengthy  gaps  between  turns.  It  is  also  important  to  note  that,  initially,  tutors  found  long 
pauses  difficult  to  cope  with,  but,  like  the  students,  as  they  became  more  familiar  with  the  medium,  they  adapted 
their  teaching  style  to  provide  learners  with  a learning  experience  optimally  suited  to  the  learning  environment. 
Yet  another  essential  component  of  this  virtual  learning  environment  is  the  learning  activity.  As  in  ‘traditional’ 
learning  environments  like  the  real-life  classroom,  task-design  is  vital  to  the  success  or  failure  of  the  experience. 
It  would  be  wrong  to  assume  that  eveiy  activity  that  works  in  a face-to-face-setting  can  simply  be  transferred  to  a 
virtual,  audio-conferencing  environment.  The  findings  of  the  current  research  support  the  hypothesis  that 
"...more  explicit  structuring  of  the  task  content,  of  the  already  agreed-on  issues,  and  of  the  planning  and 
progress  was  needed  to  maintain  a shared  understanding  and  a shared  focus" [Heeren  1995],  Given  that  the 
present  project  is  explicitly  concerned  with  language  learning,  we  would  like  to  cite  one  of  our  students  who,  in  a 
different  context  from  Heeren,  noted:  "The  level  of  concentration  required  for  on-line  communication  is 

extremely  high  and  this  should  enable  enhanced  focus  on  the  expression  of  the  language  as  well  as  enhancing  the 
understanding  of  the  sense  of  what  is  being  received,"  This,  then,  is  exactly  what  we  had  hoped  to  achieve  - the 
learning  environment  allowed  learners  to  concentrate  their  efforts  on  using  their  listening  and  speaking  skills  to 
communicate  and  interact  in  the  foreign  language  and  thus  improving  their  confidence  in  the  use  of  the  target 
language. 

To  summarise,  what  is  asserted  here  is  the  following:  this  project  enables  and  empowers  learners  to  interact  with 
each  other  via  synchronous  audio  and  to  form  autonomous  learning  communities  whose  members  are  located  at 
a distance  from  each  other.  The  learning  environment  changes  the  role  of  the  tutor  to  that  of  facilitator,  co- 
leamer  and  activity  co-ordinator  and  allows  learners  and  tutors  to  participate  equally  in  the  design  process. 
Finally,  the  technology  itself  was  chosen  precisely  because  it  can  be  used  in  accord  with  the  pedagogical  aims  of 
the  activity  designers  and  is  thus  deployed  in  a pedagogically-  rather  than  technologically-driven  manner  - 
thereby  adding  value  to  the  overall  learning  experience. 
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Abstract  : The  Vicarious  Learner  project  is  looking  broadly  at  issues  concerning  the 
development  of  a multimedia  database  system  to  promote  and  enhance  the  role  of 
dialogue  in  learning.  A specific  interest,  and  the  origin  of  the  project's  name,  is  in  the 
question  of  whether  and  how  dialogue  can  be  helpfully  "re-used".  What  benefits  can 
students  gain  from  dialogue  as  observers,  not  just  as  participants?  We  describe  our 
initial  attempts  to  generate  and  capture  educationally  effective  discourse  exchanges 
amongst  and  between  students  and  tutors.  Problems  encountered  with  available  CMC 
discourse  formats  led  to  our  development  of  a set  of  Task  Directed  Discussions  (TDDs). 
A medium  sized  corpus  of  discourse  exchanges  was  collected  using  the  TDDs.  A selection 
of  two  hundred  of  these  TDD  exchanges  formed  the  multimedia  discourse  database  to 
the  implemented  prototype  system.  Dissemination.  Initial  results  from  a large  scale 
controlled  experiment  and  evaluation  of  Dissemination  are  outlined. 


The  Vicarious  Learner  Project 

Attempts  to  develop  CMC  environments  that  could  support  appropriate  educational  discourse  has  tended  to 
simply  result  in  designers  repeatedly  making  the  same  mistakes  (Graddol,  1989).  Motivated  by  this  observation 
Mayes  (1995)  has  developed  the  ‘Groundhog  Day’  model  of  educational  courseware  in  which  the  roles  and 
interaction  between  the  three  types  of  learning  technology  are  explained;  these  being  referred  to  as  Primary, 
Secondary  and  Tertiary  Courseware.  Primary  Courseware  is  the  technology  to  support  the  presentation  of 
content.  Secondary  Courseware  the  technology  that  provides  the  support  for  the  doing  of  learning  tasks  and, 
finally.  Tertiary  Courseware  is  the  technology  that  supports  learning  dialogues,  through  communication.  The 
present  paper  looks  at  how  the  value  of  dialogue  in  learning  depends  on  the  structure  of  the  environment  in 
which  it  takes  place.  In  particular,  we  consider  dialogue  which  takes  place  in  a "computer-supported 
communication"  (CMC)  environment,  an  approach  which  is  being  adopted  in  many  courses  from  primary 
schools  through  postgraduate  university  courses  and  continuing  education,  often  uncritically  and  without  fully 
understanding  all  the  issues  which  arise  from  these  methods. 

Motivated  by  our  own  experience  in  developing  web-based  CBL  environments  we  describe  the  reasons  for 
developing  a multimedia  database  of  discourse  exchanges  as  a means  to  promote  better  discussions  (Greeno, 
Benke,  Engle,  Lachapelle  and  Wiebe,  1998).  The  development  of  this  multimedia  system.  Dissemination,  has 
been  motivated  by  a wish  to  exploit  technology  as  a means  to  promote  patterns  of  discussion  and  enquiry  that 
have  proved  difficult  or  problematic  to  initiate  in  traditional  educational  contexts  (Bligh,  1986;  Gibbs,  1992). 
We  begin  by  looking  at  the  Vicarious  Learner  projects  initial  attempts  to  generate  good  educational  discourse 
and  consider  the  various  roles  of  the  environment,  the  participants  and  the  decisions  of  the  course  organisers  as 
some  of  the  determinants  of  the  kind  of  dialogue  that  results.  These  issues  underlie  dialogue  in  learning,  face-to- 
face  as  well  as  on-line  (Newman,  Webb  and  Cochrane,  1995). 


Attempts  to  Encourage  Appropriate  Pedagogical  Discourse 

Dialogue  is  an  essential  component  of  learning,  particularly  in  complex,  discursive  domains.  This  has  been 
noted  by  many  researchers  (Voss,  1996;  Laurillard,  1993),  but  with  increasing  class  sizes  and  the  move  toward 
more  and  more  computer-based  courses,  this  component  is  ever-decreasing  and  in  danger  of  disappearing 
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completely.  The  role  of  technology  must  be  to  push  back  the  threshold  imposed  by  these  constraints,  this  being 
achieved  by  opening  up  new  media  for  discourse  that  are  not  subject  to  the  same  delivery  bottlenecks  as 
traditional  methods. 

The  Vicarious  Learner  project  propose  a new  type  of  courseware  that  aims  to  support  understanding  of  primary 
course  content  (notes,  lectures...)  by  distilling  the  experience  of  other  learners  captured  as  discourse.  Initial 
attempts  to  exploit  CMC  technologies  led  us  to  adopt  a range  of  web-based  discussion  tools,  principally  the 
’’HyperNews"  system  (LaLiberte,  1995)  - a tool  that  manages  a collection  of  HTML  pages  to  provide  a forum 
maintaining  persistent  discussion  threads  accessible  through  a normal  Web  browser  such  as  Netscape.  In  the 
first  Computers  in  Teaching  and  Learning  course  (CTLl),  this  environment  was  used  relatively  loosely. 
Students  were  prompted  by  a few  seed  questions,  but  then  were  left  to  develop  the  discussion  as  they  chose, 
with  arbitrary  but  usually  quite  sparse  contributions  from  the  course  tutors.  Thus  HyperNews,  as  used  in  CTLl, 
acted  as  a form  of  ‘discretionary  database’  (Connelly  and  Thom,  1991),  growing  and  developing  only  under  the 
motivation  of  the  students  themselves. 

Based  on  feedback  from  the  first  course  and  suggestions  in  the  literature  on  conducting  on-line  discussions 
(Sproull  and  Keisler,  1993),  we  made  the  discussions  in  the  second  CTL  course  more  structured  and 
participation  compulsory.  We  had  tutors  post  specific  questions  and  participate  more,  particularly  by 
summarising  the  main  points  at  the  end  of  each  week's  discussion. 

These  changes,  which  in  good  'user-centred'  style  were  based  on  the  comments  and  observation  of  real  users  in 
a real  setting  were  not  successful  in  improving  the  discussions.  Two  effects  were  particularly  striking.  On  the 
one  hand,  all  students  participated  in  the  discussion  in  CTL2,  whereas  in  CTLl  only  a half  of  the  students  made 
any  contribution,  though  only  half  of  these  again  (i.e.  a quarter  of  the  class)  on  a regular  basis.  On  the  other 
hand,  the  number  of  questions  raised  in  the  HyperNews  discussion  in  a given  week  in  CTLl,  was  typically  four 
times  greater  than  in  CTL2.  A content  analysis  of  these  questions  showed  those  generated  in  CTLl  to  have  been 
‘knowledge-based’  questions  arising  from  students’  interests  and  attempts  to  engage  more  deeply  in  the 
problems  underlying  the  course  content  (Scardamalia  et  al.,  1992).  That  is,  they  were  engaging  in  discussion  on 
the  theoretical  assumptions  and  motivations  defining  the  knowledge  domain.  In  contrast,  students  in  CTL2  all 
participated,  but  in  a generally  much  more  subdued  manner,  restricting  their  questions  to  a more  shallow  ‘text- 
based’  level  (Rosenshine  and  Chapman,  1990). 


Turning  Discussions  into  Learning  Tasks 

In  response  to  our  own  experience  we  have  developed  a series  of  Task  Directed  Discussions  (TDDs)  which 
encourage  students  to  engage  in  meaningful  structured  discussion  tasks.  The  series  of  tasks  gradually  demand 
students  to  engage  in  deeper  thinking  about  the  domain  concepts  and  'ease  students  in'  to  discussions.  These 
TDDs  are  a first  step  both  toward  developing  methods  which  will  be  useful  for  any  student  population  and  also 
toward  testing  and  refining  our  model  of  educational  dialogues. 


In  themselves  each  TDD  is  a discrete  language  activity  with  a set  goal,  with  all  discussions  being  based  upon 
discursive  manipulations  of  a common  set  of  domain  concepts.  Altogether  eleven  Task  Directed  Discussions 
have  been  proposed  ranging  in  discursive  focus  from  single  concepts  to  groups  of  concepts,  with  vaiying 
manipulations  of  these.  Thus,  the  discussions  vary  in  their  coverage  of  the  knowledge  domain,  the  specificity  of 
discussions  and  the  types  of  epistemic  structures  that  they  aim  to  elicit  and  make  exoteric  to  others.  This  is 
where  TDDs  have  gained  a great  deal  from  the  lessons  learnt  in  work  on  second  language  acquisition,  where 
there  is  over  thirty  years  of  work  in  using  and  embedding  complex  cognitive  tasks  in  structured  discussion 
groups  (Skehan,  1998). 


For  each  discussion  task  the  idea  is  to  focus  attention  on  to  an  explicit  and  shared  set  of  concepts  that  have  been 
derived  from  the  Primary  Courseware  (i.e.  the  course  content).  Thus  the  Primary  Courseware  remains  the  target 
for  each  discussion  task,  but  the  form  and  scope  of  each  discussion  is  controlled  through  specified 
manipulations  of  these  concepts.  So,  each  discussion  form  is  based  around  a simple  cognitive  task  and  acts  as  an 
example  of  Secondary  Courseware.  Finally,  Tertiary  Courseware  is  produced  as  an  outcome  of  these  TDDs  in 
the  form  of  recorded  discussions  about  the  course  content.  These  discussion  can  then  be  indexed  into  a 
multimedia  database  to  compliment  the  Primary  Courseware.  The  eleven  Task  Directed  Discussions  are  : 
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1.  Amphibolic  TDD  : the  goal  is  to  examine  the  multiple  interpretations  offered  by  the  organisation  and 
Primary  Courseware. 

2.  Common  Denominator  TDD  : given  a concept,  name  examples  of  its  application. 

3.  Comparison  TDD  : describe  the  connection  between  two  concepts  relative  to  a given  criterion. 

4.  Defining  Terms  TDD  : one  student  attempts  describes  a concept  well  enough  for  another  to  guess  what  this 
concept  is. 

5.  Depiction  TDD  : explore  the  multiple  representational  structures,  styles  or  media  in  which  a concept  or 
argument  may  be  depicted  (e.g.  charts,  text,  graphs...). 

6.  Gestalt  TDD  : the  explicit  goal  is  to  highlight  the  underlying  assumptions  made  by  another  student  as  part 
of  one  of  their  explanations  of  a concept  or  argument  using  a given  concept. 

7.  Hypothetical  TDD  : Encourage  learners  to  reason  to  and  from  precepts,  conjecturing  to  and  from  'possible 
worlds.*  Examples  would  be  a counterfactual  arguments. 

8.  Ranking  TDD  : rank  a given  set  of  concepts  in  terms  of  level  of  importance  along  a given  dimension. 

9.  Reconstruct  TDD  : have  learners  re-organise  the  order  and  propositional  structure  of  sections  of  the 
Primary  Courseware  to  explore  the  conceptual  and  functional  structure  implicit. 

1 0.  Repertory  Grid  TDD  : select  three  concepts  and  describe  in  what  way  two  are  similar,  but  different  from 
the  other  one  along  a common  dimension  (construct) 

1 1 . Scanning  TDD  : select  from  a designated  section  of  the  Primary  Courseware  a specified  number  of  factors 
that  are  the  most  relevant  examples  of  a given  criterion. 


TDDs  have  a number  of  goals,  but  foremost  is  the  requirement  that  through  taking  part  in  them  learners  are 
implicitly  helped  in  structuring  their  understanding  and  developing  their  knowledge,  i.e.  that  learners  are 
helped  in  developing  their  cognitive  "epistemic fluency'  (Morrison  and  Collins,  1995). 

As  mentioned  above,  as  a means  to  explore  such  issues  relating  to  multimedia  indexing  and  modality 
presentation  of  pedagogical  discourse  we  have  developed  and  implemented  an  indexed  multimedia  database  of 
these  TDDs  in  the  ‘Dissemination’  tutoring  system.  A logical  representation  of  the  interface  structure  of 
Dissemination  can  be  seen  in  Fig.  1,  while  an  example  of  the  actual  system  is  shown  in  Fig.  2,  below. 


Figure  1 : The  'Dissemination'  Tutoring  System 


Dissemination  : An  Indexed  Multimedia  Database  of  Tertiary  Courseware 

From  a potential  resource  of  over  30  hours  of  videod  discussions  Dissemination  has  been  populated  with  an 
initial  number  of  200  media  clips  as  video,  audio-graphic,  and  text  transcripts.  Further,  using  Dissemination  a 
large-scale  study  was  undertaken  in  which  37  subjects  were  required  to  undergo  a 10  hour  learning  experiment. 
The  subjects’  goal  was  to  learn  and  understand  the  Primary  Courseware  (‘The  Role  of  Computers  in  Teaching 
and  Learning’,  as  in  the  CTL  1 and  2 courses)  well  enough  to  achieve  70%  on  a test  about  it  (the  subjects  were 
told  that  anyone  achieving  this  score  would  be  awarded  a financial  bonus).  In  addition  one  set  a subjects  had 
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access  to  the  Tertiary  Courseware  in  Dissemination.  However,  there  was  no  explicit  requirement  that  these 
subjects  make  use  of  the  Tertiary  Courseware  during  the  experiment.  In  fact,  given  the  incentive  of  the  financial 
bonus  it  is  reasonable  to  assume  - in  addition  to  the  subjects  own  stated  comments  - that  they  only  made  use  of 
this  material  when  they  found  it  useful  to  do  so.  As  a result  we  a confident  that  in  using  the  Tertiary 
Courseware  the  subjects  were  attempting  to  retrieve  those  examples  of  multimedia  discourse  from  the  database 
that  would  best  help  them  in  the  interpretation  and  understanding  of  the  primary  course  content.  Each  media 
clip  entered  in  the  system  could  be  retrieved  along  three  orthogonal  dimensions  : 

1.  The  concept  or  focus  topic  of  the  TDD; 

2.  The  form  of  the  discourse  as  determined  by  the  discourse  goal  specified  in  the  TDD; 

3.  The  medium  of  discourse.  This  is  the  dimension  along  which  the  discourse  is  captured  and  entered  into  the 
database  (video,  audio,  text,  graphic). 

The  relationship  of  the  Primary  Courseware  to  the  Tertiary  Courseware  can  be  seen  in  Fig.  1 . In  reading  the 
course  notes  the  learner  can  access  Dissemination  through  an  indexed  set  of  key  concepts  highlighted  directly 
(top  left),  or  they  can  access  an  archived  TDD  indirectly  through  a search  of  the  database  (bottom  left).  For  each 
key  concept  the  learner  is  presented  with  the  indexed  set  of  all  TDDs  for  that  key  concept  in  various  media  (top 
right).  In  selecting  one  of  these  along  the  three  orthogonal  dimensions  the  learner  is  shown  the  desired  piece  of 
Tertiary  Courseware  (bottom  right). 


OuicknTM™  inda 
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Figure  2 : A Screenshot  of  the  'Dissemination'  Tutoring  System,  showing  the  Primary  Courseware  (centre),  the 
indexed  database  of  TDDs  for  the  concept  ‘reflection’  (top  left)  and  Tertiary  Courseware  presented  as  a text 
transcript  (bottom  left)  and  digitised  video  clip  (bottom  right) 


Tertiary  Courseware  as  a Vicarious  Learning  Resource 

The  results  from  the  experiment  are  extremely  rich  in  data.  Giving  us  information  on  the  cognitive,  affective, 
linguistic  and  behavioural  impact  of  Tertiary  material  on  learners.  At  the  time  of  writing  we  are  conducting  a 
series  of  structured  interviews  using  Repertory  Grid  techniques  on  the  RepGrid  II  programme  (Gaines  and 
Shaw,  1 993).  In  addition  we  are  analysing  the  computer  logs  to  look  at  the  media  access  patterns  in  relation  to 
the  content  of  the  Tertiary  and  Primary  Courseware.  Some  preliminary  data  that  is  of  interest  in  its  own  right  is 
data  showing  that  there  are  patterns  of  preference  and  individual  differences  in  subjects’  retrieval  strategies, 
and  that  these  preferences  for  Tertiary  Courseware  develop  over  time. 
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Figure  3:  Graph  of  comparative  mean  rates  of  media  use  for  each  quarter  hour. 


The  pattern  of  media  use  during  the  course  of  the  experiment  can  be  seen  in  Fig.  3.  Certainly  there  are  a number 
of  surprising  patterns  suggested  by  this  data.  The  audio-graphic  medium  is  clearly  the  preferred  medium  of 
retrieval  for  the  Tertiary  Courseware.  In  addition,  it  was  surprising  to  the  experimenters  that  the  textual  medium 
was  used  at  all,  as  these  were  verbatim  transcription  of  the  conversants’  speech  (hums  and  stutters  included). 
One  explanation  is  to  understand  these  results  in  terms  of  the  ‘grounding’  of  the  discourse  available  in  each 
medium  (Clark  and  Brennan,  1993).  Yet  the  concept  of  communicational  grounding  does  not  adequately 
explain  why  subjects  would  access  the  same  data  in  different  media  or  would  show  a preference  for  one  media 
at  one  time  and  another  later.  This  problem  is  clearly  highlighted  when  one  looks  at  the  data  on  individual 
differences.  For  each  individual  there  were  marked  patterns  of  preference  for  one  medium  over  another,  though 
these  preferences  often  changed  over  time.  Additionally,  the  subjects  varied  greatly  in  the  volume  of  material 
browsed  through  to  the  number  of  clips  they  viewed  or  read  in  their  entirety. 

Finally,  the  data  on  the  mean  number  of  clips  viewed  in  their  entirety  also  pose  a problem  to  be  explained.  Over 
the  course  of  a single  session  individuals  were  less  likely  to  listen  or  watch  a clip  in  its  entirety,  the  decline 
being  gradual  from  an  mean  of  2 per  quarter  hour  to  less  than  1 ; with  this  pattern  being  repeated  during  each 
session.  It  may  be  that  this  is  an  indication  of  subjects’  improvement  in  their  ability  to  retrieve  the  information. 
Or  that  their  growing  understanding  of  the  course  notes  allowed  them  to  confirm  their  working  hypotheses 
more  quickly.  Basically,  as  your  understanding  of  a subject  grows  do  you  simply  get  better  at  retrieving  the 
information  you  need  or  need  to  retrieve  less  information? 

Conclusion 

In  coming  to  exploit  and  fully  understand  the  nature  and  interaction  between  different  courseware  media  the 
‘Groundhog  Day’  model  (Mayes,  1995)  has  proved  to  be  a strong  motivating  factor  in  the  exploitation  of  a 
networked  multimedia  database  of  student  discussions.  In  balance  of  the  importance  of  such  learning 
opportunities  in  traditional  education  media  (Bransford,  Franks,  Vye  and  Sherwood,  1989;  Graddol,  1989)  it  has 
been  important  in  our  work  to  overcome  the  technical,  psychological  and  educational  difficulties  in  providing 
such  a resource  to  learners.  Through  our  early  work  on  promoting  discourse  we  quickly  became  aware  of  the 
need  to  develop  a set  of  Task  Directed  Discussions  that  would  guide  students  in  their  interactions  with  the 
Primary  Courseware  and  with  each  other.  Having  produced  a corpus  of  such  discussion  tasks  (some  30  hours) 
we  implemented  these  as  part  of  a structured  on-line  database  of  Tertiary  courseware  for  other  learners  to 
exploit. 

The  large  scale  experiment  conducted  using  the  discourse  database  (Dissemination)  has  produced  interesting 
and  promising  initial  results.  Using  the  information  from  this  experiment  and  from  continuing  work  in 
developing  the  multimedia  database  to  exploit  the  emerging  SMIL  standard  (Synchronised  Multimedia 
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Integration  Language)  for  integrated  multimedia  frameworks  we  hope  to  implement  the  full  Tertiary 
Courseware  corpus.  As  part  of  this  we  are  currently  subjecting  transcripts  of  the  discourse  to  a range  of 
linguistic  measures,  for  both  content  and  discourse  analysis  (Gunawardena,  Lowe  and  Anderson,  1998).  Such 
an  analysis  will  be  used  to  further  develop  the  logical  database  structure  of  Dissemination  for  exploitation  by 
researchers  interested  in  the  form  and  properties  of  pedagogical  discourse  and  CMC  environments. 

Acknowledgements 

The  research  reported  here  was  carried  out  under  the  EPSRC  Funded  Grant  Ref:  GR/K72759  : ‘Distributed  Learning 
Dialogues:  Deriving  Tertiary  Courseware.’  Initial  teaching  trials  for  the  ‘Computers  in  Teaching  and  Learning’  course  were 
conducted  at  the  Centre  for  Computer  Based  Learning,  Heriot  Watt  University,  Edinburgh. 


References 

Bligh,  D.  (Ed.)  (1986).  Teach  Thinking  by  Discussion,  Surrey:  SHRE  & NFER-NELSON. 

Bransford,  J.D.,  Franks,  J.J.,  Vye,  N.J.,  and  Sherwood,  R.D.  (1989).  New  approaches  to  instruction:  because  wisdom  can’t  be  told.  In 
Similarity  and  Analogical  Reasoning.  Editors  Vosniadou,  S.,  and  Ortony,  A,  Cambridge  University  Press,  pp.  470-497 

Clark,  H.H.,  and  Brennan,  S.E.,  (1993).  Grounding  in  Communication,  In  'Readings  in  GroupWare  and  Computer-Supported  Co- 
operative Work : Assisting  Human  Computer  Collaboration  * Ronald  M.  Baeker,  Morgan  Kaufman,  pp.  222-233 

Connelly,  T.,  and  Thom,  B.K.  (1991)  Discretionary  Data  bases  : Theory,  data,  and  implications.^ln  J.  Fulk  and  C.  Steinfield  (Eds.), 
Organisations  and  Communication  Technology,  Newbury  Park,  CA:  Sage,  pp.  219-233. 

Gaines,  Brian  R and  Shaw  Mildred  L G,  (1993)  Eliciting  Knowledge  and  Transferring  it  Effectively  to  a Knowledge-Based  System, 
IEEE  Transactions  on  Knowledge  and  Data  Engineering  5(1)  4-14. 

Gibbs,  G.  (1992).  Discussion  with  More  Students,  Cambridge:  Oxonian  Rewley  Press. 

Graddol,  D.  (1989).  Some  CMC  Discourse  Properties  and  their  Educational  Significance.  In  Mindweave  : Communication.  Computers 
and  Distance  Education,  R.  Mason  and  A Kaye,  (Eds.).  Oxford:  Pergamon  Press. 

Greeno,  J.G.,  Benke,  G.,  Engle.  R.A.,  Lachapelle,  C.,  and  Wiebe,  M.  (1998).  Considering  Conceptual  Growth  as  Change  in  Discourse 
Practices.  Proceeding  of  the  twentieth  annual  conference  of  the  Cognitive  Science  Society.  University  of  Wisconsin- 
Madison.  (August  1-4)  pp  442-447 

Gunawardena,  C.N.,  Lowe,  C.A.,  and  Anderson,  T (1998).  Transcript  Analysis  of  Computer-Mediated  Conferences  as  a Tool  for 
Testing  Constructivist  and  Social-Constmctivist  Learning  Theories.  Proceedings  of  the  14  th  Annual  Conference  on  Distance 
Teaching  and  Learning  '98  (August  5-7),  pp  139-145.  Madison,  Wisconsin. 

LaLiberte,  D.  (1995).  HyperNews,  http://union.ncsa.uiuc.edu: 80/HyperNews/get/hypemews.htmI 

Laurillard,  D.  (1993).  Rethinking  University  Education,  London:  Routledge. 

Mayes,  J.T.  (1995).  Learning  Technology  and  Groundhog  Day.  In  Strong,  W.,  Simpson,  V.B.,  and  Slater,  D.  (Eds.)  Proceedings  of 
Hypermedia  at  Work:  Practice  and  Theory  in  Higher  Education,  Canterbury:  University  of  Kent  at  Canterbury. 

Morrison,  D.,  & Collins,  A.  (1995).  Epistemic  fluency  and  constmctivist  learning  environments.  In  B.  G.  Wilson  (Ed.),  Constructivist 
learning  environments:  Case  studies  in  instructional  design.  Englewood  Cliffs  NJ:  Educational  Technology  Publications. 

Newman,  D.  R.,  Webb,  B.  and  Cochrane,  C.  (1995).  A Content  Analysis  Method  to  Measure  Critical  Thinking  in  Face-to-Face  and 
Computer  Supported  Group  Learning,  International  Computing  and  Technology  : An  electronic  Journal  for  the  21st 
Century,  ISSN  : 1064-4326,  April,  Volume  3,  Number  2,  pp  56-77. 

Scardamalia,  M.,  Bereiter,  C.,  Brett,  C.,  Burtis,  P.  J.,  Calhoun,  C.,  and  Smith-Lea,  N.  (1992).  Educational  Applications  of  a Networked 
Communal  Database,  Interactive  Learning  Environments,  Vol  2,  No.  1,  pp.  47-71. 

Skehan,  P (1998)  A Cognitive  Approach  to  Language  Learning,  Oxford  UP,  Oxford 

Sproull,  L.  and  Kiesler,  S.  (1993).  Connections:  New  Ways  of  Working  in  the  Networked  Organisation,  3rd  Edition,  London:  MIT 
Press. 

Voss,  J.  F.  (1990).  Reasoning  by  argumentation.  In  H.  Mandl,  E.  De  Corte,  N.  Bennett,  & H.F.  Friedrich  (Eds.),  Learning  and 
instruction:  European  research  in  an  international  context,  Vol.  2.1,  Oxford:  Pergamon  Press,  pp.  305-319. 


1102 


Evaluating  the  ARIADNE  Core  Tools 
in  a Course  on  Algorithms  and  Data  Structures 


E.  Duval*,  K.  Hendrikx,  K.  Cardinaels,  E.  Vervaet,  R.  Van  Durm,  B.  Verhoeven  and  H.  Olivie, 
Departement  Computerwetenschappen,  Katholieke  Universiteit  Leuven,  Belgium. 
Erik.Duval@cs.kuleuven.ac.be 


Abstract:  This  paper  presents  the  results  of  a pedagogic  verification  experiment  we  carried  out  in  the 
ARIADNE  project  (Forte  et  al.  1997a;  Forte  et  al.  1997b;  Duval  1998;  Ariadne  1998).  The  aim  of  the 
experiment  was  to  evaluate  the  use  of  the  ARIADNE  core  tools  in  a course  on  algorithms  and  data 
structures,  as  it  was  delivered  in  1997-1998  at  the  K.U.Leuven  - Campus  Kortrijk.  The  goals  of  the 
evaluation  were: 

• to  investigate  the  usefulness  and  usability  (i.e.  reliability,  ease  of  use,  effectiveness  and  efficiency, 
etc.)  of  the  ARIADNE  core  tools  and  the  ARIADNE  methodology  for  course  production  and  delivery 
from  the  perspective  of  a professor  who  is  responsible  for  the  organization,  content  and  teaching  of 
a university  course; 

• to  investigate  the  usefulness  and  usability  of  an  ARIADNE  course  for  students,  by  evaluating  the 
ARIADNE  Learner  Interface.  (The  content  provided  through  that  interface  was  developed  mostly 
with  non- ARIADNE  authoring  tools). 

With  this  paper,  we  hope  to  counter  the  general  observation  that  formal  evaluation  of  the  actual  use  of 
technology  in  education  and  training  is  often  neglected  or  even  totally  absent. 


The  Evaluation  Context 

The  ARIADNE  toolset  is  quite  broad:  it  encompasses  authoring  tools,  as  well  as  a number  of  core  tools  for 
management  and  delivery  of  courses.  The  experiment  reported  upon  here  is  concerned  with  the  evaluation  of  the 
following  core  tools  and  their  associated  methodologies: 

• the  Pedagogical  Header  Generator:  an  indexing  tool  used  to  describe  educational  resources  and  to  enter  the 
descriptions  (sometimes  called  ’metadata')  and  the  resources  themselves  into  the  KPS  (see  below),  so  that  they  can  be 
reused  in  many  courses; 

• the  Knowledge  Pool  System  (KPS):  a distributed  database  of  reusable  multimedia  educational  components,  with  circa 
10  operational  sites  throughout  Europe  at  this  moment  (Cardinaels  et  al.  1998),  that  can  be  queried  through  the 
indexing  tool  already  mentioned; 

• the  ARIADNE  Management  Interface  (AMI):  a Web  interface  that  enables  an  author  to  define  a course  as  a sequence 
of  sessions  with  associated  documents  (retrieved  from  the  KPS)  and  other  resources  (external  web  documents, 
interaction  tools  like  discussion  fora  or  email,  etc.) 

• the  ARIADNE  Learner  Interface  (ALI):  an  automatically  generated  Web  site,  based  on  the  course  definition  created 
with  the  AMI  (see  above),  that  is  used  to  deliver  the  educational  content  to  the  learner.  It  provides  him  with  access  to 
the  educational  material  included  in  the  course  and  guides  him  through  the  course  in  a way  predefined  by  the  teacher. 

The  first  three  tools  were  evaluated  by  the  authors  themselves,  as  they  were  responsible  for  the  course.  The  latter  tool 
was  evaluated  by  the  students  who  took  the  course.  This  experiment  is  part  of  a series  of  ARIADNE  pedagogic 
verification  experiments  that  were  carried  out  by  the  end  of  the  first  phase  of  the  European  ARIADNE  project 
(ARIADNE  entered  its  second  phase  in  June  1998). 


The  Course 

The  course  'Informatie-  en  Programma-Structuren'  ("Information  and  Programming  Structures")  is  delivered  to  2"^*  year 
informatics  ('2de  Kandidatuur  informatica')  students  at  the  'K.U.Leuven  Afdeling  Kortrijk'  (KULAK  for  short),  a 
distant  campus  of  K.U.Leuven,  at  circa  130  kilometers  of  Leuven.  Professor  Erik  Duval  comes  to  Kortrijk  once  a week. 
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where  a local  assistant  is  actually  based  full-time.  KULAK  students  typically  take  the  first  two  years  of  their  four  year 
program  in  Kortrijk.  Afterwards,  they  can  take  the  second  part  of  their  program  in  Leuven  itself,  or  in  another 
university. 


The  organization  of  the  course  was  not  so  much  an  ARIADNE  related  task,  but  required  substantial  work  during  the 
pre-experiment  and  the  actual  experiment  phase,  because  this  course  was  very  substantially  reorganized,  based  on  the 
findings  of  a study  committee  at  our  Computer  Science  Department.  Eventually,  the  course  consisted  of: 

• 23  lecturing  sessions  of  2 hours  each 

• 9 guided  exercise  sessions  of  2.5  hours  each 

• 2 practicals  of  20  hours  and  50  hours  individual  student  work  respectively 

• an  0.5  hour  oral  examination  with  2.5  hour  preparation  time. 

The  actual  course  material  was  developed  in  very  close  collaboration  with  prof.  Eric  Steegmans  at  K.U. Leuven  who 
was  teaching  the  same  course  to  K.U.Leuven  students.  In  concrete  terms,  the  course  content  consisted  primarily  of: 

• slides  in  Microsoft  Powerpoint; 

• text  in  Microsoft  Word; 

• text  in  LaTex,  translated  into  HTML  (manually  and/or  automatically); 

• active  existing  documents  (applets  and  the  like),  developed  by  our  team  or  located  with  Web  search  engines,  to 

• illustrate  particularly  challenging  concepts  and  algorithms  that  benefit  greatly  from  visual  animations  and  the  like. 

This  material  is  structured  in  an  ARIADNE  ’curriculum'  that  defines  sessions  and  the  appropriate  documents  for  these 
sessions.  The  table  below  shows  a small  excerpt  of  the  curriculum  for  this  course.  Note  that  the  evolution  of  the  course 
in  time  is  defined  (through  the  date  and  duration  of  the  different  sessions),  so  that  students  can  always  be  guided  to  the 
appropriate  material  at  a particular  moment  in  time.  An  assignment  is  defined  as  a so-called  'fuzzy  session'  that  has  a 
start  and  end  date,  in  between  which  students  must  complete  the  assignment. 


Session  1:  Inleiding 

Date:  2/10/97  11:00 
Duration:  2:00 

Description:  Deze  sessie  handelt  over  het  verloop  en  de  inhoud  van  de  cursus.  Tevens  wordt  kort  ingegaan  op 
Java  als  programmeertaal.  Deze  sessie  komt  overeen  met  p.1-13  in  de  cursus. 

Documents  (2): 

1.  VERLOOP  VAN  DE  CURSUS  INFORMATIE-  EN  PROGRAMMASTRUCTUREN 

2. DE  PROGRAMMEERTAAL  JAVA 

Assignment:  Practicum  2 

Start:  12/2/98  15:30  - End:  5/3/98  1 1:00 
Duration:  5:00 

Description:  Dit  is  het  tweede  deel  van  het  practicum. 

Documents  (1): 

l.OPGAVE  PRACTICUM  2 INFORMATIE-  EN  PROGRAMMASTRUCTUREN  1997-98  (KULAK) 

Session  26:  Oefenzitting  8 
Date:  19/2/98  15:30 
Duration:  2:30 

Description:  Dit  is  de  tweede  oefenzitting  rond  correctheidsbewijzen. 

If-then-else  opdrachten 

Oproep  van  procedures,  functies,  mutatoren  en  inspectoren 
Bewijs  van  eindigheid 
Bewijs  van  data-  en  klasse-invarianten 
We  behandelen  ook  een  uitgebreider  voorbeeld:  ’het  eerste  langste  plateau’. 

Documents  (5): 


1 .If-then-else  opdrachten 
2.0proep 

3. Bewijs  van  eindigheid 

4. Bewijs  van  data-  en  klasse-invarianten 

5. Eerste  langste  plateau 


Sample  from  the  ARIADNE  curriculum  for  this  course 


The  Evaluation:  Professor  Perspective 

For  the  evaluation  of  the  usejulness  and  usability  of  the  ARIADNE  core  tools  from  the  perspective  of  a professor  and 

his  team,  the  following  statements  summarize  the  main  findings.  These  are  based  on  personal  experience  by  the  authors 

as  recorded  in  an  interim  report: 

• There  is  a lot  of  work  involved  in  organizing  a new  course  (or  rather:  a heavily  restructured  existing  course).  This  is 
probably  true  in  general,  but  the  situation  was  not  alleviated  through  the  use  of  ARIADNE  technology.  In  other 
words:  the  workload  of  course  preparation  and  elaboration  certainly  didn't  seem  to  diminish  because  of  the  use  of 
ARIADNE  tools 

• An  important  concern  was  to  present  and  treat  the  content  (algorithms  and  data  structures)  in  a more  object-oriented 
way,  with  a focus  on  structured  and  carefully  planned  development  of  solutions,  using  Java  as  a programming 
language.  Few  really  good  examples  exist  that  treat  the  content  of  the  course  in  this  way.  Even  books  that  promised 
to  offer  what  we  were  looking  for  often  used  an  object-oriented  language  like  Java  for  the  notation,  but  not  the 
associated  object-oriented  programming  techniques.  This  resulted  in  the  need  for  more  original  work  than 
anticipated,  which  was  only  possible  through  close  collaboration  with  the  team  that  teached  the  same  course  in 
Leuven,  as  this  team  has  extensive  experience  in  teaching  on  related  topics.  This  problem  was  completely  unrelated 
to  the  use  of  ARIADNE  tools. 

• Using  an  ARIADNE  curriculum  to  document  the  actual  progress  of  the  course  as  it  takes  place  actually  helps  to  keep 
track  of  the  administration  of  the  course. 

• The  ARIADNE  Management  Interface  was  not  always  robust  and  easy  to  use:  updating  a course  extensively  and 
then  not  being  able  to  upload  the  new  version  is  a major  source  of  frustration.  (Current  developments  in  the 
ARIADNE  project  are  tackling  this  problem.) 

• Adding  value  with  the  ARIADNE  approach  to  a course  whose  main  mode  of  operation  is  traditional  (face-to-face 
lecturing  with  regular  practical  sessions)  is  far  from  trivial,  the  more  so  in  this  case  where  the  situation  is  relatively 
luxurious  in  that  the  students  constitute  only  a relatively  small  group  of  10  where  real  interactivity  with  the  whole 
group  can  actually  be  achieved. 


The  Evaluation:  Student  Perspective 

Because  of  the  rather  informal  nature  of  the  course,  with  the  interactive  sessions  involving  a group  of  10  students  in  a 
relatively  small  room,  and  the  very  regular  contacts  between  the  students  and  the  teaching  assistant  at  Kortrijk,  we 
opted  for  a continuous  evaluation  through  discussions  during  coffee  breaks,  email  messages,  short  class  discussion 
during  the  actual  lectures  and  regular  updates  with  the  teaching  assistant.  Close  to  the  end  of  the  course,  a standardized 
questionnaire  was  handed  to  the  students,  with  a request  to  return  the  filled  in  questionnaire  before  the  final 
examination.  Five  out  of  ten  students  did  indeed  return  the  questionnaire.  Using  a commercially  available  software  tool 
(SUM I),  the  answers  from  the  students  were  analyzed  statistically,  through  the  kind  assistance  of  SCET,  one  of  the 
ARIADNE  project  partners. 

The  students  were  generally  quite  receptive  to  the  whole  idea,  although  some  questioned  whether  the  use  of  technology 
didn’t  make  life  too  easy  for  the  professor,  as  he  no  longer  had  to  ’teach’  directly,  and  too  hard  for  the  students,  as  the 
amount  of  content  could  be  increased  substantially  without  any  immediately  apparent  organizational  problems.  It  was 
agreed  that  the  electronic  material  would  be  supportive  only  and  that  all  content  would  be  discussed  during  the 
interactive  sessions  as  well. 


The  figure  below  represents  the  overall  'Profile  Analysis  Graph'.  In  this  graph,  the  Global  scale  and  five  Usability 
Subscales  are  shown  for  the  ARIADNE  Learner  Interface,  as  evaluated  by  the  students.  For  each  scale,  the  median 
value  is  shown  circled  in  the  middle  of  the  line;  the  95%  upper  and  lower  confidence  limits  are  shown  by  the  opening 
and  closing  points.  These  limits  indicate  the  range  within  which  we  can  be  95%  certain  that  the  true  scale  median  for 
the  software  can  be  found.  The  scales  are  so  arranged  that  the  standard  for  state  of  the  art  software  is  50.  Software 
above  50  is  ahead  of  state  of  the  art  for  quality  of  use;  software  below  50  is  behind  state  of  the  art.  Each  sub-scale  is  on 
a numeric  scale  from  1 1 (low  usability)  to  71  (high  usability),  with  a mean  of  50  and  a standard  deviation  of  10. 
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Essentially,  the  graph  above  means  that  students  rated  the  usability  of  the  ARIADNE  Learner  Interface  slightly  above 
'state  of  the  arf . The  main  findings  of  a more  detailed  analysis  of  the  evaluation  results  are: 


• The  overall  ARIADNE  infrastructure  does  work  well  in  field  experiments  with  real  students  in  a real  course.  The 
students  do  not  seem  to  get  the  impression  that  they  are  treated  as  guinea  pigs  for  immature  technologies. 

• The  overall  ARIADNE  methodology  for  course  material  distribution,  based  on  content  in  a Knowledge  Pool  System 
with  structure  defined  in  a separate  curriculum  and  a student  web  site  generated  automatically  from  the  combination 
of  these  two,  is  very  compatible  with  a develop-while-you-de liver  approach  to  course  material  production.  It  should 
be  noted  that  the  latter  approach  arose  from  practical  requirements  and  limitations,  rather  than  from  an  a priori  intent. 
However,  this  approach  is  certainly  very  common  and  thus  the  above  mentioned  compatibility  seems  quite  important 
for  acceptance  of  the  ARIADNE  toolset. 

• The  relatively  structured  approach  to  learning,  which  is  typical  to  ARIADNE,  seems  to  lead,  as  expected,  to 
purposeful  behavior  on  the  students  part,  as  illustrated  in  the  graph  below  with  the  reactions  to  the  statement  "It  is 
relatively  easy  to  move  from  one  part  of  a task  to  another". 


Disagree 


and  by  the  graph  below  with  the  reactions  to  "This  software  seems  to  disrupt  the 
work". 


□ Profile 

□ Expected 


way  I normally  like  to  arrange  my 


□ Profile 

□ Expected 


• From  the  standardized  questionnaires  filled  in  by  the  students,  one  can  safely  conclude  that  the  ARIADNE  Learner 
Interface  is  easy  to  understand  and  learn  working  with.  Most  of  the  students  are  by  now  probably  quite  accustomed  to 
a World-Wide  Web  interface  with  a navigation  panel  on  the  left  and  a content  panel  that  takes  up  most  of  the 
remainder  of  screen  estate.  This  is  illustrated  by  the  graph  below,  which  represents  the  reactions  to  the  statement  "It 
takes  too  long  to  learn  the  software  commands". 
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Conclusions 

Being  responsible  for  this  course,  we  certainly  felt  that  the  ARIADNE  toolset  and  methods  helped  us  to  structure  the 
development  of  this  course. 

The  most  crucial  remaining  problem  in  our  opinion  is  that  it  is  unclear  to  see  how  exactly  ARIADNE  tools  and 
methodologies  can  add  value  to  a relatively  comfortable  traditional  situation,  where  a small  group  of  students  meet 
almost  weekly  for  2 hours  to  discuss  the  course  content,  meet  every  two  or  three  weeks  for  a 2.5  hours  guided  practical 
session  and  submit  the  results  of  their  individual  work  two  or  three  times  a year,  on  which  they  receive  relatively 
detailed  feedback.  Of  course,  this  situation  is  becoming  more  and  more  the  exception  and  therefor  in  more  general 
terms  the  usefulness  of  the  ARIADNE  approach  seems  much  more  obvious.  Remains  the  question  whether 
developments  like  ARIADNE  do  not  actually  encourage  administrations  and  political  responsibles  to  continue  to  make 
the  teaching  situation  more  problematic  ... 

Regarding  the  evaluation  by  the  students,  it  should  be  noted  that: 

• These  were  2"^*  year  students  in  informatics.  One  could  therefor  expect  that  these  students  would  be  somewhat 
positively  prejudiced  with  respect  to  the  use  of  telematics  for  education.  This  is  illustrated  by  the  graph  below  that 
lists  reactions  to  the  statement  ’There  have  been  times  in  using  this  software  when  I have  felt  quite  tense". 


Agree  UrMleclded  Disagree 


• The  students  were  explicitly  instructed  to  evaluate  the  ARIADNE  Learner  Interface,  not  the  actual  content  (i.e.  the 
pedagogical  documents)  provided  to  them  through  this  interface.  It  is  not  clear  whether  they  really  were  able  to  make 
this  difference. 

• The  students  were  typically  working  in  a LAN  environment,  with  relatively  high  bandwidth  connections  on  a local 
ethemet  network,  as  illustrated  by  the  graph  below  with  the  answers  to  the  statement  "The  speed  of  this  software  is 
fast  enough". 


Agree  Uralecided  Disagree 


□ Profile 

□ Expected 


All  in  all,  it  seems  that  the  students  were  more  positive  about  the  usability  of  the  ARIADNE  Learner  Interface  than  we 
had  anticipated.  It  is  of  course  possible  that  they  wanted  to  ’please’  the  professor  that  took  their  examinations... 
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Conclusion 


In  the  current  academic  year,  we  are  reusing  much  of  the  content  for  a new  run  of  this  course.  This  is  of  course 
absolutely  in  line  with  the  ’share  and  reuse’  philosophy  that  drives  the  ARIADNE  project.  We  will  focus  our 
developments  this  year  on  the  elaboration  of  new  and  the  reuse  of  existing  (ARIADNE  and  non-ARIADNE  alike) 
active  components  that  can  really  add  value  to  the  more  conventional  course  delivery:  these  components  will  include 
simulations,  visualizations,  self-tests,  etc. 

More  generally,  our  team  is  also  using  the  ARIADNE  toolset  for  new  courses  on  computer  science  topics  (first-year 
programming,  Java  language,  hardware,  productivity  tools,  etc.).  Beyond  that,  the  same  infrastructure  is  being  adopted 
by  a most  active  user  community  that  already  includes  about  40  universities,  a number  of  industrial  companies  and 
some  consortia  of  such  users.  We  constantly  try  to  monitor,  evaluate  and  analyze  the  actual  use  of  the  ARIADNE  tools 
in  these  very  diverse  circumstances. 
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Introduction 

At  Europe’s  premier  distance  learning  institution,  the  Open  University  (OU),  we  have  made  computer 
conferencing  central  to  our  flagship  computing  course,  M206  ‘‘Computing:  An  Object-oriented  Approach”  which 
began  with  over  5,000  students  in  Western  Europe  in  1998  and  which  this  year  is  also  being  studied  by  600 
students  at  the  Singapore  Institute  of  Management.  The  course  will  also  become  available  in  North  America  by  the 
end  of  the  year.  The  course  is  primarily  concerned  with  introducing  software  development  and  does  not  recognize 
the  distinction  between  “computer  science”  and  “software  engineering”  (see  Woodman  et  al.  1999).  It  was 
designed  to  address  the  needs  of  industry  and  has  been  given  a prestigious  IT  Award  by  the  British  Computer 
Society.  Computer  conferencing  is  central  to  the  course  and  is  used  to  encourage  our  students  to  ‘talk’  informally 
about  their  new  discipline,  as  portrayed  in  our  teaching  materials,  and  also  formally  to  participate  in  group 
working  projects  for  which  they  are  graded.  At  present  we  are  using  the  FirstClass'*'^  conferencing  system^  for  all 
conference  based  work  - both  for  the  informal  talk  and  the  formal  group  working.  FirstClass  was  chosen  because 
the  university  has  a great  deal  of  experience  of  it  and  could  commit  to  supporting  our  extremely  high  usage. 

The  Case  for  Group  Working 

People  skills  are  hugely  important  to  the  software  industry.  In  contrast  to  the  early  decades  of  software 
development,  its  cost  and  efficiency  are  now  dominated  by  people  issues.  Software  developers  (programmers, 
designers,  analysts)  exhibit  the  normal  human  characteristics  of  fallibility  and  miscommunication.  Analysts  fail  to 
understand  customer  requirements  or  fail  to  communicate  constraints  to  designers;  designers  misinterpret 
specifications  or  fail  to  understand  technological  issues;  programmers  ignore  architectures  or  skip  unit  testing. 
Developing  communication  skills  in  students  is  vital  if  they  are  to  be  useful  and,  indeed,  fulfilled  practitioners.  Of 
course,  because  the  language  of  their  discourse  is  a complex  technical  one,  and  an  inconsistent  one  at  that,  the 
acquisition  and  use  of  vocabulary  is  itself  non-trivial.  Another  of  the  problems  that  neophyte  programmers  face  is 
that  at  any  stage  of  their  learning  their  knowledge  is  incomplete  and  their  skills  deficient.  From  time  to  time  many 
of  them  believe  that  their  lack  of  competence  is  a permanent  state  and  they  fail  to  realize  that  what  distinguishes 
professional  practitioners  is  that  they  have  sufficient  knowledge  and  skills  to  be  able  to  deal  with  such  problems. 
As  well  as  helping  students  formulate  and  organize  their  knowledge,  conferencing  encourages  students  to  be 
reflective,  a necessary  attribute  for  collaborative  working.  The  benefits  of  collaborative  working  are  well 
established  and  the  practice  of  it  in  software  development  is  more  or  less  universal.  We  summarize  the  benefits  to 
students  of  the  group  working  part  of  their  study  thus: 

• it  allows  the  sharing  of  information  and  experience,  which  allows  software  development  problems  to  be 
resolved; 

• students  can  learn  from  one  another; 

• group  members  give  support  to  one  another; 

• group  members  often  provide  motivation  to  sustain  interest  and  involvement. 


* FirstClass  is  a proprietary  conferencing  system  from  SoftArc  Inc.  See  http://www.softarc.com 
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Organization,  Problems  and  Solutions 


Students  on  M206  are  assessed  by  seven  assignments  that  must  be  submitted  by  published  deadlines  coupled  with  a 
final  exam.  The  group  working  element  of  M206  is  composed  of  four  conference-based  projects  which  are 
associated  with  questions  on  three  of  these  assignments.  These  projects  are  concerned  with  concepts  students  have 
studied  several  weeks  earlier  (the  elapsed  time  for  the  course  is  33  weeks  as  it  represents  one  sixth  of  a degree). 
Hence  students  should  not  be  overly  taxed  by  the  technical  content  of  a project,  although  in  some  projects  the 
content  is  intended  to  reinforce  and  extend  ideas  previously  met.  Grades  for  group  working  are  awarded  primarily 
for  participation  in  collective  work  but  there  is  no  dependence  on  the  abilities  of  other  group  members  for  an 
individual  student’s  grades.  That  is,  there  are  no  collective  marks  divided  among  group  members,  only  individual 
marks.  These  principles  are  the  core  of  this  part  of  our  pedagogy. 

With  a course  of  such  a large  population  size,  we  have  inevitably  encountered  some  problems.  Initially  we  chose  to 
make  the  size  of  the  project  groups  between  8 and  10  students.  However  we  were  concerned  that  students  should 
fully  commit  to  a project  because  failure  to  fully  partake  could  compromise  the  ability  of  other  individuals  in  a 
small  group  to  earn  their  grades.  This  concern  about  participation  was  prompted  by  claims  from  many  students  that 
they  did  not  perceive  the  need  to  work  with  others,  despite  the  OU’s  unique  situation  of  having  a student  body 
whose  mean  age  is  37  and  so  whose  life  experience  could  be  expected  to  motivate  this  part  of  the  course. 
Furthermore,  we  implement  a substitution  mechanism  for  the  assignments  whereby  the  least  scoring  assignment  is 
substituted  with  the  average  mark  from  all  other  assignments.  Therefore,  due  to  the  distribution  of  project  work 
amongst  the  assignments,  students  can  pass  the  course  (albeit  poorly)  without  participating  in  group  working.  So, 
to  encourage  participation  and  to  maintain  the  viability  of  the  individual  groups,  for  the  1998  presentation, 
students  could  only  take  part  in  a group  working  project  by  registering  for  the  project  by  a registration  cut-off  date. 
Students  used  FirstClass  to  send  a proforma,  pre-addressed  project  registration  email  to  the  M206  Group  Working 
server.  This  server  (which  we  developed  in-house)  collated  and  acknowledged  (via  FirstClass)  the  receipt  of  all 
registrations  and,  when  registration  had  closed,  randomly  assigned  students  to  project  groups  and  hence  to 
conferences.  This  involved  the  creation  of  about  800  separate  conferences.  The  server  then  emailed  each  student 
the  address  of  their  project  conference.  However,  despite  email  reminders  and  notices  on  both  our  Web  site  and  the 
conferencing  system,  some  students  still  failed  to  register  for  projects  by  the  registration  cut-off  dates.  This  was 
generally  due  to  two  reasons:  either  they  were  unable  to  access  the  registration  mechanism  during  the  registration 
period  (one  student  was  under  the  polar  ice  cap  in  a submarine!)  or  students  assumed  that  enrollment  to  the  course 
equated  to  registration  to  all  projects.  Therefore  for  1999,  we  decided  to  make  project  groups  significantly  larger 
(typically  25  rather  than  10)  and  to  automatically  register  all  students  for  all  projects.  Due  to  the  larger  group  size, 
student  drop  out  from  project  participation  should  not  significantly  effect  the  viability  of  a group.  At  the  time  of 
writing  the  1999  students  cohort  have  not  yet  reached  this  stage  of  the  course. 

All  the  projects  have  multiple  stages,  with  each  stage  being  associated  with  a hard  deadline.  This  can  be  seen  in 
Fig.  1 which  shows  part  of  our  web-based  group  working  calendar.  As  various  stage  deadlines  approach,  the  server 
sends  out  automatic  reminders  of  the  forthcoming  deadline. 
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Figure  1:  Calendar  for  Group  Working  Projects 

Typically,  in  the  first  stage  of  a project  a student  must  study  some  new  problem  or  variation  on  a familiar  one  and 
carry  our  some  task  such  as  writing  a critique  of  a user  interface,  coding  a subclass  in  Smalltalk  or  addressing 
issues  to  do  with  the  analysis  and  design  of  some  problem  area.  Subsequent  stages  usually  involve  commenting  on, 
or  testing  other  students  submissions  and/or  going  deeper  into  the  problem. 

Collaboration  on  a project  is  only  deemed  to  take  place  if  it  has  happened  in  the  computer  conference  generated  for 
that  purpose.  This  is  crucial  for  allowing  graders  to  check  that  project  work  submitted  in  an  assignment  actually 
took  place  in  a collaborative  manner.  However,  this  strategy  has  thrown  up  challenges.  In  particular  we  had  to 
prevent  the  situation  in  which  a talented  student  gives  a complete  answer  so  preventing  others  from  making  an 
effective  contribution.  To  combat  this,  for  Projects  I and  II,  the  project  conference  is  not  made  visible  on  the 
student’s  FirstClass  desktop  until  after  the  Stage  1 deadline.  This  means  that  whilst  students  can  use  the  supplied 
conference  address  to  post  their  Stage  1 contributions,  they  cannot  see  fellow  students  postings  until  after  that 
deadline,  thus  solving  the  preemptive  contribution  problem  mentioned  above.  After  the  Stage  1 deadline  for 
Projects  I and  II  has  passed,  for  each  student,  the  server  makes  their  project  conference  visible  on  their  FirstClass 
desktop. 

A final  issue  encountered  in  1998  was  that  of  coordination  of  project  participation  which  was  related  to  the  time 
period  allotted  for  each  project  (about  4 weeks  — see  Fig.  1).  Because  of  the  varied  lifestyles  and  domestic  demands 
of  the  several  thousand  students  in  the  “class”,  they  inevitably  tended  to  want  to  work  at  their  own  pace  within  the 
given  time  frames  for  the  various  stages.  Some  students  would  start  work  as  early  as  possible,  posting  their  ideas  as 
to  how  to  approach  the  posed  problem  only  to  be  frustrated  by  having  to  wait  up  to  two  weeks  for  any  feedback,  as 
other  members  of  the  group  were  delaying  their  work  until  the  latter  half  of  the  available  time  frame.  We  expect 
that  our  decision  to  enlarge  the  group  conferences  described  above  will  also  overcome  this  problem. 


Sample  Content 

In  the  1998  presentation  of  the  course.  Project  I involved  critiquing  a user  interface  and  consisted  of  two  stages. 
Before  they  started  this  project,  students  had  an  opportunity  to  examine  a sample  project  on  the  same  topic  and  see 
how  another  group  of  students  had  tackled  it.  This  sample  project  consisted  of  a software  application  and  examples 
of  conference  postings  that  the  group  had  made  for  each  of  the  two  stages.  As  in  the  sample  project,  stage  1 of 
Project  I,  involved  students  in  having  to  analyze  and  write  a critique  of  a supplied  application’s  user  interface, 
paying  particular  attention  to  the  principles  of  visibility,  affordance  and  feedback  (principles  to  which  they  were 
introduced  early  in  the  course).  This  critique  was  then  sent  to  the  Project  I conference  by  the  deadline  for  Stage  1 . 
As  described  above,  these  Stage  1 postings  were  not  visible  until  after  the  stage’s  deadline  had  passed.  When  these 
postings  became  visible,  Stage  2 involved  students  making  some  constructive  comments  about  another  group 
member’s  critique  of  the  same  user  interface.  This  form  of  group  working  is  of  course  highly  constrained  and 
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somewhat  artificial.  However  we  deemed  this  to  be  necessary,  in  the  first  instance,  to  introduce  our  students  to  the 
notion  of  group  working  in  a managed  fashion.  By  comparison,  the  final  two  projects  are  much  more  realistic: 
students  work  together  to  solve  a common  problem.  The  group  conferences  are  visible  from  the  start  and  the 
interaction  is  much  more  fluid  and  discursive,  with  students  making  multiple  postings  for  each  stage. 

Conclusion 

Educational  institutions  often  treat  students  as  individuals  who  will  produce  independent  pieces  of  work,  and 
during  their  education  are  not  often  encouraged  to  share  ideas,  work  together  or  evaluate  each  others  work  so  that 
it  can  be  improved.  In  the  work  place  the  opposite  is  expected.  This  is  especially  true  in  the  software  industry. 
Software  developers  are  expected  to  work  together,  be  team  players,  share  ideas,  and  work  towards  common  goals. 
Increasingly  employers  want  evidence  of  such  skills  and  aptitudes,  which  often  require  some  degree  of  work 
experience  before  they  can  be  mastered.  Furthermore  group  working  helps  the  budding  software  developer  to  think 
in  terms  of  complex  systems  rather  than  just  their  part  of  that  system  or  of  traditional  simple  programs.  To  meet 
these  industry  requirements  we  have  developed  a suitable  pedagogy  and  the  mechanisms  by  which  students  can 
work  collaboratively  on  projects  over  extended  periods  of  time. 
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Introduction 

At  the  start  of  1998  Europe’s  premier  distance  learning  institution,  the  Open  University  (OU),  deployed  a new 
introductory  computing  course,  M206  “Computing:  An  Object-oriented  Approach”  (see  Woodman  et  al.  1998). 
The  course  counts  for  one  sixth  of  a degree  (440  hours  of  study)  and  represents  a radical  change  to  the  computing 
curriculum  and  a radical  change  to  how  supported  distance  learning  courses  are  presented.  Reception  of  the  course 
was  unprecedented,  with  over  5,000  students  registering  and  its  innovations  being  recognized  by  a prestigious  IT 
Award  from  the  British  Computer  Society.  Now  in  its  second  year,  some  600  additional  students  at  the  Singapore 
Institute  of  Management  are  taking  the  course,  and  with  the  OU  now  incorporated  in  the  USA,  it  should  be 
available  in  North  America  by  late  1999.  The  course  has  a full  mix  of  media:  paper,  broadcast  television, 
interactive  CD-ROM,  a Web-oriented  programming  and  learning  environment,  a Web  site  and  computer-based 
conferencing.  From  the  start  of  work  on  M206,  we  committed  to  a Web-centric  strategy:  “home  is  where  the  Web 
is”  became  the  course  team  chair’s  slogan!  In  other  words  we  wanted  to  organize  all  learning  resources  around  the 
Web  (see  Sumner  & Taylor  1998).  Ideally  we  would  like  to  have  all  such  resources  accessible  via  a common  front 
end  - a Web  browser.  However,  pragmatics  intervened,  as  described  below,  and  we  have  not  yet  achieved  our  ideal. 
In  subsequent  sections  we  explain  our  technology  architecture  and  the  problems  we  experienced,  together  with 
consequent  changes  we  had  to  implement  for  the  1999  presentation. 

The  Web  Site  as  a Repository 

We  believe  that  the  Web  should  be  the  seamless  mechanism  for  all  electronic  communication:  email,  conferencing, 
Web  pages,  downloads,  and  whatever.  For  pragmatic  operational  reasons  we  had  to  compromise  this  ideal  in  order 
to  provide  students  with  a robust  conferencing  system  that  was  fully  supported  by  the  OU.  Hence,  electronic 
communication  became  split  between  the  M206  Web  site  and  the  FirstClass™  conferencing  system*.  Faced  with 
this  split  we  needed  a clear  principle  for  deciding  what  would  be  available  via  FirstClass  and  what  would  be 
available  via  the  Web.  Such  a principle  was  not  immediately  obvious,  since  FirstClass  could  adequately  support 
more  than  just  conferencing  - for  example,  a read-only  notice  board  could  disseminate  new  information  and  offer 
downloads  and,  with  its  use  of  an  off-line  reader,  was  arguably  less  expensive  for  our  students  to  use  in  terms  of 
connection  time.  The  principle  we  arrived  at  is  enshrined  in  the  slogan,  “FirstClass  announces;  the  Web 
pronounces.”  That  is,  the  Web  became  the  repository  for  all  non-transient  learning  material  (errata  and  addenda  to 
learning  materials,  assignments  and  mock  exams,  software  downloads,  etc.).  Whenever  any  new  learning  material 
was  added  to  the  Web,  the  fact  of  the  addition  was  announced  on  the  student  FirstClass  Notice  Board,  directing  the 
students  to  the  appropriate  Web  page  by  means  of  a logical  reference,  for  example,  “Details  of  the  revised  Practical 
9 are  now  available  on  the  Chapter  15  Web  page”). 

The  1998  M206  site  was  designed  very  much  as  a compromise  between  what  was  possible  and  what  was 
acceptable.  The  main  constraints  on  our  site  were  those  relating  to  the  computers  and  communications  used  by  our 


* FirstClass  is  a proprietary  conferencing  system  from  SoftArc  Inc.  See  http://www.softarc.com 
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students.  The  minimum  computer  specification  precluded  the  use  of  graphically  intense  designs  and  greatly  limited 
the  amount  of  “bells  and  whistles”  that  could  be  included.  In  addition  there  would  be  a limited  amount  of  staff  time 
available  to  develop  and  maintain  the  site.  The  1998  site  was  therefore  designed  in  HTML  3 using  a conventional 
frame-based  solution.  Navigation  was  handled  by  means  of  a frame  on  the  left-hand  side  of  the  screen,  with  the 
contents  of  pages  appearing  in  the  right-hand  frame  (see  Fig.  1).  The  design  of  buttons  was  chosen  to  reflect  those 
used  in  other  interactive  course  materials. 

Chapter  19  : Group  Working  Project  I 

To  complete  the  work  covered  in  this  chapter  you  vwll  need  to  download  a new 
LearningBook  from  the  Web.  Instructions  for  doing  this  are  provided  on  a separate 
page.  Click  on  the  link  below  to  read  the  instructions  and  download  the  LearningBook. 

^ LearningBook  download  instructions 

Project  I is  discussed  in  detail  on  the  Project  I Web  page. 

^ Group  Workj;^<;t  Pmie-:t  I 

Review 

After  reading  this  chapter  you  should  understand  the  mechanics  of  participating  in  the 
group  working  part  of  the  course,  in  particular  that: 

• You  must  register  before  a registration  cut-off  date  (one  for  each  of  the  4 specific 
projects)  in  order  to  participate  in  a specific  group  working  project; 

• Each  project  is  composed  of  one  or  more  stages,  each  of  which  is  associated 
with  a posting  to  the  group  conference  by  a deadline; 

Figure  1:  M206  Web  site 

The  site  was  sub-divided  into  a number  of  areas  relating  to  different  aspects  of  the  course,  including  information 
about  using  the  site,  calendars,  etc.  The  main  resources  sections  were: 

Course  Blocks  - the  course  is  broken  into  seven  “blocks”  each  one  covering  a number  of  “chapters”  (any 
multimedia  unit)  with  an  associated  assignment.  Chapter  Web  pages  comprised  of  a recap  of  the  important 
points  and  a section  detailing  any  misprints  or  omissions.  These  pages  also  provided  additional  learning 
material,  such  as  background  reading,  or  transcripts  of  television  programs. 

Assessment  - this  section  contained  pages  for  the  seven  assignments  and  a specimen  exam  paper.  On  each  page 
there  were  links  to  downloadable  copies  of  the  documents. 

Software  ~ students  could  use  this  section  to  update,  download  or  test  their  software.  Links  were  provided  to 
software  vendors  and  full  installation  guides  given  for  the  software. 

Contacts  - students  must  deal  with  a number  of  different  parts  of  a large  organization  like  the  OU  whilst  studying. 
The  contacts  page  listed  the  types  of  problem  students  might  encounter  (such  as  the  failure  of  a password  or 
missing  course  materials)  alongside  the  appropriate  postal  or  email  address  or  contact  phone  number.  As  well 
as  making  the  contact  procedure  less  problematic  for  the  student,  it  reduced  the  amount  of  mis-addressed 
email  and  post  for  the  university. 

News  - originally  all  course  news  was  distributed  by  means  of  the  FirstClass  system  (see  below),  however,  it  was 
later  decided  that  the  Web  offered  a more  robust  solution  to  all  of  our  users  and  the  two  systems  were  used  in 
parallel.  Items  relating  to  the  course  (such  as  the  posting  of  new  pages,  corrections  to  materials  or  notices 
about  computer  downtime)  were  posted  on  this  page;  old  items  were  archived  on  linked  pages  for  reference. 

Conferencing  as  a Community 

Of  course,  electronic  conferencing  was  used  for  much  more  than  notice  board  style  announcements.  Conferencing 
underpinned  the  support  for  group  working  that  is  an  important  part  of  M206  as  well  as  providing  an  electronic 
forum  for  students  to  discuss  all  matters  of  interest  to  them  in  relation  to  the  course.  In  our  first  year  (1998),  we 
created  a detailed  national  conference/sub-conference  structure  based  around  the  structure  of  the  course  (fifty  two 
chapters  organized  into  seven  blocks,  essentially).  This  conference  structure  was  available  to  all  5,000  students. 

We  used  FirstClass  in  several  ways:  first  as  the  primary  means  of  communication  between  the  central  academic 
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team  and  both  the  student  body  and  the  part-time  tutors  supporting  them.  For  example  the  M206  Notice  Board 
conference  was  used  for  our  “pronouncements”.  FirstClass  was  also  the  most  direct  way  students  had  to 
communicate  with  us  via  an  email  id  called  M206  Course  Office  - a front  for  administrators  and  academics.  We 
also  used  it  for  informal  discussion  of  the  course  materials  and,  crucially,  collaborative  group  working  (see 
Griffiths  et  al.  1999). 

Ease  of  asynchronous  electronic  communication  immediately  created  a very  real  sense  of  a community  of  learners 
and  teachers,  interacting  with  the  material  and  with  each  other’s  ideas  and  views.  As  teachers,  we  felt  we  had  to  be 
part  of  that  community,  reacting  to  requests,  clarifying  issues  and  (occasionally)  defending  ourselves  against 
suggestions  about  the  elegance,  clarity  and  relevance  of  some  of  our  teaching.  Such  involvement  made  huge 
demands  on  the  academic’s  time,  with  the  course  manager  and  an  academic  often  devoting  50%  of  their  time  to 
queries.  The  locus  of  control  of  the  course  was  no  longer  firmly  teacher-centered  and  the  nature  of  the  relationship 
teacher  and  taught  was  irrevocably  changed. 

There  was  a strong  tendency  for  this  community  to  be  one  that  was  sharing  “bad  news”  or  telling  “war  stories”  - 
from  inability  to  read  some  file,  to  ambiguity  in  an  assignment  question,  and  on  to  downright  errors  in  learning 
material.  This  could  actively  construct  a perception  of  a troubled  course  when  the  reality  was  quite  different. 
Fortunately,  there  were  various  episodes  where  hundreds  (if  not  thousands)  of  students  actively  emailed  their 
general  (and  often  enthusiastic)  appreciation  of  the  course.  This  did  much  for  the  frayed  morale  of  teaching  staff! 

Problems  Leading  to  Reorganization 

Whilst  the  first  year  of  presentation  was  successful,  it  was  not  without  its  problems.  Students  began  to  voice 
frustration  at  having  to  go  to  FirstClass  to  find  a news  announcement  and  then  having  to  go  to  the  Web  for  the 
detailed  pronouncement.  Furthermore  we  broke  our  own  principle  of  “FirstClass  announces;  the  Web  pronounces,” 
when  it  came  to  dealing  with  clarifications  and  tips  and  hints  on  assessment  material  - for  both  urgency  and  ease 
of  student  use,  we  made  these  pronouncements  available  via  FirstClass  on  the  M206  Notice  Board.  To  avoid  this 
confusion,  for  the  1999  presentation  we  have  moved  the  M206  Notice  Board  to  our  Web  site.  The  Web  is  now  the 
sole  site  for  both  news  announcements  and  pronouncements  whereas  FirstClass  is  solely  used  for  conferencing. 

Another  problem  that  manifested  itself  with  the  1998  presentation  was  that  our  use  of  national  conferences  placed 
an  unacceptable  burden  on  the  central  academic  team  and  there  was  evidence  that  whilst  it  provided  a vehicle  for 
articulate  and  competent  students  it  did  not  always  support  students  in  difficulty.  Our  use  of  national  conferences 
also  ignored  an  important  resource  - the  250  tutors  who  are  employed  part-time  to  assess  student  assignments  and 
offer  face-to-face  tutorials  and  other  support  to  groups  of  (typically)  twenty  to  thirty  allocated  students.  We  have 
therefore  adopted  a somewhat  different  conferencing  approach  for  the  1999  presentation.  Instead  of  large  national 
conferences  each  tutor  now  manages  a tutor- group  conference  for  their  allocated  students.  The  tutor-group 
conference  is  a vehicle  for  (a  smaller,  more  focused  and  supportive)  discussion  of  the  course  and  associated  issues 
throughout  the  year,  as  well  as  being  the  basis  for  sub-conferences  to  support  group  working.  This  will  undoubtedly 
construct  a different  learning  experience  for  both  us  and  our  students.  Gone  will  be  the  wild  roller-coaster  ride  of 
an  electronic  community  of  5,000  students  and  teachers  discussing  every  aspect  of  the  course.  In  its  place  we  aspire 
to  a more  localized  and  supportive  structure  of  small  groups,  centered  around  a tutor  and  her/his  students. 

Conclusion 

A pragmatic  organizational  requirement  to  use  a proprietary  conferencing  system  as  opposed  to  a Web-based 
system  led  to  a split  in  our  computer-based  resources  we  would  rather  have  avoided.  Consequently  we  had  to 
establish  a clear  principle  for  dividing  material  between  two  “places”  students  would  see  as  distinct,  something  that 
would  be  reinforced  by  the  very  different  user  interfaces.  We  initially  determined  that  the  FirstClass  conferencing 
system  would  “announce”,  whereas  the  Web  site  would  “pronounce”.  This  supported  the  metaphor  that  FirstClass 
was  a place  to  meet  for  talk,  both  informally  and  (in  group  working)  formally  and  that  the  Web  was  a repository  of 
useful,  indeed  vital  information.  However  this  split  proved  unacceptable,  as  it  forced  students  to  access  two 
resources  in  order  to  find  important  course  information.  Hence  for  the  1999  presentation  FirstClass  is  being  used 
strictly  for  conferencing  and  the  Web  for  both  student  notices  and  static,  more  considered  information. 
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Abstract:  In  education,  our  desired  arena  of  presentation  has  moved  from  the  immediate 
environment  to  the  world,  but  we  have  not  been  able  to  make  the  necessary  paradigm  shift  in 
the  way  we  think  about  design  and  accessibility  to  allow  the  world  to  join  in.  Projects  are 
typically  dictated  by  the  content,  the  audience,  and  available  space.  Using  the  Internet  and 
Intranet  has  altered  some  of  these  limitations  but  the  main  concern  is  still  the  content.  It 
requires  that  we  look  more  carefully  at  how  to  organize  and  present  the  content.  Concerns  of 
how  to  display  a project  have  moved  from  a concern  of  space  availability  to  concerns  of 
usability  and  accessibility. 

Your  web  pages  cannot  be  all  things  to  all  people.  They  can  be  accessible  to  and  usable  by  the 
largest  audience  possible.  This  paper  looks  at  some  easy-to-implement  ways  to  do  this. 


They  Are  Only  Good  If  They  Can  Be  Used 

Multimedia/hypermedia  and  telecommunications  activities  are  only  good  if  they  can  be  used.  That,  obviously,  is 
common  sense.  Why  even  bother  stating  it? 

Why?  Because  in  many  instances  we  design  these  activities  to  keep  potential  learners  at  a distance.  Not 
deliberately,  of  course,  but  we  still  do  it,  especially  in  web-based  instruction.  In  education,  our  desired  arena  of 
presentation  has  moved  from  the  immediate  environment  to  the  world,  but  we  have  not  been  able  to  make  the 
necessary  paradigm  shift  in  the  way  we  think  about  design  and  accessibility  to  allow  the  world  to  join  in. 
Projects  are  typically  dictated  by  the  content  (that  which  is  to  be  taught)  the  audience  (those  who  are  to  be 
taught),  and  available  space  (the  size  of  model  or  poster  board  that  can  be  carried  to  school  or  the  amount  of 
shelf  or  wall  space  available  in  the  classroom).  Using  the  Internet  and  Intranet  has  altered  some  of  these  dictates 
but  the  main  concern  is  still  the  content. 

Use  of  the  Internet  or  Intranet  requires  that  we  look  more  carefully  at  how  to  organize  the  content  into 
meaningful  chunks  that  can  be  presented  in  a clear,  concise  manner  on  web  pages  that  are  robust  enough  to 
stand  alone,  yet  are  small  enough  to  be  downloaded  easily.  The  target  audience  is  no  longer  simply  the  class,  the 
school,  and  the  parents.  It  is  now  anyone,  anywhere  with  an  interest  in  the  school,  the  students,  or  the  content  of 
the  web  page.  Concerns  of  how  to  display  a project  have  moved  from  a concern  of  space  availability  to 
concerns  of  usability,  accessibility,  and  intuitiveness. 

Your  web  pages  cannot  be  all  things  to  all  people.  They  can  be  accessible  to  and  usable  by  the  largest  audience 
possible.  This  paper  looks  at  some  easy-to-implement  ways  to  do  this. 

The  Audience 

Knowing  that  your  audience  can  be  anyone — of  any  age  — ^with  any  interest  is  not  helpful  in  the  analysis 
process.  You  have  to  decide  who  your  main  audience  will  be  and  work  from  there.  If  you're  involved  with  a 
school,  you  can  assume  the  school’s  administrators,  teachers  and  students  will  be  part  of  your  audience. 

Students  in  other  schools,  relatives,  fnends,  former  students,  and  anyone  with  an  interest  in  the  use  of 
technology  in  educational  settings  will  also  be  interested  in  your  pages. 


Your  audience,  and  here  we  are  talking  only  about  the  people  who  actively  seek  you  out,  will  include  those  with 
the  fastest  modems  and  latest  computers  to  those  with  old  systems,  slow  modems,  and  monochrome  monitors.  It 
will  also  include  the  deaf,  the  hard  of  hearing,  those  who  are  blind  or  have  limited  vision,  individuals  with 
physical  disabilities,  and  users  who  feel  challenged  by  technology.  They  are  individuals  who  may  have 
problems  discerning  colors  in  graphics  and  text  or  those  who  have  turned  off  the  graphics  function  on  their 
computer  or  those  who  use  a screen  reader.  Unless  an  alternative  method  of  delivery  is  available,  content 
presented  only  via  audio  may  be  lost  to  individuals  with  hearing  impairments  or  those  without  soundcards  in 
their  machines.  Pages  with  closely  spaced  hyperlinks  can  create  navigational  problems  for  individuals  with 
certain  physical  disabilities.  Context  that  is  not  easy  to  understand,  distracting  graphics,  or  noisy  backgrounds 
make  pages  less  accessible  or  less  usable  for  most  users. 

Web  pages  that  do  not  meet  the  needs  of  the  user  will  suffer  the  fate  of  either  the  ’’Back"  or  the  ’’Stop"  button. 
Without  that  audience,  there  is  no  reason  for  your  web  pages  to  exist.  The  information  could  be  presented  just 
as  effectively  face-to-face  in  a classroom. 


What  Is  There  And  How  We  Get  To  It 

Focus  On  Content 

We  build  web  pages  to  convey  content.  If  there  is  no  content,  there  is  nothing  to  convey.  Pages  that  are  simply 
lists  of  links  to  other  sites  are  useless  if  those  sites  are  down.  The  links  you  put  on  a page  should  be  additional 
resources  to  give  more  value  to  what  is  presented  on  your  page,  not  the  primary  reason  for  the  page’s  existence. 

Unlike  printed  pages,  pages  on  the  web  must  be  designed  to  stand  alone.  A user  can  enter  at  any  page  using  any 
browser.  Just  as  you  would  want  a guest  in  your  home  to  feel  as  comfortable  as  possible,  a guest  at  your  web 
site  should  also  feel  comfortable.  Upon  entering  any  page,  the  users  should  know  where  they  are,  what  else  is 
available  to  them,  and  how  to  navigate  from  one  page  to  another. 


Provide  Relevant,  Reliable  Links 

• Check  your  links  frequently.  A page  filled  with  ’’dead”  links  is  frustrating. 

• Provide  a visible  link  at  the  top  of  each  page  to  allow  users  to  move  back  and  forth  between  the  graphical 
and  alternate  versions  of  the  page. 

• Have  the  text  only  version  of  the  page  link  to  other  text  only  versions.  Otherwise,  for  every  page  the  user  is 
linked  backed  to  a graphic  version,  only  to  have  to  link  back  to  the  text  version. 

• Place  non-link  printable  characters  (surrounded  by  spaces)  between  links  that  occur  consecutively,  e.g.,  the 
use  of  ”1"  between  the  links  at  the  top  and  bottom  of  this  page. 

• Screen  reader  users  often  read  the  page,  then  tab  through  the  page  moving  from  one  link  to  another.  Since 
they  are  tabbing  past  the  surrounding  text,  landing  only  on  the  links,  a link  may  be  meaningless.  Create 
meaningful  link  phrases.  The  phrase  "full  text  of  document"  is  more  meaningful  than  "click  here". 

• One  way  to  make  the  links  stand  out  clearly  for  the  screen  reader  user  is  to  make  the  links  separate  from  the 
body  of  the  text,  e.g.,  in  a list  or  a menu  following  the  text. 


Provide  Alternatives 

Providing  alternatives  makes  your  site  accessible  to  the  largest  number  of  people  possible.  This  means,  among 
other  things,  providing  alternative  versions  that  convey  the  same  message  as  the  graphics  or  audio. 

A page  does  not  have  to  be  dull  and  boring  to  be  equitable.  It  just  needs  to  present  alternatives. 

• Give  users  the  ability  to  use  the  browser  of  their  choice  and  not  to  be  forced  to  use  the  one  for  which  your 
page  was  designed. 
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• Use  the  "alt”  attribute  to  provide  alternated  text  for  images,  e.g.,  <IMG  SRC=”dod.gif*  alt=”Department  of 
Defense  Logo">  . In  using  the  "alt"  attribute,  keep  the  text  short.  It  might  be  easier  to  describe  the  function 
of  the  graphic  rather  than  the  graphic  itself 

• A D-link  is  a link  to  a long  text  only  description  for  graphics,  such  as  tables,  charts,  or  diagrams,  which 
have  important  information.  Example:  <IMG  SRC="enrollment.gif  ’ alt=”Chart  of  school  enrollment 
!998><A  HREF“’enrollment98.htmr>D</A>  The  ”D"  would  appear  next  to  the  graphic  of  the  enrollment 
chart.  When  the  user  clicks  on  the  ”D,”  a text  only  description  of  the  chart  will  open. 

• If  possible  DO  NOT  use  frames.  If  you  do  use  frames,  provide  an  alternative  text  version. 

• Provide  a text  transcript  for  all  information  presented  in  auditory  form. 

• Provide  an  auditory  alternative  for  all  information  presented  in  video  form. 

• Provide  a text  transcript  for  all  information  presented  in  video  form. 

• Provide  text  summaries  of  tables  for  non-visual  users. 

• Provide  both  graphic  and  text  versions  of  web  pages. 

• Users  should  be  able  to  determine  the  colors  for  backgrounds  and  text  as  well  as  the  size  of  font  used.  They 
should  be  able  to  work  in  an  environment  that  is  comfortable  for  them. 

• Include  a note  about  accessibility.  Encourage  users  to  notify  you  with  concerns  about  accessibility  or 
usability. 


Make  the  page  clear,  concise,  intuitive,  and  consistent 

In  your  classroom  doors  look  like  doors  and  the  doorknob  is  where  you  would  normally  expect  it  to  be,  not  on 

the  ceiling.  Elements  on  your  web  page  should  also  look  and  act  the  way  you  would  expect  them  to  do. 

Selecting  the  blue,  underlined  text  Contents  should  take  you  the  Contents  page,  not  start  a movie  about  the 

migration  of  ants. 

• 

• Be  consistent.  Page  elements  that  perform  the  same  function  on  different  pages  should  be  located  in  a 
constant  location  on  every  page.  Pages  that  are  consistent  in  layout  are  easier  to  navigate  and  easier  to 
maintain. 

• Users  should  be  able  to  determine  the  colors  for  background  and  text  was  well  as  the  size  of  font  used. 
They  should  be  able  to  work  in  an  environment  that  is  comfortable  for  them. 

• Use  an  internationally  understood  date  format:  January  9,  2002,  not  01/09/02. 

• Make  sure  there  is  clear  contrast  between  the  text  and  the  background. 

• The  background  should  be  just  that  - a background.  Don’t  let  it  be  more  interesting  and  attention-getting 
than  the  content. 

• Test  your  page  by  running  it  in  monochrome  mode. 

• Provide  a visible  link  at  the  top  of  each  page  to  allow  a user  to  move  back  and  forth  between  the  graphical 
and  alternate  versions  of  the  page. 

• Test  your  pages  with  a variety  of  web  browsers. 

• Keep  it  simple. 


HTML  is  not  a desktop  publisher 

HTML  not  a desktop  publishing  environment.  It  is  a content  markup  language.  Web  authors  prepare  and  present 
content.  They  use  HTML  to  structure  paragraphs,  headings,  lists,  and  tables.  It  is  the  users  who  can  configure 
their  browsers  to  choose  font  size  and  color.  When  building  web  pages,  start  with  the  content  and  then  add  the 
necessary,  relevant  the  links  and  images.  Don’t  use  HTML  to  create  a page  that  cannot  be  reconfigured  by  the 
users  to  meet  their  needs.  Use  it  to  create  pages  that  are  proportionally  structured  so  users  are  able  to  change 
font  size  and  color. 

• Use  style  sheets  to  control  layout,  alignment  and  positioning. 

• Use  standard  HTML.  Do  not  use  functions  that  will  work  only  on  specific  browsers,  e.g.,  <Blink>  is  not 
supported  by  all  browsers. 

• Marquee  text  is  often  read  one  word  at  a time  as  it  appears  on  the  screen.  It  may  be  read  backward  one 
word  at  a time  if  that  is  how  it  appears. 
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• Use  logical  rather  than  physical  markups:  for  italics  use  the  logical  <EM>  instead  of  the  physical  <I>.  For 
bold  text  use  the  logical  <STRONG>  instead  of  the  physical  <B>. 

• Provide  a single  downloadable  file  for  documents  that  appear  on  separate  pages. 


Keep  the  pages  small,  no  longer  than  a screen-and-a-half  in  length.  The  total  page  size,  including  graphics, 
should  be  about  30K,  40K  is  pushing  it.  It  should  take  no  more  than  10  seconds  to  download  a page.  Keeping  it 
small  means  it  is  usable  even  on  old  systems  with  slow  monitors. 

• Do  not  use  frames. 

• Keep  page  layout  simple  and  straightforward. 

• Use  text  that  is  clear,  concise,  and  easy  to  understand. 

• Keep  the  backgrounds  simple.  Users,  especially  those  with  low  vision,  colorblindness,  or  a monochrome 
monitor  may  have  trouble  reading  text  on  busy  backgrounds. 

• Leave  ample  space  around  items  to  be  selected. 

• Avoid  large  graphics  taking  a long  time  to  download.  Usually  a smaller  version  will  do.  Provide  a link  to 
the  larger  version. 


Tables  may  be  difficult  for  screen  readers  to  read.  If  tables  must  be  used,  provide  a text  only  version  of  the 
table. 

Try  this  to  see  how  a table  might  be  read  by  a screen  reader: 

1 . Open  a web  page  that  has  a table. 

2.  Hold  a piece  of  paper  up  to  the  screen  so  it  can  read  only  one  line  at  a time. 

3.  Read  the  line  aloud.  Does  it  make  sense? 

For  example: 

Distance  Learning  Traditional  Classroom 

allows  asynchronous  allows  teachers  to  grasp 

interaction  — not  the  teachable  moment, 

dependent  on  time 
or  place. 

With  a screen  reader,  this  might  be  read  as: 

Distance  Learning  Traditional  Classroom  allows  asynchronous  allows  teachers  to  grasp  interaction  — not  the 
teachable  moment,  dependent  on  time  or  place. 


Lists  can  also  be  confusing  for  non-visual  users.  A list  that  is  several  layers  deep  may  be  easy  to  follow 
visually;  however,  if  it  lacks  discemable  clues  to  indicate  the  layers  the  non-visual  user  will  be  unable  to  read  it 
easily.  Additionally,  if  text  wraps  to  the  next  line  it  may  appear  as  two  separate  entries. 

The  list: 

School  Supplies 
Paper 
Lined 
Graph 
Sticky  pad 
Pencils 

Black  #2  lead 
Red  for  marking 
Pens 

For  writing 


The  Display 


Tables 


Lists 


Black  ink 
Blue  ink 
For  marking 

Yellow  highlighter 
Blue  highlighter 

Might  be  read  like  this  by  a screen  reader: 

School  Supplies 

Paper 

Lined 

Graph 

Sticky  pad 

Pencils 

Black  #2  lead 

Red  for  marking 

Pens 

For  writing 
Black  ink 
Blue  ink 
For  marking 
Yellow  highlighter 
Blue  highlighter 

Use  formatting  options,  such  as  numbering  conventions,  such  as  compound  numbering  to  understand. 
School  Supplies 

1.  Paper 

1.1.  Lined 

1.2.  Graph 

2.  Pencils 

2.1.  Black  #2  lead 

2.2.  Red  for  marking 

3.  Pens 

3.1.  For  writing 

3,1,1  Black  ink 
3.1.3  Blue  ink 

3.2.  For  marking 

3.2.1.  Yellow  highlighter 


Eight  easy  tips 

1 . Provide  text  summaries  of  tables  for  non-visual  users. 

2.  Avoid  using  tables  if  possible. 

3.  Avoid  using  tables  as  a way  to  control  layout,  such  as  putting  text  in  columns. 

4.  If  you  are  using  a table  to  layout  text  and  numbers  the  screen  reader  may  append  all  of  the  numbers  from 
the  different  columns  in  the  same  row  into  one  number.  Provide  a text  only  version  if  you  want  to  use  the 
table. 

5.  Using  numbered  (ordered)  lists  makes  it  easier  for  those  using  screen  readers  to  keep  track  of  where  they 
are  in  the  list. 

6.  State  the  number  of  elements  in  the  list. 

7.  Provide  text  summaries  of  tables  for  non- visual  users, 

8.  Test  your  table. 

Being  aware  of  the  need  to  make  web  pages  more  accessible  and  usable  is  the  first  step.  The  next  step  is  to 
incorporate  accessibility  and  usability  design  features  into  your  web  based  instruction.  Using  the  design 
suggestions  mentioned  in  this  paper  will  not  make  your  web-based  materials  accessible  to  everyone.  Their 
usage  will,  however,  help  you  get  started.  Designing  accessible  and  usable  web  pages  may  require  a learning 
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curve  but,  like  all  good  things,  it  is  well  worth  the  effort.  To  help  you,  there  are  many  excellent  resources  on  the 
Internet  to  help  you  in  designing  accessible  and  usable  web  pages.  The  World  Wide  Web  Consortium  (W3C)  at 
http://www.w3.org,  one  of  leaders  in  this  area,  even  provides  an  HTML  validation  service  to  test  your  pages. 

Remember,  your  multimedia/hypermedia  and  telecommunications  activities  are  only  good  if  they  can  be  used. 
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Abstract:  Static  HTML  links  are  not  appropriate  for  quickly  evolving 
teachware  to  which  several  authors  independently  contribute  material  on 
overlapping  topics.  Furthermore  static  links  are  not  capable  of  providing 
students  with  an  individualized  linearization  of  the  hypertext  suited  to  their 
individual  level  of  previous  knowledge  and  competence.  We  hence  propose  a 
linking  mechanism  based  on  concept  maps  and  a respective  tagging  of  the 
source  material  for  the  course.  These  tags  are  used  to  dynamically  generate 
HTML  pages  that  reflect  the  learning  level  of  the  individual  students. 


1.  Introduction 

The  ability  to  use  mathematical  language  is  one  of  the  most  important  ingredients  for  a successful  study  of 
natural  sciences  and  engineering.  It  is  also  one  of  the  subjects  feared  most  by  students  since  training  in  its 
abstract  way  of  thinking  is  not  thoroughly  practiced  at  school.  Hence  a dislike  of  mathematics  or  the  self- 
perceived  failure  to  understand  its  concepts  is  one  of  the  most  frequent  reasons  for  not  choosing  technical 
subjects. 

The  Faculty  of  Physics  at  the  University  of  Heidelberg  is  thus  trying  to  smoothen  the  entry  into  the  abstract 
world  of  physics  by  offering  a two- week  tutorial  course  in  mathematics  for  first  year  undergraduates. 

During  this  intensive  course  the  students  can  revise  and  practice  their  skills  in  central  areas  of  mathematics 
like  calculus,  complex  numbers,  vectors,  and  matrices  that  are  required  in  physics  almost  from  the  first  day 
on.  Especially  students  who  are  not  fresh  out  of  school  but  who  have  completed  their  military  service  or  an 
apprenticeship  prior  to  their  studies  show  a great  need  for  intensive  recapitulation  of  mathematics. 

Since  the  mathematical  knowledge  of  first  year  undergraduates  differs  vastly  from  student  to  student  a 
personal  tutor  for  each  would  be  highly  desirable  though  of  course  not  affordable.  Therefore,  two  years  ago 
we  began  experimenting  with  various  electronically  enhanced  means  for  undergraduate  mathematics 
education  that  can  do  more  than  the  usual  multimedia-enhanced  teachware. 

Two  specific  challenges  characterize  our  project.  Firstly,  many  authors  with  different  objectives  contribute 
on  a continuous  basis  to  our  courseware  even  though  it  is  already  being  employed.  Secondly,  the  students 
using  the  courseware  are  rather  disparate  with  regard  to  their  previous  knowledge,  motivation  and  intended 
careers.  The  notion  of  links  as  incorporated  in  the  HTML  specifications  suffers  from  some  major 
drawbacks  with  respect  to  either  challenge: 

A link  can  only  be  established  if  its  destination  is  known  upon  coding  the  source  for  the  page  containing 
that  link.  If  for  example  an  author  decides  later  on  to  provide  additional  information  on  a specific  topic  all 
pages  that  are  to  link  to  that  new  page  will  have  to  be  re-edited. 

Links  to  a given  page  will  be  broken  if  that  page  is  removed.  If  for  example  an  author  splits  a page  in  order 
to  better  accommodate  the  topics  contained  in  the  original  page,  all  links  pointing  to  that  page  then  have  to 
be  updated  to  the  respective  locations  of  the  newly  created  pages.  This  requires  not  only  a complete  check 
of  all  pages  that  could  possibly  be  linked  to  the  page  in  question  but  also  a certain  understanding  of  both  the 
source  and  the  destination  topic. 
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Human  knowledge  and  human  thinking  differ  profoundly  in  structure.  While  the  structure  of  the  former  is 
interwoven  the  latter  is  in  principle  strictly  linear.  This  linearity  can  only  be  broken  by  again  picking  up 
points  of  thought  abandoned  earlier.  This  pseudo-linearity  is  commonly  realized  as  a hierarchical  tree-like 
structure. 

However,  HTML  courseware  based  on  this  principle  tends  to  resemble  what  can  be  done  in  books.  An 
unstructured  linkage  on  the  other  hand  is  hard  to  use  and  therefore  often  rejected  by  students. 

A novel  mechanism  able  to  circumvent  these  drawbacks  should  hence  be  able  to  link  the  different  parts  of 
the  courseware  using  invariant  features  of  the  content  itself.  It  must  not  depend  on  a specific  representation 
of  that  content.  The  mechanism  should  also  reflect  possible  linearizations  that  are  meaningful  to  the  reader. 
We  propose  a three- step  solution  to  that  problem:  Firstly,  we  describe  the  invariant  structure  of  the  course 
content  by  means  of  a simplified  type  of  concept  map.  We  then  tag  these  concepts  to  the  HTML  material. 
In  the  third  step  we  use  the  mapping  and  tagging  in  steps  one  and  two  to  dynamically  generate  HTML 
pages  that  have  now  been  individualized  for  each  user. 

Concept-based  structures  are  a common  tool  for  providing  HTML  metadata.  Concept  maps  or  graphs  are 
described  by  several  authors  as  a graphical  tool  for  structuring  ideas  [1],  [2].  Novak  presents  many 
examples  of  how  concept  maps  improve  the  results  of  teaching  efforts  [3],  [4].  Based  on  these  and  other 
experiences  concept  maps  have  meanwhile  been  directly  employed  in  several  hypertext  applications  [5]- 

This  paper  is  structured  as  follows.  In  section  1 we  have  given  an  overview  of  our  motivation  and  the 
problems  encountered  during  the  preparation  of  our  courseware.  In  section  2 we  present  our  concept  map 
approach  and  the  corresponding  tagging  mechanism  that  we  developed  to  handle  these  problems.  Section  3 
explains  the  dynamic  HTML  generation  and  gives  a practical  example.  Section  4 reports  on  the  early 
experience  with  our  tools.  In  section  5 we  summarize  our  project  and  draw  conclusions. 


2.  Concept  maps  and  HTML  tagging 

Thinking  is  often  equaled  to  speaking.  Particularly  if  we  also  consider  non-verbal  symbols  to  be  words  in 
an  abstract  language.  In  this  sense,  words  or  phrasal  combinations  of  them  are  the  building  blocks  of  our 
thinking.  They  reflect  our  concepts  of  a certain  subject.  Naming  concepts  is  a major  activity  during  the 
cultivation  of  new  areas  of  knowledge.  And  being  able  to  appropriately  name  and  relate  concepts  during  a 
dispute  is  taken  as  a sign  of  understanding. 

Relying  on  this  fact  we  made  concepts  the  features  on  which  we  base  our  linking.  Concepts,  however, 
should  not  be  confused  with  the  words  which  describe  them.  The  former  are  essential  and  unique,  the  latter 
somewhat  arbitrary.  The  scientific  context  often  establishes  a rather  strict  terminology,  mostly  with  one, 
sometimes  with  two  but  rarely  more  equivalent  names  for  a concept.  We  will  consider  this  equivalence 
problem  later,  for  the  time  being  taking  uniqueness  for  granted. 

As  the  building  blocks  of  our  thoughts  and  knowledge  concepts  are  glued  together  by  their  mutual 
relations.  These  in  turn  can  differ  greatly  in  depth  and  complexity,  ranging  from  easy  'has  a'- type  relations 
to  more  intricate  relations  such  as  'changes  color  if  in  contact  with'  when  for  example  describing  an 
indicator  - acid  relation.  A graphical  arrangement  of  concepts  and  their  relations  is  called  a concept  map 
and  can  serve  as  a tool  for  structuring  one's  knowledge.  Some  authors  explore  more  complex  mappings 
where  relations  can  also  be  equipped  with  attributes,  e.g.  further  concepts. 

For  our  purpose  that  multitude  of  relations  disturbs  the  automated  handling  of  concept  maps.  Relations  in 
our  initial  prototype  were  restricted  to  the  following  three  types:  'requires  an  understanding  of,  'explains' 
and  'is  an  example  of.  While  this  simplified  the  coding  and  usage  of  the  concept  map  it  also  spoiled  the 
clear  definition  of  concepts.  Information  had  to  be  incorporated  into  the  concept  that  originally  should  have 
been  contained  in  the  relation.  An  example  presented  in  the  course  is  no  concept  in  the  original  sense  but 
content  related  to  another  concept  already  defined  in  the  concept  map. 
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Figure  1:  Simplified  example  of  a relation  structure  used  by  our  initial  prototype.  Concepts  are  linked  to 
other  concepts  by  three  kinds  of  relations;  'requires  an  understanding  of,  'explains'  and  'is  an  example  of. 
The  distinction  into  different  types  of  relations  depending  on  the  didactic  use  of  a piece  of  content  spoils 
the  clear  definition  of  concepts. 

Although  our  initial  prototype  that  incorporated  this  scheme  was  rather  successful  we  improved  the  scheme 
of  concept  maps.  This  new,  improved  scheme  more  clearly  separates  the  two  types  of  concepts  which  had 
been  mixed  in  the  initial  prototype.  Concepts  describing  the  content  itself  and  the  associated  didactic 
aspects  are  now  handled  by  two  different  concept  maps.  The  first  contained  concepts  like  "Function"  and 
"Derivative",  the  second,  concepts  like  "Introduction",  "Example"  and  "Exercise".  Didactic  concept  maps 
are  much  less  complex  and  hence  treated  differently  in  our  system.  The  content  concept  maps  on  the  other 
hand  have  now  degenerated  into  a prerequisite  map,  i.e.  all  relations  have  the  meaning  "is  prerequisite  to". 
For  later  reference  we  call  concepts  that  are  prerequisite  to  a given  concept  "ancestors"  of  that  concept. 
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Figure  2:  Simplified  example  of  content  concept  maps  (left)  and  didactic  concept  maps  (right)  used  in  our 
improved  prototype.  Content  concept  maps  describe  the  relation  "is  prerequisite  to".  Pieces  of  content  in 
our  courseware  are  tagged  using  both  content  and  didactic  concepts.  This  allows  for  individualized 
linerarization  of  the  material. 

The  HTML  courseware  is  now  tagged  with  pairs  consisting  of  one  concept  from  each  category.  Each  piece 
of  content  is  provided  with  a concept  name  and  a tag  indicating  its  didactic  function. 

<CONCEPT  NAME=  "Derivative"  TYPE=  "Def inition" > The  function  mapping  x to  the 
slope  of  the  tangent  to  the  function  f in  x is  called  the  derivative  of  f.  It  will 
be  denoted  by  f*.  <\CONCEPT> 

This  tagging  can  be  done  in  parallel  to  the  authoring  of  the  text  itself  Different  authors  have  only  to  agree 
on  the  didactic  concepts,  i.e.  they  must  have  a common  idea  of  basic  didactic  tools  like  introductory  texts, 
examples  and  exercises.  However,  no  further  structural  definition  is  required.  Concepts  can  be  identified  by 
the  author  of  the  respective  part  of  the  course.  Overlapping  material  is  then  combined  automatically  by 
means  of  the  content  concept  map  and  the  tagging  in  the  HTML  source. 
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3.  Dynamic  HTML  generation 

The  content  and  the  respective  concept  maps  can  be  used  in  two  ways.  The  first  integrates  the  capability  to 
deal  with  concept  maps  into  the  browser.  The  second  possibility  employs  a modified  HTTP  server  which 
itself  processes  the  pages  before  they  are  shipped  to  the  students. 

Since  our  initial  prototype  pursued  not  only  the  integration  of  these  concept-based  navigation  facilities  but 
also  more  elaborate  multimedia  integration  of  mathematical  texts  and  Java  applets,  we  implemented  a 
special  browser  in  Java.  Here  Java  animations  could  be  seamlessly  mixed  with  the  bitmaps  created  from 
LaTeX  sources,  providing  full  rendering  of  all  mathematical  formulae  without  the  usual  hurdle  of  having  to 
mix  HTML  and  GIFs.. 

The  improved  prototype  is  currently  implemented  as  a Java  application  running  on  the  course's  web  server. 
The  applets  communicate  the  assessment  results  to  that  server  process  and  receive  in  return  the  URL  for  the 
next  course  segment. 
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Figure  3:  Brief  outline  of  the  data  flow  in  our  courseware.  The  sources  containing  the  concept  tags  are 
stored  on  the  web  server  together  with  the  students'  records.  A mixture  of  CGl-scripts  and  Java  applications 
ship  the  HTML  pages  to  the  browsers  and  maintain  the  records  with  the  assessment  results  provided  by  the 
Java  applets  on  the  client  side. 


With  the  help  of  the  concept  map  and  the  concept  tags  we  now  generate  HTML  pages  that  are  transferred  to 
the  student's  browser.  The  browsing  history  of  each  student  is  logged  on  the  server  together  with  the 
students'  results  obtained  in  the  assessments.  If  a student's  result  is  better  than  the  defined  threshold  of  0.5 
the  prerequisite  relation  is  assumed  to  haye  been  satisfied  for  that  student.  A concept  is  said  to  be 
established  if  all  its  ancestors  have  been  established  and  all  its  prerequisite  relations  have  been  satisfied. 
This  mechanism  provides  at  each  point  in  time  a set  of  concepts  with  fully  established  ancestors.  From  this 
set  and  according  to  the  didactic  concept  map  the  server  chooses  a piece  of  content  that  is  next  shown  to  the 
student.  Presentation  of  a new  concept  will  typically  begin  with  an  introductory  part.  Where  no  introduction 
has  been  provided  the  server  will  choose  an  example  to  enhance  an  explanation  or  definition.  Finally  the 
student  will  do  some  exercises  and  take  some  tests  as  an  assessment.  Sufficient  performance  on  these  tests 
then  establishes  the  concept. 

In  the  common  case  that  there  are  several  concepts  with  fully  established  ancestors,  the  student  is  presented 
with  several  possible  tracks.  This  provides  the  greatest  possible  freedom  to  the  students  while  guaranteeing 
a sound  trail  of  study. 
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Figure  4:  Screenshots  from  our  intial  prototype.  The  left  hand  side  shows  a content  concept  map  for  the 
current  concept  and  its  nearest  descendants.  On  the  right  hand  side  a typical  page  from  the  course  itself  is 
shown.  Besides  multiple  choice  questions  tests  can  require  entering  of  numerical  values  and  simple 
formulae. 
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In  our  implementation  we  paid  less  attention  to  the  students'  browsing  history,  relying  more  on  their 
assessment  results.  This  allows  the  students  to  quickly  pass  through  topics  that  are  already  clear  to  them. 
Accordingly  the  courseware  provides  introductory  pages  that  sketch  the  problem  and  directly  lead  to 
relatively  tough  questions  and  tests.  Students  are  free  to  try  these  questions  to  quickly  establish  the  concept 
or  take  the  long  way  through  the  explanatory  pages  of  the  course. 

4.  Early  experience 

Two  main  reasons  inspired  us  to  investigate  the  techniques  described  above.  Firstly,  our  students  have  a 
vastly  disparate  previous  knowledge  of  the  mathematics  taught  in  our  course.  Secondly,  the  hypertext 
material  and  the  accompanying  Java  applets  are  in  constant  evolution.  Especially  this  issue  could  not  have 
been  handled  conventionally.  During  the  two  years  of  this  project  the  text  for  the  entire  course  was 
rewritten  three  to  four  times,  not  counting  the  dozens  of  minor  changes  like  splitting  and  combining 
chapters,  or  the  insertion  of  new  examples  and  deletion  of  old  ones.  Finally  the  exercises  and  tests  were 
modified  almost  daily.  During  this  time  the  course  was  taught  by  five  professors  and  two  more  lecturers 
provided  additional  material. 

Despite  this  ongoing  evolution  we  were  always  able  to  provide  the  students  with  a working  prototype  of  the 
course.  That  prototype  could  be  handled  by  a single  person,  who  administered  the  web  and  video  streaming 
servers  and  supervised  the  students  programming  Java  applets  for  the  course.  Our  approach  greatly  reduces 
the  amount  of  work  required  to  maintain  our  teachware. 

Students  were  also  very  impressed  with  the  possibilities.  Although  the  staff  members  did  offer  tutoring  in 
persona  the  students  used  the  electronic  courseware.  After  the  two- week  introductory  course  more  than 
90%  expressed  the  opinion  that  this  courseware  could  provide  them  with  significant  tutoring  if  no 
traditional  tutoring  by  staff  members  was  available. 

However,  the  results  are  somewhat  unsatisfactory  since  the  capabilities  of  our  system  were  not  really  tested 
independently  of  the  traditional  course.  A profound  analysis  can  only  be  done  if  students  work  solely  with 
the  electronic  courseware.  Since  our  program  is  scheduled  to  be  published  by  a large  scientific  publisher  in 
1999,  we  are  confident  we  will  soon  obtain  real  data  on  usage  and  performance  of  our  courseware. 

5.  Conclusions 

Transforming  tutorials  that  have  been  traditionally  held  by  staff  members  into  interactive  courseware  is  a 
challenge  for  two  reasons:  Several  authors,  each  with  their  own  ideas  and  objectives  about  the  course, 
contribute  simultaneously  to  the  teachware.  The  resultant  constant  evolution  hinders  any  centralized 
planning.  Secondly,  the  students  differ  greatly  in  their  previous  knowledge  and  are  interested  in  different 
aspects  of  the  course.  It  was  hence  necessary  to  design  a mechanism  that  is  able  both  to  combine  different 
contributions  without  the  need  to  define  a fixed  structure  or  hierarchy  of  the  content  and  to  allow  for 
individualized  linearizations  of  the  material. 

We  chose  concept  maps  as  the  base  for  linking  the  different  parts  of  the  course.  Content  concept  maps 
reflect  the  structure  of  the  topics  taught  while  didactic  concept  maps  describe  the  different  functions  of  the 
individual  pieces  of  the  course.  Both  concept  types  are  used  to  tag  the  source  of  the  courseware.  Server 
applications  then  dynamically  generate  HTML  pages  that  are  shipped  to  the  students’  browsers. 

The  system  has  proven  very  successful.  Despite  the  constant  changes  in  the  course  a consistent  view  of  the 
material  could  be  given  at  any  time.  The  necessary  maintenance  work  was  minimal.  On  the  other  hand, 
students  could  quickly  navigate  the  material  since  the  system  was  able  to  present  them  the  material  best 
suited  to  their  individual  level  of  competency.  We  are  confident  to  soon  be  able  to  provide  a thorough 
analysis  of  the  capabilities  of  our  system  since  the  courseware  is  scheduled  to  be  published  for  general  use 
in  1999. 
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Abstract:  Increasingly  universities  have  supported  several  innovative  flexible  learning 
projects  over  the  past  few  years,  and  there  exists  a small  but  enthusiastic  group  of  staff  who 
have  demonstrated  that  something  is  possible.  Our  task  is  to  suggest  additional  strategies  that 
can  realise  some  of  the  strategic  goals  set  for  Flexible  Learning  over  the  next  few  years.  This 
paper  is  based  upon  the  assumption  that  a strategic  direction  has  already  been  chosen  and 
suggests  some  strategies  to  ensure  practical  and  realistic  outcomes. 


Introduction 

The  notion  of  flexible  learning  is  that  of  providing  students  with  choices  about  when,  where  and  how  to 
study.  Through  flexible  learning  programs  we  can  begin  to  cater  for  individual  needs  in  a higher  education 
system  that  is  expanding  to  accommodate  an  increasingly  diverse  student  body.  Flexibility  can  be  introduced  in  a 
number  of  different  forms  that  include: 

Flexibility  in  time:  Tuition  is  not  restricted  to  a set  schedule  of  events.  Rather,  students  can  choose  a time  that  is  of 
most  convenience  to  them  and  their  lifestyle. 

Flexibility  in  place:  Students  are  not  restricted  to  coming  to  the  University  campus  to  attend  teaching  sessions. 
Learning  materials  can  be  accessed  from  both  on  and  off-campus. 

Flexibility  in  the  curriculum:  Students  are  provided  with  choices  in  the  teaching  resources  and  methods  to  best  suit 
their  particular  learning  needs. 

Flexibility  in  pace:  Students  can  progress  through  a course  at  their  own  pace  in  accordance  with  their  academic 
background,  and  personal  circumstances. 

Traditionally  distance  education  programs  have  provided  some  flexibility  in  that  they  provide  choices  in 
the  time  and  place  of  study.  Teaching  programs  were  usually  delivered  using  a number  of  different  resources 
most  commonly  print,  audio  and  videotape.  However,  there  has  been  a strong  distinction  between  the  course 
offerings  and  learning  experiences  of  students  studying  on  and  off-campus.  The  more  recent  notion  of  flexibility 
is  one  that  aims  to  bridge  the  gap  between  on  and  off-campus  contexts  by  providing  all  students  with 
substantially  the  same  teaching  materials  and  learning  experiences.  Thus  students  are  truly  provided  with  a 
choice. 

Flexible  learning  is  concerned  with  access,  free  of  the  constraints  of  time  and  place,  and  organizational 
structures,  and  in  the  presence  of  dialogue  and  other  student  support  systems.  (Kirkpatrick,  1997,  p.l66). 

Fleming  (1993)  suggests  that  flexible  learning  is  taken  to  mean: 

...the  organization  and  resourcing  of  learning  in  ways  that  shift  the  emphasis  off  the  traditional  format  of 
lecture  and  seminar,  with  its  tendency  to  reinforce  a one-way  conception  of  teaching  and  learning  as  the 
dissemination  of  wisdom  from  teacher  to  learner.  Flexible  learning  offers  the  learner  a more  actively 
constructive  role  by  providing  a framework  in  which  learning  goals  can  be  more  independently  pursued 
(Fleming,  1993,p,322), 

When  reviewing  the  implementation  papers  at  her  university,  Kirkpatrick  (1997)  identified  four  different 
discourses  of  flexible  learning: 

• Flexible  learning  as  efficiency.  A concern  of  academics  in  that  they  feel  that  more  pressure  is  being  placed 
on  them  to  teach  more  students  with  the  same  or  declining  resources. 

• Flexible  learning  as  the  competitive  edge.  This  is  incorporated  into  the  university’s  strategic  plans  to  initiate 
flexible  learning  approaches. 

• Flexible  learning  as  equity  and  access.  A frequent  statement  in  the  flexible  learning  strategy. 
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• Flexible  learning  as  information  technologies.  This  is  part  of  every  dialogue,  while  saying  information 
technology  does  not  equate  with  flexible  learning,  there  are  often  plans  for  some  use  of  the  Web. 

And  in  several  institutions,  there  is  also  a fifth  discourse: 

• Flexible  learning  as  a strategy  to  increase  the  quality  of  learning  experiences.  This  discourse  is  in  line  with 
newer  approaches  to  learning  theory  and  the  advent  of  constructivism  as  an  increasingly  dominant 
philosophical  approach  to  learning  (Jonassen,  Mayes,  & McAleese,  1 997). 

The  Role  of  Technology  in  Facilitating  Flexibility 

With  the  convergence  of  computer  and  communications  technology  through  the  World  Wide  Web,  we 
now  have  the  capacity  to  offer  enhanced  forms  of  flexibility.  The  capacity  of  the  Web  to  hyperlink  and  layer 
information,  provide  interactivity  and  support  multimedia  formats  makes  it  a valuable  and  exciting  tool  which 
can  used  to  develop,  deliver  and  manage  flexible  learning  programs.  At  its  simplest  the  Web  can  be  used  to 
coordinate  and  manage  learning  and  act  as  a point  of  communication  for  staff  and  students.  At  its  most 
sophisticated  it  can  be  a virtual  learning  environment  that  facilitates  the  entire  educational  process. 

However,  while  the  technologies  can  be  used  to  increase  the  quality  of  the  learning  experience,  by  rigidly 
re-packaging  the  curriculum  rather  than  re-conceptualising  the  learning  required  for  a particular  knowledge 
domain,  it  is  possible  to  lock  students  into  the  need  to  use  technology  when  they  are  loath  to  do  so.  Without 
thought  the  exercise  may  simply  be  converting  the  learning  experience  from  a group  listening  experience  to  an 
isolated  and  singular  one. 

Several  studies  have  pointed  out  that  the  way  in  which  information  technologies  are  integrated  becomes 
critical  to  their  effective  use.  In  many  institutions  IT  is  “bolted  on”  to  each  of  the  traditional  operations  and 
practices,  they  in  effect  become  adjuncts  to  traditional  classroom  practices.  More  importantly,  they  need  to  be 
reconceptualised  so  that  they  are  joined  in  new  integrated  combinations  of  classroom  teaching  and  new 
configurations  of  older  media  and  technologies  as  appropriate.  It  is  logical  to  expect  that  in  time,  the  second 
approach  will  help  to  redefine  the  relationship  between  teaching,  and  research  for  each  academic.  In  particular, 
new  on-line  technologies  can  potentially  redefine  the  relationship  between  student  and  teacher,  enabling  non- 
sequential, concurrent  and  collaborative  relationships.  Moving  toward  the  extremes  might  mean  that  databases  of 
resources  (including  multimedia  and  other  new  forms  of  data  display)  can  be  individually  structured  for  each 
course  and  accessed  as  required  by  students.  These  resources  can  be  grown  by  staff  and  student  and  eventually 
hold  contributions  by  multiple  stakeholders. 

The  growth  of  the  Web  and  the  movement  away  from  individual  technological  implementations  such  as 
one  CD-ROM  or  a computer  assisted  learning  package  (CAL)  has  produced  an  environment  which  integrates  a 
range  of  teaching  and  learning  functions.  For  example,  the  web  enables  not  only  the  presentation  of  ideas 
including  using  multimedia  forms  but  also  computer  mediated  communications  (CMC)  in  discussion  spaces  and 
chat  rooms,  together  with  computer  managed  learning  (CML)  using  password  testing  and  result  collating 
software  like  Topclass.  In  effect  this  enables  the  mass  production  of  a series  of  useful  products  for  teaching  and 
learning.  For  instance,  it  is  possible  to  collect  a set  of  readings,  case  studies  and  other  resources  and  the  select 
from  them  for  different  groups  of  students  or  different  units. 

Elements  of  a Flexible  Learning  Program 

Several  design,  developmental,  implementation  and  management  issues  need  to  be  considered  when 
introducing  flexibility  into  educational  programs.  All  teaching  and  learning  resources  must  be  firmly  grounded 
in  the  aims  and  outcomes  of  the  unit.  The  introduction  of  flexible  resources  will  also  require  new  approaches  to 
teaching  and  learning.  Both  students  and  staff  may  need  assistance  in  accommodating  these  new  approaches. 

In  moving  a subject  into  a flexible  framework,  decisions  need  to  be  made  about  how  flexible  is  it  going  to 
be.  Remembering  that  the  greater  the  flexibility,  the  more  effort  is  required  to  keep  the  records  and  ensure  the 
students  have  a chance  to  experience  the  required  issues  in  the  most  effective  way.  For  instance,  the  14  week 
session  might  be  required  as  the  basis  for  group  and  joint  project  work.  It  is  very  difficult  to  create  groups  to 
work  on  joint  problems  if  they  are  all  at  different  stages  in  a subject.  The  skill  in  offering  alternative  study 
patterns  is  in  the  understanding  of  student  learning  and  how  instructional  strategies  need  to  be  redefined  for 
mediated  approaches  to  learning.  Strategies  such  as  problem-based  learning  might  be  considered  as  alternatives 
to  traditional  approaches  (Boud  & Feletti,  1997). 

In  any  flexible  learning  program  there  are  several  options  for  configuration.  Options  existing  for  the  way 
students  access  the  content  of  the  subject,  options  also  exist  for  the  way  they  are  assessed,  and  the  forms  of 
collaboration  that  are  possible.  Traditional  essays  and  examinations  are  of  course  possible  within  any  flexible 
framework.  However,  as  the  nexus  between  attendance  and  geography  is  broken,  the  assessment  patterns  and 
forms  can  provide  some  challenges:  both  for  marking  and  in  new  forms  of  assessment.  Marking  assignments  is 
not  easily  undertaken  electronically.  Reading  and  correcting  documents  on  screen  is  a learned  skill.  However, 
many  students  will  submit  assessments  through  e-mail  and  use  a variety  of  word  processing  software  awareness 
of  the  options  will  be  necessary  and  be  reviewed  regularly.  However,  there  are  also  tools  that  enable  a quality 


turnaround,  consider  the  use  of  the  revision  tools  in  Microsoft  Word  which  will  enable  a marker  to  annotate  the 
submission  in  a different  colour  font. 

One  technique  that  might  be  applicable  in  several  knowledge  domains  is  the  exercise  of  getting  the 
students  as  part  of  their  assessment  to  complete  some  form  of  Web  document  that  can  form  a basis  for  other 
students  learning.  Instead  of  writing  traditional  essays,  the  document  could  form  a guide  to  a new  area  of  study. 
It  might  also  be  a series  of  discussion  summaries  which  would  be  undertaken  verbally  in  a face-to-face  class  but 
the  on-line  class  might  rotate  the  task  of  summarising  the  computer  mediated  chat  sessions.  In  many  subject 
areas  such  as  information  systems  and  communications,  producing  a product  that  embodies  all  the  attributes 
being  discussed  in  the  class  is  the  most  obvious  method  to  test  for  application  and  transfer  of  the  principles 
learned. 

The  challenge  is  not  necessarily  one  of  adding  to  the  knowledge  presentation,  rather  it  is  the  unlearning  of 
traditional  ways  and  re-conceptualising  other  ways  of  achieving  similar  outcomes  with  less  demand  on  staff  and 
less  demand  on  redundant  systems.  Realistically,  universities  are  increasingly  in  a competitive  environment  and 
the  options  for  some  might  be  solved  by  electronic  access  to  prestigious  institutions  such  as:  Harvard  and 
Stanford  on-line.  From  the  work  described  in  this  paper,  provided  that  the  cost  differentials  are  appropriate,  face- 
to-face  access  to  a local  academic  will  be  more  appealing  to  many  students. 

An  Example  of  Implementing  "Flexible  Delivery" 

The  availability  of  the  Internet  is  facilitating  a change  in  teaching  and  learning  strategies  as  Universities 
take  up  the  challenge  of  investigating  flexible  modes  of  delivery  in  contrast  to  the  traditional  fixed  time  and 
place  lecture  style  format.  This  account  describes  the  first  two  implementation  cycles  of  one  particular 
postgraduate  subject  and  illustrates  the  change  that  has  occurred  in  terms  of  the  learning  process  and  the  nature 
of  assessment  by  virtue  of  the  technology  used. 

Implementation  Cycle  One:  A Postgraduate  Virtual  Class 

In  1 996  the  postgraduate  subject:  Implementation  and  Evaluation  of  Technology-Based  Learning  was 
chosen  as  a pilot  study  to  trial  World  Wide  Web  and  videoconferencing  technologies.  This  subject  had 
previously  been  offered  as  two  separate  classes  on  two  separate  evenings;  one  held  on  campus,  the  other  held  in 
Sydney  (SOkms  north).  The  use  of  the  Web  and  videoconferencing  would  allow  both  classes  to  be  held  on  the 
same  evening  with  one  instructor/lecturer. 

The  two  geographically  separate  classes  met  each  week  on  the  same  evening  for  three  hours  during  a 
fourteen  week  semester.  There  were  eight  students  and  a participant  observer  in  the  on-campus  class  and  six 
students  in  the  Sydney  class.  The  instructor  physically  met  with  each  group  on  alternate  weeks.  Interaction 
between  the  two  sites  was  facilitated  by  a class  web  site  and  videoconferencing  except  for  the  last  week  when  all 
the  students,  the  instructor  and  the  participant  observer  met  face-to-face. 

For  assessment  purposes,  students  were  required  to  complete  three  pieces  of  work: 

1 . A seminar  presentation  of  an  agreed  topic  to  facilitate  learning  and  discussion  (30%  of  final  grade).  This 
task  was  to  be  conducted  as  a collaborative  exercise  between  two  students,  one  from  each  site.  Students 
were  to  lead  a discussion  for  the  entire  evening  and  provide  online  activities  to  engage  the  virtual  class.  A 
web  page  was  to  accompany  the  presentation  and  each  student  was  required  to  submit  a piece  of  writing,  in 
the  form  of  a publishable  paper  for  assessment.  The  paper  was  due  three  weeks  after  the  seminar 
presentation  or  at  the  end  of  semester,  whichever  was  earlier. 

2.  A portfolio  of  resources  relating  to  implementation  and  evaluation  of  technology-based  learning  (40%  of 
final  grade).  Students  were  to  compile  useful  articles,  evaluation  instruments,  web  sites,  etc.  that  would  help 
them  further  their  understanding  of  the  issues  raised  in  the  subject.  Each  item  collected  was  to  be 
accompanied  by  a short  reflective  piece  that  explained  its  selection  as  a useful  resource.  The  portfolio  was 
due  at  the  end  of  semester. 

3.  An  evaluation  of  an  educational  software  package  or  a review  of  the  implementation  of  this  subject  as  a 
technology-based  learning  project  (30%  of  final  grade).  Th  evaluation  report  was  due  at  the  end  of  semester. 

Content  delivery  was  structured  so  that  a new  topic  was  addressed  each  week.  The  first  five  weeks 
consisted  of  instructor-led  discussions  and  workshops  and  the  remaining  nine  weeks  were  designed  as  student- 
led  seminars.  Figure  1 depicts  the  model  that  emerged  from  this  subject. 

The  themes  that  surfaced  from  this  implementation  are  summarised  as  follows: 

• Nature  of  interaction 

On-line  interaction  was  predominantly  synchronous  in  nature.  "Synchronicity*'  was  encouraged  by  the 
nature  of  the  course  design  that  was  structured  to  present  new  content  each  week.  Due  to  much  social  interaction 
taking  place  within  each  physical  class,  the  role  of  the  web  site  became  a storage  mechanism  for  student  seminar 
web  pages  and  a pointer  to  the  online  discussion  tools. 
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A problem-based  approach  was  used  to  facilitate  online  interaction.  All  the  online  discussions  were 
structured  around  a problem  or  a series  of  questions  that  students  needed  to  answer.  On-line  interaction  occurred 
mostly  during  ’’class  time"  and  was  not  linked  to  an  assessable  task.  The  quality  of  discussion  was  influenced  by 
time  constraints  and  it  is  contended  that  the  incentive  and  motivation  for  students  to  engage  in  meaningful 
discussion  was  affected  by  the  n on-assessable  nature  of  the  tasks. 

• Nature  of  student  activity 

Students  attended  class  each  week  and  worked  either  individually  or  in  small  groups  when  contributing  to 
the  online  class  discussions.  Apart  from  a small  group  exercise  that  was  organised  in  the  early  weeks  of  semester 
and  pair  work  that  occurred  for  some  of  the  seminars  (not  all  students  presented  in  pairs),  there  was  little  need 
for  group  collaboration. 

• Nature  of  assessment 

Grading  was  based  on  individual  work  submitted.  The  majority  of  assessable  work  was  due  at  the  end  of 
semester.  (Seminar  papers  were  submitted  from  Week  9 to  Week  14). 

• Nature  of  content 

Content  material  drove  the  learning  process.  Each  week  students  attended  class  expectant  to  address  a 
new  area  of  content.  Upon  reflection,  the  strategies  implemented  to  present  content  and  encourage  student 
interaction  (both  in  the  physical  groups  and  online)  were  couched  within  a traditional  university  lecture 
framework  of  fixed  weekly  meetings  (see  Agostinho,  Lefoe  & Hedberg,  1997  for  specific  examples).  The 
technology  was  used  to  facilitate  a weekly  "virtual  class"  and  little  asynchronous  discussion  transpired. 

For  the  subsequent  implementation  we  felt  that  the  features  of  asynchronous  communication  could  be 
better  utilised  and  tasks  given  to  students  could  provide  a more  flexible  and  effective  learning  environment.  The 
reliance  of  fixed-weekly  meetings  could  be  reduced.  In  an  asynchronous  mode  students  would  be  able  to  spend 
more  time  discussing  an  issue  and  contributions  to  the  discussion  could  occur  outside  the  set  "class  time".  Face- 
to-face  meetings  or  workshops  could  then  be  better  utilised  to  discuss  student  concerns,  resolve  problems  or 
provide  technical  "skilling"  workshops.  However,  to  implement  this  concept  requires  a change  in  the  learning 
process.  In  this  case  the  content  and  weekly  meetings  drove  the  learning  process.  In  the  following 
implementation  we  wanted  to  use  the  assessment  tasks  as  the  facilitator  of  the  learning  process. 
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Figure  1:  Virtual  Class  with  a "Content"  Focus 


Implementation  Cycle  Two:  A Postgraduate  Virtual  Community 

In  1997  the  subject  was  restructured  to  encourage  a more  flexible  asynchronous  approach.  During  the 
fourteen  week  semester,  students  attended  eight  class  meetings  (Weeks  1,  2,  4,  6,  8,  10,  11  and  14)  and  were  to 
participate  in  asynchronous  and  synchronous  online  discussions  during  the  non-meeting  weeks.  The  two 
geographically  separated  groups  (eleven  students  and  a participant  observer  in  the  on-campus  class  and  six 
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students  in  the  Sydney  class)  met  for  three  hours  on  different  evenings  and  the  instructor  was  physically  present 
in  each  group.  A class  web  site  facilitated  interaction  between  the  two  groups  and  amongst  all  the  students 
outside  class  time. 

Assessment  tasks  were  restructured  as  follows: 

1.  A written  paper  based  on  class  and  online  discussion  about  evaluation  theory  and  its  relevance  to 
technology-based  learning  was  to  be  completed  by  Week  4 (Worth  30%  of  final  grade). 

2.  A Web  Study  Guide  on  an  agreed  topic  to  facilitate  learning  and  discussion  was  to  be  completed  by  Week  8 
(Worth  30%  of  final  grade). 

3.  An  evaluation  of  all  the  Web  Study  Guides  was  to  be  completed  by  Week  1 1 (Worth  10%  of  final  grade). 

4.  An  evaluation  proposal  for  a technology-based  learning  project  was  to  be  completed  and  presented  as  a 
group  project  in  Week  14.  (Worth  30%  of  final  grade). 

Tasks  were  completed  at  various  inter\^als  during  the  course  rather  than  submission  at  the  end  of 
semester.  In  doing  so,  the  assessable  tasks  became  resources  that  students  accessed  to  assist  them  with  the  final 
group  project.  Thus,  the  first  three  tasks  act  as  "stepping  stones"  to  the  final  group  project. 

The  delivery  of  content  was  structured  with  a task  focus.  Content  material  was  provided  to  help  with  the 
completion  of  an  assessment  task.  The  concept  of  a Web  Study  Guide  replaced  the  need  for  content  to  be 
presented  in  class  time  in  a sequential  sequence  as  the  content  is  in  an  electronic  form  that  can  be  reviewed  by 
students  at  a time  and  place  convenient  to  them. 

The  instructor  incorporated  several  online  tasks  during  the  semester.  The  first  online  task  involved 
students  working  in  small  groups  to  asynchronously  discuss  the  content  required  for  the  first  assignment.  Each 
group  worked  on  different  content  and  posted  a summary  of  their  discussions  on  the  class  web  site.  The  intention 
was  that  students  could  then  view  the  summaries  from  the  different  groups  to  help  them  write  the  first 
assignment.  Other  online  tasks  included  synchronous  and  asynchronous  discussions  about  particular  readings. 
The  model  that  emerged  from  this  implementation  is  illustrated  in  Figure  2. 

The  themes  that  surface  are  summarised  as  follows: 

• Content,  on-line  tasks  and  assessment  are  integrated 

Delivery  of  separate  chunks  of  content  was  replaced  with  an  "assessment  chunk".  In  this  model,  content 
material  and  online  tasks  support  the  production  of  an  assessable  piece  of  work.  The  technology  is  used  in  a 
much  more  integrated  way  as  web  page  summaries  and  Web  Study  Guides  are  then  available  as  resources. 

• Nature  of  student  activity 

In  the  early  weeks  of  semester,  students  engaged  in  group  work  and  produced  a group  summary  to  assist 
in  the  completion  of  the  first  assignment.  Asynchronous  discussions  were  encouraged  to  discuss  readings  whilst 
production  of  the  student’s  individual  Web  Study  Guide  took  place.  Then  a period  of  individual  self-paced  work 
occurred  when  each  student  reviewed  other  students'  Web  Study  Guides.  Group  work  then  occurred  for 
production  of  the  final  group  project.  Therefore,  although  students  did  not  meet  each  week  there  was  a sense  of 
community  as  they  were  involved  in  either  physical  group  work,  online  class  discussions  or  reviewing  other 
student’s  work. 

It  became  apparent  that  for  many  students  the  non-meeting  weeks  were  viewed  as  a "week  off’.  In 
summary,  the  feedback  obtained  through  a questionnaire  suggests  that  students  are  still  working  to  come  to  grips 
with  this  way  of  learning.  They  exemplify  the  concern  by  asking  for  more  incentive,  more  time,  more  structure 
and  more  guidance. 

• Nature  of  on-line  interaction 

There  were  varying  patterns  of  online  interaction  during  the  semester.  For  example,  as  students  were  busy 
with  Web  Study  Guide  production  the  online  environment  became  very  quiet.  Students  expected  facilitation  in 
the  online  discussions  mainly  from  the  instructor.  There  was  little  spontaneous  online  discussion,  although  some 
students  did  attempt  to  initiate  online  contact. 

• Nature  of  content  delivery 

More  than  half  of  the  class  thought  that  reviewing  the  Web  Study  Guides  was  an  effective  learning 
strategy  in  comparison  to  sitting  in  class  listening  to  the  traditional  student  seminar  presentation.  As  Web 
authoring  tools  become  easier  to  use,  it  is  possible  for  students  to  use  such  tools  to  construct  their  own 
interpretation  of  the  course  content.  In  effect,  they  can  use  the  Web  environment  as  a cognitive  tool  (Reeves  & 
Reeves,  1997).  The  learner  is  the  designer  (Jonassen  & Reeves,  1996).  The  student  becomes  the  author. 

• The  evolving  nature  of  the  on-line  environment 

By  producing  a Web  Study  Guide  and  then  using  the  Web  Study  Guides  as  background  for  the  final 
project,  students  are  creating  resources.  This  becomes  a resource-based  learning  environment  (Hannafin,  1997). 
This  evolving  dynamic  nature  of  Web-based  resources  means  that  subsequent  groups  of  students  can  access 
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previous  student  products  and  on-line  discussions  which  enables  the  concept  of  "learning  on  the  shoulders  of 
others"  to  result.  (Hedberg,  Brown,  Larkin  & Agostinho,  in  press)  Unlike  delivering  a traditional  face-to-face 
course  where  the  same  content  is  presented  over  and  over  again  and  the  same  assessment  tasks  assigned,  this 
dynamic  learning  environment  enables  students  to  learn  from  their  predecessors  taking  "learning"  to  a new 
plateau. 
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Figure  2:  Virtual  Community  with  a "Task"  Focus 
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Conclusion 

The  focus  of  this  paper  was  to  discuss  the  rhetoric  and  the  required  infrastructure  for  the  flexible  delivery 
of  courses  in  different  ways  within  a traditional  campus  and  to  illustrate  an  example  of  how  one  subject  has 
changed  to  a more  flexible  delivery  approach.  As  the  nature  of  academic  discourse  changes,  it  will  be  expected 
that  the  infrastructure  and  the  courses  offered  flexibly  will  become  more  commonplace.  Often  postgraduate 
students  seek  out  potential  places  to  study  based  on  course  and  common  interests;  this  paper  has  described  how 
technology  can  be  used  to  promote  a virtual  community  to  provide  a flexible  yet  effective  learning  environment. 
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Abstract:  This  paper  discusses  the  emerging  trends  in  the  use  of  delivery  systems  in  the  field 
of  training  and  development,  the  process  used  to  explore  those  trends,  lists  identified  trends 
and  makes  recommendations  on  how  to  implemented  those  changes  in  the  curriculum  of  a 
Masters  of  Science  program  in  Training  and  Development. 


Introduction 

There  is  a well-documented  need  expressed  by  business,  industry,  military,  government,  and  education 
to  more  effectively  deliver  training.  This  need  is  bound  to  increase  as  the  demand  for  training  increases  and 
organizations  want  to  enhance  the  quality  of  training  while  containing  cost.  To  meet  this  need  media  systems 
are  becoming  increasingly  important  in  the  effective  and  efficient  delivery  of  training  and  are  changing  at  rapid 
rate. 

The  Situation 

The  University  of  Wisconsin-Stout  (UW-Stout)  offers  a graduate  level  program  in  Training  and 
Development.  The  "hands  on-minds  on"  portion  of  the  program  emphasizes  integrating  theory  with  practice  to 
design,  develop,  and  delivery  training  programs.  Previous  studies  and  reviews  of  the  current  literature,  indicated 
there  is  a need  to  monitor  and  continually  enhance  the  delivery  of  training  through  the  use  of  mediated  delivery 
systems.  Feedback  from  the  Training  and  Development  program  advisory  committee,  employers  and  graduates 
reinforced  this  need. 

In  a 1995  study  (Furst-Bowe)  recommended  that  programs  in  training  and  development  need  to  review 
their  instructional  technology  courses  and  competencies.  The  study  also  found  that  there  is  a need  to  examine 
the  professional  development  needs  of  trainers  in  the  area  of  instructional  technology.  A 1 996  (Schlough)  study 
found  that  it  was  important  for  all  UW-Stout  graduates  to  be  able  to  create  multimedia  presentations  and 
participate  in  videoconferences.  The  study  also  stated  because  of  rapid  changes  in  the  competencies  needed  in 
the  workplace  that  educational  institutions  continually  monitor  those  changes. 

Literature  Review 

The  following  chart  from  longitudinal  studies  conducted  by  the  American  Society  for  Training  and  Development 
(http://www.astd.org/virtual  communitv/research/bench/96stats/graph  1 3.gif)  shows  the  changes  in  the  use  delivery  systems 
over  a three-year  period  and  gives  an  overview  of  current  delivery  systems. 


Percent  of  Companies  Using  Selected 
Delivery  Systems  (1994-1996) 


1994 

1995 

1996 

Classroom 

97% 

100% 

10C>% 

Advanced  Techiiolog>’  Classroom 

53% 

56% 

47% 

Televised  Distance  lA^arnttig 

47% 

64% 

69% 

CBT 

72% 

90% 

S4% 

Intei-acCive/MuHiinedia  CBT 

47% 

85% 

81% 

1 11  te rn  et/i\c t>vork-Ba  sed  Electronic 
DisUmce  Learning 

12% 

33% 

53% 

EPSS 

34% 

45% 

37% 

Otli«>r  Scir-Paccd  Dellvwt' 

66% 

77% 

75% 

Other  Delivery 

12% 

10% 

10% 

Mote:  available. 

Copyri^t  19^7  ASTHy  fiicl  Mtrinbcrs  of  <ls«  AS'llI  ljef)clsi»irkiii^  l^onnsi 


AstoBaichinaikiig  Forum 


Table  1:  ASTD  study  of  the  use  of  delivery  systems  in  training.  Definitions  of  the  delivery  systems  follow. 

Classroom  (Instructor-led  lecture) 

Definition:  Classroom  training  that  is  lecture-  or  discussion-  based.  Also  referred  to  as 
instructor-led.  Does  not  include  advanced  technology,  interactive  classrooms,  or  electronic 
distance  learning. 

Advanced  technology/interactive  classroom 

Definition:  An  advanced  technology/interactive  classroom  is  an  instructor-led  classroom  with 
some  degree  of  automated  facilities,  including  student  response  units. 

Televised  electronic  distance  learning 

Definition:  Televised  electronic  distance  learning  refers  to  teleclasses  broadcast  via  satellite, 
microwave,  telephone  lines,  or  other  means.  May  or  may  not  be  interactive. 

Internet/Network-based  electronic  distance  learning 

Definition:  Intemet/network-based  electronic  distance  learning  refers  to  CBT  classes  made 
available  via  Internet  or  other  electronic  networks.  May  or  may  not  be  interactive. 

Computer  Based  Training  (CBT) 

Definition:  CBT  uses  a computer  as  an  integral  part  of  an  instructional  system,  with  the 
learner  engaging  in  real-time  interaction  with  the  computer. 

Interactive/Multimedia  CBT 

Definition:  Interactive/multimedia  CBT  uses  any  two  instructional  media  together;  interactive 
multimedia  platforms,  typically  employ  a computerized  learning  programs  that  are  learner- 
controlled.  Learning  response  will  initiate  one  of  several  program  sequences,  and  can  be 
presented  in  video  form  as  still  shots,  regular,  slow,  and  fast  speed  motion.  May  include  sound 
and  other  features. 
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Electronic  performance  support  systems  (EPSS) 

Definition:  EPSS  is  an  integrated  computer  program  that  provides  any  combination  of  expert 
system,  hypertext,  embedded  animation,  CBT,  and  hypermedia  to  an  employee  on  demand. 
Electronic  performance  support  systems  allow  employees  to  perform  with  a minimum  of 
support  and  intervention  by  others  (e,g,,  help  systems,  electronic  job  aids,  expert  advisors). 

Other  self-paced  instruction 

Definition:  Other  self-paced  instruction  represents  any  other  method  of  instruction  not 
identified  above  where  the  learner  controls  the  rate  of  instruction.  Typically  does  not  require 
advanced  technology  or  unusual  equipment.  Includes  such  methods  as  paper-based  instruction 
using  training  manuals  and  videotapes 


Among  the  future  trends  appears  to  be  the  movement  toward  more  Internet  delivery  of  video  and  more 
video  simulation,  John  Chambers  (1997)  CEO  of  Cisco  felt  that  training  video  distributed  over  networks  will  be 
the  application  that  drives  the  growth  of  PC  networks,  McLean  (1997)  discusses  that  the  use  of  digitized  images 
and  video  to  create  simulations  for  training  has  the  potential  to  become  a $4  billion  market. 

Based  on  these  findings,  the  following  actions  were  taken  to  assure  that  the  course.  Delivery  Systems 
for  Training,  stayed  current  and  continued  to  meet  the  needs  of  students  in  Training  and  Development  program 
at  UW-Stout, 

Methodology 

To  discover  new  trends  related  to  delivery  systems,  it  was  originally  proposed  to  do  an  in-depth  phone 
needs  assessment  and  a series  of  site  visits.  It  was  felt  the  from  the  literature  review  and  other  information 
obtained  that  it  would  be  prudent  for  the  Communications,  Training,  and  Education  Department  to  obtain  a 
digital  video  to  allow  students  to  easily  integrate  digital  video  into  their  projects,  A grant  proposal  was 
submitted  to  the  Stout  University  Foundation  to  cover  these  items.  Funding  was  provided  to  purchase  the  digital 
video  camera,  but  was  not  adequate  to  carry  out  the  in-depth  research  project,  A Hitachi  MP-EGl  A camera  was 
purchased  and  produces  acceptable  results. 

As  it  was  not  possible  to  do  the  research  as  planned,  an  alternative  was  implemented.  Taking  in  to 
consideration  that  the  students  in  the  course  Delivery  Systems  for  Training  were  graduate  students  and  the 
trends  continually  change,  the  trend  analysis  was  turned  into  the  following  student  project,  which  was  assigned 
during  the  Spring  semester  of  1998,  There  were  9 students  enrolled  in  the  course. 


Trends  in  Delivery  Systems  for  Training 

You  will  need  to  explore  future  trends  related  to  delivery  systems  to  be  proactive  in  this  rapidly 
changing  and  critical  area  for  training  and  education.  To  discover  new  trends  related  to  delivery  systems 
you  will  conduct  a trend  analysis.  This  analysis,  quantitative  in  nature,  will  focus  on  obtaining  data  from 
individuals  who  develop  leading  edge  training  programs.  As  a class  you  will  develop  the  trend  analysis 
instrument  and  procedure.  Each  student  will  contact  one  respondent,  either  by  phone  or  in  person  to 

answer  of"  What  do  you  feel  will  be  the  biggest  change  in  delivering  training  over  the  next years," 

or  similar  question(s).  Each  student  will  write  up  a write  up  a one  to  two  page  paper  on  his  or  her 
findings.  You  will  exchange  papers  and  develop  a presentation  based  on  the  findings.  The  presentation 
will  be  presented  during  the  final  class  period.  Before  you  conduct  your  interview  make  sure  the  class 
has  came  to  a consensus  on  the  instrument,  the  procedures,  format  for  the  paper,  and  format  for  the 
presentation.  Make  sure  NO  ONE  contacts  the  same  site  and  you  have  my  approval  before  you  begin. 


Evaluation  Criteria 


Paper 


Format 


012345 


Technical  Execution 


01 2345 


Content 


02  46  8 10 


1137 


Total 


Team  Presentation 

Delivery  0 1 2 3 4 5 

Visual  Appeal/Technical  Execution  0 1 2 3 4 5 
Content  0 2 4 6 8 5 

Total 


Your  evaluation  by  your  peers  is  worth  up  to  15  points 

You  are  to  evaluate  the  other  members  of  the  class  on  their  role  as  a team  member  in  the  trend  analysis 
project.  You  should  rate  each  member  on  a 15-point  scale.  I will  average  the  scores  your  classmates 
gave  you  and  that  will  be  the  number  of  points  you  earn.  The  following  questions  should  help  you 
evaluate  your  team  members. 

Was  the  individual  cooperative? 

Did  the  individual  take  initiative? 

Was  the  person  open-minded  and  accepting  of  other's  ideas  and  opinions? 

Did  the  individual  contribute  a fair  share  to  the  success  of  the  group? 

Did  you  feel  the  person  acted  in  a professional  manner? 

Did  the  person  keep  the  goals  and  objectives  of  the  group  in  mind? 

Did  the  individual  complete  their  designated  tasks  in  a timely  fashion? 

Did  the  individual  submit  materials  that  were  accurate  and  composed  well? 

Was  the  individual  in  attendance  and  punctual  for  group  meetings? 

How  would  you  rate  this  person's  overall  ability  to  function  as  an  effective  team  member? 


The  students  meet  as  a team  and  developed  procedures  and  the  following  research  questions: 

What  training  delivery  systems  has  your  organization  used  in  the  past  5 years? 

What  delivery  systems  do  you  currently  use? 

What  delivery  systems  to  you  expect  to  be  using  in  the  next  5 years? 

What  constraints  does  your  organization  face  with  the  development  and  use  of  training  delivery 
systems? 

The  students  contacted  a luxury  hotel,  a national  computer  manufacturer,  an  international  computer 
component  manufacturer,  a national  brewing  company,  a regional  high-tech  manufacturer,  a public  utility 
company,  a leading  manufacturer  of  innovative  products,  a university,  and  a national  aeronautics  company.  The 
results  of  their  study  are  presented  in  the  next  section. 

Results 

The  following  is  a composite  the  challenges  and  constraints  that  were  identified: 
o Money  & Budget 
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o Time 
o Justification 
o Development 
o Selection 
o Time  to  Train 
o Geographic 

o Information  Systems  Support 
o Staffing 
o WWW  Access 
o Language  Barriers 
o Management  Support 
o Training  Quality 
o Soft  Skills 
o Shifts 

o Process  Changes 
o Apathy  Toward  Training 
o Union  (Trust  Issue) 
o Keeping  Up  With  Technology 


The  following  table  identifies  the  use  of  delivery  systems  in  organizations: 


SYSTEM 

PAST 

PRESENT 

FUTURE 

Instructor  Led/Classroom 

9 

9 

9 

Self‘Directed  Study 

4 

5 

4 

CBTAVBT 

5 

7 

9 

OJT 

3 

5 

4 

Video  / Audio: 

6 

7 

4 

Satellite  Downlink: 

1 

1 

3 

Video  Conferencing: 

2 

4 

2 

Intranet: 

0 

0 

3 

CD-ROM 

0 

3 

3 

Table  2:  Use  of  identified  delivery  systems. 

Summary 

This  method  of  research  proved  to  be  a viable  alternative  to  the  originally  planned 
research.  Although,  the  results  of  the  research  were  based  on  a small  population  they  appear 
to  give  a good  snapshot  of  what  is  happening  in  training  delivery  systems.  The  same  project 
was  repeated  in  the  Summer  of  1998  with  thirteen  students  contacting  thirteen  different 
organizations  and  producing  similar  results.  The  advantages  of  this  methodology  is  that  the 
students  individually  talk  to  a training  manager,  collaborate  at  the  beginning  and  end  of  the 
project,  spend  time  to  together  discussing  their  results,  get  current  results,  and  have  a sense  of 


1139 


ownership  in  the  results.  It  is  planned  to  continue  to  use  the  project  and  refine  as  time  goes 
on. 
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From  Research  to  Teacher  Professional  Development  to  Technology 
and  Back  Again:  The  development  of  A Video  Exploration  of  Classroom 


Abstract:  This  paper  describes  a research-based  process  for  the  development  of  teacher 
professional  development  materials.  A Video  Exploration  of  Classroom  Assessment  is  a 
CD-ROM-based  set  of  assessment  workshops  designed  to  be  used  by  small  groups  of 
teachers  at  their  school  settings.  Unstaged  classroom  video  of  assessments  is  a key 
feature.  At  the  Institute  for  Research  on  Learning,  we  incorporated  ongoing  teacher 
professional  development  and  assessment  research  into  every  aspect  of  the  development 
of  our  assessment  CD-ROM.  Everything  from  the  target  audience  and  setting  to  the 
interface  design  incorporated  research  findings.  This  has  resulted  in  an  extremely 
functional  and  usable  product,  which  itself  can  help  us  organize  future  research. 


Introduction 

If  you’re  a teacher,  the  opportunity  to  watch  the  goings  on  in  another  classroom  is  a rare  and  exciting 
thing.  The  opportunity  to  talk  about  your  observations  with  a group  of  teachers,  and  plan  experimental 
teaching  based  on  your  discussion,  is  still  more  rare.  Even  more  unusual  is  the  opportunity  to  obtain 
perspectives  and  advice  from  teachers  who  have  tried  similar  experiments. 

This  paper  tells  the  story  of  the  development  of  a CD-ROM  called  A Video  Exploration  of  Classroom 
Assessment.  This  assessment  CD-ROM  gives  teachers  all  these  opportunities  as  they  work  to  develop  their 
repertoire  of  classroom  assessment  strategies.  Teachers  can  view  classroom  video  of  assessments  in  a real 
classroom,  and  get  perspectives  of  teachers  and  students  who  have  tried  the  various  assessment  strategies. 
The  CD-ROM  includes  complete  workshops  for  teachers  who  want  to  develop  assessment  strategies 
together. 

The  story  of  the  assessment  CD-ROM  is  important  because  it  illustrates  the  crucial  symbiosis  and 
synergy  between  educational  research  and  educational  product  development.  The  assessment  CD-ROM 
was  developed  at  the  Institute  for  Research  on  Learning  (IRL).  IRL  is  committed  to  a research  strategy  that 
is  deeply  embedded  in  real-world  practices.  The  assessment  CD-ROM  is  the  culmination  of  many  years  of 
IRL’s  research  in  classrooms  and  in  a teacher  professional  development  (TPD)  community. 

In  the  next  three  sections  we  will  describe  the  research  projects  that  inspired  and  contributed  to  the 
design  of  the  assessment  CD-ROM,  explain  how  research  findings  were  incorporated  into  the  design  of  the 
CD-ROM,  and  suggest  directions  for  further  research  uncovered  by  our  development  process. 

Historical  Overview 

An  emerging  community  in  MMAP 

The  assessment  CD-ROM  grew  as  the  next  logical  step  to  a series  of  research  and  development 
projects  at  IRL.  We  had  formed  a community  of  about  30  teachers  who  were  co-developers  of  our  Middle- 
school  Mathematics  through  Applications  Project  (MMAP)  curriculum  (funded  by  the  National  Science 
Foundation).  These  teachers  worked  with  us  over  five  years  and  came  to  IRL  for  monthly  work  days  and 
three-week  summer  institutes.  Work  days  included  MMAP  curriculum  development  activities,  math  and 
technology  workshops,  guest  speakers,  and  collaboration  on  our  research  on  math  teaching  and  learning. 
Although  nominally  about  curriculum  development,  the  MMAP  teacher  community  became  a rich 
professional  development  experience  for  participants  and  a research  opportunity  for  us  (Lichtenstein, 
Weissglass  et  al.  1998;  Greeno,  McDermott  et  al.  in  press). 

A year  of  assessment  research 

It  became  clear  as  the  work  days  progressed  that  assessment  was  a primary  interest  and  concern  of  our 
teacher  community.  Because  MMAP  is  a project-based  rhath  curriculum,  old  assessment  methods  were  not 
always  appropriate,  and  using  authentic  assessments  required  learning,  time,  and  effort  for  many  teachers. 
We  decided  to  conduct  research  on  ways  of  building  classroom  assessment  systems.  We  spent  a year  of 
intensive  work  with  one  teacher,  Mona  Muniz,  developing  assessment  strategies,  testing  them  in  her 
classroom,  and  revising  our  strategies  (Cole  1996).  We  video  taped  extensively  in  Mona’s  classroom  and 
analyzed  the  tapes  at  teacher  work  days.  We  also  solicited  comments  from  our  teachers  on  the  assessments 
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we  developed.  This  project  continued  a trajectory  of  several  years  of  this  type  of  assessment  research  at 
IRL  (Moschkovich  1994;  Hall,  Knudsen  et  al.  1995/1996). 

The  Assessment  Club 

These  activities  inspired  the  creation  of  the  Assessment  Club,  which  met  monthly  at  IRL  to  discuss  and 
design  assessment  strategies.  Teachers,  as  a result  of  their  participation  in  Assessment  Club  activities, 
began  to  experiment  in  their  own  classrooms,  and  reported  their  experiences  and  difficulties  in  Assessment 
Club  meetings.  Several  teachers  became  experts  and  resources  for  the  group  in  particular  assessment 
techniques.  Every  teacher  in  the  club  used  new  assessment  techniques  that  year,  and  most  began  to  use  a 
greater  variety  of  assessments  (Goldman  and  Cole  1997).  We  had  a growing  collection  of  very 
knowledgeable  teachers  on  our  hands, 

A Video  Casebook 

Many  of  the  liveliest  discussions  in  MMAP  and  Assessment  Club  were  inspired  by  classroom  video 
from  members’  classrooms.  Teachers  found  reasons  to  watch  the  tapes  again  and  again,  as  they  tried  to 
interpret  the  effects  assessments  had  on  student  learning.  Video  supported  in-depth  discussion  and 
examination  of  teaching  practices  in  a way  that  oral  or  written  cases  did  not.  We  began  to  compile  video 
cases  that  teachers  outside  MMAP  could  use  to  seed  discussion  about  using  standards-based  math 
curriculum.  The  result  was  a video  tape  of  real  classroom  vignettes  called  Seeing  powerful  mathematics. 
We  tested  this  tape  on  groups  of  teachers  outside  of  MMAP  and  found  that  their  reaction  to  real  classroom 
video  was  as  positive  as  that  of  our  MMAP  teachers  (Goldman  1996). 

As  MMAP  neared  completion,  we  felt  the  need  to  develop  a product  by  and  for  teachers  that  would 
capture  the  diverse  and  extensive  knowledge  MMAP  teachers  had  about  assessment.  Although  much  of 
this  expertise  had  been  fed  back  into  the  development  of  the  MMAP  curriculum  and  support  materials,  we 
wanted  to  reach  teachers  outside  MMAP.  We  also  wanted  to  inspire  teachers  to  form  supportive  and 
dynamic  learning  groups  like  the  one  that  had  developed  in  MMAP,  and  give  such  groups  materials  to 
work  with  once  they  had  formed.  The  next  section  details  how  all  this  research  work  informed  the  design 
of  the  assessment  CD-ROM. 

Creating,  a tool  for  collaborative  teacher  learning 

Valuable  communities  around  assessment  had  been  created,  and  we  needed  to  find  a way  to  keep  this 
kind  of  community  formation  and  discussion  alive.  Looking  back  at  all  of  our  research,  what  had  been  the 
key  factors  in  stimulating  discussion  and  growth  around  assessment?  We  needed  to  answer  this  question 
before  we  started  making  a tool  that  had  the  main  focus  of  producing  this  type  of  interaction. 

From  Research  Findings  to  Technology  Choices 

We  synthesized  the  findings  from  our  research  into  five  key  elements  we  wanted  to  preserve  as  we 
moved  to  a technology-based  tool: 

• Unstaged  classroom  video:  In  Assessment  Club  meetings  and  other  MMAP  teacher  workday 
activities,  discussion  and  learning  had  always  resulted  when  the  teachers  were  confronted  with 
unstaged  classroom  video  of  assessment  in  action.  This  fact  alone  mandated  that  we  needed  to 
present  real  classroom  examples  that  would  spark  discussion  around  assessment  practices.  This 
meant  that  out  tool  needed  to  have  a space  for  video. 

• Access  to  classroom  artifacts:  We  saw  the  tremendous  benefit  of  teachers  sharing  student  work 
and  assessment  tools  they  had  created.  This  meant  that  along  side  the  classroom  video,  we  also 
needed  to  have  a space  where  pictures  of  paper  documents  could  appear. 

• Multiple  Perspectives:  Our  teachers  had  learned  from  hearing  the  opinions  of  the  teachers  around 
them.  Having  multiple  teachers'  perspectives  available  in  our  tool  was  a must. 

• Workshop  structure:  We  also  realized  that  we  needed  to  imbed  a structure  inside  the  tool  that 
would  help  teachers  find  what  they  wanted  and  support  discussion  and  experimentation.  We 
decided  to  organize  the  content  into  workshops  similar  in  structure  to  Assessment  Club  meetings. 

• Clear  and  inviting  interface:  Lastly,  we  knew  that  the  tool  would  have  to  be  inviting  and 
accessible  to  teachers.  For  example,  in  our  testing  of  the  Seeing  Powerful  Mathematics  video,  we 
found  that  teachers  needed  a transcript  to  hear  video  conversations  clearly,  and  that  they  used 
these  transcripts  to  go  back  to  parts  of  the  video  for  discussion.  Easy  transcript  management  thus 
became  one  of  many  interface  requirements  for  making  the  whole  tool  useful  to  teachers. 

Keeping  all  of  these  factors  in  mind,  the  one  piece  of  existing  technology  that  met  all  of  our  needs  was 
the  CD-ROM.  A CD-ROM  would  allow  us  to  present  classroom  and  interview  video  and  all  the  supporting 
material  we  needed. 
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Figure  1 shows  how  the  workshop  screen  of  the  CD-ROM  is  organized.  The  upper  left  comer 
contains  the  workshop,  including  prompts  for  discussion,  video  watching,  student  work  analysis,  and 
experimentation.  Beneath  the  workshop  frame  is  a frame  for  showing  student  work  or  other  artifacts  from 
the  video  being  shown  at  the  upper  right.  Perspectives  from  other  teachers  and  students  also  show  in  this 
window.  Finally,  the  lower  right  frame  contains  continuously  updated  transcript  of  the  video  currently 
being  played.  Teachers  can  click  on  any  point  in  the  transcript,  and  the  video  will  begin  playing  at  that 
point. 

Figure  I:  Workshop,  classroom  video,  artifacts,  and  transcripts 


The  choice  of  the  CD-ROM  media  thus  gave  us  two  powerful  ways  to  meet  our  goals.  First,  video, 
transcripts,  and  artifacts  can  all  be  synchronized  within  a workshop  stmcture.  Second,  teachers  can  quickly 
and  easily  access  any  part  of  the  video  and  play  it  as  many  times  as  they  want  to. 

Restructuring  the  video  presentation 

Our  first  step  in  creating  the  assessment  CD-ROM,  A Video  Exploration  of  Classroom  Assessment^  was 
to  capture  the  classroom  video  that  would  be  the  core  of  our  disk’s  content.  We  turned  to  a teacher,  Mona 
Muniz,  who  was  a primary  collaborator  in  our  assessment  research.  Our  first  inclination  was  to  follow  her 
through  an  entire  year  of  her  classroom  assessment.  This  approach  matched  our  research  techniques,  where 
we  often  taped  weeks  of  class  in  chronological  order. 

When  we  brought  our  research  team  together  to  look  at  the  tape  we  were  gathering  from  Mona’s  class, 
we  grew  dissatisfied  with  the  chronological  organization.  We  remembered  that  when  teachers  went  to  a 
piece  of  classroom  video,  it  was  always  in  the  context  of  a question  they  had  or  an  issue  they  wanted  to 
explore.  Teachers  who  would  use  our  tool  needed  to  be  able  to  access  video  according  to  their  interests  and 
questions.  Our  unstaged  tape  needed  a new  organization. 

In  considering  what  a teacher  would  be  looking  for  when  working  with  a CD-ROM  focused  on 
assessment,  a natural  organization  came  to  the  surface.  Mona  was  primarily  working  with  four  assessment 
techniques:  peer  review,  notebooks,  walk-arounds,  and  testing.  We  decided  to  group  the  classroom  video 
on  these  specific  techniques,  so  that  teachers  working  with  the  disk  could  focus  in  on  one  technique  at  a 
time.  We  also  had  the  video  from  interviews  with  multiple  teachers  and  two  students.  They  had 
commented  on  specific  assessment  techniques  and  issues,  and  their  comments  could  be  paired  with  our 
classroom  examples.  This  change  from  presenting  the  content  in  chronological  order,  as  we  had  been  used 
to  doing  during  our  research  analysis  was  a large  shift  in  thinking,  and  it  was  critical  to  the  development  of 
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a useful  teacher  tool.  Figure  2 shows  how  teachers  can  select  the  six  workshops  that  resulted  from  the  new 
organization. 


A Video  Exploration  of 
Classroom  Assessment 


I Help  I IControl Panel  | 


Figure  3:  Discussion  starters,  teacher  interviews,  and  teacher  tools 
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Realizing  that  there  are  many  other  important  assessment  techniques  and  issues  outside  of  those 
covered  in  Mona's  classroom,  we  interviewed  teachers  to  explore  their  experiences  with  self  assessment, 
portfolios,  presentations,  rubrics,  working  with  parents,  and  students  with  limited  English.  Figure  3 shows 
these  "Discussion  Starters,"  where  teachers  who  are  using  the  disk  can  read  an  outline  of  an  issue,  hear 
from  teachers  on  the  disk,  and  then  discuss  the  topic  on  their  own. 
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Interface,  look  and  feel,  and  ongoing  development 

During  the  five  years  of  working  with  teachers  on  the  development  of  a middle-school  math 
curriculum  that  included  units  that  involve  technology,  we  saw  that  teachers’  comfort  with  technology 
varies  considerably.  In  designing  the  disk's  look  and  feel,  we  wanted  it  to  be  inviting  and  intuitive  to  use. 
Through  every  step  of  the  design  process,  we  kept  our  end  goal  in  mind:  creating  a disk  that  would  feel 
inviting,  warm,  and  professional.  We  combined  familiar  images  of  the  classroom  with  a binder-like 
interface  design. 

Another  challenge  we  faced  during  this  CD-ROM's  production  was  that  we  knew  that  once  the  disk 
was  finished,  duplicated,  packaged,  and  shipped,  there  was  no  way  to  further  develop  the  content  presented 
on  the  disk.  This  kind  of  finality  was  never  felt  during  the  research  process.  With  our  research,  if  another 
intriguing  theme  emerged  from  our  data,  wc  could  always  start  writing  a new  paper  and  continue  our 
analysis.  We  addressed  this  concern  by  creating  a web  site  (www.irl.org/assess/assess.html)  that  can  be 
reached  by  pressing  the  "web  site"  button  inside  the  disk.  Inside  the  workshops,  we  encourage  teachers  to 
go  to  this  web  site  to  report  on  their  use  of  these  assessment  techniques.  They  can  also  read  what  others 
teachers  have  had  to  say.  This  is  our  attempt  to  keep  the  dialogue  going  and  growing  outside  of  the  disk. 
Preliminary  testing  and  feedback 

When  we  selected  the  preliminary  video  segments,  we  tried  them  out  with  teachers  to  get  their 
reactions.  Were  we  on  the  right  track?  We  found  that  the  cases  that  we  picked  did  indeed  spark  discussion, 
and  teachers  found  it  very  easy  to  talk  about  the  positive  aspects  of  the  assessment  as  well  as  the  ways  that 
they  might  try  to  make  it  better.  These  discussions  highlighted  another  positive  aspect  about  the  disk.  It 
gave  teachers  a classroom  to  talk  about  that  was  not  their  own  or  their  peer  teacher's.  They  felt  free  to 
comment  openly  about  what  they  saw,  and  their  reactions  were  not  guarded.  The  disk  gives  teachers  a 
credibility  for  trying  new  techniques.  Having  the  teachers'  on-going  feedback  about  the  content  inside  A 
Video  Exploration  of  Classroom  Assessment  as  well  as  its  design  and  functionality  was  invaluable. 

When  teachers  had  a chance  to  work  with  the  disk,  here  were  some  of  their  reactions: 

• “I  really  like  seeing  the  classroom;  it  feels  so  real.  The  teacher  perspectives  help  with  knowing  how  to 
realistically  apply  these  techniques  in  my  classroom.  Very  nicely  done.” 

• “I  love  the  assessment  CD-ROM;  it  is  wonderful  to  be  inside  Mona’s  classroom.  Four  other  teachers 
and  I get  together  and  watch  sections  together.” 

• “It  is  thought  provoking.  It  gives  some  concrete  suggestions.  It  stimulates  conversation  between  my 
colleagues.” 

• “I  am  a MSDP  Evaluator  (Middle  School  Demonstration  Program)  for  John  F.  Kennedy  Middle 
School  in  Redwood  City,  CA.  This  school  is  doing  some  intensive  work  in  math  assessment.  In  my 
report  to  them,  I recommended  the  use  of  the  CD-ROM,  A Video  Exploration  of  Classroom 
Assessment^  as  an  invaluable  resource  for  their  summer  school  training  of  math  teachers  and  training 
throughout  the  school  year.  This  report  also  goes  to  the  State  Department  of  Education.” 

• “The  Assessment  CD-ROM  is  excellent.  Our  school  is  planning  on  using  it  in  our  staff  development 
program  in  August  after  returning  from  summer  break.” 

• “Should  you  develop  further  materials  that  explore  student  leaming/teaching  in  a similar  way,  I should 
be  most  interested  to  learn  about  them.” 

Back  again:  Directions  for  research 

Now  that  the  CD-ROM  is  finished,  we  have  shifted  our  thinking  back  to  research  mode.  Our  field 
testing  has  left  us  with  at  least  three  big  questions  to  explore.  First,  what  is  necessary  to  support  the  many 
possible  uses  for  the  CD-ROM?  It  is  our  hope  that  A Video  Exploration  of  Classroom  Assessment  will  be 
used  in  multiple  settings.  The  disk  is  useful  to  both  current  teachers  who  are  going  through  continual 
professional  development  as  well  as  to  future  teachers  who  are  in  teacher  training  programs.  Teachers  use 
the  disk  alone  and  in  groups  with  other  teachers.  We  can  improve  our  understanding  of  teacher  professional 
development  by  exploring  how  tools  are  used  differently  in  different  settings. 

Our  second  question  is,  how  can  we  keep  the  content  of  the  workshops  current  and  continue  to  spark 
discussion  as  teachers’  needs  change?  Web  delivery  is  one  obvious  future  alternative,  and  others  are 
emerging.  Whatever  technology  we  use,  our  research  will  continue  to  drive  our  innovations. 

Our  third  question  is,  how  does  using  our  tool  change  what  teachers  do  with  their  students?  We  know 
our  disk  is  successful  in  that  teachers  like  and  use  the  disk,  but  we  do  not  know  much  about  what  happens 
in  classrooms  as  a result  of  this  kind  of  professional  development  experience. 
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We  believe  the  tight  integration  of  ongoing,  multifaceted  research  with  product  development  was  the 
biggest  factor  in  the  ultimate  high  quality  of  our  CD-ROM.  This  connection  works  the  opposite  way  too: 
by  keeping  research  in  mind  through  the  product  development  cycle,  we  have  a whole  new  set  of  questions 
to  explore. 
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Abstract:  Based  on  the  design  of  a hypermedia  application  for  educational  purposes,  a 
hypermedia  course  developed  through  some  hypermedia  model  or  methodology,  it  is  possible 
to  produce  the  authoring  or  effective  implementation  of  the  course.  To  support  the  effort  of 
designing  hypermedia  courses,  here  called  pre-authoring  activity,  one  must  use  appropriate 
models  and  tools.  This  paper  presents  a pre-authoring  environment  supporting  the 
development  of  hypermedia  courses,  called  APACHE.  This  environment  offers  supporting 
tools  to  simplify  the  author's  task  of  designing  hypermedia  knowledge  domains,  based  on  a 
model  that  incorporates  both  concept  maps  and  information  mapping  techniques.  It 
propitiates,  still,  the  automated  design  of  guided-tours  for  hypermedia  courses,  according  to 
pedagogic  criteria  imposed  by  the  author,  the  target  population  and  the  learning  objectives. 
Based  on  the  designs  of  hypermedia  contents  and  related  course  guided-tours  developed  under 
APACHE,  it  is  possible  to  implement  the  hypermedia  course  easily  using  any  hypermedia- 
authoring  tool. 


Introduction 

For  designing  general  hypermedia  applications  several  models  found  in  the  literature  can  be  used,  such 
as  HDM  (Garzotto  et  al.  1993),  RMD  (Balasubramanian  1994),  OOHDM  (Schwabe  & Rossi  1995),  etc. 
However  these  models  are  not  suitable  for  modeling  hypermedia  courses,  since  they  do  not  incorporate  didactic 
and  pedagogic  aspects  in  its  processes  of  modeling  the  hypermedia  content  and  related  course  guided-tours  (or 
trails).  To  support  the  effort  of  designing  hypermedia  courses,  here  called  pre-authoring  activity,  one  must  use 
appropriate  models  and  tools.  The  problem  is  in  defining  how  to  design  both  the  hiperbase  and  related  course 
guided-tours  before  doing  the,  strictly  speaking,  authoring  task. 

This  paper  presents  such  a pre-authoring  environment  supporting  the  development  of  hypermedia 
courses,  called  APACHE.  The  paper  organization  is  the  following:  firstly  the  architecture  of  the  APACHE 
environment  and  the  research  context  is  presented;  next  the  APACHE  pre-authoring  tools,  called  SICH  and  SPR, 
are  showed. 


APACHE  Environment  and  the  Research  Context 

For  the  purpose  of  this  paper,  we  understand  a hypermedia  course  as  including  the  following: 

• A specific  hyperbase  for  the  subject 

• One  or  more  navigational  guided-tours  or  trails  defined  on  the  nodes  of  the  related  hyperbase, 
where  a hyperbase  is  a base  of  subject  nodes  related  through  concrete  links  and  a guided-tour  is  a sequence  of 
nodes  of  the  hyperbase. 

A guided-tour  is  a course  when  it  presents  a certain  subject  in  a didactic  and  pedagogic  sequence  of 
nodes.  In  this  case,  some  additional  nodes  added  to  the  sequence  do  not  refer  to  related  nodes  belong  to  the 
hyperbase.  A guided-tour  is  a hypermedia  course  when  one  or  more  hyperlinks  in  a number  of  nodes  belonging 
to  the  sequence  is  active,  except  forward-backward  links  that  must  be  always  active.  The  course  author  must 
predefine  whether  a hyperlink  in  the  guided-tour  is  to  be  active  or  not. 

Usually  three  activities  are  considered  in  the  development  of  hypermedia  courses:  pre-authoring, 
authoring  and  reader  or  learning  activities.  In  a pre-authoring  activity,  all  the  hypermedia  context  of  an  intended 
course  is  designed  beforehand.  This  includes  the  design  of  the  hyperbase  and  related  guided-tours.  In  this  case, 
the  design  of  the  contents  of  a hypermedia  course  can  be  developed  by  using  hypermedia  models  or 
methodologies.  In  an  authoring  activity,  the  hypermedia  content  of  an  intended  course  is  effectively  developed. 
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considering  or  not  previous  designs  of  hyperbases  and  related  guided-tours.  In  the  reader  activity,  an  apprentice 
can  navigate  through  the  guided-tours  of  the  course  trying  to  satisfy  the  objectives  imposed  by  the  course  author. 
Both  the  authoring  and  the  reader  activity  can  be  supported  by  academic  or  commercial  hypermedia  systems. 

Some  tools  supporting  the  pre-authoring  activity  are  reported  in  the  literature  (Gaines  & Shaw  1997, 
Moroni  & Senas  1997).  Such  a tool  is  presented  in  this  paper.  In  order  to  help  the  author  to  design  the 
hypermedia  content  of  a course  prior  to  the  authoring  activity,  the  APACHE  environment  offers  support  to  the 
pre-authoring  activities.  APACHE  is  an  acronym  for  Pre-Authoring  Environment  for  Extended  Hypermedia 
Courses.  (Fig.  1 ) shows  the  whole  research  context,  emphasizing  the  architecture  of  the  APACHE  environment. 

Through  the  SICH  tool  one  can  develop  a base  of  hyperbase  designs.  Through  the  SPR  tool  one  can 
develop  the  base  of  guided-tours  designs  based  on  the  hyperbase  designs  developed  with  SICH  and  considering 
didactic,  pedagogic  and  motivational  aspects.  In  the  authoring  activity,  by  using  commercial  tools  such  as  Quest 
and  ToolBook  or  specific  tools  for  Internet,  the  real  hyperbase  and  the  real  guided-tours  of  the  hypermedia 
course  will  be  created.  The  learner,  using  a hypermedia  system  as  an  interface,  will  be  able  to  navigate  freely 
through  the  hyperbase  nodes  or  follow  the  trails  of  hyperbase  nodes  indicated  by  the  guided-tours.  In  the  latter 
case,  the  learner  can  be  coached  by  the  system  in  a non-intrusive  way. 
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Figure  1:  Research  context  and  the  APACHE  environment. 


Pre-authoring  Tools 

The  pre-authoring  tools  help  design  and  model  the  hiperbase  and  related  guided-tours.  In  the  design  of  a 
hyperbase,  the  hypermedia  knowledge  domain  is  modeled.  In  the  design  of  guided-tours,  a sequence  of  concepts 
taken  from  the  hyperbase  is  modeled  in  a didactically  effective  way  by  the  teacher.  Next  the  two  pre-authoring 
tools  offered  by  the  APACHE  environment,  namely  SICH  and  SPR,  will  be  presented. 
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SICH  Tool 


SICH  (acronym  for  Hypermedia  Course  Construction  Tool)  offers  support  to  the  author  in  the  design  of 
hypermedia  knowledge  domains  or  hyperbases.  The  underlying  model,  called  Daphne  (Kawasaki  1996), 
incorporates  both  concept  maps  and  information  mapping  techniques  in  a harmonic  way.  The  goal  is  to  supply 
the  teacher  with  effective  means  to  design  the  hypermedia  content  of  the  course,  according  to  the  objectives  and 
the  methods  borrowed  from  pedagogic  techniques  used  for  designing  courses  for  traditional  classes, 

SICH  is  composed  of  a manager  for  hypermedia  courses  and  four  auxiliary  editors,  as  prescribed  in  the 
Daphne  Methodology:  editor  for  description  of  the  course,  concept  map  editor,  information  map  editor  and  editor 
for  direct  access  structures.  The  manager  for  hypermedia  courses  allows  doing  the  following  tasks:  creating, 
opening,  deleting  and  renaming  a course.  Each  course  will  be  the  root  of  a specific  file  directoiy.  From  the 
manager  of  hypermedia  courses  one  can  have  access  to  the  editors  description  of  the  course,  concept  map, 
information  map  and  direct  access  structures. 

Through  the  editor  for  description  of  the  course,  one  can  identify  in  a concise  form  the  whole  course, 
defining  the  area  of  interest  of  the  course  and  its  extent.  Next,  one  can  define  the  general  objective,  writing  in  a 
simple  sentence  what  is  intended  that  the  apprentice  must  know  after  completing  the  course.  The  specific 
objectives  and  sub-objectives  in  the  cognitive,  affective  and  corporal  levels  must  be  also  defined. 

The  concept  map  is  an  educational  tool  of  great  versatility  (Jonassen  & Marra  1995,  McAleese  1990, 
Novak  & Gowin  1984).  The  use  of  the  concept  map  technique  allows  structuring  the  knowledge  domain  of  a 
hypermedia  course  in  a graphic  form.  This  technique  has  been  successfully  used  as  a simple  and  explicit 
modeling  of  educational  software  (Kawasaki  1996,  Moroni  & Senas  1997).  In  the  process  of  construction  of 
concept  maps,  one  can  define  the  group  of  information  that  should  compose  the  course,  considering  that  learners 
may  bring  a degree  of  previous  knowledge  on  the  subject. 

The  concept  map  editor  helps  the  teacher  build  a specific  concept  map  for  a given  target  subject.  (Fig. 
2)  illustrates  the  four  types  of  nodes  used  in  a concept  map.  The  creation  of  a node  is  accomplished  through  the 
corresponding  buttons  Initial,  Middle,  Final  and  Context.  An  initial  node  corresponds  to  the  name  of  the  current 
concept  map.  A middle  node  corresponds  to  intermediary  concepts.  Final  nodes  correspond  to  nodes  without  any 
relation  to  child  nodes.  Finally,  context  nodes  will  indicate  other  concept  maps,  facilitating  the  nested  or 
hierarchical  construction  of  concept  maps.  The  initial,  middle  and  final  nodes  will  directly  indicate  information 
maps.  In  (Fig.  3),  a graphical  interface  shows  an  example  of  a concept  map  elaborated  by  the  concept  map  editor. 


Initial  Middle  Final  Context 


Figure  2:  Four  types  of  nodes  in  a concept  map. 

The  information  map  editor  helps  the  teacher  to  structure  the  nodes  of  a concept  map  through  the 
information  mapping  technique.  Information  maps  provide  a high  degree  of  modularization  for  the  instructional 
material.  They  help  organize,  store  and  present  information.  This  process  consists  of  dividing  the  target 
information  into  more  detailed  pieces  of  information  called  blocks  and,  in  more  general  way,  to  obtain  an  overall 
understanding  of  the  subject,  called  maps.  Blocks  of  information  represent  the  smallest  portion  of  information 
that  can  be  individually  accessed.  When  these  blocks  are  organized  in  larger  and  related  structures,  they  form 
information  maps. 

According  to  Horn,  all  basic  information  can  be  classified  in  seven  types  of  information  maps  (Horn 
1989):  Concept,  Structure,  Procedure,  Fact,  Process,  Classification  and  Principle.  Each  one  of  these  types 
consists  of  a group  of  two  or  more  blocks  of  information  referring  to  a certain  topic.  The  following  are  examples 
of  types  of  blocks:  definition,  introduction,  formula,  example,  remark,  etc.  In  the  interface  for  building 
information  maps,  the  inner  circle  represents  a topic,  whose  content  corresponds  to  an  initial,  middle  or  final 
node  taken  from  a concept  map.  In  (Fig.  4),  the  graphical  interface  shows  an  example  of  a concept  map  topic, 
namely  the  initial  node  of  the  concept  map  illustrated  in  (Fig.  3),  with  four  information  maps,  each  one  with  its 
respective  blocks. 

Finally,  the  interface  of  the  direct  access  structure  editor  provides  functions  for  defining  glossaries  and 
bibliographies.  In  this  case,  the  teacher  can  register  specific  words  and  their  respective  meanings  related  to  the 
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target  domain  or  subject.  He  can  also  register  the  authors  and  respective  data  of  publications.  The  lists  of  words 
or  bibliographic  references  are  alphabetically  ordered,  facilitating  the  search  during  the  edition  task.  The 
information  defined  through  this  editor  will  provide  a course  with  additional  resources  for  the  learner.  For 
instance,  when  navigating  through  a hypermedia  course,  the  learner  will  be  able  to  access  the  meaning  of  a word 
belonging  to  the  course  glossary  directly. 
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Figure  3:  Graphical  interface  illustrating  the  construction  of  a concept  map. 

SIGH  is  implemented  in  the  programming  language  Java,  an  object-oriented  language  that  executes 
independently  of  the  platform  and  browser.  In  the  design  of  the  interfaces  the  essential  activities  aiming  at 
minimizing  cognitive  overload,  and  the  correspondent  adaptation  effort  of  the  author  to  the  interface  were  taken 
into  account.  The  operations  of  saving  and  opening  files  are  automatic  in  all  the  four  editors.  The  tool  also 
facilitates  the  reuse  of  the  content  of  a hypermedia  course  material  in  other  courses. 


Related  Works 

For  building  concept  maps  in  hierarchical  levels,  there  are  two  main  tools  in  the  literature:  JMap 
(Gaines  & Shaw  1997)  and  MCH  (Moroni  & Senas  1997),  SIGH  is  also  a tool  for  building  hierarchical  concept 
maps.  JMap  presents  a diagrammatic  notation  with  only  three  kinds  of  nodes,  a not  so  friendly  user  interface,  and 
does  not  allow  reusing  concept  maps  in  other  similar  applications.  SIGH  presents  a better  diagrammatic  notation, 
offering  four  kinds  of  nodes.  It  has  a friendlier  and  highly  interactive  user  interface,  offering  a strategy  of 
automatically  opening  and  saving  files  related  to  a specific  course,  minimizing  both  the  cognitive  overloading 
and  adaptation  effort  of  the  author  to  the  interface.  SIGH  also  presents  a specific  editor  for  developing 
information  maps,  not  present  in  JMap,  which  allows  better  concept  structuring  and  concept  maps  reuse. 
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Figure  4:  Graphical  interface  illustrating  the  construction  of  a information  map. 

Both  SICH  and  MCH  allow  the  reuse  of  concept  maps.  However,  MCH  presents  a diagrammatic 
notation  with  only  two  kinds  of  nodes  and  only  a one-to-one  relationship  between  nodes.  Besides,  the  content  of 
each  node  is  inserted  directly  on  the  same  screen  where  the  nodes  are  created,  making  it  a confusing  operation. 
The  diagrammatic  notation  of  SICH  offers  two  kind  of  relationship:  one-to-one  and  one-to-many.  For  inserting 
the  content  of  a concept  node,  SICH  offers  another  editor,  allowing  the  developing  of  information  maps. 

There  are  other  tools  for  developing  concept  maps  in  the  literature  (Fisher  et  al.  1990,  Jonassen  1995), 
all  similar  to  Jmap  and  MCH,  but  differing  significantly  from  SICH. 


SPR  Tool 

SPR  (acronym  for  Guided- tours  Designing  Tool)  provides  special  features  for  the  automated  design  of 
guided-tours  for  hypermedia  courses,  according  to  pedagogic  criteria  imposed  by  the  author,  target  population 
and  learning  objectives. 

A hypermedia  course  is  composed  of  one  or  more  entities,  where  each  entity  is  a complete  set  of 
information  about  a certain  subject.  The  inner  structure  of  a hypermedia  course  can  be  determined  through  a 
concept  map.  In  the  context  of  the  course,  an  entity  or  topic  can  be  classified  as  belonging  to  one  of  the 
following  categories:  Body,  Complementary  Note  and  Prerequisite.  Body  is  the  set  of  new  information  that  the 
author  intends  the  learner  to  acquire.  Prerequisite  is  all  knowledge  that  the  learner  has  to  know  before  doing  the 
course.  Complementary  Note  is  all  information  used  for  illustration,  comparison  or  reinforcement  of  the  new 
concepts  introduced  in  the  Body  part. 

The  design  of  a guided-tour  takes  into  account  the  correspondent  hyperbase  design.  A hypermedia 
course  can  have  one  or  more  navigational  guided-tours.  The  goal  of  a guided-tour  is  to  facilitate  the  learning  of 
the  subject.  A course  guided-tour  has  a didactic  way  for  presenting  de  subject,  following  some  pedagogic  and 
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motivational  scheme. 

SPR  is  an  undergoing  tool,  written  in  Java,  offering  a friendly  and  interactive  interface  supporting  the  author 
of  the  course  in  the  following  tasks: 

• Selecting  the  subjects  for  each  part  of  the  course  design  and  classifying  the  selected  entities  as 
belonging  to  the  Body,  Complementary  Note  or  Prerequisite. 

• Designing  a sequence  of  pre-defined  learning  units  based  on  the  correspondent  hyperbase. 

• Designing  of  limited  and  programmed  access  to  the  hypermedia  content  related  to  the  topic  under  the 
learner's  attention. 


Conclusions 

The  process  of  modeling  hypermedia  courses  should  include  the  pre-authoring  task  for  the  design  of 
hiperbases  and  related  guided-tours.  The  APACHE  environment,  composed  of  SICH  and  SPR  tools,  will  help 
the  planning  and  development  of  a hypermedia  course.  The  SICH  tool  allows  a teacher  to  design  educational 
applications  on  different  knowledge  domains,  through  a friendly  and  interactive  interface.  This  aspect 
contributes  effectively  to  the  creation  of  educational  hypermedia  material.  The  SPR  tool  allows  the  teacher  to 
define  guided-tour  designs  for  hypermedia  courses  based  on  the  correspondent  hyperbase.  Finally,  once 
APACHE  is  implemented  in  Java,  it  can  be  used  for  designing  hypermedia  courses  through  the  WWW. 
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World  Wide  Web  Based  Simulations  for  Teaching  Biology. 
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Abstract:  The  Biology  Labs  On-Line  Project  is  an  attempt  to  create  simulations 
of  important  biology  experiments  that  students  normally  can  not  perform  in  typical 
undergraduate  laboratories.  All  of  the  simulations  are  designed  so  that  the  student 
has  the  flexibility  to  design  and  interpret  their  own  experiments.  The  programs 
generate  large  amounts  of  data  and  are  fast  enough  for  students  to  do  multiple 
experiments.  The  simulations  are  all  written  in  Java  and  are  accessed  over  the 
World  Wide  Web.  Ten  of  the  simulations,  covering  the  topics  of  evolution, 
Mendelian  genetics,  protein  translation,  human  population  demography,  protein 
structure-function,  human  genetics,  mitochondrial  electron  transport,  enzyme 
kinetics,  cardiovascular  physiology  and  photosynthesis  will  be  ready  by  the 
summer  of  1999,  with  five  more  planned  for  the  summer  of  2000. 


Introduction 

The  Biology  Labs  On-Line  Project  is  a component  of  the  California  State  University  System  (CSU) 
Integrated  Technology  Strategy  (ITS),  which  calls  for  anywhere,  anytime  access  to  information.  The  project 
initially  brought  together  biologists  from  throughout  the  CSU  system  and  the  CSU  Center  for  Distributed 
Learning  (CDL)  to  explore  ways  to  use  technology  to  improve  learning  in  introductory  biology  laboratories. 
Later,  multimedia  developers  from  Addison  Wesley  Longman  were  added  to  the  development  team.  A major 
goal  of  the  collaboration  was  to  allow  students  to  learn  as  biologists  do,  /.e.,  by  actively  designing 
experiments  and  interpreting  their  results.  Another  goal  was  to  extend  student  learning  opportunities  by 
creating  simulations  of  experiments  that  they  might  not  normally  do  because  of  expensive  and/or  inaccessible 
lab  equipment,  lack  of  field  opportunities,  time  constraints,  complexity,  danger,  or  ethical  problems. 
Eliminating  the  time  constraints  of  the  traditional  experiment  and  two  or  three  hour  laboratory  period  gives 
students  the  opportunity  to  design  and  interpret  experiments,  learn  from  their  mistakes,  and  to  revise  and  redo 
their  experiments  just  like  real  scientists.  The  simulations  are  not  designed  to  replace  the  traditional  “wet 
labs”  found  in  the  normal  biology  course,  but  rather  are  designed  to  extend  the  laboratory  experience  to 
subjects  and  experiments  that  can  not  normally  be  done,  or  not  done  enough,  in  a traditional  laboratory.  The 
simulations  are  also  not  multi-media  presentations,  stand-alone  tutorials  or  on-line  courses. 

The  project  has  produced  seven  different  educational  simulations  covering  the  subjects  of  evolution, 
Mendelian  genetics,  protein  translation,  human  population  demography,  protein  structure-function,  human 
genetics,  and  mitochondrial  electron  transport.  Current  projects  in  progress  and  due  to  be  finished  by  the 
summer  of  1999  will  simulate  enzyme  kinetics,  cardiovascular  physiology  and  photosynthesis.  While  each  of 
the  simulations  is  unique,  all  of  them  share  many  common  interface  elements  and  functions.  All  of  the 
simulations  have  been  designed  so  that  the  student  can  carry  out  many  different  experiments,  allowing  the 
student  to  design  and  interpret  their  own  experiments.  The  flexibility  of  the  programs  make  well  designed 
exercises  an  important  part  of  each  laboratory.  However,  while  sample  exercises  will  be  included  with  each 
lab,  instructors  can  easily  design  their  own  exercises  to  meet  the  needs  of  their  students. 

The  simulations  have  all  been  created  in  the  Java  programming  language,  so  that  they  can  be  easily  accessed 
over  the  web  using  any  standard  browser.  This  solves  the  problem  of  widely  disseminating  the  applications,  a 
common  problem  with  most  educational  software.  The  Java  application  provides  the  user  interface  where 
students  set  the  starting  parameters  for  their  experiment  and  get  graphical  feedback  on  their  current  settings. 
In  some  of  the  simulations  the  Java  program  also  calculates  the  results,  while  in  others  the  input  parameters 


are  passed  back  to  the  server,  where  the  real  calculations  take  place.  When  the  server  is  done  it  sends  the 
results  back  to  the  Java  application,  which  presents  the  results  to  the  student. 

The  downside  of  using  Java  is  that  only  individuals  and  schools  with  fairly  new  computers  and  software 
(Netscape  3 or  better,  etc.)  and  an  internet  connection  can  use  the  software.  Another  disadvantage  of  using 
Java  is  the  inability  of  Java  programs  to  save  to  disk  or  print.  This  limitation  has  been  overcome  through  the 
use  of  a notebook  that  can  be  exported  to  a web  page.  All  of  the  data  tables,  such  as  numbers  of  different 
types  of  progeny,  or  results  of  statistical  calculations,  can  be  imported  directly  into  the  notebook.  After  typing 
in  their  comments  the  student  can  export  the  notebook  to  a web  page  for  printing,  or  to  email  to  themselves  or 
an  instructor.  The  web  page  is  temporarily  stored  on  the  server.  Graphical  images  such  as  graphs  and  charts 
produced  by  some  of  the  programs  are  also  exportable  to  the  notebook,  where  they  can  then  be  printed. 

All  of  the  programs  share  some  common  user  interface  elements,  including  a title  bar  with  links  to  an 
introduction  to  the  lab,  help,  sample  assignments,  the  notebook,  etc.  While  there  is  much  diversity  in  how  the 
different  labs  operate,  most  of  them  start  in  an  input  mode  where  the  student  designs  their  experiment  by 
adjusting  different  parameters.  After  designing  the  experiment  the  student  runs  the  simulation.  The  program 
calculates  the  results  of  the  experiment,  usually  in  a minute  or  less,  and  then  presents  the  results  in  the  output 
mode.  In  this  mode  there  is  a tabbed  interface  where  the  student  chooses  which  type  of  output  they  wish  to 
view,  a table  of  the  data,  a graph,  the  input  values,  etc.  After  analyzing  their  results  they  can  import  them  into 
the  notebook  and  then  go  back  to  the  input  mode  to  design  another  experiment.  This  ability  to  quickly  go 
back  and  forth  between  the  design  of  an  experiment  and  the  results  is  one  of  the  powerful  advantages  of  a 
simulation  approach  to  teaching  science. 

Seven  of  the  programs  are  currently  available  for  beta  testing  and  all  ten  should  be  finished  by  the  summer  of 
1999.  Descriptions  of  each  of  the  labs  can  be  found  below.  Current  plans  call  for  a $19.95  fee  for  access  to  all 
ten  of  the  simulations  and  a lab  manual  with  printed  instructions  and  sample  assignments.  The  fee  is 
necessary  to  support  the  servers  and  for  maintenance  of  the  various  programs  as  operating  systems  and 
computers  change.  . The  option  of  selling  access  to  individual  labs  is  also  being  considered.  Below  is  a fairly 
complete  description  of  EvolvelT  along  with  brief  descriptions  of  the  other  simulations  However,  the  best 
way  to  learn  about  these  simulations  is  to  go  to  the  CDL  site,  http://www.cdl.edu/html/biology.html,  and  try 
out  some  of  the  five  simulations  currently  available  there. 


EvolutionLab 

Although  evolution  is  the  unifying  theme  of  the  biological  sciences,  it  is  perhaps  one  of  the  most 
misunderstood  and  difficult  concepts  to  convey  in  a laboratory  setting.  The  study  of  evolution  is  especially 
suited  to  computer  simulations  because  evolution  normally  occurs  over  very  long  time  intervals,  large  data 
sets  are  usually  needed  to  understand  it,  and  there  are  usually  a number  of  important  parameters  that  are 
difficult  to  control  in  real  experiments.  EvolvelT  rhttp://www. cdl.edu/EvolvelTT  is  a web  based,  interactive 
computer  simulation  designed  to  teach  the  basic  concepts  of  natural  selection  and  to  convey  the  importance 
of  time  in  the  evolutionary  process. 

Students  using  EvolvelT  observe  evolutionary  changes  in  bird  beak  morphology  in  hypothetical  populations 
of  birds  isolated  on  two  islands.  In  the  simulation  students  can  set  the  annual  rainfall  on  island(s)  containing 
finch  populations,  and  then  observe  the  effect  of  this  environment  on  the  evolution  of  the  finch’s  beaks. 
Students  may  also  change  several  other  properties  of  the  bird  populations,  such  as  initial  mean  beak  size, 
beak  size  variability,  beak  size  heritability  and  mean  clutch  size,  to  determine  their  effect  on  beak  evolution. 

To  help  students  clearly  see  the  effects  of  changes  in  the  different  variables,  the  simulation  uses  two  islands 
with  independent  and  isolated  populations.  A student  can  either  directly  compare  two  different  sets  of 
conditions,  or  do  a duplicate  run  where  both  populations  have  identical  starting  conditions.  As  many  students 
will  also  be  unfamiliar  with  some  of  the  variables,  the  program  gives  extensive  feedback  on  what  is  being 
changed  with  each  alteration  of  one  of  the  initial  variables.  For  instance,  when  the  slider  for  mean  beak  size  is 
moved  to  the  right  a picture  of  a finch  head  shows  the  finch  beak  growing  larger,  or  when  variability  is 
increased  a graph  of  the  current  distribution  of  beak  sizes  spreads  out. 


The  program  creates  several  hundred  different  virtual  “birds”  using  the  initial  parameters  entered  by  the 
student.  These  birds  then  go  through  a round  of  natural  selection  where  each  bird’s  probability  of  survival  is  a 
function  of  the  seed  distribution  (determined  by  the  setting  for  rainfall)  and  that  bird’s  beak  size.  After  the 
selection  step  the  surviving  birds  are  randomly  mated  to  one  another.  They  then  produce  offspring  based  on 
the  values  for  heritability  and  variability  entered  by  the  student.  This  new  population  of  birds  becomes  the 
starting  population  for  the  next  year.  This  repeats  for  each  year  of  the  simulation.  Because  of  the  random 
selection  and  mating  each  simulation  run  is  unique  and  will  produce  a different  result  than  any  other  run, 
even  one  that  starts  with  exactly  the  same  starting  parameters.  The  program  produces  several  different 
outputs:  a scrolling  table  with  the  mean  beak  size,  the  variability  and  the  population  size  for  every  year  of  the 
simulation,  for  both  populations;  a series  of  histograms  showing  what  proportion  of  birds  of  different  beak 
sizes  survived,  for  each  year  of  the  simulation;  a plot  of  the  mean  beak  size  versus  time  for  both  populations; 
and  a plot  of  the  population  size  versus  time  for  both  populations.  The  final  output  is  a table  with  the  initial 
values  for  the  simulation. 

The  student  can  rerun  the  simulation  with  the  same  initial  values,  revise  the  experiment,  or  start  over  with  the 
default  values  and  design  a new  experiment.  Students  can  study  the  effects  of  different  amounts  of  rainfall  on 
the  evolution  of  beak  size  to  get  a feel  for  how  natural  selection  works.  They  should  be  able  to  determine  that 
large  beaks  are  favored  in  low  rainfall,  the  optimum  beak  size  for  different  amounts  of  rainfall,  the  effect  of 
varying  the  severity  of  the  selection,  and  the  importance  of  the  environment  in  determining  the  direction  of 
selection.  Similar  experiments  can  be  done  in  which  only  population  variability  or  beak  size  heritability  is 
manipulated  to  study  how  these  parameters  affect  beak  evolution.  Changes  in  island  size  affect  the  carrying 
capacity  of  the  island  and  allow  the  student  to  investigate  the  stochastic  effects  that  can  result  from  small 
population  size.  Variations  in  clutch  size  permit  the  student  to  investigate  the  consequences  of  different 
fecundities  on  the  capacity  of  different  species  to  evolve.  Some  aspects  of  population  dynamics  can  also  be 
investigated  using  this  parameter  (rate  of  exponential  growth,  boom  bust  population  cycles,  etc.) 

While  the  simulation  is  based  on  Darwin’s  finches,  changes  in  the  species  variables  such  as  mean  beak  size, 
variability,  heritability  and  clutch  size  create  virtual  species  that  can  have  properties  similar  to  many  other 
wild  species.  Students  can  investigate  the  parameters  that  are  more  likely  to  lead  to  the  extinction  of 
endangered  species,  see  why  some  species  might  evolve  faster  than  others,  and  examine  many  other  facets  of 
evolution.  The  program  generates  large  data  sets,  one  run  can  produce  600  data  points,  so  students  can  learn 
how  to  analyze  and  interpret  large  amounts  of  data,  unlike  the  situation  in  a typical  lab.  The  great  flexibility 
of  the  program  should  allow  individual  instructors  to  tailor  student  assignments  to  their  particular  preferences 
and  provide  students  with  a real  opportunity  to  design  their  own  experiments.  Actively  engaging  students  in 
exploring  and  studying  evolution  through  this  simulation  will  provide  another  avenue  for  students  to  learn 
about  evolution  in  addition  to  the  traditional  text  and  lecture  explanations. 


Virtual  Fly  Lab 

Two  genetics  labs  are  currently  planned.  One  is  a simulation  of  classical  fruit  fly  genetics  while  the  other  one 
lets  the  student  study  human  genetics  by  analyzing  pedigrees.  The  Virtual  Fly  Lab 
(http://www.cdl.edu/Flylab')  simulation  is  an  update  to  the  Virtual  Fly  Lab  originally  created  by  Bob 
Deshamais.  In  the  Virtual  Fly  Lab  students  design  their  own  fruit  flies  by  choosing  from  many  different 
possible  phenotypes  for  characteristics  such  as  eye  color,  wing  shape,  body  color,  etc.  They  then  mate  their 
flies  and  analyze  the  progeny  to  determine  the  rules  of  inheritance  for  different  traits.  Each  experiment  is 
unique  and  students  can  have  up  to  1 0,000  progeny  produced  from  one  mating.  Offspring  can  also  be  mated 
so  a wide  range  of  different  experiments  are  possible.  There  are  29  different  traits  that  can  be  studied  in 
isolation  or  in  various  combinations  so  the  number  of  possible  experiments  is  in  the  millions.  The  traits  are  all 
represented  graphically  so  the  student  can  observe  the  phenotypes  directly.  For  instance,  if  the  student  selects 
the  white  eye  mutation  for  the  female  parent  their  picture  of  the  female  parent  will  change  to  have  white  eyes. 
After  the  mating  they  will  get  a picture  of  the  different  progeny,  along  with  numbers  beside  each  picture  to 
indicate  the  number  of  progeny  of  that  type  (number  of  females  with  white  eyes,  females  with  red  eyes,  etc.) 
The  program  includes  a Chi  Square  calculator  for  doing  statistical  tests  of  the  students  hypotheses,  and  a 
notebook  for  recording  results,  observations,  hypotheses  and  conclusions.  Students  can  import  the  numerical 
results  from  their  crosses  and  statistical  tests  directly  into  the  notebook. 
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Using  this  program  students  can  discover  or  study  most  of  the  important  principles  of  Mendelian  genetics, 
including  dominant  and  recessive  alleles,  sex-linkage,  lethal  alleles,  independent  assortment,  epistasis, 
linkage,  gene  order,  linkage  groups,  and  linkage  maps.  More  importantly,  students  can  discover  these 
principles  by  doing  the  same  sort  of  experiments  as  the  original  researchers,  only  much  faster.  The  program  is 
appropriate  for  a wide  range  of  biology  courses  as  the  assignment  determines  the  level  of  difficulty.  Students 
can  do  statistical  tests,  but  this  is  not  required.  Students  can  do  complicated  crosses  with  multiple  traits,  or 
simple  crosses  with  only  one  trait  at  a time.  If  a student  is  confused  by  a complicated  cross,  they  can  always 
do  some  additional  simpler  crosses  to  try  to  figure  out  what  is  going  on.  They  can  also  do  additional  crosses 
with  the  progeny  from  their  crosses,  and  their  progeny,  etc.  This  ability  to  devise  their  own  experiments  and 
try  many  different  permutations  is  a major  strength  of  the  FlyLab. 


PedigreeLab 

PedigreeLab  (http://caldera.calstatela.edu/PedigreeLab)  generates  a hundred  pedigrees  for  several  different 
genetic  diseases.  The  student  can  examine  the  pedigrees  to  determine  the  inheritance  pattern  of  the  particular 
disease.  In  addition,  the  student  can  examine  various  molecular  markers  and  determine  whether  they  are 
linked  to  the  genetic  disease.  Students  can  use  the  molecular  markers  to  map  the  disease  gene  to  a particular 
chromosome  or  to  a region  of  the  chromosome.  This  is  a key  process  in  the  current  search  for  human  genetic 
disease  genes  and  is  normally  very  difficult  to  explain  to  students.  The  program  keeps  track  of  the  results 
from  each  pedigree  they  use  and  can  determine  the  statistical  significance  of  their  results.  Having  them 
actually  go  through  the  process  of  mapping  a human  gene  should  significantly  improve  their  learning  of  these 
difficult  concepts. 


TranslationLab 

So  far,  the  Biolabs  project  has  produced  two  molecular  biology  simulations.  The  first.  Translate  IT, 
http://www.cdl.edu/TranslateIT.  simulates  some  of  the  original  experiments  used  to  crack  the  genetic  code, 
one  of  the  key  discoveries  in  molecular  biology.  These  experiments  rely  on  radioactive  materials  and  difficult 
to  produce  RNA  templates  so  they  can’t  be  done  in  the  normal  biology  lab.  Students  design  and  create  simple 
RNA  molecules  in  the  simulation  that  they  then  translate  in  a virtual  in  vitro  translation  mix.  The  program 
shows  a simple  animation  of  the  techniques  that  would  be  used  to  analyze  the  products  of  the  translation  and 
then  gives  them  the  amino  acid  sequence  of  any  proteins  produced  in  their  experiment.  The  student  must 
logically  analyze  the  results  of  multiple  experiments  to  deduce  the  properties  of  the  genetic  code,  just  as  the 
original  researchers  did,  only  with  the  advantage  of  being  able  to  do  experiments  in  minutes  that  normally 
take  months  to  carry  out.  Various  properties  of  the  code  that  can  be  determined  using  this  simulation  are  the 
triplet  nature  of  the  code,  that  the  code  is  non-overlapping,  codon  assignments  for  particular  amino  acids,  and 
the  existence  and  identity  of  stop  codons. 


HemoglobinLab 

In  the  second  molecular  biology  simulation,  the  HemoglobinLab,  http://www.cdl.edu/HemoglobinLab. 
students  investigate  various  aspects  of  the  molecular  biology  of  hemoglobin,  using  case  studies.  The  goal  is 
for  the  student  to  learn  how  changes  in  the  nucleotide  sequence  of  a gene  may  effect  the  protein  sequence, 
which  may  effect  the  structure  of  the  protein,  which  may  effect  the  function  of  the  protein,  which  may  effect 
the  properties  of  the  cell,  which  may  in  turn  effect  the  physiology  of  the  individual.  Students  choose  a case  by 
selecting  a patient  from  a pull  down  menu  with  a list  of  over  a dozen  patients.  For  each  case  the  students  can 
examine  the  doctors  notes  about  the  symptoms  and  medical  history  of  the  patient,  examine  the  color  of  a vial 
of  the  patients  blood,  examine  the  blood  under  a microscope  to  see  if  there  are  changes  in  the  red  blood  cells, 
run  a sample  of  the  blood  on  an  electrophoresis  gel  to  determine  if  there  are  physical  changes  in  the  globin 
protein,  and,  finally,  the  student  can  determine  the  amino  acid  sequence  of  the  patients  globin  protein.  Having 
determined  the  sequence  of  the  protein  the  student  can  go  to  the  DNA  sequence  editor  and  try  to  alter  the 


DNA  sequence  of  the  normal  gene  to  see  what  type  of  DNA  mutation  would  cause  the  changes  found  in  the 
patient  they  are  examining.  The  patients  have  a variety  of  mutations  in  the  globin  gene  ranging  from  simple 
point  mutations  that  change  one  amino  acid,  such  as  in  sickle  cell  anemia,  to  deletions  and  insertions  causing 
frameshifts,  such  as  some  of  the  thallasemias.  The  mutations  cause  many  different  patient  phenotypes,  such 
as  anemia,  brown  blood,  polycythemia  (too  many  red  blood  cells),  and  purple  skin  color. 


DemographyLab 

The  Demography  Lab,  http://caldera.calstatela.edu/DemoLab.  models  human  population  growth  in  several 
different  countries  around  the  world.  Students  can  use  this  lab  to  investigate  how  differences  in  population 
size,  age -structure,  and  age-specific  fertility  and  mortality  rates  affect  human  population  growth.  Default 
values  for  seven  countries  have  been  incorporated  into  the  program  to  allow  comparisons  between  nations 
with  very  different  demographics,  such  as  Japan  and  Nigeria.  In  addition,  students  can  change  any  of  the 
parameters  to  create  their  own  experiments.  The  proportion  of  males  and  females  in  each  five  year  age  group, 
the  total  population  size,  the  mortality  rate  for  males  and  females  in  each  five  year  age  group  and  the  birth 
rate  per  female  in  each  five  year  age  group  can  all  be  set  by  the  student  using  a simple  graphical  interface. 
After  running  the  simulation  for  100,  200  or  300  years,  students  get  summary  statistics  for  the  population  at 
the  end  of  the  time  interval,  such  as  life  expectancy,  birth  rate,  population  growth  rate,  etc.  They  can  view  a 
line  graph  of  population  numbers  over  the  course  of  their  experiment,  see  a graphical  representation  of  the 
population  structure  for  every  five  years  of  the  experiment,  or  examine  the  number  of  males  and  females  in 
each  age  group  for  each  five  year  period.  Using  the  program  a variety  of  demographic  phenomena  can  be 
demonstrated,  such  as  exponential  growth  and  decline,  stable  age  structure,  zero  population  growth, 
demographic  momentum,  dependency  ratios,  sex  ratios  and  marriage  squeezes. 


MitochondriaLab 

Three  biochemistry  and  cell  biology  labs  currently  in  development  are  MitochondriaLab,  EnzymeLab,  and 
LeafLab.  MitochondriaLab,  http://www.cdl.edu/MitochondriaLab.  simulates  electron  transport,  proton 
gradients  and  oxidative  phosphorylation  in  mitochondria.  Students  recreate  the  classic  experiments  that 
established  the  chemiosmotic  theory  as  the  mechanism  for  energy  production  in  the  cell.  They  add  various 
substrates  and  inhibitors  to  their  virtual  mitochondrial  extracts  and  then  measure  the  consumption  of  oxygen 
over  time.  From  their  results  they  can  work  out  some  of  the  steps  in  the  pathway  and  the  mechanism  by  which 
the  chemical  energy  is  converted  into  ATP  molecules. 


EnzymeLab 

In  EnzymeLab  students  will  study  the  enzyme  kinetics  by  varying  quantities  of  substrates,  cofactors  and 
inhibitors  and  then  measuring  reaction  rates.  They  will  also  be  able  to  vary  temperature  and  pH  to  determine 
their  effects  on  a typical  enzyme  catalyzed  reaction. 


The  LeafLab  will  simulate  the  photosynthetic  reactions  in  leaves.  Students  will  vary  wavelength  and  intensity 
of  light,  CO^  and  oxygen  concentrations,  temperature,  and  type  of  leaf  and  then  measure  the  consumption  of 
CO2  in  their  simulated  leaves. 


LeafLab 


CardioLab 


The  tenth  lab  currently  under  development  is  CardioLab.  This  lab  will  simulate  the  cardiovascular  function. 
Students  will  be  able  to  inject  various  hormones,  remove  or  add  blood  or  plasma,  alter  diet  and  exercise,  and 
stress  their  virtual  subjects  and  then  can  track  various  physiological  responses  such  as  heart  rate,  pressure,  and 
blood  flow  over  time.  Students  will  also  be  able  to  do  experiments  on  virtual  patients  with  various  health 
problems  such  as  obesity,  heart  disease,  atherosclerosis,  etc.  Student  experiments  should  lead  to  a greater 
understanding  of  hormone  action,  positive  and  negative  feedback  loops,  homeostasis,  and  physiological 
pathways.  Understanding  this  physiological  cycle  is  obviously  of  great  importance  to  the  students  and  will 
also  teach  them  many  important  physiological  concepts. 


Assessment 

The  precursor  to  all  of  these  labs  is  the  original  virtual  Fly  Lab.  This  simulation  of  fruit  fly  genetics  is  now 
used  in  biology  classes  all  over  the  world,  and  has  created  so  much  demand  on  the  server  hosting  the  program 
that  there  are  now  five  different  mirror  servers.  The  Virtual  Fly  Lab,  EvolvelT  and  TranslatelT  have  been 
field  tested  in  an  upper  division  genetics  course  with  encouraging  results.  98%  of  the  students  in  this  course 
considered  their  Virtual  Fly  assignments  useful  in  learning  genetics,  83%  found  Evolvelt  to  be  useful  and 
93%  found  Translated  useful.  Some  sample  comments  from  the  students  are: 

"Excellent  3 part  demonstration  of  Mendelian  genetics  - each  lesson  built  on  the  previous  one  - excellent 
practice  for  the  tests.  Also  includes  repetition  of  important  concepts  (X  linkage,  etc.)" 

"Liked  best  - actually  enjoyed!  (what  a concept)  Virtual  Fly  I,  II,  and  III.  These  were  great  fun  to  puzzle  out  - 
someone's  going  to  hate  me  for  saying  this" 

"Translated  was  enjoyable  because  it  requires  the  student  to  investigate  and  solve  the  problem." 

"Virtual  Fly  and  Translated  were  the  assignments  I got  the  most  out  of  I liked  the  way  it  made  you 
systematically  think  to  solve  the  problems." 

"I  liked  the  Virtual  fly  and  Evolved  activities  because  they  allowed  you  to  do  some  investigation  on  your  own 
and  they  made  you  think  about  what  was  really  happening,  which  made  you  understand  the  material  better." 

There  were  only  a few  negative  comments,  usually  having  to  do  with  the  difficulty  of  getting  on-line  and 
using  the  programs.  Students  who  are  uncomfortable  with  computers  are  at  a disadvantage  when  using  these 
simulations  and  special  care  must  be  taken  to  make  sure  they  get  the  most  out  of  the  simulations.  The  only 
other  negative  comment  was,  "I  liked  Translated  the  least  because  it  made  my  head  hurt."  While  this  is 
unfortunate,  if  the  BioLabs  project  can  produce  more  simulations  that  cause  some  students  heads  to  hurt,  then 
the  project  will  be  producing  simulations  that  change,  for  the  better,  the  way  biology  is  taught. 
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Facilitating  Virtual  Learning  Teams  in  Online  Learning  Environments 


The  Challenge  of  Virtual  Learning  Teams 

Exceedingly,  today’s  educators  are  expected  to  adapt  teaching  and  facilitation  techniques  to  new  and  emerging 
delivery  systems.  The  number  of  educational  courses  and  training  programs  migrating  to  the  World-Wide-Web, 
the  Internet,  and  corporate  intranets  is  astounding  (see  Bassi,  Benson,  and  Cheney  1 996).  For  most  educators, 
the  migration  to  computer-mediated  distance  education  does  not  come  easily.  Although  at  first  glance,  it  seems 
that  traditional  teaching  methods  such  as  presentation,  discussion,  and  team-based  learning  can  be  easily 
adapted  to  on-line  delivery  systems,  in  reality  research  is  showing  that  teaching  and  learning  in  on-line 
environments  is  very  different  from  face-to-face  instruction  (Luetkehans  1998;  Sherry  1996).  Of  particular 
interest  to  educators  is  the  effective  facilitation  of  collaborative  team  learning  in  on-line  environments. 

From  a student  perspective,  participating  in  a virtual  learning  team  (VLT)  is  a new,  and  perhaps  frightening, 
experience.  Mary  Lou  Crouch  and  Virginia  Montecino  (1997)  note  a phenomenon  experienced  by  on-line 
learners  called  “cyberstress.”  The  asynchronous  nature  of  many  on-line  courses  and  communication  tools,  and 
the  perceived  distance  between  learners  and  other  team  members  contributes  to  fears  of  contributions  and 
assignments  left  unnoticed  and  “lost  in  cyberspace.”  Students  have  not  developed  sufficient  experiences  for 
dealing  with  delayed  communications,  the  generative  nature  of  on-line  learning,  nor  the  ability  to  express 
themselves  effectively  to  team  members  through  written  communication. 

The  purpose  of  this  paper  is  to  present  tips  for  educators  who  would  like  to  successfully  integrate  and  facilitate 
virtual  learning  teams  within  their  on-line  courses.  The  techniques  are  grounded  in  current  research  and  the 
theoretical  foundations  of  systems  theory  and  group  dynamics.  Tips  emphasize  facilitation  of  virtual  learning 
teams  assembled  for  the  purpose  of  formal  education  who  are  supported  by  computer-mediated  communication 
tools. 


Critical  Elements  of  Virtual  Team  Learning 

Task  or  Problem 

A team  is  defined  as  “a  number  of  persons  associated  in  some  joint  action”  (Webster’s  Unabridged  Dictionary 
1992).  However,  all  virtual  teams  are  not  created  equal.  The  VLT  process  is  affected  by  the  type  of  task  or 
problem  it  is  given  (Straus  and  McGrath  1994).  As  with  other  instructional  strategies,  virtual  learning  team 
activities  can  be  classified  by  learning  domain  (Bannan  and  Milheim  1997).  Specifically,  team  activities  can  be 
classified  as  those  designed  for  a.)  motivation  or  attitude  development,  b.)  problem-solving,  c.)  skill  building, 
and  d.)  knowledge  construction.  The  instructional  goal  and  the  type  of  team  activity  has  a direct  impact  on  how 
the  team  members  explore  and  define  team  objectives,  plan  a course  of  action,  and  their  perceptions  of  success. 
A clearly  communicated  instructional  goal  and  desired  performance  outcomes  of  the  team  are  essential  no 
matter  the  delivery  system. 


Lara  Luetkehans,  Ph.D. 
Northern  Illinois  University 
DeKalb,  IL,  USA 
luetke@niu.edu 


Margaret  L.  Bailey,  Ph.D. 
Northern  Illinois  University 
DeKalb,  IL,  USA 
pbailey@niu.edu 


Team  Dynamics  and  Interaction 

As  with  face-to-face  teams,  VLTs  are  governed,  in  part,  by  the  dynamics  of  group  communication.  Group 
dynamics  is  concerned  with  both  the  productive  (task)  communication  of  members  as  well  as  the  development 
of  roles  and  relationships  among  team  members  (maintenance).  All  teams,  whether  on-line  or  face-to-face,  will 
develop  and  enforce  group  norms  (e.g.,  acceptable  and  unacceptable  communicative  behaviors)  and  methods  for 
leading,  stimulating,  rewarding  and  punishing  team  member  contributions.  Often,  on-line  educators  overlook 
these  elements  of  VLT  interaction  because  they  are  focused  solely  on  content  (task)  learning  and  interaction. 
Taking  a more  systems  view  of  team  interactions  may  help  avoid  so-called  “internet  pitfalls”  (Boettcher  1 997) 
of  on-line  communication. 


Team  Member  Roles 

The  greatest  value  of  team  learning  may  also  be  its  greatest  challenge.  That  is,  a group  is  made  up  people  with  a 
diversity  of  talents,  strengths  and  experiences.  This  brings  with  it  the  foundation  for  stimulating  discussion, 
creativity,  and  effective  problem  solving.  However,  it  also  means  that  each  member  of  the  team  comes  into  it 
with  established  habits,  learning  styles,  and  preferred  team  roles.  Most  of  the  current  literature  agrees  that 
effective  teams  are  able  to  represent  a balance  between  task  roles  (goal  accomplishment)  and  maintenance  roles 
(process  satisfaction  and  efficiency).  Task  roles  and  maintenance  roles  take  on  new  character  in  on-line 
environments.  Task  roles  dominate  and  are  performed  both  on-line  and  off-line.  Maintenance  roles,  although 
critical  to  team  connectivity,  may  not  be  performed  until  conflict  arises  or  inefficiency  is  felt.  The  maintenance 
role  of  leading  consensus  is  more  difficult  to  achieve  on-line  (Harasim  1993). 


Mediated  Communication 

Two  factors  differentiate  a VLT  from  a face-to-face  team:  1 .)  team  members  who  are  “out  of  sight”  and  unless  a 
face-to-face  introduction  has  occurred,  who  are  based  on  an  impression  established  by  text  and  description,  and 
2.)  a reliance  on  connections  between  team  members  made  through  electronic  or  computer-mediated 
technology.  Both  facilitators  and  team  members  must  rely  on  virtual  (as  opposed  to  tangible,  touchable) 
connections  in  order  to  achieve  goals.  The  result  may  be  a perceived  loss  of  control  and  security  for  both  the 
facilitator  and  the  team  member.  Members  fear  a loss  of  productivity  if  that  connection  is  not  reliable  or 
effective. 


Facilitation 

To  facilitate  is  “to  free  from  difficulties  or  obstacles;  to  make  easier;  to  aid;  to  assisf*  (Bailey  1996).  VLTs  are 
generally  facilitated  both  from  the  outside  (by  the  educator)  and  within  (by  team  members).  The  role  of  the 
outside  facilitator  (the  educator)  in  on-line  environments  is  changed  from  that  of  face-to-face  instruction.  The 
role  of  the  outside  facilitator,  because  of  the  nature  of  VLTs,  entails  1 .)  ensuring  that  no  barriers  (technical, 
informational  or  motivational)  exist  to  team  learning,  2.)  ensuring  essential  elements  for  learning  are  accessible 
(such  as  interaction,  task  information,  and  feedback),  and  3.)  aiding  team  members  in  times  of  conflict  or 
confusion. 


Ten  Techniques  for  Virtual  Learning  Team  Facilitators 
Tips  for  Facilitating  Motivation  and  Efftcacy 
Tip  U1  - Help  team  members  manage  ^^cyberstress** 

Help  learners  overcome  initial  stress  by  sending  a detailed  advance  e-mail  at  the  onset  of  the  team  activity. 
Include  a welcome,  a description  of  the  team  goals  and  desired  outcomes,  and  tips  on  being  a successful  team 
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member  in  an  on-line  course.  Ask  members  to  contribute  personal  information  about  themselves  as  a first 
communication  to  the  VLT.  Ask  them  to  include  their  background,  familiarity  with  the  task/problem,  and 
talents/contributions  they  will  be  bringing  to  the  team.  As  the  course  continues,  manage  the  stress  resulting  from 
delayed  communications  by  sending  “receipt”  messages.  A receipt  message  is  a short  feedback  message 
indicating  that,  although  you  cannot  immediately  respond  to  the  learner’s  message  or  review  the  assignment, 
you  received  it.  Encourage  team  members  to  do  the  same.  Help  teams  establish  ground  rules  for  frequency  of 
checking  computer-mediated  communications. 


Tip  #2  - Plan  frequent  e-mail  prompts  to  help  team  members  overcome  procrastination 

It  is  important  to  help  team  members  sustain  participation.  “The  major  reason  for  disappearance  has  to  do  with  a 
student’s  feelings  of  not  being  connected”  (Crouch  and  Montecino  1997).  E-mail  reminders  are  useful  in 
helping  to  keep  the  team  activity  in  the  forefront  of  learner’s  thoughts.  E-mails  should  not  be  nagging,  but  serve 
as  friendly  reminders  to  log-on  to  the  site,  ask  if  assistance  is  required  or  for  reports  on  team  progress. 


Tips  for  Facilitating  Problem-Solving 

VLT  activities  for  problem  solving  typically  require  methods  that  encourage  both  creativity  and  decision- 
making. Activities  which  integrate  knowledge  and  skill  sets  with  real  problems  and  contexts  may  include  case 
scenarios,  simulations,  team  research  and  reporting,  negotiation,  and  decision  making.  The  facilitator  of 
problem  solving  must  be  skilled  at  supporting  the  phases  of  problem  solving  and  in  helping  teams  achieve 
consensus. 


Tip  U3  - Provide  a variety  of  tools  to  support  the  different  phases  of problem  solving 

Provide  each  team  an  exclusive  area  on  the  course  website  for  posting  team  resources  and  strawman  solutions. 
Because  problem  solving  requires  both  exploration  and  consensus,  VLTs  should  have  the  ability  to  access  and 
post  hypertext  resources  for  information  searching  and  sharing,  as  well  as  tools  to  develop  and  debate  solutions. 
Provide  a combination  of  tools  for  different  stages  of  problem  solving.  If  the  problem  or  case  study  will  take 
several  sessions  to  solve,  teams  will  benefit  from  access  to  an  asynchronous  tool  that  can  be  used  to  keep  the 
history  of  the  team’s  discussions  and  negotiations.  If  the  problem  can  be  resolved  in  a single  sitting,  a 
synchronous  tool  for  discussing  and  defending  individual  perspectives  or  even  voting  can  be  useful. 
Asynchronous  problem  solving  is  best  supplemented  with  a synchronous  tool  during  stages  of  negotiation  and 
decision  making. 


Tip  §4  -Assist  team  members  when  they  struggle  with  achieving  consensus 

Teams  often  appreciate  the  intervention  of  a neutral  facilitator  when  it  comes  time  to  make  critical  decisions  or 
select  among  alternatives.  Explore  methods  for  polling  team  members,  and  for  conducting  on-line  voting.  The 
summary  is  an  effective  tool  for  leading  to  consensus.  Often  team  members  are  too  busy  looking  forward  to  see 
where  they  have  been.  Review  team  progress,  and  summarize  it  as  you  see  it.  For  example,  “I  see  you  are 
developing  two  pretty  distinct  alternatives.  The  first  appears  to  focus  on  the  navigation  needs  of  the  learner.  The 
second  focuses  on  the  simplicity  of  design.  Which  do  you  feel  are  priority  given  the  goal  of  immediate  access?” 


Tips  for  Facilitating  Skill  Building 

Activities  that  allow  for  the  development  of  skills  may  include  small  group  projects  and  expert  modeling.  The 
facilitator  of  skill  development  must  be  expert  at  designing  authentic  projects,  offering  timely  and  meaningful 
feedback,  modeling,  and  leading  effective  reflection  and  debriefs  of  skill  activities. 
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Tip  #5  - Assemble  teams  strategically  based  on  task  and  talent 

In  online  environments,  larger  groups  are  less  productive  and  have  more  difficulty  arriving  at  consensus,  so 
consider  3-4  members  per  team  as  a target  (Dennis  & Gallupe  1993).  If  possible,  teams  can  be  balanced  on 
talent  and  experience  (e.g.,  with  HTML)  so  that  an  "expert"  in  the  team  is  able  to  provide  modeling  of  the  skills 
and  scaffolding  for  other  team  members.  Assess  the  skills  targeted  for  development.  Design  tasks  around  the 
targeted  skills,  and  communicate  them  through  clearly  stated  objectives.  Focus  tasks  on  the  development  of  a 
single  competency  area,  or  a small  number  of  skills. 


Tip  #6  - Provide  timely  and  meaningful  feedback 

Skill  building  depends  on  frequent  practice  and  feedback.  The  facilitator  can  use  semi-private  and  private 
communications  for  feedback.  Semi-private  communications  can  be  established  by  providing  “exclusive” 
threaded  discussions,  chat  rooms  or  Listservs  accessible  only  by  team  members.  Individual  feedback  should  be 
conducted  via  e-mail.  The  facilitator  should  plan  and  use  process  checks  as  part  of  project  activities.  A process 
check  is  a planned  “check  poinf  ’ for  communication,  reporting,  and  questioning  on  a project’s  progress. 
Synchronous  tools  are  effective  for  conducting  process  checks.  When  the  activity  is  complete,  conduct  an 
asynchronous  debrief  discussion.  The  facilitator  should  use  questioning  techniques  that  stimulate  reflection  and 
processing.  Modeling  difficult  tasks  and  behaviors  is  another  form  of  feedback,  allowing  students  to  compare 
their  outcomes  to  a benchmark  or  standard.  Use  hyperlinks  to  show  learners  examples  of  completed  projects, 
forms,  or  reports  on  the  course  website  or  in  threaded  discussion. 


Tips  for  Facilitating  Knowledge  Construction 

Activities  that  develop  new  concepts,  contexts,  and  meaning  may  include  reading,  information  searching  and 
sharing,  discussion,  inquiry,  and  reflection.  The  facilitator  of  knowledge  construction  must  be  skilled  at 
scaffolding  discussion,  and  encouraging  exploration  and  elaboration. 


Tip  # 7 - Scaffold  topical  discussions  using  a threaded  discussion  (asynchronous)  tool 

An  effective  scaffolded  discussion  involves  a.)  providing  an  initial  structure  and  b.)  facilitating  concept 
construction  through  questioning.  The  structure  can  be  minimal.  If  the  discussion  topic  is  new  to  learners,  post  a 
topic  name  along  with  1-2  open-ended  questions  to  initiate  thinking.  This  allows  students  to  construct  the 
concepts  and  detail.  Facilitators  should  avoid  dominating  discussions  by  using  relay  questioning  techniques  and 
only  participating  when  necessary.  To  encourage  continued  contributions,  reward  participants'  thoughtful 
responses  with  short  affirmations.  Finally,  when  discussion  objectives  have  been  met,  the  facilitator  can  quickly 
“point  to”  the  learning  with  a brief  summary.  Summaries  provide  both  rewards  and  reinforcement. 


Tip  U8  - Encourage  elaboration  through  questioning  and  hypertext  linking 

As  in  face-to-face  facilitation,  effective  questioning  techniques  are  useful  to  encourage  elaboration.  Use  open- 
ended  questions  to  stimulate  a response  that  builds  on  prior  concepts.  For  example,  “Jack,  you  indicated  ‘a 
survey  as  one  ways  to  gather  information  for  a needs  assessment.’  What  other  techniques  could  you  employ?” 
Hyperlinks  to  additional  information  or  expertise  can  also  be  used  for  stimulating  elaboration.  Many  Internet 
communication  tools  accept  HTML  links  within  the  message  body.  The  facilitator  should  model  the  use  of 
hypertext  in  her  responses.  In  addition,  e-mail  links  can  be  included  as  a way  to  access  outside  experts  willing 
to  participate  in  discussions  or  private  e-mail  exchanges. 
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Tips  for  Resolving  Conflict 


Teams  will  rely  on  the  facilitator  to  intervene  in  times  of  dysfunction.  It  is  the  facilitator’s  primary  function  to 
remove  these  obstacles  to  learning. 


Tip  U9  “ Discourage  judgment,  criticism  and  personal  attacks 

Carefully  phrase  retributions  to  comments  that  are  clearly  not  constructive.  For  example,  “While  I realize  that 
we  will  not  always  agree  with  all  point  of  view  expressed  on  this  forum,  this  is  a reminder  to  reflect  and  build 
on  IDEAS,  not  judge  the  PERSON.”  This  type  of  conflict  can  be  avoided  if  teams  adopt  ground  rules  that 
encourage  members  to  suspend  judgment  and  accept  diverse  views.  Post  the  rules  at  the  beginning  of  a team 
activity  or  on  the  course  website.  Monitor  the  interaction  and  remind  members  when  a contribution  is  outside  of 
the  accepted  rules. 


Tip  UlO-  Intervene  to  highlight  areas  of  common  ground  among  conflicting  team  members 

Team  members  will  expect  the  facilitator  to  intervene  when  conflicts  get  personal  or  unproductive.  Start  by 
helping  team  members  see  areas  within  their  conflict  that  they  agree  upon.  For  example,  “Sara  and  Tom,  you 
seem  to  be  at  a stand  still.  In  reviewing  your  discussion,  it  appears  that  you  are  both  concerned  that  the  end 
product  be  visually  appealing.  Is  that  correct?”  Encourage  the  use  of  synchronous  tools  to  resolve  heated 
conflict  in  a timely  fashion.  Phone  conferences  may  be  more  effective  than  computer-mediated  communication 
to  resolve  personal  conflict. 

Drawing  on  a theoretical  framework  emergent  from  constructivism,  systems  theory  and  group  dynamics,  this 
paper  presented  ten  tips  for  facilitators  of  virtual  learning  teams  (VLTs).  An  on-line  facilitator’s  required  skills 
range  from  selecting  and  designing  appropriate  on-line  tasks  to  assembling  teams  and  removing  obstacles  to 
team  task  and  maintenance  roles.  Of  greatest  importance  is  to  intervene  on  any  matter  that  deters  a team’s 
technical,  informational,  or  motivational  needs.  As  more  courses  migrate  to  computer-mediated  environments, 
on-line  facilitation  skills  will  become  increasingly  critical  to  educators. 
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Abstract:  In  this  paper  we  present  the  prototype  version  of  MGCoop,  a set  of  CSCW  tools 
developed  for  Management  Game,  a simulation-based  learning  environment  developed  by  the 
University  of  Carnegie  Mellon.  This  paper  presents  the  design  of  CSCW  tools  that  allow  the 
students  to  plan  and  establish  virtual  meetings  through  the  internet,  maintaining  them  aware  of  their 
capacities,  commitments,  availability  and  activities,  in  a learning  environment.  The  students  can 
work  in  a cooperative  way,  being  coordinated  using  a WYSIWIS  workflow  environment,  and 
presenting  and  discussing  their  points  of  view  in  an  asynchronous  or  asynchronous  way.  We 
included  the  results  obtained  during  the  analysis  from  the  interaction  problems  presented  by  the 
members  of  the  learning  group  during  their  cooperation  meetings.  Some  examples  of  these  tools  are 
presented,  which  were  implemented  in  JAVA,  permitting  platform  independence. 

Keywords:  CSCL  (Computer  Supported  Collaborative  Learning),  Groupware,  CSCW  (Computer 
Supported  Cooperative  Work),  JAVA,  Workflow,  Simulation  Based  Learning. 


Introduction 

Currently,  learning  environments  based  on  the  cooperation  and  the  collaboration  of  students  present  new  alternatives 
to  the  teaching-learning  process  (O’Malley  1995)  in  educational  institutions  as  well  as  in  organizations.  Learning 
environments,  based  on  technologies  of  CSCW  (Computer  Supported  Cooperative  Work)  (Khoshafian  & 
Buckiewicz  1995),  allow  the  training  and  the  collaborative  work  in  companies,  increasing  their  competitiveness. 
Since  1995  research  and  development  projects  in  educational  software  have  been  proposing  the  adjustment  of  the 
concepts  of  CSCW  within  a theoretical  learning  collaborative  framework,  appearing  the  area  of  CSCL  (Computer 
Supported  Collaborative  Learning)  (Ayala  & Yano  1998)  (Ayala  & Yano  1994). 

Together  with  researchers  of  the  University  of  Carnegie  Mellon,  and  continuing  the  same  research  guidelines  (Cadiz 
et  al.  1998),  in  the  UDLA-P  we  have  developed  a set  of  CSCW  tools  for  a simulation  based  learning  environment 
(Medelez  1 998).  These  tools  have  been  designed  based  on  the  interaction  study  of  members  of  learning  groups,  who 
need  to  cooperate  and  be  aware  of  the  activities  each  one  must  accomplish  and  the  students  that  are  working  in  a 
given  moment,  supporting  in  this  way  their  coordination.  The  activity  that  the  group  must  accomplish  together  is  the 
creation  of  a report  that  must  be  presented  periodically  to  a committee.  This  activity  is  the  one  which  the  students 
require  more  coordination  and  cooperation  in  Management  Game,  which  is  the  reference  framework  of  our  research 
project. 
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Management  Game 


Management  Game  was  developed  in  the  50’s,  with  the  participation  of  teachers  of  the  Graduate  School  of  Industrial 
Administration  (GSIA)  of  the  University  of  Carnegie  Mellon.  The  purpose  of  this  game  is  to  learn  by  applying 
knowledge  of  business  administration  in  a competitive  and  dynamical  simulation  environment.  A learning 
environment  based  on  simulation  provides  a great  number  of  advantages  in  comparison  with  the  simple  analysis  of 
cases  of  studies  or  the  readings  of  textbooks.  The  simulation  promotes  the  competition  between  several  companies, 
each  one  of  they  formed  by  a group  of  students  (Lerch  et  al.  1997).  This  game,  already  a tradition  in  the  University 
of  Carnegie  Mellon,  became  an  international  game  in  90's,  when  students  of  Sweden,  Japan  and  Mexico  were 
included  (Lerch  et  al.  1997).  The  UDLA-P  participated  in  the  game  with  a group  formed  by  students  of  the  master 
course  in  Business  Administration  and  Industrial  Engineering.  Using  this  game,  the  students  put  in  practice  their 
knowledge,  competing  with  students  of  different  parts  of  the  world,  living  an  experience  of  international  training. 

In  Management  Game,  the  students  can  play  two  or  more  roles  in  their  company  (president,  counting,  investigating 
of  market,  responsible  for  marketing  and  sales,  of  production,  of  operations,  of  finances,  of  competitive  analysis, 
strategic  planning  and  of  communications).  It  is  very  important  to  assign  the  roles  to  the  most  suitable  person,  as 
well  as  to  keep  the  students  aware  of  (1)  the  roles  that  each  student  plays,  (2)  her/his  commitments,  (3)  the  assigned 
tasks  and  (4)  their  schedule  and  deadlines.  After  some  experiences  with  the  learning  groups,  it  was  determined  that 
the  learning  environment  required  a set  of  CSCW  tools  for  the  coordination  and  the  efficiency  of  the  work  in  the 
learning  group. 


Interaction  Requeriments 

After  some  meetings  with  the  students  participating  in  Management  Game,  we  found  the  following  interaction 
problems  in  the  learning  groups,  which  were  affecting  their  performance  and  therefore  their  learning: 

a)  Meeting  Difficulties.  Some  of  the  group  members  already  had  established  commitments  in  other 
enviroenments,  like  companies  or  other  institutions.  In  occasions,  they  have  to  make  trips  outside  the  city, 
which  made  difficult  to  plan  face  to  face  meetings. 

b)  Lack  of  cohesion  and  awareness  of  the  group.  The  students  presented  problems  based  on  the  ignorance  of 
who  was  available  and  what  activity  was  performing  in  a given  moment,  when  they  required  cooperation. 

c)  Lack  of  control  and  follow-up  of  the  worlfflow.  The  students  presented  coordination  problems,  not  knowing 
exactly  to  who  they  have  to  deliver  the  results  of  their  task  This  causes  a delay  in  the  accmplishement  of  the 
report  to  be  submitted  by  the  group. 

d)  Lack  of  awareness  about  the  capacities  and  commitments.  Some  groups  presented  problems  in  the 
assignment  of  roles,  since  the  student  playing  the  role  of  president  was  not  aware  that  some  peer  had  a greater 
capacity  or  experience  in  some  specific  activity  within  the  simulation.  There  were  also  awareness  problems 
concerning  the  assigned  tasks  and  commitments  by  each  student. 

e)  Heterogeneity  and  conflict  in  the  decision  making.  The  groups  include  students  of  industrial  engineering  as 
well  as  business  administration.  As  a result,  the  interaction  between  students  presented  conflicts  due  to  different 
points  of  view. 

It  was  considered  that  without  a cooperative  working  environment  designed  for  Management  Game  these  problems 
could  not  be  solved,  specially  if  in  a near  future  it  is  planned  to  have  more  heterogeneous  groups  from  different 
institutions  and,  later  from  different  countries. 
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CSCW  Tools  for  Management  Game 


In  order  to  propose  a solution  to  the  problems  mentioned  in  the  previous  section,  we  have  designed  and  developed 
an  environment  in  JAVA  where  the  students  learn  by  collaborating  with  their  peers  through  internet  while  working 
in  Mahnagement  Game,  These  CSCW  tools  allow  the  student  to: 

1)  Be  aware  of  the  participants  who  are  working  in  a given  moment, 

2)  Be  aware  of  the  activities  of  each  member  of  the  learning  group,  via  an  electronic  agenda  supported  by  a 
calendar, 

3)  Be  aware  of  the  tasks  already  concluded  by  the  members  of  the  learning  group,  the  relationship  between 
these  in  the  context  of  the  common  work,  as  well  as  of  the  deadlines  of  the  tasks,  via  an  workflow  environment, 

4)  Clear  assignment  of  the  tasks  to  perform  by  the  members,  an  agreement  of  the  role  cissigned  to  the  members 
play,  according  to  their  capacities,  via  a users  manager  tool.  There  is  a document  manager  that  includes  the 
concurrence  managing,  including  the  access  and  modification  privileges  of  shared  files, 

5)  A synchronous  communication  tool  between  peers  (chat),  as  well  as  a asynchronous  communication  tool  (e- 
mail)  developed  ad  hoc  the  needs  of  the  environment. 


MGCoop 

In  order  to  support  awareness,  the  cooperative  work  and  learning  of  the  students  that  play  with  Management  Game, 
we  developed  MGCoop,  MGCoop  has  been  implemented  in  JAVA,  We  used  the  WIMPs  (Windows,  Icons,  Menus, 
Pointers)  interfece  concepts  (Van  Dam  1997)  and  the  classes  AWT  (Abstract  Window  Toolkit)  (Sun  Microsystems 
1998)  and  Swing  (Fowler  1998), 

We  have  organized  theMGCoop  tools  in  three  groups: 

a)  Users  Manager 

b)  MGCoop  Workflow  Environment 

c)  Communication  Tools 

Users  Manager 

The  work  assignment  in  a responsible  way  is  a very  important  aspect  that  we  consider  when  we  designed  the 
environment  MGCoop,  In  this  way  we  could  improve  the  group  performance.  The  users  manager  module  includes 
tools  that  keep  awareness,  communication  and  coordination,  so  the  students  can  work  in  a cooperative  way.  These 
tools  are  referred  to  the  following  functions: 

a)  Assignment  of  the  student's  role.  Used  by  the  coordinator  of  the  groups  in  an  institution.  S/he  assigns  the  role 
to  perform  by  each  student  in  the  virtual  company, 

b)  Consulting  and  modifying  student  information.  Any  student  of  the  group  have  on  line  access  to  the  name,  role 
and  electronic  mail  address  of  her/his  peers.  The  coordinator  can  modify  the  assigned  role  previously  to  any 
student  when  it  will  be  necessary  (Fig,  1), 

c)  Users  Directory,  Keep  aware  any  student  of  the  group  about  who  is  working  in  the  environment  in  a given 
moment,  as  well  as  the  role  that  it  is  performing  (Fig,  1), 

d)  Synchronous  Communication,  A chat  tool  was  specially  designed  for  Management  Game,  Any  student  is  able 
to  communicate  and  collaborate  with  one  or  more  of  their  peers  in  real  time  (Fig,  2) 

e)  Asynchronous  Communication,  An  email  tool  was  specially  designed  for  Management  Game,  Any  student  is 
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able  to  communicate  and  collaborate  with  one  or  more  of  their  peers  at  any  time. 


Figure  1:  General  Information  of  a user 
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Figure  2:  Synchronous  conversation  between  two  members 

MGCoop  Workflow  Environment 

Within  the  technology  of  Groupware,  automated  workflow  systems  are  environments  that  permit  the  control  of 
activities,  task  assignment  and  the  coordination  and  cooperation  (Khoshafian  & Buckiewicz  1995).  For  MGCoop  we 
have  developed  a workflow  environment  based  as  a WYSIWIS  (What  You  See  Is  What  I See)  interface.  This  tool 
allows  any  student  to  observe  in  real  time  the  changes  and  the  advance  of  the  report  constructed  by  the  group, 
represented  as  a sequence  of  related  tasks.  It  was  necessary  to  study  the  interactions  between  the  members  in  order 
to  design  the  workflow  environment. 

The  students,  as  members  of  a virtual  company,  should  present  a report  to  an  established  executive  committee, 
composed  by  real  executives  supporting  Management  Game.  The  meetings  where  the  reposrt  are  prersented  are 
given  in  three  occasions  during  the  development  of  the  game.  In  the  first  is  made  known  the  strategic  plan  that  the 
company  will  follow.  In  each  one  of  the  remaining  two  a summary  of  the  company  operations  is  presented.  Each 
report  consists  of  a set  of  related  projects.  Each  one  of  these  projects  consists  also  of  a set  of  tasks  to  be  performed 
by  the  members  of  the  learning  group. 

The  characteristics  of  the  MGCoop  workflow  environment  include: 
a)  Graphic  worhflow. 

The  projects  and  the  tasks  are  represented  in  an  workflow  graph  (Fig.  3)  in  which  projects  or  tasks  are  presented 
as  squares  related  with  others  considering  their  requeriments.  Each  task  or  project  in  the  interface  are  sensitive 
to  the  action  of  the  mouse,  presenting  to  the  user  the  relevant  information,  as  name,  responsible,  requesting, 
completion  date,  requeriments  and  current  state.  A project  indicates  the  tasks  that  compose  it,  and  each  project 
has  its  respective  workflow  graph  representing  the  tasks  implied.  Tasks  and  projects  are  presented  in  red  color 
while  they  have  not  been  ended,  and  in  blue  color  when  they  have  been  accomlished.  The  environment  allows 
the  user  to  build  the  workflow  graph  using  the  mouse  and  selecting  information  related  to  the  tasks  or  projects 
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through  menus,  buttons  and  text  areas. 

The  president  must  assign  the  deadlines  of  the  projects  that  conform  the  report,  in  this  way  the  members  of  the 
group  will  know  the  completion  dates  of  each  one  of  the  projects. 

b)  Task  assignment 

The  environment  automatically  provides  to  the  president  the  suggestion  of  the  most  indicated  person  as 
responsible  for  a work,  according  to  the  specifications  of  Management  Game. 
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Figure  3:  Interface  of  the  Workflow,  showing  a report  composed  of  4 projects. 


Communication  Tools 

MGCoop  contains  tools  of  chat,  electronic  mail  and  agenda  electronics  designed  specifically  for  Management 
Game.  The  possibility  of  synchronous  and  asynchronous  communication  and  cooperation  permits  to  solve  the 
problem  of  the  physical  meeting  difficulty.  Through  these  tools  the  members  of  the  group  will  be  able  to  take 
decisions  in  group,  to  solve  their  different  points  of  view  and  to  plan  meetings  they  will  be  physical  or  virtual.  The 
electronics  agenda  provides  necessary  information  to  plan  these  meetings,  the  members  of  the  group  will  be  aware 
of  the  activities  of  each  one  of  their  peers. 


Figure  4:  Activities  to  accomplish  by  a member  of  the  group 


It  was  necessary  to  study  the  way  in  which  the  members  of  the  learning  group  would  have  to  be  communicated,  in 
an  asynchronous  or  synchronous  way.  The  asynchronous  form  that  we  employ  was  the  electronic  mail,  and  the 
synchronous  way  was  the  "Chat".  Both  tools  were  designed  in  a special  way  for  the  users  of  Management  Game  due 
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to  they  were  not  familiarized  with  already  existing  tools.  We  consider  that  to  use  various  tools  and  interfaces  for 
their  work  can  influence  negatively  their  performance. 

The  chat  tool  of  MGCoop  includes  the  representation  of  the  roles  of  the  users,  permitting  them  to  be  aware  of  their 
commitments.  The  electronic  mail  of  MGCoop  is  also  very  easy  use  and  forms  part  of  the  environment.  The  issued 
messages  can  be  read  from  any  other  email  tool,  not  only  by  MGCoop, 

The  environment  also  provides  an  electronics  agenda  that  shows  the  students  the  calendar  of  the  other  peers  actvities 
in  a graphic  way  and  sensitive  to  the  action  of  the  mouse.  When  a day  of  the  specific  month  is  selected,  the  activities 
that  the  selected  member  has  for  that  day  appear  (Fig,  4),  At  any  time  a student  can  add  a new  appointment  or 
activity  into  her/his  personal  calendar. 


Conclusions 

MGCoop  is  a set  of  colaborative  tools  designed  for  a simulation-based  learning  environment.  These  tools  were 
developed  as  a result  of  the  research  in  the  area  of  CSCW  on  the  interaction  required  by  the  users  of  Management 
Game.  The  developed  environment  permits  the  students  to  plan  and  establish  virtual  meetings,  and  be  aware  of  the 
capacities,  commitments,  availability  and  activities  of  their  peers.  Also  it  allows  the  cooperative  and  coordinate 
work  thanks  to  a workflow  environment  and  the  presentation  and  discussion  of  their  points  of  view  in  a synchronous 
or  asynchronous  way. 

Currently  we  are  working  in  the  development  of  more  classes  in  JAVA  in  order  to  include  in  MGCoop  the 
interaction  through  videoconference.  MGCoop  it  is  currently  in  a test  period  and  it  will  be  used  in  the  beginning  of 
the  following  year  when  Management  Game  is  going  to  be  played  again. 
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Abstract:  The  paper  presents  the  design  issues  of  SALLE,  a life-long  learning 
environment  based  on  software  agents  modeled  for  the  social  construction  and  use  of 
Japanese  language  knowledge.  This  research  proposes  the  new  paradigm  of  agent 
based  life-long  learning  environments  for  Japanese  language,  based  on  the  idea  that 
the  study  of  a second  language,  specially  the  Japanese  for  western  people,  is  a life-long 
learning  process. 

With  the  development  of  SALLE  (Software  Agents  for  a Life-long  Learning 
Environment  in  Japanese)  we  try  to  determine  the  real,  relevant  and  truly  useful 
educational  material  for  the  study  of  the  Japanese  language,  also  motivating  the 
discovery,  use  and  sharing  of  knowledge  by  the  learners  in  a virtual  community  in  the 
Internet. 


Introduction 

The  concept  of  life-long  learning  has  become  one  of  the  most  important  topics  in  computer  assisted 
education.  Life-long  learning  refers  to  the  continuous  education  that  implies  the  updating  of  our 
knowledge  in  order  to  be  more  capable,  productive  and  competitive,  adapting  ourselves  to  the  dynamic 
changes  in  our  ever  evolving  world.  Lifelong  learning,  according  to  the  definition  by  the  European 
Lifelong  Learning  Initiative  (ELLI)  and  presented  in  (Longworth,  96),  is: 

"a  continuously  supportive  process  which  stimulates  and  empowers  individuals  to  acquire 
all  the  knowledge,  values,  skills  and  understanding  they  will  require  through  their 
lifetimes,  and  apply  them  with  confidence,  creativity  and  enjoyment  in  all  roles, 
circumstances  and  environments." 

In  a global  and  internationalized  world,  lifelong  learning  has  become  a necessity  for  everyone  due 
to  the  rapid  changes  in  our  knowledge  based  society  (Berge,  98a).  Lifelong  learning  environments  are 
needed  for  the  competitive  advantage  and  success  of  leading-edge  companies  and  universities.  In  this 
context,  it  is  clear  that  the  study  of  a second  language  is  a life-long  learning  process,  since  language  is 
dynamic  and  in  constant  change. 

A learner  centered  approach  (Berge,  98b)  has  become  the  actual  tendency  in  educational  software 
for  second  language  learning.  From  this  point  of  view,  the  learner  has  the  responsibility  of  her/his  own 
learning,  not  the  teacher  or  the  tutor.  This  approach,  supported  by  theories  of  Constructionism  (Resnick, 
96),  has  been  promoting  the  research  and  development  of  learning  environments,  as  an  alternative  to 
tutoring  systems.  Nowadays  there  are  two  main  problems  of  actual  interest  in  this  learner  centered 
approach  (Berge,  98b),  first  we  have  to  determine  how  to  provide  access  to  real,  relevant  and  truly  useful 
educational  material;  and  second,  we  need  to  motivate  the  discovery  and  use  of  knowledge  by  the  learner, 
and  make  her/him  share  it  with  others.  We  believe  that  the  solution  to  these  problems  implies  the  design 
and  development  of  a life-long  learning  environment  based  on  the  concept  of  social  construction  of 
knowledge  (Barret,  92).  For  a life-long  learning  environment  the  continuous  social  construction  of 
knowledge  should  be  not  a specialized  activity,  but  a way  of  being  in  a community,  which  represents  a 
culture  that  consists  of  ideas,  beliefs  and  knowledge  shared  by  its  members. 
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We  are  considering  the  World  Wide  Web  as  the  main  source  of  knowledge  and  information  which 
contains  the  real,  frequently  used,  current  and  popular  language  patterns,  words,  expressions  and  kanji 
compounds  of  the  written  Japanese  language,  situated  in  a real  context.  Since  Japanese  language  changes 
continuously,  specifically  in  new  words  adopted  from  other  languages  and  expressions,  the  knowledge 
represented  in  a life-long  learning  environment  must  be  updated  with  the  language  elements  used  by  the 
producers  of  WWW  pages  in  Japanese  and  by  the  participating  members  of  the  community  of  Japanese 
language  learners.  Our  objective  is  to  develop  a computer  mediated  learning  environment  where  the 
participants  can  learn  using  and  constructing  together  a knowledge  base  representative  of  the  written 
Japanese  language  currently  and  frequently  used. 

In  this  paper  we  introduce  SALLE  (Software  Agents  for  a Life-long  Learning  Environment  in 
Japanese).  SALLE  is  a multiagent  learning  environment  where  software  agents  cooperate  and  assist  their 
learners,  providing  a learning  environment  based  on  the  coordinated  social  construction  of  a knowledge 
base  as  a representation  of  the  current  use  of  the  Japanese  language. 


Software  Agents  and  Japanese  Language  Learning 

A software  agent  (Nwana,  96)  works  on  the  behalf  of  the  user,  reducing  her/his  work  and  information 
overload  (Maes,  94).  Here  we  refer  to  the  term  agent  as  an  entity  that  functions  continuously  and 
autonomously  in  an  environment  in  which  other  agents  exist  (Genesereth,  94).  Of  particular  interest  to 
the  life-long  learning  approach  is  the  work  of  Norrie  and  Gaines,  introducing  the  Learning  Web  as  an 
agent  based  distributed  intelligent  learning  environment  (Norrie,  95). 

There  have  been  proposals  in  the  modeling  of  agents  for  Japanese  language  collaborative  learning 
environments  (Ayala,  95).  With  GRACILE  (Ayala,  98)  we  proposed  a collaborative  learning  environment 
where  foreign  students  learn  by  constructing  a dialog  together,  applying  vocabulary,  language  patterns 
and  expressions  of  the  Japanese  language.  The  agents  in  GRACILE  are  implemented  in  PROLOG  and 
work  on  a local  area  network  of  Macintosh  computers,  since  GRACILE  was  designed  as  a second 
language  collaborative  learning  for  the  enculturation  of  the  learner  (foreign  student)  in  a geographically 
defined  community  (a  school  of  a given  university),  with  its  own  language  characteristics.  In  SALLE  the 
interaction  of  learners  is  indirect,  through  their  contributions  to  a virtual  community  knowledge  base  via 
the  Internet,  and  there  is  not  a collaborative  task  to  be  solved  synchronously  by  a small  group  of  learners, 
as  in  GRACILE.  The  virtual  community  of  practice  in  SALLE  is  indefinitely  wide,  and  the  members 
participate  by  cooperating  asynchronously  at  any  time  from  anywhere.  The  software  agents  in  SALLE 
have  been  designed  based  on  an  object  oriented  methodology,  which  permits  its  implementation  as  a set  of 
software  components  in  JAVA  (Bigus,  98),  allowing  the  client  to  be  platform  independent. 

In  SALLE  we  have  modeled  four  software  agents;  an  information  agent  that  works  on  the 
acquisition  of  knowledge  from  html  files  in  the  WWW,  a facilitator  agent  that  assist  the  learners  in  the 
social  construction  of  knowledge,  a knowledge  negotiation  agent  to  allows  the  learner  to  agree  or  disagree 
in  a viewpoint  provided  from  other  member  and  a knowledge  agent  that  applies  language  patterns  and 
vocabulary,  providing  assistance  concerning  their  reading,  meaning  and  use. 


Knowledge  Agent 

The  knowledge  agent  in  SALLE  plays  three  important  roles  in  the  learning  environment: 

a)  Management  of  the  knowledge  base.  The  knowledge  elements  (nouns,  expressions,  adjectives,  verbs 
and  the  corresponding  language  patterns  for  their  conjugation,  together  with  their  meanings  and  readings) 
are  organized,  accessed  and  registered  in  a knowledge  base  of  the  community,  implemented  through  a 
DBMS. 

b)  Recognition  of  knowledge  elements  and  language  patterns.  Recognition  of  conjugated  adjectives  and 
verbs,  as  well  as  nouns  and  expressions  provided  by  the  information  agent,  obtained  from  Japanese 
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material  in  the  Web  or  by  the  facilitator  agent,  from  contributions  of  nouns  and  expressions  by  the 
members  of  the  virtual  community, 

c)  Assistance.  Pedagogical  presentation  of  the  meaning,  pronunciation  of  nouns,  expressions,  and 
language  patterns  analysis  of  conjugated  adjectives  and  conjugated  verbs.  The  knowledge  agent  provides 
the  meaning  together  with  the  conjugation  features  of  the  pattern  involved,  the  reading  of  kanji  (furigana) 
and  the  corresponding  meaning  in  English'.  In  this  way,  it  helps  the  novice  and  advanced  user  to 
understand  the  Japanese  language  pages  s/he  is  reading  while  navigating  the  Internet,  The  knowledge 
assist  the  learner,  applying  its  vocabulary  and  the  language  patterns  and  expressions.  The  requests  of 
analysis  from  the  learners  may  be  in  hiragana,  katakana  or  kanji. 

In  our  first  prototype  version,  the  knowledge  agent  contains  around  2,000  nouns  considered  of 
frequent  use.  Their  representation  includes  hiragana  and  kanji  compounds.  Around  400  of  them  are  in 
katakana.  Each  noun  object  includes  its  meaning  in  English  and,  in  the  case  of  a kanji  compound,  its 
reading  (furigana).  A request  to  the  knowledge  agent  from  a learner,  concerning  a noun,  can  be  in 
English  or  in  Japanese,  using  hiragana,  katakana  of  kanji.  The  user  can  copy  a kanji  compound  from  a 
text  (e-mail  or  WWW  site  in  Japanese,  for  example)  and  paste  it  into  the  request  dialogue.  Then  the 
knowledge  agent  will  show  the  corresponding  reading  and  the  meaning. 

Utterances  which  do  not  obey  a language  pattern  are  called  expressions  (Alfonso,  89).  These  are 
referred  to  greetings,  apologies,  etc.  The  knowledge  agent  contains  more  than  200  frequently  used 
expressions  including  their  reading  meaning  in  English. 
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Figure  1.  Analysis  of  the  knowledge  agent  from  a given  verb. 


Language  patterns 

We  consider  a language  pattern  as  a template  which  represents  a general  structure  that  includes 
conjugation  forms  and  grammatical  particles  (Alfonso,  89).  The  knowledge  agent  contains  the  language 
patterns  for  the  conjugation  of  verbs  and  adjectives,  and  applies  them  when  constructing  a conjugated 
verb  or  adjective,  under  the  learner’s  request,  or  while  analyzing  text  (tokens)  provided  by  the  information 
agent  from  html  files  in  the  WWW  or  asked  by  the  learner. 

The  knowledge  agent  currently  contains  more  than  100  adjectives,  together  with  all  the  language 
patterns  corresponding  to  the  adjective  conjugation  rules  according  to  the  Japanese  grammar.  The 
knowledge  agent  is  able  to  recognize  a conjugated  adjective  either  in  hiragana  or  kanji.  The  knowledge 
agent  provides  the  meaning  in  English  and  the  information  of  the  language  pattern  used  for  the 
conjugation  of  the  adjective,  together  with  the  respective  kanji  and  its  reading.  It  is  also  able  to  conjugate 
a given  adjective  in  English,  with  the  corresponding  conjugation  features  indicated  by  the  learner. 

The  knowledge  agent  currently  contains  more  than  900  verbs,  together  with  all  the  language 
patterns  corresponding  to  the  verb  conjugation  rules  of  the  Japanese  grammar.  It  is  able  to  recognize 
conjugated  verbs,  either  in  hiragana  or  in  kanji,  taken  from  text  obtained  from  the  WWW  by  the 
information  agent  or  requested  by  the  user.  The  knowledge  agent  manipulates  its  language  patterns  and 
provides  the  meaning,  kanji  reading,  dictionary  form  of  the  verb  and  the  conjugation  features 
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corresponding  to  its  language  pattern  for  verb  conjugation  (see  figure  1 ).  When  the  learner  asks  for  the 
conjugation  of  a verb,  s/he  selects  the  verb  and  the  conjugation  features  in  English,  and  the  knowledge 
agent  presents  the  results  indicating  the  conjugation  in  Japanese  and  the  reading  of  the  respective  kanji. 


Information  Agent 

Japanese  language  learners  have  special  topics  and  interests.  In  SALLE  people  learn  more  independently 
using  materials  that  meet  their  individual  needs.  The  information  agent  has  been  designed  in  order  to 
determine  the  real,  relevant  and  truly  useful  knowledge  of  the  Japanese  language.  It  helps  in  the  recovery 
of  Japanese  text  from  html  files,  navigating  through  the  Internet  starting  its  travel  from  the  favorite  urls 
provided  by  the  user,  which  represent  her/his  topics  of  interest.  In  this  sense,  the  information  agent  is  a 
"Web  spider"  or  "indexing  robot".  The  information  agent  keeps  track  of  the  Web  sites  already  analyzed, 
autonomously  checks  for  new  information  on  them  (nouns,  adjectives  and  verbs)  and  provide  it  to  the 
knowledge  agent,  which  will  register  it  into  the  community's  knowledge  base. 
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Figure  2.  Information  Agent  Interface 

The  learner  visits  one  of  her/his  sites  being  sure  that  it  still  contains  material  and  that  the 
knowledge  agent  already  included  nouns,  expressions,  verbs  and  adjectives  from  that  site.  This  is  because 
her/his  information  agent  autonomously  checks  all  the  favorite  sites  and  remove  from  the  list  those  which 
are  not  longer  available.  On  the  other  hand,  it  provides  a list  of  updated  sites  which  refer  to  those  favorite 
sites  that  have  been  updated  since  the  user's  last  visit  (see  figure  2). 


Figure  3.  Facilitator  Agent  Interface 


Facilitator  Agent 


With  the  facilitator  agent  we  are  implementing  how  distributed  knowledge  can  be  turned  into  shared, 
collective  knowledge  of  a virtual  community.  According  to  the  constructivism  approach  in  learning 
environments  (Resnick,  96),  knowledge  is  also  constructed  by  the  learner,  not  just  acquired  from  browsing 
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or  attending  presentations.  Interacting  via  SALLE  learners  participate  not  just  passively  reading 
educational  material  in  the  Web,  but  as  active  agents  in  the  construction  of  new  knowledge,  with  the  help 
of  others  (Berger,  98a).  According  to  this  learner  centered  approach  in  the  design  of  learning 
environments,  the  learner  develops  first  a sense  of  ownership  of  her/his  knowledge  and  later  share  it  while 
participating  in  the  virtual  community  of  practice. 

In  SALLE  learners  are  allowed  to  write  their  own  understanding  of  nouns  and  expressions 
registered  by  the  knowledge  agent  (from  the  tokens  given  by  information  agent)  and  which  has  not  a 
reading  and/or  meaning  associated.  The  learner  can  provide  the  reading  and  meaning  to  expressions  and 
nouns  used  in  her/his  personal  context.  The  facilitator  agent  allows  the  provision  of  new  nouns  and 
expressions  together  with  their  meanings  and  proper  use  to  the  knowledge  agent  (see  figure  3).  Later,  with 
the  help  of  a knowledge  negotiation  agent  the  community  will  be  aware  of  the  different  viewpoints  and 
adopt  those  believed  correct  and  meaningful  contributions.  Also,  students  are  invited  to  locate  and  define 
Japanese  Web  pages  according  to  their  interests.  New  urls  for  the  information  agent  are  provided  through 
it  (see  figure  3). 


Knowledge  Negotiation  Agent 

Knowledge  negotiation  (Moyse,  92)  is  an  important  topic  for  computer  assisted  education  and  life-long 
learning.  Life-long  learners  need  to  develop  a critical  attitude,  giving  more  importance  to  their  own 
constructions  (what  their  discover  or  believe  is  true  in  practice)  than  to  the  theoretically  correct  knowledge 
indicated  in  an  authoritative  source,  as  in  the  case  of  a book. 

In  SALLE  knowledge  is  distributed  in  each  member  and  later  shared  among  the  participants  via 
the  facilitator  agent.  Each  noun  or  expression  provided  by  the  learner  may  imply  one  with  a different 
viewpoint  concerning  its  reading  or  meaning.  Therefore  it  is  necessary  to  allow  the  learner  to  approve  or 
reject  the  ideas  of  their  peers,  justifying  her/his  position.  The  role  of  the  knowledge  negotiation  agent  is  to 
decide  the  delivery  of  new  knowledge  from  the  facilitator  agent  to  the  knowledge  agent,  based  on  the 
social  acceptance  of  knowledge,  and  including  the  comments  (justifications)  of  the  beliefs  provided  by  the 
community  members.  The  knowledge  negotiation  agent  promotes  the  agreement  of  members  and  at  the 
same  time  consider  their  status  in  the  community  (novice,  advanced  or  native  speaker)  in  order  to  help  the 
other  learners  to  consider  their  contributions  (see  figure  4). 
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Figure  4.  Knowledge  Negotiation  Agent  Interface 


Conclusions 

SALLE  is  a prototype  for  a life-long  learning  environment  based  on  software  agents  modeled  for  the 
social  construction  and  use  of  Japanese  language  knowledge.  This  is  our  first  step  in  the  proposal  of  a 
paradigm  of  agent  based  life-long  learning  environments  for  Japanese  language,  as  a space  for  the 
application  and  social  construction  of  knowledge.  We  believe  that  the  development  of  environments  where 
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learning  is  based  on  the  actual  and  real  content  and  use  of  knowledge,  taking  place  at  any  time  and  any 
place,  will  be  the  future  for  all  the  life-long  learners  of  the  Japanese  as  a second  language  worldwide. 
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Revisiting  the  Web-based  Performance  Support  Systems  for  Lifelong 
Learning:  Learner-Centered  Resource  Development  Tool 


Joanna  C.  Dunlap,  Ph.D, 
University  of  Colorado  at  Denver 
U.S.A. 

Joni_Dunlap@ceo.cudenver.edu 


Abstract:  Influenced  by  generative  and  intentional  learning  environment  strategies  and  tools, 
a Web-based  tool  has  been  developed  to  empower  learners  to  build  their  own  Web-based 
Performance  Support  Systems  (WPSS)  to  support  their  learning,  professional  development, 
and  performance  within  a domain.  The  original  version  of  the  WPSS  tool  was  presented  at 
ED-MEDIA  98.  The  original  version  did  not  address  the  need  to  create  unique  Web-based 
resources  when  available  WWW  resources  did  not  meet  the  learners’  needs.  Using  document 
sharing,  synchronous  chat  and  whiteboard  technologies,  a learner-centered  resource 
development  tool  has  been  incorporated  into  the  original  WPSS  structure.  In  this  way,  the 
WPSS  not  only  enables  learners  to  (1)  build  a learning  and  performance  resource  that  will 
provide  them  with  immediate  support  and  guidance  and  (2)  help  them  develop  structure, 
strategies,  and  skills  for  subsequent  lifelong  learning  activities,  but  also  (3)  take  responsibility 
for  creating  original  resources  that  support  lifelong  learning  and  performance  support. 


[Note:  A full  version  of  this  paper  is  available  at  http://www.cudenver.edu/'^jdunlap] 

What  do  you  do  if...? 

• Your  company  has  just  completely  a major  reorganization  in  which  a number  of  number  of  positions  — 
including  yours  — have  been  restructured  and  responsibilities  reassigned.  Now  you  are  required  to  work 
on  tasks  you’ve  not  done  before. 

• You’re  a UNIX  programmer  for  a company  that  has  decided  to  replace  its  UNIX  servers  with  PCs.  Now, 
all  new  application  development  must  be  done  for  the  PC  platform. 

• You’ve  been  a technical  writer  in  your  organization  for  three  years.  Now,  in  a move  to  reduce  costs  and 
build  in  quality  assurance  measures,  instead  of  out-sourcing  the  desktop  publishing  of  the  documents  and 
manuals  you  develop  you  will  be  required  to  “publish”  the  documents  yourself 

• Your  company  has  caught  the  WWW  bug  and  wants  to  implement  a company- wide  Intranet.  You’ve  been 
assigned  to  convert  the  print-based  Employee  Handbook  to  an  HTML  document  — by  next  Friday! 

In  a climate  of  rapid  change,  increasing  innovation,  emerging  technologies,  and  proliferating  knowledge, 
lifelong  learning  is  a necessary  professional  development  objective.  In  order  to  keep  current,  people  have  to  be 
willing  and  able  to  continually  “retool”  their  knowledge  and  skill  base.  Simply  knowing  how  to  use  tools  and 
knowledge  in  a single  domain  at  a specific  point  in  time  is  not  sufficient  to  remain  productive  and  competitive. 
People  have  to  face  new  domains  and  novel  situations  with  increasing  frequency  due  to  the  information 
explosion  (Nash,  1994).  The  need  to  be  a continuous  learner  is  especially  apparent  in  domains  influenced  by 
scientific  and  technological  advances;  these  advances  cause  knowledge  and  skills  to  become  obsolete  overnight. 
To  deal  with  today’s  complex  workplace  environment,  employers  need  personnel  who  possess  contemporary 
skills  and  knowledge,  and  are  willing  and  able  to  proactively  update  their  abilities  to  meet  the  ever-changing 
needs  of  the  organization.  Employees  who  are  able  to  keep  up  with  the  information  explosion  are  valuable 
assets;  employees  who  fail  to  “grow  with  the  flow”  are  restructured  out  of  their  positions.  Therefore,  lifelong 
learning  is  essential  to  staying  current,  competitive,  productive,  and  innovative  in  today's  workplace,  and 
therefore  employed  and  in-demand. 

Following  the  prescriptions  of  the  generative  and  intentional  learning  methodologies  that  promote  the 
development  of  metacognitive  and  self-directed  learning  skills  to  support  lifelong  learning  activities,  a Web- 
based  development  tool  was  created.  This  tool  was  designed  to  help  people  in  a workplace  environment 
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generate  their  own,  individualized  Web-based  performance  support  systems  (WPSS)  to  address  the  concerns 
described  above  by  encouraging  and  providing  a supportive  structure  for  lifelong  learning  activities. 


Lifelong  Learning  Deflned 

Lifelong  learning  is  any  purposeful  learning  that  people  engage  in  throughout  their  lives;  it  is  an  activity 
engaged  in.  to  gain  greater  individual  self-fulfillment  and  to  improve  the  quality  of  life  for  the  individual  and 
the  emerging  society  (Overly,  McQuigg,  Silvemail,  & Coppedge,  1980).  The  knowledge  explosion  requires 
professionals  to  engage  in  lifelong  learning  if  they  intend  to  stay  current  — let  alone  evolve,  advance,  and 
remain  competitive  — in  their  profession.  Therefore,  lifelong-learning  skill  development  is  imperative  if  people 
are  expected  to  learn  over  the  full  expanse  of  their  professional  lives.  In  order  to  better  prepare  for  lifelong 
learning  activities,  learners  must  be  exposed  to  learning  activities  that  require  them  to  take  on  and  develop  many 
of  the  responsibilities  normally  afforded  to  educators.  To  achieve  this  requires  moving  away  from  a view  of 
learning  that  is  controlled  outside  the  individual  — by  a teacher,  trainer,  instructional  designer,  or  subject  matter 
expert  — to  a view  of  learning  that  is  internally  controlled  by  the  individual  (Overly  et  al.,  1980).  Therefore,  in 
order  to  internally  control  the  learning  process,  the  development,  and  subsequent  successful  application,  of  two 
skill  areas  — metacognition  and  self-directedness  - is  required. 


Metacognition 

Von  Wright  (1992)  defines  metacognitive  skills  as  “the  steps  that  people  take  to  regulate  and  modify  the 
progress  of  their  cognitive  activity:  to  learn  such  skills  is  to  acquire  procedures  that  regulate  cognitive 
processes.”  Glaser  (1984)  describes  metacognitive  or  self-regulatory  skills  as  knowing  what  one  knows  and 
does  not  know,  predicting  outcomes,  planning  ahead,  efficiently  apportioning  time  and  cognitive  resources,  and 
monitoring  one’s  efforts  to  solve  a problem  or  learn.  More  specifically,  metacognitive  skills  that  are  required  for 
lifelong  learning  include  the  (Ridley,  Schutz,  Glanz,  & Weinstein,  1992): 

• recognition  of  content  and  skill  limitations; 

• ability  to  set  goals  and  create  action  plans  based  on  those  defined  limitations; 

• ability  to  activate  the  appropriate  prior  knowledge  to  achieve  set  goals; 

• ability  to  accurately  assess  progress  in  learning  and  task  performance  and  effectiveness  of  learning 
resources  selected; 

• awareness  of  what  still  needs  to  be  completed  to  reach  a set  goal,  and  how  best  to  allocate  time  and 
resources;  and 

• ability  to  modify  strategies,  tactics,  processes,  and  resource  selection  based  on  the  needs  of  the  task  at  hand. 


Self-directedness 
Self-directed  learning  is: 

...the  process  in  which  individuals  take  the  initiative,  with  or  without  the  help  of 
others,  in  diagnosing  their  learning  needs,  formulating  learning  goals,  identifying 
human  and  material  resources  for  learning,  choosing  and  implementing  learning 
strategies,  and  evaluating  learning  outcomes  (Knowles,  1975). 

The  domain  of  medicine  provides  a perfect  example  of  self-directedness.  When  dealing  with  patients,  the  doctor 
has  to  begin  assessing  the  patient’s  condition  before  having  all  of  the  data  necessary  to  evaluate,  diagnose,  and 
treat  the  patient.  Characteristically,  the  patient  provides  the  doctor  with  fragments  of  information  (“My  stomach 
hurts.  I can’t  hold  any  food  down.  No  one  else  in  my  family  is  experiencing  any  problems.”).  The  rest  of  the 
information  needed  to  solve  the  patient’s  problem  comes  from  the  study  of  a variety  of  other  resources:  patient 
and  family  history,  laboratory  results,  x-rays,  other  doctors’  opinions,  past  experiences,  similar  cases  in  the  case 
file,  and  current  research  findings  on  new  diagnostic  and  treatment  procedures.  These  skills  utilized  by  the 
doctor  to  collect  the  necessary  information  to  solve  the  patient’s  problem  are  described  as  “self-directed 
learning  skills”  (Barrows,  1985,  1986). 
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Barrows  (1995)  defines  the  process  of  self-directed  learning  as  utilizing  the  following  skills  to  solve  a problem 
or  fulfill  a learning  requirement: 

• the  ability  to  identify  and  define  a problem/leaming  need; 

• the  ability  to  identify,  find,  use,  and  critique  resources  for  solving  the  problem  or  meeting  the  learning 
requirement; 

• the  ability  to  capture  and  apply  information  from  resources  to  the  problem  or  learning  need;  and 

• the  ability  to  critique  information,  skills,  and  processes  used  to  solve  the  problem  or  meet  the  learning 
requirement. 


So,  how  do  we  develop  lifelong  learning  skills? 

In  order  to  develop  lifelong  learning  skills,  the  learners  need  to  be  directing  and  driving  the  learning  process  and 
activities  based  on  their  learning  and  performance  needs.  Two  instructional  methodologies  that  specifically 
address  the  development  of  lifelong  learning  skills  are  generative  learning  and  intentional  learning. 


Generative  Learning  Environments 

Generative  learning  environments  require  students  — individually  and  collaboratively  — to  be  responsible  for 
creating,  elaborating,  and  representing  domain  knowledge  in  an  organized  manner  (Cognition  and  Technology 
Group  at  Vanderbilt,  1992;  Hannafin,  1992;  Scardamalia,  Bereiter,  McLean,  Swallow,  & Woodruff,  1989; 
Scardamalia  & Bereiter,  1991).  Through  a this  process  of  “generating”  knowledge,  instead  of  passively 
receiving  information,  learners  develop  structure,  strategies,  and  habit  for  lifelong  learning. 

Generative  learning  environments  require  students  to  take  responsibility  for  determining  what  it  is  about  a 
particular  domain  they  need  to  know,  and  then  direct  their  activities  accordingly  to  effectively  research, 
synthesize,  and  present  their  findings.  Schank  and  Jona  (1991)  describe  a generative  learning  environment  in 
their  discussion  on  the  research  method  of  teaching.  Under  the  research  method  of  teaching,  students  are  asked 
to  research  a particular  topic  and  then  present  their  results  to  others  (the  class,  a collaborative  group,  etc.).  In 
this  way,  students  are  taking  over  the  responsibility  of  information  gathering  and  synthesis  and 
dissemination/presentation  from  the  teacher.  For  this  teaching  method  to  lead  to  successful  learning,  students 
need  to  be  allowed  to  select  their  own  topics  to  research  and  report  on,  so  that  they  have  a real  interest  in 
proceeding  with  the  assignment  and  have  more  control  over  their  learning.  Because  the  learning  is  student- 
directed,  where  each  student  makes  choices  and  takes  responsibility  for  those  choices,  the  learning  is  more 
meaningful;  “...in  general,  material  that  is  organized  in  terms  of  a person’s  own  interests  and  cognitive 
structures  is  material  that  has  the  best  chance  of  being  accessible  in  memory”  (Bruner,  1961).  In  addition, 
because  students  are  responsible  for  selecting  a topic,  developing  a question  to  research,  making  decisions  about 
how  to  gather  information,  analyzing  and  synthesizing  information,  etc.,  they  are  engaging  in  activities  that  help 
to  develop  high-level  thinking  and  problem  solving  abilities. 


Intentional  Learning  Environments 

Intentional  learning  refers  to  the  “cognitive  processes  that  have  learning  as  a goal  rather  than  an  incidental 
outcome”  (Bereiter  & Scardamalia,  1989).  Intentional  learning  encourages  students  to  take  “an  intentional 
stance  toward  cognition”  (Scardamalia  & Bereiter,  1991),  which  means  that  learners  must  learn  how  to  monitor 
and  be  aware  of  their  own  learning  processes,  and  take  responsibility  for  pursuing  desired  and/or  required 
learning  outcomes.  Intentional  learning  is  learning  that  is  actively  pursued  by  and  controlled  by  the  learner 
(Resnick,  1989).  Palincsar  and  Klenk  (1992)  describe  intentional  learning  as  an  achievement  resulting  from  the 
learner’s  purposeful,  effortful,  self-regulated,  and  active  engagement.  By  encouraging  students  to  take  “an 
intentional  stance  toward  cognition”,  intentional  learning  helps  students  learn  how  to  not  only  monitor  and  be 
aware  of  their  own  thinking  and  learning  processes  (i.e.,  metacognitive  skills),  but  also  to  take  responsibility  for 
pursuing  individually-determined  learning  goals  (i.e.,  self-directed  learning). 
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The  objective  of  an  intentional  learning  environment  is  to  create  a supportive  structure  in  which  students  can 
engage  in  cooperative  knowledge  building  as  they  move  towards  greater  autonomy.  Addressing  students’  need 
for  higher-order  abilities  in  thinking  and  learning,  intentional  learning  helps  students  develop  the  general 
metacognitive  and  self-directed  learning  skills  that  facilitate  autonomous  lifelong  learning  (Palincsar,  1990; 
Scardamalia  et  al.,  1989).  These  skills  are  developed  by  engaging  students  in  situations  in  which  they  need  to 
build  a body  of  knowledge  based  on  their  learning  interests  and  needs  using  a variety  of  information  resources. 
While  building  the  knowledge  base,  students  practice  tactics  for  making  claims,  collecting  evidence  in  support 
of  their  claims,  and  evaluating  and  responding  to  counterarguments  from  peers  and  teachers.  Throughout  this 
knowledge-building  process,  students  reflect  on  specific  aspects  of  their  learning  and  thinking  processes,  and 
consider  the  effects  of  collaboration  on  each  other’s  learning,  such  as  the  impact  of  opinion,  bias,  controversy, 
debate,  and  negotiation  (Glaser,  1991). 


Throw  them  all  together  and  what  do  you  get?:  Web-based  Performance  Support 
Systems  (WPSS) 

Influenced  by  generative  and  intentional  learning  methodologies  as  well  as  electronic  performance  support 
systems  (EPSS)  technology,  I developed  a Web-based  tool  empowers  learners  to  build  their  own  Web-based 
Performance  Support  Systems  (WPSS)  to  support  their  learning,  professional  development,  and  performance 
within  specific  domains.  Similar  to  electronic  performance  support  systems  (EPSS),  a WPSS  uses  the  Web  to 
provide  on-demand  access  to  integrated  information,  guidance,  advice,  assistance,  training,  and  tools  that 
support  high-level  job  performance.  In  fact,  using  the  Web  to  create  performance  support  systems  is  a perfect  fit 
because  the  Web  is  actively  used  by  professionals  as  a forum  for  the  distribution  of  current  and  up-to-date 
references,  instruction,  and  guidance. 

By  creating  a structure  that  supports  individualized  and  collaborative  knowledge  building  by  the  people  who 
will  actually  be  using  the  knowledge,  the  higher-order  thinking,  problem-solving,  and  decision-making  involved 
in  the  selection  and  utilization  of  appropriate  learning  materials  and  performance  support  is  done  by  those  who 
can  get  the  most  out  of  the  process.  In  addition,  because  these  activities  occur  in  the  workplace  and  are  driven 
by  the  needs  of  the  job  at  hand,  the  learning  activities  are  contextualized,  authentic,  and  meaningful.  Enabling 
people  to  utilize  an  easy-to-use  tool  to  develop  their  own  WPSS  accomplishes  two  goals: 

1 . they  learn  about  the  domain  while  they  are  locating,  evaluating  (which  requires  utilization  of  resources), 
and  organizing  resources  to  support  their  job  performance  activities  and/or  their  generative  and  intentional 
learning  activities;  and 

2.  once  the  WPSS  is  completed  it  can  be  used  to  support  performance  and  further  professional  development 
while  working  in  that  domain. 

In  this  way,  the  WPSS  not  only  enables  learners  to  build  a learning  and  performance  resource  that  will  provide 
them  with  immediate  support  and  guidance,  but  also  helps  them  develop  structure,  strategies,  and  skills  for 
subsequent  lifelong  learning  activities. 


Overview  of  Original  WPSS  Development  Tool  Components 

The  WPSS  development  tool  helps  employees  — collaboratively  and  as  individuals  - organize,  assess,  and 
utilize  Web-based  resources.  In  order  to  build  an  effective  WPSS,  the  development  tool  enables  employees  to 
organize  Web  resources  into  a variety  of  self-determined  categories.  Categories  may  include: 

• cue  cards:  brief  definitions,  reminders,  directives,  job  aids,  best  practices 

• computer-based  instruction:  tutorials,  case  studies,  practice  activities 

• wizards:  intelligent  demonstration/application  functions;  assistance 

• coaches:  response  sensitive  correction  and  feedback 

• mentors:  individualized  responses  to  questions  from  experts  in  the  field 

• practitioner  forum:  access  to  other  practitioners  in  the  field 

• examples 

• tools 
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In  order  to  build  a WPSS  that  meet  individualize,  specific  learning  and  job  performance  needs,  employees 
engage  in  a number  of  generative  and  intentional  learning  activities  including: 

• determining  their  learning  needs  and  goals 

• developing  a plan  for  action  for  finding  resources  to  help  fulfill  those  goals 

• researching  Web  resources  that  meet  the  appropriate  needs 

• utilizing  Web  resources  in  order  to  evaluate  usefulness,  difficulty  level,  strengths  and  weaknesses 

• updating  links  to  Web  resources  when  appropriate 

• responding  to  other  learners’  comments  regarding  WPSS  contributions 

• developing  Web  resources  via  HTML  pages  and  threaded  discussion  fomms 

In  other  words,  employees  practice  and  develop  the  very  skills  and  strategies  needed  to  engage  in  lifelong 
learning  activities  while  they  are  learning  domain-specific  content  and  skills  needed  for  their  jobs. 

Overview  of  New  WPSS  Development  Tool  Components: 

The  Learner-Centered  Resource  Development  Tool 

Although  the  original  implementation  of  the  WPSS  was  successful,  it  had  a major  flaw:  it  relied  on  pre-existing 
Web  resources.  People  using  the  WPSS  had  to  rely  on  the  WWW  to  provide  the  various  resources  needed  to 
create  a knowledge  base  that  would  support  their  learning  and  performance  support  activities.  For  the  most 
part,  the  WWW  provided  the  resources  needed.  However,  especially  in  Intranet  environments  in  which 
employees  needed  access  to  organizational  specific  resources  that  did  or  did  not  already  exist,  the  WPSS  failed 
to  meet  the  learners'  needs.  Therefore,  a tool  was  created  to  enable  people  - as  individuals  or  in  collaborative 
groups  - develop  unique  Web-based  resources  for  inclusion  in  the  WPSS. 

The  Learner-centered  Resource  Development  Tool  (LRDT)  combines  the  technologies  of  document  sharing  and 
synchronous  chat  and  whiteboard  to  create  an  environment  in  which  people  can  develop  — from  scratch  — their 
own  Web-based  resources.  The  LRDT  uses  synchronous  chat  and  whiteboard  features  to  provide  a fomm  for 
group  brainstorming  about  the  contents  of  a new  resource  document.  The  document  sharing  feature: 

• enables  learners  to  track  and  archive  various  versions  of  new  resource  documents, 

• utilizes  asynchronous  threaded  discussion  technology  to  allow  reflective  discussion  around  the 
development  of  new  Web-based  resources,  and 

• provides  easy  uploading  and  downloading  of  resource  documents  for  revision  purposes. 

The  LRDT,  now  in  beta  testing,  extends  the  usefulness  of  the  WPSS  by  empowering  WPSS  users  to  develop 
their  own  Web-based  resources. 

Examples  of  the  WPSS  in  Use 

Although  still  in  a formative  stage  with  enhancements  being  added  all  the  time,  there  are  a number  of  examples 
of  the  WPSS  tool  in  action.  These  WPSS  examples  can  be  viewed  for  examination  purposes  only  at  the 
following  URL:  http://www.cudenver.edu/~jdunlap/wpss.html 
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Abstract:  This  paper  examines  the  requirements  for  choosing  software  and  New  Media 
technological  solutions  to  facilitate  future  delivery  platforms  for  Education,  It  outlines  evaluation 
models  and  instruments  for  selecting  software,  and  discusses  various  media,  both  current  and 
future,  which  may  aid  pedagogic  delivery.  Through  appropriate  use  of  technology  and  software, 
selected  in  accordance  with  the  models  outlined,  learning  experiences  may  be  enriched  and  learners 
may  actively  engage  in  their  individual  learning  experience  despite  differing  learning  styles. 


Introduction 

The  explosion  of  technology  that  has  recently  occurred,  has  forced  many  educators  to  be  faced  with  the  daunting 
problem  of  identifying  appropriate  technological  solutions  for  Education.  The  task  of  selecting  appropriate 
technology  for  the  development  of  curricular  resources  for  educational  application,  is  something  for  which  few 
educators  have  received  any  formal  training,  but  are  often  required  to  undertake  in  the  course  of  their  employment. 
There  are  technologies  that  have  been  common  place  in  classrooms  for  many  years.  These  include  the  blackboard, 
over-head  projector  (OHP)  and  whiteboard. 

New  media  technologies  are  not  so  common  in  educational  institutions,  although  they  are  becoming  more  common 
as  time  goes  by.  One  relatively  new  technology  which  has  been  employed  by  many  educators,  is  computer 
projection  using  presentation  software  (i.e.  PowerPoint)  and  LCD  screens  on  OHP’s  or  dedicated  computer 
projectors.  With  the  rapid  evolution  and  convergence  of  New  Media  Technologies,  educators  are  being  told  that  they 
must  adapt  their  delivery  styles  and  make  use  of  these  new  tools  for  education.  Also,  the  entire  delivery  of  education 
is  said  to  be  about  to  undergo  a paradigm  shift  with  the  proliferation  of  distance  education. 

How  do  we  go  about  selecting  hardware  and  software  for  educational  use,  when  we  are  not  familiar  with  how  it  can 
be  used?  Do  we  just  purchase  the  latest  technology  and  then  try  to  build  a pedagogical  paradigm  around  it.  This 
approach  may  work  if  we  have  unlimited  human  and  financial  resources  available  to  throw  at  these  technological 
solutions,  however,  most  educational  institutions  do  not.  Is  it  the  technology  that  is  driving  pedagogy,  or  is  the 
content  available  to  make  the  technology  merely  a conduit  for  delivery?  It  is  often  beneficial  to  first  look  at  what  has 
not  been  successful  in  the  past.  Gayeski,  D.  (1989),  identified  a number  of  technologies  which  were  hailed  as  being 
effective  tools  for  education  but  failed  to  gain  adoption.  These  include:  Educational  Television;  Teaching  Machines; 
Dial  Access;  Videotext;  and  to  some  extent  Interactive  Cable.  Technologies  were  also  identified  with  limited 
success,  including:  Video;  and  Visual  Aids  (Business  Slides  and  Overheads),  Questionable  technologies  were 
described,  including:  Teleconferencing;  CAI;  Interactive  Video;  CD-ROM;  and  AI,  Now,  some  ten  years  later,  these 
questionable  technologies  are  undergoing  a convergence  to  form  part  of  the  New  Media  Technologies, 

We  need  to  careftilly  evaluate  technology  before  we  adopt  it.  Learning  paradigms  can  not  be  based  on  technology 
alone,  but  must  be  based  on  content.  Information  is  not  knowledge.  It  is  not  sufficient  for  educators  to  merely 
disseminate  information  to  learners,  who  in  turn  may  be  able  to  recall  this  information  when  assessed  and  examined. 
The  learners  need  to  be  able  to  apply  the  information  in  various  situations  for  knowledge  to  indeed  have  been 
gained. 
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In  evaluating  technological  solutions  we  need  to  take  into  account  many  factors,  some  of  which  are: 

• Economic  Viability  - Capital  Expenditure,  Software  Development  or  Acquisition,  Operational  Cost,  Provision 
of  User  Help  Systems,  Maintenance,  Upgrade  Path. 

• Software  Availability  - Suitability  of  Content  to  required  learning  environment,  Development  Cost 

• Flexibility  - Customisation  feasibility. 

• Reliability  - Technical,  Educational. 

• Ease  of  use  - Educators,  Students. 

When  selecting  technological  solutions,  the  paramount  concern  must  be  to  ascertain  whether  the  pedagogical  needs 
are  best  addressed  by  the  tools  under  review,  and  whether  they  are  economically  viable.  It  is  only  through  a carefully 
planned  approach  that  it  is  possible  to  reveal  the  realisation  of  instructional  objectives  through  their  implementation. 


Models 

Different  models  have  been  developed  for  evaluating  software,  however,  few  are  designed  specifically  for 
evaluating  educational  technology  and  software.  Most  of  the  existing  models  approach  the  evaluation  from  a wide 
perspective  and,  if  anything,  focus  on  user  interface  factors  and  not  pedagogical  factors.  A pedagogical  model  for 
selecting  educational  software  has  been  outlined  (Komoski,  P.K.,  1995),  which  involves  seven  steps: 


1.  Analyse  Needs 

• Needs  & Goals 

• Objectives 

2.  Specify  Requirements 

• Compatibility  with  Available  Hardware 

• Cost  - Multiple  Copies?  Site  License  Necessary? 

• User  Friendliness 

• Level  of  Interaction  Desired 

• Adequacy  of  Documentation  1 

« Access  to  Technical  Support  via  Toll-Free  Number  | 

• Direct  Correlation  with  Instructional  Objectives  and  Curriculum  Requirements  j 

Identified  in  Needs  Analysis  | 

The  following  criteria  should  be  applied  within  the  context  of  the  Objectives  and  Student ‘s  Needs, 

• Content  j 

• Instructional  Presentation 

• Demands  Placed  on  the  Learner  j 

• Technical  Features 

• Documentation  and  Management  Features 

3.  Identify  Promising  Software 

• Educational  Products  Information  Exchange  (EPIE) 

• Listservs  - AskERIC@ericir.syr.edu 

4.  Read  Relevant  reviews 

• ERIC  database  etc. 

5.  Preview  Software 

• Observe  Students 

• Are  Educational  Objectives  Achieved? 

• Preview  with  own  Students  or  Avoid  Purchase! 

6.  Make  Recommendations 

• Select  Most  Desirable  Software  after  Systematic  Evaluation  of  all  alternatives  in  terms  i 

of  Educational  Objectives  and  Constraints 

• Establish  Quantitive  Method  for  rating  each  alternative  against  the  Selection  Criteria 

established  in  Step  2 j 

• Evaluate  Relative  Importance  of  each  Selection  Criteria  - Preview  should  probably  be 

rated  high 

• Create  Written  Report 

7.  Get  Post-Use  Feedback 

• Determine  Conformance  or  Discrepancy  to  Enabling  Objectives  specified  in  Step  1 
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Software  needs  to  be  evaluated  by  educators  before  being  implemented,  in  an  attempt  to  enhance  the  learning 
experience  of  the  students.  Through  the  use  of  exemplary  resources,  students  may  then  be  offered  a richer  learning 
environment.  However,  to  be  able  to  truly  evaluate  technological  solutions,  including  curricular  resource  software, 
one  must  first  look  at  who  the  students  are,  and  what  is  the  desired  knowledge  to  be  conveyed.  This  investigation 
must  be  undertaken  before  we  can  look  at  the  media  available  to  support  the  curriculum. 

Heinich  et  al  (1996)  devised  the  ASSURE  model  as  a procedural  guide  for  planning  and  conducting  instruction  that 
incorporates  media  (technology  and  curricular  resource  software),  specifying  six  stages: 

A:  Analyse  the  Learner 

S:  State  the  Instructional  Objectives 

S:  Select  Methods,  Media  and  Materials 

U:  Utilise  Media  and  Materials 

R:  Require  Learner  Participation 

E:  Evaluate  and  Revise 

This  model  appears  to  be  more  useful  because  it  involves  the  students  from  an  earlier  stage.  It  also  facilitates  a more 
iterative  process  through  the  inclusion  of  revision.  An  entire  piece  of  software  should  not  be  ignored  and  discarded 
because  it  fails  to  meet  all  the  instructional  objectives.  If  it  meets  some  of  the  objectives,  better  than  the  other 
products  available,  sections  may  be  used  as  an  effective  tool  to  assist  students  with  a particular  segment  of  the  topic. 


Instruments 

In  order  to  be  able  to  evaluate  any  software  objectively  it  is  good  to  have  a well  developed  systematic  instrument 
such  as  an  evaluation  form.  This  can  be  a quantitative  instrument  or  checklist,  where  the  evaluator  simply  ticks  an 
appropriate  box  against  a known  grading  scale  or  rubric  (e.g.  Poor,  Fair,  Good,  Excellent)  (Children's  Software  Review 
1995).  It  can  take  the  form  of  a questionnaire,  where  sufficient  space  is  allowed  for  the  evaluator  to  compare  the 
software  product  to  a known  set  of  heuristics  and  write  detailed  comments;  or  it  can  be  a combination  of  both. 

A checklist  is  used  by  Children's  Software  Revue,  where  evaluators  respond  to  items  by  ticking  one  of  four  possible 
responses  (always;  some  extent;  never;  and  N.A.-Not  Averaged),  to  five  categories  of  criteria.  Through  the 
application  of  evaluation  rubrics,  the  California  Instructional  Technology  Clearinghouse  (CITC)  have  reviewed  and 
categorised  over  2,000  curricular  resource  software  titles  into  a searchable  database.  Their  evaluation  is  also  divided 
into  five  broad  sections,  but  in  a much  more  pedagogical  approach.  The  CITC  provides  for  three  levels  of  grading: 
makes  an  excellent  case  for  recommendation;  makes  an  good  case  for  recommendation;  makes  an  minimal  case  for 
recommendation.  This  approach  seems  to  be  a more  quantitative  approach,  through  the  use  of  a checklist.  The 
complete  checklist  can  be  viewed  at:  http://clearinghouse.kl2.ca.us 


New  Media  Technology 

New  Media  Communication  Hybrid  Technologies,  may  facilitate  learning  to  take  place  at  any  time  and  in  any 
location.  They  may  also  enable  the  delivery  of  intelligent  multi-layered  interactive  multimedia  solutions  which  will 
help  ensure  that  learners  are  absorbing  the  desired  information,  whilst  not  having  to  traverse  subject  matter  which  is 
already  known.  Some  of  the  New  Media  Technologies,  innovations,  and  devices  which  may  shape. educational 
These  technologies  are  paradigms  and  pedagogy  in  the  new  millennium  are: 

• Broadband  Cable  Data  Networks  and  Satellite  Data  Transmission 

• Video  on  Demand,  High  Definition  Television,  Digital  Television  and  3D/Omnidirectional  Video  Imaging 

• Digital  Versatile  Disc 

• Machine  Translation 

• Virtual  Displays  and  Body  Wearable  Computers 

• Intelligent  Agents  and  Multimodal  Interfaces 

These  technologies  are  outlined  in  Frohlich  (1998). 
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The  facilitation  of  Education  must  always  rely  on  content  rather  than  the  technology  of  the  delivery  medium.  A 
well-drawn  sketch  on  a blackboard  can  convey  information  much  better  than  a poorly  produced  slide-show.  The 
paradigms  utilised  in  education  have  evolved  over  many  years,  however,  it  is  only  relatively  recently  that 
Multimedia  has  been  able  to  fulfil  an  integral  role  in  facilitating  DL  through  being  able  to  provide  differing 
instructional  techniques  to  suit  different  Learning  Styles.  Technology  can  be  of  great  benefit  in  the  delivery  of 
Distance  Education  and  DL,  or  it  can  become  a barrier  as  well.  Different  people  assimilate  information  through 
different  cognitive  paths  and  processes.  Like  a "Chinese  Whisper"  situation,  re-transmission  of  information  may 
continue,  along  with  the  addition  of  'noise'  and  'distortion',  for  a number  of  steps  until  the  information  transmitted 
bears  little  resemblance  to  the  original  information.  This  particular  form  of  degradation  of  information  rarely  applies 
in  Distance  Education  situations,  because  it  is  usually  a single  stepped  process,  however,  the  information  may  still 
not  have  been  correctly  understood  by  the  learner.  This  may  be  through  the  addition  of  noise  into  the  learner's 
cognitive  process  through  the  delivery  of  the  information  not  aligning  with  their  preferred  learning  style.  To  be  sure 
that  the  information  has  been  understood  by  the  learner,  it  is  best  for  them  to  have  to  put  it  into  their  own  words  or 
images,  and  transmit  it  back  to  the  educator.  This  is  usually  achieved  through  the  learner  being  subjected  to 
answering  questionnaires,  completing  exams  and  undertaking  other  assessment  tasks. 

In  face-to-face  delivery  by  a ‘sage  on  the  stage’,  the  teacher  usually  has  the  benefit  of  having  visual  contact  with  the 
students  and  can  gain  feedback  as  to  whether  ‘the  lights  are  on’  and  whether  the  topic  is  being  comprehended  by  the 
students.  People  rarely  want  to  ‘lose  face’  and  appear  ignorant  by  asking  questions  in  front  of  their  peers,  so  students 
are  often  inhibited  in  revealing  that  they  are  unable  to  follow  the  subject  matter  in  front  of  the  whole  class.  They 
may  also  be  reluctant  to  make  suggestions  which  may  be  perceived  as  incorrect  (Koschmann  et  al  1995).  Even 
experienced  teachers  are  often  unable  to  gain  the  required  feedback  as  to  whether  they  are  teaching  the  students 
topics  which  they  already  know.  The  teacher  is  often  unable  to  move  on  to  another  topic  without  boring  and 
alienating  some  students  by  regurgitating  information.  Students  usually  respond  to  social  pressure  and  complete 
classes  even  if  they  find  the  content  boring  and  uninteresting.  If  the  learning  style  of  the  student  does  not  align  with 
the  manner  in  which  the  information  is  presented,  the  student  may  often  leave  with  little  more  knowledge  than  when 
they  entered.  Even  though  students  may  have  been  presented  with  the  required  information,  information  is  not 
knowledge.  The  cognitive  transfer  of  information  to  knowledge  may  not  have  been  facilitated  through  the  style  of 
delivery  being  mismatched  with  the  student’s  learning  style. 

In  asynchronous  DL  there  is  often  no  social  pressure  to  stay  and  complete  the  class.  Masie  et  al  (1998)  asserts  that 
when  students  are  on-line  they  are  usually  merely  one  click  away  from  exiting  the  programme.  Learners  switch-off 
and  leave  for  a number  of  factors,  some  of  which  are:  "Been  there,  done  that!";  "I’ll  never  use  that  knowledge!"; 
"This  site  is  SO  slow!";  "I  can’t  figure  out  what  to  do!"  These  factors  need  to  be  eliminated  in  order  to  keep  learners 
motivated  to  complete  the  course.  One  way  of  maintaining  interest  in  the  learning  process  may  be  to  utilise  user- 
model  based  interfaces.  A prototype  form  of  user-model  based  interface,  utilising  adaptive  link  annotation,  has  been 
implemented  on  the  WWW  in  the  form  of  Adaptive  Navigation  Support  (ANS).  This  has  proved  to  be  beneficial  to 
student’s  learning  as  long  as  they  accepted  and  followed  the  navigational  advice  provided  (Eklund  et  al,  1998).  In  the 
future,  more  advanced  user-model  based  interfaces  should  allow  us  to  make  use  of  styles  of  delivery  which  appear  to 
the  learner  as  being  less  didactic,  but  facilitate  the  absorption  of  information  and  the  cognitive  transfer  of 
information  to  knowledge.  The  subject  information  needs  to  be  designed  for  delivery  in  numerous  ways  to  provide 
for  different  learning  styles.  These  differing  styles  of  delivery  must  be  envisaged  and  planned  for  at  the  authoring 
stage  of  the  courseware,  and  the  delivery  interface  must  also  provide  for  this  flexibility  in  delivery. 

An  intelligent  agent  (Woodnorth,  1998b)  client  interface  may  provide  the  learner  with  their  prescribed  topics  in  a 
controlled  navigational  environment.  The  learner  may  unknowingly  be  restricted  from  travelling  to  wherever  they 
want,  but  would  transparently  be  directed  in  their  travels  by  the  client.  They  may  only  be  offered  a limited  choice  of 
navigational  paths  and  would  be  gently  steered  throughout  the  learning  experience.  The  intelligent  agent  would 
constantly  update  a profile  of  the  learners  knowledge  and  preferred  learning  style  through  the  use  of  audit  trails  and 
constructive  evaluative  exercises  and  tests.  The  pedagogical  preference  for  Problem  Based  Learning  (Camp,  1996) 
may  be  facilitated  with  students  being  given  problems,  then  interacting  with  an  intelligent  agent  to  find  a solution. 
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Conclusion 


Communication  Technologies  and  software,  if  selected  carefully,  may  empower  educators  to  construct  rich  learning 
environments  where  all  learners  can  be  actively  engaged  in  their  individual  learning  experience,  despite  their 
differing  learning  styles.  Through  the  hybridisation  of  these  new  technologies  the  geographical  barriers  currently 
limiting  the  use  of  technology  for  education,  such  as  lack  of  electricity,  telephone  and  broadband  infrastructure,  can 
also  be  broken  down  so  that  learners  universally  may  become  more  actively  involved  in  their  own  education.  It 
would  appear  that  given  the  correct  mix  of  technology  and  content,  DL  may  finally  come  to  the  forefront  of 
educational  delivery.  As  education  in  the  global  village  becomes  more  of  a reality,  and  distance  ceases  to  be  the 
major  limiting  factor  in  the  delivery  of  affective  education,  we  need  to  exercise  care  so  that  the  educational  benefits 
are  available  to  all  citizens  of  the  planet  and  that  we  do  not  further  the  creation  of  the  new  class  system  of 
information  rich  and  poor. 
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Abstract:  Producing  an  educational  multimedia  CD-ROM  in  a culturally  diverse  environment 
provides  a number  of  challenges.  Content  collection  and  validation,  client  education,  choice  of 
multimedia  consultants,  research  into  cultural  beliefs,  cultural  sub-groups,  geographically 
dispersed  subject  matter  experts  and  agreement  between  content  experts  from  various  para 
professional  disciplines  are  but  a few.  This  case  study  traces  the  development  of  a CD-ROM 
for  Otitis  Media,  a middle  ear  infection  which  results  in  Conductive  Hearing  Loss  in  young 
children,  particularly  indigenous  children,  from  as  early  as  a few  weeks  old  through  to  their 
teenage  years  and  beyond.  The  recurring  incidence  of  Otitis  Media  during  a child’s  early  years 
may  result  in  permanent  hearing  loss  and  is  thought  to  have  a significant  impact  on  a child’s 
educational,  social  and  emotional  development.  Experience  gained  from  this  project  highlights 
the  critical  elements  of  a project  that  must  be  considered  when  developing  a multimedia  CD- 
ROM.  These  include  educating  the  client,  scoping  the  job  accurately,  determining  content 
availability,  choosing  appropriate  consultants  and  setting  up  clear  communication  and  decision 
making  strategies  to  achieve  consensus  between  subject  matter  experts  from  diverse  disciplines. 


Background 


Otitis  Media  has  been  recognised  as  a serious  problem  for  indigenous  children  for  many  years.  When  associated 
with  Conductive  Hearing  Loss  (CHL),  it  is  the  most  common  ailment  experienced  by  children  of  the  90’s 
[National  Center  for  Health  Statistics,  1992].  In  New  Zealand  alone  Otitis  Media  is  estimated  to  cost  the 
community  $704  million  per  annum,  an  increase  of  150%  since  1975  [Allen,  1993].  Otitis  Media  is  known  to 
affect  children  from  as  young  as  a few  days  old  through  to  their  teenage  years  and  beyond. 

While  much  debate  still  exists  with  respect  to  the  overall  effect  of  Otitis  Media,  it  is  widely  recognised  that  CHL 
with  Otitis  Media  can  cause  children  to  suffer  difficulties  with  their  auditory,  linguistic,  social  and  educational 
development  [Hasenstab,  1987;  Kavanagh,  1986;  Webster,  1988].  As  a result,  it  is  suggested  that  these  children 
will  also  experience  significant  social  and  emotional  difficulties  during  their  early  development.  [Jacobs  1993] 
stated  “The  real  cost  of  recurrent  Otitis  Media  must  be  counted  in  terms  of  low  self  esteem,  children  who  cannot 
make  friends  or  children  who  are  in  the  vicious  cycle  of  failure  at  school”. 

[Rockeley  & Rhys-Evans  1986],  estimated  that  “up  to  80%  of  all  children  experience  at  least  one  episode  of 
Otitis  Media  by  the  age  of  5 years  with  a peak  occurrence  between  6 and  18  months  of  age”.  Research  in 
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Australia  has  shown  that  up  to  70%  of  indigenous  children  in  a classroom  may  experience  Otitis  Media  with 
associated  Conductive  Hearing  Loss  at  any  one  time  [Lowell,  1993]. 

Project  Objectives  and  Scope 


In  1 996  the  Education  Department  of  Western  Australia  in  conjunction  with  Edith  Cowan  University  undertook 
to  produce  a CD-ROM  to  increase  awareness  of  Otitis  Media  and  present  caregivers  with  guidelines  on  the  early 
recognition  and  appropriate  intervention  strategies  that  could  be  adopted  to  support  children  with  Conductive 
Hearing  Loss.  “PRISM  Interactive”,  a Western  Australian  based  Multimedia  Company  was  commissioned  to 
provide  design  and  production  services  for  the  development  of  the  CD-ROM. 

When  the  project  was  initially  commissioned  the  target  audience  was  identified  as  three  distinct  groups: 

• children  [4-7  yearsjwho  experience,  or  have  experienced  Otitis  Media; 

• teachers/indigenous  education  workers;  and 

• parents  of  children  in  the  high  risk  groups. 

As  the  scope  of  the  project  developed,  the  target  audience  was  extended  to  include  other  caregivers  responsible 
for  the  well  being  of  children  in  their  care  and  a range  of  para-professional  groups  such  as  health  workers, 
audiologists  and  speech  pathologists. 

The  key  objectives  of  the  CD-ROM  were  to: 

• identify  the  common  causes,  signs  and  symptoms  of  Otitis  Media  with  associated  CHL 

• identity  the  educational,  social  and  emotional  difficulties  experienced  by  children  with  OM; 

• provide  a series  of  guidelines  on  appropriate  intervention  strategies  that  can  be  used  to  support  children 
suffering  from  recurring  Otitis  Media; 

• provide  a comprehensive  data  base  of  support  agencies/groups  and  teaching  resources  that  can  be  used  to 
assist  children  with  OM;and 

• provide  children  within  the  age  group  4-7  with  an  understanding  of  the  cause  of  OM,  self  help  strategies  that 
they  can  employ  and  identify  those  within  their  immediate  family/community  that  can  provide  support. 

As  part  of  the  scoping  process,  the  Education  Department  of  Western  Australia  established  a working  party 
whose  members  were  considered  leaders  in  their  field  of  expertise.  It  consisted  of  15  people  including  teachers, 
speech  pathologists,  audiologists,  health  care  workers,  indigenous  education  workers,  ear  nose  and  throat 
specialists,  surgeons,  and  nationally  and  internationally  recognised  researchers  in  the  area  Otitis  Media. 

To  commence  the  scoping  stage  the  Education  Department  convened  a two-day  workshop  with  the  working 
party,  which  generated  much  enthusiasm  and  ideas.  The  workshop  also  highlighted  the  following: 

• the  scope  of  the  project  was  much  broader  than  first  estimated  and  the  original  project  time  frames  were  not 
achievable; 

• it  would  be  difficult  to  achieve  consensus  among  the  members  of  the  working  party  on  both  the  objectives 
and  content  of  the  CD-ROM; 

• the  Education  Department  of  Western  Australia  (the  client)  had  very  little  useable  content;  and 

• the  original  budget  was  not  going  to  be  sufficient. 

Content  Collection  Issues 

When  the  budget  for  the  project  was  initially  developed,  only  a small  amount  of  funding  had  been  allocated  for 
content  collection  on  the  premise  that  the  client  would  be  able  to  provide  the  necessary  content  to  allow  the 
project  to  commence.  During  the  scoping  process  it  became  apparent  that  the  client  did  not  have  access  to  this 
content  and  that  insufficient  funding  had  been  allocated  to  this  stage  of  the  project.  This  left  the  working  party 
with  the  dilemma  of  how  the  required  content  would  be  identified  and  collected  to  enable  the  commencement  of 
storyboarding  and  multimedia  production. 

While  this  would  normally  have  seen  most  projects  suspended,  the  working  party  who  were  very  much  aware  of 
the  impact  that  Otitis  Media  was  having  on  the  development  of  young  children  volunteered  their  services  to 


ensure  that  the  project  could  proceed.  Without  this  support,  and  the  expertise  of  this  group,  it  is  unlikely  that  the 
project  would  have  been  completed.  Such  was  the  commitment  of  the  working  party  that  on  one  occasion  a 
surgeon  left  the  operating  theatre  to  answer  a question,  leaving  his  assistant  to  continue  with  the  operation! 

During  the  scoping  process  it  also  became  apparent  that  many  of  the  para  professional  groups  represented  on  our 
working  party  had  dissimilar  views  on  some  of  the  finer  aspects  associated  with  the  impact  of  OM  on  a child’s 
development.  In  the  absence  of  any  defined  source  of  documented  content,  a literature  search  was  initiated  to 
identify  any  content  and  research  that  would  support  the  views  that  would  eventually  be  depicted  on  the  CD- 
ROM.  This  was  to  take  three  months  to  complete.  As  relevant  content  was  sourced  or  developed,  it  had  to  be 
filed,  referenced  and  cleared  for  copyright.  The  information  collected  required  careful  cataloguing  to  allow 
designers  to  quickly  retrieve  information  when  they  returned  to  this  material  up  to  three  months  later. 

However,  new  research  findings  kept  emerging  that  caused  continual  changes  in  storyboards  and  on-screen 
content.  While  this  was  having  a significant  impact  on  the  scheduled  delivery  date  the  working  party  felt  that  the 
latest  research  must  be  reflected.  At  this  point  it  was  decided  that  a Web  site  would  be  developed  to  complement 
the  CD-ROM  to  allow  access  to  future  research  findings. 

With  up  to  five  different  subject  matter  experts  across  each  of  the  professions  within  the  working  party  it  was 
often  difficult  to  reach  agreement  on  the  critical  elements  and  treatment  of  the  content.  To  overcome  this  issue,  it 
was  decided  that  the  instructional  designer  would  take  the  role  of  content  identification  and  assessment.  This 
person  was  perceived  as  not  having  a particular  bias  towards  any  particular  discipline  and  was  able  to 
communicate  and  collaborate  with  all  the  subject  matter  experts.  However,  obtaining  agreement  on  content  still 
continued  to  have  a significant  impact  on  the  project  delivery  date.  Many  of  the  subject  matter  experts  were 
living  in  different  states  across  Australia,  so  feedback  and  approval  of  storyboards  was  time  consuming,  often 
taking  up  to  two  weeks  to  be  returned.  In  some  instances  storyboards  underwent  no  less  than  twelve  iterations 
before  all  the  subject  matter  experts  would  sanction  commencement  of  production. 

To  help  make  the  process  of  content  collection  and  approval  more  efficient,  it  was  eventually  agreed  that  only 
two  subject  matter  experts  would  approve  the  initial  content.  Storyboards  would  then  be  circulated  to  the  whole 
working  party  for  final  approval.  This  helped  reduce  the  time  required  to  validate  the  storyboards  and  the  stress 
on  the  instructional  designers!. 

Cultural  Issues 

Apart  from  the  difficulties  associated  with  the  collection  of  content,  this  project  was  made  more  difficult  to 
manage  due  to  the  wide  cultural  diversity  of  the  main  target  audience,  Australia’s  indigenous  population.  The 
client  was  aware  that  within  indigenous  populations  there  exists  many  different  groups,  each  with  different 
cultural  beliefs  which  had  to  be  considered  if  the  final  product  was  to  gain  acceptance  and  be  actively  used  as 
resource.  This  presented  a major  challenge  for  the  design  team  resulting  in  the  need  to  work  closely  with  many 
members  of  different  indigenous  groups  to  identify  and  reach  consensus  on  the  appropriate  treatment,  interface 
design,  learning  styles,  graphics  styles  etc. 

While  the  design  team  was  aware  of  many  of  the  more  commonly  known  cultural  issues,  on  a number  of 
occasions  various  segments  of  the  program  had  to  be  revised  in  light  of  the  feedback  received  from  indigenous 
groups.  Examples  included  ensuring  that  graphics  did  not  depict  children  looking  directly  into  the  faces  of  adults, 
skin  textures  and  facial  features  were  not  indicative  of  any  one  group  and  that  consent  was  obtained  from  parents 
and  adults  shown  in  the  videos  or  photographs.  In  Aboriginal  culture  once  a person  dies  it  is  taboo  to  view  the 
face  of  a deceased  person.  Even  the  way  photos  were  transitioned  on  and  off  the  screen  had  to  be  carefully 
considered  to  ensure  that  we  did  not  offend  the  beliefs  of  Indigenous  people. 

This  cultural  diversity,  and  need  to  be  aware  of  many  different  customs  required  a much  wider  cross  section  of 
the  target  audience  be  sampled  than  would  normally  be  needed  during  the  formative  stages  of  the  project.  While 
this  would  have  a significant  impact  on  the  time  frame  of  the  project  it  was  considered  critical  if  the  project  was 
to  be  accepted  in  all  the  indigenous  communities  that  it  would  eventually  be  used  in. 


Project  Resourcing  & Client  Education 

As  mentioned  earlier  the  client  had  no  experience  of  multimedia  CD-ROM  development  and  had  little 
knowledge  of  the  resourcing  implications  of  such  projects.  Originally  it  was  anticipated  that  the  project  would 
take  about  three  months  to  complete  (it  eventually  took  eighteen  months),  with  the  belief  that  content  was  readily 
available  or  could  be  easily  sourced.  On  this  basis,  an  Education  Department  employee  was  allocated  25%  of 
their  time  to  coordinate  the  project.  Unfortunately,  even  after  the  full  scope  of  the  project  was  known  (and  lack  of 
available  content),  the  time  allocation  for  the  client  coordinator  could  not  be  increased. 

As  a consequence,  the  project  became  dependent  upon  the  energy  of  a few  people  who  contributed  many  hours 
outside  normal  working  hours  to  collect  content.  During  the  latter  stages  of  the  project  this  extra  burden, 
compounded  by  dealing  with  diverse  subject  matter  experts  and  the  impact  of  continual  changes  to  meet  varying 
cultural  needs,  was  the  cause  of  considerable  personal  stress  for  some  team  members. 

This  highlighted  the  need  for  “Educating  the  client”  about  correctly  scoping  a project,  and  having  content  and 
appropriate  resources  available  before  commencing  a project  of  this  size  and  complexity.  Lack  of  content  is  an 
important  issue  in  multimedia  production  which  must  be  considered  carefully  before  undertaking  a project  of  this 
size.  In  light  of  our  experiences  we  would  strongly  recommend  that  when  undertaking  a project  of  this  nature  a 
separate  contract  for  the  identification  and  collection  of  content  should  be  completed  before  attempting  to 
estimate  the  budget  and  production  schedule. 

Contracting  the  right  people  was  essential  to  complete  this  project.  Consultants  were  chosen  who  had  completed 
similar  work  in  the  past.  In  this  case  it  was  important  that  the  main  developer  had  instructional  design  and  CD- 
ROM  production  expertise.  Checking  the  quality  of  past  productions  and  asking  previous  clients  about  their 
satisfaction  level  gave  a good  indication  of  which  consultants  were  suitable.  Other  attributes  that  were  required  in 
selected  consultants  included  communicating  honestly,  clearly  and  on  a regular  basis  with  reporting  schedules 
showing  allocated  responsibilities,  and  excellent  interpersonal  and  team  skills. 

“The  client  will  want  a quality  product  at  the  cheapest  possible  price,  and  the  developers  will  want  the  biggest 
possible  profit  margin!” 

Copyright  Issues 

Copyright  clearances  were  necessary  for  all  media,  teaching  activities  and  papers  that  made  up  the  CD-ROM 
information  database.  In  some  instances  the  design  team  discovered  that  some  of  the  materials  provided  were  not 
the  legitimate  property  of  the  source.  This  resulted  in  the  need  to  implement  a policy  where  all  content  providers 
were  required  to  demonstrate  proof  of  ownership  of  the  materials  that  they  were  supply.  If  sufficient  proof  of 
ownership  was  not  available,  then  the  materials  provided  were  not  used.  While  this  occasionally  restricted  the 
amount  of  useable  content  that  was  identified  it  was  necessary  to  protect  the  interests  of  all  parties. 

Information  had  to  be  maintained  about  each  media  element  (graphics,  text,  video,  sound  & voice)  used  in 
production  i.e.  the  source,  ownership,  shipping  information,  loan  duration,  copyright  status,  licensing  and  other 
relevant  attributes  needed  to  be  stored  and  maintained. 

Implementation  Issues 

As  part  of  the  implementation  of  this  CD-ROM  a policy  on  Otitis  Media  in  Western  Australian  government 
schools  is  being  developed  by  the  Education  Department  of  Western  Australia.  Professional  development 
programs  have  been  conducted  in  the  sixteen  education  districts  throughout  the  state  to  ensure  that  all  teachers, 
school  health  workers,  indigenous  education  workers  and  other  caregivers  are  aware  of  the  implications  of  Otitis 
Media,  the  availability  of  this  CD-ROM  and  the  manner  in  which  it  can  best  be  used  to  educate  the  wider 
community.  This  process  also  involves  groups  from  outside  the  education  sector  i.e.  health,  allied  health, 
Aboriginal  Medical  Services  and  other  community  groups  must  be  invited  to  attend  the  professional  development 
sessions.  However,  as  occurred  during  the  design  and  production  phase  of  the  project,  implementation  has  been 
hindered  by  the  lack  of  resources  necessary  to  conduct  professional  development.  Without  additional  resources, 
concern  is  also  held  for  the  maintenance  of  the  program. 
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Promotional  materials  and  copies  of  the  CD-ROM  have  been  distributed  free  of  charge  by  the  Education 
Department  to  all  government  primary  schools  throughout  Western  Australia  and  will  be  provided  at  minimal 
cost  to  all  other  education  sectors  around  Australia. 

The  Education  Department  of  Western  Australia  has  received  numerous  letters  of  commendation  from  many 
professional  groups  both  nationally  and  internationally,  including  the  World  Health  Organisation. 

Although  no  structured  marketing  has  occurred  demand  for  the  CD-ROM  has  outstripped  supply  and  resulted  in 
the  Education  Department  outsourcing  the  reprint,  distribution  and  future  marketing  of  the  CD  ROM. 

Recommendations 

From  the  experience  gained  in  this  project,  the  following  recommendations  are  made  for  anybody  considering  the 
development  of  a computer  based  training  CD-ROM  within  a culturally  diverse  environment: 

• have  a thorough  understanding  of  the  content  required  and  the  resources  needed  to  collect  it; 

• consider  issuing  a separate  contract  to  identify  and  collect  content  prior  to  finalising  the  scope,  budget  and 
time  frame  of  the  project; 

• educate  the  “ultimate  client”  , subject  matter  experts  and  other  stakeholders  about  the  key  facets  of 
managing,  developing,  coordinating,  and  resourcing  a project  of  this  size; 

• insist  on  sufficient  dedicated  time  and  resources  for  a client  coordinator; 

• ensure  that  a strong  representation  of  the  cultural  groups  that  make  up  the  target  audience  are  included  on  the 
working  party; 

• have  a clear  understanding  of  agreed  objectives  and  adhere  to  these  at  all  times  to  keep  the  project  on  track; 

• ensure  that  costings  and  time  schedules  are  not  locked  in  prior  to  a proper  scoping  of  the  project  being 
completed; 

• ensure  that  the  client  is  made  aware  at  the  beginning  of  the  project  of  the  resource  requirements  required  to 
implement  and  maintain  the  final  product; 

• use  developers  and  consultants  who  are  experienced  in  this  type  of  development; 

• use  subject  matter  experts  and  consultants  who  have  good  interpersonal  skills,  communication  skills  and  a 
“good  sense  of  humour”! 
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Abstract.  Distance  learning  has  played  a major  role  in  Australian  education  for  much  of  the  20^^  century  as 
distance  and  remoteness  made  access  to  traditional  facilities  very  difficult  for  many.  Now,  however,  it  has  become 
a mode  of  necessity  for  many  people  whose  life/work  styles,  rather  than  geography,  dictate  how  they  can  access 
education  and  training 

The  Australian  National  Training  Authority  has  developed  a national  policy  on  'flexible  delivery’,  with 
considerable  focus  on  learning  in  the  workplace  or  'on-the-job'  (ANTA,  1996).  It  defined  flexible  delivery  as 
'commonly  uses  the  delivery  methods  of  distance  education  and  the  facilities  of  technology'. 

The  development  of  this  policy  was  on  the  basis  of  a number  of  unexamined  assumptions.  These  assumptions 
suggested  that  vocational  education  and  training  clients  are  sufficiently  skilled  to  access  training  programs  offered 
through  flexible  delivery,  including  on-line  delivery. 

The  research  reported  in  this  paper  set  out  to  examine  these  assumptions. 

Background 

Distance  learning  has  played  a major  role  in  Australian  education  for  much  of  the  20^^  century  as 
distance  and  remoteness  made  access  to  traditional  facilities  very  difficult  for  many.  Over  time  it 
developed  also  as  a mode  of  choice.  Now,  however,  it  has  become  a mode  of  necessity  for  many 
people  whose  life/work  styles  dictate  how  they  can  access  education  and  training.  It  is  less  about 
choice  and  remoteness  today,  and  more  about  the  pressures  of  work  and  life  dictating  an  only  viable 
mode:  distance  learning  method. 

The  Australian  National  Training  Authority  has  developed  a national  policy  on  'flexible  delivery',  with 
considerable  focus  on  learning  in  the  workplace  or  'on-the-job'  (ANTA,  1996).  It  defined  flexible 
delivery  as  'commonly  uses  the  delivery  methods  of  distance  education  and  the  facilities  of 
technology’. 

However,  the  development  of  this  policy  position  was  made  on  the  basis  of  a number  of  unexamined 
assumptions.  These  assumptions  included:  That  vocational  education  and  training  clients 

• want  to  learn  through  flexible  delivery  and  on-line  and  have  the  capacity  to  take  advantage  of  this; 

• are  sufficiently  skilled  to  access  vocational  education  and  training  programs  offered  through 
flexible  delivery,  and; 

• can  learn  in  the  workplace. 

The  research  reported  in  this  paper  set  out  to  examine  these  assumptions. 


Students:  Questionnaires  were  distributed  to  professional,  trade  and  para-professional  students  who 
were  concurrently  in  employment  (worker/leamers).  The  students  were  chosen  from  a wide  range  of 
occupations  and  from  small  medium  and  large  workplaces.  No  differences  in  response  were  found  by 
sex,  size  of  enterprise  or  course. 


Subjects 


Project  Methodology 


The  research  used  questionnaire  surveys,  and  structured  focus  group  interviews  to  address  the  research 
questions. 

Questionnaire 

This  is  a very  comprehensive  130  item  likert-type  scale  in  two  parts.  Part  one  canvasses  student 
opinions  about  their  abilities  as  learners  and  about  their  perceptions  of  and  experience  with  various 
forms  of  instructional  communication,  including  technology.  It  also  canvasses  their  perceptions  about 
their  own  abilities  as  independent  learners.  Part  two  uses  a commercially  available  scale  to  assess 
subjects’  self-directed  learning  readiness  (Guglielmino,  1977  ). 

Structured  Interviews 

These  were  conducted  in  focus  groups  with  a sample  of  the  worker/leamers’  employers  and  a sample 
of  the  TAPE  teachers  who  were  involved  in  teaching  the  worker/leamers. 

Results 

Surveys  were  completed  by  542  worker-learners  Three  hundred  and  twenty  particpants  were  male  and 
two  hundred  and  twenty  one  female  with  one  participant  not  indicating  gender.  Sixty  eight  percent 
were  under  twenty  five  years  of  age.  The  breakdown  for  size  of  workplace  in  which  they  were 
employed  showed  that  thirty  four  percent  were  in  small  workplaces  with  less  than  twenty  employees, 
thirty  eight  percent  in  medium  workplaces  with  up  to  five  hundred  employees  and  slightly  over  nine 
percent  were  in  large  enterprises  with  more  than  five  hundred  employees. 

Data  also  were  interrogated  for  age  between  those  under  25  years  and  those  above.  There  were  no 
significant  differences  for  Part  1 of  the  Questionnaire,  but  there  was  a significant  difference  in  relation 
to  self-directed  learning  readiness.  This  was  surprising  for  Part  1 because  general  assumptions  about 
technology  use  would  suggest  that  the  older  age  groups  were  less  confident  and  competent.  This  was 
not  the  case,  although  it  needs  to  be  recognised  that  neither  age  group  reported  high  levels  of 
competence  or  confidence  with  technology. 

Questionnaire  Part  1 

Participants  reported  a substantial  (high  or  medium)  degree  of  confidence  in  themselves  as  learners 
(93.8%),  said  they  were  self  motivated  (92%),  and  could  work  independently  with  little  direction 
(91.8%).  However,  nearly  one-third  (30.8%)  said  they  were  not  capable  of  self-directed  learning,  with 
only  fourteen  percent  strongly  agreeing  that  they  were  capable  of  self-directed  learning.  In  other 
words,  respondents  in  the  sample  seem  confident  working  under  the  supervision  of  a teacher,  but  not 
under  their  own  supervision.  It  is  interesting  to  note  in  a related  study  that  Australian  and  Scottish 
students  were  less  at  ease  with  teachers  than  their  United  States  counterparts,  but  were  more  open  to 
self-directed  learning  (Christie  and  Warner,  1999). 

Preferred  mode  of  Learning 

Respondents  reported  that  traditional  face-to-face  learning  was  their  most  preferred  mode  of  learning. 
This  is  shown  in  Table  1 . 

TABLE  1:  Most  Preferred  Mode  of  Learning 


PREFERRED  MODE 

% 

Face  to  face 

42 

ERIC 
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Learning  Packages 

5 

Electronic 

3 

Guided  experiential 

5 

Combination,  inc.  face  to  face 

43 

Further,  fifty-five  percent  of  all  respondents  believed  that  their  future  learning  would  be  delivered 
face-to-face. 


Learning  Technologies 

In  the  emerging  area  of  learning  technologies,  participants  reports  give  cause  for  concern  in  relation  to 
flexible  delivery.  Forty-one  percent  had  no  computer  based  learning  experiences,  fifty-two  percent 
had  not  experienced  CD  Rom  learning  and  sixty-three  percent  had  no  experience  of  the  Internet  for 
learning.  Only  the  Internet  is  reported  on  in  this  paper. 

The  Internet 

For  the  majority  of  participants,  the  Internet  was  a new  medium  for  which  they  reported  low  skill,  or 
no  experience.  Their  self-reports  of  competence  are  shown  in  Table  2. 

TABLE  2:  Level  of  Reported  Competence  in  Internet  Tasks. 


Task 

No 

Experience 

Low 

Medium 

High 

Accessing 

37 

18 

22 

19 

Browsing  www 

43 

17 

18 

18 

Emailing 

46 

19 

17 

14 

Downloading  programs 

50 

18 

15 

13 

Chat  groups 

51 

15 

19 

11 

Database  search 

47 

17 

19 

18 

Despite  the  above,  participants  did  report  that  they  perceived  that  the  Internet  would  be  useful  for  their 
learning  (66.8%). 

While  sixty-three  percent  had  no  prior  experience  of  the  Internet,  some  sixty-two  percent  said  that 
they  currently  have  access  to  it  either  at  home  (15.5%),  in  the  workplace  (6.1%),  or  through  their 
course  provider  (19.2%),.  Eighteen  percent  stated  that  they  had  access  to  the  Internet  through  more 
than  one  location. 

These  results  give  some  cause  for  concern.  They  show  that  about  two  thirds  of  respondents  had  either 
no  competence  or  no  skill  in  performing  Internet-related  tasks.  These  findings  endorse  the 
observation  that  students  require  progressive  exposure  to  the  use  of  higher-end  technologies  to  support 
their  learning. 

Questionnaire  Part  2 

Part  two  of  the  questionnaire  dealt  with  an  assessment  of  student’s  learning  preferences  and  used  a 
commercially  available  scale  to  measure  this  dimension  of  student  readiness  for  self-directed  learning. 
“Self  directed  learning  is  a strategy  or  a process  whereby  the  learner  accepts  the  responsibility  for 
planning,  seeking  out  resources,  and  implements  and  evaluates  their  own  learning”  (Brookfield,  1984. 
p.l6).  Scoring  norms  for  this  scale  are  depicted  in  Figure  1 . 
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TABLE  3:  International  Norms  for  Learning  Preference  Assessment 

(Guglielmino,  1991)  and  Australian  VET  Sample 


Score 


Level  of  Readiness 


Australian  VET 
sample 


252-290 

227-251 

202-226 

177-201 

58-176 


High 

Above  average 
Average 
Below  average 
Low 


% 

8 

19 

32 

29 

12 


Results  revealed  information  which  may  give  cause  for  concern.  For  example,  the  mean  score  on  this 
scale  for  the  Australian  VET  client  sample  is  202,  with  seventy  three  percent  of  the  Australian  sample 
scoring  226,  or  below.  Comparison  with  international  norms  reveals  that  our  sample  has  only  low  to 
average  readiness  for  flexible  learning.  Forty  one  percent  of  the  sample  score  only  below  average  or 
low  on  flexible  readiness. 

Half  of  the  students  in  the  present  sample  possess  below  average  to  low  readiness  for  self-directed 
learning  and  the  scores  for  three  quarters  of  the  sample  are  average  or  below.  This  indicates  that 
despite  massive  investment  in  flexible  learning  technology  and  infrastructure,  the  skill  level  of 
students  may  have  remained  unaddressed.  The  VET  sector  has  apparently  failed  to  acknowledge  that 
similar  investment  needs  to  be  made  to  ensure  that  clients  can  cope  with  flexible  learning  demands. 

The  results  call  attention  to  the  fact  that  mandating  a flexible  learning  mode  for  all  VET  clients  will 
potentially  disadvantage  three  quarters  of  the  client  population.  This  is  poor  pedagogy  and  poor 
business. 

However,  a significant  difference  for  age  was  identifies  between  those  under  25  years  and  those 
above.  The  latter  indicated  a greater  readiness  for  self-directed  learning  with  some  40%  scoring  above 
average  and  higher,  compared  to  21%  of  the  under  25  year  group.  This  has  implications  for  how 
course  developers  and  instructors  differentiate  between  younger  post-school  students  and  mature  age 
students  in  the  workforce. 

The  research  findings  do  point  to  a cohort  of  VET  clients  with  above  average  to  high  dispositional 
readiness  and  also  with  a range  of  self-directed  learning  skills,  including  the  use  of  learning 
technologies. 

Those  clients  with  the  attitude  and  skills  that  are  disposed  to  flexible  delivery  do  want  choice.  This  is 
not  readily  available  to  them  now.  However,  a collaborative  approach  in  establishing  ‘virtuaF 
provision  of  training  could  support  these  clients  and  provide  a new  framework  of ‘learning  efficiency’. 

Structured  interviews  with  teachers 

Overwhelmingly  teachers  believe  that  face-to-face  delivery  is  more  successful  as  a college  strategy 
than  other  modes  of  delivery  in  promoting  student  learning.  Optimun  training  is  achieved  through  a 
combination  of  face  to  face  learning  with  college  lecturers,  and  workplace  training  and  practice  with 
workplace  supervisors.  A further  critical  factor,  perhaps  the  most  important  factor  in  any  method  of 
delivery  was  described  as  interactivity. 

The  teachers  interviewed  said  that  a range  of  skills  are  required  of  students  to  profit  from  flexible 
delivery.  They  feel  that  students  may  possess  some  or  all  of  these  skills  to  varying  degrees  but  few 
possess  all  of  the  skills  at  a sufficiently  high  level.  The  skills  nominated  by  teachers  were: 


• Time  Management 

• Goal  Setting 

• Self-direction  Skills 

• Computer  Literacy 

• Technology  Skills 

• Basic  Literacy  and  Numeracy 

• Comprehension  and  Meta-Cognitive  Skills 

They  perceived  all  of  these  skills  to  be  essential  for  flexible  learning.  They  did  not  indicate  any 
teacher  or  institutional  responsibility  for  helping  students  actually  develop  these  skills.  They  generally 
attributed  'blame'  to  the  students  and  their  previous  learning  experiences  or  institutions. 

Structured  interviews  with  employers 

The  majority  of  employers  interviewed  were  familiar  with  the  term  flexible  delivery  but  most  had 
some  difficulty  or  confusion  with  the  precise  definition  of  the  term.  Most  saw  flexible  delivery  as 
somehow  giving  more  emphasis  or  focus  to  the  learner,  and  they  saw  it  as  more  focussed  on  the 
individual  rather  than  on  the  group. 

Employers,  without  exception,  mentioned  literacy  skills  and  numeracy  skills  as  the  most  fundamental 
skills  required  of  employees  to  benefit  from  any  kind  of  training.  Among  the  essential  skills  for 
flexible  delivery  success  are: 

• Literacy  and  Comprehension 

• Problem  Solving  and  Information  Retrieval  Skills. 

• Analytical  Thinking  Skills 

• Keyboard  Skills 

They  saw  these  skills  as  more  important  than  for  example,  computer  literacy  skills  which  they  believe 
can  more  easily  be  taught  on  a just-in-time  basis.  Above  all,  employers  believe  that  trainees  need  to 
come  to  the  training  experience  with  high  levels  of  motivation  to  learn  and  high  levels  of  self-direction 
skills. 

Employers  from  larger  workplaces  knew  more  about  flexible  delivery  and  the  needs  of  employees  in 
relation  to  training.  Larger  workplaces  tend  to  have  more  equipment  than  do  smaller  workplaces. 
They  also  tend  to  have  more  text-based  resources  and  are  more  likely  to  be  technologically  better 
equipped.  Overall  larger  workplaces  seem  better  able  to  support  students  with  more  non-human 
support. 

Conclusion 

This  study  has  found  that  the  assumptions  on  which  flexible  delivery  and  online  delivery  are  based  are 
untenable.  On  the  basis  of  the  current  research,  the  assumption  that  students  are  ready  for  flexible 
delivery  of  courses  does  not  seem  warranted.  Both  disposition  and  skill  for  self-directed  learning  were 
low  on  an  international  benchmark  scale.  Differentiation,  based  on  age,  however,  could  provide  better 
opportunities  for  mature  aged  workers.  Further,  there  is  evidence  from  the  interviews  with  VET 
teachers  and  employers  that  the  VET  sector  has  a limited  understanding  of  flexible  delivery  and  what 
might  be  needed  for  it  to  be  successful.  Teachers  and  employers  generally  saw  the  problem  as 
belonging  to  the  student/employee  and  resulting  from  poor  previous  school  experiences.  They 
identified  skills  needed,  but  did  not  indicate  any  belief  that  they  might  have  a major  role  to  play  in 
enhancing  the  capabilities  of  students/employees. 

In  terms  of  the  use  of  learning  technologies,  people  in  the  VET  sector  have  limited  experience  and 
competency  of  them  as  learning  tools.  While  basic  computing  skills  were  evident,  their  reports  did 
not  indicate  any  real  sophistication  in  higher-order  computing  or  Internet  skills.  For  both  present  and 
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future  learning  there  was  clearly  a preference  on  the  part  of  students  for  face-to-face  traditional 
delivery  in  college..  There  was  little  evidence  that  their  VET  experiences  exposed  them  to  learning 
using  new  technologies  or  developing  the  skills  to  be  able  to  learn  thus.  However,  there  were  some 
indications  that,  given  opportunity  and  learning  skill,  the  students  might  well  develop  the  disposition 
to  learn  differently.  Certainly,  there  is  no  evidence  to  suggest  that  VET  clients  could  not  develop  the 
capability  to  learn  through  flexible  delivery.  What  needs  to  happen  is  an  attitude  change  on  the  part  of 
students,  teachers  and  employers  and  attention  given  to  the  development  of  skills  associated  with 
flexible  learning,  particularly  skills  associated  with  new  learning  technologies. 
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Abstract:  Audiographics  technologies  enjoyed  a period  of  relatively  widespread  use 
and  progressive  refinement  in  the  early  to  mid  1990’s.  Their  growth  and  development 
was  however  paralleled  by  the  growth  and  development  of  the  Internet  and  the  World 
Wide  Web.  Since  the  mid  1990s  the  functionality  and  appeal  of  collaborative  Web-based 
tools  has  slowly  gained  favour  amongst  educators  with  the  consequence  that  they  have 
began  to  replace  the  more  specialised  functionality  of  established  audiographics 
programs  such  as  Electronic  Classroom  ® with  the  more  diversified  and  increasing 
sophisticated  Web-based  programs.  The  low  speed  and  poor  reliability  of  phone  line 
connections  and  congested  Internet  service  lines  are  factors  that  can  still  give 
audiographics  programs  the  edge  in  some  non-metropolitan  and  remote  areas.  The 
results  of  a telephone  survey  of  Electronic  Classroom  ® users  indicates  that 
audiographics  use  has  at  best  stabilised  and  at  worst  is  in  decline  in  the  face  of  an  ever 
increasing  array  of  Web-based  collaborative  tools.  The  detailed  comments  of  Electronic 
Classroom  ® users  in  relation  to  the  decision  abandon  audiographics  and  move  to 
another  educational  technologies  will  be  presented  as  part  of  the  conference 
presentation. 


1.  Introduction 

This  paper  addresses  the  effect  of  technological  paradigm  shifts  on  investments  in  educational  infrastructure, 
courseware  and  teacher  skill  sets.  The  paper  reviews  the  paradigm  shift  that  has  occurred  in  educational 
information  technology  with  the  arrival  of  the  Internet  and  the  World  Wide  Web  (the  "Web").  The  Web  has 
literally  affected  all  pre-existing  educational  information  technologies  to  a greater  or  lesser  extent.  In  this 
research  we  address  the  impact  of  the  Web  on  one  relatively  widely  adopted  educational  information  technology: 
audiographics.  This  technology  is  a form  of  synchronous,  distributed  multimedia  software.  Audiographics  is 
designed  to  support  distributed  learning  across  a range  of  educational  levels.  How  have  developers  of 
audiographics  software  responded  to  the  challenge  presented  by  the  Internet  and  the  Web?  Equally  importantly, 
how  have  educational  administrations,  institutions  and  individual  teachers  reacted  to  the  new  opportunities  and 
challenges  presented  by  the  Internet  and  the  Web? 

This  paper  proceeds  as  follows:  In  the  next  section,  the  nature  of  Kuhnian  paradigm  revolutions,  or  what  are 
more  commonly  referred  to  as  paradigm  shifts,  is  discussed.  The  next  section  introduces  audiographics 
educational  technologies.  We  review  the  adaptation  made  by  suppliers  of  audiographics  software  to  Internet 
technologies.  We  then  analyse  the  response  of  experienced  users  of  a well-established  Australian  audiographics 
product.  Electronic  Classroom  ®,  to  the  changes  brought  about  by  the  Internet  and  the  Web.  In  the  final  section, 
we  make  some  recommendations  for  further  research  into  substantial  changes  or  paradigm  shifts,  in  educational 
technology. 


2.  Paradigm  Shifts  in  Educational  Technology 

Kuhn  (1962:  83),  explains  that  a paradigm  revolution  in  science  occurs  when  a "continually  resistant 
component  of  Science  - an  anomaly  in  the  paradigm"  occurs.  He  notes  that  a paradigmatic  revolution  "is 
complete  when  the  profession  will  have  changed  its  view  of  the  field,  its  methods,  and  its  goals"  (Kuhn 
1962:84).  Paradigmatic  revolutions  "must  occur  all  at  once  (though  not  necessarily  in  an  instant)  or  not  at  all" 
(Kuhn  1962:  149).  While  the  original  Kuhnian  view  of  a paradigm  revolution  was  to  describe  the 
comprehensive  shifting  of  plate  tectonics  in  scientific  disciplines,  the  more  popularly  accepted  term  of  paradigm 
shift  has  come  to  represent  a fundamental  move  in  understanding  and  use  of  technologies  or  other  human 
activity.  For  example,  Capra  (1982:  30)  defines  a paradigm  shift  as  "a  profound  change  in  the  thoughts, 
perceptions,  and  values  that  form  a particular  vision  of  reality".  The  term  has  now  entered  into  the  public 
consciousness  and  has  been  used  widely  in  a variety  of  disciplines. 

There  has  been  recognition  in  recent  years  of  the  importance  of  information  technology  in  reshaping  a variety  of 
human  endeavours.  The  work  of  Negroponte  (e.g.  Negroponte,  1995)  and  Tapscott  (e.g.  Tapscott  1995;  1997; 
and  Tapscott  and  Caston  1992;  1993)  have  been  influential  in  pointing  to  the  fundamental  implications  of  new 
computer-based  technologies  for  the  way  that  societies  are  organised.  While  the  work  of  Negroponte,  Tapscott 
and  others  have  not  been  linked  to  any  particular  technology,  we  can  say  that  the  extraordinary  move  of  the  Web 
(Berners-Lee  and  Cailliau  1990;  Berners-Lee  1997)  from  the  relative  confines  of  the  high-energy  physics 
community  to  the  general  global  community,  constitutes  a paradigm  shift,  even  a paradigm  revolution,  under 
any  definition.  In  the  space  of  less  than  a decade  the  Web  has  grown  into  an  unfathomable  multimedia  resource. 
Intkomi  Inc.  estimates  that  there  are  more  than  200  million  publicly  available  pages  on  the  Internet  that  a 
search  engine  can  index.  There  are  probably  many  more  pages  hidden  behind  corporate  Intranets  and  database 
engines.  The  Web  has  affected  the  development  of  a wide  variety  of  pre-existing  information  technologies  from 
local  and  wide-area  networks,  to  group  decision  support  and  collaborative  systems  to  e-commerce  and  logistics. 

3.  Audiographics 

Audiographics  is  a set  of  linked  technologies  that  supports  synchronous  and  distributed  teaching  and  learning 
(Barker  and  Patrick  1989;  Rehn  and  Towers  1994;  Ellis,  Debreceny  and  Crago  1996).  In  the  "pre-Internet" 
period,  an  audiographic  installation  would  involve  computers  at  physically  distributed  teaching  sites  connected 
by  modem  and  telephone  line.  The  sites  would  also  be  connected  by  telephone  line  for  voice.  This  is  illustrated 
in  Figure  1 : 


Figure  1:  A basic  audiographics  installation 

Typically,  audiographics  software  supports  one  or  more  of  shared  whiteboards  with  graphics  and  chat  features. 
In  the  case  of  the  Electronic  Classroom  ® product,  shared  access  to  sounds  and  movies  in  QuickTime  ® format 
was  also  provided.  Audiographics  was,  for  some  time,  the  only  multimedia  computer-based  form  of  educational 
technology  that  could  support  synchronous,  different  place  interaction  with  graphical  support.  Of  course,  pre- 
Internet  technologies  such  as  bulletin  boards  and  chat  rooms  could  support  such  interaction,  but  in  a text-only 
mode.  The  place  of  audiographics  in  the  broader  span  of  educational  technologies  is  illustrated  in  Figure  2 
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Figure  2:  Typology  of  Educational  Technologies 

Three  products  took  the  major  share  of  the  worldwide  audiographics  marketplace.  These  were  Vis-a-Vis  and 
Smart  2000  from  Canada,  both  of  which  were  in  a Windows  3.1 1 and  subsequently  Windows  95  environment, 
and  Electronic  Classroom  ® from  Australia,  which  operated  in  an  Apple  Macintosh  environment  (Crago  1993). 
As  might  be  expected,  Electronic  Classroom  ® dominated  the  Australian  marketplace,  with  substantial  inroads 
being  made  in  the  Victorian  and  West  Australian  school  system  and,  to  a somewhat  lesser  extent,  the  NSW  and 
Queensland  school  systems  and  higher  education  nationally.  Electronic  Classroom  ® has  provided  the 
foundation  for  a number  of  experiments  in  distributed  teaching  and  learning,  many  of  which  have  been  reported 
at  previous  ED- MEDIA  conferences  (for  example;  Ellis  and  Debreceny  1994;  Arms  1997;  Oliver  and  Lake 
1997;  Stoney  and  Oliver  1998). 

Audiographics  has  been  applied  in  a wide  variety  of  educational  environments,  from  primary  school  to  higher 
and  further  education  (Ellis,  Debreceny  and  Crago  1996).  Audiographics  does,  however,  require  rather  intensive 
investment  of  resources.  Each  site  traditionally  required  two  telephone  lines  for  voice  and  computer-to-computer 
communication.  When  there  is  long-distance  synchronous  teaching  and  learning,  which  has  been  the  situation 
for  many  audiographics  installations,  there  are  additional  communication  costs.  These  costs  may  be  substantial. 

Ellis,  Debreceny  and  Crago  (1996)  documented  the  progressive  technical  improvements  in  Electronic 
Classroom®  functionality  from  version  1.0  in  February  1990  to  version  2.5.9  in  June  1995.  In  April  1996,  the 
release  of  version  3 allowed  users  to  substitute  point-to-point  Internet  connectivity  for  modem  connectivity.  This 
allowed  users  of  try  out  IP  networks  instead  of  dedicated  phone  lines. 

In  contrast  the  Internet,  has  provided  the  promise  of  connection  to  local  points  of  presence  (POPs)  at  the  cost  of 
local  telephone  call.  Further,  with  Internet  support,  only  one  phone  line,  for  audio,  is  required.  Migration  to  an 
Internet-based  solution  also  opens  up  many  opportunities  for  wider  collaboration  and  alternative  forms  of 
collaboration.  Synchronous  teaching  and  learning  can  be  integrated  with  asynchronous  communication  and 
collaboration  by  email,  listserv  or  Web-based  discussion  board. 

A large  number  of  products  have  been  developed  to  support  synchronous  and  asynchronous  collaboration  on  the 
Web.  Products  include  chat  tools  such  as  ChatTouring,  which,  as  the  name  suggests,  is  designed  to  support  text- 
based  synchronous  chatting  but  with  the  ability  to  take  participants  in  the  chat  to  designated  URL’s.  The  chat 
participants’  browser  follows  the  lead  of  the  URL  that  has  been  placed  on  the  ChatTouring  chat  space.  New 
synchronous  tools  have  been  developed  in  the  Internet  environment  that  was  not  possible  with  pre-Internet 
technologies.  For  example,  the  concept  of  a "pager"  such  as  Mirabilis's  ICQ  allowed  for  participants  to  enter 
into  dialogue  with  designated  parties  when  those  parties  come  onto  the  Internet. 

Widely  known  products  that  most  closely  resemble  "traditional"  audiographics  solutions  in  an  Internet 
environment  are  CUSeeMe(TM)  and  ClassPoint(TM)  from  WhitePine,  and  NetMeeting(TM)  from  Microsoft. 
Each  of  these  products  supports  audio,  video,  shared  whiteboards  and  chat.  Each  is  compliant  with  the  ITU’s 
H.323  standard  on  multi-point  video  conferencing.  ClassPoint  is  also  compatible  with  the  T.120  standard  on 
multi-point  data  sharing  . Products  based  upon  T.120  are  designed  to  interoperate  with  H.323-compliant 
products.  ClassPoint(TM)  is  an  enhanced  version  of  CUSeeMe(TM),  which  originated  as  a research  project  at 
Camegie-Mellon  University  (Steele  1994).  The  product  supports  the  multi-point  audio  and  video  of 
CUSeeMe(TM)  as  well  as  integration  of  Web-touring,  along  the  lines  of  ChatTouring  discussed  in  the  previous 
paragraph.  ClassPoint(TM)  also  provides  for  teacher  control  of  the  "classroom"  environment  as  well  as  Web- 
based  teaching  and  learning  materials.  NetMeeting  has  integrated  audio,  video,  shared  whiteboards,  shared 
applications  and  chat.  NetMeeting  is,  however,  limited  to  two  parties  sharing  audio  and  video,  although  others 
can  share  a chat  room  and  shared  applications.  MeetingTools(TM)  from  DataBeam  enhances  NetMeeting(TM) 
to  support  a large  number  of  users,  albeit  without  audio  and  video  support. 

The  pre-Internet  audiographics  products  have  made  varying  attempts  to  become  Internet-friendly.  None  has, 
however,  made  the  transition  completely  successfully.  Developers  of  some  of  the  products  have  tried  to  reduce 
the  cost  of  ownership  by  multiplexing  audio  and  data  on  the  same  telephone  line  (Klein,  Pratap  and  Prestinario 
1997).  No  evidence  has  been  found  for  widespread  adoption  of  such  a solution.  A product,  which  sits  between 
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the  traditional  audiographics  products  and  the  newer  generation  of  Internet  products,  is  FarSite(TM),  also  from 
DataBeam.  It  does  not  support  audio  or  video.  FarSite(TM)  has,  however,  extensive  support  for  Internet-based 
collaboration  tools. 

In  summary,  the  Internet  now  provides  the  promise  of  low  cost  synchronous  collaboration  with  all  the 
functionality  of  the  pre-Internet  audiographics  products.  At  the  same  time,  moving  to  an  Internet  solution 
provides  the  potential  for  teachers  and  learners  to  access  the  resources  of  the  network  and  to  leverage  alternative 
synchronous  and  asynchronous  Internet  collaboration  solutions.  However  it  must  be  remembered  that  the 
reliability  and  speed  of  Internet  connectivity  varies  widely.  How  did  the  users  of  pre-Internet  audiographics 
respond  to  this  paradigm  shift? 

4.  A Case  Study:  Electronic  Classroom® 

4.1  User  Telephone  Survey 
Format  and  questions 

As  discussed  above,  Electronic  Classroom  ® is  a well-developed  and  widely  used  pre-Internet  audiographics 
product.  It  has  been  in  use  in  the  secondary  educational  sector  since  1990.  Although  most  of  the  installations 
have  been  in  rural  and  regional  Australia,  there  have  been  applications  of  Electronic  Classroom  ® in  urban 
environments.  Typically,  schools  in  a defined  geographic  area  will  collaborate  to  jointly  offer  classes  that  could 
not  cost-effectively  offered  by  individual  schools  in  the  consortium.  By  mid  1998  a total  of  Just  over  1000 
Electronic  Classroom  ® licenses  had  been  sold.  This  research  was  designed  to  track  the  decisions  made  by 
active  users  of  Electronic  Classroom  ® in  the  period  from  approximately  mid  1996  to  mid  1998,  that  is,  since 
the  introduction  of  version  3,  which  allowed  Internet  connectivity  to  be  substituted  for  modem  connectivity,  and 
over  the  time  of  rapid  growth  of  collaborative  Web-based  tools. 

A telephone  survey  of  Australian  users  was  conducted  in  August  and  September  1998.  Revelation  Computing, 
the  developer  and  distributor  of  Electronic  Classroom(R)  supplied  some  contact  details  of  users.  Other  contacts 
were  added,  based  on  the  authors'  knowledge  of  users  from  past  research.  The  intention  of  the  survey  was  to 
sample  people  who  had  not  Just  purchased  copies  but  who  were  known  to  have  made  some  use  of  the  product  in 
the  years  prior  to  1998.  The  survey  was  intended  to  be  a simple  pilot  study.  It  was  not  designed  as  a 
comprehensive  review  of  the  entire  product  user  base.  Nevertheless  the  survey  was  stratified.  It  sampled 
metropolitan  and  country  users  in  several  Australian  states.  In  Australia,  primary  and  secondary  education  is  the 
responsibility  of  educational  administrations  at  the  state  level. 

The  majority  of  users  were  known  to  be  active  in  the  various  state  primary  or  secondary  school  systems.  The  age 
of  students  studying  via  Electronic  Classroom  ® sessions  ranged  from  nine  to  eighteen  years  of  age.  The  survey 
was  not  intended  to  be  a detailed  "product  comparison"  but  rather  an  attempt  to  establish  if  any  trends  and 
changes  were  present  and  if  so,  how  the  trends  and  changes  could  be  best  characterised. 

Some  general  points  about  the  survey  population  and  survey  questions  are  relevant.  By  the  very  action  of 
making  use  Electronic  Classroom  ® in  their  teaching  and  learning  environment,  users  can  be  characterised  as 
technologically  literate,  as  innovators  and  as  "early  adopters"  under  the  model  proposed  by  Rogers  (1983). 
Given  this,  it  is  to  be  expected  that  users  would  also  be  active  experimenters  with  other  related  products  and 
hardware  that  might  be  seen  as  offering  similar  features  of  perhaps  even  completely  "new"  instructional 
environments. 

The  structure  of  the  survey  involved  a series  of  questions  of  two  types:  a number  of  optional,  defined  responses 
(e.g.  Yes/No)  or  open-ended  responses  (a  few  sentences  to  a few  paragraphs  were  recorded).  The  initial  question 
was  aimed  at  splitting  the  sample  into  two  sub-groups: 

Opening  question:  Are  you  still  actively  using  Electronic  Classroom  ® in  your  teaching? 

Possible  responses:  Yes  or  No 

Depending  on  the  respondents  answer  to  this  question  one  of  two  other  sets  of  additional  questions  were  asked. 
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If  Yes: 


1.  Is  your  level  of  use:  Possible  responses:  (a)  higher  then  a few  years  ago,  (b)  about  the  same,  (c)  less  than  an  a 
few  years  ago. 

2.  Have  you  looked  for  a potential  replacement  system?  Possible  responses:  (a)  Yes,  actively,  (b)  No,  but  have 
checked  out  "similar"  products,  (c)  No,  not  at  all. 

3.  Have  you  considered  any  other  products  that  might  provide  supplementary  support?  Possible  responses:  (a) 
Yes,  (b)  No 

4.  What  do  you  consider  are  the  current  major  strengths  of  Electronic  Classroom  ®?  Give  details. 

5.  What  do  you  consider  are  the  current  major  weaknesses  of  Electronic  Classroom  ® ?Give  details. 


1.  What  was  your  main  reason(s)  for  abandoning  Electronic  Classroom  ® ? Give  details. 

2.  Has  this  move  proved  to  be  the  right  decision?  Possible  responses:  (a)  Yes,  (b)  No. 


5.  Results 

5.1  The  Yes  group 

The  majority  of  users  surveyed  were  in  the  Yes  group,  that  is.  Electronic  Classroom  ® was  still  being  used  by 
them  and  their  organisations.  However,  the  users  reported  that  the  level  of  use  had  stabilised  or  was  declining. 
No  one  was  actively  looking  for  a replacement  product  but,  as  was  to  be  expected,  all  were  "checking  out 
"similar  Internet-based  products  that  provided  collaborative  screen  sharing  facilities. 

The  majority  of  Yes  respondents  also  reported  they  had  considered,  or  even  trailed  the  use  of  supplementary 
products.  These  included  low  cost,  small  window  videoconferencing  such  as  ClassPoint(TM)  or  CUSeeMe(TM). 
Use  of  such  products  was  seen  as  a means  of  enhancing  the  feeling  of  personal  contact  between  teachers  and 
students  and  amongst  the  student  population. 

The  features  that  were  overwhelmingly  reported  as  being  the  major  strengths  of  Electronic  Classroom  ® were 
its  tenacity  at  maintaining  a continuous  connection  even  over  relatively  poor  quality  telephone  lines  and  its 
suitability  for  delivering  a formally  structured  lesson  based  on  already  developed  content.  These  comments  come 
as  no  surprise  as  they  represent  the  elements  of  the  niche  market  that  the  product  originally  targeted. 

In  terms  of  major  weakness  most  respondents  said  none.  One  respondent  stated  its  inability  to  link  with  other 
similar  products,  in  particular  those  that  run  on  a Windows  platform. 

5.2  The  No  group 

The  main  reasons  for  abandoning  Electronic  Classroom  ® were  related  to  the  rapid  spread  and  evolution  of  the 
World  Wide  Web  and  the  availability  of  shared  whiteboard  and  screen  control  software  as  well  as  the 
availability  of  low  cost  colour  videoconferencing  hardware  and  software.  In  metropolitan  areas  the  availability 
of  Internet  connectivity,  had  clearly  attracted  several  users  into  what  they  saw  as  a more  information  rich 
environment.  This  environment  was  seen  as  more  popular  with  the  respondents'  students.  Connection  to  the 
Internet  was  often  centrally  funded  by  the  respective  State  Education  system,  and/or  available,  at  relatively  low 
cost,  via  commercial  ISP's.  This  contrasted  with  the  cost  of  phone  connections  with  Electronic  Classroom  ® that 
requires  expenditure  of  school  budgets  or  the  continuation  of  special  grants  from  central  administrations. 

All  No  respondents  believed  that  the  move  had  been  the  right  decision.  Several  did  offer  the  comments  that  it 
had  altered  the  structure  of  how  online  lessons  were  now  delivered.  The  respondents  felt  that  the  "more  direct 
control  available  when  using  Electronic  Classroom  ® had  been  lost".  In  contrast,  the  benefits  of  an  Internet 
solution  "were  that  the  students  were  now  searching  for  and  creating  more  of  their  own  resources".  Clearly  the 
popular  appeal  amongst  the  student  population  of  creating  Web  pages  had  been  a factor  decision  to  move  from 
Electronic  Classroom  ® to  an  Internet  solution. 

Details  of  individual  responses  to  the  phone  survey  will  be  incorporated  into  the  conference  presentation. 

6.  Conclusions 


IfNo: 


In  this  paper,  we  have  introduced  audiographics  technology  against  a backdrop  of  a paradigm  shift  in 
educational  technologies.  We  noted  that  new  Internet-based  products  provided  the  functionality  of  pre-Internet 
audiographics  applications.  These  Internet-based  products  allow  users  to  use  a single  telephone  line  for  video, 
audio  and  data  as  well  as  opening  up  schools  to  the  cornucopia  of  resources  on  the  Internet  and  the  Web.  At  the 
same  time,  suppliers  of  audiographics  software  have  not  moved  quickly  to  meet  the  demand  for  Internet 
solutions. 

How  did  schools  respond  to  these  opportunities?  The  strongest  supporters  of  Electronic  Classroom  ® remain 
users  who  want  to  deliver  formally  structured  lessons  and  are  located  in  areas  where  poor  telephone  line  quality 
is  unfortunately  still  the  norm.  Reliability  of  connection  and  poor  bandwidth  in  a rural  and  regional 
environment  mitigates  against  a change.  Attempting  to  support  audio,  video,  shared  whiteboards  and  chat  on  a 
single  telephone  line  requires  high-speed  lines  and  continuous  quality.  A product  such  as  Electronic  Classroom 
® has  a high  level  of  Quality  of  Service  (QoS).  The  QoS  of  the  Internet  cannot  be  guaranteed.  Also  this  group 
has  a major  investment  in  prepared  lesson  content,  which  would  not  be  easily  exported  to  another  environment. 

The  reason  that  usage  levels  have  stabilised  and  the  user  base  is  not  expanding  is  clearly  related  to  the 
increasing  number  of  Internet  connections  in  schools,  the  availability  of  other  screen  sharing  programs  and  the 
increasingly  rich  resource  pool  provide  by  the  Web  sites  around  the  world. 
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Abstract  This  paper  will  report  the  findings  of  a second  study’  to  investigate  the  value  of  a 
Performance  Support  System  (PSS)  for  novices  undertaking  a complex  cognitive  task  (lesson 
planning),  within  an  innovative  instructional  model.  The  intention  of  the  research  was  to 
investigate  the  value  in  using  performance  support  as  a strategy  for  engaging  learning,  by 
describing  how  learning  might  occur  as  a result  of  using  a specific  PSS. 


INTRODUCTION 

The  rationale  for  this  research  lay  in  the  paucity  of  instructional  design  models  or  approaches  for  the 
development  of  educational  software  products,  particularly  for  tertiary  education.  Furthermore,  it  was 
completed  in  the  shadow  of  seminal  comments  by  Brown  (1994),  and  Park  and  Hannafin  (1993),  amongst 
others,  which  are  proving  increasingly  representative  of  the  field  of  instructional  technology,  and  which 
strongly  advocate  the  need  for  new  and  better  informed  (ie.  by  a greater  diversity  in  research  findings) 
instructional  design  models. 

This  research  hypothesised  the  value  of  using  a model  of  instruction  based  primarily  in  the  theory  of  cognitive 
tools  and  in  the  design  methodology  of  (electronic)  performance  support  systems  (PSSs),  but  also  taking 
appropriate  account  of  other  cognitivist  principles  in  instructional  design,  such  as  mental  models,  situated 
cognition  and  authentic  learning.  There  were  two  orientations  to  the  research  reported  in  this  paper: 

1 . Investigate  how  novice  student-teachers  engage  PSS  components  in  the  Lesson  Planning  System  (LPS) — a specific 
PSS  designed  to  support  the  production  of  lesson  plans. 

2.  Investigate  the  effectiveness  of  the  LPS  as  a PSS  to  support  the  completion  of  lesson  planning. 

The  space  allowed  here  prohibits  a full  reporting  of  the  data  collected.  Further  refereed  articles  reporting  on 
this  and  other  data  are  published  elsewhere — for  example,  see  Wild  (1998a;  1998c;  in  press). 

PERFORMANCE  SUPPORT  SYSTEMS 

A Performance  Support  System  is  interactive  software  that  is  intended  to  both  train  and  support  the  novice  user 
in  the  performance  of  tasks.  Raybould  describes  a PSS,  rather  widely,  as  a,  ‘computer-based  system  that 
improves  worker  productivity  by  providing  on-the-job  access  to  integrated  information,  advice  and  learning 
experiences’  (Raybould,  1990).  Traditionally,  PSSs  have  been  characterised  by  their  structure  and  the  software 
resources  they  provide,  and  these  are  usually  determined  to  include:  an  information  base  (eg.  on-line  reference 
and  help  facilities  and  case  history  databases);  interactive  and  learning  experiences;  productivity  software 
(often  used  with  templates  and  forms);  and,  an  advisory  system  (eg.  coaching  facility)  (Gery,  1995). 

THE  CONTEXT  OF  DESIGN  FOR  THE  LPS 

At  the  core  of  the  LPS  is  a model  of  lesson  planning  required  by  Edith  Cowan  University,  Western  Australia, 
and  wider  afield.  This  model  includes  essential  components  of  lesson  planning  such  as  writing  learning 
objectives,  developing  learning  experiences  and  planning  evaluation  (Barry  & King,  1993).  Each  component  is 
supported  by  activities  that  instructs  the  user  about  the  task  (eg.  provision  of  information  relating  to  reasons 
why  objectives  are  necessary,  criteria  for  quality  objectives),  and  which  also  assist  the  user  in  performing  the 
task  (eg.  provision  of  a database  of  verbs  to  assist  in  writing  quality  learning  objectives).  A set  of  software  tools 
are  available  to  support  each  activity.  A full  description  of  the  LPS  can  be  found  in  Wild  (1998b). 
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PROCEDURE 


In  the  second  of  two  major  studies^,  five  students,  all  female,  were  identified  to  provide  the  focus  for  studying 
their  patterns  of  LPS  usage  over  a two  week  period.  These  students  were  volunteers  and  self-confirmed  novices 
in  lesson  planning.  The  students  were  tutored  in  the  use  of  the  LPS,  to  the  point  at  which  they  felt  comfortable 
with  their  skill  in  the  use  of  the  technologies  (ie,  computer  and  software  use).  Students  then  used  the  LPS  to 
plan  a minimum  of  six  lessons,  over  a period  of  two  weeks — producing  these  lesson  plans  so  that  they  might  be 
implemented  in  their  professional  practice  placement  schools  within  a day  or  so  of  their  planning,  and  also 
subject  to  evaluation  by  a university  supervisor,  classroom  teacher,  and/or  school  principal  or  representative,  as 
part  of  the  normal  operating  expectations  in  the  professional  practice  period. 

Observational  data  was  collected  by  video  tape  recordings,  providing  a complete  record  of  use  of  the  LPS  for 
each  student  for  each  session  of  use.  From  the  observ'ational  data,  it  was  possible  to  determine  a count  of 
students’  interactions  with  instructional  and  performance  components  of  the  LPS,  and  the  amount  of  time  it 
took  for  them  to  complete  the  lesson  planning  task  using  the  LPS.  Also,  from  this  data,  an  instructional- 
performance  (IP)  coefficient  was  calculated  to  more  effectively  represent  the  changing  nature  of  LPS  usage  over 
all  lesson  planning  tasks.  This  coefficient  is  calculated  by  dividing  the  total  number  of  instructional  (I) 
interactions  for  each  lesson  plan,  by  the  corresponding  number  of  performance  (P)  interactions.  For  example, 
an  equal  number  of  instructional  and  performance  interactions  for  one  lesson  plan,  would  provide  an  IP 
coefficient  of  one  (1).  Whilst  there  is  no  optimum  IP  coefficient  indice,  when  calculated  over  a range  of  lesson 
planning  tasks,  for  each  student-teacher,  the  IP  coefficients  can  provide  an  indication  of  the  development  of 
expertise  in  students’  lesson  planning  skills. 

Individual  follow-up  interviews  conducted  at  the  completion  of  the  two  week  period  (ie.  within  15  days  of  the 
completion  of  the  final  lesson  plan  observed),  helped  determine  how  all  students  managed  aspects  of  the  lesson 
planning  task.  Interviews  were  conducted  one-to-one,  and  comprised  a series  of  open  questions  which  sought 
to  identify  how  students  perceived  they  completed  the  lesson  planning  tasks,  and  probed  why  students 
performed  and  managed  the  task  in  the  manner  they  described.  The  sixth  and  final  video  tape  recorded  for  each 
student,  was  played  back  to  the  student  at  this  point,  to  elicit  a delayed  think-aloud  procedure,  acting  as  a 
prompt  for  each  student  to  offer  explanatory  comment  on  their  actions  and  behaviours  in  using  the  LPS,  over 
their  whole  period  of  use,  and  particularly  addressing  the  notion  of  change  and/or  perceived  improvements  in 
skill  performance. 

Further,  each  of  the  six  lesson  plans  produced  was  evaluated  by  an  expert  lesson  planner  (ie.  lecturer  or 
teacher),  as  a measure  of  product  quality;  and,  as  a means  of  gaining  an  indication  of  the  strength  of  transfer  in 
students’  learning  over  media,  these  lesson  plans  were  then  compared  to  a lesson  plan  produced  by  each  of  the 
students  by  ‘pen  & paper’  means,  following  their  use  of  the  LPS. 

RESULTS 

Taken  together,  there  emerges  consistent  and  strong  patterns  in  the  data  (see  Table  1),  For  example,  the 
reduction  in  the  use  of  instructional  components  is  strong  (21-9,  or  57.1%),  although  less  so  if  the  fifth  and 
sixth  tasks  (#5,  #6)  are  discounted  as  unrepresentative  (for  which  only  two  students  provided  data),  (becoming 
28.6%).  Again,  the  reduction  in  the  time  taken  to  complete  the  lesson  planning  tasks,  (34-13  minutes,  or 
61.8%)  is  very  apparent — although,  again,  less  so  if  the  latter  two  tasks  are  removed  from  the  calculations 
(becoming  44.1%). 


Table  1:  Lesson  plans  1 — 6. 


Lesson  plans 

LPS  Components 

Time  (mins) 

Instruction  (I) 

Performance  (P) 

IP  Coefficient 

LI 

21 

20 

1.0 

34 

L2 

18 

19 

1.0 

25 

L3 

14 

20 

1.4 

20 

L4 

15 

21 

1.4 

19 

L5 

11 

17 

1.5 

17 

L6 

9 

14 

1.6 

13 

Perhaps  of  more  interest  are  the  figures  for  students’  use  of  performance  functions  in  the  LPS:  very  little 
change  from  one  task  to  another  over  the  first  four  tasks;  and  being  reduced  only  in  the  fifth  and  sixth  tasks 
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(20-14,  or  30%),  for  which  data  was  available  from  only  two  students.  However,  the  most  telling  figures  lie  in 
the  IP  coefficients:  the  trend  here  is  of  a gradual  and  positive  change,  rising  from  1.0  in  the  first  task,  to  1.4  in 
the  third  and  fourth,  and  then  peaking  at  1.6  in  the  sixth  task.  The  story  in  all  these  sets  of  figures  and 
particularly  those  for  the  IP  coefficients,  is  of  students  who  develop  their  cognitive  strategies  in  lesson  planning 
using  the  LPS  by  the  fourth  task,  and  then  refine  those  strategies  thereafter,  improving  upon  them  more 
gradually  over  the  remaining  two  tasks.  Indeed,  Figure  1 reveals  more  of  the  same  story,  suggesting  the 
interaction  in  the  instructional  and  performance  interactions  data  sets  that  occurs  just  before  the  second  task 
(#2),  is  the  point  at  which  the  development  in  expert  cognitive  strategies  first  takes  hold;  but  is  only  refined  and 
consolidated  at,  or  after,  the  fourth  task  (#4).  However,  as  a cautionary  note,  it  should  be  remembered  that 
generalising  in  this  way,  does  serve  to  mask  the  differences  in  the  sets  of  figures  recorded  and  analysed  for 
individual  students. 


L1  L2  L3  L4  L5  L6 
Lesson  Plans 


Figure  1:  Lesson  plans  1 — 6. 

The  temporal  data,  when  plotted  on  a single  graph  (Figure  2),  show  that  for  all  students,  the  time  taken  to 
complete  a lesson  planning  task  declined  over  the  entire  span  of  tasks  (#1— #6).  Also,  for  most  students  (ie. 
excepting  students  2 and  3),  this  pattern  of  reduction  is  very  similar:  starting  in  the  region  of  34-46  minutes  for 
the  first  lesson  planning  task;  falling  quite  rapidly  for  the  second  and  third  tasks;  and  then  declining  more 
gradually  for  the  remaining  tasks.  Of  the  two  students  (2,  3)  who  did  not  conform  to  this  pattern,  student  3 
completed  only  three  lesson  planning  tasks,  making  it  more  difficult  to  read  patterns  of  any  type  into  her  data; 
and  student  2 experienced  a hiatus  in  the  earlier  data  (where  for  tasks  #2  and  #4,  the  completion  time  rose  by 
one  to  two  minutes),  but  followed  the  broader  pattern  after  task  (#4).  The  data  for  all  students  increasingly 
converge  over  the  last  three  lesson  planning  tasks,  so  that  all  students  complete  the  final  task  in  approximately 
13  minutes. 


Figure  21  Time  taken  for  task  completion  in  the  LPS  for  lesson  plans  1 — 6. 

Although  there  are  similarities  in  both  the  interactions  and  the  task-time  data  between  the  five  students  in  their 
completion  of  the  submitted  lesson  planning  tasks,  there  are  also  some  significant  differences,  described  in  full 
in  Wild  (1998b).  Whilst  each  of  the  students  experienced  a reduction  in  the  use  of  instructional  components  in 
the  LPS  over  the  series  of  tasks  recorded,  together  with  a strongly  evident  and  corresponding  reduction  in  task 
completion  times,  the  patterns  in  their  usage  of  performance  functions  are  comparatively  less  alike — although, 
apart  from  two  students,  overall  there  are  reductions  in  these  interactions. 
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Beyond  this,  the  data  also  suggest  evidence  of  all  students  developing  appropriate  cognitive  strategies  for  lesson 
planning  using  the  LPS.  For  example,  the  data  for  tv^o  individual  students  (4,  5),  when  charted  as  line  graphs 
(not  shown  in  this  paper),  show  interactions  in  the  instructional  and  performance  components  interactivity 
data,  at  points  somewhere  between  the  second  and  third  lesson  planning  tasks.  It  is  at  these  points  that  the 
students  have  a ratio  in  the  use  of  instructional-performance  components,  of  one  (1),  and  where  their  use  of 
performance  components  begins  to  outstrip  their  use  of  instructional  components.  In  addition,  for  these  two 
students,  the  divergence  between  these  two  data  sets  that  follows  their  interaction,  suggests  that  such  patterns  in 
the  data  indicate  the  prevalence  of  cognitive  strategies  representative  of  expertise  in  the  task.  Even  for  those 
three  students  for  whom  there  is  no  interaction  in  the  instructional  and  performance  components  interactivity 
data,  there  is  a pronounced  divergence  between  these  data  sets  at  some  point  on  individual  students’ 
interactions  graphs  (again,  these  graphs  are  not  shown  in  this  paper),  a phenomenon  which  in  itself  is  perhaps 
enough  to  indicate  the  development  of  expertise  in  the  cognitive  strategics  being  deployed. 

Indeed,  Figure  1,  displaying  the  mean  average  in  the  interactions  data  for  all  students,  suggests  that  the  stage  at 
which  they  begin  to  develop  expertise  in  their  cognitive  strategies  when  using  the  LPS  for  planning  lessons,  (ie. 
the  point  at  which  there  is  an  interaction  in  the  instructional  and  performance  components  interactivity  data) 
occurs  very  early  on,  at  or  just  after  the  first  task.  However,  it  is  evident  that  the  phenomenon  of  the  interaction 
in  the  data  sets  must  be  read  in  context,  and  in  particular,  that  governing  the  nature  and  strength  of  the 
following  divergence  between  these  data  sets.  In  this  case,  when  all  students  are  taken  collectively,  as 
represented  in  Figure  1,  the  pattern  of  divergence  does  not  become  soundly  established  until  some  point  during 
or  after  the  fourth  lesson  planning  task  (#4) — it  is  at  this  point  on  the  graph,  that  the  divergence  pattern  in  the 
data  stabilises. 

INTERVIEWS 

Interview  data  allowed  for  the  richer  documentation  of  students’  cognitive  processes  and  also  increased 
accuracy  (reliability)  in  interpretations  offered  in  analysis  of  the  video  data.  Whilst  there  is  not  room  here  to 
provide  a detailed  account  of  the  analysis  of  each  student’s  interview,  it  was  clear  from  these  analyses  (Wild, 
1998b)  that  the  LPS  supported  a range  of  cognitive  strategies,  including  that  based  in  ‘value  adding’,  where 
there  was  clear  benefits  to  be  had  from  being  able  to  word  process  to  a template,  to  be  prompted  to  check 
aspects  such  as  lesson  evaluation,  and  also  to  be  able  to  provide  well-formatted  print-outs  of  lesson  plans. 
There  is  additional  evidence  that  students  came  to  see  their  use  of  the  LPS  as  a scaffold,  which  could  be 
removed  ‘once  you  know  what  to  do’  (quote  from  student  interview  data).  Furthermore,  most  students  clearly 
evolved  a confident,  reflective  and  critical  practice  in  their  lesson  planning,  as  part  of  the  process  of  using  the 
LPS:  this  is  illustrated  to  a limited  extent,  in  example  statements  for  two  students: 

After  a couple  of  times,  I really  thought  about  what  I was  doing,  about  how  I could  make  the  lessons 
better.  I used  one  of  the...  I copied  one  of  the  lessons  in  the  program.  Not  really  copied,  I suppose...  used 
it  to  get  ideas  from.  It  was  really  close  to  an  idea  I had  anyway,  of  using  cooking  in  science  to  get  the 
kids  to  think  about  heat,  the  effects  of  heating  water,  evaporation.  Once  I had  this  lesson  worked  out,  I 
used  it  as  the  foundation  for  my  other  lessons — objectives,  evaluations.  I started  a new  lesson  plan  each 
time,  but  I used  what  I did  before  to  speed  things  up. 


And: 


Yes,  I’m  thinking  about  the  evaluation  methods  to  use  (here).  I did  take  a lot  of  time,  more  time, 
thinking  about  what  I was  doing.  I think  I said  this  before.  I thought  more  about  what  I was  trying  to 
teach,  how  I was  teaching,  you  know,  individual  kids.  My  supervisor  asked  me  to  think  about  setting  up 
groups  for  maths.  This  is  what  I’m  doing.  I tried  to  find  out  about  groups  (pointing  to  the  screen).  This 
helped.  It  was  easy  when  I saw  how  to  set  out  a plan  for  working  with  groups,  so  I used  it  in  my  lesson 
(plan).  I can  see  I thought  about  what  I wanted  to  do  more.  Yeh,  OK.  When  I was  finishing  off,  I went 
through  the  lesson  again,  here,  like  the  computer  said  (when  using  the  Reflection  tool).  It  was  good.  It 
makes  you  think,  to  think  about  what  you’ve  doing,  how  things  fit  together.  Yeh,  (pointing  to  adding 
text  on  screen)  I need  to  think  about  some  of  the  special  children  in  the  class,  there  are  quite  a few 
really  intelligent  girls. 

For  all  students,  there  were  clear  indications  provided  across  the  interview  data  to  suggest  the  ‘LPS  experience’ 
was  a major  benefit,  in  both  supporting  lesson  plan  writing  directly,  and  in  providing  the  skills  and  in 
particular,  the  confidence,  to  tackle  lesson  planning  tasks  without  the  use  of  the  LPS.  Furthermore,  there  were 
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repeated  indications  in  most  interviews,  that  students  were  aware  of  the  strategies  they  developed  in  using  the 
LPS,  as  well  as  the  benefits  they  perceived  it  bestowed.  There  were  also  indications  of  metacognitive  skills 
being  developed,  of  students  consciously  thinking  of  their  own  role  in  the  lesson  planning  process,  and  of 
thinking  about  the  best  ways  of  working  with  the  LPS  to  produce  increasingly  better  lesson  plans. 

LESSON  PLAN  PRODUCTS 

All  lesson  plans  created  by  student-teachers  here  were  subject  to  grading  (F — A)  by  expert  reviewers  at  Edith 
Cowan  University.  For  use  in  providing  a graphical  representation  of  data  in  Figure  3,  below,  the  grades  were 
each  articulated  to  a numerical  equivalent  (ie.  a mark),  1 — 6 (see  Table  2). 

Table  2;  Articulation  of  lesson  plan  assessment:  outcome-grade-mark. 


Outcome 

Grade 

Mark 

Outstanding 

A 

6 

Outstanding 

B 

5 

Highly  Competent 

C 

4 

Highly  Competent 

D 

3 

Competent 

E 

2 

Unsatisfactory 

F 

1 

The  grades  for  each  of  the  five  students  over  the  series  of  lesson  plans  produced  by  use  of  the  LPS  and  that  were 
submitted  and  assessed,  show  an  improvement,  at  best,  of  two  grades  and  one  major  grade  category  (ie.  from 
Competent  to  Highly  Competent),  (students  1,  4);  and  at  worst,  of  no  improvement  at  all  (student  2).  However, 
not  all  students  experienced  increasingly  positive  results  in  this  respect,  over  the  entire  span  of  lesson  plans 
assessed — for  some  (students  2,  5),  their  grades  fluctuated  both  up  and  down,  at  different  junctures  in  this  span 
(see  Table  3). 

Table  3:  Lesson  plan  grades  [shaded  areas  indicate  lesson  plans  produced  by  means  of  ‘pen  & paper']. 


Lesson  plans 

Lesson  plan  grades 

Student  1 

Student  2 

Student  3 

Student  4 

Student  5 

Mean  (L5) 

LI 

2 

3 

2 

2 

3 

2.4 

L2 

3 

3 

3 

4 

2 

3.0 

L3 

4 

4 

3 

4 

4 

3.8 

L4 

4 

3 

> -1.  3 -b  ^ 

4 

4 

3.6 

L5 

4 

2 

#> 

3.2 

L6 

4 

3 

4.0 

L7 

4.0 

L8 

■ ; 

4.0 

L9 

4.0 

Mean  (L1-L9) 

3.9 

3.1 

3.2 

3.7 

3.3 

3.4 

St.  Dev. 

0.9 

0.6 

1.0 

0.8 

0.8 

0.6 

Figure  3:  Grades  for  lesson  plans  1 — 9 (where  1-6  corresponds  to  F-A). 


Any  improvement  in  grades  achieved  by  the  student-teachers  over  the  lesson  plans  produced  by  use  of  the  LPS, 
was  maintained  or  bettered  in  the  lesson  plans  subsequently  produced  by  ‘pen  & paper’  (see  Table  3,  and 
Figure  3).  In  the  case  of  two  students  (students  1,  3)  the  improvements  were  by  measures  of  one  major  grade 
category  (ie.  Highly  Competent  to  Outstanding).  Although  it  is  difficult  to  argue  a long-term  trend  or  measure 
of  learning  or  skill-performance  transfer  from  these  figures  alone,  there  is  clearly  an  inference  here  that  some 
level  of  transfer  did  take  place.  In  other  words,  removing  the  LPS  from  the  student-teachers  here,  did  not 
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hinder  or  impair  their  performance  in  the  lesson  planning  task;  and  it  is  likely,  these  students  transferred  their 
learning  and  their  level  of  skill-performance  in  the  task,  achieved  by  their  use  of  the  LPS,  to  the  same  type  of 
task,  without  the  use  of  the  LPS. 

CONCLUSION 

The  study  reported  here  illustrates  how  novice  students  used  the  LPS  in  a complex  task  (lesson  planning)  to 
reach  levels  of  expertise  in  the  completion  of  that  task;  and  further,  were  able  to  continue  to  call  on  this  newly 
acquired  skill  and  knowledge  base  to  complete  the  same  type  of  task  without  use  of  the  LPS.  Whilst  there  are  a 
range  of  cognitive  strategies  by  which  novice  students  developed  expertise  in  their  performance  of  lesson 
planning,  it  would  seem  that  all  students  here  benefited  in  particular  from  active  participation  in  an  holistic, 
complex  yet  supporting  environment,  organised  around  a single  performance  goal.  In  general  terms,  findings 
from  this  study  included  the  following: 

(i)  Learning  and  performance  in  students  was  transferred  substantively  across  media,  so  that  when  the  LPS  was 
removed  as  part  of  the  task  environment,  learning  and  skill-performance  in  the  same  task  continued. 

(ii)  Not  all  students  learnt  or  performed  effectively  in  a hypertext  environment;  and  at  least  one  student  appeared  to  be 
specifically  disadvantaged  by  the  embodiment  of  instructional  resources  in  a hypertext  or  hypermedia  format  in  the 
LPS,  as  a task-based  performance  environment. 

(iii)  At  least  one  student  in  this  study  was  hindered  in  achieving  meaningful  and  deep  learning,  by  the  design  of  the  task 
environment  in  the  LPS — where  learning  and  performance  was  intended  to  be  completed  simultaneously  or  at  very 
close  proximity,  (as  in  the  philosophy  of  just-in-time  learning  ). 

(iv)  Learning  in  a task-focused  environment,  such  as  that  provided  in  the  LPS,  did  not  appear  to  promote  cognitive 
strategies  in  students  that  were  primarily  guided  by  the  motivation  to  perform  the  task  better.  Students  were  just  as 
likely  to  form  strategies  guided  by  the  motivation  to  obtain  better  understanding  in  the  task. 

(v)  Students’  learning  styles  or  preferences  appeared  to  be  the  primary  factors  influencing  their  adoption  of  cognitive 
strategies  in  learning  and  performing  in  a task.  This  was  despite  the  fact  that  these  strategies  were  not  necessarily 
suited  or  optimised  to  the  cognitive  tools  available  for  use  in  the  LPS,  or  to  the  task-based  environment  in  which 
they  were  applied. 

(vi)  The  LPS  provided  strong  cognitive  support  to  novice  student-teachers  in  their  completion  of  lesson  planning  tasks. 

It  would  seem  that  whilst  PSSs  like  the  LPS,  are  well-placed  for  consideration  as  an  alternative  instructional 
model,  not  all  students  are  in  a position  to  make  the  best  use  of  them;  and  that  individual  learning  styles  are 
likely  to  be  a strong  determinant  of  students’  success  in  their  use. 
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Abstract:  We  examine  the  relationship  between  design  features,  patterns  of  use,  and  the  quality  of 
student  explanations  generated  in  response  to  inquiry  questions  in  a multi-faceted  hypermedia 
environment  ^^Seeing  Through  Chemistry.^'  We  see  that  some  design  features  appear  to  influence 
student  performance.  In  particular,  incorrect  answers  and  a model  response  seem  important  for  helping 
students  recognize  the  need  to  revise.  What  appears  to  be  lacking  are  features  which  lead  students  to 
improve  the  explanation  quality  as  they  revise. 


1 Introduction 

Hypermedia  systems  may  be  particularly  appropriate  as  learning  environments  because  the  associative 
knowledge  structures  of  a topic  or  subject  matter  are  made  explicit.  However,  in  creating  hypermedia  systems,  attention 
must  be  given  to  design  decisions  that  can  support  students  in  making  appropriate  connections  in  their  own  developing 
knowledge  structures  (e.g.,  McKendree  et  al  1995).  Activities  that  will  highlight  the  key  concepts  of  a domain  and 
engage  the  students  in  problem  solving  and  reflection  are  especially  important  in  this  regard.  Examples  include  guided 
tours  which  highlight  concepts  in  a domain  and  how  they  are  related,  multiple  possible  paths  through  materials  that 
encourage  consideration  of  the  same  topics  or  concepts  in  differing  contexts,  questions  that  promote  reflection  on  the 
relationships  among  topics  or  concepts,  and  opportunities  to  revise  one’s  response  as  understanding  develops. 

In  this  paper,  we  consider  the  impact  of  various  design  features  on  student  learning  in  a multi-faceted 
hypermedia  environment  ''Seeing  Through  Chemistry''  Psychological  principles  of  learning,  in  particular,  cognitive 
flexibility  theory  (Spiro  et  al.  1988)  and  principles  of  human-computer  interaction  (Dershimer  et  al.  1992)  provided  the 
theoretical  and  empirical  basis  for  designing  the  STC  learning  environment.  More  specifically,  STC  integrates  video, 
animations,  and  experimental  simulations  with  text  and  pictures  onto  multimedia  content  cards  to  facilitate  the 
development  of  students’  understanding  of  topics  in  an  introductory  college  chemistry  course  (Dershimer  & Rasmussen 
1990).  Four  interaction  modes,  distinguished  by  the  level  of  structure  and  support,  focus  students’  attention  on  the 
relationship  among  topics  in  each  of  the  six  modules  and  allow  for  multiple  paths  through  the  material  to  support 
individual  styles  and  interests. 

Inquiry  questions  form  the  primary  instructional  activity  in  STC.  The  questions  represent  a synthesis  of  topics 
or  content  cards,  and  the  same  topics  may  be  relevant  for  several  questions.  Revisiting  the  same  topics  (e.g., 
concentration)  in  multiple  contexts  (e.g.  pH,  precipitation)  promotes  students’  understanding  of  the  relationship  among 
related  concepts.  Students  are  asked  to  respond  to  the  inquiry  questions  with  an  answer  (select  from  among  two  or  more 
choices)  and  a written  explanation.  Answers  are  scored  right  or  wrong  and  then  students  are  shown  a model 
explanation.  The  software  prompts  students  to  revise  their  explanations  and  refer  to  content  cards  in  that  process. 
However,  students  are  not  required  to  revise  their  explanations  nor  does  the  software  evaluate  the  quality  of  their 
explanations.  Further,  students  make  decisions  on  how  the  software  is  to  be  used  - whether  and  when  to  read  the 
content  cards,  selection  and  use  of  the  four  navigation  modes,  and  whether  and  when  to  revise  their  explanations.  The 
student  makes  choices  in  interacting  with  the  software  and  it  is  this  interaction  that  effects  the  extent  of  learning,  more 
so  than  individual  design  features  in  the  abstract.  In  what  follows,  we  examine  variation  in  STC  modules  (content  cards 
and  inquiry  questions),  variation  in  student  patterns  of  use  and  their  relationship  to  students’  explanations  to  inquiry 
questions  (used  as  a proxy  for  student  learning  or  understanding).  We  conclude  with  software  design  implications. 
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2 Methods 


2.1  Study  Design 


Each  of  the  six  modules  in  STC  was  designed  with  the  same  fundamental  instructional  goal.  Nevertheless, 
characteristics  such  as  the  number  of  topics,  content  cards,  and  inquiry  questions,  and  the  nature  and  extent  of 
supporting  media  influence  the  options  students  have  to  interact  with  the  material.  We  reasoned  that  the  nature  and 
extent  of  opportunities  to  revisit  content  would  influence  student  learning.  To  this  end,  we  created  visualizations 
showing  which  content  cards  are  suggested  for  each  inquiry  question  in  each  module  and  selected  two  modules  that 
varied  widely  in  coverage  (see  Figure  1).  Acids  & Bases  has  6 inquiry  questions,  and  was  the  fourth  module  completed 
in  the  semester.  Solubility  has  10  questions,  and  was  the  fifth  and  final  module  completed. 
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Figure  1:  Coverage 'm  Acids  & Bases  and  Solubility  modules 


In  Figure  I,  bullets  (•)  indicate  when  a topic  (one  or  more  content  cards)  is  suggested  for  each  inquiry  question 
in  a module.  Reading  across  the  rows,  one  can  see  which  topics  are  suggested  for  which  questions.  For  example,  for 
Acids  & Bases  question  1,  STC  indicates  that  topics  2-4,  4-1,  4-2,  and  4-3  are  relevant  to  this  inquiry  question  (topics 
are  named  in  the  software,  but  names  have  been  ignored  for  the  purposes  of  this  discussion). 

Complete  coverage  occurs  when  each  column  has  at  least  one  bullet  (i.e.,  every  topic  or  content  card  is 
suggested  at  least  once).  Although  neither  module  has  complete  coverage.  Solubility  has  relatively  more  coverage  than 
Acids  & Bases.  The  columns  indicate  how  many  times  the  topic  is  visited;  more  bullets  indicate  the  topic  is  suggested 
for  more  questions  (i.e.,  repeat  coverage).  Overlap  is  determined  by  looking  at  the  pattern  of  bullets:  for  example.  Acids 
& Bases  question  5 covers  the  same  topics  as  question  2,  and  only  overlaps  any  other  question  with  one  topic  (3-2). 
Solubility  question  10,  on  the  other  hand,  overlaps  with  every  other  question  in  the  module.  Further,  the  Solubility 
module  appears  to  contain  distinct  subsets  of  topics:  questions  1 to  5 primarily  rely  on  topics  in  Clusters  2 and  3,  while 
questions  6 to  8 primarily  rely  on  Clusters  4 and  5.  Variety  in  overlap  is  important  for  revealing  different  facets  of  the 
topic;  e.g.  one  question  covers  saturation  and  concentration,  a second  refers  to  saturation  and  dynamic  equilibrium, 
while  a third  relates  saturation  and  solubility. 

Note  that  each  module  has  a different  number  of  inquiry  questions  and  a different  number  of  content  cards. 
Modules  with  more  inquiry  questions  provide  more  opportunities  to  visit  a given  topic:  if  there  are  10  questions,  a 
student  has  1 0 opportunities  to  visit  any  given  topic,  but  if  there  are  only  5 questions,  then  the  student  only  has  5 
opportunities  to  visit  a given  topic.  To  facilitate  cross-module  comparisons,  we  report  proportional  visits:  number  of 
potential  visits  per  content  card  per  question.  Across  the  six  modules,  the  Solubility  module  provides  the  highest 
number  of  proportional  visits  (0.276)  and  Acids  & Bases  the  lowest  (0. 1 77). 


2.2  Methods,  Procedures,  and  Measures 

Participants.  This  study  is  part  of  a larger  research  project  examining  the  use  of  Seeing  Through  Chemistry  in 
an  introductory  chemistry  course  at  a large  Midwestern  research  university.  Use  of  STC  was  required,  but  participation 
in  the  study  was  not:  20 1 of  the  450  students  enrolled  in  this  multi-section  lecture  course  volunteered  to  participate  in 
the  research  project  by  completing  additional  instruments  and  giving  researchers  access  to  course  and  background  data. 
Of  the  201  participants,  180  completed  the  modules  of  interest  in  this  study. 
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Context.  The  semester  these  data  were  collected,  all  students  used  the  software  regularly,  for  graded 
assignments.  Students  answered  the  inquiry  questions  for  five  of  the  six  modules  in  STC  and  turned  in  the  printed 
version  of  their  responses.  The  printout  shows  the  answer  choice  and  the  explanation  (initial  and  final).  Teaching 
assistants  checked  the  assignments  for  completeness.  Students  received  up  to  6 points  on  each  assignment  for  a 
maximum  possible  of  30  points  (5%  of  the  overall  course  grade). 

Log  Files  and  Inquiry  Question  Responses.  STC  automatically  captures  the  sequence  and  duration  of  choices 
made  by  the  student.  This  information  is  recorded  to  files  known  as  log  files,  one  per  student,  so  that  each  log  file 
captures  completely  a student’s  use  of  STC  throughout  the  semester.  The  log  file  also  captures  transitions  and 
navigation  modes,  as  well  as  all  answers  and  revisions  typed  in  response  to  questions. 

Explanation  Quality.  Because  STC  does  not  evaluate  the  quality  of  students’  explanations,  a post  hoc  scoring 
scheme  was  devised  for  this  puipose,  based  upon  student  response  data  and  the  content  in  the  model  response.  A 
student’s  explanation  is  assigned  one  of  five  scores,  determined  by  its  completeness,  coherence,  and  correctness  (cf. 
Chi,  Feltovich,  & Glaser  1981).  The  scores  are  0~lnadequate;  1-Fragmented;  2-Partial,  3-Good,  and  4-Elaborated.  For 
example,  a student  who  mentions  two  isolated  facts  but  doesn’t  relate  them  to  each  other  or  to  the  question  might  be 
given  a level  1 score  (Fragmented).  A student  who  mentions  all  the  relevant  concepts,  including  an  explanation  of 
“mechanism,”  and  who  adds  ideas  not  found  in  the  model  response  would  receive  a level  4 (Elaborated).  The  first 
author  scored  both  initial  and  final  explanations  to  the  inquiry  questions.  A second  rater  scored  a 25%  random  sample. 
Interrater  reliability  was  0.87  and  0.79  ^ox  Acids  & Bases  and  Solubility,  respectively. 

Patterns  of  Use,  A variety  of  software  tools  derive  patterns  of  use  from  the  participants’  log  files.  Moves  are 
extracted  from  the  log  files,  and  characterized  by  type  (e.g.  view  question  (q),  view  content  card  (c),  edit  response  (r)). 
The  sequence  of  moves  is  combined  and  then  abstracted  to  form  a pattern  label  that  is  assigned  to  each  question. 

Interviews  with  a target  group  of  students  (Jones,  Berger,  & Magnusson  1996)  revealed  two  aspects  to  patterns 
of  use:  base  patterns  and  revision  patterns.  Base  patterns  are  characterized  in  two  ways:  students  who  look  at  cards 
before  they  read  the  question  and  students  who  look  at  cards  after  reading  the  question  but  before  entering  their 
response.  Revision  patterns  consider  whether  students  revise  their  explanation  and  whether  students  refer  to  cards 
between  initial  and  final  revision.  The  possible  combinations  of  base  and  revision  patterns  are  shown  in  Table  1.  Base 
and  revision  patterns  of  use  were  identified  for  each  of  the  16  inquiry  questions  for  each  of  the  180  students. 


Revision  Patterns 

do  not  refer  after  initial  response 

refer  to  cards  after  initial  response 

Base  Patterns 

no  revisions 

revisions 

no  revisions 

revisions 

qr:  read  question,  respond 

^Hnit 

q^iniftfinal 

q^initc 

qnnitcrfinal 

cqr:  read  content  cards,  read  question,  respond 

cqrinit 

cqrinitrfinal 

cqrinit^ 

cqrinitcrfinal 

qcr:  read  question,  read  content  cards,  respond 

qcrinit 

qcrjnitrfinal 

qcrjnitc 

qcrinitcrfinal 

cqcr:  read  content  cards,  read  question,  read 

content  cards,  respond 

cqcrinit 

cqcrinitrfinal 

cqcrinitc 

cqcrinitcrfinal 

Table  1:  Patterns  of  Use  in  Seeing  Through  Chemistry 

Intuitively,  we  expect  those  who  read  the  content  cards  to  write  better  explanations.  On  the  other  hand, 
perhaps  some  students  do  not  read  the  cards  prior  to  responding  if  they  think  they  already  know  the  material.  They  may 
intend  for  the  exercise  to  serve  as  a “self  check”  on  their  understanding;  if  they  determine  they  don’t  understand,  they 
will  read  further.  Students  may  read  the  cards  before  they  answer  the  question,  or  after  they  read  the  questions  but 
before  responding.  The  former  is  like  reading  the  chapter  before  doing  the  exercises;  the  latter  is  like  pre-reading 
questions  on  a reading  comprehension  test.  Both  are  legitimate  strategies,  but  one  may  be  more  effective  in  this  setting. 


3 Results 

We  believe  that  the  design  of  instructional  software  can  greatly  influence  the  learning  that  occurs,  through  the 
nature  and  quality  of  the  interaction  that  is  supported.  Because  the  design  of  the  Solubility  module  is  more  robust  and 
interconnected  with  respect  to  topic  coverage  than  is  Acids  & Bases,  we  expected  students  to  provide  higher  quality 
explanations  for  that  module.  In  analyzing  student  responses,  we  first  considered  the  answer  choices  and  the 
explanation  quality.  We  further  examined  these  measures  in  light  of  the  process  students  undergo  while  answering 
inquiry  questions,  both  what  they  do  before  making  their  initial  response  (base  patterns  of  use)  and  what  they  do  while 
working  towards  their  final  revision  (revision  patterns  of  use). 
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3.1  Answer  Choices  and  Explanation  Quality 


In  analyzing  student  responses,  we  first  considered  answer  choices.  In  general,  the  answer  choices  to  inquiry 
questions  were  relatively  easy  for  students:  for  4 of  the  6 questions  in  Acids  &,  Bases,  80%  or  more  of  the  students 
made  the  correct  choice.  The  most  difficult  question  in  this  module  was  Question  4,  with  only  20%  initially  correct. 
The  Solubility  module  is  similar  in  the  sense  that  the  questions  were  relatively  easy  for  students  to  get  correct  on  their 
first  response.  However,  there  was  less  variation  in  the  difficulty  of  the  inquiry  questions.  For  all  of  the  Solubility 
questions,  55%  or  more  of  the  students  answered  correctly,  and  for  7 of  the  10  Solubility  questions,  80%  or  more  of  the 
students  made  the  correct  choice  initially.  Furthermore,  in  Acids  & Bases,  146  students  (^81%)  got  more  than  half  the 
questions  initially  correct,  while  in  Solubility,  1 74  students  (96%)  got  more  than  half  the  questions  initially  correct. 


Acids  & Bases  Module  (6  questions)  Solubility  Module  (10  questions) 

Answer  Explanation  Score  Totals  Answer  Explanation  Score  Totals 

Correctness  (max  possible  = 24)  Correctness  (max  possible  = 40) 


Response 

min 

max 

M 

SD 

min 

max 

min 

max 

M 

SD 

min 

max 

Initial 

20% 

95% 

7.34 

(3.97) 

0 

17 

57% 

97% 

12.01 

(6.13) 

0 

34 

Final 

100% 

100% 

12.84 

(3.68) 

4 

21 

100% 

100% 

19.06 

(6.44) 

6 

36 

Table  2:  Answer  and  Explanation  Scores  (n=180) 


Explanations  were  scored  on  a 4-point  scale.  As  shown  in  Table  2,  in  the  Acids  & Bases  module,  students 
average  around  1 (fragmented)  point  per  question  for  initial  responses  and  move  to  a 2 (partial).  Likewise  for  the 
Solubility  module,  students  improve  from  an  average  score  just  above  1 to  just  below  2.  In  both  modules  we  see  that 
students  generally  select  the  correct  answer  choice  but  they  do  not  provide  high  quality  explanations  for  their  choices. 
Moreover,  students  do  revise  their  explanations  and  the  change  in  the  quality  of  explanations  is  statistically  significant 
(t(df=179)=18.65***  and  t(df=179)=15.40***  for  AB  and  Sol,  respectively)  but  the  magnitude  of  the  difference  is 
small.  Furthermore,  contrary  to  expectations,  there  were  no  obvious  differences  between  the  two  modules  in  overall 
quality. 


3.2  Base  Patterns  of  Use 

In  examining  students’  patterns  of  use  we  asked:  do  patterns  of  use  affect  explanation  quality?  To  answer  this 
question  we  characterize  the  various  patterns  of  use  and  the  consistency  with  which  students  use  them  within  and 
between  modules.  In  Acids  & Bases,  1 19  students  use  only  one  of  the  four  base  patterns,  a further  31  use  one  pattern 
for  five  of  the  six  questions;  therefore,  150  students  (83.3%)  use  a single  pattern  almost  exclusively.  Similarly,  in 
Solubility,  139  students  use  one  pattern  exclusively,  while  19  more  use  one  pattern  for  all  but  one  question.  So  158 
(88%)  use  one  pattern  almost  exclusively.  Furthermore,  only  20  students  (1 1%)  switched  from  a ‘cards  first’  (cqr  or 
cqcr)  pattern  to  a ‘questions  first’  (qr  or  qcr)  pattern  between  the  two  modules. 

In  both  modules,  the  qcr  pattern  is  used  by  almost  half  the  1 80  students  on  each  question,  but  the  distribution 
of  students  amongst  the  other  patterns  varies  considerably  across  questions.  Furthermore,  students  are  either  in  a ‘cards 
first’  group  (cqr  and  cqcr)  or  a ‘questions  first’  group  (qr  and  qcr).  However,  the  distribution  of  students  within  these 
pairs  changes  through  the  module.  Finally,  as  the  module  progresses,  students  tend  to  refer  to  cards  between  the 
question  and  the  response  (e.g.  the  size  of  the  qr  group  decreases  while  qcr  increases,  likewise  for  cqr  and  cqcr). 


quest  ion 

qr 

M (sd) 

cqr 

M (sd) 

qcr 

M (sd) 

cqcr 

M (sd) 

sample 
M (sd) 

F 

group  differences 

ABql 

0.98  (1.11) 

1.87 

(M9) 

1.81 

(1.03) 

2.24 

(1.37) 

1.65 

(1.21) 

8.47* 

qr  < cqr,  qcr,  cqcr 

Sol  q2 

1.02  (1.22) 

1.66 

(1-26) 

0.62 

(1.02) 

1.73 

(1.32) 

1.12 

(1.25) 

9.58* 

qcr  < cqr,  cqcr 

Sol  q5 

0.83  (1.29) 

1.59 

(1.30) 

0.96 

(1.06) 

1.66 

(1.14) 

1.18 

(1.22) 

4.99* 

Sol  q7 

0.87  (0.97) 

2.06 

(0.89) 

1.70 

(0.90) 

1.85 

(0.94) 

1.69 

(0.97) 

8.43* 

qr  < cqr,  qcr,  cqcr 

Sol  qlO 

0.43  (0.99) 

1.24 

(1-16) 

0.90 

(0.86) 

1.17 

(0.95) 

0.96 

(0.98) 

3.95  * 

qr  < cqcr,  cqr 

*p<.003  (.05/  16) 

Table  3:  Initial  Explanation  Quality  Score  by  Base  Pattern  Group 
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In  order  to  examine  effectiveness  of  strategies,  we  examined  mean  scores  within  the  base  pattern  groups. 
Initial  explanation  quality  score  means  and  standard  deviations  are  shown  in  Table  3 (only  significant  results  are 
shown,  due  to  space  restrictions).  On  each  of  the  16  questions,  we  conducted  a within-subjects  (base  pattern  group:  qr 
vs.  cqr  vs.  qcr  vs.  cqcr)  ANOVA.  Base  pattern  group  effects  were  significant  for  5 of  the  16  questions.  For  4 of  these  5, 
significant  group  mean  differences  were  found.  For  two  questions  (ABl,  Sol  7),  the  qr  group  had  lower  mean  scores 
than  all  other  groups.  For  a third  question  (Sol  1 0),  the  qr  group  had  lower  scores  than  the  two  cards-first  groups  (cqcr 
and  cqr).  Consistent  with  our  expectations,  these  results  suggest  that  patterns  of  use  are  related  to  performance. 
Students  who  read  cards  (either  before  the  question  or  between  the  question  and  response)  generate  better  quality 
explanations  than  students  who  don’t  read  cards  prior  to  writing  their  response. 


3.3  Revision  Patterns 

Students  are  asked  to  respond  to  each  inquiry  question  by  selecting  an  answer  from  among  the  choices  given 
and  generating  an  explanation  to  support  their  choice.  Recall  that  students  can  then  revise  their  explanations.  Do  they 
use  this  opportunity?  What  features  might  influence  their  decision?  Do  their  revisions  lead  to  improvement? 

We  examined  whether  students  consistently  applied  revision  patterns.  In  Acids  & Bases,  20  students  only  use 
one  of  the  four  revision  patterns,  a further  28  students  use  one  pattern  all  but  once;  therefore,  only  48  students  (26.7%) 
use  a single  pattern  almost  exclusively.  Similarly  for  Solubility,  10  students  use  one  pattern  exclusively,  while  25  more 
use  one  pattern  for  all  but  one  question.  Thus,  only  35  students  (19.4%)  use  a pattern  almost  exclusively  in  Solubility, 
Contrast  this  with  the  base  patterns  of  use  where  the  values  were  83%  and  88%.  Base  patterns  are  fairly  consistent  and 
revision  patterns  are  less  so.  This  suggests  that  the  base  pattern  is  perhaps  more  characteristic  of  the  student,  while  the 
tendency  to  revise  or  refer  is  more  a reflection  on  the  design  features  of  the  software. 

What  software  features  might  influence  the  quality  or  the  revision  process  itself?  We  reasoned  that  feedback 
on  performance  would  be  an  important  determinant  of  students’  performance.  To  examine  this,  we  looked  at  the 
percentages  who  revise,  and  whether  those  revisions  improve  the  explanation  quality.  Students  are  given  feedback  as  to 
the  correctness  of  their  answer,  but  they  are  not  given  explicit  feedback  about  the  quality  of  their  explanation. 
However,  if  their  answer  is  correct,  they  are  shown  a “model”  response  or  explanation  to  which  they  can  compare  their 
explanation  as  an  indicator  of  quality.  If  the  answer  is  incorrect,  students  are  prompted  to  review  relevant  content  cards; 
a “model”  explanation  is  not  shown.  Students  then  made  a decision — not  imposed  by  the  software — about  whether  and 
how  much  to  revise  their  explanation,  and  whether  to  refer  to  cards. 

33 A Answer  Correctness  and  Tendency  to  Revise 

We  expected  students  to  revise  their  explanations  when  they  provided  an  incorrect  answer  and  as  expected  the 
questions  that  were  the  most  difficult  (i.e.,  the  largest  number  of  incorrect  answers)  had  the  highest  number  of 
explanation  revisions.  Virtually  all  students  with  a wrong  answer  revised  their  explanation.  However,  we  also  found 
that  20  to  60%  of  the  students  with  a correct  answer  (depending  on  the  question)  revised  their  explanations.  This 
finding  suggests  that  students  use  criteria  other  than  correctness  of  answer  to  judge  whether  to  revise  their  explanation. 

5.5.2  Model  Response  and  Explanation  Quality 

One  possibility  is  that  students  compare  their  explanation  to  the  model  response.  As  it  turns  out,  one  of  the 
inquiry  questions  did  not  have  a model  response.  An  examination  of  responses  to  Sol  q3  shows  that  the  question  was 
slightly  more  difficult  than  average;  80%  of  the  students  answered  correctly.  While  all  students  with  incorrect  answers 
revised,  only  20%  of  the  students  with  correct  answers  revised.  This  is  the  lowest  percentage  of  all  the  questions. 

Why  do  students  revise?  Perhaps  their  explanations  are  quite  bad,  and  their  revisions  only  bring  them  up  to  the 
quality  of  those  who  don’t  revise.  Furthermore,  revising  does  not  guarantee  improvement;  however,  the  data  reveal  that 
revisers  tend  to  improve  the  quality  of  their  explanations,  although  slightly  less  so  in  Solubility  than  in  Acids  & Bases 
(75%  and  80%  for  Solubility  and  Acids  &.  Bases,  respectively).  If  we  examine  only  those  students  who  revise,  the 
scores  for  explanations  increase  on  average  by  about  1.5  points  (on  a 4-point  scale).  Their  final  scores  are  typically 
between  2 and  2.5,  which  is  higher  than  the  overall  mean. 

In  both  modules,  there  is  one  question  where  improvement  is  much  less  common.  For  Solubility  question  3, 
only  42%  of  the  revisers  received  a higher  score.  In  contrast,  72%  or  more  of  the  revisers  improved  their  scores  for  all 
other  questions  in  that  module.  Again,  Solubility  question  3 did  not  have  a model  response  - this  perhaps  suggests  that 
many  students  rely  on  the  model  response  in  their  revisions.  In  Acids  & Bases,  43%  of  the  revisers  on  question  5 do  not 
see  an  improvement.  The  reason  is  not  immediately  obvious,  but  it  might  be  that  students  were  unable  to  pick  out  the 
crucial  concepts  in  the  model  response  and  instead  latched  onto  surface  differences. 


4 Conclusions 


We  see  that  some  design  features  appear  to  influence  student  performance.  In  particular,  incorrect  answers  and 
the  model  response  seem  important  for  helping  students  recognize  the  need  to  revise.  What  appears  to  be  lacking  are 
features  which  lead  students  to  improve  their  explanation  quality  as  they  revise.  Furthermore,  differences  in  coverage 
do  not  appear  to  influence  explanation  quality  or  revisions.  Perhaps  this  is  because  students  do  not  refer  to  much 
content  in  the  first  place,  so  any  potential  benefit  from  a more  well-connected  module  is  mitigated. 

According  to  the  STC  designers,  revision  should  be  a key  process  when  students  answer  the  inquiry  questions 
(Dershimer  et  al.  1992).  Substantial  revision  in  the  sense  of  improved  explanations  is  not  taking  place.  Why  not?  We 
see  that  students  revise  when  the  inquiry  question  choice  is  hard  (when  they  choose  the  wrong  answer).  Perhaps  the 
questions  need  to  be  harder  or  trickier.  But  also,  the  nature  of  the  interaction  - merely  expecting  students  to  revise  - 
does  not  seem  to  encourage  engagement.  Students  need  feedback  on  the  quality  of  their  explanations.  A different 
response  model  can  be  seen  in  the  Evolutionary  Thematic  Explorer  (Jacobson  1997).  In  that  system,  students  choose 
statements  from  a list  and  then  have  their  answers  critiqued  with  respect  to  two  major  theories  of  evolution. 
Experimental  results  show  learning  occurred.  Perhaps  similar  activities  here  would  help:  students  could  assemble  an 
explanation  from  sets  of  statements  that  either  capture  key  concepts  or  highlight  common  misconceptions. 

Furthermore,  students  need  an  incentive  to  revise.  While  the  inquiry  questions  were  used  here  as  the  primary 
instructional  activity,  they  are  in  one  sense  just  another  navigation  mode.  Students  are  supposed  to  choose  the  mode 
that  best  helps  them  learn.  Discussions  with  the  primary  designer  indicate  that  reliance  on  inquiry  questions  might  best 
be  used  in  a classroom  where  considerable  discussion  takes  place  before  the  final  explanation  is  written  (Dershimer 
1996).  At  a minimum,  if  these  questions  are  to  be  used  as  an  assignment,  the  graders  need  to  pay  attention  to  whether 
or  not  and  what  kinds  of  revisions  students  make.  Without  the  external  motivation  of  a grade  that  takes  explanations 
into  account,  many  students  will  not  make  the  effort  to  learn  the  material  or  to  express  their  understanding. 

Our  experience  identifying  the  extent  of  criss-crossing  would  be  a useful  exercise  for  hypermedia  designers. 
Visualizations  such  as  we  created  can  help  identify  the  gaps  in  coverage  at  the  design  stage,  rather  than  in  a later 
analysis.  When  we  began  this  study,  we  thought  “criss-crossing”  referred  to  the  content  design;  are  the  content  cards 
related  and  cross-linked?  do  the  inquiry  questions  lead  students  to  visit  all  the  cards?  But  what  we  see  instead  is  that  the 
design  of  the  questions  itself  - are  they  hard?  do  they  reveal  common  misconceptions?  - seems  to  have  considerable 
influence  over  whether  or  not  students  make  substantial  changes  in  their  explanations. 
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Abstract:  A component  framework  for  implementing  Web-based  learning  activities, 
customisable  for  local  discipline  and  teaching  requirements,  is  proposed.  The  framework  is  based 
on  a standard  protocol  for  describing  and  recording  the  state  of  Web  pages  containing  arbitrary 
arrangements  of  HTML  input  elements  and/or  interactive  objects  constructed  in  JavaScript, 
Shockwave  or  Java.  The  State  Description  Protocol  lays  the  foundations  for  independent 
development  and  reuse  of  software  components  with  which  learner  activity  can  be  recorded, 
analysed  and  re-created.  Learning  activity  pages  may  contain  embedded  references  to  previous 
tasks  and  provide  conditional  feedback  to  student  input  by  way  of  additional  components.  The 
framework  philosophy  is  to  provide  flexibility  in  educational  design,  multiple  paths  to  adoption 
and  to  maximise  collaborative  sharing  and  reuse  of  both  components  & pedagogical  techniques. 


Background 

The  Multimedia  Education  Unit  is  a central  academic  unit  supporting  the  University  in  its  move  to  integrate 
technologies  into  ’transformed'  curricula.  It  provides  educational  design,  evaluation,  media  & programming 
services  and  runs  staff  development  programs.  The  Unit  is  involved  in  the  development  of  a range  of  faculty 
projects  where  local  academic  knowledge  is  the  key  to  understanding  learning  and  discipline  requirements.  The 
Unit  has  a central  role  in  recognising,  developing  and  promoting  techniques  that  will  facilitate  effective  utilisation 
and  transfer  of  this  expertise  throughout  the  institution  and  further  afield. 

Educational  technology  is  increasingly  being  focussed  on  the  more  open,  flexible  and  apparently  democratic  world 
of  the  Internet.  This  move  is  not  without  its  difficulties,  given  the  wide  range  of  possible  solutions,  the  rapid 
evolution  of  technologies  and  challenges  to  individuals  adjusting  to  new  skills  and  roles.  Finding  a path  through  a 
complex  and  dynamic  solution  space  involves  broad  consideration  of  educational,  organisational,  technical  and 
individual  needs.  Integrated  course  systems  such  as  WebCT  and  TopClass  provide  useful  tools  and  templates  for 
efficient  management  of  learning  and  production  of  materials.  In  the  process  however,  they  also  restrict  flexibility 
of  approach,  for  example  in  the  use  of  customised  discipline  interfaces  or  in  the  richness  of  conditional  feedback. 
The  model  proposed  is  intended  to  supplement  the  capacities  of  such  integrated  systems. 


Customised  Learning  Activity  Scenarios 

We  have  tried  to  address  the  need  for  high  quality  interactive  learning  while  ensuring  that  both  software  and 
pedagogical  techniques  are  scalable  and  transferable.  Educational  requirements  targeted  include  the  capacity  to 
support: 

• customisable  user  interfaces  to  discipline  knowledge; 

• responsive  feedback  to  a student's  actions; 
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• exercises  involving  with  subjective  knowledge', 

• customisable  learning  sequences  that  can  incorporate  elements  of face  to  face  interaction:  and 

• reflective  summaries  and  overviews  of  the  learning  experience  for  both  student  and  teacher. 

Such  requirements  are  derived  from  the  experiences  gained  working  with  faculty  teaching  staff  on  real  course 
projects.  For  example,  a structured  learning  sequence  might  be  proposed  by  a teacher  as  a useful  exercise  for 
students  in  a particular  content  area.  Scenarios  such  as  this  from  different  discipline  areas  are  of  prime  relevance 
to  the  formation  of  a generic  implementation  framework.  More  detailed  ones  have  been  described  (Fritze, 
Johnston  & Kemm  1998).  It  is  the  teacher's  understanding  of  student,  content  and  teaching  strategy,  predating 
technical  implementation  considerations,  that  we  are  keen  to  harness. 

This  scenario  illustrates  how  a simple  sequence  of  learning  actions  might  take  place  between  a student  (S)  and 
mentor  (M).  It  represents  one  approach  of  a teacher  in  a face  to  face  tutorial,  seeking  to  encourage  reflective 
thinking. 

1)  S is  presented  with  an  issue  and  asked  to  articulate  an  initial  interpretation 

2)  M suggests  one  relevant  factor  and  asks  S assess  their  own  response  to  that  criterion. 

3)  M provides  feedback  based  on  this  assessment  (2  & 3 are  repeated  with  different  criteria) 

4)  M then  recalls  the  initial  view  and  asks  S to  re-examine  this  initial  interpretation. 

5)  S receives  a summary  overview  of  key  points,  demonstrating  how  their  initial  interpretation  has  developed. 

The  learning  sequence  is  expanded  (Fig.  1).  If  such  an  activity  were  to  be  facilitated,  not  by  face  to  face 
communication,  but  by  an  online  computer,  simple  text  entry  fields  and  check  buttons  are  probably  adequate  for 
articulation  of  responses  la,  2a  & 4a.  A sequence  of  disconnected  questions  however,  would  not  be  sufficient  to 
capture  the  full  spirit  of  the  dialogue.  For  example,  analysis  of  the  response  2a  is  necessary  for  useful  feedback 
(3).  In  addition,  the  initial  hypothesis  is  recalled  in  later  tasks  (2  & 4)  and  again  in  the  summaiy  overview  (5). 
Knowledge  of  the  student's  actual  efforts  needs  to  be  maintained  for  the  life  of  the  activity  and  could  possibly  be 
referred  to  in  entirely  different  contexts  in  other  areas  of  a course  (6). 


1 . Issu  e 2 . iQleria  (rep  eate  (J  4 . ^flection  5 . Gfewiew 


Figure  1:  Stages  of  a reflective  learning  sequence  1-5.  Student  entries  la,  2a,  4a  are  re-introduced  to  later  tasks 

This  example  represents  a relatively  simple  face  to  face  dialogue  interaction  to  which  many  variations  and 
extensions  possible.  We  desire  to  construct  such  sequences  within  a practical  online  environment. 


A Component  Framework  to  Support  Customised  Learning  Interactions 

A component  architecture  to  facilitate  synthesis  of  online  learning  scenarios  of  the  type  above  is  illustrated  (Fig. 
2).  This  has  evolved  from  an  earlier  work  (Fritze  & Ip  1998)  in  which  a central  'activity  manager'  object  running 
in  the  browser  controlled  the  entire  process.  The  new  modular  architecture  has  enabled  the  integration  of 
components  and  techniques  from  other  projects,  for  example  the  Virtual  Apparatus  framework  (Ip  et  al.  1997)  and 
Component-based  Object  Library  (Ji  et  al.  1998).  Web  pages  can  now  be  light  in  weight,  with  simpler,  easier  to 
maintain  software.  Additional  components  can  be  created  in  whatever  technology  is  appropriate. 
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Figure  2:  Learning  Engines  components  and  protocol  framework. 

A browser  page  contains  visible  user  interface  components  such  as  forms  elements  and  interactive  Java  or 
Shockwave  objects.  These  are  connected  by  a common  State  Description  Protocol  representing  the  state  of  the 
page  as  described  below.  SDP  conforming  components  are  supported  by  a 'glue'  script  and  linked  to  a database  via 
other  components.  Additional  functions  can  be  added  to  provide  specialised  feedback  or  management  of  page 
sequences,  for  example.  Various  page  configurations  are  indicated  (Tab.  1)  with  those  required  in  the  described 
learning  sequence  (Fig.  1)  numbered. 


Page  Style 

SDP  glue 

Interactive 
I/F  object 

DB/ID 

connect. 

Feedback 

Comp. 

Embedded 
SDP  refs 

Notes 

Resource  page 

R 

o 

Page  containing  HTML  form  elements  and/or 
interactive  objects  can  be  set  to  particular  states  on 
loading  or  by  clicking  different  HTML  buttons. 

Basic  saved  page 
(1) 

R 

o 

R 

Page  state  is  saved  to  and  retrieved  from  a 
corresponding  record  in  database. 

Conditional 
feedback  (2) 

R 

o 

o 

R 

A feedback  component  analyses  and  responds  to  page 
state  according  to  criteria  statements. 

Embedded  SDP 
References  (2,  5) 

o 

o 

o 

o 

R 

Page  containing  embedded  references  to  SDP  records  of 
any  activity.  Rendered  by  a loader  component  as  a 
summary  document,  activity  map,  progress  log  etc. 

Redrafting  task 

J4) 

R 

o 

R 

o 

R 

Previously  submitted  SDP  entry  inserted  into  a new  task 
for  redrafting. 

Table  1:  Possible  configurations  of  page  components  for  different  learning  purposes.  Each  page  requires  certain 
components  (R),  others  are  optional  (o).  Numbers  in  brackets  refer  to  page  structures  used  in  figure  1 . 


Interface  Components 

In  addition  to  HTML  text  fields,  radio  buttons,  checkboxes,  popup  menus  , framework  Web  pages  may  contain 
interactive  objects  developed  in  JavaScript,  Shockwave  or  Java.  For  example,  a large  number  of  specialised 
components  have  been  written  in  dynamic  HTML  for  an  online  course  in  Chinese  language  (Ji  et  al.  1998).  A 
Shockwave  object  for  graph  sketching  is  illustrated  here  (Fig.  3)  and  others  have  been  described  (Fritze  & 
McTigue  1 997^  The  essential  characteristics  for  such  interactive  objects  are  that  they: 


o 
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operate  within  a browser  page  supported  by  basic  'glue’  scripts; 

register  their  current  State  Description  to  the  controlling  page  in  response  to  student  actions;  and 
can  configure  themselves  to  a certain  State  Description  if  instructed  by  the  controlling  page. 


Figure  3:  A Shockwave  graphing  object  and  assessment  component  combined  on  framework  page  to  provide 
conditional  feedback  based  on  the  State  Description  of  an  entered  graph. 


Representation  of  State  - the  State  Description  Protocol 

The  central  element  of  the  Learning  Engines  framework  is  the  ability  to  record  a snapshot  of  a learning  activity 
page  containing  any  combination  of  forms  elements  or  conforming  external  interface  objects.  This  is  specified  in 
terms  of  a State  Description  Protocol  (SDP)  that  can  be  used  to  record  the  student  entries,  restore  previous  page 
states  and  for  analysis  of  students  input.  The  SDP  is  a 'list  of  lists'  consisting  of  property/value  pairs,  where  a 
value  may  be  a string,  a number  or  another  list  as  in  the  following  form: 

{propLvaluel,  prop2:  {subpropl:value3,  subprop2:value4  }} 

The  SDP  syntax  relates  closely  to  object  structure  representations  in  both  JavaScript  and  Director  Lingo  as  well  as 
to  the  pedagogical  learning  elements.  Not  immediately  obvious  is  that  the  SDP  string  format  requires  no  special 
database  configuration  for  different  page  configurations  and  rich  descriptions  of  the  customised  pages  and 
embedded  objects  can  be  readily  defined.  For  example,  the  SDP  for  the  above  page  (Fig.  3)  containing  a single 
’graph*  object  set  with  one  curve  supplied  by  the  question  ('supply')  and  another  entered  by  the  student  (’answer*) 
is; 


{graph: {supply: {style: "growthline”,  Xl:20,  Yl:20,  thl:40,  th2:220,  X2:80,  Y2:70}, 
answer;{style:’’growthline",  Xl:10,  Yl:40,  thl:45,  th2:225,  X2:60,  Y2:90}}} 


Assessment  and  Feedback  Components 

The  use  of  SDP  facilitates  analysis  of  a student’s  input  and  formulation  of  appropriate  responses.  The  SDP  string 
above  contains  detailed  information  with  which  to  interpret  the  curve  entered  by  the  student  (Fig.  3).  The  ‘Submit’ 
assessment  component  on  the  illustrated  page  (Fig.  3)  has  established  that  the  style  of  the  curve  'answer'  is  correct 
(graph.answer.style  = 'growthline'),  its  gradient  is  correct  (graph.answer.thl  = 45),  but  that  its  coordinates  place  it 
incorrectly. 
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A number  of  feedback  components  have  been  developed,  ranging  from  very  simple  JavaScript  functions  that 
compare  the  SDP  of  a page  with  a preset  value,  to  one  controlled  by  an  XML  criteria  specification  to 
progressively  reveal  feedback  comments  and  provide  access  to  further  tasks.  Further  feedback  mechanisms  can  be 
developed  as  required  for  particular  purposes. 


Embedded  SDP  References 

The  learning  sequence  (Fig.  1)  requires  that  page  states  be  carried  forward  into  other  pages.  This  is  achieved  by 
embedding  references  to  the  stored  SDPs  within  the  HTML  of  a page.  When  displayed  by  a loader  component, 
these  references  are  expanded  with  the  latest  state  information  from  the  corresponding  database  SDP  record.  Tn 
this  example  the  overview  page  (5)  refers  to  text  entered  as  the  initial  hypothesis  la,  as  well  as  to  the  self- 
assessments  2a  and  the  redrafted  entry  in  4a,  The  embedded  reference  could  be  contain  text,  HTML,  or  even  the 
state  of  an  interactive  object  such  as  the  curve  entered  in  a graph  sketcher  above.  This  capacity  to  refer  to  any 
element  of  work  submitted  makes  possible  a range  of  useful  learning  functions,  such  as  progress  reports,  entries  in 
learning  'journals',  summaries  of  key  points  or  dynamic  content  maps.  Such  a page  could  serve  multiple  purposes 
such  as  an  overview  of  the  issues  and  a navigational  menu  (Fig.  4), 


Figure  4:  A map  page  representing  the  progress  of  a student  as  a ‘jigsaw’.  Each  piece  is  an  embedded  reference  to 
the  SDP  of  an  exercise  page  and  is  filled  in  as  the  student  completes  the  activity.  The  pieces  contain  links  to  the 

relevant  activity. 


Conclusion 

We  have  attempted  to  relate  closely  to  coal-face  educational  requirements  in  building  the  foundations  of  a 
framework  for  customised  on-line  learning  interactions.  Such  requirements  relate  at  least  in  part  to  the  features  of 
face  to  face  dialogue.  Central  to  the  pedagogical  and  technical  operation  is  a protocol  for  memorising  and 
analysing  the  states  of  Web  pages  containing  an  arbitrary  arrangement  of  input  elements  or  interactive  objects. 
These  records  of  student  activity  are  accessible  from  any  other  page,  enabling  the  construction  of  simple  but 
useful  learning  structures. 

Since  the  initial  writing  of  this  paper,  the  Learning  Engines  framework  has  been  incorporated  into  OCCA,  an 
Online  Courseware  Component  Architecture  under  development  at  the  Multimedia  Education  Unit.  OCCA 
formally  brings  this  work  together  with  other  online  technologies  developed  at  MEU:  the  Virtual  Apparatus 
framework  (Ip  et  al.  1997)  and  Component-based  Object  Library  (Ji  et  al.  1998).  During  1999,  over  a dozen 
internally  funded  University  projects  will  use  this  architecture.  Work  in  MEU  will  concentrate  on  facilitating  the 
direct  involvement  of  Faculty  academics  in  the  production  of  materials.  This  will  involve  collation  of  exemplars  of 
interactive  question  configurations,  the  running  of  workshops  and  the  evaluation  of  techniques  developed. 

While  online  interactions  do  not  replace  face  to  face  dialogue,  they  can  usefully  supplement  and  enhance  other 
approaches  to  teaching  and  learning.  The  flexibility  offered  by  the  framework  can  empower  academics  to  think 


1221 


BEST  COPY  AVAILABLE 


beyond  what  might  be  offered  by  more  rigid  delivery  methods,  for  example,  by  integrated  courseware  systems  and 
traditional  lecture/tutorial  environments.  The  intention  is  to  encourage  conception  of  novel  learning  interfaces  and 
interactions  that  will  operate  in  conjunction  with  other  such  approaches.  It  is  hoped  that  the  uptake  of  OCCA  into 
the  University  and  wider  community  will  provide  a productive  environment  in  which  components  and  learning 
techniques  can  be  developed  and  shared. 
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Abstract:  This  paper  discusses  cultural  factors  in  user  acceptance  of  human-computer 
interfaces  and  presents  findings  from  the  first  phase  of  the  development  and  evaluation  of 
two  versions  of  a multimedia  program  '‘^Learn  About  PCs”  (British  version  and  Kuwaiti 
version). 

The  purpose  of  this  study  is  to  evaluate  users’  perceptions  and  opinions  about  two  versions 
of  Learn  About  PCs,  its  usability  and  suitability  to  the  users’  culture,  highlighting  some 
issues  which  designers  may  consider  when  designing  multimedia  programs  for  different 
cultures.  The  study  was  able  to  uncover  several  interesting  findings  and  to  make 
recommendations  which  are  useful  for  redesigning  the  two  versions  of  Learn  About  PCs  . 
The  overall  results  of  this  study  confirm  that  most  of  Kuwaiti  students  preferred  the  Kuwaiti 
version  of  the  multimedia  program  while  most  of  British  students,  British  experts,  and 
Kuwaiti  experts  preferred  the  British  version  of  the  program  compared  to  the  Kuwaiti  one. 


It  is  believed  that  culture  is  a discernible  variable  in  interface  acceptance,  and  that  interfaces  should  be 
designed  to  accommodate  users’  culture  (Del  Galdo  1996,  Fernandes  1995,  Uren  et  al  1993,  Nielsen  1990). 
Since  there  has  been  little  research  in  the  area  of  designing  multimedia  learning  programs  for  different 
cultures,  this  study  is  intended  to  underline  and  see  where  the  issues  lie  and  how  they  might  be  investigated. 

Two  versions  of  Learn  About  PCs,  a multimedia  learning  program,  have  been  designed,  one  for  British  and  one 
for  Kuwaiti  Higher  Education  (HE)  students.  They  carry  the  same  content  and  structure,  but  they  are  different 
in  the  way  that  media  elements  are  presented  on  the  screen.  The  two  versions  have  been  designed  and 
developed  not  only  to  look  at  how  the  interface  worked  and  how  successful  it  was,  but  also  to  examine  some 
presentation  and  screen  design  differences  which  accommodate  both  the  British  and  Kuwaiti  users.  The 
program  does  not  give  a detailed  knowledge  about  computers,  but  rather  provides  learners  with  all-round 
grounding  in  the  concepts  involved  which  serves  the  intended  outcome  of  the  study.  The  objectives  of  this  study 
are  as  follows: 

1-  To  identify  some  presentation  design  issues  when  developing  multimedia  programs  for  different  cultures. 


1.0  INTRODUCTION 


2-  To  evaluate  the  usability  and  the  suitability  of  the  two  versions  of  Learn  About  PCs  for  the  British  and 
Kuwaiti  HE  students,  and  to  provide  some  recommendations  for  future  designs. 

2.0  THE  DESIGN  OF  “LEARN  ABOUT PCs^^ 

Learn  About  PCs  was  designed  and  authored  using  Asymetrix  Multimedia  Toolbook  version  4.0.  The  process 
for  designing  the  program  were  identified  as  information  design  (defining  the  content  and  the  audience), 
interaction  design  (defining  the  navigation,  types  of  interaction,  and  control),  and  presentation  design  (defining 
the  style  and  the  layout  of  the  elements).  Theories  and  issues  which  contribute  to  an  effective  interface  design 
were  taken  into  account,  such  as:  ease  of  use,  consistency,  visual  representations,  etc.  The  prototype  of  Learn 
About  PCs  was  originally  designed  to  suit  British  HE  students.  The  design  intentionally  followed  a set  of 
theories  and  guidelines  outlined  in  the  British  literature  (Boyle  1997,  Vossen  et  al  1997,  McAtter  and  Shaw 
1995,  Davies  et  al  1994,  Leonard  1994,  Preece  1990). 

3.0  DESIGNING  THE  KUWAITI  VERSION 

In  order  to  achieve  the  objectives  of  this  study  as  outlined  in  the  introduction  part  of  this  paper,  the  initial 
program  (British  version)  was  redesigned  producing  a version  of  the  program  which  suits  Kuwaiti  HE  students 
(Kuwaiti  version).  Since  there  is  not  much  work  published  about  designing  multimedia  interfaces  for  Kuwaiti 
or  Arabic  cultures,  the  researchers  have  identified  some  presentation  design  differences  based  on  their 
investigations  and  their  knowledge  about  the  Kuwaiti  and  British  cultures.  Some  differences  were  identified  as: 
Buttons  and  icons,  Background  Colours,  Frames  and  Borders,  Graphics,  Music,  and  Speech.  Based  on  these 
differences.  Learn  About  PCs  was  redesigned  while  other  issues  such  as:  content  (in  English  language), 
structure,  functionality,  navigation  links,  and  screen  layout  were  kept  the  same. 

4.0  METHODOLOGY 

The  procedure  followed  in  the  evaluation  of  Learn  About  PCs  consisted  of  various  phases  which  have  been 
identified  by  (Catenazzi  et  al  1997).  These  phases  are:  Selecting  the  technique;  Preparing  the  evaluation: 
Conducting  the  evaluation;  and  Elaborating  data. 

4.1  SELECTING  THE  EVALUATION  TECHNIQUE 

The  first  step  of  Learn  About  PCs  evaluation  was  the  choice  of  the  most  appropriate  evaluation  technique, 
taking  into  account  the  stage  of  development  of  the  prototype  and  the  research  context  constraints.  It  was 
believed  that  the  most  suitable  way  for  evaluating  the  program  was  a limited  survey  technique,  in  which  a semi- 
structured  interview  was  used. 

4.2  PREPARING  THE  EVALUATION 

The  preparation  of  the  evaluation  involved  selecting  the  participants;  arranging  the  semi-structured  interview; 
and  defining  the  tasks  which  are  necessary  to  obtain  feedback  from  the  participants. 

4.2.1  PARTICIPANTS 

With  respect  to  the  selection  of  the  evaluators,  the  researchers  took  into  account  the  Learn  About  PCs 
evaluation  objectives,  that  is  to  verify  that  the  two  versions  of  the  program  are  suitable  for  the  users’  culture. 
Therefore,  two  groups  of  potential  users  were  required  (Experts  and  Students).  Seventeen  experts  (6  British  and 
II  Kuwaiti)  and  nine  HE  students  (4  British  and  5 Kuwaiti  HE  students). 

4.2.2  INTERVIEW 

The  semi-structured  interview  was  arranged  to  elicit  evaluators’  opinions  with  the  aim  of  assessing  the  usability 
and  suitability  of  the  two  versions.  The  interview  form  was  divided  into  four  parts:  Part  One  was  concerned 
with  participants’  personal  information;  Part  Two  was  concerned  with  common  design  issues  and  the  usability 


of  the  program;  Part  Three  sought  evaluators’  opinions  about  the  differences  between  the  two  versions  of  Learn 
About  PCs  and  their  suitability  to  their  culture;  and  Part  Four  was  concerned  with  evaluators  opinions  about 
the  preferred  version  and  their  general  comments  and  suggestions  for  improvements. 

4.23  TASKS 

With  respect  to  the  task  selection,  taking  into  account  the  objectives  of  the  study,  users  were  asked  to  navigate 
and  interact  with  the  two  versions  of  Learn  About  PCs  by  turning  pages,  activating  links,  playing  media,  etc. 
insuring  that  they  went  through  and  interacted  with  the  whole  program.  They  were  asked  to  observe  the 
differences  between  the  two  versions  which  were  identified  in  section  3.0.  It  took  about  one  hour  to  complete  all 
of  these  tasks.  It  was  considered  useful  to  have  their  comments  and  opinions  about  the  program’s  usability, 
utility,  and  suitability  directly,  by  asking  them  a number  of  questions  while  they  were  interacting  and  recording 
their  feedback  immediately  before  the  semi-structured  interview  which  was  planned  to  be  accomplished  after 
completing  the  tasks  at  the  end  of  each  session. 

4.3  CONDUCTING  THE  EVALUATION 

After  the  preparation  had  been  completed,  the  evaluation  process  started.  The  evaluation  took  place  both  in 
Kuwait  (Kuwaiti  experts  and  students),  and  in  Britain  (British  experts  and  students).  Subjects  were  informed 
that  they  are  not  being  tested,  rather  it  is  the  system  that  is  being  tested.  The  evaluation  execution  was 
conducted  individually,  that  is,  one  user  at  a time  in  order  to  follow  and  observe  them  during  their  interaction, 
stopping  them  when  needed,  asking  questions,  and  taking  notes  about  their  comments  and  suggestions.  The 
evaluation  process  lasted  between  one  hour  and  one  hour  and  a half  including  the  task  and  the  interview  at  the 
end  of  the  session.  The  evaluation  time  varied,  that  is  because  some  evaluators  were  interested  in  the  program, 
giving  more  comments  and  suggestions  and  raising  many  questions,  while  others,  (mainly  students),  were 
strictly  accomplishing  the  tasks,  and  finally  answering  the  interview  questions,  giving  some  comments  and 
recommendations. 

4.4  ELABORATING  DATA 

The  data  produced  were  elaborated,  in  order  to  transform  them  into  findings  and  recommendations  for 
improvement.  The  evaluation  results  are  reported  in  the  following  section. 

5.0  RESULTS 

Significant  differences  were  found  between  evaluators  opinions  about  the  two  versions  of  the  multimedia 
program.  The  results  of  the  evaluation  are  summarised  in  (Tab.  1)  through  (Tab.  8).  Note  that  the  scale  used  for 
(Tab.  1)  is  represented  by  (Scale:  5 = very  much  and  I = not  at  all). 


Questions 

Kuwaiti 

Students 

British 

Students 

Kuwaiti 

Experts 

British 

Experts 

Ave 

rage 

1-Are  buttons  and  icons  in  the  British  version  meaningful? 

3.8 

4.5 

4.1 

4.3 

4.2 

2-Are  buttons  and  icons  in  the  Kuwait  version  meaningful? 

2.9 

3.4 

2.7 

2.9 

3.0 

3-  Did  you  like  background  colours  in  British  version? 

3.8 

3.5 

3.4 

4.2 

3.7 

4-  Did  you  like  background  colours  in  Kuwaiti  version? 

3.6 

3.5 

3.3 

2.8 

3.3 

5-  Did  you  find  frames  and  borders  in  the  British  version  appealing? 

3.8 

3.8 

3.4 

4.0 

3.8 

6-Did  you  find  frames  and  borders  in  the  Kuwait  version  appealing? 

4.0 

3.5 

3.1 

2.8 

3.4 

7-  Did  graphics  and  images  in  the  British  version  reflect  British  culture? 

4.0 

3.3 

4.2 

4.0 

3.9 

8-  Did  graphics  and  images  in  the  Kuwaiti  version  reflect  Kuwaiti  culture? 

4.0 

3.8 

3.9 

4.0 

3.9 

9-  Did  you  notice  the  differences  in  “graphics”  in  the  two  versions? 

4.4 

3.8 

4.1 

4.3 

4.2 

10-  Is  music  appropriate  in  the  British  version? 

4.0 

3.3 

3.7 

3.0 

3.5 

1 1 - Is  music  appropriate  in  the  Kuwaiti  version? 

3.8 

3.3 

4.0 

3.5 

3.7 

1 2-  Is  speech  appropriate  in  the  British  version? 

4.2 

3.8 

4.3 

4.5 

4.2 

13-  Is  speech  appropriate  in  the  Kuwaiti  version? 

4.8 

3.5 

4.0 

3.5 

4.0 

Table  1:  Differences  in  Design:  ( Buttons  and  icons,  background  colour,  frames,  graphics,  music,  and  speech) 


1 • > 9 rr 
X ^ o 


British  Version 

Kuwaiti  Version 

Both  Versions 

None 

British  Students 

75% 

25% 

0% 

0% 

Kuwaiti  Students 

60% 

40% 

0% 

0% 

British  Experts 

66.5% 

25% 

8.5% 

0% 

Kuwaiti  Experts 

95.5% 

4.5% 

0% 

0% 

Table  2;  Which  version  did  you  prefer  with  regard  to  buttons  and  icons? 


British  Version 

Kuwaiti  Version 

Both  Versions 

None 

British  Students 

25% 

50% 

25% 

0% 

Kuwaiti  Students 

40% 

40% 

20% 

0% 

British  Experts 

67% 

0% 

16.5% 

16.5% 

Kuwaiti  Experts 

45.5% 

27% 

18% 

9% 

Table  3;  Which  version  did  you  prefer  with  regard  to  background  colours? 


British  Version 

Kuwaiti  Version 

Both  Versions 

None 

British  Students 

50% 

25% 

25% 

0% 

Kuwaiti  Students 

0% 

60% 

40% 

0% 

British  Experts 

67% 

33% 

0% 

0% 

Kuwaiti  Experts 

36% 

55% 

9% 

0% 

Table  4:  Which  version  did  you  prefer  with  regard  to  the  use  of  frames? 


British  Version 

Kuwaiti  Version 

Both  Versions 

None 

British  Students 

50% 

0% 

25% 

25% 

Kuwaiti  Students 

20% 

80% 

0% 

0% 

British  Experts 

50% 

0% 

50% 

0% 

Kuwaiti  Experts 

18% 

45.5% 

36% 

0% 

Table  5:  Which  version  did  you  prefer  with  regard  to  graphics  and  images? 


British  Version 

Kuwaiti  Version 

Both  Versions 

None 

British  Students 

50% 

50% 

0% 

0% 

Kuwaiti  Students 

20% 

80% 

0% 

0% 

British  Experts 

33% 

33% 

17% 

17% 

Kuwaiti  Experts 

0% 

64% 

18% 

18% 

Table  6:  Which  version  did  you  prefer  with  regard  to  music? 


British  Version 

Kuwaiti  Version 

Both  Versions 

None 

British  Students 

75% 

25% 

0% 

0% 

Kuwaiti  Students 

0% 

100% 

0% 

0% 

British  Experts 

83% 

0% 

17% 

0% 

Kuwaiti  Experts 

45.5% 

45.5% 

9% 

0% 

Table  7:  Which  version  did  you  prefer  with  regard  to  speech? 


British  Version 

Kuwaiti  Version 

Both  Versions 

None 

British  Students 

75% 

25% 

0% 

0% 

Kuwaiti  Students 

0% 

40% 

60% 

0% 

British  Experts 

67% 

16.5% 

16.5% 

0% 

Kuwaiti  Experts 

54.5% 

27.3% 

18.2% 

0% 

Table  8:  In  General,  Which  version  did  you  prefer,  British  or  Kuwaiti? 


6.0  DISCUSSIONS 
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This  study  was  able  to  uncover  several  interesting  findings  and  to  make  recommendations  to  designers  when 
designing  multimedia  programs  for  different  cultures.  The  overall  results,  as  shown  in  (Tab.  8),  confirm  that 
most  of  the  Kuwaiti  students,  taking  their  comments  and  suggestions  into  account,  preferred  the  Kuwaiti 
version  of  the  program  while  most  of  the  British  students  preferred  the  British  version.  In  addition,  most  of  the 
British  and  Kuwaiti  experts  preferred  the  British  version  of  the  program  stating  that  “It  has  a standard  and 
simple  look  and  feel”  compared  to  the  Kuwaiti  one.  The  following  is  an  interpretation  of  the  tables  shown 


6.1  BUTTONS  AND  ICONS 

As  shown  in  (Tab.  2),  participants  were  more  positive  about  buttons  and  icons  in  the  British  version  than  the 
Kuwaiti  version,  mainly  because  they  believe  that  buttons  and  icons  are  more  simple  and  clear.  (Tab.  1)  reveals 
that  the  average  score  of  participants’  opinions  about  buttons  and  icons  in  the  British  version  is  4.2,  while  in 
the  Kuwaiti  version  is  3.0.  Out  of  evaluators’  responses,  it  is  learnt  that  it  is  essential  to  improve  buttons  and 
icons  in  both  versions  with  emphasis  on  placing  drawn  graphic  icons  (rather  than  the  bitmaped  icons)  for  the 
Kuwaiti  version  keeping  them  as  simple  and  standard  as  possible,  and  design  them  to  better  reflect  the  Kuwaiti 


6.2  BACKGROUNDS  AND  COLOURS 

The  dominant  background  in  the  British  version  is  textured  grey  as  suggested  by  (Boyle  1997),  and  for  the 
Kuwaiti  version  is  textured  dark  brown,  as  it  is  the  norm  for  most  of  the  Kuwaiti  Press  which  represents  the 
Kuwaiti  desert  environment.  As  shown  in  (Tab.  1),  the  average  score  for  background  colour  in  the  British 
version  is  3.7  while  for  the  Kuwaiti  version  is  3.3.  (Tab.  3)  reveals  that  British  and  Kuwaiti  students  have 
similar  degree  in  liking  background  colours  for  both  versions,  while  British  and  Kuwaiti  experts  were  clearly 
positive  about  background  colour  in  the  British  version. 

Although  the  use  of  colours  can  carry  different  meanings  for  different  cultures,  there  have  been,  as  noticed  from 
participants  comments,  a lot  of  contradictions  which  indicates  that  colours  should  not  be  a cultural  issue,  but 
rather  a personal  taste  when  developing  multimedia  programs.  In  addition,  background  colours  should  fit  with 
the  domain  which  the  application  covers. 

6.3  FRAMES  AND  BORDERS 

The  British  version  was  designed  with  normal  borders  which  were  represented  by  lines  around  both  the  main 
screen  and  the  information  area.  However  in  the  Kuwaiti  version,  two  Islamic  frames  (borders)  were  used,  one 
around  the  screen,  and  one  around  the  information  area.  The  objective  of  the  use  of  frames  in  the  Kuwaiti 
version  was  to  add  Kuwaiti  cultural  taste  to  the  program. 

As  shown  in  (Tab.  1),  the  average  score  for  participants’  response  was  3.8  when  they  were  asked  whether  the 
frames  in  the  British  version  were  appealing.  On  the  other  hand,  their  average  score  for  the  Kuwaiti  version 
was  3.4.  (Tab.  4)  reveals  that  the  majority  of  Kuwaiti  students  and  Kuwaiti  experts  preferred  the  Kuwaiti 
version  in  terms  of  the  use  of  frames  and  borders,  while  the  majority  of  British  students  and  British  experts 
preferred  frames  and  borders  in  the  British  version.  This  part  of  investigation  shows  that  Kuwaitis  liked  the  use 
of  Islamic  colourful  frames  in  the  program,  because  their  eyes  are  used  to,  however,  they  suggested  to  improve 
frames’  quality,  and  to  make  better  selections. 

6.4  GRAPHICS  AND  IMAGES 

Graphics  and  Images  were  scanned  and  imported  in  the  two  versions.  They  were  used  in  the  British  version  to 
reflect  the  British  culture  (Such  as  British  people  or  places),  while  some  graphics  and  images  were  used  in  the 
Kuwaiti  version  to  reflect  the  Kuwaiti  culture.  (Tab.  5)  reveals  that  the  majority  of  Kuwaiti  students  and 
Kuwaiti  experts  preferred  graphics  and  images  in  the  Kuwaiti  version,  while  most  of  British  students  and 
experts  preferred  graphics  and  images  in  the  British  version. 


above. 


culture. 


6.5  MUSIC  AND  SPEECH 


Kuwaiti  music  were  incorporated  with  the  Kuwaiti  version  and  British  music  were  incorporated  with  the 
British  version.  As  illustrated  in  (Tab,  1),  the  average  score  for  evaluators’  opinions  about  the  appropriateness 
of  the  use  of  music  in  the  British  version  and  Kuwaiti  version  were  3,5  and  3.7  respectively  (Scale:  5 = very, 
1 = not  at  all).  (Tab.  6)  reveals  that  most  of  Kuwaiti  students  and  experts  preferred  music  in  the  Kuwaiti 
version,  while  British  students  and  experts  liked  music  in  the  two  versions  equally.  Some  British  evaluators 
suggested  the  selection  of  better  music,  while  others  suggested  to  improve  the  quality  of  the  music  for  the 
British  version. 

In  addition,  speech  was  performed  by  a Kuwaiti  person  (speaking  English)  for  the  Kuwaiti  version,  and  a native 
British  person  for  the  British  version.  As  illustrated  in  (Tab,  1),  the  average  score  for  evaluators’  opinions 
about  the  appropriateness  of  the  speech  used  in  the  British  and  Kuwait  versions  were  4,2  and  4.0  respectively, 
(Scale:  5 = very,  1 = not  at  all).  (Tab.  7)  reveals  that  all  of  the  Kuwaiti  students  preferred  the  Kuwaiti  speech 
while  Kuwaiti  experts  preferred  speech  in  both  versions  (could  be  because  most  of  Kuwaiti  experts  have 
studied  English  abroad  and  lived  with  native  English  speakers).  On  the  other  hand,  most  of  British  students 
and  experts  preferred  speech  in  the  British  version. 


As  the  two  versions  of  Learn  About  PCs  are  going  to  be  redesigned  based  on  the  results  of  this  study,  we  will 
summarise  recommendations  when  redesigning  the  two  versions.  The  objective  is  to  increase  the  suitability  of 
the  two  versions  to  users’  culture: 

1“  Use  standard  icons  for  the  two  versions,  however,  remove  the  scanned  images  which  were  used  as  icons  in 
the  Kuwaiti  version  and  replace  them  with  drawn  graphic  icons  which  reflect  objects  in  the  Kuwaiti  culture, 
2“  As  participants  have  different  perceptions  about  colours  in  general,  and  as  colours  seem  to  be  a personal 
taste  rather  than  cultural  issue  in  this  matter,  background  colours  should  be  similar  in  the  two  versions. 
Light  blue,  light  brown,  or  light  grey  background  could  be  convenient  for  the  two  versions. 

3“  Improve  the  frames  in  the  Kuwaiti  version,  choosing  better  and  more  culturally  related,  using  only  one  or 
two  types  of  frames  throughout  the  program. 

4-  Improve  the  quality  of  graphics  and  images  used  in  the  two  versions,  add  more  graphics  and  images  which 
are  closely  related  to  British  and  Kuwaiti  cultures. 

5“  Change  the  type  of  music  in  the  two  versions,  improve  their  quality,  and  make  better  selection. 
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Abstract:  In  1998  a tool  called  the  "Virtual  Constructing  Tool"  (VCT)  was  developed  within 
the  Faculty  of  Policy  Sciences.  The  VCT  aims  to  facilitate  the  process  of  reading,  discussing  and 
producing  papers  with  construction  and  communicating  tools  and  is  completely  based  on  the  use 
of  the  Internet,  Each  course  in  which  the  VCT  is  used  is  divided  into  two  parts.  In  the  first  part, 
students  construct  their  own  knowledge  system.  In  the  second  part,  personal  findings  and 
opinions  merge  into  a joint  presentation.  In  this  presentation,  the  leading  part  is  played  by  the 
structuring  of  information  with  the  help  of  hyperlinking.  This  paper  shows  the  fundaments  of  the 
VCT  and  describes  its  main  features.  More  information  can  be  found  on  http://www.kun.nl/vct. 


Introduction 

The  Faculty  of  Policy  Sciences  attaches  significance  to  teaching  in  small  groups  with  much  student  involvement 
and  interaction  between  students  and  teachers.  Students  frequently  work  on  co-operatively  assignments,  which 
result  in  a joint  paper.  The  advantages  of  group  work  (see  for  instance,  Eurelings  and  Rontelkap  (98))  are 
strengthened  by  the  use  of  hypermedia.  Because  of  the  increased  user-friendliness  of  the  Internet,  a window  has 
been  opened  to  the  development  of  a learning  tool,  which  is  completely  based  on  the  Internet  standard.  This  tool  is 
called  'Virtual  Constructing  Tool'  (VCT).  The  VCT  aims  to  enrich  learning  processes  within  groups  and  finally,  it 
is  “just  as  any  co-operative  learning  method,  a unique  solution  to  the  problem  of  how  to  structure  a classroom” 
(Kagan  85). 


Communication  and  action 

Years  ago,  Bateson  (51)  distinguished  the  'report  aspect'  and  the  'command  aspect’  within  communication.  In  the 
‘report  aspect’  communication  is  reduced  to  the  transmission  of  information  between  people.  For  instance,  a 
teacher  recites  new  theoretical  insights  with  respect  to  a subject.  There  is  no  need  for  action  as  a response  to  the 
information.  The  receiver  of  the  information  only  has  to  remember  the  information  and  to  be  able  to  recall  it  on 
demand. 

The  transmission  of  information  from  one  person  to  another  also  can  be  interpreted  as  a regulative  act.  The 
teacher’s  announcement  that  theory  ‘X’  is  most  common,  might  result  the  studentswriting  this  down  in  their  notes. 
It  is  the  ‘command  aspect’  that  is  responsible  for  this  change  in  students’  behaviour.  Changes  in  behaviour  can 
influence  the  different  actor’s  interests.  If  information  threatens  the  interests  of  the  receivers,  they  will  not  only 
reconstruct  the  message  but  also  question  its  content.  They  might  reject  its  behavioural  consequences  and  opt  for  a 
discussion  with  the  sender  of  the  message. 

Critical  social  theory  (CST)  recognizes  this  possibility.  This  theory  views  people  as  intelligent  actors  who  assess 
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the  truthfulness,  completeness,  sincerity  and  contextuality  of  the  message  they  receive.  A listener  or  reader  can  go 
beyond  achieving  a mutual  understanding  with  the  speaker  or  writer.  This  is  accomplished  by  critical  reflection, 
which  assesses  validity  claims  which  pertain  to  what  the  speaker  or  writer  means.  The  listener  questions  these 
validity  claims  and  sees  that  it  is  incomplete,  false,  unclear  or  inappropriate.  What  CST  offers  is  the  recognition 
that  communication  among  everyday  actors  also  involves  their  need  to  assess  the  validity  or  justification  of  what  is 
being  communicated  in  the  first  place  Ngwenyama  and  Lee  (97). 


Two  stages 

The  CST  has  many  applications.  In  this  paper  CST  is  a point  of  departure  in  the  exploration  of  the  contribution  of 
hypertext  in  collaborative  learning  in  general,  and  in  the  production  of  group  papers  in  particular.  Two  stages  can 
be  distinguished  in  the  accomplishment  of  a paper:  reading  and  writing. 

The  reading  stage 

In  the  initial  stage  of  the  production  of  a paper,  the  reading  of  articles  and  books  is  a common  phenomenon.  The 
preliminary  problem-formulation  acts  as  guide.  Reading  is  a process  in  which  relevant  facts  and  different  views 
are  listed.  Gradually  different  insights  are  related  and  a view  of  the  authors  themselves  arises.  In  the  reading  stage 
hypertext  is  used  to  collect  and  present  information.  In  particulary,  hypertext  is  a useful  tool  in  relating  different 
texts.  Many  authors  claim  that  hypertext  is  superior  to  conventional  texts  in  representing  the  interconnectivity  of 
concepts  (Berk  and  Devlin  91,  Jacobson  93,  Mayes  90)  and  therefore,  the  production  of  knowledge.  Knowledge  is 
taken  to  be  a semantic  network  of  related  concepts.  During  the  reading  stage  a personal  classification  of  pieces  of 
information  grows.  Research  into  the  effect  of  the  reading  of  hypertext  documents  on  the  integration  of  knowledge 
is  an  ongoing  process  (Jonassen  89,  Siemon  95,  Eklund  95). 

An  additional  advantage  of  the  use  of  hypertext  is  that  articles  quoted  can  be  made  accessible  as  a whole.  By 
making  hypertext  links  to  summaries  of  relevant  parts  of  their  sources,  students  increase  the  clarity  and  solidity  of 
their  argument. 

Some  authors  stress  the  role  of  hypertext  in  the  process  of  the  construction  of  meaning.  Hypertext  forces  the  reader 
- more  than  in  the  case  of  conventional  reading  - to  actively  relate  different  pieces  of  information  (Landow  92, 
Jonassen  89).  The  resulting  individual  web  of  knowledge  serves  as  a point  of  departure  for  communication  with 
other  students.  However,  effective  communication  requires  a logical  structure  of  information.  The  individual  web 
has  a logical  structure  for  the  individual  student  but  not  for  the  other  group  members.  Structuring  information  in  a 
logical  order  for  members  and  other  readers  is  one  of  the  main  targets  of  the  writing  stage. 

The  writing  stage 

If  a certain  threshold  in  the  reading  stage  has  been  surpassed,  then  the  writing  stage  will  commence.  Authors 
usually  aim  at  convincing  their  audience  and  therefore  papers  take  the  form  of  an  argument.  The  delivery  of  this 
argument  requires  restricting  the  readers’  freedom  to  jump  from  one  part  of  the  argument  to  another.  Instead,  the 
author  should  present  a degree  of  structure  and  sequence.  The  necessity  of  putting  structure  in  a paper  constrains 
the  otherwise  desirable  network  structure  of  information.  A network  structure  offers  only  limited  possibilities  of 
determining  what  the  sequence  of  an  argument  is.  As  a consequence,  in  the  writing  stage  a structure  based  on 
hierarchical  branches  is  preferred.  Hypertext  is  deployed  to  facilitate  this  kind  of  structuring.  Structuring  based  on 
hierarchical  branches  has  several  advantages: 

The  paper  may  contain  many  details  and  side-steps  without  damaging  the  main  line  of  argument. 

Students  are  forced  to  distinguish  between  bits  of  information  according  to  their  rank  of  importance  in  view  of  the 
argument. 

The  copying  of  pieces  of  text  form  others,  schemes  and  definitions  can  be  omitted.  Hyperlinking  to  the  original 
sources  (of  excerpts)  suffices.  More  time  and  space  are  left  for  the  student’s  own  line  of  thinking  and  supporting 
facts  and  arguments. 

Hierarchical  branching  offers  lengthy  possibilities  for  validation  by  the  readers  who  feel  its  necessity.  Those  who 
prefer  the  reader’s  argument  for  its  own  sake  will  skim  over  additional  elaboration. 

The  CST  proposes  that  while  reading  a text,  conveying  of  the  meaning  may  include  more  than  just  understanding 
the  writer’s  argument.  The  reader  is  entitled  to  critical  reflection  / validation  (Kunneman  83).  According  to 
Habermas,  each  type  of  social  interaction  brings  about  a set  of  validation  claims  of  its  own.  Ngwenyama  and  Lee 
(97)  differentiate  between  instrumental,  communicative,  discursive  and  strategic  action.  Writing  a paper  is 
considered  to  be  a communicative  action.  The  main  objective  of  writing  a paper  is  discussion  which  possibly 
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results  in  agreement.  The  possibility  of  validating  the  writer’s  claims  is  important  while  reading  an  argument. 
Hypertext  is  a tool  ‘par  excellence’  in  the  case  of  two  kinds  of  validation  claims:  completeness  and  truthfulness. 
Completeness:  In  writing  papers  with  hypertext,  it  is  no  longer  ‘What  is  the  content?’  which  is  the  main  question 
but  ‘Where  should  1 put  this  piece  of  information?’.  Recording  several  types  and  quantities  of  information  is 
allowed.  The  number  of  pages  is  not  a significant  restriction  to  the  content  of  a paper  anymore.  Redundancy  in  the 
inclusion  of  facts  and  visions  is  also  allowed.  Each  paper  contains  virtual  archives  which  can  be  used  for  validation 
purposes  or  just  for  curiosity”s  sake. 

Truthfulness:  Hypertext  allows  references  to  an  abundance  of  articles,  statistics  and  books.  By  enabling  the  reader 
to  consult  original  sources,  the  verification  of  the  validity  of  their  application  is  within  reach. 

This  option  is  not  meant  in  the  first  place  to  discover  deceit.  However,  it  is  commonly  known  that  authors 
unconsciously  interpret  their  sources  in  a way  which  is  more  advantegeus  to  themselves.  In  addition,  more  insight 
is  offered  in  the  contextuality  of  the  sources.  Rasker  et  al.  (98)  talk  about  ‘activity-based  transmission  of 
information’.  Anderson’s  ACT-R  theory  explains  the  power  of  hypertext.  This  theory  poses  that  the  delay  between 
the  content  of  someone’s  reading  and  the  reception  of  feedback  influences  learning.  The  bigger  the  delay,  the  less 
learned  (Anderson  93).  Hypertext  allows  for  immediate  feedback  in  a number  of  cases.  For  instance,  incredible 
views  can  be  checked  immediately  if  supporting  hyperlinks  are  provided.  In  the  case  of  written  texts,  the  threshold 
for  consulting  sources  is  usually  unpermeable  and  only  a few  readers  will  engage  in  the  painstaking  search  for 
sources  in  libraries.  The  addition  of  original  sources  increases  their  accessibility. 

Reading  the  authors’  sources  adds  to  the  understanding  of  the  writer’s  line  of  thinking  and  contributes  to  the 
development  of  meaning  and  the  reader’s  own  personal  interpretation.  Reconstructing  the  writer’s  argument,  he  of 
she  turns  to  the  construction  of  his  of  her  own  insights  (Schank  94). 

In  the  case  a of  disagreement  between  the  readers  and  the  author  of  a paper,  the  readers  should  ideally  also  become 
writers  too,  and  express  their  lines  of  thinking,  by  using  hyperlinking  as  well.  The  original  writer  can  respond  and 
so  forth. 


Hypertext  and  the  structuring  of  information 

According  to  Beeman  (Mayes  et  al  90),  hypertext  learning  systems  can  fundamentally  change  students’  learning. 
While  building  a hypertext  system,  students  organize  information  in  a non-linear  way.  Structuring  information  in  a 
non-linear  way  stimulates  the  processes  of  integration  and  contextualisation.  By  making  hyperlinks,  existing 
knowledge  and  new  information  are  consciously  integrated.  Students  are  constantly  stimulated  to  construct  new 
knowledge  structures.  Interchanges  between  the  old  and  the  new  structures  signify  an  increase  in  learning. 

Pointing  out  the  advantages  of  hypertext  does  not  necessarily  mean  taking  a position  in  the  debate  between  the 
devotees  of  linear  versus  non-linear  representation  of  knowledge  (Ess  96).  In  fact,  it  is  the  degree  of  linearity  or 
structure  that  counts.  Jonassen  divides  hypertext  into  twe  catogaries,  structured  and  unstructured  hypertext.  In 
unstructured  hypertext  linking  has  no  rules  (Jonassen  89). 

A fundamental  belief  behind  unstructured  hypertext  is  the  desirability  of  relating  information  by  means  of 
association  (Berk  & Devlin  91).  However,  readers  will  comprehend  the  coherence  within  the  information  only  if 
they  share  each  other's  associations.  Because  of  a personal  bias  in  regard  to  associations,  this  will  not  often  happen. 
Maybe  this  is  the  reason  why  many  users  feel  lost  in  a hypertext  network  (Mayes  et.  al  90) 

Unstructured  hyperlinking  is  allowed  only  for  private  purposes.  Our  students  are  urged  to  build  a knowledge  web 
of  their  own.  Hypertext  is  used  here  in  order  to  collect  information  and  to  relate  pieces  of  information  together.  In 
the  realisation  of  a group  paper,  unstructured  hyperlinking  is  not  tolerated  for  reasons  of  communicating 
effectively  and  the  necessity  of  logical  arguing,  which  implies  imposing  some  direction  and  sequence. 

According  to  Landow  (92),  linearity  is  not  only  typical  in  the  traditional  presentation  of  papers  but  it  can  also  be 
found  in  the  building  blocks  of  a network  structure.  A network  structure  offers  the  possibility  for  readers  to  choose 
a sequence  themselves.  The  more  the  author’s  goal  is  to  convince  readers  of  his  or  her  argument,  the  more 
structure  and  guidance  is  implied.  Within  a predetermined  structure,  hyperlinking  is  a usable  means  for  indicating 
alternative  paths  and  viewpoints. 

A predominantly  linear  structure  is  preferable  in  case  of  an  argument,  in  a situation  in  which  convincing  another 
person  is  of  essence.  A non-linear  structure  offers  more  possibilities  if  an  inventory  has  been  made  of  knowledge 
structures  or  belief  systems. 
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The  role  of  the  teacher 


The  value  attaches  to  hypertext  is  based  on  the  acceptance  of  constructivist  principles  in  regard  to  teaching  and 
learning:  In  a constructivist  view,  teaching  is  the  facilitation  of  the  rediscovery  of  knowledge  and  the  conveying  of 
meaning  to  various  parts  of  the  world.  It  is  not  the  delivery  of  information.  Students  acquire  knowledge  if  they 
become  involved  in  an  active  construction  process.  They  have  to  read  texts,  to  compare  various  views  of  diferent 
authors  and  to  be  able  to  formulate  their  own  views.  The  writing  of  an  argument  is  a useful  tool  in  reaching  these 
objectives.  The  teacher  can  support  students’  construction  activities  by  formulating  an  argument  first  and  then 
inviting  students  to  respond  critically  if  it  is  necessary  that  they  assume  a perspective  view  themselves.  To  reach 
this  goal  hypertext  is  used,  for  instance,  to  admit  students  to  the  sources  of  the  teacher’s  argument  (Schank  94). 
Divergences  from  the  teacher’s  thinking  patterns,  by  reinterpreting  his  or  her  resources,  deepens  understanding. 

A construction  process,  such  as  the  one  indicated  above  requires  a permissive  and  discovery  minded  atmosphere  in 
the  classroom.  The  Virtual  Constructing  Tool  (VCT)  which  is  illuminated  below  offers  ICT  support  in  the 
students’  collaborative  reading  and  writing  activities. 


The  composition  of  the  VCT 

The  Virtual  Constructing  Tool  has  three  different  levels:  the  general  level,  the  group  level  and  the  personal  level. 
General  level:  The  general  level  is  accessible  to  all  visitors  of  the  Internet.  Everyone  who  deliberately  or 
incidentally  passesby  can  become  acquainted  with  the  course.  E-mails  can  be  despatched  to  the  teacher  and  to  the 
students.  Students  in  their  turn,  will  find  schedules,  learning  materials,  news  items  and  answers  to  FAQ. 

Group  level:  Students  can  log  in  to  the  group  level  by  using  a password.  Only  members  of  the  group  are  admitted. 
The  tools  for  writing  a group  paper  are  accessible  from  this  level. 

Personal  level:  The  personal  level  is  reserved  for  personal  items,  for  instance,  references,  summaries,  relevant 
links,  messages  to  news  groups  and  notes.  As  these  items  are  all  accessible  from  the  Internet,  students  always  have 
them  at  their  disposal,  regardless  of  the  PC  they  use. 

On  this  level,  students  make  their  own  knowledge  web,  which  consists  of  Internet  pages  dealing  with  their  specific 
themes.  These  pages  are  the  building  blocks  of  the  group  paper. 

Students  are  invited  to  design  personal  pages.  It  is  a useful  exercise,  which  contributes  to  the  social  atmosphere  of 
the  group.  ‘‘Whether  or  not  software  tools  designed  to  support  collaborative  learning  will  actually  do  so,  depends 
mainly  on  to  which  extent  the  instructor  and  the  students  are  able  to  create  a social  structure  which  supports  these 
processes.”  (Keynes  97).  Photographs  of  the  group  members  also  contribute  to  this  goal. 


The  devices  of  the  VCT 

The  devices  of  the  VCT  are  divided  into  Construction  tools.  Communication  tools  and  Help.  All  devices  enable 
time  and  place  independence.  Obviously,  students  may  have  meetings  or  use  more  conventional  tools  such  as  a 
telephone  or  a faxmachine. 

Construction  Tools 

The  construction  tools  support  students  in  designing  Internet  pages.  Students  have  at  their  disposal  style  sheets  and 
the  so-called  ‘html  bank’.  Style  sheets  are  ready-made  Internet  pages,  in  which  students  only  need  to  type  their 
own  texts.  A professional  lay  out  and  background  is  provided. 

The  html  bank  offers  pictures,  buttons,  symbols  and  other  graphic  tools,  so  that  students  can  use  these  to  prepare 
their  papers. 

Communication  tools 

In  addition  to  all  the  usual  communication  facilities,  the  students  have  course  conferencing  and  the  so-called 
‘mutation  form’  at  their  disposal.  Course  conferencing  enables  students  to  communicate  with  all  the  course 
members.  They  can  type  in  a message  or  respond  to  messages  they  have  received.  All  messages  are  presented  in 
sequential  order.  It  is  possible  to  include  links  to  the  group  level  or  to  external  Internet  sites. 

Students  use  the  mutation  form  in  order  to  propose  mutations  in  other  students’  papers.  Mutations  may  concern 
parts  of  the  text  of  the  inclusion  or  exclusion  of  links.  The  proposed  mutations  usually  lead  to  discussion. 
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Help 

Students’  computer  abilities  differ  considerably.  The  Help  function  offers  a manual,  answers  to  FAQ  and  a guide 
for  seeking  on  the  Internet.  Of  course,  contacting  the  Webmaster  is  also  possible. 


Conclusion 

The  VCT  is  used  during  four  different  courses.  In  all  of  these  courses,  additional  changes  have  been  made  in  the 
use  and  appearance  of  the  VCT.  We  should  gain  more  experience  before  sustaining  the  possible  advantages  for 
using  hypertext  in  writing  papers.  For  the  time  being,  these  temporary  findings  can  be  given: 

Students  and  teachers  are  used  to  strictly  linear  thinking.  In  using  the  VCT,  they  must  abandon  this  way  of 
thinking  and  become  used  to  connecting  pieces  of  information  through  hyperlinking.  It  takes  time  before  they 
become  accustomed  to  structuring  information  in  this  way.  Aftcrw'ards,  the  assignment  appeared  to  not  be  suitable 
enough  for  structuring  information  by  means  of  hyperlinking.  However,  writing  a paper  in  this  way  was  not 
experienced  as  being  more  difficult  than  was  usually  the  case. 

In  addition  to  all  usual  communication  facilities,  students  can  use  the  facilities  within  the  VCT.  However,  in  true 
practice,  no  one  used  these  facilities.  The  reasons  were  mentioned,  which  were  lack  of  speed  and  interaction. 
Insufficient  computer  equipment  at  home  (whether  they  had  an  Internet  connection  or  not)  was  also  seen  as  a 
limitation. 


References 

Anderson,  J.R.  (1993).  Rules  of  the  Mind. 

Bateson, G.  (1951).  Information  and  Codification.  IN:  Ruesch  J.  & G.  Bateson,  Communication:  The  Social  Matrix  of 
Psychiatry. 

Berk,  E.  & J.Devlin  (1991).  Why  hypertext?  IN:  Berk  E.,  & J.Devlin,  hypertext/hypermedia  handbook  Berk,  E.  & J.Devlin 
(1991).  What  is  hypertext?,  IN:  Berk  E.,  & J.Devlin,  hypertext/hypermedia  handbook 

Eklund,  J.  (1995).  Cognitive  models  for  structuring  hypermedia  and  implications  for  learning  from  the  world-wide  web, 
[online]  Available  HTTP:  http://www.scu.edu.au/sponsored/ausweb/ausweb95/papers/hypertext/eklund/index.html 
Ess,  Ch.  (1996).  Modernity  and  postmodemity  in  ‘hypertext  notes’,  a call  for  theoretical  consistency  and  completeness.  In: 
Ejoumal,  Volume  6,  No.  3,  august  1996.  [online]  Available  HTTP: 
http://www.hanover.edu/philos/ejoumal/archive/v6n3/ess/ess.html. 

Eurelings  A.  F.  & F.  Rontelkap  (1996).  User  requirements  for  the  design  of  POLARIS.  An  electronic  collaborative  learning 
environment  for  problem-based  learning.  In:  Placing  the  student  at  the  centre.  Current  Implementations  of  Student-Centered 
Education,  20th  Anniversary  Conference  Maastricht  1996 

Jacobson,  M.  J.,  & J.A.,  Levin  (1993).  Network  learning  environments  and  hypertext:  constructing  personal  and  shared 
knowledge  spaces.  In  D.  Foster  & D.  V.  Jolly,  Proceedings  ofTel-Ed  '93  (pp.  190-197).  [online]  Available  HTTP: 
http://www.ed.uiuc. edu/projects/tta/Papers/J&L-Tel-Ed93.html 
Jonassen,  D.H.  (1989).  Hypertext/hypermedia 

Kagan  S.  (1985).  Dimensions  of  cooperative  classroom  structures  IN:  Learning  to  Cooperate,  Cooperating  to  Learn,  Rev 

versions  of  papers  originally  presented  at  the  Second  Conferece  of  the  International  Association  of  Cooperation  in  Education 

Keynes  M.  (1997).  Supporting  Collaborative  Learning  in  Asynchronous  Learning  Networks,  Invited  Keynote  Speaker  for  the 

UNESCO/  Open  University  Symposium  on  Virtual  Learning  Environments  and  the  role  of  the  Teacher 

Klemm  W.R.,  & J.  R.  Snell  (1996).  Enriching  Computer-Mediated  Group  Learning  by  Coupling  Constructivism  with 

Collaborative  Learning  [online]  Available  HTTP:  http://cwis.usq.edu.au/electpub/e-jist/klemm.htm 

Landow,  G.P.  (1992).  Hypertext,  the  convergence  of  contemporary  critical  theory  and  technology. 

Mayes  T.,  M.R.  Kibby  & T.  Anderson  (1990).  Signposts  for  conceptual  orientation:  some  requirements  for  learning  from 
hypertext  IN:  Me  Aleese  & Green,  Hypertext:  State  of  the  Art,  1990. 

Ngwenyama,  O.  & A.  S.  Lee  (1997).  Communication  richness  in  electronic  mail:  critical  social  theory  and  the  context  of 
meaning:  In:  MIS  Quarterly,  Volume  21,  Number  2,  pp.  145-167,  June  1997. 

Rasker,  P.C.  e.o.  (1998).  the  effect  of  two  types  of  information  exchange  on  team  self-correction.  Paper  presented  at  the  RTO 
HEM  Symposium  on  "Collaborative  Crew  Performance  in  Complex  Operational  Systems  ". 

Schank,  R.  (1994).  Engines  for  Educators.  The  Institute  for  the  Learning  Sciences. 

Siemon,  J.  (1994).  Hyperhausarbeit:  Lemwege  in  Hypertext/Hypermedia,  [online]  Available  HTTP; 
http  ://www.  wiso . gwdg.de/'^j  s iemon/W3_3 1 3 .HTM 


1233 


Getting  Computer  Information  Technologies  Used  in  Teaching  and  Learning: 
A Model  of  Technology  Diffusion  in  a K-10  School. 


Lyn  Henderson 

School  of  Education,  James  Cook  University, 

Townsville.  Australia.  481 1 
E-mail:  lynette.henderson@jcu.edu.au 
Ph:  (61)  (7)  47  814  355;  Fax  (61)  (7)  47  25 1 690 

Scott  Bradey 

Technology  Coordinator,  Charters  Towers  School  of  Distance  Education 
Queensland.  Australia. 

E-mail;  sbradey@chartowesde.qld.edu.au 
Ph:  (61)  (7)  47  875  733;  Fax:  (61)  (7)  47  875  784 


Abstract:  Technological  diffusion  in  schools  is  a complex  process  that  has  not  met  expectations. 

The  paper  identifies  a model  adopted  by  a K-10  school  of  distance  education:  the  Multi- 
Dimensional  Emersion  Model  of  Technology  Diffusion.  It  analyzes  the  process,  including 
technical,  pedagogical,  and  coercion  and  commitment  issues.  Evaluation  of  the  outcomes  of  the 
whole  school  approach  of  the  first  year  of  adoption  revealed  general  approval  of  the  process,  some 
disquiet  concerning  overt  recognition  by  administration,  and  a majority  who  were  able  to  move 
outside  their  comfort  zone. 

During  the  last  decade  we  have  witnessed  a significant  rise  in  the  number  of  computers  in  schools.  However, 
the  integration  of  educational  technology  in  school  curricula  still  lags  behind  earlier  expectations.  Reports  and 
researchers  have  tried  to  explain  this  gap,  usually  in  terms  of  a failure  by  colleges  of  education  to  expose  students 
to  educational  technology  and  to  train  them  adequately  in  this  area,  budget  limitations,  the  teachers'  fear, 
reluctance,  and  often  resentment  to  incorporate  computer  technology  in  their  classrooms,  and  infrastructure 
organizational  difficulties.  Effective  technological  innovation  is  such  a complex  process  and  so  difficult  to 
accomplish  within  school  cultures  that  the  literature  identifies  "islands  of  excellence"  (Willis,  1996).  This  paper 
reports  the  first  year  phase  in  a Pre  to  Grade  10  school  of  distance  education’s  journey  to  create  such  an  island.  The 
specific  aims  are  to  identify  the  model  of  technological  diffusion  adopted,  analyse  the  processes,  procedures,  and 
conceptual  frameworks  that  were  utilized,  and  evaluate  the  outcomes. 


Methodology 

A qualitative  research  methodology  was  employed.  The  study  utilized  the  initial  workshop  evaluation,  a needs 
assessment  survey,  a post  53  five  point  Likert  Scale  questionnaire,  open-ended  structured  questionnaire,  teacher 
project  documentation,  observational  data  from  the  teachers’  presentations  of  their  WWW  project  to  their  peers, 
and  anecdotal  evidence  from  the  technology  coordinator.  The  documentation  contained  teacher  evaluations  of 
computer  software  and  WWW  sites  as  well  as  unit  plans  detailing  the  incorporation  of  their  selected  piece  of 
software  and  WWW  site  into  their  curriculum. 

The  Charters  Towers  School  of  Distance  Education  (http://www.chartowesde.qld.edu.au/)  previously.  School 
of  the  Air,  is  situated  in  North  Queensland,  Australia.  It  provides  education  to  preschool  to  Grade  10  children, 
anywhere  in  Australia,  who  live  at  a distance  to  the  nearest  state  school,  are  in  hospitals,  belong  to  travelling 
families,  or  are  taught  at  home.  The  School’s  Development  Council  decides  the  strategic  direction  policies  of  the 
school  but  not  its  operations;  it  comprises  equal  representation  from  parents  or  caregivers  and  school  staff  The 
school  uses  traditional  means  of  communication  like  mail  and  telephone  as  well  as  radio  for  daily  half-hour 
synchronous  lessons  between  the  teacher  and  students,  video,  television,  and,  since  the  beginning  of  1998,  the 
Internet  to  cross  the  barriers  of  distance  and  create  the  bonds  of  a school  community.  Where  possible,  face-to-face 
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contact  with  students  is  achieved  through  structured  student-family  camps  at  the  School  when  formal  lessons  are 
conducted  and  informal  learning  and  socialising  occurs  between  family  and  school,  and  by  teachers  through 
excursions,  home  visiting,  and  outreach  activities. 

All  37  teachers  (Preschool  through  to  Grade  10;  one  teacher-librarian,  and  two  Languages  Other  Than 
English)  were  involved  in  the  technology  project.  There  were  five  male  teachers  and  32  female  teachers  ranging  in 
age  from  22  years  to  50  years.  Some  had  been  in  the  school  for  up  to  eight  years  and  were  therefore  experienced  in 
distance  education,  while  six  teachers  were  appointed  from  regular  urban  public  schools  to  the  distance  education 
school  in  1998.  Most  had  not  extended  their  knowledge  beyond  the  wordprocessing  applications  and  the  majority 
had  had  little  experience  with  E-mail  and  even  less  with  the  WWW  prior  to  1998. 


Results  and  Discussion 

Our  model  of  technological  change,  the  Multi- Dimensional  Emersion  Model  of  Technology  Diffusion,  was 
identified  from  the  data.  Nevertheless,  it  is  not  surprising  that  it  echoes  theoretical  models  of  school  change  or 
includes  elements  that  other  projects  have  identified  as  fundamental  for  successful  diffusion.  Understandably,  it 
also  contains  innovative  facets  that  incorporated  aspects  and  shapes  of  existing  practices  and  values  particular  to 
the  Charters  Towers  School  of  Distance  Education.  The  paper  proceeds  to  discuss  the  model  highlighting  elements 
of  similarity  to,  and  difference  from,  other  paradigms  of  technological  diffusion  and  school  change.  The  model  is  a 
combination  of  what  we  term,  "markers-of-significance",  that  were  put  in  place  at  the  beginning  of  the  technology 
diffusion  project  or  evolved  as  it  proceeded. 

One  of  the  markers  identified  in  the  model  is  the  decision-makers.  As  the  decision  to  focus  on  technology 
diffusion  was  made  by  the  School  Development  Council,  which  involved  teachers,  it  was  not  perceived  as  a "top- 
down”  imposition  by  any  of  the  teachers  (Post  Likert  Questionnaire).  The  Council  made  certain  global  decisions  or 
big  picture  decisions  to  do  with  school  change.  These  global  why,  what,  when,  and  how  decisions  involved  a 
funnelling  effect  to  the  other  stakeholders. 

The  "why"  decisions  involved  the  rationale  and  aims  for  the  school  to  be  at  the  forefront  of  computer 
information  technologies  so  that  the  students  would  be  well-served  for  the  present  and  future.  Effective  school  aims 
need  to  become  the  "common  ground,  the  shared  vision  that  compels  all  involved"  (Meacutendez-Morse,  1992). 
All  but  four  teachers  supported  the  technology  infusion  in  their  professional  lives  and  their  teaching:  ”1  embraced  it 
as  it  is  the  way  of  the  future  for  me  and  the  students";  " 1 found  it  exciting  as  1 could  see  its  potential  in  student 
learning";  I enjoy  the  challenge  of  learning  about  technology"  (Open-ended  Questionnaire).  In  this  respect  the 
multi-dimensional  emersion  model  reflects  the  re-educative/culture/fix-the-system  attributes  of  Hord’s  (1992) 
generational  model. 

The  "what”  decision  embodied  aspects  of  the  "fix-the-parts"  and  "fix-the-system"  change  model  (Hord,  1992). 
It  involved  resourcing  the  vision  or  putting  dollars  where  they  would  make  a real  difference.  The  school  funded 
equipment,  such  as  a laptop  computer  with  a standard  suite  of  software  for  each  teacher,  an  Internet  infrastructure 
and  online  costs,  and  a computer  laboratory  for  in-school  camps.  Students'  families  were  provided  the  opportunity 
to  either  purchase  or  lease  a computer  and  printer  from  the  school,  including  the  standard  package  of  software.  The 
resourcing  decision  also  resulted  in  the  appointment  of  a teacher  as  a half-time  information  technology  (IT) 
coordinator-half-time  teacher.  It  was  a significant  resolution  and  commitment  to  change.  This  decision,  which 
teacher  representatives  were  party  to,  highlights  the  teachers'  corporate  commitment  to  the  school's  vision;  their 
1 00%  agreement  (Post  Likert  Questionnaire)  meant  they  had  to  cover  the  IT  coordinator's  half-time  teaching  load 
when  he  wore  his  IT  coordinator's  hat  training  and  mentoring  the  school  community:  staff,  students,  and  home 
tutors.  The  multi-dimensional  emersion  model  therefore  did  not  lack  appropriate  resources  which  Boyd  (1992,  p. 
25)  argues  has  been  "a  major  barrier  to  sustained  change  efforts". 

The  "when"  decision  resulted  in  a two-year  time-line  for  information  technology  diffusion  to  be  the  school’s 
major  economic  and  pedagogic  focus.  This  "fix-the-system"  approach  eschewed  a global  linear  approach  where 
step  two  did  not  start  till  step  one  was  completed  (Carr  & Duchastel,  1998).  For  example,  the  school  did  not  wait 
till  all  the  staff  had  a laptop  (the  last  few  teachers  received  their  laptop  towards  the  end  of  the  research)  before 
commencing  induction  programs  or  appointing  the  IT  coordinator.  Twenty-one  percent  of  the  teachers  (Post  Likert 
Questionnaire)  voiced  a "too  much,  too  soon"  concern  in  their  questionnaires.  They  would  have  preferred  a linear 
approach  so  that  they  would  only  have  had  to  cope  with  frustrations  on  one  level,  their  own  computer  skill 


progress,  rather  than  on  a number  of  levels,  including  coming  to  grips  with  the  pedagogic  changes  needed  for 
incorporating  the  WWW  into  their  teaching  and  technical  failures  (Open-ended  Questionnaire).  According  to 
Geoghegan  (1994),  such  a preference  for  incremental  change  as  opposed  to  discontinuous  and  often  disruptive 
breakthrough  alterations  in  core  processes  is  not  uncommon. 

The  "how",  fix-the-people  - fix-the  system  (Hord  1992),  decision  resulted  in  an  emersion  model  which 
involved  all  the  teachers  rather  than  a slower  approach  to  training  volunteers  who  would  enthuse  other  staff  to 
adopt  the  necessary  pedagogic  changes  to  their  teaching.  Louis  and  Miles  (1990)  preferred  such  a slower 
evolutionary  method  and  cautioned  that  ifs  non-adoption  could  mean  death  to  innovation.  The  teachers  perceived 
the  opposite  with  only  7%  disagreeing  with  the  items:  "A  small  number  of  enthusiastic  or  knowledgable  teachers 
should  have  been  involved  first"  and  "The  Grade  6-10  teachers  [the  only  grades  whose  students  were  provided  with 
a home  computer]  should  have  been  inserviced  first"  (Post  Likert  Questionnaire). 

The  major  input  into  the  nature  of  these  global  decisions  came  from  the  school  principal,  who  also  had  the  job 
of  operationalizing  them.  One  aspect  of  this  was  communicating  the  administration's  and  the  School  Development 
Committee's  expectation  that  the  staff  would  become  meaningfully  involved  in  the  diffusion  of  technology.  That 
expectation  was  accompanied  by  a high  level  of  trust  that  this  would  naturally  happen  because  trust  meant 
recognition  and  acknowledgment  of  the  teachers'  professionalism.  These  sorts  of  elements  appear  to  echo  the 
empirical-rational  and  power-coercive  strand  of  Hord’s  (1992)  generational  model.  Operationalized,  this  meant  the 
principal  adopted  a direct  hands-off  approach.  The  data  reveals  that  the  principal  was  definitely  perceived  by  the 
teachers  as  a necessary  catalyst  for  change.  However,  21%  of  the  teachers  (Post  Likert  Questionnaire)  would  have 
appreciated  an  increased  level  of  direct  involvement,  particularly  in  terms  of  individualized  praise  and 
understanding  of  what  they  were  accomplishing. 

The  second  marker  of  significance  in  our  model  of  change  was  the  IT  coordinator.  Casey's  (1992)  distributed 
expertise  model  emphasized  the  importance  of  leadership  in  school  change,  not  just  from  the  principal  but  also 
from  others,  such  as  an  IT  coordinator.  This  role  was  a two-way  conduit-facilitator  between  the  principal  and  the 
teachers.  This  duality  was  probably  a significant  factor  in  how  the  IT  coordinator  exercised  his  expertise.  The 
teachers  reported  that  he  helped  them  develop  images  of  "how  to  get  there"  that  were  tied  to  a workable  reality. 
This  is  one  of  the  factors  that  Boyd  (1992)  argues  helps  teachers  develop  a sense  of  ownership  of  the  diffusion 
process.  This  perception  suggests  that  all  three  generations  in  the  "normative-re-education  - culture  - fix-the- 
system"  model  (Hord  1992)were  in  operation. 

The  how  and  what  of  inservicing  staff  was  left  to  the  IT  coordinator.  A multi-faceted  model  of  professional 
development  was  adopted.  According  to  Topp,  Mortenson,  & Grandgenett  (1996),  this  is  an  important  factor  in 
school  change  because  no  one  method  of  staff  development  works  for  all  teachers.  The  Consortium  for  Policy 
Research  in  Education  (1995)  maintained  that  the  most  promising  professional  development  programs  are  those 
that  stimulate  and  support  site-based  initiatives  because  they  are  not  isolated  from  classroom  realities.  The  multi- 
dimensional emersion  model  certainly  contained  this  ingredient.  It  was  based  on  a needs  analysis  the  IT 
coordinator  conducted  with  the  staff  to  ascertain  their  requirements  and  concerns.  Inservicing  involved  traditional 
workshops,  provision  for  peer  collaborations,  mentoring,  and  residential  workshops. 

The  professional  development  program  followed  a linear  time-line  unlike  the  overall  project  which  overlapped 
the  phasing  in  of  key  elements.  The  reasons  were  specific  to  the  school's  context.  The  program  capitalized  on  the 
division  of  the  Australian  school  year  into  four  terms  of  approximately  ten  weeks  each.  Each  term  had  a distinct 
focus.  The  first  phase  of  the  inservicing  was  a series  of  non-compulsory  workshops  dealing  with  wordprocessing, 
spreadsheet,  database,  desktop  publishing,  E-mail,  and  WWW  access  and  search  skills.  These  were  held  weekly  for 
most  of  the  first  term.  According  to  the  data,  93%  agreed  that  this  was  a sensible  approach  (Post  Likert 
Questionnaire);  it  allowed  the  IT  coordinator  and  other  new  teachers  to  settle  into  teaching  at  a distance  and  all 
teachers  to  obtain  a comparable  level  of  technology  skills.  It  also  afforded  staff  a welcome  distance  from  the 
networking  hassles.  In  effect,  it  gave  the  teachers  valuable  time  to  begin  to  internalize  what  the  school  vision 
would  mean  in  its  actualization.  Compulsory  workshops  were  conducted  in  second  term  and  third  term  to  explore 
the  pedagogy  involved  in  evaluating  and  using  educational  multimedia  software  and  the  WWW,  respectively,  as 
teaching  and  learning  tools.  Each  of  these  sessions  involved  three  compulsory  after-school  workshops  that 
addressed  theoretical  and  hands-on  practical  issues.  Teachers  were  not  overly-disgruntled  about  having  to  spend 
their  own  time  after  a long  day,  being  re-educated  (Post  Likert  and  Open  Ended  Questionnaires  and  Anecdotal 
Records). 

The  teachers  consistently  reported  a very  high  rating  on  items  ascertaining  the  facilitative  effectiveness,  level, 
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and  type  of  support  of  the  IT  coordinator  with  only  seven  percent  undecided  about  the  appropriateness  of  the  level 
of  support  (Post  Likert  Questionnaire).  During  the  whole  process,  the  IT  coordinator  provided  ”just-in-time" 
mentoring  and  support  for  individuals  and  groups  as  either  he  or  the  teachers  saw  the  need  (Open-ended 
Questionnaire).  Many  teachers  reported  that  they  perceived  the  IT  coordinator's  praise  and  enthusiasm  for  their 
achievements  as  motivating  and  rewarding  with  a few  explaining  their  belief  that  this  attitude  was  not  tokenistic  as 
it  was  blended  with  useful  critique  (Open-ended  Questionnaire).  Sashkin  and  Egermeier  (1992)  concluded  that  the 
"more  personal  assistance  and  continuing  support  from  a skilled  and  knowledgable  local  agent,  the  more  likely  that 
the  innovation  will  be  used  for  a long  duration"  (p.  7).  Hopefully,  long-term  evaluation  will  prove  this  correct. 

The  third  marker-of-significance  in  the  model  is  the  teachers.  For  a variety  of  reasons,  teachers  are  not 
particularly  malleable  to  change  (see  Hord,  1992).  The  school  principal  made  it  clear  to  staff  that  the  school's  two 
year  focus  on  the  integration  of  technology  this  would  involve  commitment  to  the  inservice  that  the  IT  coordinator 
devised.  This  certainly  contains  elements  of  the  power-coercion  - political  - fix  the  people  change  model  (Hord 
1992)  yet  it  also  involves  the  re-educative  - fix  the  system  model  (Hord  1992)  because  a majority  of  the  teachers 
had  taken  on  board  their  personal  commitment  to  their  re-education  (Post  Likert  and  Open-ended  Questionnaires). 
The  innovation  involved  exemption  from  non-project  commitments,  such  as  a residential  workshop  based  off- 
campus  at  the  district  university,  and  after  school  hours  workshops.  Interestingly,  few  teachers  commented 
negatively  on  these  compulsory  after-hour  workshops  (Likert  Questionnaire).  Many  reported  that  the  time  was 
more  than  compensated  by  the  chance  "for  working  together  and  learning  together"  (Open-ended  Questionnaire).  It 
is  not  only  technical  resources  that  count  but  also  the  time  and  energy  demanded  of  people  to  plan,  share,  observe, 
and  take  action.  (Boyd,  1992). 

Teachers  were  offered  the  opportunity  to  work  individually,  in  pairs,  or  in  a small  groups  on  their  curriculum 
units.  Only  four  teachers  and  the  teacher-librarian  worked  individually.  Most  opted  to  work  in  pairs  for  the 
interactive  multimedia  CD-ROM  activity  with  most  working  as  groups  of  three  on  the  WWW  activity.  There 
appear  to  be  four  major  reasons  for  their  collaboration:  (a)  it  permitted  the  pre-school  and  primary  teachers  to 
brainstorm  ideas  for  their  common  grade  levels  while  the  secondary  teachers  collaborated  in  discipline  areas;  (b) 
time  constraints  meant  collaboration  allowed  a reduction  in  individual  time  spent  resource  hunting  and  unit 
preparation;  (c)  it  provided  security  through  peer  support;  and  (d),  it  divided  the  responsibility  for  designing  units 
that  incorporated  the  software  and  WWW  site  in  meaningful  contextualized  ways.  The  data  reflects  what  Boyd 
(1992)  reported:  skill  building  and  training  are  part  of  the  process  of  change  and,  hence,  learning  to  do  something 
new  involves  initial  doubts  about  one's  ability  and  incremental  skill  development. 

Outcomes  are  obviously  another  marker  of  significance.  The  teachers'  self-reported  outcomes  included: 
increased  confidence;  improved  competency  in  computer  skills  and  Internet  usage,  particularly  search  strategies; 
recognition  of  self  and  others  efforts;  ability  to  integrate  CD-ROMs  and  the  WWW  as  learning  tools  in  their 
curriculum;  using  CD-ROMs  and  the  WWW  with  students  and  witnessing  their  growth  and  enjoyment.  A number 
still  acknowledged  their  need  for  continued  improvement  due,  in  part,  to  not  utilizing  the  skills  or  implementing 
the  strategies  regularly.  There  appeared  to  be  little  technofatigue.  Three  months  after  the  first  year,  all  but  three 
teachers  (one  was  indifferent)  reported  that  they  still  found  learning  about  technology  and  its  incorporation  into 
curriculum  enjoyable  and  were  "not  getting  tired  of  it":  "No  way!";  "To  live  is  to  learn";  "I 'm  still  learning  and 
enjoy  the  challenges"  (Open-ended  Questionnaire). 

Examination  of  the  documentation  data,  their  unit  plans  and  the  teachers'  presentations  of  the  WWW  units, 
revealed  that  most  teachers  treated  the  professional  development  tasks  seriously  and  professionally.  The  others 
handed  in  tokenistic  work.  A majority  of  teachers  appeared  to  change  their  pedagogy  to  integrate  new  ways  of 
teaching  and  learning  into  their  curriculum.  They  had  grappled  with  the  necessary  pedagogical  changes  to  their 
thinking  and  practice.  The  Open-ended  Questionnaire  reported  an  acknowledgment  of  their  successes  and,  for 
many,  a current  determination  not  to  "backslide",  as  one  respondent  worded  it.  The  outcomes  of  the  minority  group 
of  teachers  can  be  depicted  as  a brief  sojourn  out  to  the  world  of  the  Internet  and  interactive  multimedia  CD- 
ROMs.  Overall,  for  the  majority,  the  data  revealed  that  their  world  was  not  the  comfortable  status  quo  oyster  where 
a minority  of  the  teachers  still  positioned  themselves. 


CONCLUSION 


Any  change  paradigm,  like  the  multi-dimensional  emersion  model  of  technology  diffusion,  will  always 
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incorporate  elements  of  coercion,  pressure,  and  support.  It  is  getting  the  weightings  and  timing  of  them  about  right 
that  is  often  crucial.  The  key  to  any  change  strategy  is  the  word  "persistence".  Resource  provision,  professional 
development,  teacher  commitment,  ongoing  assistance,  and  recognition  can  not  be  one  shot  events  or  quick  fixes 
(Hord  92;  Beer,  Eisenstat,  & Spector,  1990).  Currently,  at  the  end  of  phase  one,  it  seems  that  the  change  effort  was 
more  than  an  illusion  for  a majority  of  teachers.  Implementation  of  phase  two  needs  to  monitor  the  "persistence" 
factors  if  long-term  success  is  to  be  the  outcome. 
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Abstract:  This  paper  addresses  the  research  question:  What  are  some  of  the  key  information 
and  communication  technology  issues  from  a fractal  management  perspective  when 
implementing  the  virtual  class  in  conventional  higher  education?  It  also  suggests  effective 
management  strategies  for  dealing  with  the  identified  issues.  The  specific  manifestation  of 
the  virtual  class  referred  to  in  this  study  is  when  the  virtual  class  is  based  on  Internet  and 
Intranet  technologies.  The  term  “networked  education”,  coined  by  the  writer  is  used  to 
describe  this  expression  of  the  virtual  class.  The  study  concludes  that  the  implementation  of 
the  virtual  class  in  conventional  higher  education  has  an  extensive  and  profound  impact  on, 
and  demands  new  ways  of  using  and  managing  information  and  communication 
technologies. 


1.  Introduction 


**The  single  most  frequent  failure  in  the  history  of forecasting 
has  been  grossly  underestimating  the  impact  of  technologies^^ 

Anon 

The  research  question  addressed  in  this  paper  is:  What  are  some  of  the  key  information  and  communication 
technology  issues  from  a fractal  management  perspective  when  implementing  the  virtual  class  in  conventional 
higher  education?  This  paper  attempts  to  identify  what  some  of  the  key  technology  issues  are  when  the  virtual 
class  is  implemented  in  conventional  higher  education  and  ways  to  address  these. 

Tiffin  and  Rajasingham  (1995)  argue  that  communications  have  a fractal  dimension  that  is  "a  node  in  a 
communications  network  can  prove,  on  closer  examination,  to  be  a communications  network  itself  (p.  37).  The 
technological  issues  are  also  viewed  from  a fractal  management  level  that  is  to  acknowledge  the  various  levels 
and  dimensions  of  management  the  management  processes  of  the  institute,  administration,  the  departments 
(both  academic  and  administrative),  the  design,  development  and  delivery  of  teaching  materials,  the  student's 
learning  and  so  forth.  “Management”  in  tertiary  education  does  not  refer  to  “administration”  only.  To  limit 
management  to  administrative  practices  only  and  thus  excluding  the  teaching  area  would  be  to  argue  that 
planning  and  decision  making,  organising,  leading  and  control  do  not  occur  in  the  area  of  teaching,  but  only 
in  administration.  “Management”  refers  therefore  to  management  on  all  levels  of  the  organisation,  both  in  the 
teaching  and  administrative. 


2.  Creating  Visibility  in  A Virtual  World 

Rayport  and  Sviokla  (1995)  summarise  the  essence  of  the  virtuality  of  the  virtual  class  when  they  write  about  a 
corresponding  phenomenon  in  the  business  world  that  they  term  the  "marketspace"  that  is  a virtual  market 
place.  They  characterise  it  as  being  "virtual"  because  the  value-adding  processes  “...are  performed  through  and 
with  information”. 


Tiffin  & Rajasingham  (1995)  first  used  the  term  "virtual  class"  to  refer  to  the  learning  process  that  is  enabled 
solely  by  telecommunications.  Tiffin  & Rajasingham  (1995)  distinguishes  the  concept  of  the  virtual  class  from 
the  “Virtual  Classroom”  coined  by  Roxanne  Hiltz  (1986)  as  “...it  suggests  that  the  place  a virtual  class  is  held 
is  an  electronic  simulation  of  a conventional  classroom”  (p.  10)  because  Hiltz  described  it  as  the  use  of 
computer-mediated  communications  “...to  create  electronic  analogue  of  the  communications  forms  that  usually 
occur  in  a classroom  including  discussion  as  well  as  lectures  and  tests”  (p.  95). 

The  virtual  class  can  be  described  as  the  process  that  occurs  when  teacher,  learner,  problem  and  knowledge  are 
joined  solely  through  communication  and  information  technologies  for  the  purpose  of  learning  and  teaching.  It 
is  an  educational  experience  of  real  people  in  a virtual  dimension.  In  the  virtual  class  the  teaching  and 
learning  is  performed  without  the  movement  of  physical  objects  (eg  getting  students  and  lecturers  into  a 
physical  venue). 

The  specific  manifestation  of  the  virtual  class  referred  to  in  this  study  is  when  the  virtual  class  is  based  on  the 
Internet  or  Intranet  technologies.  The  term  “networked  education”,  coined  by  the  writer  is  used  to  describe  this 
expression  of  the  virtual  class.  Networked  education  emphasises  the  high  level  of  connectivity  that  it  enables 
through  creating  a network  between  student  en  student,  student  and  teacher,  student  and  resources,  teacher 
and  resources  as  well  as  the  past  and  the  present  (through  availability  of  on-line  resources  of  one  course 
occurrence  for  a next  occurrence).  It  also  indicates  that  the  education  is  network-based  (Internet  or  intranet) 
and  computer-mediated,  that  it  includes  teaching,  learning  and  research  (“education”),  and  emphasises  the 
distribution  both  of  the  control  of  learning,  as  well  as  the  on-line  learning  and  teaching  materials  among  the 
students  and  teacher(s). 

The  virtual  nature  of  this  type  of  learning  can  remove  the  conventional  prompts  to  assist  the  learner  to  focus 
on  their  studies  for  example  when  the  lecturer  starts  to  speak  in  a face-to-face  on-campus  lesson,  or  the  arrival 
of  study  materials  in  paper-based  distance  education  mode.  It  is  therefore  essential  in  the  virtual  environment 
to  build  in  prompts  to  assist  in  this  area  for  example  automatic  e-mail  based  on  the  course  schedule  as  well  as 
on  the  participation  of  each  individual  student. 


3.  Providing  Mobility  in  the  Virtual  Class 

The  most  significant  loss  students  often  experienced  in  an  on-line  environment  are  the  lack  of  mobility  of  the 
study  materials. 

The  affordability  and  weight  of  portable  computers  such  as  Laptop  computers  and  notebooks  negate  these 
technologies  from  being  an  adequate  provider  of  mobility.  Smaller  hand-held  palmtop  computers  with  infrared 
updating  capabilities  are  a positive  step  in  re-creating  the  mobility  for  on-line  students. 

Another  development  which  is  currently  experimented  with  by  MIT  (Nicholas  Negroponte  at  the  1997  ICDE 
Conference)  and  by  commercial  companies  in  the  US,  is  to  address  and  program  pixels  on  paper  which  has  all 
the  characteristics  of  wood-based  paper  except  that  it  is  made  of  specialised  materials.  Pages  in  such  a book 
can  be  loaded  with  information  form  the  Web  or  other  electronic  sources  and  copied  and  pasted  into  other 
applications.  Each  page  has  a huge  capacity  to  store  information  and  can  be  continuously  updated  and 
reloaded.  This  will  be  a major  breakthrough  to  provide  students  of  the  virtual  class  with  both  mobility  and  a 
totally  new  level  of  flexibility. 


4.  The  Pervasiveness  of  Communication  and  Information  Technologies  in  the 
Virtual  Class 
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A complete  set  of  technologies,  which  have  become  transparent  through  their  pervasive  integration  into  every- 
day life  is  required  to  make  conventional  on-campus  or  distance  education  possible.  This  set  of  technologies 
include  roads,  vehicles,  electricity,  air  conditioning,  buildings,  ducting,  clothing,  aids  such  as  spectacles,  food 
preparation  technologies,  piping,  systems  dealing  with  waste,  entertainment  systems  and  so  forth.  At  the  same 
time  these  technologies  require  skills  and  knowledge  which  in  most  cases  have  become  totally  transparent 
through  our  frequent  and  natural  use  of  it  for  example  navigating  buildings  and  roads. 

The  virtual  class  is  per  definition  based  on  communication  and  information  technologies.  Education  in  this 
environment  similarly  needs  a whole  range  of  technologies,  which  often  corresponds  to  the  ones  described 
above.  At  this  stage  the  technologies  required  to  enable  the  virtual  class  are  however  not  only  in  its  infancy 
and  undergoing  revolutionary  changes,  but  the  full  complement  of  what  is  required  is  not  even  fully  clear  yet. 
In  the  current  phase  of  establishing  the  virtual  class,  there  are  often  new  realisations  of  totally  new 
technologies  needed  or  significant  changes  required  to  existing  technologies  (like  Word-processing  HTML 
documents).  It  might  be  that  the  virtual  class  at  this  stage  is  creating  more  questions  than  being  nicely  tied 
down  according  to  our  traditional  conventions.  In  this  scenario  communication  and  information  technologies 
can  be  seen  to  be  more  of  a hurdle  than  an  enabling  agent. 

The  virtual  class  therefore  requires  of  both  student  and  lecturer  to  assume  a sophisticated  level  of  computer 
literacy  and  use.  While  the  on-line  lecturer  may  have  support  within  the  tertiary  educational  institute  through  a 
computer  department  and  a network  of  colleagues,  the  students  often  may  not  have  the  same  advantage.  It  is 
therefore  essential  to  provide  technical  assistance  not  only  via  on-line  help  information,  but  also  through  a 
technical  help  desk  function  which  can  be  reached  by  telephone  and  fax. 

A conventional  tertiary  educational  institute  interested  in  using  technology-based  education  like  the  virtual 
class  seems  to  require  therefore  an  extensive  use  of  CIT  throughout  the  organisation.  At  the  University  of 
Melbourne,  Australia,  where  a campus  wide  information  system  (CWIS)  was  implemented,  Goldenfarb  (1995) 
noted  that  one  barrier  to  the  successful  implementation  of  the  CWIS  was  low  IT  skills.  Goldenfarb  also  notes 
that  “...if  the  problem  is  identified,  it  can  be  overcome  and  the  department  can  become  a successful  adopter". 

In  embracing  the  virtual  class,  an  organisation  per  definition  also  embraces  communication  and  information 
technologies  on  a wide  base  and  with  it  the  requirement  for  higher  and  new  levels  of  computer  literacy  of  their 
academic  an  administrative  staff  as  well  as  their  students. 


5.  Managing  Discontinuity  in  the  Virtual  Class 

In  networked  education  the  materials  and  teaching  process  is  in  a state  of  continuity  while  these  are  in  a state 
of  discontinuity  in  conventional  tertiary  education.  Once  a course  is  on  the  Web,  it  remains  available  and  no 
special  arrangements  need  to  keep  it  continually  available;  special  arrangements  however  have  to  be  made  to 
discontinue  availability. 

Effectively  managing  the  discontinuity  of  on-line  materials  as  a result  of  the  discontinuity  of  human 
involvement  is  vital  to  meet  student  expectations  and  provide  ongoing  support,  and  in  so  doing  avoiding  that 
the  institute  comes  in  disrepute.  An  example  of  discontinuity  is  when  a course  is  in  a specialised  academic 
area  and  the  lecturers  discontinue  their  involvement  with  the  course.  The  on-line  course  materials  will  still 
remain  on-line  and  might  become  obsolete,  or  when  removed  causes  frustration  to  on-line  users  when 
unsuccessfully  attempting  to  locate  it. 

In  a paper-based  distance  education  environment  or  a face-to-face  physical  teaching  environment,  the 
discontinuity  of  a lecturer  can  be  addressed  by  the  discontinuity  of  the  mailings  or  the  classes  at  an  appropriate 
point.  Two  factors  contribute  to  the  problem.  Firstly  the  on-line  materials  are  often  registered  with  search 
engines  and  guides  on  the  Internet.  The  URL  is  then  bookmarked  within  a web  browser  by  users  who  may  also 


share  the  URL  with  others  in  a network  of  contacts.  Secondly  the  on-line  materials  need  to  be  kept  up  to  date  if 
it  needs  to  remain  available. 

The  virtual  class  thus  needs  special  approaches  to  ensure  a seamless  discontinuity  like  de-registering  the 
materials  with  the  search  engines  and  guides,  replacing  the  course  materials  with  clear  notices  to  that  effect  or 
contacting  those  whom  the  institute  know  have  bookmarked  the  materials. 


6.  Effective  Data  Management  in  the  Virtual  Class 

It  is  quite  staggering  to  see  how  many  computer  products  for  the  development  of  on-line  courses  take  the 
notion  for  granted  that  course  elements  (“course  objects”)  - including  media  elements  - are  to  be  stored  in  flat 
directory  structures. 

Often  the  HTML  files  media  elements  and  scripts  are  being  stored  on  servers  in  an  organised  flat  directory 
structure,  which  is  reminiscent  of  how  data  was  stored  before  the  1980’s.  Since  then  relational  databases  and 
object  oriented  databases  have  clearly  emerged  as  more  effective  and  sound  ways  of  storing  information  in 
computer  systems. 

Some  comprehensive  and  specialised  databased  software  have  been  developed  for  on-line  course  generation  eg 
“Hyperwave”  (initially  developed  as  “Hyper-G”)  (Hyperwave  1998)  and  TLM  (The  Learning  Manager)  and 
Lotus  Learning  Space. 


7.  Bridging  Transactional  Space  in  the  Virtual  Class 

A major  challenge  in  the  virtual  class  is  addressing  the  issues  of  dialogue  across  the  response  and 
psychological  distance  between  teacher  and  learner.  "Distance"  in  the  virtual  class  however  is  no  longer 
defined  in  terms  of  physical  proximity  but  in  response  time  (comment  made  by  Nicholas  Negroponte  at  the 
18th  World  International  Council  for  Distance  Education  Conference  in  June  1997). 

There  is  also  another  type  of  distance  which  Caladine  (1993)  explains  as  'transactional  distance'  which  is  the 
psychological  distance  between  learner  and  learner,  and  learner  and  teacher.  Caladine  indicates  that 
transactional  distance  in  technology-delivered  education  is  greatly  impacted  by  the  technology  itself:  “...one  of 
the  most  important  determining  factors  is  the  medium  of  communication”.  Terry  Evans  and  Daryl  Nation 
(1992)  seem  to  agree  when  they  state  that  virtual  class  practices  “...do  not  eliminate  the  problems  of  distance 
between  teachers  and  learners  but  create  their  own”  (p.  9). 

Complicating  this  issue  is  a changing  relationship  between  student  and  lecturer.  In  the  conventional  class,  the 
teacher  often  has  a one-to-many  relationship  with  the  students,  which  is  based  on  the  conventional  teaching 
model  as  well  as  on  convenience  for  the  students.  Students  in  a course  will  often  relate  with  the  academic 
community  solely  via  the  convenor  of  the  course  when  seeking  clarification,  feedback,  additional  instruction  or 
wish  to  challenge  ideas.  The  on-line  student  however  is  “...no  longer  confined  to  our  campus  and  its  teachers 
and  students  and  activities”  Tiffin  (November  1996).  The  student  now  has  various  teachers  accessible  by  e- 
mail  and  geographically  located  anywhere  in  the  world.  This  was  vividly  illustrated  when  the  writer  wanted  to 
clarify  an  aspect  of  Roger’s  (1983)  diffusion  of  innovation  theory  and  was  able  to  use  electronic  mail  to  contact 
Everett  Rogers  personally  and  received  clarification  within  24  hours  through  a reply  electronic  mail  (Rogers, 
E.  Re:  Top-down  approach,  everett  rogers  <erogers@unm.edu>,  10  July  1998).  The  student  in  the  virtual 
class,  through  an  extended  group  of  on-line  teachers,  can  therefore  be  more  challenging  and  also  more 
knowledgeable  than  the  conventional  student.  Furthermore,  lecturers  might  have  to  deal  with  a new 
relationship  with  their  students  which  is  not  one-to-many  but  one  of  many. 
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Synchronous  on-line  communication  facilities  like  Internet  Relay  Chat  (IRC),  voice,  video-conferencing  and 
shared  whiteboard  facilities  over  the  Web  provide  social  interaction  in  a more  natural  way,  and  also  build 
some  accountability  into  on-line  courses.  Asynchronous  mechanisms  like  electronic  mail,  hypermail-threaded 
message  and  newsgroups  are  effective  in  bridging  both  time  and  space. 

Addressing  the  social  needs  of  on-line  students  and  on-line  teachers  are  critical.  Strategies  like  newsgroups, 
electronic  mail  discussion  lists,  synchronous  on-line  meetings,  arranging  some  kind  of  meetings  in  the 
physical  realm  with  other  students,  designing  courses  with  a high  level  of  interactivity  and  using  photographs 
and  video  clips  of  students  and  lecturers  within  a course  can  all  be  used  to  address  these  needs. 


8,  The  Nature  of  Change  Required  When  Implementing  the  Virtual  Class 

Taylor,  Lopez  and  Quadrelli  (1996)  comments  on  the  work  of  Morrison  believe  it  to  be  improbable,  as 
technology  becomes  more  pervasive  in  all  aspects: 

. . . and  suggests  that  our  thinking  about  those  issues  itself  must  recognise  that  a 'smooth  evolutionary 
transition'  to  the  widespread  use  of  more  flexible  modes  of  delivery  is  not  likely.  Indeed,  his  work 
suggests  it  may  be  far  more  productive  to  conceptualise  the  process  of  evolution  in  terms  of 
dislocations,  dilemmas  and  uncertainties  rather  than  projections  from  'what  is’  to  'what  is  needed'. 

At  the  University  of  Melbourne,  Australia,  where  a campus  wide  information  system  (CWIS)  was 
implemented,  (Goldenfarb  1995),  a research  project  looked  closely  at  the  first  ten  departments  adopting  the 
use  of  CWIS  and  set  out  to  test  if  critical  success  factors  in  diffusing  innovations,  identified  in  the  literature 
and  at  other  universities  played  key  roles  in  diffusing  the  CWIS  in  this  University.  Goldenfarb  found  that: 

In  the  final  analysis  of  departments  against  criteria  measures  for  successful  adoption,  the  departments 
that  recognised  the  low  skills  as  barrier  found  solutions  to  this  problem  and  rated  highly  in  the 
ranking  order.  On  the  other  hand,  the  department  that  had  very  high  IT  skills  but  did  not  see  a clear 
advantage  in  adopting  ranked  last.  The  department  that  did  not  see  a clear  relative  advantage  didn't 
have  full  commitment  from  the  head  of  the  department.  Many  other  departments  reported  success  in 
obtaining  commitment  from  the  leader,  when  clear  benefits  were  demonstrated  in  trial/pilot  projects. 

Understanding  the  benefits  of  the  virtual  class,  like  possible  productivity  increases  of  academic  staff,  is 
therefore  essential  for  administrative  managers  and  academic  staff  to  take  a positive  interest  in  the 
implementation  of  the  virtual  class. 


9.  Conclusion 

The  possibility  in  the  virtual  class  to  work  collaboratively  has  become  a strong  motivation  to  consider 
networked  education.  This  is  not  surprising  if  one  considers  the  nature  of  the  underlying  distributed 
technologies  of  the  virtual  class  that  is  the  Internet  and  intranets  - the  purpose  of  which  in  essence  is  to  enable 
connectivity  - be  it  to  share  resources  or  contact  other  people.  This  fundamental  characteristic  of  the  virtual 
class  needs  to  be  fully  explored  in  areas  like  on-line  communication,  use  of  on-line  resources,  collaborative 
research  activities  and  the  transnational  recognition  of  prior  learning  and  qualifications. 

Does  conventional  tertiary  education  have  the  ability  to  change  as  dramatically  as  the  new  media  seem  to 
demand>  Can  it  respond  in  such  a flexible,  dynamic  and  adaptable  way?  Thomas,  Carswell,  Price  and  Petre 
(1998)  argues  for  the  “transformation  of  practices  (both  teaching  and  administrative)  to  take  advantage  of 
technology  in  order  to  provide  needed  functions,  rather  than  superficial  translation  of  existing  practices”. 

Taylor,  Lopez  and  Quadrelli  (1996)  believe  it  to  be  improbable: 
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As  technology  becomes  more  pervasive  in  all  aspects  of  teaching  and  administration,  both  academic 
and  general  staff  roles  are  being  transformed.  New  positions  and  skills  are  required  across  all  key 
areas.  From  the  diversity  of  staff  development  strategies  and  activities  that  universities  are  adopting, 
we  identified  three  approaches  to  deal  with  this  challenge.  These  approaches  will  need  to  support  an 
accelerated  shift  from  teaching  to  learning,  delivered  not  by  individual  lecturers  but  by  multi- 
functional teams.  Universities  are  poorly  equipped  and  under  resourced  to  manage  this  strategic 
change. 

It  is  important  that  the  future  must  not  be  inhibited  by  what  we  have  been  doing  in  the  past;  we  have  a great 
opportunity  to  rethink  teaching,  learning  and  research;  to  uncritically  replicate  in  the  virtual  class  what 
happens  in  conventional  on-campus  and  distance  education  would  be  a tragedy. 
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a time  of  drastic  change  it  is  the  learners  who  survive; 
the  ^Uearned**  find  themselves  fully  equipped 
to  live  in  a world  that  no  longer  exists** 
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A major  benefit  of  incorporating  multimedia  and  technology  into  foreign  language  learning  is  that  it  can  provide 
students  with  something  resembling,  albeit  loosely,  an  immersion  experience.  Linking  the  classroom  to 
authentic  sounds  and  images  engages  the  student  in  the  learning  process;  it  makes  language  learning  more  than 
a series  of  verb  conjugations,  and  instead  a richer  experience.  If  we  can  engage  the  student,  then  we  have  a 
better  change  of  making  him/her  a successful  language  learner. 

One  of  the  goals  of  our  language  programs  is  to  help  students  achieve  advanced  levels  of  oral  proficiency  in  the 
foreign  language  while  giving  them  as  much  exposure  as  possible  to  authentic  cultural  information.  In  order  to 
enhance  language  learning  possibilities,  we  decided  to  take  advantage  of  advanced  digital  imaging  technology  to 
provide  culturally  rich  virtual  learning  environments  for  our  students.  A special  program  at  Franklin  & 
Marshall,  the  Academic  Technology  Venture  Fund  for  Curricular  Innovation,  allowed  us  to  create  an  intra- 
departmental  archive  of  culturally  authentic  French  and  Spanish  digital  images.  We  are  using  these  images  to 
create  innovative  activities  which  can  be  tailored  to  various  levels  of  language  learners. 

The  Academic  Technology  Venture  Fund  for  Curricular  Innovation  was  established  to  promote  faculty 
initiatives  that  enhance  opportunities  for  curricular  innovation  through  instructional  technology  support  that 
foster  teaching  and  learning.  Through  a competitive  application  process,  the  Venture  Fund  awards  several  mini- 
grants of  up  to  $5000  each  to  support  curricular  project  design,  development,  and  implementation.  Collaborative 
projects  exceeding  $5000  are  also  possible.  One  of  the  advantages  of  the  Venture  Fund  is  the  collaborative 
nature  of  the  projects.  Content  experts  can  rely  on  technology  experts  to  help  create  the  applications  needed  in 
order  to  deliver  the  class  materials  in  systematic,  efficient,  and  professional  package  to  students. 

The  first  priority  of  this  project  was  to  establish  an  archive  of  culturally  rich  images;  we  sent  a student  who  was 
fluent  in  French  and  highly  proficient  in  Spanish  armed  with  both  a digital  camera  and  a standard  35mm  to 
Spain  and  France  in  order  to  record  these  images.  We  provided  him  with  a "shot  list"  that  included  items  from 
monuments  to  bidets  to  couples  holding  hands.  We  expected  the  student  to  return  with  about  1000  pictures  — he 
returned  with  nearly  2000,  including  those  of  Paris  after  the  1998  World  Cup!  Approximately  1200  of  these 
images  were  recorded  digitally  while  the  remaining  750+  were  taken  with  print  film  and  then  transferred  to 
Photo  CD™. 

Upon  the  student's  return,  our  second  priority  was  to  create  a database  and  categories  for  that  database  in  which 
to  insert  information  about  each  image.  This  included  information  about  where  the  picture  was  taken,  the 
elements  within  the  photo,  and  a series  of  keywords.  This  aspect  of  the  project  will  result  in  a searchable  web 
site  that  instructors  can  use  to  create  classroom  applications  and  that  students  can  use  to  research  both  oral  and 
written  projects. 

While  the  images  have  already  been  used  for  multiple  purposes,  one  specific  project  that  we  have  developed 
promotes  oral  proficiency.  As  we  began  to  view  the  images,  it  became  clear  that  they  could  be  used  to  create 
narratives,  a key  feature  of  advanced  level  speakers.  One  of  the  most  difficult  leaps  that  a language  learner 
makes  is  the  progression  from  being  an  intermediate  level  speaker  with  the  ability  to  describe  the  here  and  now 
to  an  advanced  level  speaker  who  is  able  to  talk  about  events  which  occurred  in  the  past.  Textbooks  at  every 
level  spend  time  talking  about  how  to  talk  in  the  past,  but  students  are  rarely  given  enough  time  to  really  work 
on  this  aspect  of  acquisition  before  they  are  moved  on  to  another  grammar  point.  The  "storyboards"  that  we 
create  for  students  give  them  the  opportunity  to  further  practice  this  skill  using  authentically  appropriate  images 
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that  are  linked  to  the  vocabulary  that  they  have  been  studying.  These  storyboards  can  be  a brief  as  four  images 
or  as  long  as  eight.  For  intermediate  level  speakers,  there  is  a logical  sequencing  to  the  pictures  and  a common 
theme.  For  instance,  one  storyboard  (below)  contains  six  images:  a young  woman  on  a plane,  a picture  of  a 
beach  in  Spain,  a close-up  of  sunbathers,  a young  man  renting  out  lounge  chairs  on  the  beach,  a shady  character 
smoking  a cigarette,  and  a police  officer.  Students  have  the  opportunity  to  create  a story  using  these  images  so 
that  their  language  creation  also  occurs  within  a context.  Students  also  record  their  stories  orally  without  the 
opportunity  to  write  anything  down  since  our  goal  is  to  improve  oral  proficiency  with  this  project.  We  expect 
that  students  will  complete  five  to  six  of  these  stories  throughout  the  semester  with  the  instructor  providing 
feedback  for  each  story.  Students  access  the  storyboards  via  our  campus  network,  and  eventually  they  will  be 
able  to  do  so  via  a website. 


This  classroom  application  is  something  that  we  would  not  have  been  able  to  do  within  a traditional  classroom. 
We  would  need  to  either  spend  classroom  time  passing  the  pictures  to  each  student  or  create  20  different 
storyboards  so  that  students  could  take  them  home  and  work  with  them.  Given  the  fact  that  we  intend  to  have 
students  complete  five  of  these  narrations,  that  would  involve  over  100  storyboards  and  at  least  400  pictures. 
Technology  has  given  us  a tool  to  deliver  this  application  to  students  in  an  efficient  manner  and  has  already 
provided  us  with  essential  information  about  how  non-native  speakers  tell  stories  in  a foreign  language. 
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1.  Research  Framework 

Problem-based  learning  (PBL)  is  an  instructional  approach  that  uses  problems  as  the  stimulus  and 
focus  for  student  activity  (Baud  & Feletti,199l).  Research  suggests  that  PBL  may  help  students  develop  the 
skills  necessary  to  pursue  learning  independently  (Aspy,  Aspy,  & Quinby,  1993). 

In  PBL,  instruction  begins  with  the  presentation  of  a complex  problem  situation  and  all  learning  occurs 
as  a result  of  students'  efforts  to  solve  the  problem.  As  students  realize  that  they  lack  information  necessary  to 
develop  a solution,  they  identify  learning  needs,  then  engage  in  self-directed  study  to  meet  those  needs 
(Barrows  & Tamblyn,  1980).  During  self-directed  study,  learners  must  sift  through  information,  distinguishing 
that  which  is  pertinent  from  that  which  is  not.  Yet,  inexperienced  problem  solvers  tend  to  focus  on  the  surface 
features  of  a problem,  failing  to  recognize  what  information  is  useful  (de  Jong  & Ferguson-Hessler,  1 986).  Even 
in  the  field  of  medical  education,  both  faculty  and  students  have  expressed  concern  that  students  may  not  be 
able  to  determine  what  should  be  learned  (Barrow  & Tamblyn,  1980).  When  attempting  to  use  PBL  with  young 
learners,  support  for  the  development  of  self-directed  study  seems  warranted. 

Responsibility  for  this  support  can  be  left  to  the  classroom  teacher,  but  the  extent  and  variety  of 
supports  needed  can  be  overwhelming.  One  of  the  potential  benefits  of  computer  technology  in  educational 
environments  is  that  it  can  share  the  responsibility  for  this  support  with  the  teacher.  Collins  (1991)  argues  that 
technology  can  offer  good  process  models  of  expert  performance  embedded  within  the  situations  in  which  they 
are  useful.  The  purpose  of  this  study  is  to  determine  if  the  provision  of  hypermedia  based  models  of  expert 
cognitive  processes  can  support  students'  work  during  self-directed  study. 


2.  Research  Questions 

1 . Does  hypermedia  delivered  cognitive  modeling  affect  students’  use  of  a cognitive  tool  during  self-directed 
study? 

2.  Do  students  transfer  strategies  taught  through  hypermedia  delivered  modeling  to  other  tasks? 

3.  Design  of  Study 

3.1  Sample 

Two  sixth  grade  science  classes  in  a suburban  school  district  in  the  southwestern  United  States 
participated  in  the  study. 

3.2  Treatments 

Alien  Rescue,  a hypermedia  PBL  program  dealing  with  the  solar  system,  was  used  with  both  classes. 
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Two  treatment  conditions  were  developed.  In  the  cognitive  modeling  treatment,  the  character  of  an 
expert  scientist  appeared  in  a video  in  the  hypermedia  program  to  model  the  cognitive  processes  she  would  use 
in  collecting  information  pertinent  to  an  identified  learning  need.  In  the  help  treatment,  the  expert  scientist 
explained  the  function  of  the  same  tools  used  in  the  modeling  treatment,  but  without  modeling  how  she  would 
use  these  tools  to  meet  a learning  need.  Most  students  worked  in  pairs,  though  some  worked  individually. 


3.3  Dependent  Variables 


Students’  notebooks  and  a log  of  their  tool  access  were  used  to  examine  the  effects  of  the  modeling  on 
their  work  during  a three  day  period  of  self-directed  study.  To  examine  transfer,  a problem  situation  unrelated 


to  the  problem  presented  by  the  software  was  given  to  students  at  the  end  of  the  self-directed  o. 


t''-' 


3.4  Data  Analysis 

The  notebooks  were  analyzed  descriptively  for  evidence  of  the  effect  of  modeling  during  self-directed 
study.  For  the  transfer  measure,  six  relevant  facts  (needs)  and  ten  irrelevant  facts  (non-needs)  provided  in  the 
measure  were  identified.  Students  received  one  point  for  each  relevant  fact  they  identified,  and  one  point  for 
each  irrelevant  fact  they  did  not  include,  and  these  were  added  for  a total  score.  Three  one-way  ANOVAs  were 
conducted  with  the  needs,  non-needs,  and  total  scores  as  the  dependent  variables. 


4.  Results  and  Discussion 

The  descriptive  analysis  of  students’  notebooks  showed  differences  in  students’  work  during  self- 
directed  study.  These  results  are  summarized  in  Table  1 . 


Cognitive 

Modeling 

Help 

Average  number  of  words  in  notebook 

207 

188 

Average  number  of  times  notebook  was  accessed  in  three  weeks 

73 

63 

Average  number  of  meaningful  sections  created  by  4^*^  day  of  study 

4.1 

2.8 

Percent  of  notebooks  containing  evidence  of  lists 

91% 

25% 

Percent  of  notes  that  represented  relevant  facts 

68.6% 

47.0% 

Table  1:  Results  of  Descriptive  Analysis  of  Notebooks 


For  the  transfer  measure,  the  analysis  of  variance  for  the  needs  and  total  scores  showed  non-significant 
results,  while  the  ANOVA  for  the  non-needs  score  was  significant  (F=  4.869,  df  = 1, 39;  p < .05). 

The  results  of  this  study  suggest  that  students’  self-directed  study  is  impacted  by  the  provision  of  a 
hypermedia  based  tool  in  which  an  expert  models  the  cognitive  processes  she  would  use,  and  that  students  will 
transfer  skills  modeling  during  PBL  to  other  problems  where  they  are  useful.  Of  particular  note,  the  results 
suggest  that  students  who  received  modeling  were  better  able  to  distinguish  information  useful  in  solving  a 
problem  from  other,  less  useful  facts.  This  suggests  that  problem-based  learning  software  should  be  designed  to 
include  tools  that  provide  expert  modeling  of  the  cognitive  processes  used. 
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Introduction 

The  University  of  Alberta’s  Faculty  of  Education  has  undertaken  a restructuring  process  with  the  goal  of  better 
preparing  teachers  to  effectively  integrate  technology  into  their  teaching.  A major  component  of  this  restructuring 
process  is  the  development  of  a program  of  courses  that  make  extensive  use  of  Web-based  technologies.  This  paper 
describes  one  course  development  project:  the  transformation  of  a traditional  Lab-based  instructor  led  course,  on 
multimedia  technology  tools,  to  a “Wed-Mediated  Instruction”  (WMI)  course  that  exploits  various  communication 
features  of  the  Internet.  The  underlying  goal  of  this  transformational  process  was  to  preserve  the  constructionist 
learning  environment  (Nicaise  & Barnes,  1996)  of  the  traditional  course  while  at  the  same  time  optimizing  the 
course  delivery  mode  to  make  it  more  accessible  to  a wider  audience  of  students. 


Project  Overview 


A course  on  multimedia  in  the  classroom  (Carbonaro,  1997)  taught  entirely  in  Lecture/Lab  format  was  transformed 
by  moving  the  course  resource  materials  onto  the  World  Wide  Web  (Web)  and  by  making  extensive  use  of  Internet- 
based  telecommunications.  The  course  (now  called  EDPY  485  - Multimedia  Tools  for  Teachers)  employs  what  we 
like  to  call  Web-Mediated  Instruction  (WMI).  A course  delivered  by  WMI  is  not  necessarily  delivered  completely 
over  the  Internet,  rather  the  Internet  serves  to  provide  resources  and  facilitate  communication.  Face-to-face  (F2F) 
instruction  and  F2F  student  collaboration  are  still  important  components  of  our  course.  This  allows  for  increasing 
student  accessibility  (Internet-based  support)  while  simultaneously  maintaining  a constructionist  collaborative 
learning  environment  that  incorporates  multimedia  tools  for  creativity  and  problem-solving. 

The  Course 

The  theoretical  considerations  of  the  project  resulted  in  the  development  of  a course  that  made  heavy  use  of 
hypermedia  construction  applications  and  Computer-Mediated  Communication  (CMC)  which  are  viewed  by 
Jonassen  (1996)  as  superlative  Mindtools.  The  course  website  (http://www.quasar.ualberta.ca/edpy485dev)  provides 
a great  deal  of  flexibility  in  that  students  can  proceed  through  the  course  experiencing  varying  degrees  of  F2F 
instruction.  Some  students  like  to  be  helped  in  person,  so  they  attend  scheduled  lab  times.  Others  choose  to  work 
more  independently,  from  home,  work  or  wherever.  Some  students  collaborate  via  telecommunications  while  others 
prefer  in-person  collaboration.  Another  factor  in  our  particular  course  is  that  few  students  can  afford  the  wide  array 
of  application  software  and  peripheral  devices  available  at  the  on-campus  lab  facility.  This  course  requires  that  the 
students  utilize  special  hardware  and  software  including:  scanners,  digital  cameras,  postscript  printers,  video 
digitizing  hardware,  ZIP  drives,  and  high-end  video  and  graphic  editing  software.  Thus,  in  our  course,  economics 
dictates  that  students  must  work  on-campus  for  certain  assignments.  In  any  case,  the  website  provides  the 
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information  the  students  need  in  order  to  understand  what  is  required  of  them  to  complete  the  course:  course  outline, 
schedule,  list  of  required  materials,  explanation  of  assignments,  marking  criteria,  tutorials,  supplemental  reading 
material,  etc.  The  course  requires  that  six  modules  be  completed,  they  are:  Internet  tools,  multimedia  tools,  digital 
poster,  presentation  software,  hypermedia,  and  webpage  construction. 

The  EDPY  485  course  is  continuously  undergoing  many  changes  to  enhance  the  instructional  environment. 
These  include:  (a)  modifications  to  the  Web-based  resource  center;  (b)  greater  use  of  the  Internet  for  course 
communication;  (c)  allowing  greater  choice  in  the  time  and  location  of  instruction;  and  (d)  improving  the  assessment 
of  student  performance.  In  particular,  we  have  devoted  considerable  effort  to  developing  the  course  website  in 
support  of  course  objectives.  The  course  website  is  not  just  a syllabus  representing  the  instructor's  own  perspective 
(Duchastel,  1 997).  TTiere  is  a vast  amount  of  educational  material  available  on  the  Internet  that  is  very  useful  to  pre- 
service or  in-service  teachers.  A website  makes  it  easy  to  point  the  students  to  resource  sites,  and  to  Internet-based 
search  tools  which  aid  them  in  finding  additional  resources  on  their  own.  A lot  of  the  information  presented  in  this 
course  is  very  dynamic.  A website  enables  us  to  constantly  enrich,  update,  and  disseminate  this  information.  The 
course  website  also  enables  us  to  easily  show  students  exemplary  models  of  previous  students'  work,  which  students 
can  view  whenever  they  wish,  from  any  computer  with  Internet  access. 


Conclusions 

After  some  initial  offerings  we  obtained  feedback  from  students  via  an  online  anonymous  survey  and  from 
informal  conversations.  The  survey  was  developed  by  adapting  the  online  questionnaires  currently  in  use  in  two 
other  Web-based  educational  technology  courses  to  assess  students  attitudes  and  course  quality  (Montgomerie  & 
Harapnuik,  1997;  Szabo  et  al.,  1997). 

Initial  results  indicate  that  we  have  largely  managed  to  combine  the  best  of  these  two  very  different  worlds 
(both  traditional  F2F  instruction  and  individualized  Internet-based  instruction).  Students  have  appreciated  the 
personal  support  offered  by  the  traditional  component  of  the  course  (particularly  computer  novices)  and  the 
flexibility  and  access  to  resources  offered  by  the  Web-based  component.  Hopefully,  this  course  models  how  the 
needs  of  a wide  range  of  students  can  be  answered  by  supplementing  traditional  teaching  methods  with  an 
individualized,  self-paced  component.  For  example,  some  students  take  the  website  and  run  with  it,  others  like  the 
instructor  to  accompany  them  on  a slower  walk  through  the  course  content.  This  course  also  demonstrates  how  a 
website  can  dynamically  provide  suggested  resources  to  students  and  therefore  models  a WMI  approach  to  learning 
and  teaching.  Student  feedback  indicates  that  the  constructive,  collaborative,  student-centered  nature  of  this  course 
has  proved  to  be  a very  engaging  learning  environment.  There  appears  to  be  support  for  the  idea  that  the 
incorporation  of  CMC  into  EDPY  485  furthers  the  Mindtools  cause  by  providing  teachers  with  a glimpse  of  how 
WMI  technology  can  contribute  to  the  creation  of  engaging  and  relevant  learning  environments. 
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Background 

In  1995  the  School  of  Post  Compulsory  Education  and  Training  at  the  University  of  Greenwich  piloted  the  use 
of  Computer  Mediated  Communication  (CMC)  as  an  Enterprise  in  Higher  Education  project  in  support  of  the 
In-Service  Distance  Certificate  in  Education  (Lewis,  T.,  Gould,  M.,  Ryan,  M.,  1997),  The  reasons  for  wanting  to 
use  CMC  were  to  provide  students  with  regular  access  to  their  peer  group  and  tutors  through  conferencing  and 
e-mail.  A virtual  learning  environment  was  created  using  the  groupware  product,  Lotus  Notes  (Ryan  and 
Culwick,  1997).  A customised  desktop  was  created  which  included  a set  of  virtual  classrooms  for  each  of  the 
units  of  study,  an  electronic  resource  centre  and  student  common  room.  We  believed,  and  it  was  subsequently 
confirmed,  that  improved  access  would  reduce  the  feelings  of  isolation  often  associated  with  distance 
programmes  and  provide  opportunities  for  peer  exchange  and  enhanced  tutor-student  interaction  (Singletary  and 
Anderson,  1995).  Following  the  success  of  this  pilot,  the  use  of  CMC  as  a support  tool  was  offered  to  any 
distance  student  who  had  the  necessary  equipment  and  computer  literacy  skills. 

It  is  our  experience  that  those  students  and  staff  who  are  aware  of  and  exploit  the  technology  find  it  rewarding 
and  beneficial  to  both  their  learning  and  teaching.  If  the  use  of  information  and  communication  technologies 
(ICTs)  is  integrated  into  curriculum  thinking,  planning  and  delivery  from  the  start  then  we  believe  it  is  likely 
to  be  beneficial.  Whilst  providing  a highly  structured  and  integrated  learning  environment,  a Notes  desktop  did 
not  seem  to  be  an  appropriate  technology  when  potentially  large  numbers  of  students  (exceeding  a thousand) 
would  need  to  be  supported  within  the  newly  validated  Certificate  in  Education  programme. 


The  New  Certificate  in  Education 

This  new  programme  includes  students  who  are  studying  a common  set  of  units  in  full-time,  part-time  and 
distance  modes.  The  majority  of  materials  are  print  based  and  form  the  core  of  both  content  and  mediation  for 
all  modes  of  study.  We  believe  that  tutor  and  peer  support  is  fundamental  to  all  teacher  education  programmes 
and  ICTs  are  known  to  be  effective  delivery  methods.  It  is  important  that  students,  tutors  and  mentors,  remain 
up-to-date  in  their  knowledge  of  relevant  policies,  practices  and  uses  of  ICTs  in  education  and  training.  One 
way  of  addressing  this  is  to  integrate  the  use  of  ICTs,  including  hybrid  CD-ROM  technology,  into  the  delivery 
of  the  new  certificate  programme. 


Proposed  Structure  of  On-Line  Campus  (OLC) 

Building  upon  our  earlier  success  in  using  Notes,  a web-based,  virtual  campus  has  been  established  to  support 
all  students  irrespective  of  mode  of  study  consisting  of: 

• a Discussion  Forum, 

• Electronic  Resource  Centre, 

• Student  Common  Room. 


The  Discussion  Forum  allows  any  registered  student  to  pose  a question,  share  an  experience  or  ask  for 
guidance.  If  they  wish,  tutors  may  set  up  activities  to  support  the  delivery  of  their  unit  that  can  then  be  accessed 
by  students  irrespective  of  mode  (providing  they  have  the  skills/equipment).  The  Resource  Centre  can  be  used 
to  store  copies  of  files  created  by  tutors  in  a range  of  software  applications,  (enabling  easy  access  by  both 
students  and  colleagues),  and  links  to  other  web  sites  and  OLC  pages  can  be  easily  created  for  the  benefit  of  all 
(Rohfeld  and  Hiemstra,  1995).  The  concept  of  a Common  Room  has  been  established  as  benefiting  distance 
students  and  can  provide  opportunities  for  virtual,  social  interaction  between  students  irrespective  of  mode, 
time  or  place  (Mason,  1993;  Moore,  1989). 


One  of  the  principles  established  by  our  use  of  CMC  in  support  of  Continuing  Professional  Development 
(CPD)  programmes,  is  the  need  for  a range  of  skills  to  be  employed  when  designing  and  using  an  on-line 
environment.  Consequently,  this  project  draws  upon  the  expertise  of  the  Notes  developer,  the  curriculum 
designer  and  the  subject  specialist.  It  is  also  intended  that  students,  who  are  themselves  a valuable  resource 
(Knowles,  1 990),  will  contribute  dynamically  to  the  content,  nature  and  role  of  the  OLC. 


Training  and  Development 

The  need  for  staff  and  students  to  have  the  appropriate  skills  and  confidence  in  order  to  exploit  technology  is 
well  documented.  A number  of  staff  have  experience  of  working  with  Lotus  Notes  and  a growing  number  are 
able  to  ‘surf  the  web.  A Notes  desktop  provides  more  features  and  greater  control  over  the  placing  of 
documents  and  management  of  the  discussion  forum.  It  is  envisaged  that  during  this  coming  academic  year  all 
unit  co-ordinators  will  be  equipped  with  and  supported  in  using  Notes,  enabling  them  to  be  more  effective 
managers  of  and  contributors  to  the  OLC.  Other  staff  and  all  students  will  access  the  OLC  via  a web  browser. 
Training  in  the  use  of  the  web  is  already  available  to  all  centrally  which  will  remove  the  problems  previously 
associated  with  inducting  even  small  numbers  of  students  into  the  use  of  the  Notes  desktop. 


What  Next? 

A major  reason  for  employing  an  OLC  at  this  time  is  that  it  allows  us  to  promote  the  appropriate  use  of  ICTs 
in  the  delivery  of  our  teacher  education  programmes  and  to  support  colleagues  in  exploiting  its  potential.  It  is 
likely  that  IT  competence  will  be  a core  component  of  future  teacher  accreditation  schemes  and  we  should 
prepare  ourselves,  our  students  and  programmes  to  incorporate  the  technology  strategically.  As  a relatively 
large  scale  pilot  project  we  are  interested  in  examining  such  factors  as: 

• the  extent  to  which  the  learning  community  exploits  the  technology 

• whether  the  mode  of  ‘attendance’  determines  the  use  of  the  OLC 

• what  ‘additionality’  is  brought  to  the  certificate  programme 

• its  cost  effectiveness 

• the  merits  of  Lotus  Domino  as  a web  development  tool 

• the  relationship  between  hybrid  CD-ROM  products  and  the  OLC 

Answers  to  these  questions  will  help  us  to  plan  a longer-term  strategy  for  the  use  of  the  OLC  in  support  of 
other  programmes  from  certificate  to  doctorate  studies.  This  will  form  part  of  an  integrated  approach  to  the  use 
of  ICTs  in  teaching  and  learning  which  ignores  time  and  distance  (Berge  and  Collins,  1995)  and  focuses  on 
quality  and  value. 
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This  document  is  a very  brief  summary  of  a larger  paper  and  presentation.  It  is  not  possible  to  discuss 
the  issues  presented  two  pages,  so  we  will  restrict  ourselves  to  providing  an  annotated  outline  of  the  topics  here, 
and  invite  you  to  download  the  larger  document.  More  information  about  the  hypertextbook  Visual  Design  for 
Instructional  Multimedia,  including  previews  of  most  sections  of  the  book,  is  available  from 
http://members.home.net:80/m4.multimedia/.  A complete  version  of  the  paper,  presented  earlier  this  year,  is 
available  for  downloading  as  a PDF  file  [190  KB]  from 

www.extension.usask.ca/Papers/Misanchuk/EDMEDIA99/EDMEDIA99paper.pdf 

Large-scale  hypertext  implementations  designed  to  provide  information  are  common  enough 
(especially  as  websites);  however,  large-scale  instructional  equivalents  are,  at  this  juncture,  more  rare.  The 
complete  paper  and  presentation  include  the  following: 

• Description  of  the  design  and  development  of  a CD-ROM-based  hypertextbook,  entitled  Visual  Design  for 
Instructional  Multimedia,  which  consists  of  more  than  3,100  HTML,  JPEG,  GIF,  and  MOV  files 

• Morphology  of  the  project  through  several  stages,  highlighting  issues  arising  from  developing  the  product 
with  HTML  rather  than  with  a more  hospitable  authoring  program 

• Description  of  a number  of  things  we  learned  from  the  "school  of  hard  knocks" — some  profound,  some 
mundane — in  the  hope  that  by  sensitizing  you  to  some  of  the  problems  we  encountered  and  decisions  we 
made  (and  sometimes  re-made),  we  can  help  you  avoid  some  of  the  pitfalls  we  encountered. 


Outline  of  the  Paper  and  Presentation 


Need  for  the  Book 
Content  and  Scope. 

Structure 

The  “modified  lineaF’  design 
Mainstream  text  and  diversions 


1 Or  fr 
-L  ^ O O 


• reference  citations 

• examples 

• elaborations 

• navigation  links 
Diversions  from  diversions 
Orientation  and  location  cues 

Process 

Getting  started 
The  distance  factor 
Working  together  and  apart 
The  overall  process 

Work  Methods 

Pages  on  the  Fly 
Document  First,  Pages  Later 
Outline,  Just  Like  Mr.  Hankel  Says 
The  Traditional  Book  Publishing  Approach 

Design  Issues 

Settling  on  a Visual  Design 

The  impact  of  technology  on  design  options  and  the  design  process. 
The  impact  of  usability  testing  and  technology  on  navigation  design. 
Navigation  Schemes  and  Screen  Geography 

Hypertext  advantages  with  simple  navigation. 

Linking  to  the  same  page  from  different  locations  in  the  text 
Image  map  menus  vs.  frames 
Cues  for  different  types  of  links 
Academic  Writing  and  HTML 
Formality  of  language. 

Editorial  issues. 

Web  or  CD-ROM  versus  Print  for  Distribution 
Why  not  a printed  book? 

As  long  as  it’s  electronic,  why  make  it  essentially  linear? 

Why  HTML  instead  of  another  authoring  environment? 

Why  then,  a CD-ROM  and  not  the  Web? 

Expert  Review  and  Usability  Tests 
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The  current  education  scenario  appears  to  place  the  Internet  as  a timely  tool  for  educators  who  are  interested  to  reform  education 
delivery  (McLain  & DiStefano,  1995),  The  technology  sophistication,  compatibility  and  user  friendliness  to  education 
environment  makes  education  more  accessible  from  distance.  In  fact  learners  may  receive  academic  degrees  without  stepping  on 
university  ground.  In  addition,  Kinner  and  Coombs  (1995)  found  that  utilizing  computers  in  the  classroom  can  reduce 
communication  barriers  and  facilitate  interaction  among  learners  that  are  inhibited  in  the  traditional  classroom.  However,  the 
differences  between  classroom  instruction  and  computer-based  learning  are  very  significant  (Steinberg,  1991),  therefore 
educators  should  address  the  issue  very  carefully.  If  it  is  planned,  designed  and  executed  correctly,  and  learning  can  be  placed  in 
a meaningful  context,  then  the  learners  will  be  actively  engaged  in  the  learning  situation  (Bridges,  1 992), 

This  paper  will  report  concerns  for  conducting  on-line  courses,  including  instructional  format,  methods  of  instruction,  methods  of 
interactions,  methods  of  communication,  instructional  media,  and  participants.  It  will  address  some  of  the  concerns  when 
conducting  an  on-line  course.  The  intention  is  to  share  the  information,  recognize  and  realize  the  challenges  for  professionals  to 
be  better  prepared  for  the  electronic  learning  environment. 

Instructional  Format 

Using  the  traditional  class  instructional  format  such  as  preparation,  presentation,  application,  and  summary  may  be  considered  for 
an  on-line  course.  However,  its  affect  on  student’s  concentration,  motivation,  thought,  mastery,  and  comprehension  (Verduin 
&Clark,  1996)  could  be  a big  challenge  without  the  right  strategies  to  transfer  classroom  activities  into  on-line  activities.  Among 
these  factors,  on-line  educators  are  most  concerned  about  the  assessment  process,  especially  when  looked  at  in  the  context  of 
traditional  classroom  environment  or  exam  oriented  learning  formats  regarding  the  validity  of  student’s  works. 

Methods  of  Instruction 

Brookfield  (1985)  contended  class  facilitation  will  promote  greater  student  involvement  in  the  learning  process  when  compared 
to  lecture  method.  However,  in  an  on-line  environment  the  method  can  be  very  challenging,  especially  when  students  have 
different  learning  styles.  The  factors  that  determine  how,  what,  and  when  to  interact  are  the  most  critical  factors  to  be  addressed 
in  order  for  the  teaching  and  learning  to  be  efficient  and  effective. 

Methods  of  Interaction 

In  an  on-line  environment  the  discussion,  group  work,  and  question-answer  sessions  can  be  accomplished  by  means  of  both 
asynchronous  and  synchronous  modes,  Anderson  (1993)  discussed  these  factors  for  effective,  successful  teaching  and  learning  in 
a classroom.  However,  the  absence  of  physical  presence  in  an  on-line  environment  makes  the  execution  of  such  factors  more 
challenging,  especially  when  much  of  the  success  of  the  on-line  delivery  method  using  these  factors  depend  on  both  the 
capability  of  computer  system  and  software,  and  the  ability  of  the  instructor  to  manage  the  class  effectively. 

Method  of  Communication 

In  a traditional  classroom  visual,  verbal  and  non-verbal  communications  are  readily  integrated  in  the  teaching  and  learning 
process  to  express  the  meanings  of  actions,  words,  and  texts  (James  & Blank,  1993),  Cochenour  (1994),  however,  explained  the 
difficulty  to  integrate  these  factors  in  electronic  environment  such  as  on-line  course.  There  are  many  variables  that  have  to  be 
considered  for  the  integration  to  be  successful  such  as  capability  of  system  networks,  students’  and  instructors’  computers, 
software,  and  course  design. 

Guest  Speakers 


In  a traditional  classroom  instructor,  students  and  guest  speakers  are  common  participants  (James  & Blank,  1993).  On  the 
contrary,  in  on-line  environment,  guest  speakers  are  absent.  Among  the  reasons  are,  they  have  to  face  the  challenges  of  learning 
computer  operation  skills,  managing  the  software,  availability  of  time,  log  on  process  and  so  on.  Their  involvement  is  more 
difficult  when  the  primary  communication  strategy  is  mainly  based  on  text  like  the  asynchronous  mode  of  delivery.  The  lack  of 
or  no  physical  contact,  facial  expression,  and  social  cues  that  could  promote  normal  relationship  as  in  the  traditional  classroom 
will  contribute  further  to  the  absence  of  the  guest  speaker  in  an  on-line  environment. 

Instructional  Media 

Miller  and  Clouse  (1993-1994)  contended  the  role  of  technology  is  significant  to  disseminate  primary  course  information  in  the 
education  settings.  It  can  be  applied  as  the  main  instmctional  strategy  or  as  a supplement  to  enhance  learning  and  teaching.  In  an 
on-line  course  the  use  of  technology  is  a must.  However,  finding  the  right  technological  tool  to  enhance  learning  and  teaching 
can  be  very  challenging.  Among  the  variables  that  have  to  be  considered  in  the  selection  process  are  the  capabilities  of  the 
system  networks,  end  user  machines,  compatibility  and  accessibility  of  software,  support  personnel,  and  policy.  These  will 
detemiine  to  what  extent  media  can  be  used  to  conduct  communication,  interaction,  and  instruction. 

Conclusion 

This  shift  of  new  education  paradigm  with  regard  to  the  advance  of  technology  is  altering  educators’  perceptions  about  teaching 
and  learning  strategies.  Many  educators  are  involved  with  the  new  paradigm  of  teaching.  The  concern  of  (1)  using  appropriate 
instructional  format,  methods  of  instruction,  methods  of  interactions,  methods  of  communication,  and  instructional  media,  (2) 
encouraging  participants’  involvement,  and  (3)  requiring  the  correct  strategies  should  be  the  primary  issues  for  on-line  educators. 
Some  questions  have  been  answered  but  many  are  still  unsolved.  Educators  should  share  their  ideas  with  their  colleagues  and 
continue  the  debate  for  a better  practice. 
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The  One-Minute  Paper  (OMP)  has  been  used  to  generate  feedback  to  instructors  for  some  time 
(Harwood  1996;  Ludwig  1995;  Kloss  1993).  Traditionally  this  has  involved  having  students  spend  a few 
minutes  at  the  end  of  a class  session  writing  down  what  was  understood  and  what  was  still  unclear  from  a 
particular  classroom  session.  Bryan  et  al.  (1993)  reported  that  instructors  had  favorable  perceptions  about  the 
timely  feedback  of  OMP's  obtained  from  accounting  courses.  In  addition  to  providing  feedback,  the  cognitive 
effort  that  is  required  by  students  to  complete  the  OMP  has  been  shown  to  positively  effect  learning  under 
certain  conditions.  Bryan  et  al.  (1993)  reported  that  students  from  various  accounting  classes  reported  a 
perceived  augmentation  to  learning  as  a result  of  the  OMP. 

Although  the  empirical  effectiveness  of  the  OMP  are  sparse  some  studies  do  indicate  a beneficial 
relationship  between  the  preparation  of  an  OMP  and  learning.  Aimer  et  al.  (1998)  has  shown  that  accounting 
students  completing  an  OMP  performed  significantly  better  on  quizzes  than  students  who  did  not  complete  the 
OMP^'l  Similar  results  have  been  found  with  economic  students  (Chizmar  and  Ostrosky  1998)  studying 
microeconomics. 

The  OMP  is  typically  performed  in  class,  at  the  end  of  a lecture.  This  research  will  examine  what 
effect  if  any  will  endure  by  having  students  complete  their  OMP  electronically  (or  electronic  OMP)  some  time 
after  the  class  lecture  has  finished  Having  students  complete  the  OMP  electronically  can  change  the  nature  of 
the  OMP  in  several  ways.  First,  if  students  do  no  have  access  to  a computer  in  the  classroom  for  which  the 
OMP  is  to  be  completed  the  benefit  of  the  immediate  recall  of  the  class  contents  may  be  lost.  However,  an 
added  benefit  of  completing  the  electronic  OMP  (or  e-OMP)  is  that  the  students'  e-OMP  can  be  made  available 
to  other  students,  thereby  providing  students  with  feedback  similar  to  what  an  instructor  would  receive  from  a 
traditional  OMP.  In  addition,  the  e-OMP  lends  itself  to  the  possibility  of  having  students  provide  comments 
and/or  explanations  to  other  student's  e-OMP  about  unresolved  questions.  In  other  words  the  e-OMP  can 
become  a virtual  component  of  class  discussions  in  which  students  from  a class  can  post  questions  about  areas 
that  they  do  not  understand  and  have  their  fellow  colleagues  help  them  by  providing  explanations. 

Numerous  questions  stemming  from  the  use  of  the  e-OMP  can  be  formed.  The  first,  will  the  cognitive 
effort  used  in  completing  the  e-OMP  result  in  better  performance?  Also,  will  the  e-OMP  completed  up  to  a day 
after  a class  session,  be  as  useful  as  the  e-OMP  completely  shortly  after  class?  The  second,  what  effects  will 
collaboration  have  on  learning  within  the  OMP  framework?  That  is,  will  students  who  have  responded  to  an  e- 
OMP  with  helpful  suggestions  outperform  students  who  have  only  completed  the  e-OMP  without  responding  to 
any  fellow  students?  Third,  will  the  e-OMP  have  distinct  effects  depending  on  exam  type  (multiple-choice, 
problem  solving  or  essay)? 


Experimental  Procedure 

The  e-OMP  data  presented  here  was  gathered  during  the  fall  1998  semester  from  54  MBA  students^^l 
The  e-OMP  was  implemented  using  Learning  Space  and  Lotus  Notes.  Students  were  told  that  the  posting  of  e- 


Further  analysis  by  Aimer  et  al.  (1998)  indicated  that  the  benefit  of  the  OMP  was  only  significant  for  essay-type  quizzes 
vs.  multiple-choice  type  quizzes. 

The  students  were  from  two  separate  Financial  Statement  Analysis  classes.  One  of  the  groups  was  from  an  off-site  MBA 
program.  Analysis  of  the  two  groups  indicated  no  significant  differences  between  them  so  the  results  from  the  two  groups 
are  combined. 
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OMP  would  be  anonymous,  but  that  if  any  inappropriate  comments  were  posted  the  person  responsible  could  be 
determined.  The  e-OMP  contained  the  following  questions:  \)  lam  most  confused  about  the  following  point  that  was 
brought  up  in  class  tonight;  2)  If  there  is  one  thing  I completely  understand  now,  it  is  the  following:;  3)  I would  like  to  learn 
more  about  the  following  topic:;  4)  Finally,  please  rate  your  perception  of  understanding  for  this  weeks  materials:  Got  Itl, 
On  my  way  to  getting  it,  or  Oh  boy  Fm  LosPK  The  response  to  the  e-OMP  was  worded  as  follows:  I believe  that  if 
you  consider  the  following  points,  it  might  help  clear  up  your  confusion.  The  response  to  the  e-OMP  was  not 
anonymous  and  was  used  in  the  determination  of  the  overall  participation  grade  for  the  student. 


Data  Analysis 

To  examine  the  relationship  between  the  e-OMP  and  student  performance  the  students  overall  grade 
(scored  on  a 1000  point  scale)  was  divided  into  its  component  parts:  exams  and  mini-cases.  Pearson 
correlations  were  run  to  determine  any  relationship  between  these  components  (and  their  totals)  with  either  the 
e-OMP  or  the  responses  to  the  e-OMP.  Results  indicate  a positive  relationship  between  the  e-OMP  and  students 
total  overall  grade  (.378,  p<.005).  With  respect  to  the  e-OMP  response,  results  indicate  a positive  relationship 
between  the  response  and  students  overall  score^"^^  (.569,  p<.000),  the  total  exam  score  (.349,  p<.01),  and  the 
combined  exam  and  mini-case  score^^^  (.389,  p<.004). 

Next  ANOVA  was  used  to  examine  if  the  differences  among  the  students'  performance  could  be 
attributed  (in  part)  to  the  e-OMP  or  responses  to  the  e-OMP^^l  The  results  indicated  that  even  though  a 
significant  positive  relationship  existed  between  the  use  of  the  e-OMP  and  overall  student  performance  there 
were  not  any  significant  differences  that  could  be  ascribed  to  the  number  of  e-OMP  completed.  However,  the 
results  indicate  a significant  difference  between  the  number  of  responses  to  the  e-OMP  and  the  total  exam  score 
(F=2.382,  p<.064),  the  combined  exam  and  mini-case  score  (F=2.934,  p<.03)  and  the  overall  score  (F=7.619, 
p<.000).  Post-hoc  analysis  on  the  overall  score  indicated  a significant  difference  between  0 responses  and  any 
response  (either  1,2,3  or  5 responses).  Of  practical  importance  the  differences  were  as  follows:  5 responses: 
122  points  (p<.001),  3 responses:  103  points  (p<.000),  2 responses:  63  points  (p<.039),  and  1 response:  51 
points  (p<.034).  Post-hoc  analysis  of  the  combined  exam  and  mini-case  score  also  indicated  a significant 
difference  between  0,  3 and  5 responses.  These  differences  were  as  follows:  5 responses:  63  points  (p<031);  3 
responses:  51  points  (p<.019). 
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This  last  question  was  implemented  via  a check-box  on  the  e-OMP. 

The  overall  score  includes  points  for  participation,  which  were  partially  based  on  completion  of  e-OMP  as  well  as  the 
responses  to  the  e-OMP. 

This  score  does  not  include  the  participation  points. 

The  mean  and  standard  deviation  of  the  e-OMP  were  6.9  and  3.1  respectively.  The  minimum  number  of  student 
responses  to  the  e-OMP  was  0 and  the  maximum  was  5. 
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Introduction 

In  a recently  developed  series  of  training  materials  designed  to  help  school  personnel  work  with 
youngsters  with  behavioral  problems.  Teacher  Problem-Solving  Skills  in  Behavioral  Disorders,  interactive 
multimedia  case  studies  were  combined  with  electronic  performance  support  system  (EPSS)  tools  for  teachers. 
Funded  in  part  by  a grant  from  the  U.S.  Department  of  Education  (Semrau  & Fitzgerald,  1993-1997),  the  goal 
of  the  project  was  to  develop  interactive  training  programs  to  enhance  problem-solving  skills  of  teachers  to 
better  serve  children  with  social,  emotional,  and  behavioral  difficulties.  Through  authentic,  case-based 
problems,  users  learn  to  plan  therapeutic  instruction,  design  behavioral  interventions,  and  create  cognitive- 
behavioral  intervention  materials  to  teach  children  self-management  procedures  (Fitzgerald,  Standi fer,  & 
Semrau,  1998). 

To  supplement  the  training  materials  for  educators,  initiatives  are  underway  which  attempt  to  extend  the 
real-world  use  of  the  training  materials  by  developing  a companion  series  of  EPSS  software  programs  for 
children  to  use  in  classroom  settings  (Fitzgerald  & Semrau  1998).  The  software  programs  for  kids  are 
compatible  with  the  cognitive-behavioral  strategies  in  the  instructional  materials  for  teachers.  The  self- 
management  materials  provide  a set  of  electronic  tools  that  children  and  youth  can  use  to  create  their  own 
behavior  plans  for  self-management.  By  helping  children  take  responsibility  for  their  own  behavior,  internal 
controls  for  behavior  are  established  and  the  need  for  external  control  of  behavior  are  reduced. 

Computer-based  training  and  support  mechanisms  are  an  innovative  approach  for  helping  students  gain 
control  over  personal  behaviors.  Although  there  are  limited  data  on  the  use  of  computer-based  instruction  to 
support  behavior  change  in  students  to  date,  research  results  are  promising.  Fitzgerald  and  Werner  (1996) 
reported  success  with  a computerized  verbal  mediation  essay  as  a cognitive  retraining  procedure  to  assist  a 
student  with  significant  behavioral  disorders  in  changing  his  behavior;  the  computerized  essay  provided 
consistent  practice  and  focused  the  child's  attention  and  thoughts  on  behavioral  choices  and  consequences.  In 
another  case  study,  the  same  researchers  reported  a procedure  in  which  software  templates  were  developed  for  a 
student  to  create  self-monitoring  materials.  This  study  was  the  pre-cursor  to  this  work  to  develop  and 
investigate  the  use  of  EPSS  tools  with  children. 

Instructional  Design  of  Program 

The  EPSS  programs  are  designed  as  easy-to-use  templates  that  can  be  individualized  by  children  and/or 
their  teachers.  To  use  the  templates,  the  child  simply  clicks  on  "hot  words"  on  the  template  form  to  enter 
personalized  content  and  then  print  it  for  use  in  the  classroom.  The  program  automatically  enters  the  child’s 
name,  the  date,  and  establishes  a datapath  for  recordkeeping  purposes.  When  complete,  the  templates  will  be 
appropriate  for  youngsters  at  two  levels:  elementary  and  middle  school/junior  high.  The  programs  are  created 
with  Authorware  4 and  operate  on  both  Macintosh  and  Windows  platforms. 
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KidTools  incorporate  several  literacy-related  design  features  to  make  them  suitable  for  children’s  use. 
The  teacher  tools  were  modified,  making  the  templates  child  friendly  and  incorporating  the  use  of  natural 
language  of  children.  Graphic  characters  were  added  to  serve  as  “guides”  to  the  different  tools.  Audio  support 
now  supplements  simplified  text  instructions. 

The  following  graphic  displays  an  example  of  a KidTool.  This  is  a point  card  for  monitoring  success  on 
three  behaviors.  The  child  selects  pictures  from  a graphics  library  to  represent  the  behaviors.  Each  behavior  is 
given  a simple  descriptive  name.  The  monitoring  grid  is  printed  on  the  card  to  allow  the  child  to  monitor 
success  on  the  behaviors  four  times  during  the  school  day. 


Formative  Evaluation 

Teachers  from  three  elementary  schools  in  rural  and  urban  areas  in  Missouri  participated  in  designing 
and  evaluating  KidTools.  A small  cadre  of  teachers  in  each  building  shared  the  materials  with  interested 
teachers  and  contributed  suggestions  for  template  adaptations  as  well  as  additional  forms  needed  in  their 
schools.  Project  staff  observed  children's  use  of  the  software  using  think-aloud  techniques.  User  datapath 
records  were  examined  to  determine  common  template  usage  and  to  gain  samples  of  natural  language  from 
children.  At  the  close  of  the  school  year,  interviews  were  conducted  with  teachers  who  implemented  the 
materials.  Based  on  formative  evaluation  data,  KidTools  will  be  revised  during  the  summer  months  for  full 
implementation  in  the  schools  during  the  1999-2000  school  year.  Recommendations  will  be  forthcoming 
regarding  effective  implementation  of  EPSS  tools  with  elementary-age  youngsters. 
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The  Wake  Forest  Strategic  Plan  for  Technology 


David  G.  Brown,  International  Center  for  Computer  Enhanced  Learning 
Wake  Forest  University  E-mail  = brOWn@wfu.edu 


All  incoming  Wake  Forest  University  freshmen  students  receive  new  laptop  computers 
upon  entering,  exchange  them  for  a more  powerful  laptop  upon  returning  for  their  third 
year,  and  take  their  second  computer  with  them  upon  graduation.  In  Fall  1999  the 
computer  will  be  the  IBM  Thinkpad  390  with  Pentium  II,  333  Mhz,  6 GB  Hardrive, 
1 28MB  Ram,  4.1”  active  matrix  screen,  DVGA  video,  ethemet,  and  CD  Rom.  Standard 
software  will  include  Windows  98,  MS  Office  Professional  97,  Netscopae 
4.5,Dreamweaver,  SPSS  9.0,  and  Maple  V5.1. 

All  Wake  Forest  faculty  and  staff  are  on  a two-year  refresh  cycle.  All  students,  faculty, 
and  staff  therefore  have  24-hour,  365-day,  worldwide  access  to  one  of  two  standard 
computers  with  standard  software  packages.  Standardization  enables  round-the-clock 
staffing  of  knowledgeable  help  desks,  coaching  by  roommates  and  office  neighbors, 
loaner  pools,  and  (most  meaningfully)  teaching  with  the  knowledge  that  all  students  have 
access  to  powerful  electronic  tools.  The  methods  students  learn  for  one  class  (e.g. 
creation  of  a web  page)  are  transferable  to  other  classes  and  to  all  types  of  community 
activities  (e.g.  fraternities  and  special  interest  clubs). 

This  ubiquitous  computer  environment  has  transformed  the  quantity,  quality,  and 
character  of  teaching,  learning,  and  campus  life.  Frequent  surveys  confirm  that  more 
students  are  more  intensely  involved  with  more  subgroups  over  more  months,  even  when 
subgroup  members  may  be  farther  apart.  Surveys  show  that  faculty  are  in  touch  with 
more  students  more  frequently,  as  well  as  with  more  professional  colleagues  in  more 
countries  and  for  more  years.  The  capacity  for  network  conferencing,  for  sharing 
videotapes,  for  synchronous  and  asynchronous  threaded  conversations,  for  sophisticated 
archiving  and  indexing  of  messages,  and  for  random  access  of  images  now  contributes  to 
a much,  much  richer  community  of  learners.  Students  in  a freshman  writing  course 
exchange  rough  draft  essays  with  a similar  class  in  Canada.  Shakespeare  students 
participate  in  fortnightly,  live,  two-way  video  conferences  with  the  archivists  at 
London’s  Globe  theatre.  Business  students  view  and  review  cyber-lectures  so  that  class 
time  can  be  reserved  for  discussion  and  analysis. 

Because  Wake  Forest  was  one  of  the  first  to  implement  a ubiquitous  computing  program, 
we  have  been  visited  by  individuals  and  delegations  from  over  300  U.S.  colleges  and 
universities  and  over  40  colleges  and  universities  in  other  countries.  In  Winston-Salem, 
Southeast  Asia,  and  Europe  we  have  been  sharing  "the  Wake  Forest  Story."  Our  intent  is 
to  catalyze  other  institutions  to  implement  ubiquitous  computing  for  the  benefit  of  their 
students.  This  story  includes  a multiyear  financial  plan,  the  steps  followed  toward 
achieving  faculty  and  trustee  votes  for  a standardized  computer  initiative,  faculty 
development  initiatives,  printer  and  network  strategies,  etc. 
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In  COMPUTER  ENHANCED  LEARNING  nittp://iccel.wfu.edu)  nineteen  leaders  of  the 
implementation  have  collected  this  list  of  lessons  learned: 


^ PC’s  are  only  10%  of  the  Challenge  (support/networks/policies/train/expose) 

^ Most  sunk  costs  can  be  ignored 

^ Expectations  need  management 

^ Develop  a comprehensive  plan  first,  and  quickly  match  it  with  a multiyear  financial  plan 

^ Standardization  pays  rewards  well  beyond  those  anticipated;  non-standard  configurations  require  3- 
4 times  support 

^ Students/Faculty  want  specific  computer  training  that  is  centered  around  a task-at-hand;  general 
classes  don’t  work  well 

^ Be  prepared  to  outsource  challenges 

^ Reliability  is  critical,  especially  the  Help  Desk 

^ Provide  academic  units  staff  of  their  own  & plenty  of  equipment  without  hassle 
^ Defer  wiring  every  seat  and  every  classroom 
^ Provide  printers  to  every  student 
^ Adopt  a common  course  shell 
^ Avoid  mandating  the  use  of  computers  by  all  faculty 
^ Insist  upon  at  least  a three  year  refresh  rate 
^ Improve  communications  weekly;  rumors  fly  faster 

^ Spread  the  gains  from  & ownership  of  innovation  throughout  all  units 

This  beginning  list  is  cited  for  the  purpose  of  catalyzing  a sharing  of  lessons  learned  by  other  early 
implementers  of  ubiquitous  computing,  and  thereby  enabling  others  to  avoid  some  of  the  missteps  others 
have  made. 


ERIC 


Faculty  Development  Strategies 


David  G.  Brown,  International  Center  for  Computer  Enhanced  Learning 
Wake  Forest  University,  USA.  Email  = brown@wfu.edu 


One  of  the  last,  yet  most  important,  stops  along  the  road  to  utilizing  powerful  and  newly 
available  multimedia  technologies  is  motivating  and  training  the  nation’s  faculty.  Now, 
more  than  at  any  time  in  the  last  fifty  years,  the  academic  disciplines  are  focused  upon 
quality  teaching.  Faculty  must  be  encouraged  to  start  with  their  educational  objectives 
and  their  successful  learning  experiences,  and  then  consider  how  technology  might  allow 
the  pursuit  of  these  objectives  in  more  effective  ways.  In  this  “teachable  moment”  we 
must  efficiently  provide  faculty  with  the  ability  to  use  an  array  of  new  tools — from 
downloading  clips  from  the  internet,  to  accompanying  lecture-discussions  with 
PowerPoint  slides  and  creating  animated  videos  to  illustrate  difficult-to-understand 
concepts. 

The  infrastructure  must  precede  everything.  A first  challenge  faced  by  faculty 
development  specialists  is  assuring  access  to  appropriate  computers.  To  be  fair,  all 
students  must  have  easy  and  equal  access,  and  faculty  will  insist  upon  fairness  before 
they  ask  students  to  rely  upon  computers.  Equally  important  is  a reliable  network,  for 
without  it  faculty  will  have  to  prepare  failsafe  assignments  which  means  double-work. 
Classtime  is  too  precious  to  risk  losing. 

A second  challenge  is  assuring  access  to  appropriate  advice.  Required  is  “just  in  time” 
training  when  there  is  a “need  to  know.”  Alternatives  include  subsidized  summer 
workshops,  special  classes  available  upon  demand,  computer-based  training  modules, 
well-trained  help  desk  personnel,  course  designers,  and  single-purpose  seminars.  At 
Wake  Forest  we  have  come  to  rely  most  heavily  upon  STARS  (well-trained  students 
assigned  for  semester-long  periods  to  help  individual  faculty  members  incorporate 
computer  enhancements  into  their  teaching),  Academic  Computer  Specialist  (Ph.D.’s  in 
the  discipline  hired  by  departments  to  spend  full  time  helping  departmental  colleagues 
design  and  implement  computer  enhancements  in  their  teaching  and  research),  and  RTAs 
(trained  students  residing  in  residence  halls  and  available  for  face-to-face  consultation 
especially  in  the  evening).  . 

A third  challenge  is  motivating  faculty  to  make  the  time  to  learn  how  to  use  these  new 
tools.  While  assuring  faculty  that  they  will  not  be  penalized  for  deciding  against  the  use 
of  computer  enhancements  in  their  teaching,  multiple  programs  encouraging  exploration 
are  essential.  Some  of  the  most  successful  programs  are  release  time  grants,  summer 
fellowships,  technology  fairs,  iimovations  awards,  and  collecting  information  about 
teaching  iimovations  in  conjunction  with  promotion  and  tenure  decisions. 

A fourth  challenge  is  dissemination  strategies,  especially  sharing  information  about 
successful  adoptions.  Swap  and  share  seminars,  benchmarking  trips,  guest  lecturers, 
subsidy  for  attendance  at  national  computer  meetings,  computer-assisted  learning 


newsgroups,  and  “computer  tip  talks”  by  students  are  among  the  more  successful 
programs  for  dissemination. 

The  real  key  to  success  is  a faculty  culture  that  supports  experimentation  and  encourages 
each  faculty  to  do  his/her  best  for  the  students.  Such  a culture  is  most  likely  to  thrive 
when  most  faculty  are  using  technology  in  some  way,  and  when  colleagues  can  share 
knowledge. 


Nothing  is  m*ore  important  than  keeping  it  simple!  By  shortening  faculty  learning  curves 
and  standardizing  campus  practices,  widespread  adoption  is  greatly  accelerated. 
Specifically,  two  practices  are  advocated: 

1 . When  possible,  standardize.  Enable  all  students  and  faculty  to  use  the  same  machines, 
hardware,  and  software.  Friends  and  neighbors  will  teach  each  other.  Once  a crisis  is 
fixed  for  one  machine,  it  may  be  applied  to  all.  Once  a student  learns  how  to  use  the 
computer  in  one  class,  the  knowledge  is  relevant  for  the  next.  The  negative 
consequences  of  physical  breakdowns  can  be  minimized  by  backup  systems.  Training  is 
greatly  simplified. 

2.  Emphasize  “high  benefit-  low  cost”  uses  of  the  computer.  Specifically,  train  faculty 
to  use  the  computer  to  improve  communication,  both  instructor-student  and  student- 
student.  Encourage  the  use  of  group  e-mail,  URL  citations,  and  course  web  pages.  At 
first,  discourage  the  time-consuming  development  of  on-line  exercises  and  of  fancy 
visualizations.  That  can  come  later. 

The  nation’s  faculty  is  eager  to  do  well  by  students.  Strong  networks,  early  successes, 
and  standardized  systems  can  be  the  backbone  for  effective  dissemination  strategies.  It 
will  not  be  long  before  courses  without  computer  enhancements  will  be  as  unfamiliar  as 
courses  without  textbooks.  The  faculty  is  ready!  The  students  are  there! 
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Exploring  Curriculum  Delivery 


Yanlu  Xu 

Wrexham  Business  School 
North  East  Wales  Institute 
United  Kingdom 
xuy@newi.ac.uk 


Thrss  Psrsni6t6rs  of  Curriculuin  Delivery  System 

This  paper  reports  on  an  attempt  to  conceptualize  curriculum  delivery  within  the  framework  of  new  technology. 
When  we  analyze  the  differences  of  curriculum  delivery  in  a social  context,  we  see  there  are  three  main 
parameters  that  strongly  influence  the  curriculum  delivery  paradigm.  These  parameters  can  be  set  into  a social 
context  along  with  human  history  either  vertically  or  horizontally.  The  three  parameters  are  knowledge, 
pedagogy,  and  technology.  First  we  discuss  the  three  parameters  in  a vertical  time  dimension  which  represents 
human  history.  Knowledge  based  on  human  beings’  understanding  of  the  real  world  has  developed  as  different 
self-contained  disciplines,  such  as:  physics,  medical  science,  chemistry,  social  science,  engineering,  computer 
science.  The  different  disciplines  nowadays  have  much  more  to  offer  their  learners  than  in  the  past  due  to  the 
development  of  the  disciplines  themselves.  Knowledge  is  the  basis  of  any  curriculum  delivery  system.  It  is  the 
content  of  delivery  and  it  affects  the  richness  of  the  curriculum  delivery  system.  Another  parameter  is  pedagogy 
and  we  conclude  that  the  present  curriculum  delivery  system  in  education  puts  much  more  effort  into 
pedagogical  issues  than  before.  Educators  are  concerning  more  about  how  students  learn  and,  consequently, 
how  they  themselves  teach.  Many  institutions  have  set  up  instruction  design  centers  to  improve  and  enhance 
curriculum  delivery.  Pedagogy  has  been  paid  much  more  attention  as  the  discipline  has  matured.  This 
parameter  affects  the  flexibility  and  reflects  the  methodology  of  curriculum  delivery.  The  third  parameter  is 
technology.  This  has  had  considerable  influence  in  the  past  and  we  can  project  an  immense  future  impact  on 
curriculum  delivery.  Projects  such  as  China’s  TV  University  and  the  UK’s  Open  University  are  both  good 
examples  of  technology  bringing  new  ways  to  teach  and  learn.  Future  technology  will  allow  us  to  provide  a 
more  open  and  flexible  curriculum  deliveiy  system.  Now  we  consider  those  three  parameters  in  a horizontal 
dimension.  A horizontal  time  dimension  implies  the  same  point  in  history  and  the  same  society.  We  could  see 
that  different  institutions  might  offer  different  standard  curricula  at  the  same  historical  point.  Thus  Harvard 
University  might  offer  better  management  curricula  than  others  do.  In  other  words  even  if  we  consider  the  same 
historical  point,  institutions  have  their  own  practice  in  the  context  of  the  three  parameters.  The  knowledge 
parameter  reflects  the  capabilities  and  experience  of  staff  The  technology  parameter  represents  the  extent  of 
use  of  teaching  technology.  This  can  include  such  things  as  multimedia  curriculum  deliveiy,  online  Internet 
platform  course  delivery,  CNC  distance  learning  and  ITV  distance  learning.  At  one  extreme  some  institutions 
may  have  not  even  have  started  the  use  of  transparent  slides!  Finally,  the  pedagogy  parameter  affects  the  effort 
an  institution  puts  into  its  instructional  design.  The  ones  that  put  in  more  effort  should  have  better  outcomes  for 
their  curriculum  paradigm  than  others  that  have  perhaps  put  in  less  effort.  There  are  a number  of  articles 
arguing  that  we  should  pay  more  attention  to  pedagogy,  but  how  can  those  pedagogical  issues  be  integrated  into 
instructional  design  along  with  the  use  of  technology?  This  remains  unclear.  Overall,  with  these  three 
parameters  we  can  build  a three-dimensional  paradigm  of  curriculum  delivery  that  can  reflect  its  current 
standing. 


The  CLEAR  Curriculum  Delivery  Framework 

We  have  concluded  that  we  can  construct  a curriculum  delivery  paradigm  with  these  three  parameters.  The 
CLEAR  framework  (see  the  figure  below)  offers  guidance  on  how  to  make  this  system  work.  It  is  a mechanism 
that  can  provide  an  open  and  flexible  access  and  approach  to  curriculum.  And  it  has  been  designed  keeping 
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these  ideas  in  mind.  In  the  CLEAR  framework  there  are  five  basic  components,  namely,  Communication 
channel,  Learner,  Educator,  Assistant,  and  Resources  (CLEAR).  Four  of  those  components  are  based  on  an 
initial  investigation  (Xu,  97)  of  the  current  framework  of  the  Wrexham  Business  School.  The  assistant 
component  was  derived  from  a literature  review. 


In  the  framework  figure,  the  lines  indicate  the  communication  channels.  The  bubbles  indicate  the  other  four 
components.  The  communication  channel  component  (also  referred  to  as  the  ‘knowledge  transfer  routine’) 
always  links  any  two  of  the  other  four  components  together.  When  the  five  components  are  set  into  one 
framework  and  if  somehow  the  resources  transfer  routine  can  reflect  the  openness  and  flexibility  of  access  to  the 
curriculum,  it  is  not  hard  to  arrive  at  the  conclusion  that  the  traditional  approach  has  very  little  openness  and 
flexibility  for  the  learner.  This  is  because  in  the  traditional  process,  the  educator  draws  knowledge  from  the 
resource,  and  mediates  mainly  by  face  to  face  communication  in  transferring  knowledge  to  a learner.  In  the 
CLEAR  framework,  the  knowledge  transfer  routine  has  been  expanded  by  added  the  role  of  an  assistant  into  the 
delivery  system.  Also  the  communication  channels  mean  more  than  just  face  to  face  communication  in  the 
CLEAR  framework.  They  also  include  all  the  possible  communication  methods  relying  on  advanced 
technology.  In  consequence  the  resource  becomes  richer  and  easier  to  access  in  the  sense  of  content  and  format 
due  to  the  contribution  of  the  assistant,  the  learner,  and  the  educator. 


Future  Research 

In  comparison  with  traditional  methods,  the  CLEAR  framework  can  provide  more  opportunities  to  learners, 
since  it  sets  curriculum  delivery  into  an  open  and  flexible  framework  by  adding  transfer  routines  and 
communication  channels  and  it  also  enriches  the  format  of  knowledge  resources.  However,  it  is  admitted  that 
these  goals  can  be  achieved  only  if  we  can  find  ways  to  take  advantage  of  technology,  because  the  framework 
relies  heavily  on  the  use  of  advanced  technology.  Therefore  future  research  in  this  project  aims  at  developing  a 
suitable  technology  platform.  Currently  the  links  between  technology  and  the  five  components  are  being 
mapped.  The  components  of  educator,  learner,  assistant,  and  resource  will  centralize  on  their  own  roles  in  the 
system,  and  the  communication  channel  will  maximize  the  possibility  of  assisting  their  different  roles.  The 
main  activity  for  the  educator  is  to  facilitate,  for  the  learner  it  is  to  explore,  for  the  assistant  it  is  to  cooperate 
and  for  the  resource  it  is  to  be  retrievable.  The  communication  channel  will  have  a key  role  in  extending  and 
supporting  the  system  functionally.  The  system  will  be  implemented  into  an  object-oriented  paradigm  of  the 
methodology  of  software  development.  The  roles  played  by  educator,  learner,  assistant,  and  resource  can  all  be 
integrated  as  objects  and  attributes  into  the  system  by  different  communication  methods  to  interact  with  the 
system.  This  work  will  be  reported  in  more  detail  at  a later  stage. 
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Introduction 

Learning  styles  are  approaches  to  learning  and  studying  (Dunn,  1987).  We  all  have  learning 
preferences,  which  enable  us  to  learn  more  effectively.  When  introduced  into  a learning  environment  that 
supports  our  learning  style(s),  learners  have  a higher  level  of  understanding  the  material.  The  learning 
styles  theory  implies  that  how  much  individuals  learn  has  more  to  do  with  whether  the  educational 
experience  is  geared  toward  their  particular  style  of  learning.  In  a traditional  classroom  environment,  there 
is  one  instructor  and  several  learners,  which  is  an  one-to-many  relationship.  The  instructor  presents 
information  with  his/her  personal  style  of  instruction.  If  the  instructor's  style  of  instruction  is  conducive 
with  the  majority  of  the  learner's  learning  style,  then  the  class  as  a whole  will  perform  well.  In  the  general 
case,  the  instructor’s  style  is  conducive  with  most  of  the  learner’s,  but  not  a perfect  match.  In  this  case,  the 
majority  of  the  class  will  have  an  average  performance  with  fewer  people  doing  either  very  well  or  very 
bad,  which  establishes  a bell  shaped  grade  distribution. 

Imagine  a classroom  full  of  instructors  and  only  one  learner,  which  is  a many-to-one  relationship. 
Each  instructor  is  an  expert  in  the  same  field  of  study,  but  each  uses  a different  style  of  instruction.  Hence, 
the  learner's  chances  of  doing  well  in  this  classroom  would  appear  to  be  significantly  better  than  in  a 
classroom  with  one  instructor  because  each  learner  would  adapt  to  the  instructor(s)  that  would  facilitate 
his/her  learning  style.  In  the  sections  that  follow,  we  will  introduce  an  implementation  of  the  many-to-one 
relationship. 


Arthur 

We  have  developed  a web  based  instruction  system  that  provides  adaptive  instruction,  Arthur. 
Arthur  takes  several  different  styles  of  instruction  from  several  different  instructors  and  makes  them 
available  to  each  learner,  which  defines  a many-to-one  relationship.  A group  of  instructors  with  different 
instruction  styles  from  the  same  field  collaborate  to  create  a course  map,  which  is  similar  to  a syllabus,  for 
the  course  content.  The  course  map  is  divided  into  small  sections  that  are  called  concepts.  A concept  is  a 
basic  unit  of  instruction  or  a fundamental  concept  that  must  be  covered  within  the  course.  In  a traditional 
class,  the  topics  that  are  listed  on  a syllabus  would  be  considered  course  concepts.  After  creating  the 
course  map,  each  instructor  will  create  a web  deliverable  course  module,  using  their  own  personal 
instruction  style,  that  adheres  to  the  course  map.  Upon  completion  of  their  individual  course  modules,  each 
instructor  will  add  their  course  information  into  Arthur  via  the  web.  This  creates  an  instruction  pool  of 
course  modules  accessible  from  the  world  wide  web.  Each  course  module  represents  a unique  method  of 
delivering  instruction  via  the  web  utilizing  a different  instruction  style.  Upon  entry  into  the  system, 
learners  will  be  assigned  course  modules  from  the  instruction  pool. 
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Learning  Experience 


When  a learner  enters  Arthur  using  a login  and  password,  Arthur  will  deliver  the  first  concept  of 
one  of  the  course  modules  from  the  instruction  pool  The  courses  are  initially  selected  at  random. 
Therefore,  each  student  will  be  assigned  their  first  course  module  by  chance.  Each  concept  is  terminated 
by  a short  evaluation  quiz  entered  by  the  instructor.  Learners  will  take  the  quiz  at  the  end  of  each  concept. 
They  are  expected  to  pass  each  section  with  a score  of  eighty  percent  or  better  in  order  to  continue  within 
the  current  course  module.  This  evaluation  method  introduces  the  term  Mastery  Learning,  which  is  used 
by  Arthur  to  adapt  the  instruction  style. 


Adapting  Instruction 

Mastery  Learning  is  based  on  the  assumption  that,  given  enough  time  and  proper  instruction,  most 
students  can  master  any  learning  objective  (Bloom  1968).  The  normal  distribution  of  scores  learners  exhibit 
on  any  performance  test  arise  from  the  use  of  one  instruction  style  given  by  one  instructor  and  the  practice 
of  holding  instructional  time  constant  for  all  students  and  allowing  learning  to  vary.  Bloom  (1976)  suggest 
that  learning  should  be  held  constant  and  time  allowed  to  vary.  Arthur  uses  mastery  learning  to  adapt 
instruction  for  each  learner.  When  a learner  completes  a quiz  at  the  end  of  a concept,  Arthur  employs 
mastery  learning  to  adapt  the  instruction  based  upon  the  learner's  score  for  each  quiz.  If  the  learner  scores 
80  percent  or  better,  Arthur  will  allow  the  learner  to  move  onto  the  next  concept  using  the  current  course 
module.  Otherwise,  Arthur  will  force  the  learner  to  repeat  the  same  concept  under  a different  course 
module,  which  utilizes  a different  instruction  style. 


Conclusions 

The  theory  of  learning  styles  states  that  people  have  different  approaches  to  learning  and  studying 
(Dunn  1 987).  Several  studies  show  that  there  is  "No  Significant  Difference"  when  technology  is  applied  to 
instruction  (Gerhing  1994,  Goldberg  1996,  & Moore  1996).  The  most  commonly  used  instruction 
environments  use  an  one-to-many  or  one-to-one  instructor/leamer  relationship.  We  have  developed  an 
environment  that  utilizes  technology  to  deliver  a many-to-one  instructor/leamer  relationship.  Before  the 
recent  use  of  technology,  this  type  of  education  experience  would  have  been  too  expensive  to  implement. 
Using  the  world  wide  web  and  supporting  technologies,  we  can  deliver  a many-to-one  instructor/leamer 
relationship  such  that  individual  learning  styles  can  be  accommodated. 
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Web-Based  Learning  Environments  in  Action:  The  Search  for  Luna,  A 
Nautical  Archaeology  Expedition 


Introduction 

Web-based  learning  environments  can  offer  learners  an  opportunity  to  participate  in  real-world, 
authentic  situations  that  can  provide  them  with  exciting  learning  experiences.  Anchored,  authentic,  and  situation 
learning  environments  such  as  MayaQuest,  Net-Frog,  Jasper  Woodbury,  and  Voyage  of  the  Mimi,  provide 
learners  with  a unique  opportunity  to  participate  in  complex-real  world  learning  experience.  In  these  learning 
environments,  learners  collaboratively  participate  in  experiences  within  meaningful  contexts  to  permit  the 
learner  to  become  independent  and  develop  problem  solving  and  reasoning  skills  that  can  be  transferred  to  other 
situations.  Learners  participate  in  activities  to  develop  skills  that  involve  the  analysis,  synthesis,  and  evaluation 
of  information  and  strategies  used  to  solve  a wide  variety  of  problems  (CTGV,  1 990).  In  turn,  learners  help 
define  and  refine  the  learning  environment  to  create  a multi-dimensional  microworld. 

Reform  and  accountability  measures  across  the  nation  have  challenged  teachers  to  alter  the  ways  in 
which  they  teach  (e.g.,  SCANS,  Goals  2000).  Integrated  curriculum  and  technology-rich  learning  environments 
meet  these  challenges.  In  Florida,  responses  to  these  measures  are  seen  in  the  Florida  Sunshine  State  Standards 
which  describe  skills  that  students  should  attain  in  eight  subject  areas  (in  four  grade  level  clusters  PK-2,  3-5,  6- 
8,  and  9-12):  Language  Arts,  Mathematics,  Science,  Social  Studies,  the  Arts,  Health  and  Physical  Education, 
and  Applied  Technology.  Both  national  and  state  standards  involve  a wide  range  of  high-level  outcomes  that 
require  students  to  work  cooperatively  to  solve  problems,  apply  solutions,  and  reflect  upon  their  experiences. 

Development  of  the  Learning  Environment 

The  Search  for  Luna  was  anchored  around  the  framework  of  an  Archaeological  Field  School.  A team 
of  professional  archaeologists,  elementary  school  teachers,  instructional  designers,  and  technology  experts 
worked  together  to  develop  and  implement  the  expedition.  In  Northwest  Florida,  there  is  a rich  archaeological 
tradition,  both  on  land  and  sea.  Nautical  archaeologists  have  found  and  excavated  a sixteenth  century  Spanish 
ship  in  Pensacola  Bay  and  believe  that  there  are  six  additional  sunken  ships  still  to  be  found.  Field  School 
participants  worked  with  archaeologists  to  make  meaning  from  artifacts  of  the  current  find  and  assist  in  the 
exploration  for  and  excavation  of  other  ships.  Two  major  themes  guided  students  through  the  expedition 
experience:  Become  an  Archaeologist,  and  The  Expedition  Challenge. 
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In  "Become  an  Archaeologist,"  Field  School  participants  learned  the  basics  of  becoming  an 
Archaeologist  through  four  thematic  weeks:  1 .Exploration,  Immigration,  and  Colonization  (History),  2. How 
Does  Archaeology  Help  Us  Learn  About  Our  Past  (Searching),  3.Whaf  s the  Scientific  Method?  How  Do 
Archaeologists  Use  It?  (Excavation),  4.  What  Happens  to  Artifacts?  (Conservation).  For  the  Expedition 
Challenge,  learners  were  encouraged  to  form  an  expedition  team  and  plan,  organize,  and  pretend  to  carry  out 
their  own  expedition  to  explore  and  colonize  a new  territory.  For  both  the  Field  School  and  the  Expedition 
Challenge,  learners  were  encouraged  to  collaborate  with  other  learners  throughout  the  State  of  Florida  through  a 
Share  Network. 

To  support  the  constructivist  nature  of  the  learning  environment,  extensive  resources  were  available  to 
the  participants.  Learner  resources  were  required  to  develop  the  necessary  knowledge  skills  to  solve  the 
designated  problems  (Wilson,  1996).  The  Field  School  resources,  for  learners,  included:  an  Artifact  Database, 
Glossary,  Web  Sites,  and  Books.  These  resources  formed  the  background  materials  that  learners  needed  to 
answer  expedition  problems.  To  help  teachers  during  the  experience,  materials  were  also  available  to  provide 
assistance  such  as  additional  resources,  sample  in-class  activities,  and  assessment  and  data  collection 
techniques.  Teachers  also  had  access  to  a sample  model  unit,  available  through  the  EPSS,  STEPS,  that  outlines 
standards  alignment,  integrated  curricular  activities,  resources,  best  practices,  and  technologies  that  can  be  used 
in  a Nautical  Archaeological  Unit  (Northrop,  Pilcher,  & Rasmussen,  1998;  Support  for  Teachers  Enhancing 
Performance  in  Schools,  1998).  Teachers  were  kept  up-to-date  on  expedition  status  through  online  newsletters. 
All  teachers  were  part  of  a listserv  that  was  used  for  group  communication. 

Implementation  of  the  Learning  Environment 

The  expedition,  which  consisted  of  both  live  and  simulated  elements,  permitted  learners  to  participate 
in  student-centered  learning  activities  along  with  their  classmates  and  other  learners  across  the  State  of  Florida 
from  October  12  through  November  6,  1998.  Approximately  150  Florida  teachers  joined  the  expedition, 
participating  in  a variety  of  ways.  Some  classrooms  completed  the  "Become  an  Archaeologist"  sequence;  others 
worked  on  the  Expedition  Challenge.  Participants  included  teachers  and  learners  from  public  school  systems, 
private  schools,  home-bound  students,  and  home-schooled  students. 

Many  tasks  involved  a focus  on  cooperatively-determined  problems  and  the  solving  of  complex,  real- 
world  problems,  that  permitted  an  interaction  with  archaeology  experts.  Learners  and  teachers  corresponded 
daily  with  the  Expedition  Team  to  receive  approvals  for  class  teams,  equipment  lists,  and  answers  to  the  daily 
Mystery  Artifact  and  Help  Find  It!  (a  Florida  geography  quiz).  There  were  also  four  live  broadcasts  where 
learners  were  able  to  e-mail  or  call-in  questions  to  historians  and  archaeologists.  Archival  records  have  been 
retained  so  that  teachers  can  use  the  materials  in  their  classrooms  for  years  to  come. 

Conclusion 

Web-based  learning  environments  can  be  exciting  experiences  for  both  learners  and  teachers.  Learners 
can  help  construct  the  learning  environment  to  meet  their  individual  learning  needs  and  teachers  can  take 
advantage  of  pre-developed  and  emerging  materials  to  enhance  their  curriculum.  The  development  and 
implementation  of  virtual  learning  environments  is  a complex  endeavor  that  requires  a strong  theoretical 
framework  to  permit  effective  development  and  implementation. 
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Introduction 

Multimedia  technologies  have  made  rapid  progress  in  recent  years.  By  using  these  technologies 
advantageously,  we  can  create  more  powerful  and  effective  educational  teaching  materials.  However, 
developing  multimedia  teaching  materials  requires  computer  programming,  which  may  be  an  insurmountable 
obstacle  for  ESL  teachers  who  want  to  develop  such  materials,  but  are  unfamiliar  with  computer  programming. 
As  a solution  to  this  problem,  we  are  developing  a tool  that  helps  a teacher  create  Web-based  exercises  and 
quizzes  for  their  classes  without  requiring  any  programming.  An  ESL  teacher  will  be  able  to  create  various 
types  of  exercises  by  preparing  teaching  materials,  according  to  a simple  format,  without  the  need  for  any 
programming  code.  Our  tool  will  be  able  to  automatically  create  the  appropriate  HTML/JavaScript  forms  and 
pages  by  extracting  necessary  information  from  the  prepared  materials.  The  tool  now  under  development  is 
called  A-MATE:  A Multimedia  Authoring  System  for  Teaching  ESL.  We  first  began  the  A-MATE  project  in 
order  to  incorporate  the  option  of  creating  individualized  ESL  teaching  materials  into  our  “Hypermedia 
Pronunciation  Power  Program,”  (Matsuno  & Tsutsumi,  1998).  “Pronunciation  Power”  helps  ESL  students 
improve  their  English  pronunciation.  However,  we  have  found  that  A-MATE  also  has  great  potential  as  a 
stand-alone  program,  and  would  be  useful  to  ESL  teachers. 


The  user  interface  of  A-MATE  should  be  as  easy  to  use  as  possible,  so  that  an  ESL  teacher  can  devote  his  or  her 
main  energies  to  developing  teaching  materials,  rather  than  struggling  through  a computer  program.  After  our 
analysis  of  TOEFL  and  TOEIC  exercise  texts,  as  well  as  ESL  textbooks,  we  found  several  types  of  exercise 
patterns.  Those  patterns  are:  1)  Presenting  pictures,  2)  Presenting  spoken  dialogues,  3)  Presenting  sentences,  or 
4)  Presenting  (playing)  videos,  and  then  giving  questions.  Additionally,  there  are  several  question/answer 
patterns:  A)  Choose  one  answer  from  -n  possible  answers,  B)  Connect  -n  to  -n  items,  C)  Complete  sentences, 
and  D)  Arrange  -n  mixed-up  items  into  the  proper  order. 

To  sum  up,  we  can  abstract  typical  ESL  exercise  activities  as  follows: 

1)  Present  example  problems  utilizing  a (multi-)media  type  and  format. 

2)  Choose  one  out  of  a group  of  question/answer  patterns. 

We  have  plotted  our  program  strategy  to  produce  student  exercises  based  on  above  two  ESL  development 
activities  adopted  in  general: 

1)  A teacher  chooses  one  exercise  pattern  from  the  group  of  exercise  types. 

2)  The  teacher  also  chooses  one  question/answer  pattern  from  the  question/answer  groups. 

That  is  basically  all  that  a teacher  has  to  do.  We  will  describe  A-MATE  in  more  detail  in  the  following  section. 


Strategy 


Description  of  A-MATE 


A teacher  prepares  his  or  her  teaching  materials  in  advance.  The  teacher  may  collect  these  materials  via  the 
Internet,  or  capture  them  from  other  media  formats  such  as  CD-ROM,  Laser  Disk,  and/or  VCR.  As  we 
mentioned  above,  A-MATE  internally  treats  each  type  of  exercise  in  a similar  way.  However,  A-MATE  offers 
several  types  of  exercises  for  the  teacher's  convenience.  Figure  1 shows  the  A-MATE  Menu.  In  Figure  1 , if  a 
teacher  clicks  the  "TOEFL"  button,  the  menu  for  TOEFL  exercises  pops  up.  Then,  the  teacher  can  choose  an 
exercise  type  from  three  options  as  follows:  "Listening  Comprehension,"  "Structure  and  Written  Expression,"  or 
"Reading  Comprehension."  In  Figure  1,  if  a teacher  clicks  the  "Other  Exercises"  button,  Figure  2 pops  up. 

If  a teacher  clicks  the  "Next"  button  after  choosing  menu  items  as  in  Figure  2,  another  card  appears,  and  then 
the  teacher  is  urged  to  input  his  or  her  specific  questions,  the  number  of  possible  multiple  choice  answers,  and 
both  the  correct  answers  and  the  incorrect  answers  for  each  multiple  choice  question.  We  think  the  more  the 
number  of  correct/'wrong  answer  cuiubinations  grows,  the  more  students  can  enjoy  the  vanety  of  quizzes.  This 
is  due  to  the  runtime  randomization  function  of  A-MATE.  At  runtime,  A-MATE  chooses  one  correct  answer, 
and  chooses  n-1  wrong  answers  randomly  from  the  group  of  wrong  answers  that  a teacher  has  prepared  in 
advance,  and  mixes  up  the  correct  answer  with  the  wrong  answers.  This  means  students  can  try  different 
patterns  of  quizzes,  and  A-MATE  should  automatically  judge  if  students'  answers  are  right  or  wrong.  Table  I 
shows  how  A-MATE's  actions  correspond  with  each  exercise  type  at  runtime; 
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Figure  1:  Main  Menu  Figure  2:  Sub  Menu  of  “Other  Exercises” 


Table  1:  A-MATE’s  Action 


Type 

Required  Inputs 

A-MA  TE  ’s  Action  at  Runtime 

Choose  one  from  -n 
possible  answers 

-n,  one  correct  answer,  and  at 
least  n-1  wrong  answers 

Choose  one  correct  answer  and  n-1  wrong 
answers,  and  then  mix  them  up 

Connect  -n  to  -n  items 

-n  and  at  least  -n  pairs  of  items 

Chooses  -n  pairs  of  items  and  mix  them  up 

Complete  sentences 

Input  sentences,  and  then  mark 
words  a teacher  wants  to  hide 

Hide  marked  words 

Arrange  -n  mixed  up 
items  into  proper  order 

-n  sentences  in  proper  order 

Mix  -n  sentences  at  random 

Discussion 

There  are  several  excellent  authoring  tools  such  as  Macromedia  Director  and  Asymetrix  Toolbook.  However, 
these  packages  demand  a certain  degree  of  programming  knowledge  and  expertise,  and  can  be  difficult  or  time- 
consuming  to  effectively  utilize.  Nonetheless,  such  tools  are  quite  useful  for  courseware  design.  If  our  goal  were 
to  make  more  complex  courseware  programs,  with  many  additional  options,  A-MATE  would  not  adequately 
serve  the  goal,  and  authoring  tools  such  as  those  above-mentioned,  would  be  preferable.  However,  with  A- 
MATE,  it  is  not  necessary  for  the  ESL  teacher  to  acquire  any  knowledge  of  scripting,  as  the  design  parameters 
can  cover  most  of  the  commonly  used  ESL  exercise  patterns,  without  any  additional  programming  on  the 
teacher’s  part.  In  the  future,  we  also  plan  to  develop  software  solutions  for  recording  and  calculating  students’ 
exercise  results,  and  add  methods  for  student  grading  optionally. 
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Abstract:  Postsecondary  Institutions  (PSI)s  in  Alberta,  Canada  are  exploring  ways  to  promote  the 
development  of  educational  technology  for  on-campus,  multi-campus  and  distance  delivery  of 
instruction.  It  became  apparent  that  professional  development  was  the  way  to  harness  the 
intellectual  capital  of  the  instructors  and  to  provide  the  critical  mass  to  carry  out  the  enormous  task 
before  them.  A collaborative  initiative  was  proposed  to  develop  a common  set  of  instructional 
materials  on  the  topic  of  educational  technology  for  professional  development  in  all  PSIs  in 
Alberta.  Phase  I resulted  in  the  production  and  validation  of  25  print-based  modules  on  key  topics 
in  educational  technology.  Faculty  and  staff  from  13  of  Alberta’s  27  technical  institutes,  colleges, 
and  universities  participated  in  developing  and  validating  one  or  more  of  these  modules. 
Participants  may  obtain  certificated  credit  from  a university  and/or  continuing  education  credit 
from  their  home  institutions.  Phase  II,  scheduled  for  completion  by  the  summer  of  1999  will 
involve  the  transformation  of  the  original  25  plus  an  additional  5 modules  on  leadership  training  to 
interactive,  multimedia  format  suitable  for  delivery  via  the  World  Wide  Web.  In  addition,  one 
educational  technology  facilitator  has  been  hired  at  each  of  the  principle  institutions  to  provide 
professional  development  opportunities  for  faculty  and  staff  using  the  print-based  modules  at  their 
respective  institutions. 


In  1997,  the  Department  of  Advanced  Education  and  Manpower  established  a series  of  grants  to  post 
secondary  institutions  (PSE)  in  the  Province  of  Alberta.  The  grants  are  designed  to  promote  the  use  of 
instructional  technology  in  order  to  increase  achievement,  learning  efficiency  and  most  of  all  access  to  a 
broader  range  of  residents  of  Alberta.  A secondary  goal  is  to  foster  cooperation  among  the  institutions  in 
the  Province. 

For  Phase  I,  the  Northern  Alberta  Institute  of  Technology  (NAIT)  and  Grant  MacEwan  Community 
College  (GMCC)  formed  a consortium  to  create  a series  of  40  print-based  modules  for  professional 
development  of  faculty  and  staff  in  the  area  of  educational  technology.  The  modules  were  produced  and 
validated  by  faculty  and  staff  from  over  a dozen  institutions  during  the  1997-1998  academic  year.  Faculty 
may  opt  for  certification  from  the  Faculty  of  Extension  of  the  University  of  Alberta. 

Phase  II  begin  during  the  summer  of  1998  under  the  leadership  of  NAIT,  GMCC  and  the  University  of 
Alberta  (UA).  The  first  of  two  goals  is  to  enhance  the  print  modules  with  interactivity  and  convert  them  to 
web-based  instructional  delivery  (WBI)  for  increased  access  and  to  model  WBI  as  part  of  the  professional 
development  of  faculty.  The  second  goal  is  to  hire  a trainer  at  each  of  the  3 lead  institutions  to  train  faculty 
in  educational  technology  concepts  using  the  print-  and  later  web-based  modules. 

To  date,  the  print  modules  are  being  heavily  used  and  the  trainers  from  each  of  the  three  institutions 
are  actively  conducting  both  large  and  small  group  training. 

Development  of  the  ETPD  Print-Based  Modules 

The  Province  of  Alberta  has  been  on  the  leading  edge  of  educational  technology  since  the  early  1960s 
when  the  UA  acquired  an  IBM  1500  CAI  System.  The  tradition  has  continued  with  subsequent 
applications  at  many  other  Alberta  PSEs.  The  application  of  educational  technology  proceeded  through 
many  research  and  development  initiatives  from  1965  through  to  the  late  1980s.  At  that  time,  recession  and 
budget  cuts  reduced  the  availability  of  resources  to  continue  on  the  cutting  edge.  In  the  middle  1990s, 
however,  the  government  of  Alberta  began  to  return  some  funding  from  the  major  cuts  of  previous  years  in 
the  form  of  targeted  grants.  These  grants  were  aimed  at  development  of  educational  technology  systems 
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which  would  increase  access  to  all  citizens  of  the  Province,  rather  than  just  those  who  could  physically 
attend  one  of  the  many  campuses. 

At  the  same  time,  the  demographic  trends  began  to  be  recognized;  older  adults  who  want  basic  and 
relevant  skills,  minimal  career  interruption,  corporate  training  competencies,  and  concerns  over  non- 
traditional  institutions  beginning  to  take  over  training. 

The  Educational  Technology  Professional  Development  (ETPD)  Project  was  established  in  recognition 
of  the  need  for  training  of  faculty  and  staff  in  the  use  of  educational  technology.  Secondarily,  it  is 
recognized  there  is  a need  for  faculty  of  institutions  to  come  to  grips  with  the  issues  of  change  and  reform 
which,  some  argue,  educational  technology  can  provide  (will  bring?).  To  accomplish  the  training,  the 
consortium  identified  the  necessary  content,  format,  and  expertise  within  the  Province  to  design,  develop 
and  validate  the  modules.  The  modules  are  organized  into  the  four  areas  of  Computer  Literacy  (9), 
Institution-Specific  Topics  (7),  Educational  Technology  Core  (9),  and  Specialization  Areas  (15). 

Examples  from  Computer  Literacy  include  Basic  Keyboarding,  Basic  Word  Processing,  Image 
Scanning,  Creating  WWW  Searches,  Basic  Spreadsheet  and  Database  Management.  From  the  Institution- 
Specific  topics  include  Electronic  and  Voice  mail.  Knowledge  of  Printing  and  Support  Services,  and 
Network  Familiarity.  The  Educational  Technology  Core  includes  such  topics  as  Instructional  Design, 
Survey  of  Educational  Technology,  Independent  Study  and  distance  Education,  and  Layout  and  Design  for 
Print  Materials.  Specialization  Areas  include  things  such  as  Web  Page  Design,  Advanced  CD-ROM, 
Presentation  Graphics,  Audio-Teleconferencing,  and  Introduction  to  Computer-Assisted  and  -Managed 
Instruction. 

The  modules  were  created  and  validated  by  faculty  members  from  ten  different  PSEs  in  Alberta  and 
are  available  through  GMCC.  Interested  faculty  may  arrange  to  complete  a specific  number  of  these 
modules  in  a program  operated  by  the  Faculty  of  Extension  of  the  University  of  Alberta.  Several  of  the 
PSEs  are  using  selected  modules  as  part  of  their  graduate  and  undergraduate  programs  in  Instructional 
Technology/Distance  Education. 

Development  of  the  ETPD  Web-Based  Instruction  Modules 

Phase  II  recognizes  the  need  to  provide  the  modules  in  modem  electronic  format  using  the  WWW.  At 
the  same  time,  using  the  WWW  must  include  adding  value  to  the  modules  in  the  form  of  interactivity,  ease 
of  access,  and  finally  to  provide  a model  of  what  WBI  can  do  for  the  faculty  users.  Funding  was  obtained 
by  NAIT,  GMCC  and  UA  to  re-design  and  develop  30  of  the  modules  into  WBI  format. 

As  of  this  writing,  standards  have  been  established  and  development  of  modules  is  underway.  Issues 
such  as  who,  how,  where  and  what  are  being  resolved.  In  addition  issues  of  platform,  servers  and 
telecommunications  links  and  speeds  have  been  resolved. 

In  recognition  of  the  fact  that  training  is  a necessary  but  insufficient  condition  to  bring  about  change  in 
the  use  of  the  innovative  nature  of  educational  technology,  a series  of  5 modules  on  dealing  with  the  change 
process  in  the  context  of  educational  technology  is  being  developed.  These  will  complement  and  be  a part 
of  the  30  WWW  modules.  Example  topics  in  this  series  include  Dealing  with  Mental  Models,  Sharing  a 
Vision  (for  Educational  Technology-Driven  Change)  and  Long  Range  (multi-year)  Planning. 

Also  in  Phase  II,  a full  time  trainer  has  been  hired  at  each  institution  to  provide  group  and  individual 
support  to  faculty  with  respect  to  the  use  of  educational  technology.  During  this  presentation,  an  update  on 
the  usage  and  WWW  development  (scheduled  for  completion  for  summer  1999)  will  be  provided.  In 
addition,  sample  modules  will  be  made  available  for  examination  of  structure,  content  and  methodology. 

For  More  Information  on  ETPD  Modules  and  Training,  contact 

Clayton  R.  Wright,  Ph.D. 

Coordinator,  Instructional  Media  and  Design 
Grant  MacEwan  Community  College 
Edmonton,  Alberta,  Canada 
P.O.  Box  1796 
Edmonton,  Alberta 
Canada  T5J  2P2 

wrightc@ADMIN.GMCC.AB.CA 
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introduction 

Many  people  - including  legislators  and  governing  boards  of  educational  systems,  college  and  school  administrators  and  faculty, 
owners  of  learning  technology  companies  - are  searching  for  a signpost  that  indicates  the  best  way  forward  for  education. 
Appreciating  the  fact  that  knowledge  is  being  reorganized,  none  can  say  with  certainty  how  education  will  be  done  ten  years 
from  now.  One  crucial  question,  the  answer  to  which  would  provide  an  important  clue  as  to  what  may  lie  ahead,  is  “what 
should  be  the  role  of  modem  computer  and  communication  technologies  in  the  future  of  education?” 

Considering  the  fact  that  effective  communication  is  at  the  heart  of  good  teaching  and  learning,  interactive  multimedia  CD’s 
and  resource-based  learning  using  the  Internet  would  appear  to  have  a bright  future  in  education.  Moreover,  advances  in  these 
and  other  technologies  continue  to  change  many  aspects  of  modem  life,  often  in  very  significant  ways.  Notwithstanding  these 
facts,  just  how  far  new  technologies  will  move  us  along  the  path  towards  more  effective,  enjoyable  and  efficient  learning  (E^ 
learning)  is  an  interesting  question  to  contemplate.  It  is  possible  that  the  use  of  modem  computer  and  communication 
technologies  in  education  might  not  advance  far  beyond  their  current  role  as  useful  additions  to  traditional  methods  of  teaching 
and  learning.  However,  the  same  technologies  could  lead  to  the  development  of  entirely  new  learning  materials  and  learning 
environments  that,  if  successful,  would  require  a large-scale  reassessment  of  the  function  of  educational  institutions. 

The  Learning  Model 

The  answer  to  the  question  of  where  education  goes  from  here  might  be  another  question:  “what  (technological)  developments 
in  the  quest  for  learning  will  capture  the  interest  of  students?”  Most  current  educational  applications  of  computer  and 
communication  technologies  do  not  demonstrate  the  full  potential  that  they  possess  to  improve  learning.  For  an  effective 
demonstration,  it  will  be  necessary  to  create  a highly  developed  “whole  product”  - an  ‘'integrated  learning  system  ” - that 
includes  all  components  of  a course  and  facilitates  the  human  interaction  necessary  to  create  a supportive  environment  for 
learning.  Integration  of  new,  comprehensive  and  sophisticated  methods  of  communication  into  an  optimal  technology-driven 
learning  environment  is  a prerequisite  for  the  success  of  advanced  technology-driven  learning  models  at  the  post-elementary 
levels. 

It  is  quite  possible  that  the  reaction  of  students  to  new  learning  systems  will  be  the  signpost  that  legislators,  administrators  and 
others  pay  most  attention  to.  Appropriate  technology-based  systems  providing  complete,  flexible  learning  experiences  will 
attract  students.  When  these  systems  also  demonstrate  the  potential  to  be  well  integrated  with  other  school  or  college  activity 
then  institutions  will  shake  off  uncertainty  and  nervousness  and  become  enthusiastic  about  their  future  in  a new  learning 
environment. 

The  transformation  of  conceptual  components  of  a new  technology-driven  model  of  learning  into  products  that  have  worldwide 
appeal  to  students  requires  a clear  objective  and  a sound  plan  to  achieve  it.  The  objective,  broadly  defined,  is  as  stated  above. 
That  is,  to  create  an  integrated  learning  system  by  combining  the  latest  computer  and  communication  technologies  with  the 
desirable  components  of  traditional  education.  If  development  of  new  computer-based  learning  products  is  the  force  driving 
this  work,  then  the  first  requirement  of  the  plan  - an  element  that  is  crucial  to  the  success  of  the  entire  endeavor  - is  to  have  the 
appropriate  mix  of  people  on  the  technology  research  and  development  team.  Persons  with  a vision  of  what  education  could 
become,  and  possessing  not  only  creative  talent  but  also  relevant  (good  or  bad)  experience  of  current  education  systems,  these 
special  people  will  possess  vital  components  of  the  knowledge  and  dedication  necessary  to  achieve  the  objective.  The 
“appropriate  mix”  will  draw  from  many  disciplines,  and  include  persons  from  traditional  populations  of  the  academic 
community  as  well  as  further  afield.  This  is  a very  important  point.  Whereas  many  educational  applications  of  technology 
have  been  developed  by  college  faculty  or  specialists  employed  by  learning  product  companies,  sophisticated  “whole  products” 
will  result  from  the  combined  efforts  of  people  with  diverse  talents.  Faculty,  students^  alumni  and  others  having  knowledge  in 
the  area  of  learning  being  redesigned,  or  with  expertise  in  computer  or  communication  technologies,  or  the  learning  sciences, 
communication  arts,  etc.,  etc.,  must  work  together  to  create  the  highly-developed  products  having  potential  to  reinvigorate 
education  at  all  levels. 
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To  emphasize  this  point  even  further,  some  school  and  college  faculty  learn  the  elements  of  multimedia  or  other  types  of 
computer  software  in  order  to  develop  learning  “modules’*  that  illustrate  parts  of  courses  they  teach.  Faculty  and  students  in 
media  arts  use  the  same  software  to  develop  creative  works  that  typically  communicate  something  quite  different.  When 
education  technologists  utilize  the  software  they  do  so  primarily  from  a learning  sciences  viewpoint.  All  of  these  talents  must 
be  combined  - at  a higher  level  than  ever  before  - to  create  the  new  technology-driven  learning  methods! 

The  CoBaLT  Team 

In  1995  the  author  created  an  informal  partnership  of  students,  faculty  and  alumni  of  Arizona  State  University  to  explore  the 
iisft  of  advanced  multimedia  software  to  develop  computer-based  learning  modules  for  college-level  engineering  courses  (Hinks 
& Lerman,  1996).  Members  of  the  partnership,  named  CoBaLT  for  Computer-Based  Learning  and  Training,  has  ranged  from 
3 to  7 persons  representing  a number  of  academic  disciplines,  including  computer  science,  engineering,  chemistry,  architecture, 
education,  music  and  the  media  arts.  The  members  have  had  extensive  discussions  on  the  shortcomings  of  traditional  education 
and  the  best  ways  to  improve  learning.  Our  experience  has  underscored  the  importance  of  research  and  development  taking 
place  in  an  interdisciplinary  setting,  and  “hands-on”  experience  has  led  to  a high  level  of  confidence  in  the  use  of  advanced 
multimedia  software.  Several  prototype  multimedia  modules  have  been  developed  and  demonstrated.  Perhaps  most 
importantly,  an  appreciation  of  the  opportunities  and  challenges  offered  by  the  new  computer  and  communications  technologies 
has  been  gained. 

Findings 

The  CoBaLT  team  believes  there  are  three  aspects  of  learning  model  development  that  are  critical  to  success,  namely: 

1.  The  project  manager  needs  to  pay  constant  attention  to  the  interaction  among  all  members  of  the  model  development  team. 
This  requires  every  participant,  regardless  of  area  of  expertise,  to  fully  understand  and  support  the  team’s  objective. 
Regularly  scheduled,  well-planned  meetings  are  essential,  and  must  include  the  full  participation  of  all  team  members. 

2.  Another  significant  challenge  is  to  establish  a viable  model  of  the  multimedia  design  necessary  for  optimal  E^  learning. 
The  goal  here  is  to  develop  a clear  and  concise  understanding  of  the  entire  multidimensional  framework  that  replaces  the 
traditional  linear  approach  to  the  presentation  of  knowledge.  High-level  software,  representing  the  latest  in  integrated 
media  technology,  has  to  assemble  the  knowledge  contained  in  each  module  of  the  framework  using  a standard  --  but 
flexible  - “skeleton”,  the  components  of  which  allow  connections  among  the  multiple  formats  of  integrated  media  to  be 
represented  in  the  same  way  from  module  to  module. 

3.  The  third  component  of  optimal  model  development  is  efficient  utilization  of  computing  hardware  and  identification  and 
use  of  the  most  suitable  multimedia  software.  Software  for  sound  (including  music),  text,  graphics  (including  animation) 
must  be  compatible.  In  our  experience,  it  is  advantageous  if  more  than  one  team  member  is  familiar  with  any  given 
software. 
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Interactivity:  Strategies  that  Facilitate  Instructor-Student  Communication 


Introduction 

Instruction  through  the  World  Wide  Web  is  a viable  alternative  to  the  traditional,  formal  classroom 
environment.  Students  from  kindergarten  through  post-graduate  classes  take  advantage  these  opportunities  to 
enhance  their  learning  experiences.  One  of  the  criticisms  directed  toward  WBI  is  that  there  is  lack  of 
interactivity  and  this  lack  of  interactivity  consequently  reduces  instructional  effectiveness.  There  is  a perception 
that  students  are  isolated  and  that  there  is  a lack  of  communication  among  instructor  and  students.  However, 
using  interactive  instructional  strategies  can  facilitate  the  development  of  intellectually  motivated  students. 

The  Fundamentals  of  Interactivity 

Over  the  past  two  decades  the  term  interactivity  has  been  used  in  many  contexts.  Interactivity,  in  the 
1980s,  referred  to  the  hardware  required  for  Levels  I-IV  of  interactive  videodisc  technology.  In  the  past  few 
years,  researchers  began  examining  the  nature  of  the  interaction  rather  than  the  capabilities  of  specific 
hardware.  Schwier  (1993)  proposed  a taxonomy  of  interaction  that  incorporates  levels,  functions,  and 
transactions  for  a multimedia  environment.  The  three  levels  of  interaction  proposed  were  reactive,  proactive, 
and  mutual.  Reactive  interaction  is  basically  used  for  responding  to  given  questions.  Proactive  interaction 
emphasizes  generative  activity  and  construction  of  new  and  meaningful  knowledge.  Mutual  interaction  is 
reserved  for  artificial  intelligence  and  virtual  reality  environments. 

Based  on  this  earlier  research  in  multimedia  design,  proactive  interaction  is  of  most  interest  in  the 
design  and  implementation  of  interactive  WBI.  Self-awareness,  self-regulation,  problem-solving,  and  decision- 
making skills  are  most  supported  by  a generative  environment  (Bruning,  1994).  Instruction  created  for  the  web 
environment  can  be  developed  that  is  generative  in  scope  and  interactive  in  nature  through  a student’s  active 
involvement  in  the  instructional  process.  Interaction  and  subsequent  communication  levels  are  facilitated 
through  three  different  modes:  (1)  student-to-content;  (2)  student-to-student;  and  (3)  student-to-instructor.  Each 
mode  is  not  mutually  exclusive  as  a combination  of  interaction  types  actually  strengthens  the  learning 
environment  and  promotes  multiple  instructional  strategies  for  communities  of  diverse  learners. 

Designing  Instruction  for  Interaction  and  Communication 

Answers  to  the  question,  ’’what  is  the  role  of  instruction,”  suggest  prominent  instructional  strategies  to 
facilitate  modes  of  communication  and  interaction  within  web-based  instruction.  A sampling  of  activities  and 
instructional  strategies  should  be  used  to  facilitate  the  different  modes  of  interaction  and  levels  of 
communication.  For  each  mode  and  level,  unique  strategies  can  be  developed  within  the  context  of  the  web- 
based  learning  environment.  As  web  sites  are  developed  to  facilitate  the  instruction,  appropriate  design  elements 
can  be  incorporated  that  assist  students  in  the  interaction  and  communication  processes. 
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Student-to-Content  Interaction  Strategies 

Instructional.  One  of  the  primary  ways  that  students  participating  in  WBI  receive  information  is 
through  interacting  with  the  lesson  content.  Comprehensive  electronic  lectures  may  be  used  to  present 
information.  Organizing  strategies,  such  as  advance  organizers,  tables  of  contents,  and  lists  help  guide  the 
student  through  the  lesson.  Students  may  also  explore  teacher-included  web  links  for  elaborated  information  or 
information  from  a various  sources  and  opinions.  Embedded  questioning  techniques  can  help  to  keep  the  learner 
on  task  and  engaged  in  the  learning  activity.  Questioning  frames  can  also  be  used  to  direct  students  through 
searching,  discovery,  and  exploration  activities.  Summaries  of  the  lesson  can  refocus  the  student  on  important 
points  and  prepare  them  for  supplemental  activities  that  promote  retention  and  transfer  of  information. 

Web  Page  Designs,  Strategies  for  interaction  are  important  for  student-content  interaction;  however, 
design  of  the  web  page  is  equally  critical.  Pages  should  be  aesthetically  pleasing  and  appropriate  for  the 
learning  situation  and  should  follow  appropriate  screen  design/layout  principles.  For  example,  relevant  graphics 
can  be  used  as  needed.  The  text  can  be  formatted  so  to  permit  students  to  follow  the  lesson  and  to  motivate  them 
to  continue  working  through  it.  Lines  and  lines  of  text  should  be  avoided  to  prevent  information  overload. 
Instructions  and  directions  need  to  be  clearly  stated  and  highlighted  so  that  they  are  set  apart  from  the 
instructional  material.  Likewise,  web  links  should  be  distinctly  noted  and  return  instruction  included, 

Student-to-Student  Interaction  Strategies 

Instructional.  Student-student  relationships  are  difficult  to  generate  by  the  very  nature  of  WBI  itself 
Students  are  never  actually  in  the  same  room  with  each  other  and,  consequently,  traditional  ways  of  building  a 
cohort  (e.g.,  personal  communication,  before,  during,  and  after  classes;  in-person  group  activities)  are  not 
possible.  Discussion  and  responses  take  place  over  extended  periods  of  time,  reducing  spontaneity  and 
conversation  flow.  Strategies  such  as  early  formation  of  groups,  changing  groups  often,  and  mentoring  among 
novice  and  expert  students  can  be  used  to  assist  students  in  developing  effective  group  techniques. 

Web  Page  Designs.  When  building  group  discussions,  activities  such  as  chats,  threaded  discussions,  e- 
mail,  listservs  are  typically  used  to  facilitate  student-student  interactions.  Other  technologies  such  as  web  boards 
and  desktop  video  conferencing  can  be  also  used. 

Student-to-Instructor  Interaction  Strategies 

Instructional.  The  student-instructor  interaction  in  WBI  is  potentially  very  personal  and  can  be 
viewed  as  a mentor-student  relationship.  Through  e-mail,  personal  communication  is  promoted;  instructors  can 
develop  one-to-one  relationships  with  their  students  that  may  not  be  possible  in  a traditional  course. 

Web  Page  Designs.  As  with  suggested  designs  for  student-student  interaction,  student-instructor 
design  elements  include  consistent  location  of  communication  links  and  easily  available  help  and  frequently 
asked  questions.  Strategies  that  instructors  may  use  to  facilitate  student-instructor  interaction  include:  automatic 
response  (sending  receipt  acknowledgements)  and  responding  to  all  when  asked  a question  by  one. 

Conclusion 

There  are  many  instructional  strategies  can  be  used  to  facilitate  interaction  in  a web-based 
environment.  Effective  use  of  these  strategies  permits  students  and  instructors  to  actively  engage  in  the  learning 
process  and  develop  positive,  meaningful  relationships  that  build  generative,  interactive  learning  environments. 
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Universities  are  showing  increasing  interest  in  raising  the  standard  of  teaching  that  occurs  in  their 
courses  There  is  a growing  that  the  traditional  model  of  I ectu re/ workshop/ tutorial/assignment  may 

not  be  the  best  approach  in  promoting  learning  in  students.  Concurrently  with  this  change,  there  is  increasing  use 
of  the  World  Wide  Web(WWW)  to  deliver  courses  to  internal  and  external  students. 

It  is  timely  then  to  examine  the  role  of  the  WWW  in  moves  towards  better  teaching  and  learning  for  internal  and 
external  students  at  university. 


Learning  approaches 

The  first  challenge  is  to  begin  regarding  the  use  of  the  WWW  not  as  a deliveiy  medium  but  as  teaching  and 
learning  tool.  Once  viewed  this  way  we  can  allow  pedagogical  constructs  to  be  applied  to  the  use  of  the  WWW 
rather  than  just  examining  the  technical  issues  surrounding  the  use  of  the  WWW. 

The  usual  approach  in  most  internal  courses  at  universities  involves  a lecture/tutorial  format  in  which  material  is 
delivered  at  lectures  and  discussed  at  tutorials.  The  unspoken  assumption  is  that  delivery  of  the  material  (lecture) 
results  in  learning  of  the  material.  This  approach  is  called  transmissive  and  assumes  a strong  link  between  the 
means  of  education  and  the  ends.  That  is,  having  delivered  the  material  then  students  are  assumed  to  have  learnt. 
For  external  students,  materials  delivered  by  the  web  offer  access  to  easily  updated  textual  materials,  some 
limited  interactivity  with  programs  on  the  WWW  and  access  to  audio  and  video  that  can  be  streamed  in  real 
time.  This  makes  it  technically  possible  for  them  to  watch  a lecture  in  real  time  without  attending  the  university. 
This  is  also  a transmissive  mode  of  learning. 

Approaches  that  encompass  collaboration  and  build  a sense  of  community  would  seem  to  be  ideal  for  support  by 
the  use  of  the  WWW  and  are  becoming  an  important  focus  of  research  (McMahon,  1997;  Dillenbourg  & 
Schneider,  1995;  Walker  &Lambert,  1996). 

If  we  start  to  regard  the  WWW  as  a teaching  and  learning  tool  that  can  support  social  constructivist  approaches 
to  learning,  then  University  lecturers  will  need  support  as  they  undergo  personal,  social  and  professional  change. 


Joining  the  community 

The  WWW  has  probably  been  the  main  influence  that  has  changed  the  way  we  conceive  of  computers.  They  are 
no  longer  machines  which  have  to  be  conquered  or  commanded  but  have  become  transparent  (though  maybe 
some  operating  systems  and  browsers  make  them  translucent!)  windows  into  an  information  world.  Students 
need  a sense  of  this  world,  a sense  of  the  audience  participating  and  an  understanding  of  the  mostly  unwritten 
rules  that  govern  its  behaviour  in  order  to  successfully  use  the  new  medium.  The  second  challenge  facing  the  use 
of  WWW  then  is  to  equip  our  tertiary  students  with  the  conceptual  models  and  practical  skills  to  enable  them  to 
participate  in  this  community.  This  is  necessary  because,  according  to  a social  constructivist  view,  participation 
can  result  in  good  learning.  Once  they  can  participate  then  we  face  the  challenge  of  equipping  our  students  with 
the  critical  thinking  skills  necessary  so  that  they  can  confidently  use  information  on  the  web  for  learning 
purposes.. 


Using  the  technical  features  for  learning 
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Students  can  fill  in  forms,  watch  video,  complete  a multiple  choice  test,  have  it  marked,  watch  live  video,  listen 
to  live  broadcasts,  watch  animations,  submit  written  assignments,  email  other  students,  talk  to  other  students  and 
look  at  other  students  through  the  use  of  the  WWW,  These  capabilities  represent  clever  technical  achievements 
and  there  is  no  reason  to  think  that  development  in  this  area  is  likely  to  slow.  However  these  achievements  pose 
a third  challenge  (related  to  the  first)  of  how  to  best  implement  technical  capabilities  so  bring  about  meaningful 
learning.  Which  means  finding  good  pedagogical  practices  that  will  build  on  the  inherently  engaging  nature  of 
the  web  and  produce  good  learning. 

Assessment 

To  answer  this  question  would  require  a consideration  of  a broader  range  of  assessment  techniques  currently 
used  at  universities  in  order  to  uncover  differences  in  learning.  Such  consideration  is  likely  also  to  occur  if 
universities  contemplate  outcome-based  approaches.  Assessment  should  be  reconsidered  in  most  university 
courses  with  the  advent  of  the  use  of  the  WWW  and  this  presents  a fourth  challenge  of  how  to  best  use  the  web 
for  good  assessment.  Good  assessment  is  valid  and  authentic  and  would  mean  practices  like  providing 
opportunity  for  assessing  student’s  procedural  skills,  allowing  students  to  critique  set  problem  and  allowing 
students  to  make  up  their  own  questions. 


Learning  to  learn 

The  fifth  challenge  facing  the  use  of  the  WWW  in  universities  is  to  how  best  use  the  web  to  encourage  good 
learning  behaviours  in  students.  As  an  example  of  the  kind  of  behaviour  to  be  encouraged,  metacognition  is 
universally  regarded  as  an  essential  attribute  of  good  learning.  If  students  have  good  knowledge  of  the  nature  of 
their  learning,  of  their  effective  learning  strategies,  of  their  learning  strengths  and  weakness  and  can  be  given 
some  control  over  their  learning  then  better  cognitive  and  affective  outcomes  will  result.  Further,  conceptual 
change  requires  metacognition  and  metacognition  helps  recognition,  evaluation  and  revision  of  personal  views 
(Baird  & Northfield,  1992),  The  WWW  with  its  essentially  individual  approach  to  education  has  potential  in  this 
area  to  meet  this  challenge  if  appropriate  design  strategies  are  employed.  If  the  design  of  courses  allows 
individual  exploration  coupled  with  reflection  and  the  comparison  of  views  with  others  then  metacognition  can 
be  enhanced  and  good  learning  can  result. 


Change 

A final  challenge  to  the  use  of  the  WWW  is  faced  by  lecturers  themselves.  Using  the  WWW  means  that  lecturers 
will  have  to  change  their  usual  practice.  Change  to  embrace  the  WWW  and  to  meet  the  above  challenges  would 
probably  be  driven  by  a belief  that  learning  in  their  courses  is  not  as  good  as  it  could  be  and  that  assessment 
could  be  improved.  It  is  likely  that  the  use  of  the  WWW  for  distance  education  students  is  likely  to  be  restricted 
initially  to  its  use  as  a delivery  platform  as  good  instructional  design  is  required  to  meet  the  above  challenges.  It 
is  far  more  likely  that  the  challenges  can  be  met  through  the  use  of  the  WWW  to  support  the  internal  delivery  of 
courses  as  lecturers  can  more  easily  implement  a different  approach.  For  this  reason  the  WWW  is  likely  to  be 
used  in  good  teaching  and  learning  mostly  in  internal  courses. 
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Introduction 

The  rapid  evolution  of  science  and  technology  causes  changes  in  all  aspects  of  every  day  life.  It  has  become 
necessary  for  everybody  to  develop  learning  skills  and  have  the  opportunity  for  life  long  training  in  order  to 
conform  to  the  demands  these  changes  impose.  Especially  for  the  teachers  who  play  major  role  in  the 
application  of  the  educational  system,  mechanisms  providing  continual  training  and  daily  support  need  to  be 
established.  In  our  country,  the  teachers’  training  is  facing  a lot  of  difficulties  due  to  the  fact  that  there  are  a lot 
of  distant  and  secluded  regions  and  islands.  The  use  of  the  information  and  communication  technologies 
opened  up  new  opportunities  for  accessing  educational  material  and  participation  in  distance  courses,  which 
can  be  better  adapted  to  the  trainee’s  needs,  offering  greater  interactivity,  providing  more  possibilities  of 
individualisation  of  the  learning  process,  in  which  the  trainer  is  not  a mere  transmitter  of  knowledge  and  the 
trainee  is  not  just  a passive  receiver.  The  Hellenic  Pedagogical  Institute  has  introduced  a Distance  Training 
Centre,  called  “Hellenic  Distance  Training  Centre”  (HDTC,  http://hdtc.pi-schools.gr)  which  provides  new  ways 
for  training  and  collaboration  between  scientists,  teachers  and  students. 


HDTC  Services 

In  the  context  of  making  the  teachers  active  members  of  a world-wide  educational  community  and  giving  them 
the  opportunity  to  find  out  the  educational  potential  of  the  new  technologies  (ICT),  the  HDTC  aims  at  the:  a) 
provision  of  in-service,  distance  training;  b)  elimination  of  teachers’  and  schools’  isolation;  c)  removal  of 
barriers  to  learning,  ensuring  accessibility  for  the  majority  of  teachers  and  students;  d)  provision  of 
collaborative  opportunities  via  the  Web;  e)  development  of  added-value  services  utilising  the  existing 
technological  infrastructure  at  schools;  f)  contribution  to  the  formation  of  new  attitudes  towards  the  teachers’ 
training  and  the  exploitation  of  new  technologies  for  the  improvement  of  teaching  and  learning. 

The  implementation  is  Web  based,  incorporating  all  the  necessary  tools  for  interpersonal  communication.  The 
provided  services  are  based  on  the  following  three  scenarios  i)  supported  self-learning  ii)  collaborative 
learning  iii)  virtual  classroom.  These  services  are  structured  as  follows: 

Training  material:  It  is  very  important  for  teachers  and  students  to  learn  about  the  role  of  ICT  and  its 
implications.  The  use  of  ICT  has  significant  advantages  both  for  the  students  who  may  gain  a lot  in  information 
handling,  motivation,  problem  solving,  etc,  and  the  teachers  who  have  more  opportunities  for  individual 
teaching  and  group  work,  for  rethinking  and  applying  new  teaching  and  learning  strategies,  etc.  This  material 
concerns:  a)  information  and  communication  technology  in  learning',  b)  basic  technical  knowledge',  c) 
scenarios  - lesson  plans  for  the  use  of  ICT  in  the  classroom',  d)  educational  software',  e)  research  projects. 

Curricula  - textbooks'.  The  teachers  have  the  possibility  to  be  informed  in  time  for  the  new  curricula  and 
textbooks  and  contribute  to  their  development  by  sending  their  comments  and  suggestions.  They  can  also  send 
relevant  questions  or  ask  for  clarifications  in  order  to  be  prepared  before  their  application  in  schools. 

Discussion  fora:  The  discussion  fora  environment  intents  to  promote  and  facilitate  the  exchange  of  information 
among  teachers  and  contribute  to  the  solution  to  common  problems.  The  educational  community,  through  this 
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environment,  has  the  opportunity  to  join  several  on-going  discussions  relevant  to  either  specific  curriculum 
subjects  (e.g.  Mathematics,  History,  Physics)  or  cross  curriculum  subjects  (e.g  students’  assessment,  use  of  ICT 
in  education),  raise  issues  to  be  discussed  or  answer  to  existing  ones. 

Educational  websites  and  Conferences:  A “guide-lisf’  of  Web  sites  is  provided  to  the  teachers  and  students 
which  focuses  on  those  Web  sites  that  mainly  offer  lesson  plans,  additional  educational  material  and 
collaborative  activities.  Also,  bulletin  boards  for  educational  matters  and  conferences  are  presented. 

Collaborative  activities:  Through  several  activities,  the  teachers  are  motivated  to  use  ICT.  These  activities 
include  evaluation  of  Web  sites  according  to  specific  criteria,  design  of  lesson  plans,  development  of  Web  pages 
with  educational  material  etc.  The  results  of  their  work  are  presented  via  HDTC. 

Help  desk:  This  is  a service  of  major  importance,  which  aims  at  supporting  effectively  all  the  teachers  to  get 
over  minor  or  major  every  day  problems.  It  concerns  the  following:  a)  tools  and  techniques;  b)  pedagogical 
issues;  c)  FAQs;  d)  submission  of  questions;  e)  mailing  lists;  f)  bibliography  search;  g)  comments-  suggestions. 

Virtual  classroom:  The  Training  Centre  has  the  necessary  infrastructure  to  organise  telecourses.  It  is 
responsible  for  the  choice  of  the  subjects,  taking  into  account  the  teachers’  preferences  and  needs.  A group  of 
experts  from  the  academic  community  may  be  invited  to  engage  in  the  telecourse.  The  communication  is 
bidirectional,  so  that  questions  can  be  raised,  clarifications  can  be  given  and  collaborative  activities  can  take 
place.  The  careful  choice  of  the  subjects,  the  trainers,  and  the  effective  management  of  organisational  issues  can 
result  to  active  trainees’  participation  and  the  overall  success  of  the  telecourse  series. 


Conclusions  - Future  plans 

The  first  conclusions,  as  these  are  drawn  from  the  comments  sent  by  e-mail  as  well  as  from  the  questionnaires 
that  were  completed  by  teachers  are  very  positive  and  encouraging  and  point  to  the  overall  usefulness  of  such  a 
centre.  Unfortunately,  in  our  country,  it  seems  that  the  ISDN  technology  is  still  immature  to  be  widely  used  at 
schools  - the  ISDN  connection  cost  is  high  and  the  technical  support  is  inadequate.  Pedagogical  Institute’s 
forthcoming  plans  are  focused  on  the  following: 

• the  qualitative  improvement  and  the  renewal  of  equipment  and  services  in  the  light  of  technological 
advances.  More  specifically  we  will  work  on  a)  the  enrichment  of  the  educational  material  b)  the 
minimisation  of  the  response  time  to  the  users’  requirements,  questions  c)  the  best  fulfilment  of  the 
teachers’  needs  and  d)  the  promotion  of  public  awareness  and  discussion  about  the  HDTC  and  its 
benefits. 

• the  development  of  a reliable  model  for  organising  telecourses. 
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Preparing  teachers  for  the  21st  Century  has  been  a concern  for  educational  leaders  in  this  country 
during  the  last  two  decades.  The  National  Council  for  Accreditation  of  Teacher  Education  (NCATE)  issued 
a report  called  "Technology  and  the  New  Professional  Teacher;  Preparing  for  the  21st  Century  Classroom" 
in  1997.  The  NCATE  technology  task  force  suggests  that  perhaps  the  best  way  teacher  education  faculty 
can  inspire  future  teachers  to  use  technology  is  "to  cast  themselves  as  learners  and  to  experiment  fearlessly 
in  the  applications  of  technology,"  making  themselves  "role  models  of  lifelong  learning"  (p.  9).  The  task 
force  states  that  re-educating  the  existing  teaching  force  requires  extensive  professional  development,  but 
that  the  problem  will  be  compounded  if  future  teachers  are  inadequately  prepared  to  use  new  technology 
(p.  7).  NCATE  has  challenged  higher  education  to  incorporate  technology  across  the  entire  teacher 
education  program,  not  just  as  a "computer  literacy"  class  added  to  the  existing  curriculum  (p.7).  The  task 
force  states  that  "today's  teacher  candidates  will  teach  tomorrow  as  they  are  taught  today"  (p.4).  NCATE 
states  that  teacher  education  is  in  a time  of  transition,  calling  for  experimentation  and  a new  attitude  that  is 
"fearless  in  the  use  of  technology"  (p.6). 

The  U.S.  Department  of  Education  states  that  technology  offers  "numerous  possibilities  for 
integrating  assessment  into  the  daily  life  of  the  classroom"  (1997,  April,  p.  6).  Portfolios  have  been 
helpful,  not  only  in  assessing  student  performance  in  teacher  training  courses,  but  in  helping  students 
reflect  upon  and  organize  the  material  learned  in  the  courses.  Developing  portfolios  electronically  provides 
students  with  new  ways  to  demonstrate  a broad  range  of  abilities,  as  well  as  give  them  greater  awareness  of 
the  unique  capabilities  of  technology  in  creating  meaningful  forms  of  alternative  assessment,  A well- 
designed  electronic  portfolio  system  can  support  this  approach  and  offer  expanded  potential  for  collecting 
different  kinds  of  records  of  students'  work.  Final  products  in  a variety  of  media  (text,  graphics,  video, 
multimedia),  students'  oral  presentations  or  explanations,  interviews  that  capture  students'  development  and 
justifications  for  their  work,  and  in-progress  traces  of  thinking  and  problem  solving  processes  can  now  be 
included  using  video  and  computer  technologies. 

This  study  focuses  on  the  use  of  technology  as  a tool  for  portfolio  assessment  of  teacher 
candidates  as  evidence  of  achieving  course  objectives.  Students  from  a small  California  university  created 
multimedia  portfolios  as  a final  assessment  for  a reading  methods  course.  Course  objectives  were  linked  to 
state  teaching  standards,  as  well  as  the  Reading  Instruction  Competence  Assessment  (RICA)  goals.  Initial 
attempts  to  use  an  HTML  web  format  caused  problems  due  to  the  students'  lack  of  computer  experience 
and  expertise.  Templates  were  pre-designed  using  HyperStudio  in  order  to  provide  a more  workable 
framework  for  inexperienced  students.  A separate  stack  was  created  for  each  course  objective  and  a menu 
stack  provided  space  for  personal  artifacts  and  a philosophy  statement.  Each  objective  stack  included  a 
cover  sheet  text  box  for  self-reflection  and  self-assessment.  Artifacts  were  evaluated  in  terms  of  meeting 
course  criteria  and  cover  sheets  were  examined  for  evidence  of  student  self-reflection  and  self-assessment. 
Students  were  interviewed  to  gain  insight  into  the  experience  of  collecting  and  preserving  artifacts 
electronically. 

Generally,  the  students  responded  positively  to  the  electronic  portfolio  experience  despite  the 
initial  concern  over  the  technological  expectations.  Several  students  expressed  frustration  at  having  to 
work  with  computer  technology  in  a reading  methodology  course.  One  student  stated;  “I  was  scared  of  the 
whole  assignment  at  first  because  I didn't  know  how  to  use  any  other  programs  than  word  processing 
basically.”  Another  student  described  her  change  of  attitude  toward  the  electronic  portfolio  process  as  she 
progressed  through  the  semester; 

I was  really  negative  about  it  at  first  because  it  was  technology  and  I didn't  know  how. . .1  didn't  see 

how  it  really  related  to  the  English  and  Language  Arts  class  and  I didn't  want  to  do  it,  because  I 
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didn't  see  how  it  related  to  that.  But,  after  I did  it,  I saw  the  relation  and  how  it  correlated. 

Actually  I'm  glad  that  I did  it. 

Students  generally  responded  favorably  to  the  use  of  templates.  One  student  said:  “I  liked  having 
the  prepared  one  (template).  That  was  great,  'cause  I was  getting  started.”  Another  student  said:  “The  first 
one  (disk)  that  we  had  was  of  the  templates,  which  were  good,  I really  liked  the  templates.  Those  were 
helpful,,,”  Some  students  indicated  an  interest  in  developing  their  own  design  in  the  future.  One  student 
stated:  “I  liked  having  the  template,  I could  change  things  later  to  spice  it  up.”  Another  student  described 
how  she  would  improve  her  process  in  the  future:  “I  would  use  HyperStudio  but  I'd  design  my  own 
template.  The  ease  of  the  template  took  the  stress  off  for  the  first  time,  but  I'd  do  my  own  next  time.” 

When  students  were  asked  what  they  saw  as  the  purpose  of  the  electronic  portfolio  project,  one 
student  replied:  “I  saw  it  as  primarily  a reflectionary  tool.  It  was  helpful  to  know  you  were  gaining  what 
you  should  get  out  of  the  course.  It  fulfilled  various  purposes,  but  I used  it  more  for  reflection  on  goals.  It 
could  also  be  a tool  for  interviews.”  Another  student  stated:  “One  of  the  purposes  was  to  get  me  used  to  a 
new  computer  program,  because  I know  that's  one  of  the  big  things  that  we  need  to  start  getting  used  to 
now  is  all  the  technology.  The  other  purpose  was  to  show  that  I had  met  the  necessary  objectives  for  the 
class  and  that  I understood  that  I met  the  objectives.” 

Students  generally  indicated  a positive  response  to  the  use  of  portfolios  for  assessment.  According 
to  some  students,  electronic  portfolios  offered  a more  creative  means  of  demonstrating  evidence  of  meeting 
objectives. 

As  with  any  portfolio,  I was  in  the  position  to  review  all  my  work  and  the  initial  objectives  of  the 
course.  In  doing  this  I was  able  to  assess  my  work  and  reflect  upon  how  the  work  I completed  met 
these  objectives  just  by  being  able  to  tell  you  what  I did  for  all  these  objectives  or  what  stuff  I put 
in  there.  I think  I got  more  out  of  it  than  if  I would  have  if  I'd  had  to  study  for  a test  and  just 
basically  memorize  and  cram  into  my  brain  a bunch  of  stuff  for  just  a couple  of  days,  and  then  just 
spill  it  out  on  a piece  of  paper. 

Evidence  of  self-reflection  and  self-assessment  was  present  in  all  portfolios  examined.  One 
student  stated:  "The  class  really  helped  me  feel  prepared,  but  I think  sticking  it  all  in  one  place  and 
making  connections  between  the  objectives  and  the  things  helped  me  say,  oh  look,  I am  prepared.  Not  only 
do  I think  I am,  but  I can  tell  you  why,”  Another  student  explained  the  electronic  process: 

As  with  any  portfolio  development,  you  are  critically  examining  your  work  to  determine  which 
way  you  have  met  the  selected  criteria,  and  how  you  can  best  example  or  highlight  your  work.  The 
process  of  developing  the  portfolio  is  a constant  self-assessment.  However,  I don't  feel  that  the 
electronic  portfolio  was  more  of  an  assessment  than  a hard-copy  portfolio  would  be.  The  process  is 
the  same,  it  is  the  final  output  that  is  different. 

NCATE  states  that  teachers  need  new  understandings  about  how  technology  has  changed  society 
as  a whole  and  new  approaches  to  using  a wide  range  of  tools  and  software  as  a part  of  their  own 
"instructional  repertoire"  (p.  6).  In  the  year  2000,  NCATE  expects  to  present  new  accreditation  standards 
that  make  effective  use  of  technology  a central  requirement  for  teacher  preparation.  Infusing  technology 
into  teacher  training  presents  a challenge  to  educational  institutions  due  to  the  ever-expanding  accreditation 
requirements  and  time  limitations.  Student  perceptions  provide  greater  insight  into  the  strengths  and 
weaknesses  of  the  electronic  portfolio  as  a tool  for  assessment  and  as  a viable  means  of  integrating 
technology  into  teacher  training. 
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New  communications  and  strategic  planning 

Communication  is  a fundamental  concept  in  educational  practice.  The  attitude  of  professional  educators  and 
students  towards  distance  education  is  influenced  directly  by  the  quality  of  the  communication  methods  available 
[Price  1995].  The  technological  developments  in  the  television,  computing  and  communication  industries  are 
having  a major  impact  on  all  aspects  of  life.  The  convergence  of  these  technologies  is  bringing  about  new  business 
opportunities,  new  home  entertainment  possibilities,  new  learning  opportunities  in  traditional  classroom  teaching, 
self  learning  and  distance  learning.  Developments  in  personal  computer  and  interactive  video  techniques  make  it 
increasingly  possible  to  demonstrate,  experiment,  analyse  and  simulate  complex  processes  on  screen.  The  World 
Wide  Web,  for  example,  provides  new  opportunities  for  quality  distance  education  over  the  Internet.  The  Web, 
combined  with  other  technological  advances  such  as  digital  video  compression,  video  systems,  multimedia 
networking,  voice  activated  computers,  virtual  reality,  etc  will  offer  a wide  range  of  high  quality  on-line  distant 
learning  opportunities  [Underwood  & Underwood  1994]. 

Many  research  activities  have  been  carried  out  over  the  years  in  the  area  of  strategic  planning  and  scenario 
methodologies  (see  for  example  [Georgantzas  & Acar  1995])  but  no  great  efforts  have  been  devoted  to  facilitating 
their  fruition  by  advanced  learners.  Enormous  attention  has  been  paid  over  the  time  to  theoretical  aspects  of  formal 
strategic  planning  but  the  gap  between  analytical  details  of  the  ensuing  findings  in  various  application  fields  and 
the  necessity  of  easy-to-leam  knowledge  has  not  been  adequately  bridged.  Therefore,  methodological  and  technical 
efforts  have  to  be  made  in  order  to  make  these  disciplines  easier  to  be  learned  during  advanced  education  courses. 
These  attempts  can  be  made  even  by  exploiting  the  tools  offered  today  by  technology,  in  particular  by  the 
“networked  hypermultimedia”  epoch  which  we  are  entering,  characterized  by  the  evolving  global  digital 
superhighway  infrastructure  and  dominated  by  the  huge  planetary  diffusion  of  the  networked  hypermedia  World 
Wide  Web. 

In  this  context,  scenario  engineering  includes  a general  corpus  of  scenario  methods  and  techniques  for  strategic 
planning  and  pre-planning.  Scenario  engineering  can  be  regarded  as  a pragmatic  approach  arisen  in  the  area  of 
scenario  methodologies  which  has  been  conceived  and  developed  for  the  construction  of  scenarios  to  be  produced 
even  by  exploiting  multisectorial  technology  resources. 


NET-SIMULAB  for  advanced  education 

NET-SIMULAB  [Nicolo’  & Sapio  1998a,  1998b]  is  an  advanced  work  environment  suitably  arranged  to  carry  out 
research  activities  in  the  field  of  scenario  engineering  and  to  handle  multimedia  information  in  an  interactive  way. 
As  regards  the  hardware  configuration  of  NET-SIMULAB  (Figure  1),  there  are  three  main  workstations. 
Workstation  1 and  workstation  2 are  basically  used  for  scenario  generation  and  analysis.  Workstation  3 is  used  for 
scenario  presentation  and  documentation.  A portable  workstation  is  used  for  remote  elaboration  and  can  be 
connected  to  fixed  workstations.  Additional  hardware  includes  external  mass-memory  units,  hi-fi  devices,  video 
equipment  and  digital  photographic  facilities. 

In  particular,  scenario  transfer  [Nicolo’  & Sapio  1998c],  a subset  of  scenario  engineering  specifically  devoted  to 
make  scenario  studies  easy  to  use,  can  be  oriented  to  education  objectives.  To  this  purpose,  the  instructional  pattern 
for  learners  may  include  advanced  navigation  hypermedia  tools  and  allow  distance  learning  over  the  global 
network.  Thus,  NET-SIMULAB  based  scenario-engineering  training  management  can  be  performed.  As  an 
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example,  instructional  projects  concerned  with  scenario  transfer  methodology  could  be  addressed  to  scenario 
modellers,  decision  makers  as  well  as  scholars  and  students  in  the  field  of  strategic  planning.  In  this  context,  NET- 
SIMULAB  is  intended  to  serve  as  a knowledge  and  expertise  centre  oriented  to  “produce”  scientific  methodology 
for  scenario  transfer.  This  task  can  be  significantly  facilitated  and  strengthened  by  the  access  of  NET-SIMULAB  to 
the  Web, 

The  success  of  advanced  education  in  strategic  planning  greatly  depends  on  the  effective  transfer  of  information 
about  application  hypotheses  and  methodological  outlines  as  well  as  about  results  and  findings  to  learners.  Clear 
(intrinsically),  comprehensible  (for  non-specialists),  concise  and  correct  (as  to  formal  aspects)  information  can 
assure  immediate  and  time-saving  catching  of  key  aspects  by  learners.  We  can  call  this  important  requirement  the 
“four  ‘C’s  rule”  (due  to  the  initial  letters  of  the  words  clear,  comprehensible,  concise  and  correct)  and  we  consider 


In  this  context,  multimedia  technology  can  provide  powerful  capabilities  that  should  be  suitably  employed. 
Exploitation  of  sound  and  images  (still  images  and  full  motion  pictures)  can  significantly  contribute  to  delivering 
easily  understood,  synoptical  studies  and  therefore  to  facilitating  their  utilization.  Furthermore,  scenario 
documentation,  presentation  and  communication  are  more  effective  if  they  are  research-engineers — driven  (i.e, 
modellers-,  analysts-,  simulationists-,  methodologists-driven)  through  the  use  of  multimedia  facilities.  As  a matter 
of  fact,  such  capabilities  provide  modellers  with  flexible  technological  tools  which  are  suitable  to  calibrate  and 
modulate  the  complexity  level  of  the  transfer  contents  according  to  the  circumstances  and  the  category  of  learners. 
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Introduction 

In  many  educational  organizations,  networks  are  used  for  educational  support.  On  the  other  hand,  there  are 
many  traditional  classes  that  have  no  educational  support  using  computer  networks.  When  one  intends  to 
utilize  computer  networks  or  the  Internet  for  traditional  classes,  he  will  suffer  from  problems  such  as  time 
for  investigating  teaching  materials,  manpower,  arrangements  of  equipment  and  preparation  of  presentation 
data.  In  this  paper,  taking  above  points  into  consideration,  we  propose  a method  of  introducing  educational 
support  using  computer  networks  for  traditional  classes. 

The  method  of  educational  support  for  traditional  classes  using  computer  networks 

We  set  following  goals  for  the  proposed  method.  First,  the  purpose  of  the  method  is  review  and 
self-studying.  Second  goal  is  that  the  method  could  preserve  the  various  teaching  style  of  teachers.  Third 
goal  is  that  the  method  should  be  simple  so  that  the  number  of  staffs  and  a budget  should  not  be  large. 
Fourth  goal  is  that  the  method  is  able  to  be  adopted  to  traditional  lectures  and  computer  exercises.  Last  goal 
is  that  more  than  one  simultaneous  classes  should  be  supported  using  the  method.  To  satisfy  these  goals,  we 
adopt  the  following  means: 

• The  scene  of  a class  is  recorded  on  video  tape  as  it  is.  Afterwards  the  video  is  opened  to  the  students 
through  networks. 

In  this  way,  teachers  do  not  have  to  make  additional  preparations  for  classes.  Watching  the  video  is 
enough  for  students  to  relive  the  class. 

• A written  material  or  a computer  screen  of  the  teacher  operates  is  also  recorded  on  another  video  tape. 
This  video  and  a teacher's  video  are  synthesized  to  a single  image  for  publication. 

In  this  method,  as  the  written  material  is  created  during  the  traditional  class,  teachers  need  not  prepare 
anything  than  the  usual  lecture  and  need  not  change  their  teaching  style. 

• The  camera  for  recording  teacher's  movements  has  fixed  view  and  never  follows  of  teacher's  movements. 
Written  material  is  more  important  than  the  teacher’s  movements.  Especially  for  reviewing  and  self- 
studying,  precise  movements  of  the  teacher  are  not  necessary.  Only  the  feeling  of  actually  being  in  the 
class  and  the  motion  that  the  teacher  points  to  something  are  necessary. 


Implementation 

There  are  following  three  steps  to  implement  our  proposed  method.  First  step  is  to  record  the 
class.  In  the  case  of  a class  in  which  the  teacher  uses  a blackboard,  he  must  use  a visual  presenter  instead  of 
a blackboard.  The  output  from  the  visual  presenter  is  recorded  and  presented  to  students  using  a projection 
device.  Another  video  camera  records  movements  of  the  teacher  from  the  rear  side  of  the  classroom.  The 
view  of  the  camera  is  fixed  and  it  is  left  during  the  class  as  it  is.  In  a case  of  a computer  exercise  class,  we 
use  a scan  converter  to  record  a screen  of  a teacher's  computer.  Another  video  camera  records  a keyboard 
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and  a mouse  that  the  teacher  operates.  Second  step  is  to  make  digital  video  data.  Video  of  the  teacher  and 
video  from  the  visual  presenter  are  captured  into  a computer  and  merged  into  a single  video.  For  image 
compression,  we  use  the  MPEG-1  format.  Figure  1 shows  an  example  of  an  MPEG  video  picture.  Voice 
data  is  compressed  in  MPEG  Layer  II  form.  Third  step  is  making  digital  video  data  public  to  students.  We 
use  the  WWW  as  user  interface  for  viewing  MPEG-1  video.  The  WWW  page  contains  names,  syllabuses, 
titles  and  hyperlinks  to  MPEG-1  video  for  each  class. 


Figure  1:  An  example  of  an  MPEG  video  picture. 


Results 

As  experiments,  we  applied  our  proposed  method  to  following  two  subjects  in  1 998.  The  number 
of  supporting  staffs  was  only  two. 

• “Exercise  for  information  processing”  course.  This  subject  teaches  a computer  literacy  for  freshman 
students  in  the  Japanese  culture  course.  Each  student  uses  a computer  in  the  course.  We  applied  our 
method  to  this  course  from  April  17  to  Jun  26  1998  (10  classes).  The  number  of  access  times  to  the 
videos  was  84  until  July  1 1998.  This  number  is  fairly  large  because  the  number  of  absentee  of  each  class 
was  less  than  10. 

• “Computer  literacy”  course.  This  subject  aims  to  introduce  the  computer  science  to  freshman  students  in 
the  information  science  course.  Explanation  of  this  subject  was  mainly  carried  out  by  speaking  and 
writing.  We  applied  our  method  to  this  course  from  Jun  9 to  Jun  23  1 998  (three  classes).  The  number  of 
access  times  to  the  videos  was  134  until  July  1 1998. 

In  both  courses,  teachers  commented  that  this  method  was  effective  as  an  educational  method  for 
students  who  were  absent  from  a class  or  students  who  had  not  understood  well.  Students  who  actually 
watched  the  video  commented  that  this  service  was  good  because  the  points,  which  were  difficult  to 
understand  during  the  class,  were  able  to  be  reviewed.  They  also  commented  that  self-studying  watching 
the  video  was  necessary  since  one  might  be  absent  from  class  because  of  compelling  reasons. 
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Introduction 

In  Hong  Kong  there  has  been  positive  support,  from  educationalists,  academia,  and  universities,  for  the  integration  of  computer-based  pedagogy 
in  classroom  teaching  and  learning  (Li  et  al  1998).  This  study  is  in  an  attempt  to  support  those  initiatives  by  investigating  the  effective  uses  of 
computerized  presentational  tools  (CPT)  for  enhancing  teaching  and  learning.  The  central  focus  was  to  explore  the  potential  benefits  and  usage 
the  CPT  for  classroom  teaching  and  learning  and  its  contribution  to  the  preparation  of  teaching  materials. 

There  were  two-fold  objectives  of  this  study.  They  are:  1 ) to  gain  feedback  on  classroom  teaching  and  learning  through  the  use  of  a CPT  from 
the  perspective  of  both  students  and  teaching  staff;  and  2)  to  explore  the  possibility  of  using  CPT  as  a standard  tools  for  the  preparation  of 
teaching  materials  at  the  university.  Triangulation  methodology  was  used  with  combination  of  semi-structured  questionnaire  survey  and  in- 
depth  qualitative  interviews.  The  results  are  then  used  as  the  reference  to  enhance  the  teaching  and  learning  through  the  use  of  educational 
technology. 

In  focusing  on  the  effects  of  classroom  learning,  the  components  of  the  assessment  were  divided  into  six  aspects.  They  are:  the  organization  and 
structure  of  teaching  material;  clarity  and  legibility;  visual  stimulation;  vertical  interaction;  efficiency  in  delivery  of  information  material; 
learning  environment  and  atmosphere.  A sample  of  300  students  ranging  from  Diploma  to  Master  Degree  level  in  the  field  of  nursing  and  health 
sciences  was  studied  through  questionnaire  survey  together  with  in-depth  interviews  on  30  students. 

In  exploring  the  experiences  using  CPT  for  improving  classroom  teaching  and  its  potential  to  become  a major  foundation  tools  for  developing 
teaching  materials,  the  lecturers  representing  3 different  health  disciplines  (i.e.  nursing  science,  biomedical  science  and  behavioral  science)  who 
involved  into  this  study  were  recmited  for  an  in-depth  interviews. 

Conceptual  framework 

This  framework  was  comprised  with  the  components  indicating  the  flows  of  knowledge  transformation  through  classroom  teaching  and  learning 
as  shown  in  the  following  Fig.  1 . 


Part  of  knowledge  transformation  can  only  be  achieved  through  the  process  of  teaching  and  learning  within  which  the  intentions  is  believed  as 
one  of  the  major  factor  to  affect  those  outcomes.  However,  these  intentions  could  be  enhanced  through  the  six  dimensions  of  domains  as 
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mentioned  previously  (i.e.  organization  & structure  of  teaching  material;  clarity  & legibility;  visual  stimulation;  vertical  interaction;  efficiency  in 
delivery  of  information;  enhancement  of  learning  environment  and  atmosphere). 


Procedures 

All  students  from  7 different  classes  ranging  from  Diploma  to  Master  Degree  level  received  CRT  instructed  classroom  teaching.  The  contents  of 
those  teaching  material  were  made  by  their  subject  lecturers  and  accordingly  to  their  subject  areas  and  academic  levels.  Questionnaires  were 
given  to  each  of  these  classes  immediately  after  lecture.  Feedback  was  received  from  300  students.  In  addition,  in-depth  interviews  were 
conducted  with  30  students  regarding  to  their  opinion  on  the  uses  of  CPT  for  classroom  learning  and  teaching.  The  lecturers  involved  in  using 
CPT  instructed  material  for  teaching  were  interviewed. 


Findings 

Apart  from  the  descriptive  statistics,  the  combinations  of  the  statistical  methods  was  used  for  the  quantitative  data  analysis  by  using  SPSS  while  NUD.IST 
package  was  used  for  the  qualitative  data  analysis. 

The  results  indicated  that  the  computerised  presentational  packages  for  classroom  teaching  were  well  received  by  the  students  as  effective  tools  to  enhance 
the  process  of  learning.  The  study  also  indicated  some  problems  in  enhancing  a vertical  interaction  between  the  lecturer  and  the  students,  and  feasibility  of 
performing  a computerised  presentational  package  classroom  teaching.  The  score  comparison  for  the  six  domains  on  an  overall  reaction  to  a CPT  instructed 
learning  were  shown  in  Fig.  2. 

Interviews  with  lecturers  reflected  that  they  felt  the  CPT  instructed  classroom  teaching  would  contribute  to  the  process  of  teaching  as  well  as  the  students’ 
learning.  They  found  the  combination  of  different  text  fonts  and  colours  display,  which  makes  for  the  production  of  high  quality  teaching  material  in  the 
textual  format  as  well  as  integrating  other  formats  such  as  pictures,  graphics,  sound  and  video.  The  flexibility  in  arranging  the  sequence  of  the  presentation 
allows  the  lecturers  to  have  a better  organisation  of  the  teaching  materials  and  also  serves  as  an  information  bank  for  learning  materials. 

In  summary,  they  felt  that  CPT  allowed  better  oiiganisation  of  teaching  materials  in  logical  sequences  combined  with  speedy  display  of  the  teaching  contents 
to  enable  a smooth  delivery  of  lectures.  That  also  helped  the  lecturer  to  keep  the  time  at  their  required  pace.  Consistency  of  similar  lecture  contexts  for 
different  classes  can  be  maintained  through  this  standardisation  in  the  format  of  teaching  materials  being  presented.  It  also  facilitates  the  future  development 
of  modular  or  credit  based  system  in  our  university. 


Fig.  2:  Score  comparison  for  the  six  domains  on  an  overall  reaction  to  CPT  instructed  learning. 

There  were  some  situational  constraints  imposed  on  the  CPT  instructed  teaching.  The  lecturers  expressed  that  the  physical  environment  of  the  classroom, 
such  as  the  size  of  the  classroom,  lighting  arrangement  and  its  intensity,  will  directly  affect  the  quality  of  the  display  or  projection.  The  technical  set  up  of 
that  equipment  was  also  relatively  time  consuming.  In  addition,  they  pointed  out  that  the  operation  of  CPT  instmcted  teaching  required  practical  computer 
skills  in  order  to  ensure  the  smoo^  running  of  the  lecture.  In  operating  such  educational  technology,  many  lecturers  are  still  green  in  experience. 

In  conclusion,  the  indicated  factors  influencing  the  perceived  effectivaiess  of  CPT  instructed  classroom  teaching  from  the  students  and  lecturers  may  serve  as 
indicators  for  the  future  improvements  of  teaching  strategies.  We  are  optimistic  that  additional  resources  and  support  from  the  university  can  make 
computerised  presentation  teaching  valuable  for  students  and  teachers. 
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The  World  Wide  Web  has  been  largely  used  on  distance  learning,  but  some  problems  arise,  mainly  related  to  students’ 
feelings  of  loneliness  [Schmm,  1997].  We  decide  to  focus  on  this  problem  and  also  applying  Cognitive  Flexibility  Theory 
principles  to  a 19th  century  Portuguese  novel  "Cousin  Basilio"  [Carvalho  & Dias,  1997a]  available  at  the  following  URL:  http:// 
www.iep.uminho.pt/primobasilio 

On  this  research  we  focus  on  learners’  feelings  and  opinions  about  the  web  course  structure,  and  communication  between 
learners.  We  also  attend  to  subjects’  attitude  towards  the  computer  and  to  their  knowledge  as  users,  and  also  to  their  "Learning 
Preferences"  based  on  a scale  that  we  developed  [Carvalho,  1998a  & 1998b].  Finally,  we  describe  their  opinion  about  being 
enrolled  in  other  courses  on  the  web. 

"Cousin  Basilio"  Design 

During  the  design  of  this  document  we  had  two  main  concerns:  first  of  all  to  apply  Cognitive  Flexibility  Theory  principles 
to  the  novel  "Cousin  Basilio"  from  E^a  de  Queiros  and  secondly  to  recreate  the  19th  century  ambience  on  the  interface.  If  you 
access  to  the  URL  mentioned  previously  you  will  see  an  ancient  book  and  on  its  cover  you  read  "O  Primo  Basilio:  multiplas 
travessias  tematicas"  ("Cousin  Basilio:  multiple  thematic  criss-crossings").  Then  you  have  some  information  - a Help  function 
"AJuda"  - about  the  function  of  each  button  and  the  functionality  of  each  path  as  soon  as  you  press  them. 

Each  page  has  some  objects  (buttons)  available  that  recreate  a 19th  century  atmosphere.  All  information  and  instructions  to 
the  user  appear  in  blue,  using  a well  designed  handwriting.  It  has  two  menus:  the  main  menu  is  on  the  left  side  top  comer,  and 
looks  like  an  ancient  street  plate;  the  complementary  menu  is  under  the  main  menu.  The  information  available  on  this  second 
menu  is  dependent  of  the  path  chosen  on  the  main  menu,  nevertheless  it  gives  access  to  "Themes”  (a  general  description  of 
each  of  the  nine  themes  selected  for  analysing  the  novel)  and  to  the  "Bibliography"  (authors  mentioned  on  "thematic 
commentaries"  or  on  the  description  of  the  general  Themes).  At  the  end  of  the  page  there  is  a black  pen  that,  when  pressed, 
allows  the  learner  to  write  his/her  own  notes.  On  the  bottom,  near  the  pen,  there  are  three  coins  that  when  pressed  give  access 
to  the  content  material  and,  later  on,  allow  the  user  to  quit  the  document  properly.  When  this  button  (coins)  is  pressed  in  order 
to  quit  the  document,  some  possible  links  related  to  the  novel  and  to  the  author  appear.  The  texts  of  the  novel  have  a light 
yellow  background  that  imitates  old  stationery.  All  comments  that  help  to  understand  the  texts  appear  on  a blue  background 
(this  resembles  an  ancient  official  Portuguese  stationery)  that  is  sealed. 

The  main  menu  is  always  available  and  shows  all  the  possible  paths  to  the  learner.  The  first  path  named  "Casos"  (Cases) 
presents  five  sequences  of  the  book,  each  sequence  containing  several  small  texts  (mini-cases)  of  the  novel.  According  to  Spiro 
and  Jehng  (1990:  181)  "the  mini-case  (a  segment  drawn  from  a larger  case)  is  the  starting  point  for  all  instruction".  Advanced 
knowledge  acquisition  has  significant  context  dependent  variations,  so  it  is  important  to  analyse  several  mini-cases  according  to 
multiple  representations.  The  second  path  "T6picos  de  Reflexao"  (Topics  for  Reflection)  has  five  topics.  For  the  study  of  each 
Topic,  several  relevant  mini-cases  and  thematic  commentaries  have  been  selected.  "The  notion  of  'criss-crossing'  from  case  to 
case  in  many  thematic  dimensions  serving  as  routes  of  traversal,  is  central  to  our  theory"  [Spiro  et  al.,  1987:  187].  The  third 
path  "Travessias  Tematicas"  (Thematic  Criss-Crossings)  allows  the  user  to  do  a personal  search,  choosing  the  themes  and 
cases.  The  last  option  is  "Tabela  de  Conteudos"  (Table  of  Contents)  with  all  cases  and  respective  mini-cases  and  the  indication 
of  the  applied  themes  for  each  mini-case. 

Distance  learning  course  procedures 

This  distance  learning  course  on  the  web  sets  a pre-requisite  on  the  subjects'  background:  they  have  to  be  undergraduate 
students  or  teachers  of  Portuguese  literature. 

This  course  integrates  three  packages  to  be  sent  to  each  subject  with  several  instruments  like  Questionnaires  (a 
Questionnaire  about  subjects'  computer  literacy  and  a Questionnaire  of  Opinion  about  the  web  document),  tests,  a "Learning 
Preferences"  scale  (containing  the  following  dimensions:  'complex  knowledge  acquisition',  'autonomy  in  learning',  and 

'preference  for  complex  knowledge'^  [Carvalho,  1998a  & 1998b]),  guidelines  for  the  session,  participants'  e-mail  addresses,  and 


^ The  "Learning  Preferences"  scale  has  three  dimensions:  FI:  alpha=0.83;  F2:  alpha=0.73;  F3:  alpha=0.63;  the  scale  has  a 
Cronbach's  alpha  =0.81  (Carvalho,  1998a  &1998b). 
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researcher's  contact.  Exchange  e-mail  is  not  compulsory,  but  recommended  if  they  like  to  share  some  ideas  or  some  doubts.  At 
the  end  of  this  distance  learning  web  course,  we  ask  the  subjects  to  write  a report  commenting  on  several  aspects  like  the 
document  design,  the  structure  of  the  course  and  their  opinion  about  courses  on  the  web  for  lifelong  learning. 

When  they  finished  to  answer  all  instruments  on  package  I , they  had  one  session  on  a computer  laboratory  at  the  university 
with  the  purpose  of  making  them  feel  comfortable  to  use  and  explore  the  document  on  the  Web.  At  the  end  of  this  session,  they 
received  the  second  package.  After  having  read  all  the  descriptions  on  the  general  themes,  and  done  the  first  path  on  "Cases" 
and  the  first  path  on  "Topics",  the  users  were  challenged  to  try  to  build  their  own  knowledge  that  they  could  check  on  the 
document.  This  kind  of  challenge  has  been  successful  in  another  study  (Carvalho  & Dias,  1997b).  Each  subject  worked  at  his 
own  pace.  They  were  applying  the  packages  according  to  their  pace  of  learning. 

Sample  characterisation 

We  had  nineteen  subjects,  1 male  and  eighteen  females,  3rd  year  undergraduate  students  enrolled  in  Portuguese  Literature, 
ranging  from  twenty  to  twenty-nine  years  (the  mean  was  22.5  years  and  the  mode  was  20  years).  We  realised,  when  analysing 
the  Questionnaire  of  Computer  Literacy,  that  2 subjects  never  used  a computer,  all  the  others  were  using  it,  but  not  often.  Most 
of  them  had  never  explored  an  interactive  environment,  only  6 subjects  had  (31.5%)  explored  the  Internet.  We  analysed  their 
attitude  towards  the  computer  according  to  two  parameters:  attitude  and  feelings.  The  correlation  obtained  in  these  two 
parameters  is  high  (.71).  Most  of  them,  thirteen  subjects,  liked  to  work  with  a computer,  four  subjects  didn't  like  and  two 
avoided  it.  Most  subjects  (15)  felt  comfortable  when  working  with  a computer,  eleven  of  them  liked  to  work  with  it,  but  four 
did  not.  Only  two  subjects  felt  "a  little  bit  nervous"  when  they  needed  to  use  a computer.  Finally,  two  subjects  felt  nervous  and 
avoided  to  use  a computer,  but  anyway  they  accepted  to  participate  in  this  research.  The  results  on  the  "Learning  Preferences" 

scale  pointed  out  a positive  attitude^  in  all  three  dimensions. 

Results  and  Conclusion 

Although  subjects'  computer  literacy  was  low,  they  felt  it  was  easy  to  use  the  web  document  and  to  navigate  on  it.  In 
addition,  not  only  their  opinion  about  the  web  document  as  a whole  was  favourable,  but  also  in  what  concerns  its  structure, 
interface,  orientation  and  its  content. 

All  of  them  accepted  positively  the  suggestion  of  having  access  to  documents  like  ''Cousin  Basilio"  on  the  web,  as  a support 
to  their  learning  modules  in  the  university  and  for  further  education.  Some  of  the  subjects  that  accepted  the  challenge  of 
constructing  the  possible  explanations  of  the  applied  themes  to  each  text  in  the  path  "Cases"  and  to  select  the  texts  that  support 
a given  "Topic",  said  that  "it  helps  to  a deeper  study",  another  one  mentioned  that  she  "realised  that,  when  trying  to  explain 
thematic  commentaries,  some  of  her  ideas  were  wrong,  but  others  only  need  to  be  completed".  Learning  by  doing  (constructing) 
takes  a great  effort  and  strong  involvement  that  are  rewarded  in  the  learning  achieved. 

All  orientations  and  tasks  to  be  achieved  have  been  considered  clearly  enough.  Most  of  the  subjects  used  the  university 
computer  laboratories.  Some  went  to  the  lab  with  a peer  (12  subjects),  others  alone,  but  usually  they  met  someone  there  with 
whom  they  shared  some  ideas.  Only  four  subjects  never  shared  ideas  with  others  in  the  lab  or  after  the  sessions.  They  did  not 
use  the  e-mail  facility  because  they  knew  each  other  and  they  met  regularly  at  the  university. 

We  think  that  it  is  important  for  learners  to  have  a face-to-face  meeting  with  the  teacher  responsible  for  the  course.  In  this 
way,  they  have  the  opportunity  to  see  each  other  and  to  know  the  people  to  whom  they  are  going  to  send  a message,  to  clarily 
any  doubt  they  have  about  the  course,  besides  the  introduction  to  the  web  document  functionality.  With  this  methodology,  even 
those  subjects  that  never  used  a computer  or  that  seldom  used  it  have  the  opportunity  to  learn  to  use  it  properly.  The  participants 
must  be  comfortable  with  the  system  and  the  software,  so  that  they  may  concentrate  on  the  academic  tasks. 

Finally,  we  think  that  if  we  want  to  have  a real  Information  Society  our  undergraduate  students  have  to  explore  the  web  for 
their  study.  The  USA  President's  Advisory  Panel  recommend  that  "teacher  education  should  focus  on  learning  with  technology 
while  emphasising  content  and  pedagogy"  (Davis,  1997:  221).  Today's  undergraduate  students  are  the  teachers  of  tomorrow  and 
they  will  know  how  to  use  the  web  with  their  students,  if  they  are  used  to  explore  it. 
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[Carvalho,  1998a  & 1998b]. 


A Web-Based  Multimedia  Instructional  System  for  Language  Learning 


Miwha  Lee 

Pusan  National  University  of  Education,  Korea 
E-mail:  mlee@ns.pusan-e.ac.kr 

Aesun  Yoon 

Pusan  National  University,  Korea 
E-mail:  asvoon@hvowon.cc.pusan.ac.kr 


Introduction 

A great  amount  of  information  is  conveyed,  during  the  communication,  by  means  of  verbal  and  non-verbal 
elements.  The  relationship  of  these  various  components  is  so  pertinent  and  consistent  that  we  might  fail  to 
converse  if  we  cannot  understand  those  non-verbal  elements  or  use  them  appropriately  (e.g.,  Bateson  et  al., 
1981).  Multimedia-based  materials  have  a great  potential  to  present  both  verbal  and  non-verbal  elements  and  to 
combine  them  efficiently  for  language  education.  Moreover,  the  Internet  has  been  increasingly  utilized  as  an 
effective  instructional  tool  for  language  learning,  since  the  Web  can  become  a multimedia-based  content 
provider  with  versatility  and  interconnectedness  (Harasim  et  al.,  1996;  Khan,  1997;  McManus,  1995;  Owston, 
1997;  Ritchie  & Hoffman,  1996).  The  purpose  of  the  present  study  was  to  design  and  develop  a Web-based 
multimedia  instructional  system  for  French  language  learning.  The  instructional  system  was  designed  to 
encourage  students  to  actively  participate  in  the  learning  process  by  providing  students  with  a learning 
environment  that  is  flexible,  individualized,  interactive,  and  efficient. 

The  Content  Structure 

The  content  structure  of  the  instructional  system  was  designed  to  be  adaptive  to  individual  learning  situations  on 
a non-real  time  basis.  Students  can  navigate  the  hyperlinked  multimedia  contents  without  a pre-ordered  learning 
schedule.  Through  their  exploration  and  navigation,  thus,  students  can  design  their  own  instruction.  The  contents 
are  divided  into  two  levels:  beginning  and  intermediate.  Each  level  consists  of  15  lessons  developed  to  be  easily 
integrated  into  the  school  curriculum  in  Korea.  As  shown  in  Figure  1,  each  lesson  is  composed  of  eight 
components:  listening,  reading,  speaking,  grammar,  drills,  games,  pronunciation,  and  reading  comprehension. 
The  Interaction  facilities 

The  interaction  facilities  of  the  instructional  system  consist  of  four  components:  help,  bulletin  board, 
announcements,  and  e-mail.  These  interaction  facilities  were  designed  to  provide  various  types  of  asynchronous 
communications  among  three  different  user  groups:  teachers  or  tutors,  students,  and  system  administrators 
(Boverie  et  al.,  1998;  Laney,  1996;  MacKnight,  1998;  Repman  & Logan,  1996). 


The  User  Interface 

In  designing  and  developing  the  user  interface  of  the  instructional  system,  a special  emphasis  was  placed  on 
user-friendliness  and  efficiency.  The  user  interfaces  are  differentially  provided  to  each  of  the  user  groups, 
depending  on  their  authorization  status,  since  the  user  groups  need  different  modes  of  interaction.  For  the 
consistent  and  systematic  delivery  of  information,  the  subsequent  hyperlinked  information  is  presented  in  the 
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same  page.  As  shown  in  Figure  2,  the  navigation  bar  on  the  left  side  of  the  screen  provides  a cascaded  pull-down 
menu  from  which  students  can  easily  choose  their  lessons  and  appropriate  interaction  facilities  needed.  The 
position  indicator  on  the  upper  side  of  the  screen  allows  students  to  locate  themselves  during  navigation  and 
exploration. 
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Figure  2:  The  User  Interface 

Conclusion 

In  conclusion,  this  study  has  presented  the  design  and  development  of  a Web-based  multimedia  instructional 
system  for  French  language  learning.  The  instructional  system  developed  in  the  study  appears  to  be  one  of  the 
first  instructional  systems  for  computer-assisted  French  language  learning  in  Korea.  It  can  provide  a learning 
environment  where  students  can  actively  participate  in  the  learning  process  through  exploring  and  navigating  the 
hyperlinked  multimedia  learning  contents.  This  system  has  been  successfully  integrated  into  the  existing 
curriculum  and  has  been  currently  used  in  the  foreign  language  courses.  Furthermore,  the  interaction  facilities  of 
the  system  will  be  more  expanded  to  make  synchronous  as  well  as  asynchronous  communications  possible. 
Finally,  an  experimental  study  is  underway  to  examine  the  effectiveness  of  the  instructional  system  with  regard 
to  students’  learning  process  and  product. 
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Very  clearly  multicultural  education  is  an  important  concept  for  both  preservice  and  inservice  educators. 
The  U.S.  macroculture  is  shared  by  all  our  the  nation’s  citizens.  Many  cultural  groups  or  microcultures  are 
also  part  of  the  American  Society.  A goal  of  multicultural  education  is  that  a process  of  thinking  will 
develop.  Thus  the  authors  have  categorized  several  variables  of  multicultural  education:  age,  belief,  social 
class,  exceptionalities,  gender,  language,  ethnicity,  and  sexual  orientation.  The  authors  have  carefully 
selected  websites  which  represent  various  points  of  view  in  order  to  assist  the  student  or  educator  in  the 
process  of  thinking  and  in  developing  a multicultural  perspective. 


1297 


Addressing  the  Changing  Roles  ofTeachers  and  Learners  in 
Internet-Delivered  Materials 


Yin  Leng  Theng 
School  of  Computing  Science 
Middlesex  University  (United  Kingdom) 
y.theng@mdx.ac.uk 


Introduction 

Since  its  release  in  1991,  the  World  Wide  Web  (or  Web)has  changed  the  Internet  to  the  extent  that  it  has 
become  almost  synonmouswith  the  modem  use  of  the  Internet.  In  recent  years,  we  are  beginning  tosee  more  and 
more  open  learning  materials,  distance  learning  programmes,  virtualinstitutes,  etc.  springing  up  across  the  globe  to 
cash  in  on  these  seemingly  attractivebenefits.  Organisations  such  as  schools,  professional  bodies  and 
businessesrecognising  the  benefits  are  also  turning  to  internet-based  mode  ofteaching  to  enhance  their  learning 
programmes. 

But,  before  we  get  so  hyped  up  with  this  novel  mode  of  teaching  andleaming,  we  should  really  take  a step 
back  and  askourselves  these  questions:  how  has  internet-based  teaching  and  leamingchange  the  roles  and 
relationships  of  teachers  and  learners?  What  are  thesechanging  roles?  How  can  teachers  and  learners  be  helped  to 
cope  with  thesechanging  roles  so  that  effectiveuse  of  internet-based  materials  can  be  realised? 

Gone  is  the  traditional  concept  that  the  teacher  is  thesource  of  all  knowledge.  In  internet-based  teaching, 
the  teacher  is  nolonger  the  dictator  of  content  and  pace  of  leaming.In  other  words,  face-to-face  teaching  in  the 
classrooms  is  replaced  byleamers’  interaction  with  learning  materials  through  the  Intemetvia  the  computer  monitor, 
replacing  the  teacher  (Forsyth,  1997). In  internet-based  teaching,  there  is  a shift  towards  learner- 
centred  ness,transferring  the  ownership  of  learning  to  the  learners.  Producing  internet-delivered  materials  is  not  just 
simply  putting  coursematerials  on  the  Internet,  we  need  to  address  these  changing  roles  andprepare  teachers  and 
learners  for  them,  in  order  that  the  Internet  iseffectively  used  for  teaching  and  learning. 

We  learn  from  experience  that  many  computer  projects  failed  not  onlybecause  of  poor  design  but  also  poor 
delivery  in  terms  olpreparation  and  training  of  people  involved  in  the  final  systems(Ellington,  Percival  and  Race, 
1995;  etc.).  To  address  the  changing  roles  of  teachers  and  learners,  this  papersuggests  doing  things  well  both  in  the 
design  as  well  as  the  delivery  of  internet-delivered  materials. 


Design  Principles 

Very  often  the  criticism  against  intemet-deliveredteaching  is  that  the  human  element,  that  is  the  teacher,  is 
absent  andleaming  becomes  impersonal.  A real  challenge  to  designers  is  to  retain  asmuch  of  the  traditional 
classroom“features”  as  possible  in  the  design  ofintemet-delivered  materials,  and  to  ensure  that  teachers  and  students 
seeits  benefits,  and  to  accept  it  as  an  effective,  alternative  form  ofleaming. 

The  second  challenge  to  the  design  of  intemet-basedmaterials  is  to  incorporate  the  well-accepted 
ingredients  for  successfulleaming  by  (Ellington,  Percival  and  Race,  1995):  (i)  making  the  materials  stimulating  and 
interesting  enough  to  makeleamers  want  to  learn;  (ii)  incorporating  sufficient  activities  tohelp  learners  experience 
learning  by  doing;  (iii)  providing  sufficientchannels  of feedback  to  the  learners;  and  (iv)  enabling  learners  to  digest 
and  relate  whatthey  have  learned  to  the  real  world. 

Theories  in  instructional  design,  constructivism,  minimalism  andhuman-computer  interaction  can  be 
applied  to  translate  these  basicprinciples  for  successful  learning  experience  into  practical  design: 

• Instructional  design  represents  asystematic,  coherent  approach  to  the  design  of  computer-based 
instructionalmaterials.  Gagn^et.  al.  (1979)  propose  thatleaming  materials  must  include  nine  instructional  events 
that  shouldnormally  be  followed  in  sequence:  (i)  gaining  attention;  (ii)  informingleamer  of  lesson  objective;  (iii) 
stimulating  recall  of  prior  leaming;(iv)  presenting  stimuli  with  distinctivefeatures;  (v)  guiding  learning;  (vi) 
eliciting  performance;  (vii)  providinginformative  feedback;  (viii)  assessing  performance;  and  (ix) 
enhancingretention  and  learning  transfer. 

• Constructivism  proposes  that  the  knowledge  of  the  worldis  constructed  by  the  learner,  making  learning  rather 
than  instruction  thefocal  issue.  If  learning  has  a constructive  character  inherently,  then  it  follows  that  teaching 
practices  need  to  besupportive  of  the  construction  that  occurs.  Cummingham  et.  al.  (1993)  propose  seven 
principles  of  instructional  design:  (i)  embedleaming  in  realistic  and  relevant  contexts;  (ii)  embed  learning  in 


socialexperience;  (iii)  encourage  the  use  of  multiple  modes  of  representation;(iv)  provide  experience  and 
appreciation  of  multiple  perspectives;  (v)  encourage  ownership  and  voice  in  theleaming  process;  (vi)  provide 
experience  in  the  knowledge  constnictionprocess;  and  (vii)  encourage  self-awareness  of  the  knowledge 
constructionprocess. 

• Minimalism's  central  idea  is  to  minimise  the  amount  of  explicit  instructionalmaterials,  and  that  learning  takes 
place  in  small  steps  (Carroll,  1990).Therefore,  learning  materials  should  be  structured  in  a highly  modular 
wayof  self-contained  units,  allowing  learners’  strategies  and  preferences  to  drive  the  selection  andstnicturing  of 
contents. 

• Knowledge  in  human-computer  interaction  is  important  in  helping  designers  to  take  into  account  human 
capabilitiesand  limitations,  and  predicting  how  design  choices  might  affect  human. Design  guidelines  from 
human-computer  interaction  on  text,  graphics, colours,  etc.  are  useful  in  helping  designers  to  create  interfaces 
that  are  cl ear,consi stent,  simple  and  uncluttered  so  that  complex  information  can  beeffectively  perceived  by 
learners. 

The  designprinciples  discussed  above  have  been  used  to  teach  ninety  second-yearcomputing  science 
students  at  Middlesex  University  (London).  As  part  of  theassessment,  these  student  designers  were  asked  to  design 
and  develop  ahypermedia,  computer-aided  learning  package  incorporating  established  principles  in  human-computer 
interaction,leaming  theories  and  instructional  design,  and  translating  them  intopractical  design.  These  principles 
have,  in  general,  been  useful  in  guiding  these  novice  student  designers  in  the  design  anddevelopment  of  their 
learning  packages. 

Delivery 

In  a recent  conference  on  “Educationaluses  of  the  Internet  and  European  identity  construction”  held  in 
Strasbourg  (France)  in  September  1998  (seehttp://in-tele.u-strasbg.fr),  teacher  representatives  at  the  conferencefrom 
across  Europe  voiced  their  concerns  that  not  enough  is  done  to  prepare  teachers  and  students  to  use  theintemet 
effectively.  There  is  a general  sense  of  fear  and  apprehensionamong  teachers  because  they  are  not  trained  to  use  the 
computer  andlntemet  for  teaching.To  avoid  rejection  of  this  new  medium  by  teachers  and  students  who  are 
moreaccustomed  to  traditional  classroom  teaching,  and  find  it  difficult  to  giveup,  education  and  training  should  be 
provided  to  address  the  lack  ofconfidence  in  using  the  Internet  as  a teaching  and  learning  tool.  Teachers  as  well  as 
students  should  betaught  appropriate  skills  to  use  the  Internet  which  include  knowing  how  to  search  and  query, 
transfer  materials  and  files,  use  ofe-mails,  chat  and  bulletin  boards,  computer-aided  materials,  etc. 


Conclusion 

This  paper  examined  the  changing  roles  of  teachers  and  learners  inintemet-based  teaching  and  learning. 
Effective  design  and  careful  deliveryof  the  materials  are  some  ways  suggested  to  address  these  changing 
roJes.Future  work  involves  drawing  up  a framework  for  an  application  design  modelfor  developing  hypermedia 
materials  for  teaching  and  learning.  The  aim  ofthis  framework  is  to  provide  designers  with  some  sort  of  a checklist 
toensure  thatdesign  issues  crucial  to  the  successful  delivery  of  intemet-basedmaterials  are  considered. 
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Introduction 

In  this  paper  we  discuss  work  in  progress  by  describing  the  conceptual  framework  we  are  using  to  identify  patterns 
of  collaboration  in  collaborative  teleleaming  scenarios  within  the  Norwegian  project  DoCTA 
(httn:.//wvvw, ifi.iiih.no/docta/").  Project  DoCTA  focuses  on  the  design  and  use  of  artefacts  in  collaborative 
teleleaming  scenarios  aimed  at  teacher  training.  Various  scenarios  utilising  the  Internet  are  used  to  engage  the 
students  in  collaborative  learning  activities.  An  ongoing  exploratory  study  is  analysing  four  different  scenarios. 

In  the  first  scenario,  a pilot  study  is  analysing  the  use  of  Teamwave  Workplace  (http://www.teamwave.com/)  for 
collaborative  activities  in  a graduate  university  course*  at  the  University  of  Bergen  (UiB).  The  next  two  scenarios 
involve  European  inter-cultural  simulations  where  the  goal  is  to  design  a textual  artefact  (such  as  a treaty  or  policy 
statement).  In  IDEELS^  teams  of  Norwegian  students  at  UiB  and  Nord-Trondelag  College  (HiNT)  collaborate  with 
teams  in  Germany,  Spain  and  France  to  develop  a treaty.  In  Demeter^  Parliament,  Norwegian  students  at 
Stord/Haugesund  College  (Stord)  collaborate  with  students  from  13  countries  to  contribute  solutions  to 
contemporary  problems  facing  the  European  community.  In  VisArt,  a fourth  scenario  being  designed,  developed 
and  deployed  for  use  between  Norwegian  educational  institutions,  the  goal  is  to  design  a visual  artefact  to  be  used 
in  teaching  a subject  of  choice.  In  this  scenario,  teams  will  comprise  students  from  the  three  Norwegian 
participating  educational  institutions  (UiB,  HiNT  and  Stord). 

Collaboration  Patterns 

From  a research  perspective,  the  exploratory  study  being  carried  out  within  DoCTA  will  provide  us  with  insight 
into  the  processes  of  collaboration  enabling  us  to  identify  collaboration  patterns  and  further  our  understanding  of 
how  instructors,  students  and  other  learning  facilitators  organise  their  learning  and  work. 

The  community  of  study  includes  teachers,  learners  and  facilitators  participating  in  the  various  collaborative 
teleleaming  scenarios.  The  main  research  question  has  been  formulated  to  ask  how  these  students,  teachers  and 
facilitators  organise  their  learning  and  work  given  the  different  scenarios.  The  four  collaborative  teleleaming 
scenarios  vary  with  respect  to:  (1)  actor  characteristics  (e.g.,  within  a common  community  vs.  disparate  and 
divergent  cultural  backgrounds;  similar  knowledge  and  preparation  vs.  different  knowledge  and  preparation;  etc.); 
(2)  aspects  of  the  learning  activity  (e.g.,  text  based  vs.  visually  based;  well-defined  learning  tasks  and  goals  vs.  ill- 
structured  tasks  and  goals;  etc.);  (3)  the  kinds  of  artefacts  they  have  access  to  (e.g.,  the  artefacts  provided  in  the 
various  internet  environments'*);  and  (4)  the  kinds  of  artefacts  they  are  to  design  (e.g.,  textual  or  visual). 

Conceptual  approach 

The  underlying  conceptual  framework  adopted  in  this  research  is  taken  from  three  different,  although  closely 
interrelated  approaches,  namely:  activity  theory  (Leontev,  1978,  Engestrom  1987),  distributed  cognition  (Hutchins, 
1995),  and  situated  action  (Suchman,  1987,  Lave,  1988,  Mantovani,  1996).  One  of  the  goals  of  this  research  is  to 
argue  that,  together,  these  approaches  make  up  a rich  framework  for  describing,  evaluating  and  analysing 
collaborative  teleleaming  scenarios.  All  three  approaches  underscore  the  need  to  look  at  real  activities  in  real 
situations  (Nardi,  1996,  our  italics),  and  always,  in  some  way,  include  the  context  in  studies  of  human  activity. 

The  rationale  for  combining  these  three  approaches  as  the  conceptual  foundation  of  this  study,  is  that  they  all  fall 
under  what  is  called  a sociocultural  perspective  (Wertsch,  del  Rio  & Alvarez,  1995),  that  highlights  learning  and 
thinking  as  phenomenon  that  can  not  be  studied  in  isolation.  Rather,  they  are  complex  processes  situated  or 
distributed  in  an  environment  — it  is  impossible  to  separate  them  from  the  context  in  which  they  occur.  The 
different  approaches  each  emphasise  slightly  different  elements  of  the  framework  that  is  important  to  be  aware  of 


* http://www.ifi.uib.no/staff/barbara/courses/host98.htnil 
^ http://ftp.uni-bremen.de/wwwgast/fehb/ideels/public  html/index.html 
^ http://hugin.hsh.no/prosiekt/demeter/index.htm 

'*  E.g.,  in  IDEELS  the  artefacts  they  have  access  to  include  their  own  email  system,  Teamwave  Workplace,  and  OPUSi  a web- 
based  conferencing  system  developed  at  the  University  of  Bremen,  Germany. 


in  these  kinds  of  studies.  Situated  action  emphasises  the  emergent,  contingent  nature  of  human  activity,  the  way 
activity  grows  directly  out  of  the  particularities  of  a given  situation  (Nardi,  1996).  Distributed  cognition  on  the 
other  hand,  asserts  as  a unit  of  analysis  a cognitive  system  composed  of  individuals  and  the  artefacts  they  use 
(Hutchins,  1991,  Nardi,  1996).  This  approach  underscores  the  distributed  nature  of  cognitive  processes,  and  the 
role  that  different  artefacts  play  in  these  processes.  Activity  theory  also  emphasises  the  mediating  role  of  artefacts, 
but  stresses  that  these  artefacts  carry  with  them  a particular  culture  and  history,  thus,  focuses  on  the  institutional 
and  cultural  elements  involved  in  the  learning  activity  (Kuutti,  1 996).  Adoption  of  these  approaches  provides  a 
strong  and  fruitful  conceptual  framework  that  informs  the  evaluation  of  collaboration  patterns  in  collaborative 
teleleaming  scenarios. 

Evaluation  approach 

The  evaluation  aims  at  a naturalistic  study  of  how  participants  in  collaborative  teleleaming  organise  their  work  and 
learning  activities.  Ethnography  (e.g.  Hammersley  Atkinson,  1983)  influences  the  design  of  our  evaluation 
approach  including  the  choice  of  data  collection  (e.g.,  participant  observations,  unstructured  interviews,  video 
recordings)  and  analysis  techniques  (e.g.,  discourse  analysis,  video  analysis).  This  means  that  the  evaluation  is  an 
iterative  process  where  an  ongoing  analysis  guides  the  data  collection  emphasis  in  successive  phases. 

In  order  to  collect  data  about  the  activities  that  the  students  engage  in  during  their  participation  in  the  scenarios, 
different  methods  and  techniques  will  be  used.  The  most  important  sources  of  information  will  be  derived  from 
observing  the  students  as  they  collaborate  and  interviewing  them,  and  also  from  electronic  logging  of  artefacts  used 
for  collaborating  (e.g.,  email,  shared  whiteboards,  chats,  to-do-lists)  and  artefacts  designed  (e.g.,  a web  page) 
during  the  collaboration.  It  is  a challenge  to  carry  out  the  participant  observations  since  a large  number  of  the 
students  are  geographically  distributed  over  Norway  thus  direct  observation  of  all  the  students  is  unrealistic  and  too 
costly.  Rather,  an  alternative  technique  consisting  of  immersing  ourselves  in  the  virtual  environment  in  order  to 
observe  their  activities  will  be  used.  For  this  reason,  the  electronic  data  logs  will  be  an  extremely  important 
supplement  to  the  “online”  and  “offline”  observations.  The  data  being  logged,  is  not  just  statistical  data  recording 
who  is  logged  on  when,  but  includes  a periodic  chronological  recording  of  all  artefacts  in  the  environment.  This 
means  that  we  can  recreate  versions  of  the  environment  to  study  the  use  of  artefacts  over  time  and  the  creation  and 
development  of  the  artefacts  produced  in  the  collaboration  process.  Data  collected  in  the  fall  of  1998  has  been 
analysed  and  used  to  inform  the  data  collection  in  the  spring  1999  scenarios.  At  EDMEDIA’99  we  will  be  able  to 
provide  a preliminary  report  on  our  findings. 
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1.  The  Transformations  of  the  Individual 

Historians  of  technology  show  us  in  myriad  ways  how  the  arrival  of  technology  changes  the  conception 
that  individuals  have  of  the  world,  the  relationships  with  the  objects  that  surround  us  and  social  relationships 
(Delfgaaw  1975).  We  can  distinguish  different  levels  of  transformation,  depending  on  the  impact  of  the 
technology  itself  (Attali  1985).  Information  technologies  are  already  announcing  a veritable  revolution:  the 
information  revolution.  This  is  an  unfinished  revolution  that  forms  part  of  the  worldwide  tendency  toward 
societal  acceleration,  a re-figuration  of  identity  and  our  place  in  the  world  (Morin  & Kern  1993),  and  a reduction 
in  the  restrictions  of  time  and  space,  at  least,  in  communicational  transactions  (Tiffin  & Rajaasingham  1996). 

Our  relationship  with  information  has  two  basic  dimensions:  operation  and  representation.  In  both  cases, 
the  information  refers  to  the  object  being  spoken  of,  and  also  to  what  is  being  said  about  the  object. 

In  the  operational  dimension,  we  manipulate  information  to  extract  data  on  the  object  being  referred  to. 
We  operate  the  information  in  order  to  get  more  information;  in  this  way,  information  is  understood  as  “a 
difference  that  makes  a difference”  (Bateston  1985).  Operating  information  requires  a previous  question  that 
imposes  intention  upon  the  action  of  operation. 

The  representational  dimension  has  to  do  with  the  products  that  result  from  operating  information;  it  is 
its  expressive  dimension.  The  representation  contains  an  internal  reference  to  the  intention  of  the  individual 
expressing  it  and  another  external  reference  linked  to  the  questions  of  the  individual  receiving  the  infomiation.  In 
both  cases  there  is  an  object  of  information  constructed  multi-dimensionally  according  to  the  socio-cultural  codes 
of  the  people  involved.  In  this  way,  information  is  a communication  bridge  between  individuals  that  are 
expressing  the  world,  and  therefore,  expressing  themselves.  Here  is  where  a first  level  of  transformation  appears, 
in  the  relationship  with  others  derived  from  information  technologies:  what  it  allows  us  to  represent. 

A second  level  has  to  do  with  the  way  it  allows  us  to  represent.  Information  technologies  are  intrinsically 
linked  to  the  communication  technologies  that  enable  communicational  expressions  to  take  place  a different  way: 
the  object  expressed  by  one  person  can  be  manipulated  by  another,  using  the  representation  itself 

So  our  relationship  with  information  (operation  and  representation)  and  with  others  (communication 
transactions)  is  transformed  by  information  and  communication  technologies  in  all  aspects  of  our  lives.  That  is 
why  we  consider  it  important  to  identify  the  elements  that  make  an  individual  computer  literate,  so  as  to  propose 
an  appropriate  literacy  process. 

2.  Computer  literacy 

In  the  human-computer  interaction,  we  find  a process  that  begins  with  a casuistical  exercise  of  action 
upon  the  tool,  and  that  leads  up  to  the  integration  of  the  tool  as  part  of  the  environment  of  “action  upon  the 
information”.  We  understand  this  process  in  the  following  way: 

The  process  of  computer  literacy  begins  with  a casuistical  interaction  of  the  person  with  the  tool,  which 
manifests  itself  as  “I  want  to...  How  can  I. . .?  Do  it...”  In  time,  the  individual  begins  to  identify  a set  of  patterns 
that  organize  his  actions  when  interacting  with  the  tool. 

Several  patterns  organize  interaction  with  informational  tools  on  a level  at  which  these  tools  are  seen  and 
operated  as  artifacts  for  completing  tasks  without  necessarily  being  integrated  into  daily  activity  as  an  integral 
part  of  the  individual’s  environment. 

The  organizational  and  representational  patterns  refer  to  the  interface  structures  that  allow  us  to 
understand  what  can  be  done  with  the  artifact.  These  are  structures  that  appear  in  such  a way  that  they  make  clear 
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how  the  computer  functions  internally.  In  this  way,  we  find  “iconic”  organizations  in  the  graphic  environments 
that  describe  what  is  immediately  possible,  and  indicate  the  most  common  actions. 

The  patterns  of  objects  and  their  structures.  Computers  contain  objects,  each  one  made  up  of  defining 
attributes  and  other  objects.  A document  is  an  object  whose  attributes  are  those  definitions  applied  to  each  of  its 
parts.  It  might  contain  sections  with  specific  attributes  that  are  different  from  the  attributes  of  other  parts  of  the 
document.  One  section  contains  paragraphs  each  paragraph  contains  words,  and  these  contain  letters.  An  object’s 
attributes  affect  all  of  its  parts  except  those  made  up  of  objects  with  their  specific  attributes 

As  for  the  pattern  of  processes,  they  allow  individuals  to  organize  the  set  of  actions  that  they  carry  out  to 
achieve  what  they  want.  There  is  a process  pattern  that  we  could  call  “second-level”  inasmuch  as  it  refers  to  the 
process  for  knowing  the  process  for  doing  what  I want  to  do.  It  is  the  search  process  we  apply  every  time  we  do 
not  remember  all  the  steps  for  obtaining  a specific  result. 

On  this  second  level  of  patterns  constructed  by  the  individual  for  interacting  with  information  tools,  one 
might  be  skillful  at  operation  without  necessarily  being  computer  literate.  This  is  achieved  on  a third  level  where 
among  other  things,  the  person  consciously  proposes  the  production  of  different  representations  of  the  object  in 
question  in  order  to  know  it  better,  to  know  more  about  it. 

This  level  is  made  up  of  the  combination  of  two  patterns:  the  handling  of  information  in  such  a way  that 
implies  intentional  action  upon  the  object  referred  to,  which  is  an  exercise  in  reorganizing  and  processing 
information;  and  the  pattern  of  metacognition  that  allows  individuals  to  account  for  the  way  they  produced  what 
they  produced.  This  third  level,  organized  on  the  basis  of  the  individual’s  intention,  implies  opening  new 
horizons  of  possibility. 

So  the  computer  literate  are  those  who  can  conceive  of  information  and  communication  tools  as  part  of 
their  environment  and  operate  them  as  such  intentionally  in  the  search  of  more  knowledge  about  the  objects 
referred  to  by  the  information  they  are  manipulating  and  expressing. 


3.  The  computer  literacy  process 

The  essence 

The  computer  literate  that  we  are  describing  is  basically  an  apprentice,  intentionally  looking  for  more  knowledge. 
This  leads  us  to  take  a look,  especially  in  formal  education,  at  the  way  the  individual  conceives  and  operates 
“how  to  know”.  We  identity  two  key  elements,  and  educational  strategies  must  be  devised  to  activate  them:  1 . 
The  individual’s  intention.,  as  the  origin  of  any  process.  2.  Metacognition.  As  a substantial  element  that  gives  the 
individual  a reference  point  in  his  or  her  process  of  knowing. 

Skills  for  handling  information 

We  are  talking  about  an  individual  that  searches  for  more  information:  he  or  she  proposes  various  representations 
of  these  objects  in  order  to  know  them  better.  In  this  sense,  we  cannot  conceive  separately  computer  literacy  from 
information  literacy.  If  a person  is  not  already  information  literate,  then  when  educating  for  computer  literacy,  we 
have  to  educate  for  information  literacy  as  well. 

Tool  work  in  spite  of  technological  change 

In  the  computer  literacy  process  we  should  recognize  that  the  capacity  for  tool  work  should  be  immune  to 
technological  change.  The  patterns  described  conform  a model  that  should  be  worked  on  three  different  levels: 

1 . Identifying  the  patterns:  Individuals  approach  to  technologies  is  quickly  and  systematically. 

2.  Operating  patterns:  Individuals  use  the  tool  for  their  particular  purposes.  It  implies  that  the  individuals  have 
the  capacity  to  build  and  rebuild  work  strategies  around  their  own  purposes. 

3.  Appropriating,  rebuilding  and  applying  the  model  and  the  personal  way  of  operating  it:  Individuals  move 
among  different  platforms  and  continue  to  operate  in  spite  of  technological  change. 
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Project  Description 

This  paper  is  a case  study  of  the  technical  trial  for  the  development  of  mobile  multimedia  technology  to 
create  a virtual  learning  space  on  the  World  Wide  Web  (NOMAD  - New  Online  Media  Alternate 
Delivery).  The  subject  of  the  trial  is  the  genius  loci,  or  spirit  of  place,  of  rock  art  to  support  a college  level 
Architectural  History  course.  The  fundamental  notion  is  that  architecture  is  the  ’gift  of  creating  places  for 
ritual’  (Kostof,  1995)  and  that  the  act  of  creating  a place  with  a handprint  is  the  beginning  of  architecture. 
In  part,  this  project  is  an  exploration  and  contrast  of  the  first  notion  of ’place'  (Casey,  1998)  as  created  by 
the  first  technologies  with  the  current  notion  of  'place'  created  by  current  technologies,  for  example 
cyberspace  (Meyrowitz,  1985). 

The  format  in  which  the  material  is  presented  is  intended  to  generate  interest  and  enhance  participation. 
Real  time  processing  and  presentation  of  material  through  the  web  is  used  to  create  a sense  of  participation 
in  the  adventure.  The  journey  unfolds  as  a story  in  serial  form:  Like  a novel  by  Dickens  published  in 
weekly  installments,  and  is  intended  to  whet  the  appetite  and  create  expectation  for  the  next  installment. 
The  general  structure  is  in  the  form  of  a serial  narrative  (Murray,  1997)  with  a beginning,  middle,  and  end 
but  with  each  phase  indeterminate,  unfolding  as  a journey  in  real  time.  The  NOMAD  Project  is  similar  to  a 
number  of  on-line  virtual  adventures,  for  example,  Terraquest’s  Virtual  Antarctica,  but  is  focused 
specifically  on  the  significance  of  place  and  ritual  in  the  making  of  architecture. 

Architecture 

Architecture  as  the  ’first  art’  is  a gift  of  the  gods.  Inspiration  is  the  ’breathing  in'  of  the  'genius’,  the  spirit  of 
the  Muses  (Frye,  1957).  Both  Humans  and  animals  live  in  caves  but  animals  do  not  have  architecture. 
Homo  Faber  - Man  the  Maker  changes  his  environment  to  support  his  activities,  understanding  the 
distinction  between  utility  and  meaningfulness  expressed  linguistically  as  the  difference  between  “in  order 
to”  and  “for  the  sake  of’(Arendt,  1958).  For  example,  making  a fire  pit  to  support  cooking  and  heating  is 
more  than  just  functional.  The  technology  creates  a place  of  social  gathering  forming  a group  with  a 
hierarchy  as  expressed  in  each  individual’s  relationship  to  the  fire.  Caves  are  perceived  as  connections  to 
the  underworld  and  the  womb  of  mother  earth.  By  placing  one's  handprint  on  the  cave  wall,  one 
participates  in  the  mystery  of  life.  The  simple  act  of  making  rock  art  marks  the  place  as  special  spiritually 
and  elevates  the  cave  to  the  realm  of  architecture. 

The  Cartesian  notion  of  space  and  its  relationship  to  time  (Casey,  1997)  is  distinct  from  my  use  of  the 
notion  ’place’  and  its  relationship  to  ’event'  which  is  fundamentally  phenomenological.  Cyberspace  is  a 
misnomer.  All  that  is  required  for  the  concept  of  space  is  the  definition  of  boundaries  and  limits.  The 
universe  as  limitless  space  is  curved  conceptually  providing  a boundary.  Place  always  involves  human 
agency,  even  if  it  is  as  minimal  as  giving  a name  to  a space.  Death  Valley  has  a history  and  image  that 
makes  it  a ‘place’  in  our  culture.  Any  material  intervention  that  supports  this  iconography  is  architectural. 
Cyberspace  does  not  meet  the  physical  requirements  of  space  but  it  can  be  thought  of  as  place,  or 
Cyberplace  as  it  meets  the  phenomenological  requirements. 


The  Aesthetic  of  the  Digital  Environments 


Murray  (Murray,  1994)  has  documented  the  three  principal  notions  that  enhance  the  experience  of  digital 
environments  as  immersion,  agency,  and  transformation.  Immersion  is  the  ability  for  consciousness  to  get 
'lost'  in  the  environment  as  in  being  'buried'  in  a book.  Agency  is  the  ability  to  impact  outcomes. 
Transformation  is  the  ability  to  'play'  at  being  something  else,  often  with  cathartic  effects.  How  can  these 
pleasures  be  exploited  in  NOMAD  and  to  what  end? 

Immersion  is  a measure  of  interest  in  the  content  (Murray,  1 994).  It  can  be  enhanced  through  quality 
graphics  and  text  and  the  narrative  structure  of  a journey  that  generates  a beginning,  middle,  and  an  end. 
The  unfolding  nature  of  the  serial  dclivciy  gerierates  expectation  m what  comes  next.  Key  is  the  timely 
generation  of  data  and  its  mounting  on  the  web  site. 

Agency  promotes  ownership  (Murray,  1994).  It  is  enhanced  through  quick  responses  to  online  queries  with 
answers  from  the  field  and  participation  in  the  online  forum.  A notion  of  the  project  is  that  students  will  do 
research  in  specific  areas  that  will  generate  questions  for  field  investigation  and  therefore  have  an  impact 
on  what  happens  next.  Agency  in  the  interface  can  be  enhanced  through  the  use  of  QuickTime  VR  which 
allows  the  user  manipulation  of  view  and  through  the  use  of  live  video  of  locations  from  which  the 
participants  can  capture  images  as  they  chose. 

Virtual  transformation  often  produces  cathartic  effects  that  generate  real  transformation  (Murray,  1994). 
The  hope  is  that  by  being  a virtual  traveler  one  will  become  a real  traveler  and  go  to  the  sites  visited 
virtually.  By  researching  a specific  area  of  the  content,  participants  break  down  the  barriers  between 
student  and  teacher.  By  being  a member  of  the  virtual  team,  the  participant  develops  real  team  skills. 

Responding  to  the  aesthetic  of  the  new  media  and  enhancing  notions  of  immersion,  agency  and 
transformation  in  the  construction  of  the  digital  environment  will,  I believe  advance  pedagogical  objectives 
through  increased  participation  and  user  satisfaction. 

Further  Research 

The  unique  elements  of  NOMAD  include: 

□ Using  multimedia  in  near  real  time  to  communicate  sense  of 'place' 

□ Using  a narrative  structure  to  organize  course  material 

□ A roving  or  mobile  instructor  developing  and  delivering  content  from  remote  locations 

□ Integrating  a variety  of  relatively  new  multimedia  applications  and  wireless  technologies 

□ Operating  under  remote  and  technology  challenging  conditions 
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Introduction 

The  computer  has  come  a long  way  from  a device  that  merely  performs  mathematical  calculations  for 
science  and  engineering  to  a device  such  as  multimedia  systems  that  can  mimic  the  mental  and  sensory 
abilities  of  a human  (Dannenberg,  R.  and  Blattner,  M 1992).  Multimedia  systems  have  made  possible  the 
different  representation  of  information  in  a variety  of  ways.  Through  its  interface,  multimedia  has  made 
possible  the  interaction  with  these  various  forms  of  information  through  the  various  human  senses. 
Multimedia  interfaces  uses  multiple  media  viz.  language,  graphics,  animation,  video,  non-speech  audio,  to 
communicate  with  users  (Roth,  S.  and  Hefley,  W.  1993).  Research  in  this  area  consist  of  a few  studies 
involving  the  effect  of  using  various  combinations  of  media  on  the  user’s  performance  offering  little  for 
clear  cut  designs  for  mixing  media.  Multimedia  generation  entails  the  following  processes: 

1 . Selecting  the  content  of  the  information  to  be  communicated 

2.  Choosing  what  media  to  use  in  order  to  communicate  the  information 

3.  Deciding  how  to  present  the  media 

4.  Coordinating  the  presentation  of  the  various  selected  media 

Just  like  in  a movie  where  the  effect  of  the  medium  depends  highly  on  the  proper  mix  and  relationships  of 
the  script,  choice  of  characters,  storyline,  cinematography,  music  and  setting,  multimedia  processes  raise 
issues  on  the  optimal  mix  and  relationships  between  the  various  formats,  use  of  models,  metaphors, 
navigation,  input  and  manipulation,  synchronization  and  exploitation  of  new  media  (Blattner,  M.  1994). 
This  is  because  historically,  design  of  interactive  multimedia  is  driven  more  by  intuition  rather  than 
empirical  research  (Park,  I.  and  Hannafin,  M.  1994;  Faraday,  P.M.  and  Sutcliffe,  A.  1997).  This  is 
exacerbated  by  the  designers  not  being  aware  that  design  principles  exist.  This  optimal  mix  issue  is  dealt 
with  through  empirically  guided  principles  formulated  by  Park  and  Hannafin,  Faraday  Sutcliffe  and, 
through  tools  for  presentation  with  embedded  expert  systems  in  the  field  of  artificial  intelligence. 

Multimedia  Interface  Design  Principles 

Park  and  Hannafin  formulated  20  research-based  principles  for  designing  interactive  multimedia.  These 
learning  principles  are  empirically  rooted  in  the  psychological,  pedagogical  and  technological  domains. 

The  psychological  principles  provide  important  insights  into  how  individuals  think  and  construct 
meaningful  learning.  These  are  based  on  theories  such  as  schemata,  meaningful  learning,  elaboration  and 
situated  cognition.  The  pedagogical  principles  combine’s  Gagne’s  taxonomy  of  learning  and  learning 
outcomes,  Ausubel's  theory  of  advance  organizer  and  Mayer’s  theory  of  meaningful  learning.  Together 
they  form  the  structure  for  multimedia  teaching  and  learning  environment.  The  technological  principles  are 
meant  to  provide  a vision  on  how  future  technologies  will  influence  the  teaching-learning  systems  and  the 
changing  role  of  the  teacher  and  the  student.  These  serve  to  suggest  improvements  for  creating  more 
powerful  and  efficient  technologies  for  the  future.  These  20  principles  are  intended  to  root  and  strengthen 
multimedia  in  contemporary  research  and  theory  without  unnecessarily  constraining  the  multimedia 
designer's  creativity  by  limiting  them  to  conventional  thought.  However,  it  would  be  equally  unwise  to 
allow  multimedia  designer  to  continue  their  design  approach  without  due  consideration  to  human 
processing  and  cognition  (Park  and  Hannafin  1994;  Blattner,  M.  1992). 

Peter  Faraday  and  Alistair  Sutcliffe  (1993)  drew  up  guidelines  for  designing  multimedia  interface  for  the 
Esprit  INTUITIVE  project  consisting  of  four  components  viz.: 

1 . creation  of  a task  model  with  information  requirement  specifications 
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2.  Resource  Model  describing  the  media  available 

3.  Selecting  the  media  appropriate  for  the  task 

4.  Scripting  a coherent  dialogue  combining  information  in  different  media 

In  another  project,  ’Etiology  of  Cancer’,  Faraday  and  Sutcliffe  (1997)  tackles  the  design  at  a micro-level 
through  a ”moment-by-moment  selection  of  appropriate  media  in  response  to  specific  cognitive  needs  and 
task  requirements”. 

Intelligent  Multimedia  Interface 

An  Intelligent  Multimedia  Interface  such  as  the  Intelligent  Multimedia  Presentation  System  IMMPS 
basically  assists  the  multimedia  system  designer  by  collecting  the  information  and  the  goal  for 
communicating  that  information  to  the  user  which  are  then  matched  up  with  IMMPS  archive  of  alternative 
media  and  presentation  techniques  (Roth,  S.  and  Hefley,  W.  1993,  Shih  1997a  and  Shih  1997b,  Bordegoni, 
M.  et.  Al.  1997)).  The  laboratory  prototypes  developed  are  of  the  type  that  generates  coordinated 
multimedia  presentations  such  as  WIP  which  presents  and  understands  combination  of  graphics,  text  and 
pointing  gestures  with  the  capability  of  interpreting  questions  and  designing  multimedia  answers  (Andre,  E. 
1997).  The  PPP  Personalized  Plan-Based  Presenter  which  creates  a script  with  an  accompanying  method  to 
present  the  material  (Andre,  E.  1997).  There  is  the  Columbia  Operations  and  Maintenance  Explanation 
Testbed  COMET  which  automatically  designs  integrated  textual  and  3-D  information  (Shih,  T.  1997).  The 
SAGE  automatically  creates  business  graphics  display.  The  TEXTPLAN  provides  narrated  animations  of 
directions  over  an  object-oriented  map.  AIMI  involves  the  user  into  a multimedia  dialogue  and  produces 
graphics  that  the  user  can  interact  with. 

Conclusion 

With  a complex  time-varying  nature  of  multimedia  elements,  issues  for  further  research  and  investigation 
of  multimedia  interface  will  continue  to  be  multimedia  input  and  output  (Roth  and  Hefley  1993,  Blattner 
and  Dannenberg  1992),  navigation  , visual  design  (Hannafin,  M.  and  Hooper,  S.  1989,  Grabinger,  S.1989, 
Lucas  L.  1991),  media  allocation  (Maybury,  M.  1993;  Arens,  Y.,  Hovy,  E.  and  Vossers,  M.  1993;  Faraday. 
P.M.  and  Sutcliffe,  A..  1997)  and  integration  of  media  (Roth  and  Hefley  1993).  To  be  generalizable,  the 
already  existing  models  and  tools  need  to  be  validated  and  tested  on  a variety  of  design  settings  and  on 
different  types  of  users. 
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Some  visitors  of  a website,  browsers,  simply  want  to  become  acquainted  with  a topic.  They  browse  the  home 
page  and  click  topics  to  see  what  further  information  the  site  has  to  offer  them.  The  success  of  the  site  partly 
depends  on  enticing  these  visitors  to  continue  the  exploration.  The  success  also  partly  depends  on  conveying  the 
site’s  underlying  structure.  The  browser  should  get  an  impression  of  the  ways  in  which  the  topic  is  treated. 

Other  visitors  come  to  the  site  with  a distinct  question  in  mind.  These  visitors,  searchers,  are  focused  on 
getting  a specific  answer  to  their  question.  They  want  an  immediate  answer  and  they  do  not  want  to  have  to  click 
through  several  layers  of  information  or  (sub)topics  to  get  it.  The  site  is  a success  for  these  visitors  if  it  gives  a 
comprehensive  and  correct  answer  without  requiring  an  inordinate  amount  of  eneigy  in  delving  it  up. 

Its  not  easy  to  serve  both  audiences  as  what  attracts  the  browser  may  repel  the  searcher.  In  this  paper  we 
sketch  our  attempt  to  handle  the  browser-searcher  paradox  in  designing  a website  on  technical  documentation. 
The  design  approach  we  have  adopted  for  dealing  with  this  issue  is  known  as  minimalism.  Minimalism  has  been 
used  predominantly  for  supporting  the  (new)  user  of  software  programs  such  as  word  processors  and  object- 
oriented  programming  languages.  Due  to  the  limited  space  that  is  available  for  describing  our  design,  we  discuss 
only  the  major  design  principle  to  anchor  the  topics  in  the  task  domain.  Also,  we  concentrate  on  a single  topic 
within  the  domain,  namely  screen  captures. 

A key  design  principle  is  that  the  site  structure  - the  number  of  levels  and  the  division  within  levels  - should 
reflect  the  underlying  domain  structure.  Some  topics  about  technical  documentation  can  be  treated  in  a single 
level.  For  example,  the  statement  that  quality  writing  evokes  quality  reading  is  treated  simply  as  a nice-to-know 
fact  appearing  only  on  the  Home  Page  to  entice  browsers.  Other  topics  need  stratification.  Screen  captures  are 
such  a topic.  Screen  captures  can  be  characterized  according  to  their  general  appearance  (i.e.,  design)  and 
function  (i.e.,  role).  Typically,  the  design  or  role  of  screen  captures  requires  further  specification  to  support  the 
work  of  a designer  of  technical  documentation. 

The  treatment  of  screen  captures  begins  not  on  the  first  or  on  the  second  level  of  the  website  but  on  Level  3, 
the  Theme  Page  (see  the  Figure).  The  reason  is  that  the  topic  represents  a rather  specialized  part  of  technical 
documents.  Along  with  other  parts  such  as  a goal  statement,  one  or  more  action  steps,  and  possibly  some 
problem  solving  information,  a screen  capture  - or  some  verbal  equivalent  - is  a constituent  element  of  a 
procedure.  The  main  function  of  a screen  capture  is  to  support  the  joint  handling  of  manual,  software  and  input 
device.  For  example,  a screen  capture  may  give  feedback  with  which  the  user  can  check  whether  (s)he  is  still  on 
the  right  track.  In  other  words,  screen  captures  give  coordinative  information.  As  this  kind  of  information  can 
also  be  given  verbally,  there  is  a need  to  compare  and  contrast  these  modes  in  delivering  coordinative 
information  in  procedures.  This  explains  the  rubric  and  its  organization  on  Level  3. 

The  stratification  on  the  levels  3 and  4 complies  with  the  taxonomy  of  screen  captures.  The  Theme  Page 
gives  a comprehensive  treatment  of  screen  captures,  briefly  describing  and  illustrating  the  main  design 
dimensions  and  roles.  The  Case  Page  details  each  of  the  four  dimensions  or  roles.  For  example,  users  who  turn  to 
this  Page  for  information  about  coverage  see  a page  with  an  identical  lay-out  and  structure  as  the  Positioning 
Case  Page.  In  addition,  the  organizer  with  brief  descriptions  of  the  four  design  dimensions  remains  on  top  of 
each  of  these  pages  to  convey  the  impression  that  these  dimensions  all  lie  on  the  same  level  in  the  hierarchy. 
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A special  problem  with  the  presentation  of  screen  captures  on  the  Theme  Page  and  the  Case  Page  is  that  there 
is  a conflict  between  the  preferred  technical  solution  for  displaying  pictures  on  the  website  and  our  design 
philosophy.  All  experts  that  we  consulted  on  the  question  of  how  to  display  the  (many)  pictures  (screen  captures) 
along  with  the  explanations  suggested  that  we  use  clickable  thumbnails  on  the  Page.  Clicking  the  thumbnail 
would  then  display  the  larger  picture.  This  is  strongly  advised  against  in  our  design  approach.  What  belongs 
together  should  also  be  presented  together.  Thus  we  sought  for  another  (technical)  solution.  One  of  the  two 
design  options  that  we  have  come  up  and  that  keep  picture  and  explanatory  text  together  is  using  an  expandable 
thumbnail.  That  is,  the  visitor  can  simply  enlarge  the  picture  by  clicking  and  dragging  a comer.  The  result  is  that 
the  text  is  pushed  downward,  but  still  remains  on  the  same  Page  (or  level). 

Surely,  there  is  a need  for  people  to  see  how  screen  captures  fit  into  a wider  context.  For  screen  captures  this 
context  is  a procedure  which  is  always  a main  component  or  chunk  of  information  in  technical  documentation. 
Therefore,  the  topic  is  treated  on  level  2 which  is  characterized  as  a Procedure  Environment  to  signal  the  wide 
variety  of  topics  that  one  should  expect  to  be  discussed  here.  Screen  captures  also  make  their  appearance  on  the 
first  level,  on  the  Home  Page.  On  this  Page  the  browser  is  given  a preview  of  a set  of  rather  diverse  screen  shots 
- in  thumbnail  format  - to  enhance  the  attractivity  of  that  page  and  to  invite  the  person  to  explore  that  (or 
another)  element  of  a procedure. 


Website  level  & Name 

Description 

Content 

Level  l^iHomelKafil 

Visitors  are  informed  in  an 
attractive  way  of  what  this  site 
has  to  offer.  All  Environments  are 
introduced.  Some  themes  and 
Cases  are  illustrated  with  an 
example. 

Environments 

- Testing 

- Design  approaches 

- Procedures 

Screen  captures^  preview 

Jievef  2 ilEn^rolimeht. 

Visitors  are  shown  how  each 
Environment  is  structured  or 
organized.  Typically,  these 

structures  are  based  on  validated 
taxonomies  or  typologies. 

Procedures 

- Conceptual  information 

- Constructive  information 

- Corrective  information 

- Coordinative  information 

jEevelf^STtminlfali 

Visitors  can  get  to  know  the  main 
topics  within  a Theme.  Typically 
there  is  an  overview,  a limited  set 
of  examples,  a recap  of  the  wider 
context,  a discussion  of  research, 
and  a preview  of ‘more  details’ 

Coordinative  information 

- Textual  variants 

- Screen  captures  - roles 

- Screen  cappires  - designs 

|^el|4:l@ase  PSq 

Visitors  receive  the  basic  facts 
about  the  topics  in  a theme.  There 
is  a definition,  an  exhaustive  set 
of  (annotated)  examples,  a 
discussion  of  design  variations 
and  related  research  findings,  and 
a section  on  design  considerations 

Screen  captures  - designs 

- Cueing 

- Size 

- Positioning 

- Coverage 

Full  vs  partial  screens 
Use  of  context 

Many  topics  are  still  to  be  resolved.  One  of  these  concerns  finding  a way  of  making  it  easy  to  access  information 
found  on  the  deeper  levels  of  the  site  (we  limit  the  site  to  four  levels,  as  shown  for  screen  captures).  An  option 
we  are  considering  here  is  using  a special  form  of  a preset  query  system  to  direct  searchers  quickly  to  the 
relevant  Level  and  topic.  For  example,  the  searcher  should  be  able  to  reach  level  3 or  level  4 directly  from  Home 
by  responding  to  the  query  ”I  want  information  about  ...”  with  respectively  ’’screen  captures",  or  "positioning 
screen  captures". 

In  creating  our  website  we  have  led  our  choices  be  affected  by  the  following  main  factors:  usability  issues 
(e.g.,  what  is  the  audience,  what  do  visitors  want)  topical  or  domain  characterizations  (e.g.,  taxonomies  or 
typologies),  the  minimalist  philosophy  with  its  corresponding  design  principles  and  heuristics,  and  finally, 
technical  considerations  (e.g.,  the  decision  to  abstain  from  using  plug-ins  to  serve  a wider  audience).  We  have 
found  the  approach  meaningful  but  also  progressing  only  slowly.  This  is  what  can  be  expected  in  minimalism.  It 
simply  takes  a lot  of  time  and  effort  to  create  a website  that  allows  people  to  do  more  with  less. 
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1.  Introduction 


Altliough  many  instructors  now  utilize  the  Web  to  supplement  their  courses,  the  Web  pages  they  create  are  mostly  ‘‘static**  in 
nature.  These  HTML  documents  are  written  and  then  posted  onto  the  Web  server,  where  they  sit  until  requested  by  a browser.  Materials 
on  these  documents  are  outdated  overtime  until  the  Web  developer  revises  them  one  page  at  a time,  which  requires  considerable  time 
and  effort,  especially  as  Web  pages  multiply.  In  the  long  run,  a better  strategy  to  maintain  a Web  site  is  to  use  “dynamic**  pages  linked  to 
databases.  These  Web  pages  are  ’’dynamic"  in  the  sense  that  they  are  created  “on  the  fly.”  They  don’t  exist  until  a browser  makes  the 
request.  Upon  the  information  user’s  request,  the  database  inserts  the  requested  information  into  preformatted  HTML  templates.  If  the 
information  in  the  database  is  updated,  the  Web  page  will  display  the  new  information  the  next  time  it  is  requested  by  the  user.  The 
purpose  of  this  paper  is  to  provide  an  overview  of  how  educational  Web  sites  can  be  more  easily  administered,  modified,  and 
customized  by  using  Web-database  connectivity. 

2.  Developing  Database-Driven  Web 

2.1  Advantages  of  using  database-driven  Web 

Web  browsers  provide  a Graphical  User  Interface  (GUI)  that  can  be  used  to  access  many  things,  including  a database.  Unlike 
tlie  traditional  database  management  systems,  the  user  would  not  have  to  configure  and  learn  to  program  database  client  software. 
Instead,  using  a Web  browsers  built-in  forms  capability,  users  can  access  a database  by  simply  filling  in  the  data  they  want  and  pressing 
a button.  The  returned  data  can  then  be  presented  in  an  easily  readable  format.  Furthermore,  the  Web’s  cross-platform  support  allows 
users  on  many  different  types  of  computers  or  platforms  (Windows,  Mac,  Unix)  to  access  a database  from  anywhere  in  the  world. 
Information  can  be  disseminated  with  a minimum  of  time  and  effort,  without  having  to  solve  compatibility  problems  (Lee  and  Liu, 

1998). 

There  are  several  advantages  of  using  database-driven  Web  development  over  the  conventional  “static**  Web  pages. 
First  of  all,  Web  pages  can  be  created  “on  the  fly*’  by  querying  current  information  stored  in  the  database.  New  and  fresh 
information  can  be  presented  to  the  user  at  each  visit.  This  technology  also  makes  it  easier  to  administer,  modify,  update,  and 
customize  a Web  site.  In  addition,  users  can  tap  into  vast  existing  databases  including  many  legacy  systems  through  Web- 
database  interaction.  This  allows  for  cross  platform  access  without  requiring  custom  client  applications.  Furthermore,  database- 
driven  Web  makes  it  possible  such  useful  functions  as  maintaining  user  state,  i.e.,  user’s  preferences  and  other  user-specific 
information  can  be  stored  between  visits  for  various  uses.  In  addition  to  these  benefits,  database-driven  Web  renders  particular 
advantages  to  educational  Web  sites.  Databases  are  “an  excellent  way  to  manage  the  media  components  of  a Web  site  . . . 
instructional  Web  sites  comprise  huge  numbers  of  media  objects  and  multiple  of  content  creators;  databases  make  managing 
this  information  and  longer-term  maintenance  issues  easier  to  address”  (Ashenfelter  1998). 

2.2  Mechanism  of  database-driven  Web 

Typically,  the  Web  server  delivers  a block  of  text  written  in  the  FTTML  to  the  browser,  which  parses  the  HTML  and  may 
request  additional  content,  such  as  graphical  data.  The  model  works  well  for  static  data,  but  what  about  live  data?  For  example,  how 
would  a Web  server  deliver  content  that  it  generates  based  on  user  input?  One  answer  is  to  use  a CGI  (Common  Gateway  Interface) 
script  to  handle  the  database  queries.  CGI  scripts  are  programs  that  let  Web  servers  execute  other  programs  and  incorporate  their 
output  into  HTML  documents.  The  Web  server  executes  a CGI  program  as  a separate  process  to  fulfill  a user’s  request,  which  can  be 
as  simple  as  a Web  page  hit  counter  or  as  complex  as  a database  query.  Because  a CGI  program  is  external  to  the  Web  server,  it  can  be 
written  in  virtually  any  language,  whether  compiled  or  interpreted.  Some  Web  servers  provide  libraries  and  interpreters  for  Java  and 
Visual  Basic  for  use  by  CGI  programs. 

While  CGI  programs  can  handle  basic  Web-database  connectivity  ftmctions,  programming  requires  a steep  learning  curve. 
CGI  programming  is  also  subject  to  such  limitations  as  input  data  validations  and  weak  security.  Recently  various  “middleware” 
products  have  been  developed  to  minimize  the  CGI  programming  time  and  additional  features  of  Web-database  connectivity,  including 
superior  security,  input  data  validation,  e-mail  protocol,  etc.  Middleware  is  a general  term  for  any  programming  that  serves  to  glue 
together  or  enhance  two  separate  programs.  Users  now  can  create  Web-database  applications  by  using  a fourth  generation  language 
middleware  products  such  as  Cold  Fusion  or  Active  Server  Pages. 
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Database-driven  Web  publishing  involves  three  major  components:  the  Web  forms,  CGI  or  middleware,  and  database 
system.  The  mechanisms  for  Web-database  interaction  are  shown  schematically  in  Figure  1 . There  are  four  major  stages  in  this 
interaction.  First,  the  user  fills  the  Web  form  using  the  browser,  requesting  pertinent  information.  The  browser  submits  the 
request  to  the  Web  server.  Second,  the  server  invokes  middleware  or  CGI  scripts,  accessing  and  querying  the  database.  Third, 
the  middleware  or  CGI  retrieves  or  produces  a HTML  document.  Finally,  the  Web  server  sends  the  result  to  the  browser  for 
display,  CGI  script  or  middleware  makes  it  possible  for  the  operations  such  as  opening  and  updating  appropriate  database 
tables  to  link  form  objects  to  a database  structure  specified  in  the  database  design.  The  Web  server  and  database  server  can  be 
hosted  in  a single  computer  or  two  separate  machines  connected  through  the  Internet, 


2.3  Design  of  a database-driven  Web 

To  design  an  effective  database-driven  Web,  three  major  design  activities  must  be  performed,  including  Web  form 
design,  database  development  and  CGI  or  middleware  programming,  Web  forms  are  necessary  as  a user  interface  for  a 
database-driven  Web.  The  information  user  can  send  a request  to  a database  by  submitting  it  via  Web  form,  A relational 
database  is  composed  of  rows  (for  records)  and  columns  (for  fields)  in  table  format.  Compared  with  a flat  file  structure, 
database  approach  has  the  advantages  of  minimal  data  redundancy,  information  sharing,  and  data  consistency.  Finally,  some 
programming  is  required  to  connect  the  Web  forms  and  databases.  For  example.  Active  Server  Pages  utilize  extensively  Visual 
Basic  programming  knowledge.  Visual  InterDev  utilizes  more  user-friendly  graphical  interface.  Cold  Fusion  eliminates  some  of 
the  complex  CGI  coding  by  combining  standard  HTML  with  a server-side  markup  language  called  the  Cold  Fusion  Markup 
Language  (CFML).  For  example,  the  CFML  tag  <CFQUERY>  can  replace  many  lines  of  CGI  programming  codes.  In  addition. 
Cold  Fusion  supports  such  advanced  features  as  security  integration,  dynamic  Java  forms,  data  entry  valuation,  e-mail 
integration.  Lightweight  Directory  Access  Protocol  (LDAP)  support,  and  advanced  SQL  (Structural  Query  Language). 

3.  Conclusion 

Static  Web  documents  are  cumbersome  and  time-consuming  to  maintain.  Whenever  part  of  the  contents  is  changed,  the  Web 
pages  need  to  be  updated.  While  dynamic  Web  development  may  require  greater  up-front  costs,  these  database-driven  Web  pages  will 
minimize  the  cost  of  providing  up-to-date  materials  to  the  information  user  in  the  long  run.  Instructors  can  simplify  the  means  of  adding 
and  updating  materials  on  the  Web  by  connecting  their  Web  pages  to  databases.  Carefully  designed  database-driven  course  Web  sites 
can  significantly  enhance  student-teacher  interactions  without  taxing  heavily  on  the  instructors'  valuable  time  and  effort. 


Figure  L Web -database  Interaction 
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This  paper  will  focus  on  two  applications  of  web  technology  to  learning  chemistry:  the  development  of  web 
sites  for  students  to  use  for  the  understanding  of  particular  topics  in  a physical  chemistry'  course,  and  the 
use  of  web  supplements  for  students  in  freshman  chemistry  and  physical  chemistry  lab. 

I.  WEB  SITES  FOR  UNDERSTANDING  PHYSICAL  CHEMISTRY  CONCEPTS 

The  author  is  developing  several  different  sites  for  use  in  the  teaching  of  a two-semester  course  in  physical 
chemistry.  There  will  be  a physical  chemistry  main  site  called  the  web  ’’handbook”  for  the  course  and  there 
will  be  many  other  sites  called  introductory  sites.  The  on-line  ’’handbook”  being  developed  will  reference 
introductory  sites  so  students  can  make  optimum  use  of  their  on-line  time.  The  ’’book”  will  include  the  use 
of  many  topics  which  the  students  can  ’’click  on”  to  go  to  one  of  the  introductory  sites  for  further  study. 

students  who  do  not  know  the  topic  can  go  to  the  site  for  an  in-  The  introductory  sites  will  contain  rather 
complete  introductions  to  concepts  involved  in  the  physical  chemistry  course,  and  will  include  examples  of 
applications  of  the  concepts.  The  author  is  particularly  concerned  with  the  students’  metacognitive 
development,  and  these  introductory  sites  will  attempt  to  clearly  show  how  the  understanding  of  the 
concepts  is  developed  so  that  further  applications  of  the  concepts  should  be  easier.  Those  students  who  are 
already  familiar  with  the  particular  topic  can  ignore  the  site  or  go  to  it  for  a quick  review.  Those  depth 
study  of  the  concept. 

To  help  students  judge  their  level  of  understanding  of  the  introductory  topic,  Java-based  concept  tests  will 
be  available  which  provide  instant  feedback  to  the  student.  A certain  level  of  understanding  is 
recommended  before  the  student  should  proceed  to  further  studying  in  the  handbook.  If  the  student 
performs  below  the  recommended  level  then  he/she  should  return  to  the  introductory  topic  for  further  study 
and  a retaking  of  the  concept  test.  Once  the  student  has  achieved  a satisfactory  score  on  the  concept  test 
then  he/she  should  be  prepared  to  proceed  to  the  topic  development  in  the  handbook.  The  handbook  will 
also  have  problems  for  the  student  to  work  in  applying  the  concepts. 

For  an  example,  one  may  go  to  this  site  (http://stem.kennesaw.edu/Hcombs/inter)  and  see  how 
intermolecular  forces  are  discussed  for  those  who  need  a review  or  for  those  who  seek  a more  complete 
study  of  an  introduction  to  this  concept.  This  introductory  site  would  be  referenced  from  the  on-line 
handbook  which  further  develops  applications  of  the  concept  of  intermolecular  forces. 

Will  there  continue  to  be  a hard  textbook  for  the  course?  At  this  point  the  answer  is  yes  for  the  ’’handbook” 
is  presently  being  developed  as  a supplement  to  a standard  textbook.  However  as  the  ’’handbook”  further 
develops  it  is  possible  that  it  will  become  the  only  text  and  it  may  be  in  the  form  of  a web  site  or  a CD, 
either  of  which  would  reference  the  introductory  sites.  The  author  envisions  the  introductory  sites 
remaining  as  web  sites  since  they  can  easily  be  modified  as  the  need  arrives. 

It  is  possible  that  this  method  of  study  can  be  extended  for  use  in  courses  taught  entirely,  or  mostly,  via 
distance  education.  We  are  developing  videos  of  pre-lab  lectures  so  that  students  can  view  the  pre-lab 
lectures  at  any  time  and  from  any  place,  and  as  we  develop  our  techniques  of  video  presentation  over  the 
web,  we  plan  on  making  use  of  the  technology  for  lectures  also.  The  web-based  video  lectures  will  use  an 
Elmo  for  showing  what  would  be  done  on  a chalk  or  white  board.  Although  it  is  difficult  to  imagine 
physical  chemistry  being  taken  totally  as  a distance  education  course,  it  is  possible  that  a market  for  such 


could  develop.  We  do  anticipate  offering  a course  in  Chemical  Literature  and  a course  in  General 
Chemistry  about  90%  over  the  web  by  next  fall  using  this  video  technology  and  web-site  technology  such 
as  will  be  described  in  part  II.  The  students  will  have  to  come  to  campus  or  other  proctored  sites  for  tests. 

This  short  talk  will  show  portions  of  the  "handbook”  and  portions  of  some  of  the  introductory  sites.  It  is 
hoped  that  by  the  time  of  the  meeting  that  some  video  portions  will  also  be  ready  to  demonstrate. 

II.  Web  Supplements  in  Chemistry 

The  author  has  been  using  a platform  called  Web-Course-in-a-Box  (WCB)  for  a couple  of  years.  WCB  is 
being  developed  at  Virginia  Commonwealth  University  and  a commercial  company  has  been  formed  for 
further  development  and  sale  of  the  product.  The  author  has  used  WCB  for  freshman  chemistry,  a senior 
seminar  course,  and  for  physical  chemistry  labs.  WCB  contains  six  components:  a section  on  the  class  roll 
which  allows  for  e-mail  to  be  sent  to  all  students  and  allows  the  students  to  develop  portfolios  for  the 
course,  a section  for  announcements,  a section  for  the  class  schedule,  a section  which  allows  the  students  to 
make  their  own  web  page  and  to  obtain  help  using  WCB,  a section  on  class  information  which  can  be 
anything  relating  to  the  class  including  web  sites  or  class  notes,  and  a section  which  can  contain  multiple 
bulletin  boards  and  a fully  developed  lesson  for  the  course.  The  section  for  the  bulletin  boards  is  very 
useful  in  that  many  bulletin  boards  can  be  formed  by  the  professor.  Separate  bulletin  boards  may  be  formed 
for  each  small  group  within  the  class  so  that  only  members  of  that  group  can  use  the  bulletin  board.  The 
small  group  boards  are  very  useful  for  group  projects. 

The  author  has  found  WCB  particularly  useful  for  a university  such  as  KSU  which  is  a commuter 
university  of  about  13,000  students.  Most  of  the  students  work  and  they  find  WCB  a very  helpful  way  of 
working  on  small  group  projects,  taking  concept  tests,  and  finding  notes  for  classes  that  they  have  missed. 
They  can  assess  the  web  site  after  they  leave  work  at  3:00  a.m.,  read  missed  notes,  send  e-mail  to  a study 
partner  or  to  the  professor,  and  work  on  the  bulletin  board  project  with  their  partners. 

KSU  is  changing  to  WebCT  this  summer  but  all  of  the  advantages  of  WCB  are  applicable  to  WebCT  which 
has  more  functionality  and  is  easier  to  administer. 

During  this  portion  of  the  talk,  the  author  will  demonstrate  the  use  of  WCB  in  several  courses  with 
particular  emphasis  upon  applications  in  a freshman  chemistry  course  being  taught  the  spring  semester  of 
1999.  The  corresponding  WebCT  site  will  also  be  demonstrated.  The  author  has  found  that  it  is  most 
useful  to  construct  a web  page  for  the  course  which  contains  the  course  syllabus,  homework  assignments, 
lecture  notes,  old  tests,  and  other  such  information  and  have  links  to  WCB  or  WebCT  from  this  web  site.  It 
is  much  easier  to  do  the  necessary  housekeeping  from  semester  to  semester  with  the  items  stored  on  a 
separate  web  site  rather  than  to  the  course  supplement.  The  author’s  general  web  site  for  a freshman 
chemistry  course  will  be  briefly  discussed. 
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Introduction 

Distance  learning  is  a fast  developing  field  with  many  different  aspects  to  it.  One  commonly  accepted  differentiation 
is  between  the  synchronous  and  the  asynchronous  aspects  of  distance  learning.  The  synchronous  aspect  refers  to 
learning  scenarios  such  as  study  groups  or  class  rooms.  Asynchronous  teaching  is  mainly  represented  by  exercises 
(Zitterbart  et  al.  98).  To  provide  a complete  environment  for  distance  learning  it  is  necessary  to  consider  both 
aspects.  Asynchronous  learning  has  been  investigated  to  a great  extend  and  many,  mostly  WWW  based  systems 
have  been  devised  to  support  it.  Synchronous  learning  in  this  respect  is  still  less  developed.  There  are  a number  of 
systems  available,  but  these  do  not  completely  cover  the  needs  of  synchronous  learning,  as  many  underlying  topics 
are  still  open  to  research. 

The  Modular  Advanced  Collaboration  System  (MACS)  developed  at  the  Institute  of  Operating  Systems  and 
Computer  Networks  (IBR)  of  the  Technical  University  of  Braunschweig  aims  at  providing  a flexible,  scalable  and 
easy  to  use  system  for  synchronous  distance  learning.  New  ways  of  visualizing  interactions  among  learners  and 
teacher  provide  network-based  learning  that  is  similar  to  real  life  scenarios.  Asynchronous  aspects  of  distance 
learning  are  covered  by  the  WWW-based  learning  environment  Tele  online/offline  Learning  (To/oL)  also  developed 
at  IBR  (IBR  98). 

Structure  of  MACS 

MACS  offers  an  open  framework  for  Computer  Supported  Collaborative  Work  (CSCW)  applications  as  needed  for 
synchronous  distance  learning.  Due  to  its  modular  design,  MACS  can  easily  be  configured  to  serve  different 
scenarios,  e.g.,  study  groups  or  class  rooms.  The  study  group  scenario  assumes  a small  number  of  participants  with 
equal  rights,  while  the  class  room  scenario  provides  extended  rights  to  the  teacher.  Larger  scenarios  to  model  other 
environments,  e.g.,  a conference  with  hundreds  of  participants,  are  thinkable. 

MACS  consists  of  three  different  levels:  network,  tools  and  control.  At  the  network  level  the  best  suited  protocol 
for  the  chosen  scenario  can  be  configured  by  taking  the  number  of  participants,  the  type  of  media  and  the  available 
infrastructure  into  account.  The  number  of  participants  is  mainly  determined  by  the  type  of  scenario.  The  type  of 
media  involved  can  result  in  drastically  different  requirements  (e.g.,  a text  tool/chat  vs.  high  quality  audio/video). 
The  infrastructure  available  to  the  participants  has  to  be  considered,  as  this  might  limit  the  choice  of  media  (tools) 
and  suitable  protocol,  thereby,  limiting  session  size  (i.e.,  number  of  participants).  The  tool  level  contains  the  main 
functionality  of  the  available/used  tools.  A session  is  created  or  joined  without  any  tools  activated.  Once  in  the 
session  it  is  up  to  the  learner  to  start  a selection  of  tools  and,  thereby,  create/join  the  corresponding  media  stream. 
An  example  for  the  motivation  of  not  automatically  joining  all  media  streams  is  a mobile  participant,  who  due  to  the 
bandwidth  limitations  of  the  wireless  connection  may  not  be  able  to  support  video  streams.  The  control  level  is 
responsible  for  system  and  session  administration.  The  system  administration  includes  the  maintenance  of  databases 
listing  the  current  sessions,  known  users  and  available  applications.  Mechanisms  to  create,  join,  conduct,  leave  and 
terminate  a session  are  offered,  as  well  as  mechanisms  to  invite  or  expel  a participant.  The  session  administration 
and  visualization  is  performed  by  a session  controller.  This  is  necessary,  as  it  is  possible  to  simultaneous  participate 
in  more  than  one  session. 

Session  Control 
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In  MACS  session  control  is  performed  by  a session  control  module,  such  as  the  Visual  Session  and  Floor  Control 
module  (ViSCO).  ViSCO  performs  session  and  floor  control  in  a virtual  meeting  room,  in  which  participants  are 
represented  by  thumbnails  (Fig.  1).  Various  state  information  about  the  participants  is  provided  through  the 
thumbnail  and  symbols  associated.  An  example  is  the  participants  multimedia  (e.g.,  audio  and  video)  equipment, 
represented  by  the  symbols  above  the  participants  thumbnail.  A camera  symbol  means  that  the  participant  has  a 
camera  available  and  can,  therefore,  source  a video  stream. 

Floor  and  session  control  are  also  performed  within  the  virtual  meeting  room.  If  a learner  requests  the  audio  floor,  a 
speaking  bubble  is  displayed  on  top  of  the  corresponding  thumbnail.  This  can  easily  be  recognized  by  the  other 
participants,  especially  by  the  current  floor  holder  who  may  decide  to  release  the  floor.  This  type  of  visualization  of 
social  situation  within  a session  helps  paiticipants  to  interact  in  a more  natural  way,  just  like  in  a conventional 
teaching  and  learning  scenario.  The  graphical  representation  depends  on  the  learning  scenario.  For  the  study  group 
scenario  (Fig.  1)  the  participants  are  arranged  around  a simple  table,  while  the  class  room  scenario  puts  the  teacher 
in  front  of  his  class. 

The  integration  of  the  session  control  into  the  virtual  room  provides  a consistent  user  interface  and  a strong  grouping 
of  control  elements.  This  prevents  the  well-known  problem  with  complex  user  interface  due  to  a large  number  of 
buttons,  sliders  and  other  control  elements  (e.g.,  MBone-tools)  (Kuma  96). 


Figure  1:  MACS  user  interface. 


Conclusion  and  Outlook 

There  are  different  scenarios  in  distance  learning,  each  with  its  own  set  of  requirements.  MACS  provides  a flexible 
and  modular  system  which  can  easily  be  configured  to  fit  the  scenarios  needs.  A strong  emphasis  is  placed  on  ease 
of  use  to  help  novice  learners  and  teachers  to  learn  the  system.  Feedback  about  the  current  session  situation  is 
provided  by  VISCO,  allowing  for  more  natural  learning.  Further  extensions  are  planned  and  other  scenarios,  besides 
study  groups  and  class  rooms  will  be  investigated.  For  further  details  please  visit  the  MACS  homepage  at 
htttp://macs.ibr.cs.tu-bs.de/. 
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Introduction 

This  paper  is  about  the  integration  of  multimedia  cases  in  elementary  science  teacher  education. 
These  cases  have  been  developed  within  the  framework  of  the  MUST-project  (Multimedia  in  Science  & 
Technology).  The  MUST-project  is  a joint  venture  on  behalf  of  three  Teacher  Education  Colleges,  the  National 
Institute  for  Curriculum  Development  and  the  University  of  Twente  in  the  Netherlands.  The  project  aims  at 
developing  multimedia  cases  for  the  professional  development  of  prospective  teachers  in  elementary  science 
and  technology  education.  In  the  first  project  year  two  working  prototypes  on  CD-ROM  were  developed. 
During  this  year  the  MUST-team  became  increasingly  aware  of  the  significance  of  the  implementation. of 
multimedia  cases  in  teacher  education  programs.  This  paper  ends  with  the  notion  of  flexibility-in-use  that  in 
our  opinion  is  a promising  way  to  reconcile  the  voice  of  theory  and  the  voice  of  practice. 

Multimedia  cases  and  a constructivist  learning  environment 

In  this  section  the  design  of  the  MUST  multimedia  cases  are  described  according  to  principles  for  the 
design  of  constructivist  learning  environments  (cf.  Honebein,  1995).  This  description  is  not  limited  to  the 
cases  themselves,  but,  if  appropriate,  also  the  broader  scope  of  teacher  education  programs  is  taken  into 
account. 

Embed  learning  in  a realistic  and  relevant  context 

Non-scripted  edited  video  of  (an)  elementary  science  lesson(s)  forms  the  core  of  every  MUST-case. 
These  video  clips  are  both  a realistic  and  relevant  context  for  prospective  teachers.  The  clips  are  edited  in  a 
way  that  they  provide  ample  opportunity  for  practising  analysis  and  contemplate  action  (cf  Merseth,  1996). 
So,  they  are  not  meant  to  follow  uncritically.  On  the  contrary  they  intend  to  stimulate  reflective  thought  and 
communication.  These  activities  are  also  supported  by  assignments  in  which  prospective  teachers  are 
encouraged  to  analyse  the  video  from  different  perspectives  and  use  these  knowledge  constructions  in  planning 
and  implementing  elementary  science  lessons  themselves. 

Different  perspectives  and  multiple  modes  of  representation 

The  MUST  CD-ROMS  include  comments  on  the  lesson  by  the  video- teacher,  experts,  and  prospective 
teachers.  So,  students  are  provided  with  experiences  from  different  perspectives.  However,  there  is  some 
controversy  among  specialists  in  case-based  instruction  about  whether  or  not  to  include  experts’  comments  into 
a case.  For  instance,  Merseth  (1996)  thinks  those  comments  may  inhibit  the  construction  of  knowledge  by 
novices.  When  novices  have  read  what  "experts"  say  about  the  case,  they  may  tend  to  abandon  or  suspend  their 
own  beliefs  in  favor  of  the  "delivered  wisdom"  (p.  733).  On  the  other  hand,  Shulman  (1992)  argues  that,  for 
example,  experts’  comments  provide  additional  perspectives  or  lenses  through  which  to  view  the  events  of  the 
case.  So,  they  add  complexity  and  richness  that  gloss  rather  than  simplifying  or  trivializing  the  events  (p.  12). 
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In  the  MUST-project  the  use  of  multiple  modes  of  representation  has  been  applied  in  different  ways.  Firstly, 
the  earlier  mentioned  edited  non-scripted  videotapes  and  audio  comments  on  the  video  represent  science 
lessons  in  elementary  classrooms  in  different  modes.  Moreover,  other  modes  of  representation  are  depicted  by 
all  kinds  of  textual  information. 

Ownership  an  voice  in  the  learning  process 

A third  design  principle  is  to  warrant  ownership  and  voice  in  the  learning  process  by  prospective 
teachers.  This  implies  that  they  are  encouraged  to  reflect  on  their  knowledge  construction  processes  and  take 
responsibility  for  setting  learning  goals  and  pursue  learning  processes.  In  the  design  of  the  CD-ROMS  of  the 
MUST  cases  this  principle  has  been  applied  by  constructing  an  open  uon-iinear  interface.  A second,  and  more 
important,  way  to  account  for  this  design  principle  is  to  formulate  assignments  that  have  an  open  character  and 
stimulate  the  users  to  reflect  on  their  learning  processes. 

The  voice  of  practise:  teacher  educators 

Especially  the  four  teacher  educators  in  the  MUST-team  refer  repeatedly  to  the  issue,  that  students  are 
not  familiar  with  working  in  a constructivist  learning  environment.  The  teacher  educators  question  how  much 
responsibility  their  students  may  handle.  Moreover,  they  are  worried  about  their  role  both  in  terms  of  their 
responsibility  and  in  terms  of  practical  implications.  The  former  point  may  be  illustrated  by  a remark  of  one  of 
the  teacher  educators:  she  stated  that  it  is  her  responsibility  to  do  her  utmost  that  students  will  reach  the 
predefined  learning  objectives,  so  she  wants  to  implement  one  of  the  multimedia  cases  in  a rather  traditional 
setting.  This  setting  consists  of  a science  method  course,  which  is  assessed  by  a uniform  test  for  all  students. 
The  latter  point  consists  of  concerns  of  teacher  educators  to  assign  all  different  types  of  more  open  assignments, 
such  as  portfolios.  Or  as  one  of  them  formulated  it:  For  time  reasons  it  is  simply  impossible  for  me  to  grade  all 
these  assignments. 

Conclusions:  towards  flexibility-in-use 

In  this  paper  a dilemma  is  sketched  between  the  ideal  of  multimedia  cases  in  a constructivist  learning 
environment  and  the  reality  of  teachers  and  students  in  teacher  education  programs.  In  order  to  overcome  this 
dilemma,  the  MUST  project  introduced  the  notion  of  flexibility-in-use.  This  notion  implies  that,  especially, 
teacher  educators  have  a considerable  freedom  in  the  way  they  want  to  use  the  multimedia  cases,  because  from 
an  implementation  perspective  it  is  neither  possible  nor  desirable  to  impose  a change  in  practice  on  teacher 
educators.  By  doing  so,  we  take  the  warning  of  Louden  and  Wallace  (1994)  seriously.  They  warn  reformers 
not  to  fall  into  the  trap  of  the  constructivist  paradox.  This  means  that  reformers  press  teachers  to  make  the 
transformation  from  conventional  to  constructivist  practise  in  a single  step.  We  opt  for  a more  realistic  reform 
agenda  in  which  learning  to  teach  with  multimedia  cases  is  perceived  as  a process  of  gradual  reformation  and 
elaboration  of  teacher  educators'  established  patterns  of  teaching. 
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Background:  CTC  Education  and  the  Virtual  Workshop 

The  Cornell  Theory  Center  (CTC)*  began  offering  asynchronous  distance  learning  over  the  Web  in 
1995,  with  its  first  offering  of  the  Virtual  Workshop.  The  Virtual  Workshop  is  a Web-based  set  of  modules  used 
to  provide  distance  education  in  high-performance  computing.  To  date,  we  have  conducted  ten  Virtual 
Workshops  for  about  1000  participants  from  academia,  corporations,  and  government  offices. 

A goal  in  the  Virtual  Workshop  evolution  is  experimentation  with  techniques  intended  to  provide  more 
interactive  components,  as  well  as  options  for  participants  with  different  learning  styles.  Features  we  have 
added  include  a glossary,  interactive  quizzes,  personalized  navigation  via  frames,  animations,  audio-tagged 
foils,  discussion  forums  and  whiteboard  conferencing,  and  a web  browser  interface  to  lab  exercises. 

One  interesting  new  technique  we  are  currently  incorporating  is  the  use  of  video  clips.  We  are  working 
with  two  types  of  clips.  The  first  type  is  screen  and  audio  capture  for  tool  demonstration.  The  second  is  using  a 
film  clip  of  a speaker  in  conjunction  with  other  Web-based  materials.  The  video  clips  will  be  used  to  enhance 
existing  online  materials.  The  clips  are  intended  to  improve  our  online  educational  materials  by  making  the 
content  more  engaging  and  easier  to  understand. 


Tool  Demonstration  Video  Clip 

The  tool  demonstration  video  clip  is  a screen  recording,  with  a speaker  voice-over.  Previously,  our 
online  materials  covering  use  of  graphical  tools  such  as  a debugger  or  execution  analyzer  would  use  text  and 
screen  stills  followed  by  a tutorial  which  walks  the  reader  through  the  steps  of  a simple  use  of  the  tool.  This  is 
very  tedious.  We  feel  that  providing  a method  analogous  to  looking  over  the  expert's  shoulder  while  s/he 
explains  the  steps  of  using  a tool  is  a faster,  more  effective  introduction. 


Design  Issues 

• Web  page  composition:  This  video  clip  is  presented  alone  in  the  web  browser  so  that  the  viewer  focuses 
fully  on  the  screen.  The  clip  will  include  typing,  mouse  movement,  and  menu  selection.  Too  much 
information  or  activity  on  the  screen  might  lose  the  viewer. 

• Video  clip  length:  If  the  topic  cannot  be  covered  in  less  than  the  ideal  web  video  length,  how  should  the 
parts  be  made  available?  Our  ultimate  goal  is  to  provide  an  index  into  the  video,  which  will  allow  the 
viewer  to  either  jump  into  the  desired  part  directly,  or  to  view  the  whole  segment  in  one  piece.  Other 
possibilities  are  to  leave  it  as  one  long  clip,  and  let  the  viewer  fast  forward  as  desired,  or  break  it  up  into 
logical  pieces,  then  play  one  at  the  time.  In  practice,  we  found  that  the  clip  for  one  topic  is  short  enough 
that  the  index  is  unnecessary  at  this  time. 

Additional  enhancements  we  hope  to  include  in  the  near  future  include  offering  several  clips  on  a 
single  topic  to  cover  different  scenarios  and  programming  languages,  and  making  supporting  material  available, 
such  as  a text  version  with  relevant  hyperlinks. 

There  are  several  design  options  that  we  have  decided  not  to  address  at  this  time,  due  to  tool 
constraints  and  development  costs.  They  include  slow  motion,  additional  screen  highlights,  and  offering  to 
execute  a lab  exercise  synchronized  with  the  demonstration. 


The  tool  for  the  clip  should  be  able  to  stop,  start,  pause,  resume,  forward,  rewind,  and  show  length  and 
current  time  through  the  clip.  These  are  common  features  that  are  readily  available.  Other  desirable  features 
include  editing  and  file  compression.  We  are  currently  using  HyperCam  from  Hyperionics. 


Speaker  Video  Clip 

The  second  type  of  video  clip  is  a speaker  talking  for  approximately  five  minutes.  Reasons  to  add  this 

type  of  clip  to  our  web-based  materials  include: 

1.  Give  the  materials  a human  look  and  feel.  Most  of  the  people  who  use  our  materials  never  see  the 
developers  and  consultants;  many  have  never  seen  Ithaca.  Adding  a "Welcome  to  the  Virtual  Workshop" 
clip  or  a clip  that  introduces  Virtual  Workshop  participants  to  the  consultants  would  serve  to  put  a face  on 
the  materials,  and  hopefully,  to  make  them  more  comfortable  interacting  with  us  by  email  or  chat  room. 

2.  Help  in  understanding  a difficult  concept.  Supplementing  words  and  pictures  with  an  expert  speaker  might 
provide  additional  help  for  difficult  points. 

3.  Break  up  the  monotony.  Although  the  materials  have  interactive  elements,  such  as  animations,  diagrams, 
quizzes,  simulation  programs,  and  lab  exercises,  they  can  still  benefit  from  an  interlude  of  this  type. 


Design  Issues 

• Web  page  layout  has  almost  unlimited  possibilities.  In  contrast  with  the  tool  demonstration  clip,  a 
speaker  clip  should  be  paired  with  other  activity  on  the  screen.  One  possible  usage  would  be  to  show  the 
speaker  next  to  the  html  page,  with  the  html  page  keyed  to  advance  at  the  proper  time  in  the  video.  This 
picture  most  closely  simulates  a live  lecture,  and  has  enough  activity  to  keep  the  viewer's  attention.  Early 
efforts  may  concentrate  on  having  a content  expert  introduce  a new  topic. 

• Length  is  also  at  issue  for  this  type  of  clip.  If  the  topic  cannot  be  covered  in  less  than  the  ideal  web  video 
length,  we  will  again  pursue  an  indexing  solution. 

As  with  the  tool  demonstration  clip,  the  tool  used  for  the  speaker  clip  should  be  able  to  stop,  start, 
pause,  resume,  forward,  rewind,  and  show  length  and  current  time  through  the  clip.  We  will  also  need  encoding, 
to  pair  the  video  with  html  pages,  and  acceptable  file  size.  We  plan  to  experiment  with  a method  to  highlight 
portions  of  the  screen  at  the  appropriate  time,  used  sparingly  to  focus  attention,  but  not  to  the  point  of 
annoyance.  We  are  currently  experimenting  with  RealPlayer  from  RealNetworks,  which  appears  to  have  the 
best  combination  of  availability  to  our  workshop  participants  and  desired  features. 


Production  and  Evaluation  Plan 

We  are  continuing  to  research  design  issues  and  prototyping.  We  expect  to  evaluate  both  types  of  clips 
in  production  workshops  in  1999.  Evaluation  will  include  questions  on  whether  the  layout  is  both  clear  and 
aesthetically  pleasing,  if  the  use  of  video  clips  enhance  understanding,  and  if  it  is  clear  how  to  use  and  navigate 
the  page.  Early  responses  show  that  over  75%  of  respondents  feel  that  the  tool  demonstration  clips  would 
enhance  the  materials,  but  less  than  half  feel  that  speaker  clips  would  be  helpful. 
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The  three  legs  of  the  stool:  Teaching,  Scholarship  and  Service  remain  the  three-pronged  pitchfork  by  which 
professors  are  raised  high  to  receive  promotion  and  tenure  or  pitched  out  of  the  bam  and  sent  on  their  way  to  find 
another  position.  Historically,  scholarship  has  been  judged  by  the  number  and  quality  of  publications  and  the  level 
of  prestige  of  the  journal.  Service,  the  weakest  of  the  legs  of  the  stool,  is  measured  in  terms  of  dedication  to 
thankless  positions  on  committees  or  in  furthering  other  university  goals.  Recently,  partnerships  with  school  systems 
and  community  organizations  has  been  a goal  of  many  universities  seeking  financial  support  from  federal  and  state 
grants  which  emphasize  collaborative  partnerships  between  universities  and  business  or  community  and  human 
service  organizations. 

In  the  past,  and  even  today,  the  quality  and  quantity  of  scholarship  and  service  can  be  estimated,  to  some  extent,  by  a 
paper  trail,  including  books,  journals,  and  a well  designed  vita.  But,  how  can  teaching  be  evaluated?  For  too  long, 
end  of  the  term  student  evaluations  have  been  the  major  source  for  judging  the  quality  and  effectiveness  of  teaching 
in  higher  education.  As  many  professors  and  administrators  realize,  this  form  of  evaluation,  administered  near  the 
end  of  the  quarter  or  semester,  is  often  reduced  to  a popularity  contest.  Year  after  year,  merit  pay  and  promotion  and 
tenure  decisions  are  often  based  to  a great  extent,  on  these  popularity  polls.  Short  of  video  taping  professors  in 
interaction  with  students,  there  seems  to  be  few  alternative  methods  to  document  and  subsequently  evaluate  teaching 
effectiveness. 

For  several  years,  authentic  assessment  has  become  an  increasing  priority  in  the  field  of  education  for  teachers, 
administrators,  and  curriculum  developers.  There  are  many  models  available  for  use  with  K-16  students.  Children 
in  preschool  and  primary  grades  make  judgments  with  the  help  of  their  parents  and  teachers  to  choose  their  best 
work  and  place  it  in  a folder  or  expanding  file.  Students  in  the  middle  and  upper  grades  may  continue  this  method 
and  may  take  pictures  of  their  projects  to  place  in  their  growing  portfolios.  With  increased  availability  of  computers 
in  the  schools,  students  in  third  and  fourth  grade  may  begin  to  use  simple  multi-media  software  to  document  their 
best  work  and  projects.  Goodlad,  Comer  and  other  proponents  of  educational  reform  recommend  that  schools  and 
colleges  of  education  provide  future  educators  with  multiple  experiences  in  classroom  settings  beginning  early  in 
their  educational  careers.  This  proposal  further  emphases  the  need  for  authentic  assessment  as  professors  attempt  to 
evaluate  the  pre-service  teacher‘s  growing  levels  of  expertise  in  the  educational  process.  Whether  the  decision  is  to 
develop  a paper  or  an  electronic  model,  the  concept  of  portfolio  development  is  a solid  one. 

The  College  of  Education  and  Human  Services  (CEHS)  of  Wright  State  University  has  been  recognized  nationally 
as  a leader  in  teacher  education.  A primary  concern  CEHS  is  that  pre-service  teachers  develop  skills  of  authentic 
assessment  to  document  their  own  professional  growth  and,  in  turn,  be  able  to  apply  in  their  classrooms  in  the 
future.  CEHS  has,  for  several  years,  assisted  pre-service  teachers  to  develop  hard  copy  portfolios.  These  portfolios 
are  constructed  during  the  foundation  portion  of  the  education  program,  added  to  during  the  methods  and  materials 
classes  and  groomed  during  the  student  teaching  experience.  More  recently,  the  authors  have  offered  pre-service 
instruction  in  developing  electronic  models  of  professional  and  teaching  portfolios,  including  CD  ROM  format  and 
web  page  format. 
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For  the  pre-service  teacher  to  the  president  of  a university,  the  professional  teaching  portfolio  is  becoming  an 
integral  part  of  the  educator’s  professional  files,.  A professional  portfolio  could  be  constructed  to  document  all  three 
areas:  scholarship,  service,  and  teaching,  or  any  one  of  the  three  in  isolation.  Professional  portfolios  may  be 
constructed  in  several  ways,  but  to  be  used  for  authentic  assessment,  the  portfolio  must  be  more  than  a scrapbook, 
containing  stubs  from  movies  or  basketball  games.  An  educator  could  begin  with  a hard  copy  and  easily  develop 
this  into  a CD-ROM  by  simply  scanning  the  documents  and  pressing  them  on  a CD-ROM.  Modem  technological 
advances  allow  for  use  of  multi-media  to  enhance  the  format.  For  example,  a simple  portfolio  could  be  constructed 
using  commercial  software  packages,  such  as  Scholastic's  Electronic  Portfolios  or  the  Grady  Profile.  Indeed,  any 
multimedia  presentation  software  may  be  used  to  develop  a template  for  student  portfolios.  Multimedia  programs 
such  as  HyperStudio,  Astound,  Podium,  Chisel  or  Powerpoint  could  be  used  to  develop  a portfolio  which  might 
include  actual  video  and  audio  footage.  This  addition  would  provide  a glimpse  of  the  educator  in  the  classroom  or  in 
individual  interaction  with  students.  Many  books,  as  well  as  the  World  Wide  Web  itself  may  be  utilized  as  models  in 
for  the  development  of  a portfolio.  The  resulting  document  might  then  be  stored  and,  in  turn,  viewed  from  a CD 
ROM  or  disk.  One  disadvantage  with  a CD-ROM  is  that  it  does  take  special  equipment  to  press  and  it  cannot  be 
readily  updated.  Current  technology,  however,  does  provide  for  re-writable  CD-ROMS. 

A web  page  may  allow  for  the  greatest  flexibility  in  creation,  storage,  updating  and  presentation.  In  the  past,  web 
design  through  programming  and  required  knowledge  of  html  limited  the  widespread  use  of  the  web  page  as  a 
professional  portfolio.  Now  that  programs  can  be  downloaded  from  the  web  and  software  purchased  at  a modest 
price,  web  authoring  has  been  simplified  and  made  understandable  to  a larger  population.  A web  search  engine  will 
identify  a number  of  on-line  tools  which  would  assist  the  user  in  webpage  development.  Some  World  Wide  Web 
sites  have  been  dedicated  to  housing  references  on  web  authoring  such  as: 

. . , WWW  Authoring  Information  (http://www.netspace.org/users/dwb/www-authoring.html) 

. . . The  Virtual  Library  of  WWW  Development  (http://www.stars.com/Vlib/) 

...  Educators  familiar  with  Clarisworks  will  adapt  easily  to  Claris  Homepage. 

. . . Current  versions  of  Pagemill  offer  the  novice  a path  to  webpage  production. 

More  flexible,  but  advanced  programs  might  include: 

...  Lotus  eSuite  DevPack  (http://esuite. lotus. com/preview/preview.nsf/Content/Home+Page) 

. . . GoLive  CyberStudio  3.0  (http://www.golive.com/three/gogetit/index.ehtml) 

...  Virtual  Media  Technology's  f)  Virtual  Media  Publishing  Suite  which  includes  HDK3,  INTRAWEB,  Web*lite, 
and  Hypershelf  3.0.(http://www.virtualmedia. com.au) 

. . . Matterform  Media  - HTML  Grinder  and  Theseus(http://www.matterform.com/design/) 

...  E-Publish  (http://www.stattech.com.au/) 

Educators  may  design  their  portfolios  to  document  items  included  on  their  curriculum  vita  in  the  areas  of 
scholarship,  teaching,  and  service.  In  the  area  of  scholarship,  text  files  with  grants  received,  and  proposals  which 
have  been  accepted  for  presentation  at  professional  conferences  might  be  included  on  the  web  page.  Educators  might 
provide  hypertext  links  to  their  articles  published  in  journals  or  to  the  professional  organization's  web  page 
announcing  a annual  conference  at  which  they  are  presenting  a paper.  In  the  area  of  teaching,  examples  of 
assignments  and  projects  might  be  included  as  a hypertext  file  or  as  a scanned  photo  with  sound  bytes  allowing  the 
students  to  talk  about  their  projects.  Students'  individual  web  pages  might  be  accessible  through  a professional 
portfolio  of  their  professor,  documenting  the  student's  professional  growth  even  after  graduating.  Service  could  be 
documented  on  a professional  educator's  webpage  by  providing  links  to  community  resources  and  information  on 
local  events.  Again,  text  files  of  grants  which  involved  partnership  schools  and  other  community  agencies  could  be 
included.  Letters  of  appreciation  from  students  and  others  might  be  collected  and  stored  in  an  electronic  portfolio 
format.  The  choices  of  what  might  be  included  in  the  portfolio  are  limited  only  by  the  creativity  of  the  educational 
professional  and  the  knowledge  of  the  software  available  to  make  it  happen.  As  an  exemplar  of  authentic 
assessment,  the  professional  educator's  electronic  portfolio  may  become  a template  used  to  document  individual 
and  professional  growth  in  the  business  world. 
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Introduction 

Many  medical  school  curricula  have  been  reorganized  in  the  past  20  years  to  incorporate  problem-based, 
application-driven  learning  (Norman  & Schmidt,  1992).  Similarly,  several  MBA  programs  have  been 
developed  with  projects  rather  than  separate  disciplinary  courses  as  the  primary  learning  activity  (e.g.,  Stinson 
et  al.,  1998).  However,  faculty  at  any  level  of  education  may  fear  that  a curriculum  organized  around  problems 
or  projects  will  not  provide  an  adequate  grounding  in  the  individual  scholarly  disciplines  that  are  the  basis  of 
course  sequences  in  a traditional  curriculum.  Providing  students  with  access  to  interactive  web-based 
disciplinary  resources  (chosen  or  developed  by  faculty)  for  independent  study  and  as  a problem  solving 
resource  is  one  way  to  address  faculty  concerns  about  disciplinary  coherence  in  an  application-driven 
curriculum. 

In  medical  education,  the  recently-developed  “clinical  presentation”  approach  to  organizing  curriculum 
content  capitalizes  on  the  motivational  value  of  problem-based  learning  while  providing  a framework  for 
learning  based  on  an  expert’s  organization  of  the  knowledge  (Mandin,  Harasym,  Eagle,  & Watanabe,  1995; 
Mandin,  Jones,  Woloschuk,  & Harasym,  1997).  The  clinical  presentation  approach  provides  students  with  a 
concept  map  for  each  of  a series  of  common  problems  (e.g.,  anemia,  chest  pain,  difficulty  swallowing,  fever) 
that  shows  how  an  expert’s  knowledge  is  organized,  linking  basic  causal  mechanisms  and  disciplinary  concepts 
to  specific  diagnostic  categories  in  a problem-specific  manner  (for  an  example,  see  Dane,  1 997).  Knowledge 
learned  in  this  framework  does  not  have  to  be  reorganized  in  order  to  be  applied.  When  the  causes  of  a problem 
are  learned  in  logical  groupings  that  help  learners  to  distinguish  among  the  causes,  diagnostic  judgments  are 
improved  (Klayman  & Brown,  1 993). 

The  Ohio  University  College  of  Osteopathic  Medicine  (OU-COM)  is  reorganizing  its  traditional 
disciplinary  curriculum  into  a Clinical  Presentation  Continuum  (CPC)  for  implementation  in  Fall  of  1999  for 
first-year  medical  students. 

Project  Goals  and  Design 

The  general  goal  of  the  Hematology  Web  project  is  to  provide  a complete  online  medical  hematology 
course  outline  with  slides  that  can  be  used  1)  in  a disciplinary  curriculum  for  large-group  presentations  or 
independent  study,  and  2)  in  a new  integrated  curriculum  via  links  to  application  topics.  Hematology  Web 
should  deliver  images  of  medical  quality  usable  on  a low-bandwidth  connection,  and  the  user  interface  should 
support  comparison  and  contrast  of  any  two  images  from  the  entire  project. 

Hematology  Web’s  user  interface  was  designed  to  emulate  the  menubar  and  pull-down  menus  of  a standard 
computer  program  in  order  to  make  efficient  use  of  screen  space  and  to  provide  consistent,  simple  navigation. 
Two  pull-down  menus  support  navigation  between  and  within  topics.  The  16  course  topic  titles  are  always 
available  in  one  menu,  and  the  web  page  anchors  for  the  currently-selected  topic  appear  in  a second  pull-down 
menu. 

Images  in  Hematology  Web  are  available  in  three  resolutions:  thumbnail,  medium,  and  large  (600  x 400 
pixels).  Space  on  the  right  side  of  the  screen  is  reserved  for  two  medium-sized  image  windows.  After  two 
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images  have  been  loaded,  the  next  requested  image  replaces  the  “older”  of  the  two  current  medium-sized 
images.  This  design  makes  it  relatively  simple  to  compare  and  contrast  any  two  images  in  Hematology  Web. 
Clicking  on  a medium  image  loads  the  large  version  of  the  image  in  a new  centered  browser  window. 

For  independent  study,  the  thumbnails  are  embedded  in  context  in  each  topic’s  textual  outline.  For 
presentations,  a slide  list  at  the  end  of  each  topic  contains  just  the  image  names  and  thumbnails  with  no  other 
text. 

The  user  interface  was  created  with  Javascript  and  frames.  Javascript  programming  manages  the  pull-down 
menus  and  the  loading  and  sequencing  of  images  and  labels  in  the  image  frames. 

Eighty  second-year  medical  students  using  Hematology  Web  as  a course  supplement  in  Fall  1998  were 
surveyed  concerning  user  satisfaction  and  suggested  improvements  pertaining  to  use  of  the  program  in  large - 
group,  coriipLHci  lao,  and  home  settings. 

Hematology  Web  has  met  its  design  goal  of  providing  a user-friendly  presentation  and  independent-study 
program  for  the  content  of  an  entire  medical  school  course  for  a conventional  or  application-driven  curriculum. 
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Introduction 

The  purpose  of  the  research  study  was  to  observe  the  application  of  learning  theories  to  cross-functional 
teams  in  new  product  development  and  how  individuals  acquired  skills  and  competencies  as  a result  of  their 
collaboration.  A cross-functional  team  consists  of  individuals  with  specific  but  different  skills,  who  work  together 
toward  a common  goal,  have  a common  purpose  and  follow  a common  approach  (Katzenbach,  1993).  For  purposes 
of  this  study,  skills  were  defined  as  those  relating  to  design  and  development.  Competencies  were  defined  as  the 
ability  of  the  team  members  to  communicate  effectively,  their  perceptions  of  working  in  a collaborative 
environment  and  their  level  of  confidence.  The  new  product  development  process  referred  to  the  process  of 
designing  and  developing  an  innovative  product  for  the  market. 


Theoretical  Framework 

The  theories  of  learning  and  cognition  included  in  the  study  were  cooperative  learning,  collaborative 
learning  and  situated  cognition.  Cooperative  learning  is  an  instructional  strategy  which  proposed  that  when 
learners  are  teamed  up  for  attaining  a common  goal,  their  cognitive  learning  gets  enhanced  (Slavin,  1991). 
Collaborative  learning  is  a learning  process  where  the  team  members  through  interacting  with  one  another  define 
the  learning  process  (Bruffee,  1993).  Situated  learning  is  a theory  dealing  with  the  acquisition  of  knowledge  in  a 
given  context.  (Brown,  Collins  & Duguid,  1989). 


Method  of  Inquiry 

The  research  study  was  conducted  using  the  naturalistic  research  design  (Guba  & Lincoln,  1985).  In 
naturalistic  research,  the  researcher  assumes  that  it  is  difficult  to  understand  any  phenomenon  outside  its  context. 
This  research  design  was  chosen  because  it  would  offer  a rich  understanding  of  the  process  of  learning  of 
individuals  involved  in  the  new  product  development  process. 


Participants 

The  participants  for  the  research  study  consisted  of  18  team  members  of  cross-functional  teams  working  in 
software;  hardware  and  telecommunications  related  companies.  Out  of  the  1 8 participants,  1 2 were  core  team 
leaders  and  decision-makers. 
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The  Instrument 


The  instrument  for  the  research  study  consisted  of  a researcher-developed  questionnaire  with  open-ended 
questions.  After  each  interview  was  transcribed,  case  studies  were  prepared. 


Results 

Most  of  the  team  iriembers  expressed  the  view  that  the  collaborative  environment  offered  them  a window 
into  the  product  development  process  as  a whole  and  not  just  their  part.  The  team  members  who  were  software 
engineers  were  able  to  understand  the  marketing  aspect  of  the  product  development  since  in  most  cases  the  market 
defined  the  specifications  for  the  new  product  under  construction.  The  team  members  were  usually  brought  forth 
skills  for  their  particular  area  of  expertise  but  often  times  had  to  learn  new  skills  during  the  course  of  the  project  as 
and  when  the  need  arose.  These  skills,  like  design  and  development  skills,  were  learned  in  collaboration  with  their 
peers.  Team  meetings  were  considered  crucial  since  in  some  cases  it  became  a platform  for  getting  feedback  from 
one’s  peers  on  the  work  done.  These  team  meetings  then  became  a learning  board  for  the  team  members.  It  was 
deemed  important  to  leave  the  lines  of  communication  open  at  all  times.  Once  the  team  members  felt  that  their 
opinions  counted  they  had  a sense  of  ownership  towards  the  project  and  attitudes  changed  for  the  better. 

One  aspect  that  came  up  time  and  time  again  was  the  fact  that  cross-functional  teams  were  the  way  to  go 
in  today’s  market.  The  products  were  complex  and  the  time  frame  short,  which  made  it  imperative  that  a cross- 
functional team  be  put  together  for  meeting  the  deadline  for  the  product.  Also  it  was  felt  that  no  single  person 
possessed  all  the  skills  needed  and  hence  the  need  for  a team  of  people  with  specific  skills. 


Importance  of  the  Study 

The  study  leads  one  to  believe  that  there  is  a need  for  cross-functional  teams  in  any  product  development 
because  no  single  person  could  provide  all  the  necessary  skills  needed.  The  study  leads  one  to  wonder  if  successful 
collaboration  is  due  to  good  facilitation  or  due  to  the  team  selection  process  or  due  to  the  projects  being 
challenging.  It  also  leads  to  the  question  of  whether  we  need  more  cross-functional  teamwork  among  the  faculty  in 
the  field  of  education.  And  if  so,  how  will  the  design  and  development  of  curriculum  be  and  how  will  this  affect 
the  delivery  of  instruction.  Finally,  in  the  area  of  Instructional  Technology,  collaboration  of  skilled  people  from 
different  content  areas  will  result  in  the  creation  of  innovative  multimedia  instructional  materials.  With  the  rapid 
growth  of  the  World  wide- web,  it  becomes  imperative  to  study  the  impact  of  collaboration  among  learners  spread 
in  different  parts  of  the  world.  How  will  this  impact  classroom  teaching  and  learning  remains  to  be  seen.  What 
adjustments  will  be  made  to  the  existing  classroom  structures  in  order  to  deliver  the  instructional  materials  in  an 
effective  manner  remains  to  be  seen. 
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Introduction 


The  Lilienthal  project  arose  due  to  both  the  emerging  interest  of  several  European  pilot  schools  for  computer 
aided  learning  and  the  decision  of  the  European  Community  to  foster  European  educational  multimedia 
applications. 

Apart  from  six  pilot  schools’  three  universities  and  the  Fraunhofer  Institut  fur  Arbeitswissenschafl  und 
Organisation  participate  in  Lilienthal.  The  latter  and  the  University  of  Stuttgart  do  the  project  management.  The 
other  two  universities  advice  with  their  theoretical  background  the  development  of  the  "Virtual  European  Pilot 
School"  and  care  for  the  research.  Hereby  the  University  of  Amsterdam  is  responsible  for  socio-economic 
concerns  whereas  the  Institut  fur  Didaktik  der  Physik  of  the  Humboldt-Universitat  zu  Berlin  focuses  on 
pedagogical  aspects.  Lilienthal  started  ‘98  and  will  last  for  two  years  including  demonstration  and  evaluation.  A 
pilot  study  will  be  carried  out  in  March  ‘99  and  it  is  foreseen  to  present  its  results  at  this  conference. 


Lilienthal 


A complete  pilot  training  takes  about  two  years  and  involves  several  examinations  with  differences  between  the 
different  nations.  The  whole  training  requires  about  750  hours  of  instruction  and  some  hundred  hours  of  flight 
training.  The  Private  Pilot  License  (PPL)  as  the  base  of  all  other  licenses,  is  handled  with  up  to  100  hours  of 
instruction  and  approx.  40  hours  of  flight  training.  This  varies  from  nation  to  nation.  The  participating  pilot 
schools  in  this  project  develop  an  European-wide  accepted  instruction  material  focusing  on  the  approx.  100 
hours  of  PPL-Instruction  to  reach  the  open  European  market  of  private  pilots.  This  instruction  material  is 
basically  developed  as  an  computer-mediated  system  not  only  in  order  to  make  didactic  use  of  the  new  technical 
possibilities  but  also  in  order  to  reduce  the  costs  for  the  students  as  well  as  for  the  often  recently  privatised 
schools.  This  system  will  finally  result  in  an  internet  based  Distance  Learning  Platform  (DLP).  It  offers  HTML- 
pages  as  well  as  multimedia  CBT  lessons,  which  can  be  run  through  and  discussed  asynchronously 
(Newsgroups,  email)  or  synchronously  (Electronic  Classroom  containing  audio  conference). 


The  Course 

Sense-Giving-Structure  SGS  is  a term  used  within  Lilienthal  for  the  idea  to  build  the  PPL  course  not  subject 
divided  but  in  interrelated  stages,  which  partly  run  parallel  to  the  practical  instruction.  Currently  the  number  of 
subjects  that  are  taught  for  the  PPL  varies  from  pilot  school  to  pilot  school  but  this  number  is  adjusted  to  the  nine 
subjects  of  the  general  European-wide  requirements  defined  be  the  Joint  Aviation  Authorities  (JAA).  These  are 
foreseen  to  be  integrated  into  the  national  laws.  If  you  write  the  9 subjects  like  Navigation  or  Meteorology  into  9 
columns  and  8 stages  of  the  practical  instruction  like  First  Touch  or  First  Flight  in  8 rows  a matrix  emerges.  Into 
this  matrix  were  provisionally  placed  the  108  topic  titles  suggested  by  the  JAA.  Corresponding  learning 
objectives  were  formulated.  These  were  discussed  in  a Delphi-process  [Dalkey&Helmer  1963]  and  finally  led  to 
an  European  Curriculum  based  on  agreed  detailed  learning  objectives. 

The  two-dimensional  structure  takes  the  opportunity  not  to  be  restricted  to  a weekly  timetable  any  more.  The 
student  finds  himself  in  semi-organised  environment,  represented  by  the  matrix-structure.  He  is  free  to  follow 
links  within  the  subject,  to  a Basics  course  about  the  science  behind  some  phenomenon  or  to  external  links  for 
additional  information. 


’ Lufthansa  Flight  Training,  Ecole  de  Pilotage  Amaury  de  la  Grange  Societe  Anonyme  (with  two  subcontractor 
schools),  KLM  Luchtvaartschool  B.V.  and  Horizon  Swiss  Flight  Academy  Ltd. 
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The  Knots  in  Hypermedia  Environment 


For  the  SGS  in  addition  to  the  subject  matter  experts  stage  responsibles  were  determined  who  crosscheck  the 
learning  objectives  for  possible  strong  relations  and  intersections.  These  central  points  match  in  particular  the 
possibility  of  hypermedia:  idea  converted  in  hypertext  is  to  organise  a topic* s content  in  single  information 

units  and  to  electronically  represent  these  in  a network  database  in  form  of  knots  and  links  between  knots.  ” 
[Tergan  95,  translation].  Research  will  focus  on  the  way  students  work  with  these  knots.  In  Lilienthal  most  have 
a scientific  content  like  the  concept  pressure.  The  student  will  meet  the  term  pressure  at  various  occasions  during 
his  way  through  the  course.  The  expectation  is  that  the  medial  representation  of  multidirectional  knots  assist  the 
linking  of  knowledge.  A hint  for  this  assumption  could  be  drawn  from  mc-tests  with  three  different  groups  of 
students.  The  first  group  will  meet  these  Science  Modules  via  links  from  the  SGS  as  described,  while  the  second 
will  be  given  the  modules  as  precourse  and  the  third  will  be  taught  in  traditional  classroom  teaching.  The  test 
will  enumerate  topics  and  ask  for  subjects  which  contain  these  topics. 


Learning  Modules 

All  modules  will  have  three  levels.  The  student  will  always  enter  a module  on  the  ’’mandatory”  level.  This 
starting  page  is  an  overview  about  the  linear  sequence  of ’’recommended”  learning  steps.  By  using  optional  links 
(the  links  are  marked  according  to  their  level)  the  student  can  enter  the  ’’nice  to  know”  level.  This  might  be  an 
external  homepage,  an  historic  anecdote,  a discussion  group.  The  ’’nice  to  know”  level  shall  serve  as  the 
* motivating  narrative  element**  [Waraich  98].  If  its  pages  assist  the  learning  process  is  open  for  investigation. 


Links 

The  important  difference  between  learning  in  a hypermedia  environment  and  traditional  learning  with  a book  or 
in  a classroom  is  the  easy  and  comfortable  use  of  links.  ** There  has  been  insufficient  time  for  the  field  to 
formulate  a detailed  theory  of  learning  from  such  systems**  [Shapiro  1998]  Hence  explorative  investigations  of 
the  new  medium’s  possible  advantages  are  needed  to  increase  our  understanding  hypermedia  learning.  In 
Lilienthal  research  will  focus  on  the  assumption  that  students  look  up  something  they  already  met  before  but  feel 
uncertain  about  more  often  when  they  use  the  comfortable  and  direct  link  to  the  corresponding  information.  It  is 
to  prove  whether  this  assumption  is  true  and  if  so  whether  this  really  leads  to  a safe  basic  knowledge. 


Evaluation  Design 


The  evaluation  methods  will  be  threefold:  questionaires,  tests  and  protocols.  Questionnaires  are  needed  to 
receive  the  students’  impression  of  the  learning  modules  and  their  personal  opinion  about  their  learning  progress. 
It  will  be  based  on  the  questionnaire  used  by  the  EuroMET-Goup  for  the  user  evaluation  results  [Phelps  and 
Reynold  1998].  Pre-  and  posttests  are  carried  out  in  order  to  measure  the  learning  progress.  Of  course  there  is  no 
guarantee  though,  that  individual  gain  of  knowledge  origins  from  the  corresponding  learning  module,  when 
students  can  interfere  with  all  kinds  of  information  sources  for  a period  of  3 months.  Hence  the  learning  success 
must  be  related  to  the  learning  paths  through  the  whole  course.  With  individual  learning  protocols  hypermedia 
offers  a unique  opportunity  to  recheck  the  students  actual  use  of  the  learning  media. 
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Introduction 

Web-based  instruction  (WBI)  is  an  emerging  delivery  system  used  by  public  school  systems, 
postsecondary  institutions,  military  training  centers,  and  business  and  industry  to  provide  learners  with 
education  and  training  opportunities.  There  are  literally  millions  of  web  sites  on  the  Internet  which  use  a variety 
of  themes  on  almost  every  topic  from  antiques  (http://www.antiquesonbroadway.com/)  to  zoos 
http :/www. zooatlanta.org/),  providing  global  users  with  a wealth  of  information  on  almost  any  topic.  In  the 
great  majority  of  cases,  these  sites  are  designed  for  information  sharing  rather  than  for  instruction.  Today, 
educators  use  the  web  for  a wide  variety  of  education  and  training  purposes  such  as  having  students  locate  sites 
to  gain  information  about  instructional-related  topics  (Ackerman,  1996;  Gamer  & Gilingham,  1996;  Kehoe  & 
Mixon,  1997;  Reddick  & King,  1996).  Others  have  used  the  web  for  sharing  information  through  listservs, 
newsgroups,  chat  rooms,  MOOS,  MUDs,  etc. 

WWW  & Instructional  Design 

To  harness  the  WWW  for  instruction  rather  than  just  gathering  information,  web-based  learning 
environments  must  be  created  following  principles  of  instructional  design.  Models  such  as  the  ADDIE  model, 
as  well  as  other  traditional  design  models,  follow  a fairly  linear  path  to  the  design  and  development  of 
instruction.  When  developing  instruction  for  the  WWW,  alternative  views  of  design  and  development  must  be 
employed,  taking  into  consideration  the  unique  capabilities  and  characteristics  of  the  web. 

A critical  component  of  successful  instruction  delivered  through  the  WWW  involves  the  active 
engagement  and  promotion  of  interactivity  in  the  learning  environment.  Instruction  that  provides  for  active 
engagement  depends  on  a careful  plan  that  integrates  strategies  that  attend  to  various  learner  characteristics  and 
WWW  features.  This  precisely  planned  instmction  leads  to  the  design  and  selection  of  appropriate  performance 
objectives,  instmctional  strategies,  and  technologies  (even  with  selecting  the  WWW  as  the  medium,  there  are 
other  media  selection  considerations)  for  the  delivery  of  the  instruction.  This  integrated  approach  to  analysis 
and  design  ensures  that  the  development  of  the  resulting  instruction  meets  the  desired  goals,  or  purposes  of  the 
lesson. 

Emerging  technologies  such  as  the  WWW  can  be  used  to  facilitate  and  promote  learning  in  web-based 
environments.  Learning  is  promoted  through  the  use  of  instructional  strategies  which,  in  turn,  provide  a 
framework  for  presenting  instructional  materials.  The  learning  environment  is  defined  by  the  boundaries  of  an 
instructional  situation  - class  session,  part  of  a course,  or  an  entire  program  or  course  of  study.  The  integration 
of  a variety  of  instructional  strategies  within  a learning  environment  facilitates  the  successful  learner.  It  is 
critical  for  designers  to  remember  that  these  technologies  serve  as  our  tools  that  help  students  learn  and 
instructors  teach.  The  designer’s  skills  and  creativity,  as  well  as  capabilities  of  the  delivery  systems  being 
accessed,  are  the  only  limits  on  how  these  technologies  can  be  used  in  the  development  of  instruction. 


There  are  a plethora  of  instructional  design  and  development  models  within  the  Instructional  Design 
and  Development  field.  Typically  they  cover  from  the  point  of  design  (micro  level)  to  those  that  look  at  the 
entire  process  (macro  level).  Andrews  and  Goodson  (1980)  conducted  an  analysis  of  the  basic  models  and 
described  specific  attributes  of  characteristics  of  over  60  models.  Since  that  time,  other  design  models  have 
been  revised  (Dick  & Carey,  1995)  and  new  ones  have  been  developed  (Smith  & Ragan,  1994).  Commonalties 
among  models  in  terms  of  elements  revolve  around  the  notion  that  they  are  systematic  in  process,  have  several 
stages  of  design,  development,  and  evaluation,  and  have  an  initial  stage  of  analyzing  goals,  learners,  and 
context. 

Web-Based  Instructional  Design  Mode! 

Web-Based  Instructional  Design  (WBID)  contains  the  basic  stages  of  analysis,  design,  development, 
implementation,  and  evaluation  that  are  also  found  in  other  models.  However,  the  arrangement,  or  structure,  of 
these  stages  is  less  linear  than  in  some  of  its  predecessors.  The  design  and  development  process  using  the 
WBID  model  is  an  iterative  and  ever-changing  and  evolving  process.  The  WBID  model  begins  with  an  analysis 
stage,  moves  to  a combined  design/development  stage,  followed  by  the  implementation  stage.  What  makes 
WBID  unique  is  that  once  the  analysis  stage  is  completed,  the  evaluation  stage  also  begins.  Evaluation  is 
continuous  through  the  design  and  development  stage  and  is  carried  through  the  implementation  stage.  Because 
evaluation  is  such  an  important  element  when  designing  effective  instruction  and  learning  environments,  it  must 
be  planned  and  conducted  throughout  the  product  design  and  development  stages. 

Each  stage  of  the  model  contains  and  culminates  in  decision  points,  action  plans,  or  products,  so  that 
the  shift  from  one  stage  into  another  and  back  again  is  relatively  seamless.  This  seamless  shift  is  especially  seen 
in  the  design-development  stage  where  prototypic  materials  and  products  are  developed  initially  before  moving 
into  production  of  a finished  product. 

This  model  is  a natural  way  for  instructional  designers  develop  materials  for  learning  as  well  as 
providing  a constant  refinement  and  enhancement  of  the  product  based  on  the  evaluation  information. 
Evaluation,  then,  is  no  longer  viewed  as  an  add-on  or  tail-end  stage,  but  an  integral  part  of  the  design  and 
development  process.  While  most  designers  say  that  they  conduct  periodic  reviews  of  materials  and  formative 
evaluation,  they  may  not  make  it  as  integrated  as  our  model  suggests. 

Conclusion 

The  World  Wide  Web  offers  many  challenges  and  opportunities  to  those  who  design  and  develop  web- 
based  instruction.  Traditional  models  of  instructional  design  do  not  meet  the  needs  of  designers  who  are 
designing  instruction  for  this  emerging  technology.  The  WBID  Model  facilitates  the  design  of  web-based 
instruction. 
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Introduction 

Pedanet  Project: 

1.  Supports  schoolbased  projects 

2.  Developes  web  based  services  for  schools 

3.  Implements  more  advanced  telecommunication  infrastructure  for  schools  in  Central  Finland 

4.  Implements  the  Peda.net  Schoolnet  in  Central  Finland 

Pedanet  is  a R&D  project  for  utilizing  the  opportunities  of  modem  information  technology  in  learning  and 
communication  as  a part  of  the  operation  of  educational  institutions.  The  target  of  Pedanet  is  to  contribute  to  all 
the  educational  institutions  and  the  municipalities  in  Central  Finland  by  setting  up,  and  advancing  the 
performance,  compatibility  and  economy  of  their  telecommunication  networks. 

The  essential  objectives  of  the  Pedanet  project  are: 

1.  To  activate  and  to  support  the  schools  in  the  planning,  development,  and  implementation  of  their  own  projects. 

2.  To  promote  equal  access  to  information  technologies  in  all  schools  regardless  of  current  levels  of  resources  and 
community  support.  Our  primary  concern  is  giving  small  schools  the  same  opportunities  currently  available  to 
larger  schools.  Through  the  schools,  we  reach  out  to  include  the  community  as  a whole  through  continuing 
education  and  the  provision  of  public  access  workstations. 

3.  To  evaluate  the  use  of  data  networks  in  the  schools;  and  evaluate  co-operation  between  schools. 

Goals  and  Methods 

The  goal  of  the  project  is  to  build,  by  the  end  of  the  year  1999,  an  integrated,  user-friendly,  and  pedagogically 
enriched  electronic  network  for  the  use  of  all  the  nearly  60  000  students  and  300  schools  operating  in  the  region 
of  Central  Finland, 

The  Pedanet  project  supports  the  development  projects  of  the  educational  institutions  in  helping  them  create  their 
own  innovative  ways  of  utilizing  telecommunication  networks  in  teaching  and  learning.  Teachers  are  offered 
opportunities  to  disengage  from  the  daily  teaching  work  for  a certain  period  to  work  in  their  own  Pedanet-related 
projects.  All  projects  include  research  activities  that  connect  the  projects  to  international  discussions  about  the 
utilization  of  information  networks  for  pedagogical  purposes. 

The  purpose  of  Pedanet  is  not  to  externally  guide  the  school’s  activities  in  their  own  development  work  but  to 
support  and  advance  the  implementation  of  these  projects  through  training,  evaluation,  research  work  and 
resourcing  by  offering  pedagogic  and  technical  expertise  in  form  of  various  information  network  services. 

How  Pedanet  supports  school  based  projects 

Finds  experts 

We  locate  experts  in  the  use  of  information  technology  in  each  region.  These  experts  try  to  identify  potential 
projects  that  would  be  valuable  to  communities  in  their  region. 

Starts  projects 


Ten  pilot  projects  were  selected  for  Pedanet  support.  These  projects  may  be  specific  to  a region  or  may  cover  all 
of  Central  Finland.  Projects  subjects  include:  teaching  methods,  teaching  contents,  materials,  and  processes  that 
have  the  potential  to  reform  pedagogical  practices  in  the  schools  involved. 

Supports  project  development  and  disseminates  project  achievements  to  others 

Pedanet  facilitates  contact  between  project  participants  and  experts  that  may  provide  them  assistance.  We 
provide  teachers  with  release  time  so  that  they  may  concentrate  on  their  projects.  We  also  provide  project 
participants  software  and  other  technological  support  for  instance  establishing  small  services  for  schools  on 
Internet. 

When  projects  near  completion,  we  will  publish  the  results  on  the  web  and  sponsor  conferences  to  share  what 
project  participants  have  accomplished:  their  successes,  the  problems  they  encountered,  and  anything  they  have 
learned  as  a result  of  participation  in  their  projects. 


Tools  and  Contents  of  Peda.net  Service 

Origo,  the  Heart  of  Peda.net  Service  and  the  Main  Tool  for  Virtual  School  in  Central  Finland 
All  the  300  schools  in  Central  Finland  will  have  an  access  to  Peda.net  Origo. 

The  Peda.net  Origo  is  developed  by  The  Future  Learning  Environment  (FLE)  project,  coordinated  by 
Medial aboratory  / University  of  Art  and  Design,  Helsinki,  University  of  Helsinki  and  Sonera  Ltd. 

Origo  includes  modules  for  learning  with  new  technology.  These  modules  are  called  FLE-tools.  (The  working 
title).  The  FLE-tools  are  designed  to  help  the  students  engage  in  productive  work  with  knowledge  objects  and 
artefacts  and  generate  new  objects  and  designs.  FLE-Tools  is  a groupware  system  designed  for  supporting 
collaborative  knowledge  building  over  the  Internet.  FLE-tools  consist  of  several  modules  that  are  designed  to 
facilitate  collaborative  knowledge  building  through  a constructive  process. 

The  Webtop;  The  environment  provides  each  user  with  a personal  open  ’Desktop’  in  the  web  ('webtop’).  The 
webtop  is  used  to  store  digital  materials  and  to  share  them  with  other  students. 

The  Discussion  Module  is  a shared  space  for  discussing  topics  and  concepts  generated  by  the  users. 

The  Jam  Session  module  is  a space  for  the  collaborative  construction  of  digital  artifacts. 

The  Library  is  an  adaptive  medium  to  publish  and  browse  multimedia  learning  materials. 

Other  Peda.net  Tools  and  Services: 

Interactive  OnLine  Magazine 

A tool  for  pupils  to  publish  shared  web  Magazines  online. 

Net  Publishing  system  for  schools 

A tool  for  schools  to  publish  information  and  notices  on  Web 
CUR-PRO'*  CURriculum  PROcessor 
Internet  accessible  application  for  Curriculum  development 
Document  conferencing  service 

Voice  Chat  - Text  Chat  applications,  internal  IRC  server 

Mailing  lists,  Open  Discussion  forums.  Thematic  Discussion  forums 

Notice  Boards,  Web  Calandars 

Pedanet  Commercial  - E-Shop  for  schools 

Information  and  News  service  for  schools 
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Introduction 

Its  about  time  professional  development  is  seen  as  something  other  than  teachers  sitting  in  a stuffy  room, 
drinking  cool  coffee  from  styrofoam  cups  and  listening  to  a speaker  who  may  or  may  not  be  an  expert.  Furthermore, 
instruction  online  and  instruction  6on-demand6  is  about  time  - time  that  teachers  have  in  small  segments  before  and 
after  school  and  during  preparation  periods.  And  finally,  the  nation  has  entered  the  Information  Age,  itOs  about  time 
professional  development  joins  the  revolution  and  participates  in  the  move  from  physical  space  to  cyberspace  for  the 
continued  training  of  its  teachers. 


Rethinking  Professional  Development 

With  ever  shrinking  budgets,  schools  and  school  districts  are  no  longer  able  to  provide  quality  professional 
development  in  the  traditional  manner.  With  budgetary  constraints  in  mind,  many  people  are  realizing  that 
professional  development  must  be  delivered  differently  and  must  be  constructed  differently. 

In  addition  to  changes  in  delivery,  professional  development  requires  revisions  in  the  way  it  is  constructed. 
First,  professional  development  in  the  Information  Age  must  include  a wide  array  of  learning  opportunities  that 
encourage  active  participation  in  experiencing,  creating,  and  solving  real  problems  (Jakson,  1997).  Second, 
professional  development  must  not  only  be  available  to  teachers  24  hours  per  day,  seven  days  per  week,  but  must 
take  into  consideration  the  audience  of  learners  created  by  those  teachers.  Finally,  professional  development  must 
provide  the  opportunity  for  teachers  to  reflect  critically  on  the  material  and  to  fashion  new  knowledge  and  beliefs 
about  the  content,  the  pedagogy,  and  the  application  in  the  classroom. 

Moving  inservice  training  from  stuffy  rooms  to  the  endless  expanses  of  cyberspace  allows  all  three  criteria 
for  revision  to  be  accomplished  and  for  professional  development  to  be  delivered  into  the  hands  of  the  Information 
Age.  The  following  article  will  present  not  only  an  overview  of  a professional  development  project,  entitled 
Excellence  thru  Linkages,  but  also  illustrate  how  the  professional  development  was  re-constructed. 


First  a Brief  Overview 

Excellence  thru  Linkages  was  a two  year,  funded  project  designed  to  deliver  technology  inservice  training 
to  a Native  American  school  in  the  northwestern  United  States.  The  first  year  of  the  project  involved  traditional 
training  and  focused  on  the  hardware  and  software  available  at  the  school.  Although  instruction  and  discussion  both 
included  the  integration  of  technology  into  the  classroom  curriculum,  little  or  no  evidence  of  computer  usage  existed 
at  the  end  of  the  first  year. 


An  overview  of  the  project  and  integration  of  critical  reflection 

Forced  by  the  inclimate  weather  of  the  winters  in  the  northwest,  a team  of  professors  and  graduate  students 
developed  the  idea  for  online  training.  Part  I of  the  venture  had  been  offered  in  a traditional  format  and  had  given 
the  faculty  at  the  Native  American  school  the  opportunity  to  become  computer  literate.  Although  the  technology  was 
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readily  available  and  they  had  the  basic  knowledge,  none  were  using  computers  in  their  classrooms.  Part  II  was, 
therefore,  developed  to  maximize  the  synthesis  of  technology  with  the  current  curriculum.  Nineteen  teachers  and 
three  administrators  participated  in  Excellence  thru  Linkages.  The  material  presented  was  divided  into  three 
modules: 

* integrated  software 

* research  and, 

* presentation. 

In  the  first  module,  integrated  software,  teachers  were  instructed  on  how  to  utilize  word  processing, 
database  and  spreadsheet  as  classroom  tools.  Activities,  applicable  to  numerous  subject  areas  and  grade  levels  were 
suggested  as  were  Internet  sites  for  lesson  plans  using  integrated  software.  Teachers  were  given  three  weeks  to 
review  the  online  material  and  to  try  one  of  the  activities  with  their  students^  and  to  reflect  on  the  results  through  the 
use  of  an  online  journal.  The  online  journal  was  a critical  component  in  this  project,  as  the  process  of  reflection  and 
writing  created  a stimulus  for  teachers  to  better  understand  their  own  fears  and  motivations  for  teaching  with 
technology.  When  attempted  lessons  did  not  result  in  the  expected  outcome,  the  journal  again  allowed  for 
reflection.  At  the  end  of  the  three  weeks,  a required  chat  encouraged  teachers  to  share  their  successes  and  failures 
and  to  communicate  ideas  of  how  this  software  package  could  be  utilized  in  the  classroom.  Teachers  were  also 
required  to  submit  no  less  than  two  ideas  for  using  integrated  software  at  their  grade  level. 

The  second  module,  research,  provided  information  for  use  of  the  Internet  as  a classroom  tool.  Again, 
teachers  were  given  three  weeks  to  visit  the  sites  offered  in  the  online  professional  development  and  evaluate  the 
activities  with  their  students.  The  chat  again  encouraged  sharing  of  ideas.  Mandatory  activities  and  lesson  plans 
were  not  required  by  the  administration  during  the  research  module,  as  teachers  were  becoming  more  self-motivated 
and  willingly  posted  ideas  to  the  listserv. 

The  third  and  final  module,  presentation,  reviewed  the  use  of  PowerPoint,  HyperStudio  and  Digital  Chisel 
as  classroom  presentation  tools.  The  same  time  line  and  format  was  again  used.  This  time  teachers  and  their  students 
posted  the  results  of  their  integration  of  the  software.  PowerPoint  and  HyperStudio  presentations  were  uploaded  to 
the  webpage  that  hosted  the  online  professional  development. 


Summary 

While  the  project  had  many  positive  outcomes,  the  world  of  online  professional  development  still  has 
numerous  hills  and  valleys  to  traverse  before  becoming  the  ultimate  answer  for  continued  teacher  training.  This 
project  allowed  for  the  reconstruction  of  teacher  training  and  the  launch  of  that  training  into  the  Information  Age. 
Excellence  thru  Linkages  has  only  touched  on  the  possibilities  of  what  is  to  come  as  the  Internet  and  modem 
technology  combine  in  the  world  of  education. 
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Background 

At  the  University  of  Gothenburg  we  have  for  two  years  been  working  with  a particular  course 
curriculum  as  a “stomping  ground”  for  developing  net  related  pedagogics.  The  courses  in  Social 
economy  have  Problem  Based  Learning  (PBL)  as  a pedagogical  approach  and  the  courses  are  built 
upon  the  use  of  computer  mediated  communication  in  distance  education.  The  PBL  approach  is 
described  by  the  CLEA’  management  as  a structured  model  in  which  a number  of  necessary  steps  will 
be  used.  A number  of  cases  are  the  starting  point  for  the  PBL  process.  The  case  is  presented  through  a 
vignette  that  might  be  presented  as  a picture,  a saying  or  a description  of  a situation  and  it  will  be  of 
relevance  to  the  aim  of  the  course.  The  students  in  the  course  will  then  build  the  case  around  the 
starting  point  depending  upon  what  they  have  found  to  be  interesting  in  the  presentation.  In  addition  to 
the  introductory  presentation,  a number  of  relevant  terms  will  be  supplied,  which  will  mark  out  and 
enforce  the  character  of  the  particular  case.  The  idea  is  that  the  students,  working  in  dedicated  groups, 
will  seek  knowledge  on  their  own  from  different  sources. 

The  evolution  of  the  course  has  led  to  a new  entirely  net-based  application  called  Workspace,  we  will 
return  to  the  thoughts  behind  the  creation  of  this  below. 

Method 

In  all  and  together  we  have  covered  most  aspects  of  this  course,  enabling  us  to  give  “thick 
descriptions”  of  the  processes  at  hand.  This  paper  merely  presents  an  overview  of  the  rich  material  that 
has  not  yet  been  fully  analyzed.  The  students  answered  an  electronic  questionnaire  on  the  web.  The 
questionnaire  dealt  with  questions  about  the  students’  background  as  well  as  how  the  start  of  the  course 
turned  out  and  what  problems  the  course  had  at  that  moment.  Both  individual  interviews  of  students 
and  a group  interview  of  the  supervisors  have  been  carried  out.  The  individual  interviews  have  been 
structured  out  of  a questionnaire  to  the  respondent.  In  the  interview  situation  the  questionnaire  has  been 
used  as  a support  for  the  interviewer,  not  as  a fixed  agenda.  The  overall  aim  has  been  to  let  the 
respondent  talk  freely  as  much  as  possible.  Almost  all  of  the  supervisors  were  present  at  the  group 
interview  as  well  as  the  system  administrator.  Kvale  (1996)  finds  that  the  interaction  among  interview 
subjects  in  a group  interview  often  leads  to  spontaneous  statements  about  the  topic  being  discussed,  but 
that  group  interviews  ’’generally  require  that  the  interviewer  has  a good  grasp  of  the  interview  topic” 

(p.  101). 

The  interviews  have  been  carried  out  from  May  1997  to  May  1998.  So  far,  this  work  has  resulted  in  31 
interviews  with  students  (out  of  50  students),  and  the  additional  interview  with  the  supervisors.  Most  of 
the  interviews  have  been  written  down  in  order  to  improve  the  quality  of  the  analysis  (compare  Palmer, 
1969). 

Results 

Results  from  the  questionnaire  showed  that  the  students  experienced  the  course  as  time  consuming  and 
the  students  had  also  problems  in  finding  literature  to  solve  the  tasks/cases  given  to  them.  The 


^ CLEA  was  the  name  of  the  European  Union  project  that  produced  the  original  courses. 
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questionnaire  also  showed  that  some  students  had  left  the  course.  The  teachers  each  had  a different 
approach  to  working  with  the  students  on  the  net.  However,  the  main  common  role  for  the  teachers  is 
that  of  facilitation.  This  enhanced  the  learning  process  of  the  students  as  the  teacher,  who  became  the 
“facilitator”,  acted  merely  with  a supporting  role  based  on  what  he  knew  about  the  learning  process  - 
this  rather  than  taking  part  of  the  specific  learning  process  discussing  subject  matters.  In  participating 
in  the  course  the  teacher  learnt  a lot  as  well  as  the  students,  something  that  we  find  to  have  a lot  in 
common  to  the  discussion  about  symbolic  interactions  (Blumer,  1962). 

One  of  the  facilitators  chose  to  further  his  experiences  from  this  course  together  with  the  technician  of 
the  course  and  developed  a new  computer  application  called  Workspace.  The  advantages  of  this  can  be 
summarized  in  the  sense  that  Workspace  does  not  reproduce  the  classroom  in  the  way  that  the  used 
software  WEST  did.  It  also  has  many  more  possibilities  to  create  discussion  groups  and  to  search  for 
information  in  these  through  an  engine  for  statistics  as  well  as  content.  Workspace  is  built  with  PBL  in 
mind  and  is  therefore  fully  developed  for  the  encounters  that  PBL  can  create. 

Discussion 

A description  of  the  development  process,  from  the  perspective  of  how  the  students  experience  their 
learning  situation  connected  to  the  learning  of  the  teacher,  is  the  primary  target  of  this  section.  This 
perspective  is  imperative  to  the  continuous  process  of  building  and  maintaining  the  group  of  students 
that  emanates  (Husserl,  1960)  out  of  the  understructure  of  the  student  body.  This  understructure  is 
enhanced  by  the  process  of  role  taking  (Mead,  1934)  over  the  net,  which  has  somewhat  different 
preconditions  to  the  role  taking  or  socialization  (Berger,  1967)  of  a student  in  an  ordinary  course.  The 
mere  knowledge  of  these  mechanisms  is  a precondition  to  facilitate  this  process.  For  indeed  it  is 
facilitation  rather  than  teaching  or  supervising.  The  “facilitator”  rather  than  the  teacher  has  not 
restricted  the  students’  learning  process  and  therefore  both  the  learning  process  and  the  subject  matter 
is  developed  more  intensively  than  among  the  students  of  a teacher.  One  of  the  large  advantages  with  a 
course  like  this  is  that  the  student  is  no  more  and  most  important  no  less,  than  their  identity  on  the  net. 
You  are  not  intimidated  by  “normal”  social  handicaps  like  shyness,  looks,  speaking  problems  etc.  The 
preconditions  seem  to  be  others,  like  primarily  ability  to  express  yourself  through  writing. 

Time  being  the  foremost  variable  of  human  interaction,  rather  than  space  due  to  the  psychological 
permeation  of  that  dimension  (Bergson,  1912)  This  is  obvious  in  that  the  space  dimension  is  almost 
evidable  and  has  a mere  representation  (Chaib,  1996)  in  the  virtual  room  created  for  the  Workspace. 
Time  being  the  factor  expressed  as  synchronous  or  asynchronous  representation  in  the  process  of  the 
student  discussion.  Being  asynchronous  in  the  beginning  of  the  learning  process  and  as  it  becomes 
closer  to  the  deadline  of  the  task  the  process  becomes  more  synchronous.  This  process  also  accounts 
for  the  student’s  experience  that  the  course  was  time  consuming.  This  is  due  to  the  experience  that 
synchronous  interaction  on  the  net  with  instant  feedback  is  more  effective  towards  the  end  of  a task 
since  the  result  needs  to  be  summarized  and  systemized.  This  whereas  the  process  of  brainstorming  of 
presentation  of  new  ideas  or  thoughts  is  better  presented  in  an  asynchronous  environment  since  it  gives 
more  liberty  of  thought  for  the  particular  student,  since  it  is  not  met  by  a direct  argument. 
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Abstract:  This  research  examines  how  telementoring  can  provide  introductory  field 
experiences  for  preservice  teachers  through  Internet  links  with  classroom  teachers  as  well  as 
improve  student  telecommunication  skills.  It  contains  a discussion  of  factors  impacting 
successful  mentoring  derived  from  three  semesters  of  data  collection. 


NCATE/ISTE  standards  (ISTE,  1998)  require  that  students  have  a working  knowledge  of 
telecommunications  and  its  use  in  the  classroom.  The  State  of  Illinois  and  NCATE  standards  recommend  more 
field  based  training  such  as  Professional  Development  Schools  (PDS)  for  preservice  teachers.  Although  Illinois 
State  University  continues  to  increase  the  number  of  PDS  sites,  increased  field  placement  is  difficult  given  the 
numbers  of  students  enrolled  in  elementary  education  (more  than  2,000).  This  research  examines  how 
telementoring  can  provide  introductory  contact  with  classroom  teachers  as  well  as  improve  student 
telecommunication  skills. 

Campbell  and  Yong  (1996)  had  preservice  teachers  at  two  universities  participate  in  a collaborative  Internet 
project  to  examine  classroom  issues  from  multiple  perspectives.  Additional  objectives  were  for  the  preservice 
teachers  to  acquire  skills  with  networking  technology  and  to  develop  a positive  attitude  toward  CMC.  Data 
collected  from  assignments  submitted  through  e-mail,  e-mail  messages,  e-mail  process  journals,  and  interviews. 
Participants’  use  of  e-mail  and  trouble  shooting  of  technical  problems  improved  during  the  semester.  They  also 
found  that  technology  was  more  effectively  learned  when  it  was  embedded  in  content-base  projects. 

Norton  and  Sprague  (1997)  studied  on-line  collaborative  lesson- writing  between  pairs  of  inservice  teachers 
and  preservice  teachers.  They  found  that  preservice  teacher’s  attitudes  about  telecommunications  improved  both 
for  personal  use  and  use  with  students.  They  suggest  that  on-line  collaboration  between  preservice  and  inservice 
teachers  may  be  used  to  enhance  preservice  field  supervision. 

This  research  examined  how  telementoring  of  preservice  teachers  (students)  by  practicing  teachers  can  be  an 
effective  extension  of  classroom  experience  and  identifies  factors  which  promote  successful  mentoring.  A pilot 
study  was  conducted,  fall  1997  (Brehm,  1998).  During  that  semester  and  the  two  following  semesters,  data  were 
collected  using  questionnaires,  journals,  interviews,  e-mail  exchanges,  and  Web  discussion  forum  interactions. 

All  students  and  mentor  teachers  had  access  to  Internet  e-mail  and  the  Web  from  their  school  computer, 
home  computer,  or  university  computer  labs.  Students  received  two  hours  of  Internet  training.  All  teachers  were 
experienced  Internet  users. 

A discussion  of  factors  impacting  successful  telementoring  follows  a brief  summary  of  each  semester’s 
research.  Resulting  suggestions  are  divided  into  mentor  recruitment,  type  and  structure  of  the  interaction,  and 
the  process  of  the  interaction. 

The  first  two  semesters,  undergraduate  students  were  enrolled  in  language  arts  methods  classes  taught  by 
the  author.  The  third  semester  students  were  enrolled  in  an  introductory  methods  class.  Curriculum, 
Management,  and  Assessment.  During  Fall  1997  (Sem  I),  mentor  teachers  were  enrolled  in  a Language  Arts  & 
Technology  class  delivered  on-site  in  a school  district.  The  groups  discussed  language  arts  related  topics  on 
netWorkPlace,  a Web  discussion  space.  During  Spring  1998  (Sem  II)  and  Fall  1998  (Sem  III),  individual  e-mail 
between  mentors  and  students  was  added  to  project.  Mentors  were  assigned  to  individual  students  to  personalize 
the  interaction  and  create  community  (Brehm,  1999).  Figure  1 illustrates  the  project  participants  by  semester. 


Fall  1997  (Sem  I) 

Spring  1998  (Sem  II) 

Fall  1998  (Sem  III) 

Students 

24  language  arts  class 

23  language  arts  class 

65  intro  education  class 

Teachers 

34  single  district  class 

13  many  districts 

37  many  districts 

Figure  1 Telementoring  Project  participation  by  semester. 


Based  on  suggestions  from  the  Fall  semester  1997,  modifications  were  made  in  the  recruitment  of  mentors, 
building  community,  and  the  types  of  mentoring  activities  planned  for  Spring  semester  1998.  Community 
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between  preservice  teachers  and  mentors  was  built  through  pairing  of  a teacher  with  two  students,  an 
introductory  e-mail  exchange,  the  development  of  a collaborative  project,  and  a campus  visit  by  the  mentors. 

After  the  initial  introductory  e-mail  exchange,  pairs  identified  a topic  for  a thematic  unit  that  the  classroom 
teacher  was  currently  teaching  or  developing  for  the  future.  The  preservice  teachers  developed  a lesson  plan  for 
the  unit  and  the  teachers  gave  feedback  on  the  lesson  plan. 

Students  reported  pleasure  in  creating  a lesson  plan  for  a real  classroom.  Students  had  some  difficulty  creating 
lessons  that  were  appropriate  for  the  developmental  level  of  the  students  and  lasted  the  length  of  time  that  they 
were  designed  to  cover.  An  unanticipated  finding  was  that  all  of  the  teachers  appreciated  the  new  ideas  in  the 
preservice  teacher’s  lesson  plans  and  taught  part  or  all  of  the  lesson  plan  in  their  classrooms  or  planned  to  do  so 
during  the  1998-99  school  year. 

A iTustration  of  the  researcher  was  that  some  teachers  were  reluctant  to  send  negative  feedback  to  the 
students  resulting  in  a few  lesson  plans  that  lacked  quality  even  after  revision.  As  a result,  the  role  of  the 
researcher  for  fall  1998  was  changed  from  a teacher  responsible  for  grading  to  a moderator  responsible  for 
student  and  mentor  support.  Responsibilities  of  the  moderator  included  solving  technical  problems,  posting 
bulletins  and  information,  encouraging  participation,  and  solving  communication  problems  between  mentors 
and  students. 

During  Sem  III  students  discussed  course  content  (curriculum,  classroom  management,  and  assessment) 
with  mentors  through  e-mail  and  on  a NetForum.  At  the  end  of  Sem  III,  65  students  and  25  mentors  completed 
open-ended  questionnaires  about  the  class.  When  students  were  asked  what  they  learned  from  the  mentoring 
project,  they  reported  the  following:  ideas  for  teaching  from  real  teachers  (28%),  content  (27%),  Technology 
(13%),  and  very  little  or  nothing  (13%).  Teachers  reported  the  benefits  of  participating  as  reflective  thinking 
(45%),  helping  future  teachers  (25%),  and  fresh  ideas  (15%). 

Students  reported  that  mentors  rated  as  outstanding  or  good  (66%)  responded  quickly,  answered  questions 
in  detail  were  knowledgeable,  and  enthusiastic.  Mentors  rated  as  fair  or  poor  (19%)  mentors  who  responded 
infrequently.  During  the  three  semesters,  mentors  were  recruited  through  required  participation  in  classes, 
volunteer  participation  in  classes,  volunteers,  and  paid  mentors.  Mentors  who  volunteered  were  the  most 
involved  and  enthusiastic.  Mentors  must  also  have  convenient  access  to  Internet  to  fully  participate. 

This  researcher  suggests  that  on-line  discussion  be  integrated  into  classroom  discussion  for  the  most 
benefit.  The  mentoring  should  continue  for  at  least  two  semesters  for  maximum  effectiveness.  Initial  time  is 
consumed  with  community  building,  overcoming  technical  problems,  and  learning  the  software.  Technical 
problems  were  reported  by  25%  of  the  students  and  20%  of  the  mentors  during  Sem  III. 

Explicit  structure  is  a requirement  for  telecommunications  projects.  Across  all  semesters,  the  most  frequent 
suggestions  from  both  groups  for  future  mentoring  projects  was  a rubric  for  evaluating  the  participation  and  a 
clear  timeline  of  when  electronic  exchanges  were  due. 

The  researcher  suggests  that  WebCT  or  similar  groupware  be  used  for  mentoring.  Such  software  allows  e- 
mail,  discussion  participation,  and  chat  with  one  sign-on.  It  also  allows  student  work  or  presentations  to  be 
posted  for  student  and  mentor  review.  Pictures  of  mentors  and  students  can  be  posted,  Web  pages  can  be 
downloaded  or  linked,  and  FAQs  can  be  posted  to  save  the  moderator  and  participant’s  time. 
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Introduction 

This  paper  presents  the  research  project  called  A PONTE.  The  overall  objective  of  the  project  is  to  design  the 
appropriate  setting  and  perform  a range  of  “pedagogical  experiments”  integrating  the  use  of  Information  and 
Communication  Technologies  (ICT)  in  rural  secondary  education  centers.  The  main  goal  is  to  observe  the 
inference,  identify  solutions  and  establish  clear  guidelines  for  similar  cases  in  other  regions  with  a considerable 
rural  population,  not  only  in  Europe,  but  also  in  farther  regions. 

The  research  within  A PONTE  is  carried  out  in  the  areas  of  Galicia  (Northwest  Spain)  and  Northern  Portugal. 
Although  part  of  two  different  member  states,  both  regions  show  a similar  model,  human  townships  as  well  as 
correspondences  in  terms  of  socio-cultural  and  geographic  features. 


Setting  the  context 

Two  fields  of  study  have  been  considered  to  constitute  the  research  framework  of  the  project:  the  use  of  ICT  in 
education  and  the  rural  areas'  idiosyncrasy.  A third  underlying  field  of  interest  is  the  issue  concerning  the  gap 
between  secondary  education  and  university  studies. 

It  is  of  common  sense  that  the  introduction  of  new  technologies  into  educational  contexts  requires  a transition 
from  traditional  learning  methods  to  innovative  ones.  It  is  also  widely  accepted  that  distance  education  is  one  of 
the  educational  contexts  which  takes  most  advantage  of  the  use  of  ICT. 

Besides,  rural  areas,  defined  as  such  as  they  have  to  cope  with  specific  problems  as  large  distances  and  lack  of 
appropriated  physical  means  of  communication,  are  conditioned  by  a hostile  geographic  environment  making 
access  to  information  resources  difficult.  For  example,  the  information  available  for  pupils  when  deciding  on 
the  type  of  studies  they  want  to  take  up  seems  to  be  insufficient  in  many  cases.  Consequently  the  passage  from 
secondary  education  to  university  is  much  more  complex  than  the  transition  between  the  different  levels  of 
education.  Both  studied  regions,  Galicia  and  Northern  Portugal,  are  aware  that  the  use  of  remote  applications 
using  ICT  is  the  best  way  to  overcome  the  large  existing  differences  between  rural  and  urban  population,  or  at 
least  to  reduce  them. 

Concerning  the  pedagogical  framework  underlying  the  setting  and  performance  of  the  experiments,  the 
following  demonstration  actions  have  been  identified: 

1.  Many  studies  have  highlighted  that  there  is  still  widespread  lack  of  awareness  about  the  implications  of 
using  new  ICT  in  education  and  training,  and  skepticism  about  the  benefits.  There  is  a need  for  broader 
dissemination  activities  to  encourage  the  education  communities  to  become  involved. 

2.  In  Galicia,  surveys  have  shown  that  there  is  an  important  gap  between  secondary  studies  and  university 
studies.  The  problem  is  that  the  only  information  given  to  the  students  finishing  secondary  schools  on  the 
possibilities  to  continue  their  studies  is  presented  during  a single  day  session  and  is  far  from  being 
complete.  A growing  amount  of  web-based  data  readily  available  is  provided  by  the  Universities  and 
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constitutes  a valuable  source  of  information  that  can  be  presented  adequately  to  the  students. 

3.  The  consequence  of  low  density  and  dispersion  of  population  in  rural  areas  is  that  classrooms  are  made  up 
of  small  numbers  of  pupils.  Due  to  this  fact  and  to  the  diversity  of  the  curricula  in  secondary  education, 
many  subjects  are  chosen  only  by  a few  students  in  each  school.  This  means  that  maintaining  teachers  in 
each  center  for  all  possible  subjects  is  not  affordable  and  seldom-demanded  subjects  are  merely  not  on  offer 
in  many  institutions.  ICT  applications  can  help  to  overcome  this  situation,  as  a wide  range  of  tools  to 
support  distance  education  is  available  on  the  market.  Besides,  participation  in  multicultural  projects 
involving  the  use  of  the  Internet  under  European  ongoing  initiatives  will  be  encouraged  and  supported. 


Project  development 

The  methodology  applied  in  this  project  is  based  on  the  following  approach:  educmiuual  needs  analysis, 
demonstration  and  evaluation  plan  design,  experiment  setting  implementation,  results  analysis  and  best- 
practice  reports  dissemination. 

The  report  on  educational  needs  is  centered  on  the  description  and  analysis  of  aspects  susceptible  to  influence 
the  pedagogical  experiment's  design.  These  considerations  allow  establishing  the  selection  of  schools  in  which 
the  experiments  will  take  place  and  describe  the  actual  state  of  rural  secondary  education  in  Galicia  and 
Portugal  with  special  focus  on  issues  related  to  the  use  of  ICT.  The  crucial  issue  within  A PONTE  will  be  to 
define  the  evaluation  parameters  in  order  to  carry  out  the  experiments  and  assess  the  results.  The  objective  is  to 
identify  and  analyze  the  project  evolution,  from  complete  successes  to  deep  failures.  Evaluation  actions  and 
reports  will  focus  on  issues  as  the  response  to  the  regions  special  needs,  the  pedagogical  value  of  the 
experiments,  the  technical  value  of  the  ICT  applications  implied,  the  socio-economic  impact  and  the 
consequences  for  educational  institutions.  The  intermediate  analyses  and  the  final  results  will  be  presented 
through  public  workshops  involving  teachers,  students  and  parents.  The  interactive  character  of  these 
workshops  should  allow  the  transfer  of  knowledge  between  the  different  actors,  the  confrontation  between 
designers  and  end-users,  and  will  give  to  the  organizers  the  opportunity  to  collect  feedback  data. 

As  research  on  the  impact  of  the  use  of  distance  learning  facilities  in  rural  areas  for  pupils  to  access  less  taught 
subjects  will  be  one  aspect  of  the  demonstration,  a web-based  environment  (called  Aula  da  Ponte)  will  be 
integrated  according  to  the  specific  needs  of  rural  schools.  This  tool  will  provide  an  environment  enabling 
instructional  designers,  monitors  and  students  located  in  different  rural  areas  to  communicate  and  study  in  an 
interactive  way.  The  idea  is  to  integrate  already  developed  components,  as  a preliminary  study  demonstrated 
that  a valuable  quantity  of  existing  tools  is  available  on  the  market. 


Expected  outcomes 

At  local  level,  the  expected  outcomes  of  the  pedagogical  experiments  should  be  an  increase  in  the  awareness  of 
ICT  applied  to  education  in  Galician  and  Portuguese  Secondary  Centers,  as  well  as  the  schools'  access  to 
learning  communities,  international  ICT  services  and  educational  programs.  On  a wider  scale,  experience 
gained  within  A PONTE,  guidelines  and  best-practice  reports  on  how  to  use  ICT  for  pedagogical  purposes  in 
rural  areas  should  be  made  available  to  the  international  educational  community. 
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Hybrid  web-based  courses  combine  technologies  of  distance  delivery  with  face-to-facc  interaction.  This 
combination  of  modes  poses  special  challenges  for  instructors  who  hope  to  foster  collaborative  learning 
environments  based  on  (inter)dependencies  (Bourdeau  & Wasson  1998).  This  discussion  is  based  on  the  delivery  of 
a number  of  online  courses  since  1996,  mainly  in  the  fields  of  Media  Studies  and  Computer-Mediated 
Communication  (English).  (Websites  for  these  courses  can  be  viewed  at  http://mala.bc.ca/~soules/.) 

The  theoretical  contexts  for  this  discussion  have  been  explored  in  previous  papers  (Soules  1997;  Soules  1998) 
which  explore  how  online  learning  takes  place  in  a distinct  performance  medium,  and  how  it  is  important  for 
instructors  to  reflect  on  the  new  expectations,  demands,  and  social  dynamics  resulting  from  the  new  medium. 
Research  on  video-conferenced  courses  offered  in  1996  (Soules  1996)  revealed  that  "student  satisfaction  and 
success  are  highly  dependent  on  [a]  sense  of  membership,  or  engagement  by  the  learner  in  the  educational  process. 
Our  research  found,  for  example,  that  unlike  the  membership  construct,  the  technology  itself  does  not  lead  to  high 
satisfaction  rates:  once  the  purpose  of  establishing  clear  and  reliable  communication  has  been  met,  further  efforts 
to  develop  more  sophisticated  systems  are  not  likely  to  result  in  more  student  satisfaction.  Similarly,  once  basic 
student  support  services  have  been  provided,  more  elaborate  administrative  functions  do  not  significantly  increase 
student  satisfaction  or  success"  (Dolan  1996;  Soules  1998). 

What  we  discovered  in  our  attempts  to  create  collaborative  interdependencies  was  the  need  to  supplement 
classroom  lectures  and  discussions  with  web-based  learning  resources.  Students  at  the  remote  video-conferencing 
site  felt  they  were  missing  something  that  those  in  the  host  classroom  were  experiencing,  and  asked  that  lecture 
notes  and  resources  be  available  to  them  through  other  means.  For  this  video-conferenced  course,  we  evolved  a 
hybrid  mode  which  combined  synchronous  delivery  technologies  with  supplementary,  often  redundant,  web-based 
technologies  designed  to  foster  greater  course  participation  and  involvement  for  an  "ensemble"  of  learners. 

Since  1 996,  I have  delivered  a number  of  courses  which  combine  elements  of  face-to-face  with  online,  web-based 
delivery  techniques.  In  some  cases,  all  course  participants  were  expected  to  attend  classroom  discussions,  and  then 
accomplish  certain  online  learning  tasks  in  lab  sessions.  In  other  course  configurations,  there  have  been  two 
sections  of  the  same  course:  one  section  met  in  the  classroom,  the  other  was  solely  for  online  students  studying 
from  a distance.  All  course  materials  were  provided  on  websites,  and  assignments  for  both  sections  were  submitted 
electronically.  In  still  another  version  of  the  hybrid  online  course,  I accepted  students  who  wanted  to  take  a course 
completely  online  which  had  been  scheduled  as  a traditional  classroom  session.  Finally,  I have  delivered  courses 
with  no  scheduled  classroom  meetings.  In  all  of  these  instructional  scenarios,  the  hybrid  nature  of  delivery  posed 
specific  problems,  and  revealed  important  insights  into  the  dynamics  of  online  learning. 

As  one  might  expect,  online  students  often  express  the  suspicion  that  they  are  missing  something.  Not  only  are 
they  missing  what  they  might  learn  from  any  material  presented  in  class  not  included  in  the  resource  materials, 
they  are  also  missing  the  learning  that  comes  from  discussion  and  interaction.  Additionally,  some  people  do  not 
find  it  easy  to  ask  questions  online.  These  concerns  are  valid  in  my  experience:  students  do  benefit  from  classroom 
discussions,  from  the  clarification  of  difficult  material,  and  from  interaction  with  an  instructor  and  each  other. 
Online  students  do  not  participate  as  easily  in  the  process  that  a group  goes  through  in  the  construction  of  a 
commonly-held  understanding  of  material--not  that  everyone  has  to  have  the  same  understanding,  only  that  the 
group  has  collaborated  on  an  exploration  of  the  material  from  which  they  take  their  own  conclusions.  This 
collaboration  defines,  in  effect,  the  condition  of  interdependency. 
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These  impressions  that  something  is  missing  in  the  educational  experience  is  most  keenly  felt  by  students  who  take 
a course  completely  online  and  know  that  there  is  a group  of  students  who  meet  regularly  in  the  classroom.  If  all 
the  students  are  online,  the  feeling  that  something  is  missing  is  less  urgent. 

Self-motivated  online  students  are  able  to  compensate  for  their  distance  from  the  classroom  by  making  use  of  email 
and  newsgroups  to  communicate  with  both  instructor  and  other  course  participants.  However,  for  students  who  are 
less  sure  of  themselves  or  their  abilities,  I am  discovering  that  there  are  a variety  of  strategies  that  can  be  used  to 
foster  membership  in  the  course  and  its  activities: 

1.  One  obvious  Strategy  for  promoting  a learning  synergy  in  hybrid  courses  is  to  ensure  that  the  online  learning 
resources  available  from  the  website  are  up-to-date  and  as  engaging  as  possible. 

2.  Similarly,  it  is  important  that  instructions  for  assignments  are  clear,  and  do  not  require  further  in-class 
explanations  to  give  fuller  direction.  For  most  of  my  online  assignments,  I specify  the  form  and  goals  of  the 
assignment  in  detail,  and  allow  course  participants  considerable  latitude  in  their  choice  of  approach  to  the  content. 

3.  1 grade  the  assignments  of  online  students  first,  and  respond  to  all  email  queries  as  quickly  as  possible  to 
demonstrate  that  someone  is  responsive  to  their  needs. 

4.  After  the  earlier  video-conferencing  experience,  I have  used  mainly  asynchronous  technologies  for  online 
courses-email  and  distribution  lists,  newsgroups,  and  websites.  My  use  of  newsgroups  has  evolved  considerably, 
and  become  more  structured.  Where  previously  I asked  students  to  participate  in  a newsgroup  discussion  over  a 
period  of  time  and  graded  them  on  their  level  of  participation,  now  I assign  short  weekly  assignments  which  are 
posted  to  the  course  newsgroup.  These  short  assignments  reflect  on  the  course  material,  provide  writing  practice, 
and  generate  focused  discussion.  These  weekly  assignments  allow  me  to  monitor  who  is  actively  participating  in 
the  course.  Collectively,  all  course  participants  are  building  a tangible  dialogue  related  to  the  course  content. 

5.  Since  most  of  the  courses  I deliver  emphasize  writing  skills,  I ask  participants  to  collaborate  on  the  production  of 
an  electronic  journal  which  provides  tangible  evidence  that  all  course  participants  can  contribute  equally  to  the 
learning  experience.  It  is  not  altogether  surprising  that  those  who  volunteer  for  the  production  and  design  of  these 
journals  are  often  online  students  who  want  to  participate  more  fully. 

Strategies  for  creating  membership  in  hybrid  online  courses  should  acknowledge  that  we  are  operating  in  a unique 
performance  medium  with  its  own  protocols  for  effective  interaction.  The  use  of  structured  newsgroup  discussions 
related  to  course  material,  and  the  production  of  electronic  journals  of  participants’  writing  offer  tangible  evidence 
of  interaction  and  collaboration.  Participants  are  able  to  build  dynamic  documents  testifying  to  their  participation, 
and  thus  their  rightful  membership  in  the  course. 
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It  is  one  of  the  concept’s  inherent  objectives  to  be  applicable  to  many  of  those  subjects  concerned  with  electrical 
engineering’s  curricula.  To  meet  these  special  requirement,  a structure  was  chosen,  that  consists  of  eight  different 
modules  and  six  tools. 

The  modules  could  be  derived  from  investigations  of  the  general  demands  of  professional  life  nowadays 
and  an  analysis  of  the  curriculum’s  single  subjects’  contents.  They  have  two  things  in  common,  first  of  all,  each 
of  them  is  in  a way  related  to  a special  subject’s  contents  and  second,  they  try  to  combine  computer  based 
learning  with  traditional  methods  in  learning,  e.g.  solving  exercises  by  writing  on  a piece  of  paper.  But  the 
modules  difFer  with  regard  to  the  learning  objectives,  didactic  approach,  and  the  use  of  media  elements. 


name 

description 

learning  objective 

course 

ofFers  the  contents  of  special  subjects  as  it  is  done  in 
specialist  books,  here  however  in  multimedia  form 

gaining  knowledge  by 

instruction 

exercise  book 

offers  a number  of  exercises,  derived  from  a typology  of 
exercises,  especially  elaborated  for  engineering  subjects 

using  knowledge  to  solve 
problems 

media  corner 

presents  strategies,  hints,  and  tools  for  autonomous 
information  acquisition 

knowledge  acquisition  and 
knowledge  processing 

base  knowledge 

gives  a brief  summary  of  other  subjects’  contents  necessary 
to  comprehend  the  special  subject 

gaining  basic  knowledge 

real  life 

points  out  the  actual  use  of  special  subjects’  contents  in 
reality  and  professional  life 

making  knowledge  transfer- 
able 

laboratory 

ofFers  simulation  environments  for  special  subjects’  contents 

gaining  knowledge  by 

autonomous  exploration 

formulary 

present  a combination  of  single  subjects’  formulas 

gaining  basic  knowledge 

cup  of  coffee 

offers  opportunities  for  recovering 

learning  by  entertainment 

Table  1:  Modules  recommended  by  the  new  concept.  The  table  gives  the  name,  a short  description  and  the 

learning  objective  for  each  of  the  eight  modules. 
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In  dependence  of  the  typical  demands  of  a single  subject,  the  developer  can  decide  for  his  current  distance 
education  course,  whether  to  stress  one  or  a few  of  the  eight  modules,  perhaps  leaving  out  one  and  so  on.  In 
addition  to  that,  it  is  still  left  in  the  hands  of  the  special  subject’s  expert  to  determine  the  composition  of  a certain 
animation,  figure,  video,  etc..  The  concept’s  intention  is  to  provide  a didactic  framework.  This  framework  is 
enriched  by  a detailed  description  of  functions  for  each  module.  Thus,  it  serves  as  a guideline  for  the  developer  to 
facilitate  an  implementation  of  the  modules  in  the  concept’s  sense. 

Each  module  should  be  divided  into  smaller  information  units,  each  of  which  can  be  visualized  on  a 
screen  of  its  own(using  scrolling  if  the  information  unit’s  size  exceeds  screen  possibilities  for  visualization).  For 
accessing  the  modules’  information  units,  the  concept  recommends  a linear  navigation  structure  for  each  module, 
enlarged  by  a hypertext/media  structure  connecting  all  modules.  This  allows  guided  as  well  as  self-determined 


exploration  of  the  single  infcmiuiiL/ii 


In  addition  to  the  modules,  the  new  concept  recommends  six  tools  for  coordinating  and  supporting  the 
student’s  learning  sessions  to  become  more  efficient  (“to  do  things  in  the  right  way”)  and  effective  (“to  do  the 


name  of  the  tool 

description 

learning  optimizer 

offers  adaptability  to  different  strategies  in  learning, 
which  have  been  detected  by  empirical  research 

flipchart 

gives  an  overview  of  the  user’s  navigation  schemes, 
comments,  times  for  learning,  test  ratings,  etc. 

calendar 

offers  to  plan  a single  learning  session  and  to 
elaborate  learning  schedules 

plant 

motivates  the  student  using  the  Tamagotchi-effect^*^ 

search  engine 

offers  quick  access  to  special  information  units  or 
parts  of  them 

calculator 

offers  a scientific,  interactive  calculator 

Table  2:  Tools  for  supporting  students’  learning  sessions.  The  table  gives  the  name  and  a short  description  for 

each  of  the  six  tools. 

As  the  tools  search  engine  and  calculator  are  standard  in  highly  sophisticated  computer  based  learning,  the  other 
four  tools  are  new.  They  could  be  derived  from  reflecting  the  concept’s  authors’  experiences  during  their  studies 
of  electrical  engineering.  They  are  all  intended  to  take  advantage  of  the  medium  computer,  that  is  the  general 
possibility  of  recording  learner’s  activities  data. 

Following  the  new  concept  when  developing  a special  distance  education  course,  has  three  great 
advantages: 

• With  its  modular  structure  the  human  learning  process  (adopting  knowledge  up  to  using  it)  is  not  only 
supported  on  the  whole,  but  due  to  the  clear  separation,  it  is  in  the  student’s  hands  to  choose  his  own 
aspiration  level  at  any  time  of  his  learning  session. 

• With  the  integration  of  traditional  methods  in  learning  and  computer  based  learning  in  a didactic  concept  an 
pedagogical  surplus  value  could  be  achieved. 

• Assimilating  the  tools  learning  optimizer^  flipchart,  calendar,  plant  further  contributes  to  the  student’s 
ability  to  learn. 

Up  to  now,  the  concept  has  exemplarily  been  realized  in  form  of  an  HTML-based  distance  education 
course  for  the  contents  of  mechanical  engineering  basics  in  automation  technology. 


In  1997  a popular  and  controversial  discussed  electronic  toy  for  children.  Children  have  to  keep  alive  a figure  in  a mini 
computer  by  regular  care. 


1343 


The  Effectiveness  of  Metasearch  Tools  in 
Web-Based  Information  Retrieval 


Judi  Repman 
Randal  D.  Carlson 

Department  of  Leadership,  Technology  and  Human  Development 
College  of  Education 
Georgia  Southern  University 
Statesboro,  GA  30460-8131 
j repman@gasou.edu 
rcarlson@gasou.edu 


Basch  (1998)  refers  to  metasearch  tools  as  “umbrella”  sites,  where  the  user  enters  a query  once. 
The  meta-engine  then  translates  it  and  sends  the  query  to  several  search  engines  and/or  directories 
simultaneously.  Notess  (1998)  calls  these  tools  megasearch  engines  or  parallel  or  multiple  search  engines. 
In  looking  at  the  features  of  tools  that  have  been  labeled  metasearch  engines,  one  characteristic 
differentiates  “true”  metasearch  tools:  following  a simultaneous  search  of  multiple  search  engines  or 
directories,  the  end  user  is  given  a single  integrated  set  of  results.  In  the  technical  terms  used  by  Dreilinger 
and  Howe  (1997)  a metasearch  tool  must  contain  three  components:  a dispatch  mechanism,  interface 
agents,  and  a display  mechanism.  Choice  of  search  engines  for  a specific  query  is  the  function  of  the 
dispatch  component,  while  interface  agents  “translate”  the  user’s  search  query  into  the  format  required  by  a 
specific  search  engine  or  directory.  When  the  results  have  been  obtained  from  the  different  search  tools,  the 
display  mechanism  merges  them.  Sometimes  the  results  list  can  be  sorted  or  ordered  as  the  user  specifies. 
Most  metasearch  engines  also  claim  to  have  display  mechanisms  that  eliminate  or  reduce  duplicate  links 
and  which  may  check  whether  or  not  links  are  active. 

True  metasearch  tools  also  fall  into  several  categories.  The  largest  group  is  web-based  and  free  to 
anyone  using  a web  browser.  Next  is  a group  of  desktop  applications.  Some  can  be  downloaded  from  the 
web  for  free  and  are  used  as  an  add-on  to  a single  search  engine.  InfoSeek  Express  < 
http://express.infoseek.com/>  is  one  example  in  this  group,  as  is  Sherlock,  which  is  integrated  into  the  new 
Mac  0/S  8.5  and  is  the  only  desktop  tool  that  can  be  used  on  a Mac.  The  final  category  also  consists  of 
desktop  applications,  but  these  metasearch  tools  are  shareware  or  licensed  software.  Some  sort  of 
registration  fee  or  purchase  price  must  be  paid  either  at  the  time  of  download  or  following  a trial  period. 
Sherman  (1998)  labels  these  as  search  software  utilities  or  agents  since  they  reside  on  the  user’s  computer. 

Unfortunately,  there  are  some  fairly  significant  limitations  to  metasearch  tools.  Basch  (1998) 
likens  the  approach  of  many  of  these  tools  to  the  lowest-common-denominator  effect.  Interface  agents  may 
not  be  sophisticated  enough  to  translate  a Boolean  query  into  the  syntax  required  by  a specific  search 
engine.  Advanced  searching  features  may  only  be  available  as  part  of  a registration/customization  process. 
This  approach  is  only  practical  in  single  user  computer  settings,  a luxury  few  public,  school,  or  university 
libraries  would  find  affordable.  The  speed  of  metasearch  tools  also  means  that  they  retrieve  only  the  few 
highest  ranked  hits  from  individual  search  tools.  Most  metasearch  tools  will  time-out  search  engines  or 
directories  that  take  too  long  to  respond  to  queries.  (Notess,  1999)  The  cost  of  this  emphasis  on  speed 
means  that  most  metasearch  tools  sacrifice  comprehensiveness,  returning  from  20-50  results  per  query. 

The  comparison  and  evaluation  process  began  by  identifying  metasearch  tools.  Lists  of  metasearch 
tools  were  identified  at  the  Open  Directory  Project  <http://dmoz.org/>,  Yahoo!  < http://www.yahoo.com/>, 
and  Beaucoup!  <http://beaucoup.com/>.  An  initial  examination  of  the  tools  indicated  that  while  many  were 
called  metasearch  engines,  the  majority  did  not  fit  our  definition.  After  eliminating  some  tools  that  didn’t 
seem  to  be  functional,  we  were  left  with  a set  of  16  “true”  metasearch  tools.  We  examined  each  of  these  in 
depth,  reading  any  background  information,  FAQ’s  or  descriptions  of  features  that  were  available  and 
tested  each  with  several  sample  queries. 

Developing  criteria  for  identifying  top  performing  metasearch  engines  was  an  interesting  task. 
Since  successful  searching  is  closely  related  to  relevance  this  becomes  a qualitative  assessment.  None  of 
the  metasearch  tools  examined  provided  anything  beyond  a general  statement  of  how  relevance  was 
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determined,  since  this  information  may  be  viewed  as  proprietary.  A metasearch  tool  met  our  initial  criteria 
for  success  if  a result  matching  our  information  need  was  listed  on  the  first  page  of  the  results  display.  The 
number  of  search  engines/directories  available,  the  look  and  functionality  of  the  interface,  and  the  results 
display  were  also  important  factors  in  arriving  at  any  final  judgments.  Some  results  displays  were  so 
cluttered  with  advertising  that  the  user  has  to  scroll  down  almost  a full  screen  before  any  results  were 
visible.  Metasearch  tools  that  made  it  easy  for  users  to  customize  their  search  by  selecting  which  search 
engines  to  use  or  specifying  how  results  should  be  presented  were  also  viewed  more  favorably.  Another 
convenient  addition  was  a window  that  followed  you  on  your  desktop  so  that  you  could  search 
immediately.  Finally,  metasearch  engines  using  frames  when  results  are  linked  were  identified.  When  this 
happens  end  users  don’t  see  the  URL  for  the  site  they  have  linked  to  in  the  location  bar,  which  means  that 
writing  down  or  bookmarking  a URL  is  problematic.  This  feature  can  be  toggled  off,  but  that  is  an 
unnecessary,  and  confusing  step. 

On  the  whole,  most  of  the  16  metasearch  tools  examined  performed  adequately.  None  failed  to 
answer  any  of  our  queries,  but  in  some  cases  an  acceptable  hit  was  well  down  on  the  results  list.  Duplicate 
links  were  usually  eliminated.  In  the  end,  we  agreed  that  five  metasearch  tools  (byteSearch 
<www.bytesearch.com>.  Mamma  <www.mamma.com>,  Metacrawler  <www.go2net.com/search.html>, 
Profusion  <www.profusion.com>,  and  Savvy  Search  <www.savvysearch.com>)  were  superior  performers 
but  did  not  attempt  to  rank  them.  Metasearch  tools  offer  an  excellent  alternative  for  end  user  searching.  All 
five  of  our  top  group  did  an  above  average  job  of  translating  a search  into  the  syntax  required  by  different 
search  engines  and  directories.  They  provide  a quick  way  for  the  end  user  to  have  some  sense  of  the 
quantity  of  web  resources  available  on  a topic.  If  users  pay  attention  to  the  attribution  of  results  to  search 
engines,  it  is  also  possible  to  determine  which  individual  search  engine  might  be  the  most  promising  search 
for  more  advanced  or  in-depth  searching.  The  breadth  of  SawySearch  and  the  options  available  with 
Profusion  make  them  stand  out  from  other  metasearch  tools  currently  available,  but  as  with  most 
discussions  of  web  resources  these  features  can  and  will  change.  Try  to  forget  some  of  the  negative 
comments  you’ve  heard  about  metasearch  tools  in  the  past  (that  is,  two  months  ago).  Their  capabilities 
continue  to  improve  and  our  experience  has  shown  us  that  they  have  an  important  place  in  the  information 
searching  process. 
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Introduction 

ISLA  (Information  System  for  Los  Angeles)  is  a digital  research  resource  of  Los  Angeles 
materials  held  in  a very  large,  University-based  archive.  These  resources  are  in  multiple  information 
formats  and  are  used  for  teaching,  research,  and  public  access.  The  scope  of  resources  includes  the  widest 
variety  of  information  from  all  historical  periods,  linked  by  spatial  and  temporal  coordinates.  The 
primary,  long-term  goal  is  to  create  a system  that  will  enable  all  kinds  of  users,  including  students,  to 
search  and  access,  via  the  WWW,  a rich  and  diverse  range  of  research  materials  about  a specific 
geographic  location.  The  ISLA  project  has  been  supported  by  many  organizations,  including  the  U.  S. 
National  Endowment  for  the  Humanities  (NEH),  the  City  of  Los  Angeles,  and  the  Thomas  Brothers  Map 
Company.  It  is  envisioned  that  the  software  system  will  be  transportable  and  other  cities/locations  can 
acquire  the  software  to  provide  access  to  their  own  digital  resources 

The  first  version  of  the  system  was  released  in  September  1998.  This  is  Version  1 .0,  the  proof-of- 
concept.  This  version  contains  a library  of  multidisciplinary  digital  materials  (texts,  photographs)  and 
provides  components  of  the  sophisticated  search  and  retrieval  method. 

This  paper  provides:  1)  an  overview  of  the  current  system;  and  2)  a description  of  ISLA  training 
and  development  of  K-12  curriculum  integration  plans 


Overview  of  Current  System 

The  ISLA  interface  provides  space/time/fulltext/format  indexing  to  allow  for  searching  and 
retrieval  of  materials.  This  interface  allows  access  by  the  conventional  indices  of  author,  title,  subject,  and 
other  cataloging  fields,  but  in  addition  the  attributes  of  Geographic  Information  System  (GIS)  vector 
layers,  and  timeframes.  A user,  a student  for  example,  can  take  a "core  sample"  of  a geographic  area,  such 
as  a neighborhood,  street,  or  census  area  and  examine  this  through  layers  of  time. 

The  current  materials/resources  in  ISLA  include  photographs,  newspaper  texts,  aerial  and 
satellite  images,  digital  spatial  data  made  available  from  Thomas  Bros.  Map  Company  and  the  Bureau  of 
Engineering  of  the  City  of  Los  Angeles,  data  from  the  1939  WPA  (U.S.  Work  Project  Administration), 
and  land  use  maps.  Additional  items  are  being  added  to  the  growing  ISLA  digital  archive.  Other  material 
formats,  such  as  audio-visual  media  (audio,  video,  film)  and  demographic  data,  will  be  added  in  the 
future. 


Education  Applications/Curriculum  Integration 

A view  of  this  project  is  that  one  of  the  keys  to  its  success  is  the  integration  of  ISLA  into  the 
classroom.  We  believe  that  ISLA  is  particularly  suited  to  assist  in  the  fostering  of  participation  and  critical 
thinking  skills,  and  in  the  development  of  information  literacy  skills.  At  present,  graduate  students  in 
education  (teaching  and  learning)  and  in  instructional  technology  are  being  trained,  using  the  training-of- 
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trainers  model,  in  the  use  of  I SLA.  They,  in  turn,  are  providing  training  sessions  for  school  administrators 
and  teachers,  and  to  a group  of  local  teachers  involved  in  an  in-service  program  in  computer  competency 
and  curriculum  integration 

Individuals  and  groups  are  developing  humanities  and  social  science  focused  lesson  plans  for 
their  specific  K-12  curriculum  interest  level.  ISLA-based  curriculum  planning  can  also  include  integration 
of  other  content  areas,  such  as  mathematics,  inclusion  of  other  supporting  instructional  resources,  and 
connection  to  standard  curriculum  guides,  such  as  the  California  History-Social  Science  Framework.  Most 
planning  includes  a description  of  the  learning  objectives,  required  tasks,  and  specific  learning  activities 
associated  with  I SLA,  as  well  as  a description  of  methods  for  learner  evaluation.  The  following  are  two 
possible  e.xamples  of  ISLA  curriculum  scenarios. 


Geography  (Reading  Maps);  Social  Science  (Demographics);  and  History 

The  general  learning  objectives  associated  with  this  curriculum  scenario  include  the  following. 
Learners  will  understand  and  work  with  cardinal  orientation,  scale,  and  proximity;  interpret  map  symbols; 
work  with  concepts  of  social  categorization  and  elementary  comparative  statistics;  understand  the 
settlement  patterns  of  different  ethnic  groups  during  different  historical  periods;  and  understand  the 
modes  of  livelihood  practiced  by  ethnic  groups  in  various  historical  periods. 

The  tasks  required  of  the  students  include:  orientation  of  unmarked  aerial  photos,  correlation  of 
topography  with  cartography,  comparisons  in  space,  time,  and  by  category;  searching  selected 
neighborhoods  for  evidence  of  ethnic  settlement  and  ways  of  life. 

The  activities  include  the  following.  Students  choose  or  are  assigned  different  regions  of  the  city 
and  each  is  asked  to  construct  a changing  demographic  profile  of  that  neighborhood  at  three  different 
points  in  time:  1939,  1960,  and  1990,  on  three  variables:  occupation,  race-ethnicity,  and  average  income. 
After  viewing  and  extracting  data  from  ISLA,  students  perform  simple  summary  proportion  statistics  and 
charts  using  typical  spreadsheet  programs.  Students  share  results  and  then  write  reports  about  the  nature 
of  the  changes  within  regions  over  time,  and  across  regions,  offering  hypothetical  explanations  for  those 
differences. 


Art  and  Architecture 

The  general  learning  objective  includes  developing  an  understanding  of  historical  architectural 
styles  and  their  creator’s  intended  meanings. 

Tasks  include  identifying  a style  in  different  types  of  construction  and  then  ’’reading" 
unidentified  constructions  to  determine  date  of  construction. 

Activities  include  the  following.  Students  search  ISLA  to  discover  buildings  built  during  the 
1920s,  using  dates  in  photographs  and  using  "year  built"  data  in  the  1939  WPA  Household  Survey.  They 
search  for  photographs  and  identify  Art  Deco  styles  in  both  downtown  office  buildings  and  residential 
structures.  Students  form  teams  and  each  team  assembles  a collection  of  residential  and  commercial 
buildings  from  the  period  1910  through  1940.  Each  team  attempts  to  identify  the  buildings  constructed 
based  on  the  styles  of  the  sample  chosen  by  the  other  team.  As  appropriate  and  if  possible,  teams  will  visit 
these  buildings. 


Conclusion 

The  ISLA  archive  continues  to  grow  and  the  interface  is  still  undergoing  development.  But  in  the 
meantime  teams  of  education  students  and  local  teachers  will  continue  to  evaluate  ISLA  in  terms  of  its 
curriculum  integration  capabilities  and  will  develop  additional  ISLA-based  lessons  for  eventual  classroom 
use. 
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Now  available  on  the  World  Wide  Web  (WWW),  information  on  any  subject,  by  anyone!  There 
exists,  for  the  first  time  in  history,  a channel  that  spans  the  globe  where  freedom  of  expression  is  open  to 
anyone.  It  is  this  openness  that  signals  a note  of  caution.  Little  of  the  information  on  the  Internet  is  filtered  as 
it  is  in  the  more  conventional  channels  such  as  in  libraries,  newspapers,  magazines,  books,  or  television.  What 
is  accurate?  Evaluating  information  available  on  the  Internet  is  cmcial  if  the  information  is  used  as  teaching 
material,  learning  resources,  or  research  reports. 

Critical  literacy  is  a prerequisite  skill  that  must  be  fostered  when  using  information  from  the  Internet. 
This  invaluable  skill  will  carry  over  into  a real  world  that  contains  ever  more  sources  and  quantities  of 
information.  On  the  Internet,  where  anyone  may  publish  anything  relatively  easy,  the  huge  quantity  of  extant 
information  makes  the  use  of  search  engines  commonplace.  However,  search  engines  cannot  rank  the  returned 
documents  for  credibility.  This  is  left  to  the  student,  teacher,  or  researcher,  who  is  faced,  with  a potential 
mountain  of  chaff  mixed  in  the  kemals  of  knowledge.  As  such,  the  source  is  not  always  easy  to  evaluate, 
especially  according  to  reputation,  because  of  the  vast  number  of,  often  anonymous,  authors  involved.  In  the 
end,  it  is  usually  left  to  the  document  to  speak  for  itself  in  terms  of  its  content  validity,  and  ultimate  credibility 
lies  in  the  critical  literacy  of  the  reader.  (Farah,  1995). 

An  Example:  The  staff  at  Bishops  High  School,  Newfoundland,  Canada  began  to  question  the  validity 
of  information  included  in  recent  student  term  papers.  Students  were  accepting  almost  all  home  pages  as 
accurate  regardless  of  content  because  they  were  "in  print."  Print  materials  commonly  used  for  student  research 
have  usually  passed  through  either  peer  research  or  editorial  boards  before  publication.  However,  no  such 
validation  exists  for  Internet  materials,  which  may  be  posted  by  both  genuine  authorities  and  those  with 
questionable  credentials.  In  response  to  problem,  the  Memorial  University  Faculty  of  Education  Action 
Research  began  a project  to  develop  a set  of  guidelines  for  students  and  teachers  who  want  to  use  the  Internet  as 
a source  of  information  for  research  papers.  (Doug  F.  etal.  1996),  As  a basis  for  their  guidelines  they  define 
validity  as:  The  quality,  fact  or  condition  of  being  supported  by  commonly  accepted  facts  or  recognized 
authorities.  Additionally  they  suggest  using  two  types  of  validity  with  respect  to  home  pages.  The  first, 
"concrete  validity"  is  based  upon  a page  being  well  referenced  to  known  authorities.  The  second,  "context 
validity"  is  based  upon  the  content  of  a page  being  confirmable  with  accepted  facts.  In  his  presentation  at  the 
Association  for  Educational  Communications  and  Technology,  Dr.  Don  Descy  of  Makato  State  University  in 
Minnesota  defined  information  accuracy  in  using  a two  part  definition  containing  these  same  basic  points. 
(Descy,  1997). 

Using  above  definitions  as  a basis,  the  following  is  a list  of  pertinent  questions  developed  as  a 
guideline  for  determining  the  accuracy  of  information  gleaned  fi*om  the  Internet. 

With  respect  to  "concrete  validity"  the  reader  should  be  able  determine  the  answer  to  the  majority  of  the 
following  questions. 

• What  is  the  source  of  the  information? 

• Does  an  individual,  group,  or  organization  sponsor  this  site? 

• Can  the  location  of  the  site  be  determined  by  the  URL?  (*See  information  on  Uniform  Resource 

Locator) 

• Is  the  author/publisher  a recognizable  authority? 

• Is  the  author  the  creator  of  the  information  or  a compiler  of  other  information  resources? 

• Does  the  author/publisher  provide  his/her  e-mail  address? 

• Is  there  a common  link  to  the  page  Iforn  recognized  authority? 

• Does  the  page  provide  other  sources  that  could  be  contacted  for  confirmation  or  further  information? 

• Does  the  page  cite  a bibliography  or  provide  references  to  confirm  the  accuracy  of  the  information? 
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• Has  the  site  been  reviewed  by  a "content"  reviewing  agency? 

Some  pages/sites  contain  information  that  appears  valid  but  does  not  meet  the  majority  of  the  criteria  for 
"concrete  validity."  To  deal  with  these  pages  or  further  confirm  the  accuracy  of  the  given  information,  which 
appears  valid  in  comparison  to  other  pages,  "valid  in  context,”  the  reader  should  be  able  to  answer  all  of  the 
following  questions, 

• Who  is  the  intended  audience  for  this  site. 

• What  is  the  intent  of  this  information? 

• Why  was  it  put  there  - entertainment,  educational? 

• Is  the  information  presented  in  a well-reasoned  manner? 

• Can  you  detect  any  evidence  of  bias  from  reading  the  page? 

• Is  supportive  informatioR  available  in  print? 

• Does  the  information  contradict  or  confirm  the  information  from  other  sources? 

• Does  the  site  have  links  to  other  sources  on  the  same  topic? 

Developing  skills  in  determining  the  accuracy  of  information  gleaned  from  the  WWW  has  far  reaching 
potential  for  lifelong  learning.  The  Internet  provides  a virtual  classroom  for  hands  on  learning  of  critical 
thinking  skills  needed  in  the  real  world  to  avoid  the  pitfalls  of  false  advertising,  pseudo-science,  narrowed 
reality  (gangs,  cults),  get-rich-quick  schemes  (scams,  pyramid  schemes,  mail  frauds),  political  rhetoric, 
indoctrination,  media  bias,  twisted  statistics,  and  other  ills  that  prey  on  the  unsuspecting  in  this  information 
society  we  live  in. 

*URL  information  example:  http://  www.mannlib. Cornell. edu.reference/workshops/surf/url.html 
The  Protocol:  This  tells  the  Web  browser  how  to  load  and  interpret  the  document.  For  example,  http://  tells  the 
browser  that  the  file  is  a hypertext  document.  Other  protocols  include  gopher,  ftp,  and  news. 

The  Server:  The  second  part,  www.mannlib.comell.edu  is  the  name  of  the  machine  on  which  the  Web  page  is 
located.  The  following  domain  names  provide  information  about  the  type  of  organization  using  or  maintaining 
the  server. 

".com"  names  are  used  by  commercial  (for  profit)  organization, 

".org"  names  are  usually  used  by  non-profit  organizations  or  groups. 

".edu"  is  for  4-year,  degree  granting  colleges/universities. 

".net"  is  for  network  infrastructure  machines  and  organizations. 

".gov"  is  for  US  Federal  Government  agencies. 

The  Path:  The  last  portion  of  the  URL,  reference/workshops/surf/url.html,  gives  the  path  or  directory  of  the 
document  plus  the  name  of  the  file  itself  When  the  symbol  "~"  is  included  in  the  path  it  often  denotes  a private 
directory  or  file.  To  determine  more  information  about  the  files  located  on  a particular  server,  in  the  location 
window,  try  deleting  the  file  and  folders  - one  at  a time  from  right  to  left.  (First  delete  the  file  and  press  enter  to 
logon  to  the  site,  then  delete  each  of  the  folders  in  turn  from  right  to  left,  trying  to  logon  after  each  deletion. 
Many  times  this  will  open  up  other  directories  and  files  also  housed  on  the  site)  Suggested  sites  for  additional 
information  Truth,  Lies  and  the  Internet  http://www.cnet.com/ContentyFeatures/Dlife/Truth/ssQl.html;  Kathy 
Schrock's  Guide  for  Educators  http://www.capecod.netyschrockguide/eval.htm  The  Six  Quests--  WWW  repair  - 
Evaluative  approaches  http://www.ciolek.com/PAPERS/OUEST/Quest6.html  Information  Quality  WWW 
Virtual  Library  http://www.ciolek.com/WWWVL-InfoQualitv.html. 
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Introduction 

Over  the  past  ten  years  we  have  been  involved  in  the  design  and  implementation  of  many 
Educational  Multimedia  applications.  We  have  found  that  the  selection  of  media  has  often  been  a difficult 
decision  as  we  try  to  balance  the  "best’  media  to  meet  the  instructional  objective  with  practical 
considerations  such  as  cost  and  availability  of  appropriate  designers. 

Since  1993,  we  have  applied  several  theories  used  in  Drama  to  the  design  of  six  Educational 
Multimedia  applications  as  an  additional  design  tool  to  aid  us  in  the  selection  of  media  in  the  interface. 
These  theories  have  proven  useful  as  design  aids  as  well  as  a documentation  tool  for  the  design  process. 

Background 

In  her  book  "Computers  as  Theatre",  Laurel  (1991)  argues  that  dramatic  theory,  which  is 
centuries  old,  can  be  applied  directly  to  the  contemporary  field  of  Human-Computer  Interaction.  She 
suggests  that  using  dramatic  theory  provides  "a  new  place  to  stand"  for  interface  designers.  She  explains 
that  Aristotle’s  theory  of  drama  provides  six  points  which  are  to  be  considered  when  creating  a play  or 
analyzing  a play  which  has  already  been  completed.  A successful  play  should  contain  all  six  points.  The 
six  elements  are:  action,  character,  thought,  language,  melody  or  pattern,  and  spectacle  or  enactment. 

The  ideas  put  forth  by  Laurel  have  received  considerable  attention  (see  for  example,  Frenkel 
1994)  but  we  have  struggled  with  applying  these  elements  to  the  interface  design  of  our  Educational 
Multimedia  applications  because  Laurel  does  not  provide  practical  examples.  Within  the  overall  spirit  of 
Laurel’s  argument,  we  have  been  using  three  dramatic  theories  which  provide  practical  techniques  that  we 
can  use  in  the  design  of  our  interfaces.  In  the  following  section  we  will  describe  these  dramatic  theories. 

Dramatic  Theories 

A)  Laban’s  Mastery  of  Movement  System 

Laban  (1975)  collected  information  about  human  movement  and  in  fact  the  terminology  used  in 
the  field  evolved  on  the  basis  of  his  work.  His  book  "The  Mastery  of  Movement  on  the  Stage"  referred  to 
movement  on  the  stage  in  the  theatre  but  Laban  also  used  his  ideas  of  movement  in  relation  to  the  stage  of 
human  life.  According  to  Laban,  humans  move  in  order  to  satisfy  a need.  For  example.  Eve  had  to  move  to 
pick  the  apple  from  the  tree  of  knowledge.  An  actress  representing  Eve  must  attempt  to  pick  the  apple  from 
the  tree  in  an  appropriate  manner  so  that  an  audience  member  who  is  not  familiar  with  the  biblical  stoiy 
would  be  made  aware  of  both  her  aims,  the  tangible  and  the  intangible.  The  movement  is  all  that  the 
audience  member  has  to  interpret  this  moment.  It  should  be  noted  that,  according  to  Laban's  definition, 
movement  includes  sound. 

Laban  determined  that  there  are  four  motion  factors:  time,  weight,  space,  and  flow.  Each  motion 
factor  may  have  one  of  two  measurable  and  classifiable  effort  element  states:  fighting  and  yielding. 
Categorizing  movement  in  this  manner  allows  either  for  the  analysis  of  any  given  movement  or  for 
guidance  in  creating  effective  movement.  Thus,  for  example,  a director  should  be  able  to  consider  any  actor 
in  any  given  scene  and  categorize  the  actor's  movements  in  one  of  the  potential  states.  Also,  any  given  actor 
should  be  able  to  use  these  motion  factors  as  a guideline  in  rehearsal. 


B)  Stanislavskies  Source  of  Physical  Actions  and  Intentions 


Stanislavski’s  (1976)  book  ’’Creating  a Role"  changed  the  way  many  actors  and  acting  schools 
approached  acting.  Stanislavski’s  main  thesis  was  that  the  action  performed  by  the  actor  is  the  most 
important  part  of  acting  and  that  feeling  is  not  important: 

” Let  each  actor  give  an  honest  reply  to  the  question  of  what  physical  action  he  would 
undertake,  how  he  would  act  (Not  feel,  there  should  for  heaven’s  sake  be  no  question  of 
feeling  at  this  point)  in  the  circumstances  given  by  the  playwright....  When  these  physical 
actions  have  been  clearly  defined,  all  that  remains  for  the  actor  to  do  is  to  execute  them... 

Note  that  I say  show  physical  actions,  not  feel  them,  because  if  they  are  properly  carried 
out  the  feeiiiigs  will  be  gciieratcd  Spontaneously.  If  you  work  the  other  way  and  begin  by 
thinking  about  your  feelings  and  try  to  squeeze  them  out  of  yourself,  the  result  will  be 
distortion  and  force....  ’’ 

Stanislavski  makes  many  points  such  as  for  every  action  there  should  be  a beginning,  a middle  and 
an  end  and  each  of  these  structural  elements  should  be  played  clearly  and  precisely.  He  also  says  that  the 
context  (or  moment)  of  the  movements  are  very  important.  He  explores  the  purpose  of  the  moment  as 
follows: 


"To  have  any  meaning,  whatever  happens  on  stage  must  have  a purpose.  It  must  serve 
some  end  beyond  some  accomplishment  of  the  action  itself  The  meaning,  both  for  the 
actor  and  the  audience,  rarely  lies  in  the  action  but  in  the  purpose  for  which  it  is  done. 
Even  loading  a gun  or  mixing  poison  is  not  itself  dramatic.  We  must  know  why,  toward 
whom  the  lethal  effort  is  directed  ( McGaw  1975,  p.  23)." 


C)  Perceptual  Mode  Dominance  Theory 

The  psychotherapists  Bandler  and  Grinder  (1975-1976)  have  a very  simple  yet  powerful  theory 
called  Perceptual  Mode  Dominance.  The  theory  says  that  of  the  five  senses  - hearing,  seeing,  touching 
(body  sensations  in  general  which  are  called  kin  aesthetic),  tasting  and  smelling  - the  first  three  are  the  most 
important.  Interpersonal  communications  are  almost  totally  governed  by  these  three  modes.  Human  beings 
tend  to  favor  one  of  the  three  modes.  They  tend  to  perceive  life  dominantly  either  through  what  they  see, 
what  they  hear,  or  what  they  touch  (or  kinaesthetically  feel).  They  learn  or  understand  best  if  the 
information  is  communicated  either  visually,  auditorially,  or  through  physical-spatial  means. 


Conclusion 

The  three  dramatic  theories  introduced  in  this  paper  do  appear  to  be  a useful  tools  for  aiding  the 
design  and  development  of  Educational  Multimedia  applications.  Our  current  interest  is  to  (a)  continue 
performing  experiments  to  empirically  evaluate  the  usefulness  of  the  theories  and  (b)  get  actual 
Instructional  Designers  to  use  these  theories  as  a tool  in  the  Instructional  Design  process  so  that  we  can  get 
their  feedback.  Our  long-term  interest  is  in  the  possibility  of  incorporating  these  tools  into  an  on-line 
Instructional  Design  toolkit  that  we  have  developed. 
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Generally,  most  definitions  of  the  term  distance  learning  include  the  concept  of  time-  and  space-independent 
teaching  and  learning.  Lately,  definitions  also  include  the  notion  of  employing  information  and 
communications  technology,  interactive  video,  and  computer  networks  to  enable  asynchronous  and 
synchronous  learner-instructor,  learner-learner,  and  learner-advisor  interaction.  Simply  stated,  distance  learning 
provides  learners  with  educational  opportunities  offered  outside  of  a formal  classroom  setting,  combining 
independent  study  with  guided  instruction  that  is  coordinated  by  a central  facility  providing  a network  of 
educational  services.  Applying  this  definition,  instructors,  support  personnel  such  as  academic  advisors,  and 
technology  combine  to  provide  an  instructional  delivery  system  that  fulfills  high  academic  standards  while 
being  flexible  and  considerate  of  distance  learners’  constraints. 

Learners  who  choose  distance  education  often  face  a number  of  obstacles: 

• They  live  in  remote  geographic  areas. 

• They  have  limited  program  options  at  conveniently  located  institutions. 

• They  have  work  schedules  that  conflict  with  campus-bound  course  schedules.  This  includes  people  who 
work  shifts,  travel  frequently  on  business,  work  long  hours,  and/or  are  in  the  armed  forces. 

• They  have  personal  and  family  commitments  that  conflict  with  campus-bound  course  schedules.  This 
includes  having  children  at  home  and  taking  care  of  aging  parents. 

The  obstacles  to  fulfilling  their  educational  goals  and  objectives  give  rise  to  special  needs  which  must  be  met  if 
distance  learners,  and  the  programs  that  serve  them,  are  to  be  successful.  The  special  needs  that  impact  distance 
learners’  success  fall  into  two  main  categories:  instructional/resource  access  and  communication  needs,  and 
advising  and  learner  support  service  needs. 


Needs  of  the  Distance  Learner 

The  needs  of  the  distance  learner  provide  a starting  place  for  determine  what  a distance  learner  support  services 
program  (DLSSP)  must  effectively  respond  to  in  order  to  ensure  that  learner  satisfaction  and  success  in  meeting 
learning  plans  are  maximized.  The  following  list  includes  some  of  the  more  obvious  needs  of  distance  learners 
that  should  contribute  to  the  shape  of  a distance  instructional  as  well  as  learner  support  services  program: 

• To  learn  and  accomplish  learning  plans.  This  is  the  primary  need  for  the  learner.  All  instructional  and 
service  program  components  should  ultimately  support  this  endeavor. 

• To  have  access  to  what  the  learner  chooses  to  learn  at  the  time  and  in  the  delivery  mode  preferred  by  the 
learner. 

• To  receive  accurate  and  timely  information  inclusive  of  the  following: 

• Program  offering  requirements  (i.e.,  degrees,  certificates,  and  credentials). 

• Educational  provider,  policies  and  procedures  (e.g.,  registration,  payment  of  account, 
drop/add/ withdrawal). 

• Course  offerings  and  content  descriptions  as  well  as  prerequisite/corequisite  information. 

• Self-assessment  tools  available  to  assist  the  learner  in  evaluating  eligibility  and  preparedness  for 
courses  and  programs  of  interest. 
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• How  courses/credits/competencies  apply  to  programs  or  transfer. 

• Learning  outcomes  expected  for  courses  and  programs  and  their  relationship  to  personal/professional 
goals. 

• Technical  support  assistance  with  required  software  and  hardware  for  selected  instructional  delivery 
method. 

• Learner  support  service  availability. 

• To  ensure  individual  attention  from  the  educational  provider.  The  learner  wants  to  have  one  point  of 
contact  to  the  organization  and  have  the  DLSSP  network  internally  to  meet  learner  needs.  The  learner 
wants  the  educational  provider  to  respond  to  learner  needs  rather  than  the  learner  adjust  to  the 
organization's  structure. 

• To  have  a relationship  of  support  with  educational  provider  staff.  The  distance  ieciiiiei  wants 
acknowledgment,  positive  feedback  and  encouragement  (as  appropriate),  an  early  warning  system  for 
unnoted  obstacles  or  potential  problems,  assistance  in  clarifying  and  validating  learning  plans,  and  a 
responsive  problem-solver/trouble-shooter  to  assist  when  any  academic  or  administrative  issue  arises. 

• To  be  referred  appropriately  when  the  point  of  contact  is  not  able  to  address  the  learners  concern. 

• To  have  a sense  of  connection  and  community  with  the  educational  provider  and  other  learners  with  similar 
learning  plans. 

• To  have  special  learner  needs  (e.g.,  accommodation  of  disabilities)  addressed  effectively  both  in  the 
instructional  and  service  arenas. 


Roles  and  Responsibilities  of  the  Distance  Learner  Support  Services  Provider 

The  general  roles  and  responsibilities  of  the  Distance  Learner  Support  Services  Provider  (DLSSP)  that  respond 
to  the  distance  learner  needs  are  summarized  into  three  primary  categories. 

• Communication  --  There  must  be  a variety  of  means  for  the  learner  to  connect  with  the  educational 
provider  to  gain  learner  support  services.  Because  the  DLSSP  is  working  with  distance  students,  personnel 
must  not  only  have  excellent  interpersonal  communication  and  instructional  skills  to  be  able  to  concisely 
and  clearly  explain  policies,  procedures,  and  requirements,  but  be  able  to  effectively  utilize  those  skills  in  a 
telephone-  or  computer-mediated  communication  environment. 

• Information  — Information  should  be  made  available  to  learners  in  hard  copy  and  Internet  form.  In  order  to 
accomplish  this,  a DLSSP  need  to  be  able  to  plan  for  learners'  informational  needs  and  organize  materials 
and  resources  to  meet  those  needs,  develop  and  write  supporting  materials,  and  present  information 
verbally  and  in  written  format  in  a clear  and  concise  manner. 

• Learner  Advocacy  and  Liaison  --  Addressing  all  learner  issues  with  a customer  service  orientation,  the 
DLSSP  serves  as  the  interface  with  the  learner  and  the  educational  provider.  The  most  critical  aspect  of 
being  an  advocate  and  liaison  for  distance  learners  is  advising  and  counseling,  which  includes: 

• Understand  learners’  unique  characteristics  and  ability  to  adjust  advising  and  learner  support  style 
appropriately 

• Develop  learners’  personal  learning  plans 

• Clarify  learners’  career  and  learning  goals 

• Select  of  appropriate  courses  and  other  educational  experiences 

• Increase  learner  awareness  of  resources  available 

• Evaluate  learner  progress  toward  established  goals  and  provide  appropriate  response 

Web  and  Computer-mediated  Communication  Support  and  Structure  for  Distance 
Learner  Support  Activities 

Examples  of  online  learner  support  services  for  distance  learners  that  utilize  the  Web  and  other  computer- 
mediated  communications  technologies  can  be  found  at  http://ceo.cudenver.eduMoni  Diinlap/lssexamples.html 
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Electronic  communication  is  a new  and  expanding  medium  for  the  professional  development  of  teachers. 
This  paper  reports  on  some  work  conducted  to  begin  to  develop  a framework  for  discussing  the  forms  of 
participation  that  take  place  in  the  context  of  professional  development  on-line.  (Cole,  1998) 

As  the  use  of  email  or  other  forms  of  electronic  communication  grows  as  a method  of  professional 
development  research  has  been  conducted  in  a variety  of  areas.  Existing  research  on  teacher- to-teacher 
electronic  communication  has  focused  on  diverse  topics  including  the  reasons  teachers  used  the  network  at 
particular  times  (West  et  al.,  1989);  the  professional  ties  fostered  by  the  network  (Broholm  and  Aust,  1994; 
Stahlhut  & Hawkes,  1994);  the  number,  content,  and  writing  style  of  messages  posted  by  individuals  (Weir, 
1992;  West  et  al.,  1989);  and  the  impact  of  participation  on  the  professional  lives  of  both  active  and  passive 
participants  (DiMauro  and  Jacobs,  1995;  Jacobs  and  DiMauro,  1995).  No  matter  what  the  focus  of  the  research 
all  work  points  out  that  individual  teachers  participated  in  different  ways.  However,  these  differing  forms  of 
participation  are  rarely  described  systematically.  The  goal  of  the  work  reported  here  was  to  begin  to  develop  a 
means  of  carefully  describing  on-line  participation  in  a professional  development  setting. 

The  work  focused  on  the  Elementary  School  Mathematics  Project  (ESMP),  part  of  MATHLINE,  a project 
of  the  Public  Broadcasting  System  (PBS).  ESMP  has  two  components:  a set  of  lessons  presented  on  paper  and 
video,  and  a system  to  link  the  participants  electronically  in  a small  group,  a learning  community,  and 
throughout  the  country.  This  work  was  concerned  only  with  the  learning  community  on-line  component. 

To  investigate  forms  of  participation  four  representative  learning  communities  were  selected.  All  of  the 
messages  posted  on  each  of  the  learning  communities'  electronic  bulletin  boards  were  collected  for  each  of  four 
weeks  spread  through  the  1 996-97  school  year. 

Close  examination  of  the  messages  reveled  three  distinct  types  of  messages,  statements,  inquires,  and 
replies.  A statement  was  defined  as  a message  that  makes  a statement  or  tells  a story.  In  a statement  the  author 
may  present  an  opinion  or  may  simply  state  facts.  A message  was  classified  as  an  inquiry  if  the  purpose  of  the 
message  was  to  seek  information  or  to  elicit  a response.  Replies  were  defined  as  messages  in  which  it  was 
clear  that  the  message  was  prompted  by  another  message.  These  message  types  were  used  as  the  basis  for  the 
development  of  categories  of  participation. 

All  of  the  messages  collected  were  coded  as  one  of  the  three  message  types  or  as  a combination  of  two 
types.  There  were  a total  of  271  messages  collected  of  which  24%  were  classified  only  as  statements,  13%  only 
as  inquiries,  42%  only  as  replies,  6%  as  both  statements  and  inquiries,  10%  as  both  statements  and  replies,  and 
5%  as  inquiries  and  replies.  Once  the  messages  were  classified,  they  were  sorted  by  participant  in  order  to 
investigate  the  possibility  of  trends  in  participation.  Some  possible  trends  appeared  and  were  used  to  identify 
seemingly  different  forms  of  participation  or  participation  profiles.  Previous  work  in  the  field  (DiMauro  and 
Jacobs,  1995;  Jacobs  and  DiMauro,  1995;  West  et  al.,  1989)  predicted  two  broad  categories,  active 
participation  and  passive  participation.  Jacobs  and  DiMauro  (1995)  had  named  one  of  the  types  of  passive 
participation  "active  readers."  These  were  participants  who  read  almost  all  of  the  on-line  communication  but 
rarely,  if  ever,  posted  messages.  These  ideas  of  active  and  passive  participation  and  active  readers  influenced 
the  formation  of  the  profiles  in  this  work. 

Once  messages  were  sorted  by  participant,  careful  examination  of  each  participant's  messages  took  place 
in  order  to  identify  similarities  and  differences  among  participants.  This  examination  suggested  the  existence 
of  five  distinct  forms  of  participation,  three  passive  and  two  active. 

The  first,  and  most  easily  identifiable  form  of  participation  was  non-participant.  These  were  participants 
who,  while  enrolled  in  the  project,  did  not  participate  on-line.  The  second  form  of  passive  participation  was  the 
limited  participants,  these  teachers  read  and  posted  messages  infrequently.  The  exact  definition  of  infrequent  is 
difficult  because  in  this  data  set  there  was  no  clear  gap  in  terms  of  percentage  of  messages  read  or  posted.  The 
final  form  of  passive  participation  was  the  form  described  in  previous  literature,  the  active  reader.  These 


participants  read  all  or  most  of  the  messages  posted  to  the  network,  but  never  posted  their  own  messages  or  did 
so  very  infrequently. 

There  were  two  forms  of  active  participation  identified,  informers  and  active  participants.  The  informers 
were  perhaps  the  most  unexpected  profile.  These  participants  read  and  posted  actively,  but  all  of  the  messages 
they  posted  were  either  statements  or  replies.  This  group  never  posed  inquiries  always  dispensing  rather  than 
seeking  information.  This  was  surprising  since  previous  literature  suggested  that  teachers  participate  in  on-line 
communities  in  order  to  get  answers  to  their  questions  (Broholm  and  Aust,  1994;  West  et  al.,  1989),  clearly 
these  teachers  participated  for  other  reasons. 

The  final  form  of  active  participation  was  named  the  active  participants.  These  teachers  read  all  or  almost 
all  of  the  messages  posted  and  contributed  regularly  to  the  on-line  conversation  using  all  three  message  types. 
These  were  the  people  who  kept  the  network  going.  In  the  data  examined  all  of  the  people  identified  as  on-line 
facilitators  fell  into  this  category  as  did  others. 

While  it  is  difficult  to  develop  exact  definitions  in  this  preliminary  work  with  a limited  data  set,  some 
reasoned  decisions  were  made  about  cut-off  points  in  order  to  classify  each  participant  into  one  of  the  five 
styles  of  participation.  When  this  was  complete  for  the  particular  data  set  from  MATHLINE,  12%  of  the 
participants  fell  into  the  non-participant  category,  24%  in  the  limited  participant  category,  29%  active  readers, 
1 2%  informers,  and  23%  active  participants. 

While  it  is  clear  that  further  work  is  required  to  verify  and  further  refine  the  profiles  identified,  the 
participants  in  each  profile  were  examined  to  investigate  possible  connections  between  the  profiles  and  other 
factors.  It  was  found  that  the  most  active  participants  were  also  the  most  experienced  teachers  with  the  active 
participants  having  a mean  of  24,6  years  of  teaching  experience  while  the  next  most  experienced  group  were 
the  active  readers  with  a mean  of  17,3  years.  Almost  all  of  the  participants  listed  highlights  of  participating  in 
ESMP,  but  the  highlights  varied  with  the  level  of  participation.  Those  with  limited  on-line  participation  named 
the  lessons  as  highlights  while  the  highlights  for  those  who  were  more  active  were  mixed.  There  was  no 
connection  between  the  level  of  on-line  participation  and  the  grade  taught  or  the  amount  of  professional 
development  in  the  year  prior  to  the  ESMP  project.  Finally,  active  participants  tended  to  have  access  to  the 
system  at  home  (44.4%)  or  both  at  home  and  at  school  (44,4%)  rather  than  only  at  school  (1 1 .2%). 

The  results  of  the  study  provide  a beginning  in  developing  a means  of  describing  what  happens  in  on-line 
professional  development  projects  for  teachers.  The  profiles  should  be  verified  and  then  further  explored. 
While  this  study  is  preliminary,  there  are  some  recommendations  or  considerations  possible  from  the  results. 

First,  organizers  should  expect  differing  forms  of  on-line  participation.  Providing  lessons,  as  MATHLINE 
did,  or  other  materials  may  be  important  in  keeping  those  with  differing  on-line  participation  engaged  in  the 
process.  In  addition,  if  possible,  efforts  should  be  made  to  make  the  on-line  component  accessible  to  teachers 
in  their  homes.  While  this  does  not  guarantee  participation,  it  appears  to  contribute.  Finally,  effort  should  be 
made  to  include  teachers  with  a variety  of  levels  of  experience  and  not  to  exclude  interested,  experienced 
teachers  who  in  this  case  were  the  majority  of  the  active  participants. 
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Introduction 

We  are  living  in  the  emergence  of  the  Information  Age.  Telecommunications  and  networked  systems  are 
becoming  bigger  and  more  functional.  The  Internet  is  considered  the  world’s  largest  computer  network  and 
community  of  computer  users.  The  Web,  as  the  Internet  is  commonly  known,  provides  cheap  and  simple  access  to  a 
fast-growing,  global  information  service,  for  both  providers  and  consumers  of  information  (Ford,  1995). 

Most  of  the  recent  literature  published  about  technology  in  the  classroom  refers  to  the  highest  profile 
element  of  the  Internet,  the  World  Wide  Web.  Faculties  are  involved  in  practices  using  the  Web  as  an  information 
resource  or  as  a way  to  communicate  with  others.  The  use  of  e-mail  and  the  World  Wide  Web  in  education  is  part 
of  the  daily  activity  in  most  of  the  colleges  and  universities.  Educators  have  developed  special  classroom  activities 
and  collaborative  projects  using  electronic  media.  Electronic  information  access  and  retrieval,  e-mail.  Bulletin 
Boards  (BBSs),  and  Computer  Conferencing  are  some  of  the  possibilities  of  Computer  Mediated  Communication 
(CMC)  that  are  effecting  education  daily. 

Certainly,  colleges  and  universities  are  particularly  conducive  to  computer  network  use.  Professors  are 
invited  to  use  or  re-invent  the  use  of  the  medium  as  they  wish.  However  there  has  been  resistance  to  many 
educational  innovations  introduced  in  the  higher  education  over  the  years.  Some  educators  possibly  view  CMC  as 
another  innovation  that  will  come  and  go  (Cuban,  1986).  Nevertheless,  the  diffusion  of  an  innovation  and  its  use  will 
depend  on  its  acceptance  among  innovators  and  early  adopters  (Rogers,  1 995). 

Computer  Mediated  Communication  is  being  implemented  in  the  College  of  Education  and  Health 
Professions  at  the  University  of  Arkansas.  Most  of  the  professors  search  the  web  daily  and  are  using  e-mail  as  a 
way  to  communicate  with  their  students.  However,  little  or  no  research  regarding  the  concerns  of  faculties  related  to 
the  utilization  of  CMC  have  been  done. 


Research  Design 

The  purpose  of  this  study  is  to  determine  the  level  of  concerns  among  professors  who  use  or  do  not  use 
CMC  for  instruction  in  the  College  of  Education  and  Health  Professions  at  the  University  of  Arkansas.  Survey 
methodology  was  selected  as  the  method  of  investigation.  A questionnaire.  Stages  of  Concern  ©(SoCQ),  developed 
by  Hall  G.  E.,  and  Hord,  S.M  in  Texas  Research  and  Development  Center,  was  selected  to  gather  data  on 
demographics  and  levels  of  concerns  among  the  teaching  faculty  at  the  College  of  Education  and  Health  Professions 
at  the  University  of  Arkansas  in  Fayetteville,  AR.  The  College  of  Education  and  Health  Professions  at  the  University 
of  Arkansas  was  founded  in  1934.  It  is  located  in  the  northwest  part  of  Arkansas  in  Fayetteville.  The  target 
population  for  the  study  will  be  the  entire  teaching  faculty  at  the  College  of  Education  and  Health  Professions  at  the 
University  of  Arkansas.  The  research  questions  are: 

1 . What  are  the  stages  of  concern  among  professors  who  are  using  or  not  using  CMC  on  instruction? 

2.  To  what  extent  do  stages  of  concern  among  the  professors  vary  according  to  age,  rank  and  by  departments? 


Instrumentation 


The  instrument  will  be  sent  through  the  campus  mail  to  the  entire  teaching  faculty  during  the  Spring  Semester 
1999.  Descriptive  statistics  will  be  used  for  all  variables.  The  resulting  descriptive  statistics  will  include  frequency 
distributions  and  percentages.  For  the  SoC  questionnaire,  raw  scores  for  each  of  the  seven  sub-scales  will  be  tallied 
and  converted  to  norm  percentiles  for  each  of  those  seven  categories.  Also,  stages  of  concerns  will  be  combined  and 
analyzed  by  departments,  age,  and  academic  rank.  The  chi  square  test  of  association  will  be  used  to  determine  the 
significant  difference  among  independent  groups. 


Results 

This  is  an  ongoing  research  project.  Data  will  be  collected  during  Spring  Semester.  In  June,  all  the  data  will  be 
collected  and  analysis  will  be  completed. 


Implications  and  Conclusion 

As  a result  of  the  information  age  tool  (i.e.,  the  Internet)  in  school,  innovative  educational  activities  are  growing. 
Opportune  questions  regarding  the  cognitive  and  social  benefits  of  new  interactions  must  be  raised.  Also  questions 
about  how  educators  concern  about  technology  in  school.  This  research  suggests  that  change  in  teaching  practice  is 
based  on  the  teachers’  practices  and  believes  (Richardson,  1990).  It  is  important  to  consider  the  control  that  teachers 
have  over  change.  Studies  that  are  more  comprehensive  might  take  into  consideration  the  nature  of  CMC,  the  way  it 
has  been  used,  the  instructional  context,  and  so  on.  Further  research  might  look  for  the  variability  of  these  findings 
across  other  colleges  of  education. 
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Introduction 

Aging  is  a natural  phenomenon  that  refers  to  "changes  that  occur  during  the  lifespan  and  result  in  differences  in 
structure  and  function  between  the  youthful  generation  and  the  elder  generation,  but  these  processes  do  not  occur  at  a given 
age  or  set  time  in  all  individuals"  (Kennedy,  1992  p.  10).  The  ability  to  learn  and  acquire  knowledge  does  not  end  at  the  onset 
of  old  age.  Rather,  the  focus  of  learning  in  adults  changes  from  curiosity  to  a need  to  learn  to  function  in  society,  solve 
problems,  and  to  grow  as  an  individual. 

As  we  approach  the  twenty-first  century,  there  is  a need  for  the  older  adult  population  to  acquire  knowledge  and 
compete  with  the  younger  generations.  With  the  onset  of  new  technologies,  millions  of  older  adults  are  learning  to  use  the 
Internet  (World  Wide  Web).  Older  adults  are  gaining  Internet  skills  that  are  becoming  a common  aspect  of  life,  yet  few  of 
these  older  adults  have  had  formal  instruction  on  how  to  navigate  the  web.  The  majority  of  them  rely  on  self-directed  learning 
and  informal  knowledge  (Gaboon,  1998).  According  to  Knowles  (1980),  self-directed  learning  is  only  one  facet  of  adult 
learning.  Knowles  states  that  adults  are:  (a)  motivated  to  learn,  (b)  pragmatic  learners,  (c)  self-directed  learners,  and  (d)  they 
struggle  to  balance  learning  projects  against  the  constraints  of  time,  space,  economic  resources,  and  personal  relationship. 
In  fact,  self-direction  is  a familiar  characteristic  of  adult  learning  that  is  important  for  the  acquisition  of  basic  Internet  skills 
and  for  successful  participation  in  Internet-based  distance  learning  (Gaboon,  1998). 

Older  adults  are  drawn  to  the  new  communicative  channels  of  e-mail,  on-line  conferencing,  and  Web  publishing 
because  of  its  convenience.  However,  frustration  and  isolation  can  be  found  by  the  lack  of  familiar  social  cues  when  working 
in  on-line  environments.  Older  adults  are  facing  the  need  to  learn  and  at  the  same  time,  they  have  to  deal  with  the  demands 
of  adult  life.  The  Internet  is  becoming  a demanding  instructional  tool  among  older  adults  because  of  the  availability  and 
flexibility  of  Internet  resources.  According  to  Grodsky  and  Gilbert  (1998),  "older  adults  are  not  moving  slowly  and 
cautiously,  but  are  racing  into  the  computer  age"  (p.  70). 

In  the  past,  combining  older  adults  and  computers  seemed  impossible.  Now,  with  seniors  returning  to  the 
workforce,  contacting  family  and  friends  via  e-mail,  exploring  new  worlds  through  the  Internet,  or  just  the  seniors  increased 
ability  to  expand  and  gain  new  knowledge  are  defining  new  boundaries  for  the  older  adult  learning  environment  (Spiezle, 
1999).  According  to  Kennedy  (1992),  "humans  maintain  an  intellectual  curiosity  about  the  natural  world  throughout  the 
lifespan"  (p.  1 1).  Therefore,  it  is  important  to  encourage  older  adults  to  explore  the  avenues  of  new  technology  and  to  have 
access  to  the  world  of  the  Internet. 

Many  older  adults  are  experiencing  the  Internet  through  a national  non-profit  organization  called  SeniorNet. 
SeniorNet  is  dedicated  to  “making  sure  that  seniors  are  not  left  by  the  side  of  the  road  on  the  Information 
Superhighway”(Grodsky  & Gilbert,  1998  p.72).  SeniorNet  has  been  enhancing  older  adults’  knowledge  about  computer 
technology  since  1986.  SeniorNet  not  only  has  a Web  page  dedicated  to  older  adult  learning,  but  also  the  organization  has 


established  128  learning  centers  in  35  states  where  adults  over  50  years  old  can  participate  in  classes  ranging  from 
introduction  to  computers  to  developing  their  own  Web  page.  Amazingly  enough,  the  key  to  the  success  of  the  SeniorNet 
classes  is  that  the  classes  are  peer  taught  (Grodsky  & Gilbert,  1998). 


Methodology 

A survey  was  developed  by  the  researchers  to  collect  information  pertaining  to  how  and  why  adults  50  and  older 
are  learning  on  the  Internet.  A Web  page  was  then  developed  containing  the  survey  (see  Appendix  A).  The  URL  address 
(http://ww"W.infopoll.corri/'infopoll/surv'ey5/^53 253.htm)  of  the  sur/ey  and  a short  description  of  the  purpose  of  the  survey  was 
then  sent  to  Marcie  Schwarz,  SeniorNet  Director  of  Education.  She  added  a link  to  access  the  survey  from  the  SeniorNet 
Web  page  in  March,  1999.  This  is  an  ongoing  research  project.  Data  will  be  collected  once  a month  to  ensure  that  people 
are  still  accessing  the  survey.  The  SeniorNet  population  will  have  access  to  the  survey  for  an  entire  year.  In  March,  2000, 
all  the  data  will  be  collected,  and  analysis  will  be  completed. 


Implications  and  Conclusion 

Knowing  how  and  why  adults  50  years  and  older  are  accessing  the  Internet  to  learn  can  be  beneficial  in  several 
ways.  First  of  all,  knowing  this  information  allows  higher  education  and  senior  organizations  to  participate  in  the  changing 
environment  of  the  Internet  users  and  provide  methods  of  lifelong  learning  opportunities  to  the  aging  population.  This  will 
in  turn  provide  a chance  ”to  deliver  services  that  will  upgrade  the  social,  economic,  educational,  and  personal  conditions  of 
the  older  adult  population"  (Swift,  1986  p.  19). 

Imagining  the  future,  there  will  be  more  access  to  on-line  learning  in  the  form  of  courses,  distance  learning, 
computer-based  learning,  and  Web-based  learning  environments.  Educators  can  take  this  information  gained  from  this  survey 
and  incorporate  it  into  curriculum  for  older  adults  learning  in  both  informal  and  formal  situations.  Since  the  next  century  will 
be  shaped  by  technology,  it  is  important  to  understand  learning  on  the  Internet  of  all  users  (Gaboon,  1998). 

Looking  into  the  future,  there  is  one  thing  for  certain;  the  Internet  will  only  continue  to  evolve,  and  that  technology 
will  continue  to  create  learning  opportunities  for  the  older  adult.  The  number  of  older  adults  using  the  Internet  will  continue 
to  grow  as  access  to  personal  computers  becomes  easier  and  programs  become  more  user  friendly.  Society  should  not 
underestimate  the  abilities  of  the  older  adult  population  and  how  important  it  is  to  the  older  adult  population  to  keep  up  with 
societal  and  technological  changes. 
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Introduction 

Due  to  the  relatively  easy  use  of  the  tools  involved  in  developing  educational  multimedia,  an  individual  could  be 
Subject  Matter  Expert,  graphic  artist,  computer  programmer,  and  Instructional  Designer  on  a project  that  they 
manage  and  distribute  themselves.  This  method  has  been  referred  to  as  the  "Lone  Ranger  and  Tonto"  approach  by 
Tony  Bates  of  the  University  of  British  Columbia  (Bates,  1995).  According  to  Bates  this  system,  with  the  "Lone 
Ranger"  as  faculty  member  and  "Tonto"  as  graduate  student  is  not  sustainable  over  time.  An  uncoupled  content 
and  team  approach  to  educational  multimedia  development  may  lead  to  more  applicable,  more  cost-effective,  and 
more  re-usable  educational  multimedia. 

Team  Approach 

The  team  approach  requires  collaborative  authoring  of  educational  multimedia.  The  methods,  approaches,  and 
tools  in  collaborative  authoring  vary  (Dobson,  1993).  At  the  University  of  Calgary,  collaborative  authoring 
involves  Instructional  Designers,  professors,  multimedia  professionals,  students,  and  other  Subject  Matter  Experts. 
The  initial  collaboration  is  often  between  Subject  Matter  Experts  (SME)  and  Instructional  Designer. 

What  is  the  best  way  that  Subject  Matter  Experts  can  contribute  to  authoring  the  educational  multimedia, 
recognizing  that  in  many  settings  the  best  way  is  a balance  between  efficiency,  cost  effectiveness,  and  simplicity 
and  accurate,  innovative,  and  effective  educational  multimedia?  This  paper  will  suggest  a process  of  collaborative 
authoring  in  educational  multimedia  development.  The  author  is  currently  conducting  research  attempting  to  find 
out  how  best  Subject  Matter  Experts  contribute  in  the  educational  multimedia  development  process,  in  order  to 
understand  how  to  build  sustainable,  flexible,  and  applicable  media. 

Consider  the  following  methods  for  dealing  with  content  in  educational  multimedia: 

1 . Teach  Subject  Matter  Experts  multimedia  and  instructional  design 

2.  Teach  subject  matter  to  Instructional  Designer. 

3.  Team  Subject  Matter  Experts  with  Instructional  Designers 

The  first  two  methods  listed  above  have  had  various  levels  of  success,  but  importantly  they  can  be  considered  less 
sustainable  and  applicable  overall  (Bates,  1995).  As  we  develop  skills,  methods,  and  processes  as  specialists  in  the 
various  disciplines  in  educational  multimedia  development  we  move  towards  a sustainable,  cost  effective,  and 
innovative  production  system.  The  final  approach  from  above  when  combined  with  uncoupled  content  holds 
promise  for  developing  more  high  quality  educational  multimedia  better  and  faster. 

Uncoupling  the  Content 

Recently,  the  University  of  Calgary’s  Advanced  Media  for  Learning,  has  implemented  a design  process  that 
separates  the  content  from  the  engine  that  drives  the  educational  media  (Figure  1).  By  taking  the  content  out  of  the 
programming  environment  we  achieved  shortened  development  cycles;  and  sustainable,  versionable,  reusable,  and 
applicable  educational  multimedia  and  multimedia  components.  Importantly,  the  model  also  gives  the  authorship 
of  multimedia  to  Subject  Matter  Experts  and  lends  to  collaborative  authoring. 


Figure  1,  Uncoupled  Content  Model.  The  model  demonstrates  the  separation  of  content  from  the  engine, 
concurrent  development,  collaborative  content  authorship,  and  integration  of  content  and  media  at  run-time. 

A team  approach  combined  with  uncoupling  the  content  from  the  programming  environment  allows  authors  of 
multimedia  to  be  Subject  Matter  Experts,  the  graphic  artists  to  produce  graphics,  the  computer  programmer  to 
focus  on  the  programming,  etc.  And  if  the  uncoupled  content  is  stored  in  a consistent  manner,  e.g.  a database,  the 
assembly  of  educational  multimedia  is  executed  at  run-time  by  drawing  the  appropriate  information  from  content 
and  media  databases.  With  the  content  separated  from  the  engine,  changing  the  content  does  not  require  changes  to 
the  programming,  only  to  the  content  data  store.  To  change  the  delivery  platform  of  the  content,  one  may  only  have 
to  program  a new  engine;  one  does  not  have  to  re-write  the  content. 

This  approach  basically  eliminates  the  need  for  an  individual  to  integrate  and  assemble  the  production.  The 
uncoupled  content  model,  as  pictured  in  Figure  I,  demonstrates  the  simultaneous  processes  that  can  occur,  namely 
content  production,  engine  production,  and  media  production.  These  simultaneous  events  shorten  development 
cycles  and,  importantly,  lead  to  a more  predictable  development  cycle.  The  process  also  enables  Subject  Matter 
Experts  and  Instructional  Designers  to  collaborate  effectively  to  structure  and  author  content  using  a standard  word 
processor.  Making  it  possible  for  multiple  authors,  using  the  authoring  templates,  to  provide  content.  This  can  help 
increase  the  buy-in  factor  among  potential  users  increasing  applicability  and  validity  of  the  end  product. 

The  real  "authoring  tool"  of  educational  multimedia  then  becomes  a word  processor  not  Director®,  or 
Authorware®.  Subject  Matter  Experts  don’t  have  to  learn  new  tools,  and  software  to  produce  educational 
multimedia  for  their  classroom.  Of  course,  they  can  if  they  want  and  there  is  plenty  of  software  and  support 
available.  But  on  larger  projects  that  could  apply  to  multiple  situations,  a team  approach  that  uncouples  the  content 
from  the  multimedia  engine  is  beneficial. 

Conclusion 

Decreasing  development  times,  increasing  applicability,  reusability  and  versionability  have  been  realized  through 
an  uncoupled  and  team  approach  to  multimedia  development.  This  paper  partially  describes  a process  that  is 
currently  undergoing  further  study.  The  lack  of  metrics  available  to  measure  outputs  will  likely  force  the  research 
to  take  the  form  of  a case  study.  Some  of  the  preliminary  observations  of  the  case  study,  following  the  production  of 
a 4 CD-ROM  set  titled,  “A  Practical  Guide  to  Communication  Skills  in  Clinical  Practice”  will  be  presented  at  ED- 
Media  ’99. 
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Introduction 

With  the  development  of  technology  capable  of  enhancing  communication  through  electronic  media, 
universities  and  schools  are  rapidly  being  encouraged  to  take  advantage  of  these  advances  to  both  enhance 
instruction  and  to  provide  educative  opportunities  to  people  previously  prohibited  from  instruction  because  of 
spatial,  financial,  or  time  constraints  (Gordon,  H.,  1 996).  Consistent  with  these  national  and  international  trends, 
San  Diego  State  University  has  a recent  history  of  encouraging  faculty  to  re-envision  what  post-secondary 
education  looks  like  through  the  use  of  the  multiple  technologies  available  on  campus.  For  the  first  time  last 
year  one  Exercise  and  Nutrition  Sciences  introductory  class  was  offered  completely  on-line.  The  questions  that 
this  raises,  particularly  in  a content  area  based  on  physical  performances  of  various  sorts  and  geared  toward 
human  interaction  (like  most  health  professions),  are  what  are  students  learning  and  how  is  the  learning 
enhanced  or  limited  by  the  electronic  media.  The  purpose,  then  of  this  study  was  to  document  the  interactions 
students  had  with  the  introductory  kinesiology  course  content  generated  by  the  instructor  via  electronic  media. 
A further  purpose  was  to  document  the  interactive  changes  the  instructor  made  in  response  to  student  feedback 
in  an  effort  to  make  the  class  engaging  and  challenging  for  all  learners. 


Methods 

All  of  the  approximate  150  students  enrolled  in  the  introductory  course  were  invited  to  be  participants  in  the 
study.  Qualitative  methods  (Bogdan  & Biklen,  1 992)  were  used  to  document  learning  and  describe  the  learning 
processes  of  students  throughout  the  study.  Data  sources  included  student  work,  teacher  response  to  student 
interaction,  and  unstructured  interviews  of  the  current  course  instructor,  graduate  assistant,  and  students. 

Student  work  was  documented  using  student  assignments,  electronic  journals  in  response  to  queries  by  the 
instructor  about  perceived  problem  issues,  and  grades.  The  timeliness,  completeness,  and  depth  of  student  work 
was  used  to  document  on-going  student  learning.  Teacher  response  to  student  interaction  was  documented 
through  e-mail  interaction,  in-class  interaction,  and  phone  calls.  E-mail  interaction  was  documented  by 
archiving  e-mails  by  the  date  they  arrived.  In-class  interaction  and  phone  calls  were  documented  by  a weekly 
log  of  interactions  kept  by  the  instructor  that  include  the  content  or  technological  issue  at  the  heart  of  the 
interaction.  Teacher  response  to  student  interaction  documented  by  any  change  the  teacher  makes  with  the  class 
in  response  to  students  was  documented  by  additional  information  posted  on  WWW  and  additional  e-mail 
information  or  queries. 

Student  interviews  occurred  based  on  their  technology  and  course  content  interest.  After  the  first  test,  students 
were  grouped  by  their  performance  on  the  content  knowledge.  Students  who  fell  into  the  A,  B,  C,  D,  and  failure 
assessment  regions  were  provided  the  opportunity  to  participate  in  the  study.  One  or  two  students  participating 
in  the  study  and  randomly  selected  from  each  grade  level  served  as  the  interview  respondents  after  the  midterm 
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and  final.  Toward  the  end  of  the  semester,  several  students  representing  the  five  grade  assessments  were 
assembled  for  a focus  group  interview  (Fontana  and  Frey,  1994)  to  determine  critical  issues  important  to 
learning  in  the  class. 


Data  Analysis 


Student  work,  teacher  generated  documents,  transcribed  interviews,  and  teacher  notes  were  analyzed  using 
document  analysis  (Marshall  & Rossman,  1995),  To  organize  analysis  of  these  documents,  the  theoretical 
perspectives  and  methodological  approaches  of  structuralism  were  enlisted.  Structuralism  views  “documents”  as 
“texts”  (Manning  Sc  Cuiium-Swan,  i 994,  p.  467),  As  such,  oiice  the  data  is  accumulated  m wTittcn  form,  it  was 
read  through  several  times.  With  each  reading  a set  of  tentative  themes  was  developed  and  refined.  The 
emerging  themes  and  issues  arising  from  the  readings  created  the  codes  or  interpretive  frames  that  guided  and 
structured  the  analysis.  Trends  and  commonalties  informed  researcher  understandings  of  students’  perceptions 
of  the  internet  based  course. 


Data  was  organized  chronologically  to  observe  for  trends.  Within  the  chronological  patterns  that  emerged,  data 
was  further  organized  into  similar  themes  and  categories,  Rossman  and  Rallis  (1998)  suggest  that  once  data  has 
been  coded  into  categories,  themes,  and  patterns  researchers  have  to  “challenge  the  very  pattern  that  seemed  so 
apparent”  (p.  181).  To  do  this,  once  themes  emerged  within  the  chronology,  negative  cases  or  cases  that  did  not 
fit  the  emerging  trends  were  sought.  Further,  the  trends  across  the  chronology  were  compared  to  document  how 
the  context  impacted  the  data.  For  instance,  if  some  pedagogical  decision  were  followed  by  trends  in  student 
response,  this  chronological  trend  was  tracked  across  the  chronology  of  the  semester  to  see  if  similar  trends 
existed.  The  credibility  of  data  generated  under  qualitative  research  is  determined  by  a network  of 
methodological  and  analytical  issues  that  focused  on  “establishing  the  match  between  the  constructed  realities 
of  stakeholders  and  those  realities  as  represented  by  the  evaluator  and  attributed  to  various  stakeholders”  (Guba 
& Lincoln,  1989,  p.  237).  Three  methodological  issues  that  added  to  the  study  credibility  are  the  length  of 
engagement,  the  persistence  of  observation,  and  triangulation  of  data  from  multiple  sources. 


The  results  of  this  study  add  to  the  collective  knowledge  of  distance  learning  and  to  what  we  know  about 
student  socialization  into  fitness,  nutrition,  and  education  disciplines.  In  addition,  the  efficacy  of  learning  based 
on  teacher/course  expectations  and  actual  learning  were  documented.  It  is  further  intended  to  support  changes  in 
on-line  course  design  and  presentation  by  describing  the  process  students  engage  in  to  learn  the  necessary 
content.  Data  documented  both  the  process  of  learning,  the  complexities  of  technologically  based  resources  and 
how  the  process  of  accessing  information  enhances  or  limits  student  content  learning  and  discovery. 
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The  purpose  of  this  paper  is  to  describe  the  development  of  a college  course  for  rural  adults  in  principles  of 
marketing  that  utilized  both  distance  education  and  class  meetings. 


Faculty  Support 

Many  adults  know  they  need  to  upgrade  their  skills,  but  need  encouragement  and  personal  recognition  to  do 
so.  In  the  past,  regular  class  meetings  have  provided  the  opportunity  for  the  teacher  to  provide  this  support 
and  encouragement.  In  many  rural  settings,  students  and  families  are  new  owners  of  computers  and 
neophytes  at  using  the  Internet.  They  often  need  assistance  with  using  the  Internet  as  a means  of  course 
work  and  resource  while  at  the  same  time  learning  the  subject  matter. 


Utilization  of  Student  Time 

Adult  learners  who  are  employed  full-time  often  cannot  attend  classes  on  a daily  or  even  weekly  basis. 
When  the  instructor  posts  course  work  including  assignments,  readings,  notes,  and  web  sites  on  the 
Internet,  students  can  access  these  materials  at  their  convenience.  By  utilizing  e-mail  for  the  transmission  of 
homework  and  questions  to  the  teacher,  the  teacher  can  respond  in  a timely  manner  without  having  to  wait 
for  the  class  to  meet.  In  addition,  regular  postings  of  homework  assignments  help  to  keep  each  student 
focused  and  moving  forward.  Using  the  Internet  as  a means  of  communication  between  groups  and  partners 
avoids  extra  travel  or  time  off  from  work. 


Utilization  of  Class  Time 

While  some  people  may  be  able  to  work  alone  at  home  at  the  computer,  most  careers  today  involve 
personal  interaction  with  others.  In  addition,  not  all  courses  or  teaching  styles  lend  themselves  to  total  web 
presentation.  Scheduling  class  time  for  teacher  presentations,  group  discussions,  student  presentations,  and 
exams  provides  face-to-face  opportunities  for  learning  and  interaction.  Utilizing  technology  to 
communicate  with  students  saves  valuable  class  time  for  subject  matter  discussions  and  the  presentation  of 
original  materials  not  available  in  the  text.  Students  come  to  class  prepared  having  sent  homework 
assignments  via  e-mail  several  days  prior  to  the  scheduled  class  meeting.  By  utilizing  the  Internet  students 
receive  individual  attention  and  acquire  group  skills. 


Class  Description 


During  the  fall  1998  semester,  a totally  web~based  course  was  changed  to  include  several  class  meetings. 
None  of  the  students  were  proficient  in  the  use  of  the  Internet  and  e-mail,  therefore  time  was  allowed  for 
orientation  to  the  Internet  as  a resource  and  as  a medium  for  learning.  This  is  a very  small  rural  branch  of  a 
small  college  where  the  only  technical  assistance  available  is  from  the  instructor.  Class  time  was  used  for 
exams,  instruction  in  presentation  skills,  and  class  discussions  of  readings  and  assignments.  Individual  and 
group  presentations  of  projects  were  completed  in  person. 


Teaching  Goals  and  Objectives 

This  course  was  developed  for  the  rural  adult  learner,  degree  or  non-degree  seeking  student.  Behavioral 
goals  included  helping  adults  to  achieve  a positive  attitude  toward  life-long  learning  and  technology,  and 
increased  self-identity  through  the  development  of  individual  potential.  Instructional  goals  included 
knowledge  about  the  Internet  and  its  application  to  marketing,  the  preparation  and  presentation  of 
marketing  plans,  and  the  understanding  of  the  specific  objectives  associated  with  marketing  concepts. 


Summary 

By  the  midpoint  of  the  course  students  were  regularly  sending  assignments  to  the  instructor  via  e-mail 
without  difficulty.  Assignments  that  included  searching  the  Internet  were  well  done  and  thoughtfully 
completed.  Utilization  of  class  time  was  very  focused.  Students  expressed  concern  that  the  course  took  a lot 
of  work  and  were  reminded  of  the  time  saved  by  not  driving  to  and  attending  some  classes.  Presentations 
and  projects  were  completed  in  a professional  manner.  Responses  to  the  students  by  the  instructor  needed 
to  be  better  planned.  This  format  was  a good  alternative  to  totally  web-based  courses,  it  allowed  for 
personal  interaction  among  students  and  teacher  while  encouraging  students  to  use  new  technology. 
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Since  Schon's  (1983)  work  on  the  reflective  practitioner,  the  teacher  education  literature  has  featured  numerous 
calls  for  teachers  (both  pre-  and  inservice)  to  reflect  upon  their  teaching  as  a way  to  take  control  of  their 
personal  professional  development  and  their  teaching  success  (Vogt  & Au,  1995),  Educators  believe  reflection 
should  be  encouraged,  however,  there  is  less  clarity  regarding  how  that  process  best  occurs.  This  is  especially 
true  when  it  comes  to  the  use  of  technology  in  encouraging  personal  reflection. 

While  the  use  of  technology  has  been  discussed  as  a supplement  to  instruction  (Poling,  1994),  a unique  form  of 
communication  (Eiser,  1990),  and  a provider  of  new  information  sources  (e,g,,  web  sites),  (Chandler  & 
Maddux,  1998),  these  concepts  are  not  without  their  drawbacks.  Providing  computer  access  for  students  is 
frequently  cited  as  a major  cause  of  frustration  with  students  participating  in  e-mail  oriented  projects  (Tannehill, 
Berkowitz  & LaMaster,  1997),  Helping  students  gain  ease  and  familiarity  with  the  technology  is  another 
challenge  in  implementing  technology-based  education  procedures,  requiring  time  for  students  to  gain 
confidence  in  their  computer  skills  (Delcourt  & Kinzie,  1 993). 

A recent  pilot  study  (Bayles  Martin  & LaMaster,  in  process)  confirmed  the  roadblocks  of  technology  access  and 
familiarity  in  a graduate  education  course.  Based  upon  findings  of  a pilot  study  conducted  in  1997-98,  the 
following  research  question  was  used  to  guide  this  study:  How  can  a selected  group  of  reading  graduate 
students  most  ably  use  technology  to  support  and  enhance  their  reflection  about  teaching?  It  is  the  thesis  of  this 
project  that  students  must  be  given  time  and  support  to  develop  the  facility  to  reflect  on  their  practice  in  various 
contexts  and  that  this  support  can  and  should  extend  to  use  of  technology  in  the  reflection  process. 


Methodology 

This  study  involves  a semester  of  graduate  student  preparation  in  technology.  During  the  previous  study,  it  was 
determined  that  students’  overt  expression  of  reflection  rarely  surfaced  without  specific  and  explicit  interaction 
by  the  course  instructors.  In  other  words,  when  asked  simply  to  “reflect”  about  their  teaching  on  email,  students 
rarely  did  so.  It  was  difficult  to  determine  whether  students  were  less  reflective  than  expected,  or  whether 
technology  challenges  artificially  truncated  student  responses. 

To  address  issues  of  technology  access  and  familiarity,  graduate  students  in  an  education  reading  assessment 
course  were  introduced  to  email  and  listserv  communication  through  weekly  case  study  analysis.  Students’ 
ability  to  express  their  personal  reflections  and  consider  reflexive  issues  (beyond  their  classrooms)  were  also 
supported  through  the  use  of  personal  dialogue  journals  and  weekly  professional  article  response  papers.  Both 
of  these  activities  involved  more  traditional  communication  means  (paper  and  pencil;  word  processing)  so  that 
students’  learning  about  reflection  was  not  confounded  with  their  familiarity  with  electronic  communication. 


The  subjects  for  this  study  were  22  graduate  students  enrolled  in  a Reading/Language  Arts  Master's  program 
and  participating  in  a clinical  reading  experience.  During  clinics  graduate  students  worked  in  small  groups  and 
individually  with  learners  experiencing  difficulty  with  reading.  All  subjects  had  campus  access  to  e-mail 
accounts  and  computers  through  the  university  or  their  professional  sites.  Participants  were  assigned  to  the  same 
listserv.  The  primary  researchers  and  a graduate  assistant  were  also  included  on  the  lists  to  monitor  and  provide 
guided  reflection.  At  least  once  a week,  participants  posted  to  their  class  lists  reflections  to  the  case  studies 
posted  by  the  instructor.  Peers  read  and  responded  to  each  others  postings.  All  participants  were  urged  to 
maintain  communication  with  their  peers  through  the  e-mail  class  lists. 


Analysis 

Two  goals  directed  the  data  analysis:  1)  to  identify  the  kinds  of  reflection  graduate  students  engage  in 

throughout  the  study;  and  2)  to  determine  whether  graduate  student  reflections  change  throughout  the  semester;. 
A grounded  theory  approach  was  used  for  data  analysis  of  electronic  forum  entries  and  other  related  artifacts 
(Strauss  & Corbin,  1990).  Forum  entries  were  read  and  reread  by  researchers  and  categorized  by  themes. 
Because  this  study  will  span  16  weeks,  opportunities  for  triangulation  (Lincoln  & Cuba,  1985)  will  occur 
naturally  and  vary  over  time.  For  example,  as  students  articulate  concerns  and  reflections  in  email  addresses, 
their  actual  clinical  teaching  can  be  observed  for  congruence  with  their  comments.  On-line  member-checking 
will  occur  whenever  instructors  provide  electronic  (or  verbal)  feedback  to  students. 

Implications  from  this  study  are  best  articulated  as  potential  benefits  to  the  profession.  First  among  these  is  the 
opportunity  to  gain  further  insight  into  the  reflective  processes  of  teacher  educators  during  an  intensive  and 
supervised  course  and  field  experience.  A second  benefit  involves  the  exploration  of  the  role  of  technology  in 
facilitating  (or  impeding)  attempts  first  at  reflection,  and  second  at  guided  reflection.  A third  benefit  accrues 
from  the  opportunity  to  observe  participants'  actual  instruction  at  the  Reading  Center  to  see  whether  (or  how) 
various  reflections  and  guided  reflections  are  enacted  with  Center  clients. 
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Courses  delivered  on  the  Internet  provide  "convenience  to  learners  both  in  terms  of  time  and  freedom  of  space" 
(Schlough  & Bhuripanyo,  1998).  In  their  study  of  a course  delivered  on  the  Internet  students  were  surveyed  in 
an  effort  to  identify  advantages,  disadvantages,  and  suggestions  concerning  asynchronous  course  delivery. 
Beyond  convenience,  advantages  included  learners  accessing  and  reading  course  materials  at  their  own  pace 
and  richness  of  materials  through  linkages  to  other  sites.  Weaknesses  of  this  style  of  instructional  delivery 
included  students  having  to  be  self-disciplined,  the  instructional  style  was  not  appropriate  for  all  learners,  and 
some  course  sections  were  confusing  to  use.  Suggestions  included  establishing  a chat  room  for  learner  support, 
improvements  in  the  site,  provide  a printed  version  of  the  text,  and  more  in-depth  orientation  to  the  course. 

As  the  use  of  Internet  based  instruction  continues  to  develop  it  will  be  important  to  examine  the  issues 
associated  with  this  style  of  learning.  Studies  concerning  instruction  on  the  Internet  have  examined  the 
student’s  perspective  (Mory,  Gambill,  & Browning,  1998),  provided  suggestions  concerning  course 
development,  technical  issues  (Schuttloffel,  1998),  and  interaction  guidelines.  However,  studies  that  examine 
and  question  pedagogical  concerns  are  not  as  evident. 

San  Diego  State  University  has  encouraged  faculty  to  re-envision  what  post-secondary  education  looks  like 
through  the  use  of  the  multiple  technologies.  In  response  to  the  campus  wide  interests,  the  Department  of 
Exercise  and  Nutritional  Sciences  changed  their  introductory  kinesiology  course  from  traditional  learning  to 
asynchronous  learning,  using  the  Internet.  The  purpose  of  this  study  was  to  examine  and  explore  various 
pedagogical  concerns  from  the  perspective  of  the  introductory  course  instructor.  The  results  of  this  study  will 
add  to  the  collective  knowledge  of  Web  based  instruction  methods.  It  is  further  intended  to  support  changes  in 
on-line  course  design  and  presentation  by  describing  the  pedagogical  issues  of  concern  to  the  instructor. 


Methodology 

This  research  study  was  designed  to  examine  pedagogical  issues  associated  with  a Web  based  course. 
Introduction  to  Kinesiology  is  a required  course  taken  by  all  students  entering  the  department.  Course 
information  is  designed  to  expose  students  to  the  department,  faculty,  campus,  and  potential  careers  through  a 
series  of  lectures,  assignments,  and  projects.  Information  for  the  course  (syllabus,  lecture  notes,  assignments, 
announcements,  etc.)  were  organized  through  designed  web  links  to  individual  parts  of  the  course.  There  were 
two  weekly  course  meeting  times  where  the  instructor  was  available,  but  no  on-campus  student  attendance  was 
required.  Communication  between  students  and  instructor  could  occur  through  class  session,  e-mail,  telephone, 
and  office  hours. 

The  subject  for  this  study  was  the  instructor  for  the  Web  based  course  taught  to  approximate  200  undergraduate 
students.  This  instructor  was  very  familiar  with  pedagogical  strategies,  but  had  not  designed  the  Internet 
course.  It  was  the  first  semester  that  the  subject  had  taught  the  course.  Throughout  the  first  several  weeks  of  the 
semester  the  subject  kept  notes  of  pedagogical  concerns  and  reflections.  After  the  first  few  weeks  the  subject 
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engaged  in  several  informal  interviews  with  the  researcher.  Both  the  subject  and  researcher  kept  notes, 
journals,  and  created  further  questions  aimed  at  probing  certain  pedagogical  topics  identified  by  the  subject. 


Data  Analysis 


Qualitative  methods  (Bogdan  & Biklen,  1992)  were  used  to  document  and  describe  the  process  of  identifying 
pedagogical  concerns.  Data  sources  included  subject  and  researcher  notes,  journals,  and  reflections.  Researcher 
and  teacher  generated  documents  and  interviews  were  analyzed  using  document  analysis.  To  organize  analysis 
of  these  documents,  the  theoretical  perspectives  and  methodological  approaches  of  structuralism  were  enlisted. 
Structuralism  views  ''documents"  as  “texts”  (Manning  & Cullum-Swan,  1994,  p.  467).  As  such,  once  the  data 
was  accumulated  in  written  form,  it  was  read  through  several  times.  With  each  reading  a set  of  tentative  themes 
was  developed  and  refined.  The  emerging  themes  and  issues  arising  from  the  readings  created  the  codes  or 
interpretive  frames  that  guided  and  structured  the  analysis. 


Results 

Several  pedagogical  issues  emerged  throughout  this  investigation  including  course  content,  testing,  and 
communication.  The  subject  identified  concerns  with  the  depth  and  breadth  of  lecture  notes  on  the  web  sites. 
The  question  arose,  how  to  provide  focus  and  content  emphasis  for  students.  Many  of  the  instructor  reflections 
were  directly  impacted  by  feedback  from  the  students.  Student  feedback  was  obtained  through  anonymous 
comment  mailforms,  e-mail  message  questions,  questions  asked  during  class  sessions,  and  phone  messages. 

Developing  an  environment  conducive  to  testing  was  another  great  concern  for  this  instructor.  Issues  of 
honesty  and  integrity  were  an  important  concern,  but  was  not  overshadowed  by  location  and  proctoring 
problems.  All  of  the  exams  were  to  be  taken  using  a computer.  In  past  semesters  students  were  able  to  attend 
an  open  lab,  show  identification  and  take  the  exam  at  their  convenience.  This  previous  arrangement  was  not 
available  during  the  semester  that  the  study  took  place  and  resulted  in  several  time  management  issues  for  the 
instructor  and  students. 

Providing  quality  communication  with  students  was  another  theme  which  emerged  throughout  the  study.  The 
course  used  e-mail,  provided  two  weekly  one  hours  sessions  where  the  instructor  was  available,  and  instructor 
office  hours.  These  strategies  seemed  to  be  sufficient,  yet  students  were  constantly  phoning  and  visiting  the 
instructor  outside  of  these  prescribed  times.  Previously  identified  suggestions  for  instructors  of  Internet  courses 
include  “don’t  be  available  to  your  students  all  the  time”  (Boettcher,  1997).  Results  from  this  study  indicated 
that  further  examination  of  this  web  based  course  should  occur. 
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Abstract:  The  purpose  of  this  paper  is  to  illustrate  the  incorporation  of  the  World  Wide  Web  into  Spanish 
pedagogy  at  a local  high  school  in  South  Carolina.  This  on-going  project  fits  within  the  communicative  emphasis  on 
language  learning.  The  open-ended  nature  of  hypertext  imposes  an  active  role  in  the  learning  process  which 
motivates  the  individual,  crosses  curriculum  boundaries,  and  improves  basic  skills.  Such  an  environment  provides 
students  an  ability  to  “learn  language,  learn  about  language  and  through  language”.  (Warschauer,  1997).  However, 
what  level  or  quality  of  learning  actually  occurs  as  a result?  In  examination,  this  paper  provides  pieliiniimry 
findings  and  theoretical  support. 


WALL 

“For  every  hour  of  instruction,  there  are  34  hours  spent  in  the  wilderness  ”. 

(McCarthy,  1992). 

As  a teacher  in  rural  South  Carolina,  I facilitate  the  learning-process  of  my  high  school  students  through  the  implementation  of  a web  page 
project  in  their  Spanish  class.  Assigned  cultural  topics,  these  students  become  so  absorbed  in  the  work,  that  they  are  unaware  of  the  skills  being 
utilized,  the  cognitive  thinking  levels  used  to  complete  the  task,  or  the  hours  of  time  given  outside  regular  class  meetings.  This  paper  examines  Web 
Assisted  Language  Learning  (WALL)  in  the  context  of  these  on-going  student  projects  with  the  primary  foci  being  the  effectiveness  of  the  learning 
that  takes  place. 

Based  on  theoretical  principles  laid  out  by  cognitive  researchers  there  is  support  for  effective  student  learning  facilitated  by  these  projects. 
“Effective  learning  involves  a desire  on  the  part  of  the  learner  to  grapple  with  and  understand  material  in  order  to  carry  out  higher  cognitive 
functions.”(Koppi,  1997).  In  other  words  the  enticement  of  the  learner  to  take  a deeper,  more  internalized  approach  to  his/her  learning. 

Traditionally,  language  learning  has  been  viewed  as  a process  of  acquiring  language  skills,  and  evaluation  has  been  seen  in  terms  of 
achievement.  “On-line  language  learning,  however,  is  a different  process.  The  educational  objectives  should  not  be  defined  behaviorally.  Language 
learning  on  the  Internet  is  more  similar  to  painting,  singing  a chorus,  or  playing  a ball  game.”  (Kojiro,  1 997)  WALL  therefore  is  more  in  keeping 
with  John  Dewey  who  recognized  the  need  for  more  “cultural”  education,  one  that  combines  theory  and  practice,  to  help  create  more  well  rounded, 
intelligent,  and  adaptable  citizens.  (Dewey,  1964). 

Like  a word  processor  a student  engages  his/her  mind  to  comply  with  active  learning.  Engaged  are  his/her  thoughts,  imagination,  emotions,  and 
preconceptions  as  he/she  formulates  understanding  of  new  information.  (Caine,  1994).  To  examine  the  vast  array  of  information  the  student 
encounters  WALL  requires  this  processor  to  operate  at  a higher  cognitive  level.  As  a tool,  WALL  has  potential  to  implement  student  reflection, 
evaluation,  organization,  analyzation,  creativity,  comparative  thinking,  and  information  discovery  which  combine  to  encourage  students  to  learn  and 
to  construct  their  own  meanings  regarding  the  assigned  topic.  This  multiplicity  of  higher  order  thinking  skills  are  automatically  used  as  the  processor 
sifts  new  information.  In  the  words  of  “Plato  and  Aristotle,  truth  does  not  seep  into  the  mind,  but  rather  is  discovered  through  thought  and 
inquiry.”(Harlow  & Johnson,  1998).  These  web  pages  “enable  students  to  create  different  maps  or  models,  and  to  explore  multiple  answers”.  (Ervin 
& Baldwin,  1997). 

If  one  changes  the  tools  of  thinking  available  to  a child,  his  mind  will  have  a radically  different  structure. 

(Vygotsky,  1978) 

Vygotsky  lends  credence  to  WALL  and  these  hypertext  projects,  particularly  in  the  development  of  holistic  theories  of  language  learning. 
Vygotsky  places  language  learning  as  a forerunner  to  thought,  asserting  that  only  language  use  will  cause  the  higher  mental  processes  to  become 
operational.  (Zepp,  1989).  Language  is  viewed  as  a tool  used  by  the  learner  in  social  interaction  resulting  in  the  achievement  of  higher  cognitive 
processes.  Vygotsky’s  zone  of  proximal  development  encompasses  “the  distance  between  actual  developmental  level  ...  and  the  potential 
development  as  determined  through  problem  solving  under  adult  guidance  or  in  collaboration  with  more  capable  peers.”  (Vygotsky,  1978).  During 
these  projects  the  Web  is  used  as  the  tool  of  social  interaction  between  student  and  learning,  and,  at  the  same  time,  encourages  students  to  go  beyond 
their  current  skill  and  knowledge  levels.  Given  specific  topics,  students  explore  the  Web  in  search  of  data.  This  requires  them  to  read  for  a specific 
purpose,  evaluate  the  authenticity  of  data,  compare  different  view  points,  analyze  and  synthesize  diverse  sources  of  information,  and  construct  their 
own  understanding  of  the  topic  by  creating  their  own  web  pages.  Bruner  explores  the  implications  of  tutoring  and  guided  learning  through  his  notion 
of  “scaffolding” — step-by-step  assisted  learning  until  gradually  gaining  independence  (Bruner,  1986).  The  active-learner  goes  beyond  the 
information  provided  by  internalizing  the  information,  and  constructing  new  ideas  and  concepts.  The  learning  that  takes  place  with  these  projects  is 
student-directed.  Each  student  makes  choices  and  takes  the  responsibility  for  those  choices  thereby  resulting  in  more  meaningful  learning  (Dunlap, 
1998)  “In  general,  material  that  is  organized  in  terms  of  a person’s  own  interests  and  cognitive  structures  is  material  that  has  the  best  chance  of  being 
accessible  in  memory.”(Bruner,  1961)  Likewise  Summers  (1995)  points  out  that  when  students  are  acquiring  and  evaluating  on-line  information 
found  on  the  web,  students  gain  extensive  reading  practice  and  make  use  of  complex  comprehension  and  analysis  skills.  Students  employ  strategies 
such  as  scanning  to  retrieve  information.  Despite  unknown  vocabulary  structures  students  are  motivated  to  decode  text  in  the  target  language  thereby 
extending  their  knowledge.  Additionally,  the  Web  offers  language  students  multiple  modalities  through  combined  text,  sound  and  visuals.  “Visual 
stimuli  can  become  memory-assisting  devices”  (Shekels  & Schwartz,  1987). 

The  hypertext  documents  created  by  my  students  exemplify  cognitive  learning  and  use  of  critical  thinking.  Located  at 
iittp://members.tripod.com/~clinch  1 . these  web  pages  demonstrate  the  comprehension  of  information,  organization  of  ideas,  analysis  of  data, 
application  of  knowledge,  synthesis  into  their  written  document  and  evaluation  of  ideas.  Within  the  cognitive  domain  identified  by  Bloom 
(knowledge,  comprehension,  application,  analysis,  synthesis,  evaluation)  this  project  provides  the  opportunity  for  students  to  develop  and  to 
effectively  use  these  skills. 

The  student  produced  web  page  project  is  an  embodiment  of  teaching  students  how  to  learn....  Not  what  to  think  or  how  to  regurgitate  facts.  But 
how  do  we  assess  this  learning? 

Literature  is  lacking  in  viable  instruments  for  measuring  hypertext  learning,  (Ryser,  1996),  particularly  in  foreign  language  acquisition.  “When 
students  exchange  e-mail  with  their  peers  from  different  cultures  and  create  web  pages,  their  interest  is  directed  to  the  content”  (Ervin,  1997)  rather 
than  the  grammatical  structures  of  the  traditional  classroom  experience.  WALL  enables  the  language  learner  to  learn  in  actual  context  as  well  as  from 
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each  other.  (Kojiro,  1997).  My  students  are  observed  to  ask  questions  of  each  other,  aid  each  other,  and  collaborate  together  as  they  seek  to  achieve 
success  in  their  projects.  Instructional  and  educational  activities  assume  new  meaning.  No  longer  is  a paper  written  just  as  a course  requirement,  read 
just  for  a grade,  but  rather  in  the  context  of  authentic  communication  read  by  “real”  readers. 

Bearing  this  in  mind,  there  are  a number  of  research  studies  assessing  the  effects  of  computer  technology  in  the  classroom.  According  to  a study 
by  McKinnon,  Nolan  and  Sinclair  (1996)  in  New  Zealand,  computer  usage  enabled  students  to  not  just  become  technologically  literate,  but  also  to 
become  producers  of  knowledge  as  they  analyzed  data  and  information.  (McKinnon,  1996).  The  study  noted  that  within  a given  learning  activity 
there  was  a broader  cooperation  among  individuals,  more  spontaneous  interest,  an  expansion  of  time  and  attention  devoted  to  the  activity,  and  a 
stimulus  for  developing  the  research  spirit.  Like  examples  have  been  observed  in  my  classroom.  These  high  school  students  willingly  assist  each 
other  in  the  process  of  the  web  page  creation,  from  simple  data  retrieval,  cut  and  paste  activities  to  HTML  writing.  Hours  of  extra  time  are  dedicated 
to  the  activity,  during  lunch  and  after  school.  They  even  get-together  at  the  homes  of  friends  connected  to  the  Internet  to  continue  working.  The 
search  for  more  extensive  information  was  exemplified  by  a student  researching  human  rights  issues  in  various  South  American  countries- — “How 

can  these  things  happen?  Why?... .This  is  just  so  interesting.  I’ll  just  read  a bit  more.  I’m  almost  through” an  hour  passed  without  noticing  that 

lunch  was  missed. 

A.nother  study  by  Liu  and  Rutledge  (1997)  in  Texas  ascertain  the  same  quality  learning,  motivation,  and  time  on  task.  “Students  began  to  exert 
pressure  on  those  who  tended  to  neglect  the  project....  students  reiiiinucu  each  ether  not  the  teacher”.  Results  of  this  study  suggest  that  web  page 
design  is  a motivation  to  learn,  and  program  development  for  “real”  audiences  provide  its  impetus.  Their  research  also  revealed  student  exercise  of 
higher  order  thinking  skills  such  as  project  management,  research,  organization  and  presentation. 

Further  credibility  to  student  independence,  motivation,  critical  thinking  was  made  in  a 1997  comparative  study  in  7 urban  school  districts 
within  the  United  States.  (Follansbee).  “Students  who  had  access  to  on-line  use  not  only  were  able  to  identify  and  articulate  a problem  more  clearly, 
but  also  were  more  informed  and  had  projects  that  were  more  compelling.”  (Chavez,  1998).  This  study  concluded  that  the  Internet  can  increase 
learning. 

Personal  web  pages  can  be  expressions  of  a students  own  ideas,  thereby  promoting  ownership  of  the  learning  environment  according  to  Kapur  & 
Stillman  (1997).  Their  study  provides  evidence  of  improved  quality  learning  as  well  as  enhanced  levels  of  student  satisfaction.  Case  studies  indicate 
that  Web  supported  learning  causes  a significant  learning  curve  according  to  Gillham,  Buckner  & Butt  (1998).  Favorable  student  attitudes  and 
motivational  benefits  are  also  provided  by  Keeler,  1996;  Hill  & Smith  1998;  Forrester,  1995  to  name  a few. 

All  of  these  studies  lend  support  to  the  observed  behavior  and  learning  outcomes  of  my  students. 

Learning  in  the  hypennedia  environment  of  the  Web  is  a complex  and  dynamic  interacting  process,  yet  research  provides  no  definitive  answers 
to  the  “learning”  that  actually  takes  place  with  each  student.  I can  only  attest  to  the  learning  that  I have  observed  in  my  classroom.  My  students 
thrive  and  interact  with  the  Web  as  they  embrace  their  Spanish  projects.  It  is  a key  to  improved  learning  in  my  language  classroom.  Within  a 
structured  activity,  WALL  is  a tool  to  be  capitalized  upon  by  language  instructors  to  improve  learning.  These  activities  are  not  just  “cool”  and  “fun” 
but  provide  for  “real  learning”  in  a “real”  world.  Linguistic  skills  are  strengthened,  peer  interaction-collaboration  is  established,  while  at  the  same 
time  learning  about  contemporary  culture  and  everyday  life  in  the  target  country  is  maintained.  (Standards,  1996;  SC  Frameworks,  1994).  My 
analysis  of  WALL  and  stimulated  “learning”  continues  as  further  activities  are  implemented  and  a qualitative  study  is  undertaken. 

REFERENCES 

Bloom — http://www.wested.org/tie/dlm/blooms.html 

Bmner,  J.  S.  (1961).  The  act  of  discovery.  Harvard  Educational  Review,  31(1),  21-32. 

Bmner,  J.  (1986).  Actual  minds,  possible  worlds.  Cambridge,  MA:  Harvard  University  Press.  74-76.  In  Arnold,  S.M  (1996)  The  Contribution  of  Vygotsky — WWW 
document. 

Caine, R.  & Caine,G.  ( 1994).  Making  Connections:  Teaching  & the  Human  Brain,  Addison- Wesley.  http://www.2 1 learn. org/cats/CS/fca ine.html 

Chavez,  Paul  (1998)  Do  Students  Really  Learn  on  the  A/cf.^httP://www.msnbc.com/news/105763.asp 

Dewey,  J.  (\ 964)  John  Dewey  on  education:  selected  IVorks.  (R.D.  Archambault,  Ed.).  New  York:  Random  House. 

Dunlap,  J.  (1998). Encouraging  Lifelong  Learning  with  Learner-Constructed  Web-Based  Performance  Support  Systems.  CD-Rom.  Proceedings  of  Ed-media/Ed-Telecom- 
98,  June  20-25. 

Ervin,  B.  & Baldwin,  P.  (1997)  Power  at  Play  in  the  Comuter  Classroom,  http://leahi.kcc.hawaii.edu/org/tcc-conf 

Follansbee,S.,  Hughes,  R.,  Pisha,  B.,  & Stahl,  S.  (1997)  Can  Online  Communication  Improve  Student  Performance?  Results  of  a controlled  study.  ERS  Spectrum,  15(1), 
15-26.  (http://www.cast.Qrp/publications/st.studv/J 

Forrester,  M.  (1995)  Indications  of  Learning  Processes  in  a Hypertext  Environment  Innovations  in  Ed  & Training  International,  32(3),  256-268. 

Gilliam,  M.,  Buckner,  K.,  & Butt,  R.  (1998)  Web  Supported  Learning  - A user  evaluation  of  a Media  Studies  application.  CD-Rom.  Proceedings  of  Ed-media/Ed- 
Telecom98. 

Harlow,  S.  & Johnson,  D.  (1998)  An  Epistomology  of  Technology.  Educational  Technology  Review  1998Spring-Summer,  15-19. 

Kapur,S.  & Stillman,  G.  (1997)  Teaching  & Using  the  WWW:  A Case  Study.  /£77  34(4),  316-322. 

Keeler,  C.  (1996)  Networked  Instructional  Computers  in  the  Elementary  Classroom,  & their  Effect  on  the  learning  Environment:  A Qualitative  Evaluation.  Journal  of 
Research  on  Computing  in  Education,  28(3),  329-345. 

Kojiro,  A(1997)Evo/MO/i/ig  Evaluation:  Online  Instruction  of  EFL/ESL.  http://1eahi.kcc.hawaii.edu/org/tcc  conf97/pre.s/asao.html 

Koppi,  A.,  Lublin,  J.,  & Chaloupka,  M.  ( 1 997). Effective  Teaching  & Learning  in  a High-tech  Environment.  Innovations  in  Ed.  & Training  International,  34(4),  245-25 1 . 
Liu,  M.,  & Rutledge,  K.  (l997)The  Effect  of  a “Learner  as  Multimedia  Designer”  Environment  on  At-risk  High  School  Students’  Motivation  & learning  of  design 
Knowledge.  Innovations  in  Ed.  Training  International,  34(4). 

McCarthy,  B.  (1992)  Developing  CALL  Materials  for  the  Foreign  Language  Classroom:  Ideals  and  Practicalities.  Paper  presented  at  1 7‘*'  Annual  Congress  of  the  Applied 
Linguistics  Association  of  Australia.  University  of  Sydney,  10-14,  July. 

McKinnon,  D.,  Patrick,  C.,  Nolan,  P.,  & Sinclair,  K.  (1996)  The  Freyberg  Integrated  Studies  Project  in  New  Zealand:  A Longitudinal  Study  of  Secondary  Students’ 
Attitudes  Towards  Computers,  Their  Motivation  and  performance.  International  Conferences  on  Technology  and  Education,  p.  463-465. 

Ryser,  G.,  Beeler,  J.,  & McKenzie,  C.  (1995)Effects  of  a Computer-Supported  Intentional  Learning  Environment  on  Students  Self-concept,  self-regulatory  behavior,  & 
Critical  Thinking  Ability  Jbwrnc//  Educational  Computing  Research,  13(4),  375-385. 

Standards  (1996).  Standards  for  Foreign  Language  Learning:  Preparing  for  the  2 1st  Century.  Lawrence,  KS:  Allen  Press. 

Stickels,  L.  & Schwartz,  M.  (1987).  Memory  Hooks:  Clues  for  Language  Retention.  (ERIC  Document  Reproduction  Service  No.  ED  337  013) 

Summers,  B.  (1995)  Computer  Communications  and  LOTE.  Melbourne:  WWW  document.  In  Gregoire,  R.,  Bracewell,  R.  & Laferriere,  T.  (1996) 
http://www.tact.fse.ulaval.ca/fr/html/impactnt.html 

Vygotsky,  L.S.  (1978).  Mind  in  Society.  Cambridge,  MA:  Harvard  University  Press.  pl26  & 86.  In  Arnold,  S.M.  (1996)  The  Contribution  of  Vygotsky  — 
http://www.aamt.edu.au 

Warschauer,  M.  (1997)  Computer  Mediated  Collaborative  Learning:  Theory  to  Practice.  The  Modern  Language  Journal,  81,  470-48 1 . 

Zepp,  R.  (1989).  Language  and  Mathematics  Education.  Hong  Kong:  API  Press,  p.30-32. 

ACKNOWLEDGEMENTS 

Special  thanks  goes  to  Dr.  Tony  Fetherston  (Edith  Cowan  U.)  and  Dr.  Pat  Pecoy  (Furman  U.)  for  their  continued  support,  encouragement  and  time  spent  in  my  behalf. 


1370 


Technology  and  Urban,  Elementary  School  Reform 


James  M.  Laffey 
Linda  Espinosa 
Dale  Musser 

Center  for  Technology  Innovations  in  Education 
University  of  Missouri-Columbia 
iim@coe.missouri.edu 

The  Panel  on  Educational  Technology  of  the  President’s  Committee  of  Advisors  on  Science  and 
Technology  (PCAST)  summarized  its  findings  about  the  state  of  technology  in  K-12  education  as  follows: 
“During  a period  in  which  technology  has  fundamentally  transformed  America’s  offices,  factories,  and 
retail  establishments,  however,  its  impact  within  our  nation’s  classrooms  has  generally  been  quite  modest.” 
(PCAST,  1 997  p.  6)  When  measured  against  the  promise  of  educational  technology  the  use  and  impact  of 
technology  in  schools  is  small,  and  even  less  so  in  elementary  schools.  PCAST  recognized  the  potential  of 
technology  to  support  fundamental  changes  in  pedagogic  models  moving  toward  a more  “contructivist” 
paradigm.  In  reality,  however,  computers  are  used  primarily  for  learning  about  computers  (keyboarding) 
and  software  applications  (word  processing,  databases,  and  spreadsheets).  In  elementary  schools 
computers  are  primarily  used  for  practicing  isolated  basic  skills  and  for  playing  educational  games. 

The  Jefferson  School  Project  focuses  on  implementing  technology  to  bring  about  school  improvement  in  an 
elementary  school  in  an  impoverished  urban  community.  The  framework  for  the  project  starts  with  a model 
of  education  that  can  be  improved  through  the  use  of  technology.  The  goal  of  the  project  is  to  (1)  transform 
the  school  facility  and  culture  from  a factory  model  of  isolated,  and  sequenced  production  units  to  a 
networked  knowledge  center  of  an  involved  community,  (2)  build  instructional  practices  that  embody  a 
concept  of  the  child  as  a constructor  of  knowledge,  (3)  improve  the  learning  outcomes  of  children,  and  (4) 
understand  the  potential  of  technology  and  appropriate  development  methods  for  achieving  goals  1, 2 & 3. 

The  Jefferson  School  Project  is  a collaboration  among  the  Center  for  Technology  Innovations  in  Education 
of  the  University  of  Missouri  (CTIE);  McCormack,  Baron  & Associates,  a nationally  recognized  leader  in 
community  and  housing  development.  Southwestern  Bell,  the  Danforth  Foundation,  and  the  St.  Louis 
Public  School  Board.  The  children  of  Jefferson  School  are  in  need  of  significantly  improved  educational 
opportunities.  They  live  in  an  impoverished  community  and  attend  a school  with  low  achievement  test 
scores.  The  climate  of  the  school  is  characterized  by  a high  emphasis  on  control  (e.g.,  silence  during 
lunchtime);  whole  class  instruction,  drill  and  practice  of  isolated  skills,  and  a faculty  who  felt  left  behind  as 
the  school  district  invested  its  few  resources  in  magnet  schools  at  other  locations. 

Rapid  and  dramatic  changes  are  planned  for  the  Jefferson  School  and  community.  A new  principal  was 
hired  to  lead  the  staff  The  Danforth  Foundation  has  committed  $200,000  for  professional  development 
over  a two-year  period.  Southwestern  Bell  has  committed  approximately  $ .75  million  to  create  an 
exceptional  networking  capacity  for  the  school  and  has  provided  support  for  CTIE  to  assist  the  principal 
and  community  in  planning  reform  efforts  for  the  school  in  ways  that  take  advantage  of  technology. 
McCormack,  Baron  & Associates  is  developing  new  low-cost  housing  for  the  community  and  is  providing 
leadership  and  initiative  to  reform  the  school  to  serve  the  community,  including  efforts  to  fund  the  purchase 
of  new  computers  for  school  and  adult-learning  objectives.  In  effect,  the  project  is  an  emergence  of  an 
elementary  school  that  will  have  great  technological  capacity  and  serve  at-risk  minority  children  and  their 
community. 

Most  efforts  to  use  technology  in  elementary  education  use  technology  to  deliver  computer  assisted 
instruction.  We  agree  that  elementary  education  is  a process  of  helping  children  develop  the  knowledge  and 
skills  that  are  the  foundation  of  citizenry  and  competency  in  our  society.  In  addition,  however,  during  their 
pre-kindergarten  and  elementary  school  years  children  form  important  self-concepts  and  build  habits  of 
mind  that  lead  to  a social,  intellectual,  and  economic  destiny.  Development  of  the  young  child  is  shaped  by 
the  social  relationships  formed  with  parents,  other  children,  and  teachers.  The  knowledge  and  skills  that 
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children  develop  are  constructed  in  the  context  of  these  important  social  relations.  The  role  of  technology  is 
not  to  replace  these  relationships  as  it  builds  skills,  but  rather  to  support,  extend,  and  amplify  the  social 
relationships  and  levers  they  provide  for  early  education.  New  and  advancing  technology  can  play  this  role 
through  augmenting  educational  processes  that  are  fundamental  to  early  education  and  that  have  potential 
for  significant  improvement. 

Technology  can  play  this  role  by: 

• Enriching  experiences  though  visualization,  simulation  and  other  forms  of  mediation, 

• Supporting  active  learning  and  individualization  in  the  classroom, 

• Making  all  members  of  the  process  more  resourceful,  and 

• iiiiplementing  new  and  powerful  forms  of  communication,  sharing,  and  coordination  for  the  children, 
teachers,  parents,  and  extended  members  of  the  community  (school  board  and  administration,  university 
partners,  employers,  etc.). 


During  the  Summer  of  1998  a new  principal  was  hired,  an  advanced  network  installation  and  other  facilities 
improvements  were  undertaken,  a professional  development  plan  was  created  for  the  faculty,  and  an  initial 
workshop  was  held  to  develop  a vision  statement  and  build  awareness  about  the  use  of  active  learning  and 
technology  in  instruction.  The  key  form  of  professional  development  is  to  get  the  faculty  engaged  in  active 
learning  with  technology,  and  providing  opportunities  and  support  for  reflection  on  those  practices,  so  as  to 
drive  new  models  of  instruction  with  their  children. 

During  the  school  year  of  ‘98-‘99  the  following  activities  are  underway: 

(a)  workshops  for  teachers  focused  on  technology,  project  based  learning,  and  safe  schools  (Nine  day  long 
sessions  to  be  held  on  Saturdays  as  per  the  request  of  the  teachers), 

(b)  placing  two  computers  in  each  classroom  with  a goal  of  having  at  least  6 per  class. 

(c)  having  network  services  come  on-line  (including  email  and  web  servers) 

(d)  grade-level  teacher  teams  that  will  implement  interdisciplinary,  technology-rich  projects,  on  the  theme 
of  “community”, 

(e)  teacher  teams  becoming  engaged  in  the  decision-making  of  the  school  improvement  efforts,  and 

(f)  individual  support  for  teachers  as  they  try  new  teaching  approaches  and  use  new  technology  in  their 
classrooms. 

Along  with  these  work  efforts  to  implement  a technology  infrastructure  for  the  school,  change  instructional 
practices,  and  improve  learning,  we  are  documenting  and  studying  the  processes  of  implementation  and 
socialization.  Research  methods  include  multiple  interviews  with  all  teachers  and  other  key  participants, 
systematic  classroom  observations  of  teaching  and  learning  practices,  records  of  learning  outcomes, 
patterns  of  technology  usage,  and  analysis  of  artifacts  from  teacher  and  student  use  of  technology. 

Early  findings  suggest  several  key  themes: 

(a)  Irresistible  technology  meets  immovable  bureaucracy  - the  realities  of  large  school  districts  often 
mitigate  against  rapid  innovation. 

(b)  Through  learning  about  technology,  teachers  can  begin  to  feel  and  understand  the  power  of  hands-on, 
individualization  and  support,  and  then  contrast  this  approach  with  traditional  teacher  telling  and  classroom 
seatwork. 

(c)  Through  learning  with  technology,  teachers  can  begin  to  feel  the  power  of  multi-media  representation, 
networked  resources  and  learner  initiation. 

(d)  Network  and  media  technologies  are  engaging  for  children  and  adults. 

(e)  True  curriculum  change  will  be  a long  process. 

PCAST  (1997)  Report  to  the  President  on  the  Use  of  Technology  to  Strengthen  K-12  Education  in  the  United  States. 
President’s  Committee  of  Advisors  on  Science  and  Technology. 
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1.  Introduction 

The  University  of  Auckland  with  an  enrolment  of  26  000  students  shares  with  other  tertiary  educational 
institutions  the  vision  of  providing  life-long  learning  for  its  students  while  attempting  to  deal  with  the 
current  restraints  on  budget  and  a lack  of  progress  toward  optimal  use  of  technology  in  teaching  and 
learning. 

The  School  of  Business  and  Economics  at  the  University  of  Auckland  has  developed  a computer- 
supported  learning  system  (CSL  aka  Cecil)  that  is  now  in  its  fourth  year  of  operation  and  delivers 
browser-based  assessments  and  other  learning  materials  to  tens  of  thousands  of  students  in  business  and 
the  sciences.  On  a weekly  basis  Cecil  generates  many  thousand  unique  practice  sessions  and  quizzes, 
sends  out  hundreds  of  diagnostic  email  messages  and  at  times  is  processing  more  than  300  transactions 
per  second.  The  work  of  developing  Cecil  and  designing  it  for  life  long  learning  has  been  reported 
earlier  at  EdMedia  conferences  (Sheridan,  1997  & 1998) 

In  this  paper  we  report  on  our  efforts  to  move  the  curriculum  content  of  a large  business  school  onto 
the  web-environment,  as  provided  by  Cecil,  using  as  a guide  the  model  provided  by  the  University  of 
Tennessee  (Little  & Derco,  1998).  The  principle  actors  in  this  success  were  Extroverted  Learning 
Facilitators  (ELFs)  an  idea  derived  from  (Holt,  1997).  This  paper  will  outline  the  process,  achievements, 
problems  and  current  state  of  play. 

2,  The  ITC  Model 

The  Innovative  Technologies  Center,  University  of  Tennessee  provides  a practical  model  for  moving  an 
institution  toward  broadly  based  web  courses  (Little  & Derco,  1998).  The  four  phases  include:  general 
information,  augmentation,  stand  alone  instructional  pieces,  and  totally  on-line. 


Phase  1:  General  Information 

syllabus,  course  calendar,  bibliography,  communications  (email),  last  date  updated 

Phase  11:  Augmentation 

expanded  course  calendar,  expanded  bibliography,  announcements,  lecture  notes,  grades,  glossary, 

communications 

Phase  111:  Stand  Alone  Instructional  Pieces 

All  of  the  earlier  phases  plus,  tutorials,  self-assessments,  skills  survey,  communications  (threaded 

discussions,  chat,  MQO/MUD) 

Phase  IV;  Totally  Online 

All  of  the  earlier  phases  and  no  face  to  face,  student  biographies  and  introductions,  class  photos, 
course  authentication,  online  testing 

This  model  does  not  prevent  individuals  from  forging  ahead,  but  it  does  give  managers  and 
administrators  some  achievable  targets  at  the  early  stages  of  online  development.  Given  the  inevitable 
fiscal  constraints  then  the  question  arises:  How  to  get  the  existing  materials  converted  and  online. 
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3.  The  ELF 

Course  development  for  web  based  delivery  poses  a significant  problem  if  the  only  people  available  are 
fully  tasked  academics.  It  seemed  reasonable  to  adopt  a model  implemented  at  Athabasca  University 
where  students  worked  on  course  conversion,  design  and  development  with  positive  results.  (Holt,  1997) 
It  was  noted  that  students  often  have  a clear  vision  of  what  their  peers  would  want  in  a learner  centered 
environment  and  they  also  see  the  possibilities  of  the  technology  in  achieving  it. 

In  our  instance  we  employed  nine  summer  students  across  seven  departments.  We  sought  out 
students  to  become  Extroverted  Learning  Facilitators  (ELFs).  The  characteristics  we  needed  were 
principally  students  who  were  in  the  honours  or  masters  program  of  each  department,  and  at  the  same 
time,  enthusiastic  about  their  discipline  and  technology  in  general.  Their  infectious  enthusiasm  would 
carry  the  day  and  allow  us  to  pull  the  necessary  information  and  knowledge  from  academics  to  ensure 
we  could  push  the  school  to  Phase  II  within  three  months. 

The  ELF  worked  principally  in  a designated  department.  Training  was  provided  on  the  Cecil 
interface,  Microsoft  Office  97  to  HTML  conversion,  simple  HTML  authoring  and  editing  skills, 
document  conversion  to  Adobe  Acrobat,  Lotus  ScreenCam  production,  and  document  scanning  and 
editing  using  Adobe  Photoshop. 

4.  The  Results 

Over  the  summer  recess,  the  ELFs  exceeded  our  expectations  by  persuading  even  the  most  reluctant 
academics  to  participate.  In  meeting  the  Phase  I and  II  objectives  we  now  have  871  papers  on  line  with 
total  enrolment  of  27,000+  students.  More  than  240  staff  are  using  the  Cecil  Explorer  (interface).  Staff 
training  was  designed  and  carried  out  by  the  ELFs.  A CD  was  also  produced  with  learning  materials 
better  suited  to  off-line  access.  These  included  HTML  documents,  Office  97  and  Acrobat  files,  software 
applications,  browser  updates,  and  a video  and  audio  segment.  The  Dean  and  department  heads  were  so 
impressed  with  the  results  they  decided  to  extend  part-time  employment  to  the  ELFs  for  the  next 
academic  year.  ELFs  are  now  providing  individualised  assistance  to  their  staff  as  well  as  working  the 
Cecil  help  desk  for  the  benefit  of  all  the  academic  community. 

Cecil  system  provides  a central  focus  for  the  integrity  of  course  content  by  storing  learning  resources 
within  its  database.  Cecil  is  a multi-tiered  computer  architecture.  The  database,  SQL7.0,  located  on  a 
multiprocessor  Sequent  NT  has  grown  to  8GB  in  only  a few  months.  The  dual,  fault  tolerant,  front-end 
computers  are  the  internet  servers  and  process  approximately  3M  hits  per  week.  It  has  become  a 7 day 
24  hour  operation,  (www.cecil.edu) 

While  Cecil  is  being  used  a repository  for  learning  materials,  it  is  really  an  instructional  management 
and  assessment  system.  At  this  point,  only  a few  academics  have  created  personal  bodies  of  knowledge 
(BoK)  on  their  desktops  and  replicated  portions  of  the  BoKs  across  to  the  major  Cecil  site.  More 
progress  is  expected  on  this  front  as  the  academics  come  to  understand  the  significance  of  this  feature 
since  the  goal  is  to  provide  a shared  BoK  that  resides,  in  part,  on  the  academic’s  desktop,  the 
university’s  Cecil  system,  and  a student’s  laptop.  Collaboration  among  peers  whether  they  be  academics 
or  students  will  result  in  dramatic  changes  in  the  way  knowledge  is  created,  shared,  and  managed. 

This  an  on-going  project.  Phase  I & II  was  for  many  academics  a hint  of  the  new  frontier.  An  update 
will  be  provided  at  the  EdMedia  99  conference. 
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Introduction 

The  Software  Engineering  Research  Network  (SERN)  is  funded  by  industry  and  administered  by  the  Industrial 
Software  Engineering  Chair  at  the  University  of  Calgary  to  support  the  dissemination  of  good  practice  in  software 
engineering.  One  component  of  SERN's  activities  is  a thesis-based  masters  program  with  a specialization  in  software 
engineering  targeted  at  students  with  industrial  experience  in  full-time  employment  (Shaw  & Gaines,  1998).  This 
program  has  dual  objectives:  to  develop  highly  qualified  personnel,  and  to  encourage  industry-based  software 
engineering  research  with  a focus  on  good  practice.  Course  work  includes  requirements  that  have  been  developed  in 
consultation  with  the  industry  partners.  The  learning  environment  is  unconventional  and  reflects  the  industrial 
experience  of  the  students.  After  the  first  lecture  in  a required  course,  students  take  on  much  of  the  responsibility  for 
presentations  on  the  various  topics  that  draw  upon  their  experience  and  that  of  their  colleagues.  The  instructors'  role 
is  that  of  facilitator,  "guide  on  the  side",  managing  a process  of  debate  and  exploration  rather  than  attempting  to  be 
the  "sage  on  the  stage"  authority  in  the  domain. 

The  overall  aim  of  the  masters  program  is  to  provide  a supportive  and  nurturing  learning  environment  in  which 
experience  and  knowledge  can  be  shared  and  created,  and  errors  and  omissions  made  without  censure  but  with  ready 
access  to  diagnostic  help.  Student  assignments  are  submitted  on  the  World  Wide  Web  making  them  accessible  to 
others.  A list  server  is  used  for  continuing  discussion  of  the  course  topics  outside  the  class  environment.  These  two 
features  have  made  it  possible  for  students  whose  companies  allocate  them  to  work  in  other  locations,  or  who 
change  jobs  to  companies  in  other  locations  world-wide,  to  continue  to  participate  in  the  courses.  Remote  students 
who  have  already  come  to  know  their  colleagues  can  continue  to  present  and  share  material  through  the  web  and 
participate  in  discussion  through  the  list  server.  It  becomes  clear  to  students  as  they  progress  through  the  program 
that,  while  there  are  no  easy  answers  to  the  core  questions  of  industrial  practice  in  software  engineering,  there  are 
many  useful  perspectives  and  that  simplistic  answers  generally  have  very  limited  applicability. 

We  are  currently  addressing  the  challenge  of  making  our  graduate  program  formally  available  and  accessible  to 
students  who  want  to  participate  from  a distance.  A first  step  has  been  to  develop  and  offer  parallel  sections  of 
SENG  61 1,  a six-week  graduate  course  in  Requirements  Engineering.  One  section  of  SENG  611  was  delivered  on 
campus,  and  the  other  delivered  at  a distance  using  facilities  provided  by  WebCT.  Of  the  21  students  in  the  course,  7 
chose  to  participate  in  the  distance  section,  and  14  on  campus  in  a conventional  face-to-face  learning  environment. 
In  addition  to  individual  assignments,  all  students  completed  a major  group  project  that  was  presented  during  the 
final  lecture.  The  on-campus  students  recorded  and  summarized  classroom  discussions  that  were  posted  on  the  Web 
for  the  distance  students. 

WebCT  and  Technical  Support  for  SENG  611 

The  technical  support  requirements  for  SENG  611  were  concerned  with  two  main  issues:  1)  supporting  the 
organization  and  delivery  of  the  course,  and  2)  minimizing  the  differences  in  student  engagement.  Although  the 
course  material  already  existed  as  web  pages,  WebCT  was  chosen  as  the  web-based  software  tool  to  support  the 
dissemination  of  course  related  information  in  this  experiment  (see  Goldberg,  Salari,  & Swoboda,  1996).  WebCT 
offered  encapsulation  and  paths  through  the  existing  course  content,  student  presentation  areas  for  groups  of 
students,  individual  student  tools  for  note-taking,  and  a chat  facility  for  synchronous,  online  discussions.  It  was  the 
preferred  choice  for  a number  of  reasons,  including  free  testing  of  WebCT  with  full  functionality  at  the  University  of 
Calgary,  inexpensive  licenses,  a large  customer  base,  proximity  of  developers,  ease  of  access  via  common  web- 
browsers,  and  ease  of  customization  (if  needed). 
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In  Spring  1998,  WebCT  was  installed  on  a local  server  so  we  could  experiment  with  many  of  its  features.  This  led  to 
enough  confidence  in  WebCT  for  its  use  with  the  graduate  course  in  Fall  1998.  The  technical  support  person  and  a 
course  instructor  were  well  versed  in  HTML,  scripting,  and  other  computer-related  activities  which  influenced  the 
evaluation  and  usage  of  WebCT.  Based  on  the  current  experiment  using  WebCT,  we  have  made  the  following 
observations: 

1 . Support  from  developers  of  WebCT  is  very  good.  Queries  and  requests  were  dealt  with  quickly  and  to  our 
satisfaction. 

2.  It  is  easy  to  (a)  install  WebCT  (beta  1 .3  on  Windows  NT  took  slightly  more  time  than  1.2  on  Unix),  (b)  create 
courses  as  an  administrator,  and  (c)  encapsulate  existing  conrse  materiaL 

3.  It  is  moderately  difficult  to  get  used  to  all  of  the  WebCT  concepts  and  to  develop  a mental  map  of  the  course 
structures,  and  to  keep  information  up-to-date  in  a course,  because  the  tool  imposes  a certain  approach  to 
structuring  on-line  course  materials.  One  must  become  familiar  with  the  affordances  and  constraints  inherent  in 
WebCT's  design  in  order  to  construct  well-organized  course  materials. 

4.  It  was  difficult  to  customize  WebCT  to  add  connections  to  the  external  list  server;  Perl  scripts  had  to  be 
modified  to  add  a mailto:  hyperlink  (referring  to  the  list  server)  to  WebCT’s  common  button  bar. 

WebCT  is  geared  to  non-technical  course  designers;  faculty  with  little  or  no  prior  experience  with  HTML  will  find 
the  inherent  structure  and  design  of  WebCT  to  be  useful  starting  points.  However,  it  was  our  experience  that  the 
WebCT  designer  interface  often  hampered  the  HTML-knowledgeable  instructor's  efficiency  in  building  and 
maintaining  the  course  material.  For  example,  sequencing  course  materials  requires  the  designer  to  set  up  a "path", 
which  involves  a number  of  intermediate  steps  that  would  not  be  required  if  the  designer  was  using  only  HTML. 
However,  for  an  instructor  who  is  new  to  creating  web-based  documents,  the  structure  and  tool  set  offered  by 
WebCT  provide  valuable  entry-level  supports  and  templates  as  they  construct  course  materials. 

We  found  that  WebCT  was  not  completely  applicable  to  the  current  offering  of  the  SENG  611  course.  However, 
based  upon  research  that  compared  WebCT  to  other  web-based  course  tools  (e.g.,  Kristapiazzi,  1 998),  we  expect 
that  other  web-based  course  tools  are  also  not  completely  applicable  and  would  require  customization  for  SENG 
courses.  Courses  in  the  research-based  M.Sc.  program  have  been  developed  within  an  open  architecture  philosophy 
and  the  belief  that  there  should  be  public  access  to  past  and  present  SENG  course  materials  and  student  work. 
Inherent  to  the  design  of  WebCT  is  a closed  architecture  philosophy  - courses  are  password  protected,  and 
information  in  courses  is  not  easily  accessible  to  the  general  public. 

WebCT's  student  tools,  in  particular  note  making  and  group  presentation  areas,  offer  useful  tools  for  online  courses. 
The  e-mail  and  chat  facilities  in  WebCT  were  not  used  for  the  current  experiment;  an  external,  publicly  accessible 
list  server  remained  in  use.  WebCT's  quizzes,  which  employ  selected  response  and  open-ended  questions,  and  so  on, 
were  not  useful  for  this  graduate-level  course.  However,  we  did  employ  these  features  in  the  service  of  a midterm 
examination  for  a large  undergraduate  course  (CPSC  533)  in  Artificial  Intelligence.  We  have  found  the  online 
testing  features  of  WebCT  offer  a reliable  and  useful  environment  for  exams.  In  order  to  provide  some  practice  with 
the  WebCT  exam  environment  prior  to  the  midterm,  we  required  students  to  login,  change  their  password,  and 
complete  a short,  pre-exam  survey.  In  this  way,  those  students  who  experienced  difficulties  had  an  opportunity  to 
seek  assistance  before  attempting  the  online  midterm  exam.  Our  findings,  in  brief:  students  were  frustrated  with  a) 
the  time  clock  feature,  which  only  updates  when  an  answer  is  submitted,  thus  giving  a false  impression  about  time 
left  to  complete,  and  b)  having  to  submit  each  answer  individually  rather  than  the  entire  completed  exam. 
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Problem 

Colorado  business  and  marketing  education  programs  face  a state  of  emergency  due  to  the  shortage  of 
qualified  teachers,  A qualified  teacher  is  one  who  has  compleieu  a teacher  licensure  program  and  vocational 
credentialing  courses  in  business  or  marketing.  According  to  personnel  at  Colorado’s  Department  of  Education, 
approximately  60  positions  will  be  opened  next  year  in  business  and  marketing,  while  only  20  to  25  new 
teachers  will  be  licensed  and  credentialed  in  business  and  marketing  education  for  the  1999-2000  school  year. 

Because  of  the  shortage  of  qualified  business  and  marketing  teachers,  secondary  schools  are 
requesting-- in  record  numbers— emergency  credentials  for  instructors  so  they  will  be  able  to  teach  in  a business 
or  marketing  classroom  while  they  go  back  to  school  for  needed  course  work.  The  majority  of  teachers  on 
emergency  credentials  are  licensed  teachers  from  other  disciplines,  teachers  from  other  states,  or  business 
people  who  have  become  teachers.  Currently  44  business  and  marketing  teachers  across  Colorado  need 
between  four  to  ten  credit  hours  to  be  certified  in  business  and/or  marketing  education.  With  the  number  of 
retiring  business  teachers  increasing  and  the  small  number  of  new  teachers  being  licensed  and  credentialed, 
emergency  credentialing  of  teachers  soon  will  become  the  norm  instead  of  the  exception. 

One  barrier  in  Colorado  to  the  credentialing  of  marketing  and  business  teachers  is  the  availability  of 
credentialing  courses.  Only  Colorado  State  University  (CSU)  can  offer  these  required  credentialing  courses. 
Some  courses  are  offered  only  once  or  twice  a year.  In  most  cases,  students  must  attend  classes  at  CSU's  main 
campus,  a hardship  on  the  students  because  of  long  driving  distances  in  Colorado 

The  shortage  of  business  and  marketing  teachers  is  further  acerbated  by  the  current  delivery  structure 
of  teacher  licensure  courses.  Most  students  wishing  to  become  a business  or  marketing  teacher  already  have 
completed  their  bachelors’  degree  in  business.  These  preservice  teachers  need  about  34  credits  in  education 
coursework  to  obtain  a teacher  license,  plus  eight  to  twelve  credits  in  vocational  education  to  obtain  a 
vocational  credential.  The  current  arrangement  of  courses  at  CSU  dictates  an  investment  by  the  preservice 
teachers  of  at  least  a year  and  a half  of  full-time  study,  during  which  time  it  is  difficult  to  maintain  a job  to 
support  themselves  and  their  families. 


Project  in  Progress 

To  meet  this  need,  Colorado  State  University  has  begun  to  turn  to  technology  in  the  delivery  of 
vocational  credentialing  and  teacher  education  courses.  A combination  of  face-to-face  interaction  (through  the 
use  of  seminar-like  sessions)  and  online  learning  will  be  used.  For  the  online  learning  piece,  CSU’s  School  of 
Education  will  use  WebCT  for  learning  module  coordination,  online  testing,  threaded  discussions,  chat  rooms, 
email,  and  bulletin  boards.  The  inclusion  of  distance  learning  via  digital  satellite  will  be  available  as  a teaching 
methodology  sometime  during  Fall  1999  through  CSU’s  partnering  with  the  Agricultural  Distance  Extension 
Consortium. 

The  first  phase  of  this  project  will  be  to  bring  online  the  graduate-level  methods  courses  in  business 
and  marketing  education  which  are  used  in  credentialing.  During  Fall  1999,  the  undergraduate  methods  courses 
in  business  and  marketing,  and  the  other  credentialing  courses  will  be  placed  online.  During  2000, 
approximately  four  teacher  licensure  courses  (still  to  be  finalized  by  the  teacher  licensure  committee)  will  be 
coverted  to  include  heavy  (but  appropriate)  levels  of  online  activity.  Possible  teacher  licensure  courses  to  go 
online  are  Schooling  in  the  United  States,  Educational  Psychology,  and  parts  of  Classroom  Management  and 
Classroom  Technology. 

This  project  allows  new  teachers,  especially  those  entering  the  educational  field  from  industry  or  from 
another  teaching  area,  to  gain  the  knowledge  and  skills  necessary  for  success  in  the  classroom  in  a timely,  but 
quality-based  manner.  In  addition,  the  potential  teacher  could  remain  employed  for  a longer  period  of  time 
while  acquiring  the  skills  and  content  needed  to  teach. 
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Data  Available  at  the  Time  of  Publication 


Several  areas  already  have  been  researched:  (1)  the  technological  capabilities  that  are  available  and 
which  ones  will  be  used  (described  briefly  in  the  section  above),  (2)  standard  format  for  online  credentialing 
courses,  (3)  the  topics  that  should  be  covered  in  credentialing  courses,  and  (4)  what  linkages  should  be  made 
between  the  Colorado  Department  of  Education  (CDE),  Colorado  Community  College  and  Occupational 
Educational  System  (CCCOES),  “expert”  teachers,  and  business  people. 

Online  credentialing  courses  will  assume  one  of  two  formats.  Format  1 uses  ten  to  fifteen  hours  of 
face-to-face  time  at  the  beginning  of  the  course,  with  the  remainder  of  the  course  to  be  completely  online. 
Format  2 also  has  the  majority  of  educational  time  online,  but  will  utilize  face-to-face  seminars  once  a month, 
tu  cuver  tiie  iiiOi'c  delicate  cducatiOiial  issues.  Witli  both  formats,  guest  speakers  will  either  be  present  dunng 


the  face-to-face  meetings  or  available  during  assigned  chat  times.  Students  will  share  projects,  either  during  the 
seminars  or  online.  Collaborative  learning  environments  will  be  developed,  utilizing  WebCT’s  ability  to  aid 
inquiry-based  learning  through  bulletin  broads,  email,  and  web  site  posting.  Philosophically,  the  educational 
theory  that  will  underpin  the  building  of  these  courses  is  pragmatism  and  constuctivism  (Miller  1996). 

In  preparation  for  online  delivery  of  credentialing  courses,  research  was  conducted  to  determine  the 
salient  topics  that  should  be  included.  According  to  (Lynch,  Smith,  & Rojewski  1994),  the  major  educational 
reform  themes  that  should  be  addressed  in  secondary  vocational  education  include  secondary/postsecondary 
institution  linkages,  equal  access  and  opportunity  for  all  students,  flexible/altemative  delivery  systems,  support 
services  and  funding,  curriculum  redesign,  business-education  partnerships,  alternative  assessment  techniques, 
world  class  standards  and  assessment,  and  on-the-job  training  models.  According  to  a 1999  survey  conducted 
by  myself  with  Colorado  business  and  marketing  teachers  who  have  taught  three  years  or  less,  the  areas  where 
additional  training  is  needed  are:  the  integration  of  vocational  student  organizations  into  the  curriculum, 
funding  sources  for  student  and  school  projects,  recruitment  of  students,  marketing  of  their  vocational  program, 
and  classroom  management.  Obviously,  a marriage  will  need  to  be  made  between  reform  themes,  which  may 
not  be  first  on  a new  teacher’s  “need  to  know”  list,  and  the  practical  needs  identified  in  the  1999  survey. 

Four  graduate  courses  have  been  converted  to  an  online  format  and  will  be  offered  during  the  1999 
summer  session.  These  courses  are:  Methods  in  Vocational  Marketing  Education,  Programs  in  Vocational 
Marketing  Education,  Management  of  Business  and  Office  Departments,  and  Organization  of  Business  and 
Office  Education.  The  topics  identified  above  have  been  integrated  into  these  courses,  along  with  the  areas  of 
technology  in  the  classroom,  Colorado  Standards,  and  career  preparation  and  development. 

Linkages  have  been  established  with  CDE  and  CCCOES.  Representatives  from  both  agencies  have 
reviewed  the  content  for  the  graduate-level  methods  of  business  and  marketing  education  courses  (phase  one). 
Suggested  topics  for  inclusion  were:  integration  of  academic  and  vocational  education;  evaluation  tools  for 
employers,  students,  and  follow-up;  program  approval;  professional  organizations;  and  support  systems  for  new 
instructors.  Staff  from  both  agencies  will  participate  as  guest  speakers.  A survey  is  out  now  in  regard  to 
linkages  that  could  be  established  with  expert  teachers.  Business  linkages  will  be  established  through  expert 
teachers. 


Additional  Data  to  be  Available  at  the  Conference 

At  the  time  of  the  conference,  module  outlines  for  business  and  marketing  education  and  other 
credentialing  courses  will  be  available.  Specific  issues  as  orientation  to  an  online  course,  building  collaboration 
online,  using  threaded  discussion  and  chat,  and  evaluation  of  online  courses  will  be  discussed.  Field  test 
comments  on  two  or  three  courses  will  be  available.  Expert  teachers'  participation  level  will  be  identified. 
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Two  growing  factors  in  today's  classrooms,  diversity  and  technology,  are  impacting  teachers'  ability  to 
influence  student  growth  (Grossman,  1995;  Male,  1994).  Even  for  prepared  teachers,  today's  heterogeneous 
classrooms,  with  bilingual  and  disabled  children,  are  a constant  challenge.  The  information  highway  can  benefit 
both  students  and  teachers  if  the  proper  structures  are  in  place  (Male,  1994). 

In  California's  urban  schools,  the  challenge  is  paramount.  The  overwhelming  majority  of  the  students 
are  from  culturally  and  linguistically  diverse  backgrounds,  and  over  ten  percent  of  the  students  qualify  for 
special  education  (Grossman,  1995).  A typical  classroom  has  several  students  for  whom  English  is  a second 
language  and  who  have  either  learning  disabilities  or  language  disorders.  It  is  difficult  for  a teacher  to  prepare 
interesting,  appropriately  adapted  lessons  for  every  child  in  her  class  six  hours  a day,  1 80  days  a year. 

Given  the  mandate  to  include  all  students  in  the  general  education  classroom  and  to  create  a classroom 
climate  that  is  an  arena  for  goal-oriented,  reflective  problem  solving  (Laboratoiy  of  Comparative  Human 
Cognition,  1989),  schools  need  new  technologies.  An  integrated  curriculum  is  key  to  including  all  students 
(Sheppo  et  al,  1995).  Multimedia  technologies  have  significant  potential  for  supporting  an  integrated  and 
adaptive  curriculum  (Willis,  1992).  Gardner  (1993)  argues  that  individual  children  need  to  be  taught  differently, 
because  individuals  understand  the  world  in  different  ways.  Gardner  recommends  that  schools  be  filled  with 
"apprenticeships,  projects,  and  technologies"  so  that  every  kind  of  learner  can  be  accommodated. 

Telecommunications  can  open  up  vistas  of  new  learning  for  students  and  expand  collaboration 
between  colleagues  (Reinhardt,  1 995^  Teachers  can  interact  and  share  ideas  with  colleagues  and  the  interaction 
need  not  occur  in  "real  time".  The  benefits,  however,  go  far  beyond  simple  communication.  Databases  can  be 
developed  which  will  be  a resource  for  trying  various  methods  and  collaborate  about  their  effectiveness. 

The  present  project  was  created  to:  (1)  use  web  instruction  in  a University  teacher  preparation  program;  (2) 
prepare  student  teachers  to  transfer  learning  from  University  classroom  to  school  district  classroom;  and  (3) 
provide  a structure  for  student  teachers  and  classroom  teachers  of  mainstreamed  learning  disabled,  limited 
English  proficient  students  to  collaborate  about  adapted  lessons. 

The  ongoing  project  begins  with  student  teachers  in  an  urban  general  education  teacher-training 
program  by  providing  training  during  a required  course.  Teaching  Exceptional  Students.  The  instructor  uses 
Web-CT  software  to  develop  a course  website  and  place  course  components  on  the  internet  (agendas, 
requirements,  activities,  quizzes,  discussion  forums,  and  teaching  cases).  Students  all  have  internet  accounts 
provided  by  the  University. 

The  on-line  curriculum  adaptation  project  requires  a training  phase.  The  training  covers  teaching 
students  with  learning  disabilities,  understanding  stages  of  language  acquisition  with  implication  for  instruction, 
strategies  for  adapting  integrated  lessons  for  disabled  and  limited  English  proficient  students,  and  instruction 
regarding  web  site  navigation.  A long-term  goal  of  the  project  is  to  develop  an  adapted  curriculum  database  on 
the  internet.  In  order  to  do  this,  student  teachers  are  also  given  training  regarding  inputting  lessons  on  a 
curriculum  development  tool  placed  on  Web-CT.  Once  this  training  is  complete,  the  student  teachers  form 
cooperative  learning  groups  to  begin  applying  their  learning. 

After  the  training  phase,  groups  are  given  teaching  cases  that  include  written  descriptions  and  the 
Individual  Education  Programs  of  actual  special  education  limited  English  proficient  students  mainstreamed 
into  an  actual  neighboring  inner-city  classroom  (names  are  changed  to  provide  confidentiality).  Students  then 
work  together  to  examine  previously  written  lessons  designed  for  "typical"  general  education  students  and 
develop  ideas  for  adapting  lessons  for  these  students  using  strategies  for  working  with  their  learning  disabilities 
and  language  comprehension  needs.  The  cooperative  learning  groups  then  input  adapted  lessons  in  the  web. 


1379 


The  project  then  goes  one  step  further  by  conveniently  putting  the  student  teachers  in  touch  with  the 
actual  classroom  teacher  of  the  students  that  have  been  studied.  This  is  an  advantage  of  using  the  internet  far 
beyond  the  collection  of  a database.  At  her  convenience,  often  at  home  late  in  the  evening,  the  classroom 
teacher  is  able  to  access  the  curriculum  development  tool  on  the  web  and  comment  on  the  lesson  and 
adaptations.  The  student  teachers  are  then  able  to  reflect  on  their  adaptations  and  comment  back  to  the 
classroom  teachers,  again  on  the  internet.  This  dialogue  provides  real-life  feedback  and  two-way 
communication. 

The  college  instructor  also  accesses  the  lessons  on  the  web  and  comments  on  the  groups'  work,  giving 
immediate  feedback.  While  this  feedback  is  an  effective  teaching  tool  for  the  course,  the  responses  from  the 
classroom  teacher  prove  to  be  the  most  helpful.  Finally,  the  students  revise  their  curriculum  adaptations,  based 
on  ilic  feedback. 

The  project  is  evaluated  in  three  ways.  Students  complete  course  evaluations,  in  addition  lo  this,  they 
also  respond  to  a survey  instrument  regarding  attitudes  about  adapting  curriculum  and  online  instruction.  But 
perhaps  the  most  important  evaluation  is  careful  observation  of  the  process  from  beginning  to  end,  with  special 
attention  paid  to  the  two-way  feedback  process. 

Course  evaluations  regarding  use  of  the  curriculum  development  tool  have  been  extremely  positive, 
indicating  that  success  relates  to  the  structure  of  the  tool  itself,  the  teamwork  involved,  and  the  on-line  feedback 
component.  Additionally,  student  teachers  self-report  that  they  are  using  other  students’  adapted  lessons,  which 
are  available  on  the  internet. 

Survey  responses  from  participants  in  the  first  semester  of  the  project  were  categorized  into  two 
general  areas.  There  was  a mean  score  of  4.4,  somewhere  between  “agree”  and  “strongly  agree,”  for  all 
participants  on  the  items  focusing  on  perceived  ability  to  adapt  instruction,  indicating  that  the  project  increased 
their  ability  to  make  curricular  and  instructional  adaptations  for  differing  student  needs.  The  mean  score  for  all 
participants  on  the  items  regarding  the  perceived  effect  of  on-line  collaboration  and  feedback  with  the 
classroom  teacher  was  4.0.  This  indicates  agreement  with  the  notion  that  the  collaborative  aspect  of  the  project 
was  a positive  factor. 

The  results  of  the  two-way  feedback  between  the  student  teachers  and  the  classroom  teacher 
continually  go  beyond  the  project's  original  expectations.  Each  time  groups  receive  feedback  from  the 
classroom  teacher,  usually  in  the  form  of  questions,  group  members  brainstorm  and  respond  back  in  a more 
sophisticated  manner.  The  case  responses  are  not  quantified,  however,  it  is  apparent  to  both  the  professor  and 
the  5th  grade  teacher  that  the  group  responses  to  the  on-line  feedback  demonstrate  expanded  reflection  and 
adaptation  ideas. 

The  implications  for  web  instructional  activities  are  far-reaching.  This  project  shows  that  the 
convenience  and  speed  of  communication  and  collaboration  on  the  web  can  enhance  traditional  teacher  training. 
The  skills  that  student  teachers  continue  to  develop  during  this  ongoing  project  will  contribute  to  the  success  of 
special  education  limited  English  proficient  students  in  the  mainstream. 
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1.  Introduction 

Recently,  there  are  a lot  of  researches  on  real  time  online  teaching.  Until  now,  online  teaching  focused  mainly 
on  communicating  from  teachers  to  learners,  that  is  to  say,  in  one  direction  only.  However,  learners  would  deepen 
their  understanding  by  performing  themselves  experiments  in  the  field  of  science.  Therefore,  we  proposed  a 
scientific  experiments  system  in  which  learners  could  operate  experimental  apparatus  through  the  Web  (Saisho  et 
al.  1998).  Moreover,  we  discussed  issues  in  the  development  of  WWLab  (World  Wide  Laboratory)  for  scientific 
experiments.  We  classified  scientific  experiments  in  the  following  three  groups,  1)  Live  experiments,  2)  Web- 
based  experiments,  3)  Experiments  by  computer  simulation. 

This  paper  describes  the  classification  of  scientific  experiments  and  what  kinds  of  experiments  were  suitable  for 
the  system.  We  have  a plan  to  apply  the  system  for  K4-K8  learners. 

2.  Outline  of  WWLab 

WWLab,  shown  in  Fig.l,  consists  of  a web  server  machine,  a video  camera  and  experimental  apparatus 
including  a computer-controlled  robot  arm.  A learner  can  see  our  Web  page  using  an  Internet  Browser,  such  as 
Netscape  Navigator  and  Internet  Explorer.  On  the  page,  the  learner  can  operate  the  experimental  apparatus  using 
controlling  a robot  arm.  At  the  same  time,  the  learner  can  see  the  state  of  the  experiment  by  the  video  camera.  The 
learner  can  also  control  the  video  camera  on  our  Web  page.  Real  time  computer  display  of  the  experiment  is 
observed  simultaneously  by  other  learners.  And,  the  experimental  process  is  recorded  by  means  of  the  MPEG 
method  on  the  server  computer.  And  then,  learners  can  replay  the  MPEG  movie  file  on  the  Web. 


3.  Advantages  and  aspects  of  WWLab 

We  will  discuss  advantages  and  aspects  of  WWLab  that  we  have  proposed.  The  advantages  of  the  system  are  : 
1 . The  result  of  the  web-based  experiment  is  not  predictable  because  learners  operate  an  experimental 
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apparatus  by  themselves  although  its  operation  is  indirect 

2.  Expensive  experimental  apparatus  is  made  commonly  available. 

3.  Even  if  an  experiment  may  not  go  well,  learners  can  have  insight  into  a subject  through  failure. 

4.  Learners  can  collaborate  with  other  on  an  experiment  through  the  Web. 

5.  The  system  records  experimental  status  in  an  MPEG  file.  Consequently,  learners  can  repeatedly  review  the 
experiment. 

There  are  a few  problems  of  the  system  outlined  below. 

1.  Learners  do  not  have  real  experience  because  they  can  only  operate  experimental  apparatus  through  the 
Web. 

2.  A robot  arm  carries  out  a learner’s  instructions.  Therefore,  this  system  is  not  suitable  for  precise 
experiments  because  the  arm  is  not  able  to  move  with  precision. 

3.  Though  there  are  various  methods  for  an  experiment,  this  system  is  restricted  to  the  experimental  method 
which  we  have  prepared. 

4.  A response  to  a learner’s  instruction  is  delayed  when  the  network  traffic  is  crowded. 

4.  Classification  of  experiments  in  online  teaching 

We  classified  experiments  under  the  following  three  groups  in  order  to  discuss  what  kinds  of  experiments  are 
suitable  for  WWLab. 

1 . Live  experiments 

This  group  includes  experiments  in  which  learners  are  easily  able  to  find  necessary  experimental 

apparatus  in  their  own  environment.  Also,  it  includes  experiments  that  are  not  suitable  via  the  Web  such  as 
those  concerning  smell,  taste  and  tactile  sensation,  as  well  as  experiments  that  are  influenced  by  the 
surroundings  circumstances,  e.g.  temperature  and  humidity. 

2.  Web-based  experiments 

This  group  includes  experiments  in  which  learners  are  not  easily  able  to  get  experimental  apparatus 

because  it  may  be  very  expensive  and  specialized.  Moreover,  it  includes  experiments  that  are  not 
influenced  by  the  surroundings,  e.g.  time,  place,  temperature  and  humidity,  as  well  as  experiments  which 
are  limited  by  the  requirement  of  a place  such  as  a photographic  darkroom  and  so  on. 

There  are  two  kinds  of  web-based  experiment.  These  are  experiments  in  which  the  parameters  are  fixed 
and  experiments  in  which  the  parameters  are  freely  set.  The  former  has  the  advantage  that  learners  can 
easily  conduct  experiments.  The  latter  has  the  advantage  that  learners  conduct  experiments  with  their  own 
ideas.  However,  in  this  case  there  is  the  possibility  of  having  different  results  according  to  whoever 
conducted  the  experiment. 

3.  Experiments  by  computer  simulation 

Suitable  experiments  in  this  group  are  those  which  would  normally  require  much  time  to  conduct. 

Computer  simulation  would  decrease  the  experimental  time.  Also,  it  would  be  suitable  for  experiments 
where  equipment  is  normally  destroyed  and  costs  would  be  very  high.  Moreover,  there  are  other  groups  of 
experiments  apart  from  1 and  2,  in  which  learners  may  wish  to  review  1 and  2. 

5.  Conclusion 

In  this  paper,  we  discussed  issues  in  the  development  of  WWLab.  We  concluded  experiments  that  require  high 
accuracy  and/or  quick  response  are  not  suitable  for  WWLab.  On  the  other  hand,  suitable  are  experiments  that 
learners  can  not  prepare  experimental  apparatus  around  them  because  of  too  expensive  or  special  ones.  We  expect 
learners  to  develop  positive  creativeness  and  collaboration  because  they  can  conduct  experiments  and  have 
communication  with  each  other  through  the  Web.  In  the  future,  we  plan  to  develop  other  kinds  of  scientific 
experiments  system  and  discuss  problems  of  implementation  in  the  system. 

References 

Saisho,  M.,  Tsutsumi,  Y.,  & Matsuno,  R.  (1998),  A Web-based  Remote  Controlled  Scientific  Experiment  System, 
Proceedings  of  WEBNET  98,  831. 


Virtual  Lectures  Free  Students  to  Think  Critically 


Harry  R.  Matthews 
University  of  California, 
Davis,  CA  95616 
hrmatthews@ucdavis.edu 


Introduction 

This  paper  is  based  on  results  from  a pilot  project  now  offered  twice.  The  method  depends  on  reconstructing 
lectures  on  the  World  Wide  Web,  to  free  up  time  for  associated  small  group  discussions.  The  distinguishing 
features  of  this  approach  include  the  use  of  virtual  lectures  to  free  the  small  groups  from  responsibility  for 
content  and  the  use  of  a comprehensive  database  that  addresses  all  aspects  of  offering  a course,  with  a focus  on 
content  organization  and  delivery  (Matthews  1997a,  1991b,  1998). 

Some  university  teachers  espouse  the  lecture  approach  and  some  prefer  the  group  discussion  approach,  of  which 
problem-based  learning  is  a subset.  The  strengths  of  lectures  include  an  organized  approach  to  the  subject 
matter  and  a comprehensive  coverage  of  topics  deemed  appropriate  by  the  instructor.  The  strengths  of 
discussions  include  better  context  and  more  active  student  involvement.  Virtual  lectures,  now  evolving  into  a 
virtual  course,  aim  to  combine  the  advantages  of  both  approaches,  to  improve  higher  education. 

The  prototype  course  comprises  twenty-three  50-minute  lectures  on  Molecular  & Cell  Biology.  The  students, 
about  95  per  class,  are  at  the  beginning  of  their  first  year  in  Medical  School  and  come  with  a very  wide  range  of 
backgrounds.  They  are  intelligent  and  motivated  to  learn  medicine  but  are  not  necessarily  interested  in 
molecular  biology.  The  new  approach  was  evaluated  by  comparing  examination  results  with  historical 
performance  and  by  a detailed  anonymous  student  questionnaire.  The  results  show  that  factual  and  conceptual 
learning  was  at  least  as  effective  as  with  the  traditional  methods  and  that  student  acceptance  was  very  high. 
Students  believed  their  intellectual  skills  improved  but  this  needs  to  be  evaluated  objectively  and  historically. 


Ideas  Behind  the  Course  Design 

Our  students  do  well  on  our  examinations  and  on  those  set  by  the  National  Board  of  Medical  Examiners. 
Nevertheless,  when  they  have  completed  their  two  years  of  basic  science  and  move  to  the  clinical  environment 
they  often  feel  that  their  basic  science  knowledge  is  irrelevant  to  clinical  medicine  and  clinical  instructors 
complain  that  the  students  lack  basic  science  knowledge.  More  likely  is  that  the  students  lack  the  ability  to  take 
material  learned  in  one  context  and  apply  it  in  another  and  to  take  a general  principle  and  apply  it  to  a specific 
situation. 

Many  of  our  students  learn  very  well  visually  and  aurally.  The  computer  has  some  advantages  for  this.  First, 
students  choose  their  own  pace  through  the  material  as  well  as  their  own  time  and  place  for  studying.  Second, 
once  the  transformation  to  the  computer  was  complete  my  time  was  freed  up  to  focus  on  the  development  of  the 
intellectual  skills  that  I feel  our  students  need  badly. 


Computer  Techniques 

The  complete  course  content  is  now  accessed  exclusively  through  a web  browser  running  on  a personal 
computer.  I have  in-person  office  hours  but  students  rarely  use  them.  Students  voluntarily  attend  the  small- 
group  discussions  but  these  are  oriented  towards  specific  clinical  cases  that  may  or  may  not  be  germane  to  the 
topics  in  the  virtual  lectures. 


The  database  integrates  all  aspects  of  a course,  including  course  content,  assignments,  examinations  and  other 
components.  The  structure  allows  an  instructor  to  select  the  parts  (s)he  wants  to  use  and  ignore  the  rest,  which  is 
then  hidden  from  both  instructor  and  student,  until  the  instructor  wants  to  extend  the  application.  The  course 
content  is  both  structured  and  flexible  accommodating  many  different  types  of  multimedia.  The  multimedia 
files  are  distributed  over  additional  Web  servers  and  local  CD-ROM  drives  while  their  display  remains  under 
the  control  of  the  central  database.  This  architecture  provides  good  performance  under  heavy  load  and  over 
standard  telephone  lines. 

The  virtual  lectures  include  animated  graphics,  sound  and  text.  The  format  is  flexible  and  modularized  into  the 
short  topics  so  that  each  lecture  can  be  easily  edited  and  reorganized.  The  text  is  hyperl inked  to  a glossary  and 
the  lectures  are  linked  to  a bank  of  self-test  questions  also  hyperl  inked  to  the  glossary.  Additional  tutorials, 
including  video  modules,  and  advanced  topics,  as  well  as  readings  and  interactive  3-dimensional  molecular 
models  are  integrated  into  the  virtual  course.  Students  have  a personal  notes  page  and  pre formatted  email  for 
asking  questions  by  email.  The  course  is  available  at  http://trc.ucdavis.edu/coursepages/bcm4 1 Oa/. 

The  database  tracks  and  records  key  features  of  each  student's  use  of  the  material  and  records  the  time  and  IP 
number  for  each  logon.  The  time  and  IP  number  permit  detailed  analysis  of  the  server  log  files  and  correlation 
with  individual  logons.  This  information  shows  objectively  which  components  of  the  virtual  course  are  most 
used,  and,  by  comparing  individual  outcomes,  which  components  are  most  effective  as  learning  tools. 


Measured  Results 

Examination  results  from  the  virtual  offerings  of  BCM410A  were  better  than  from  the  face-to-face  offerings 
although  the  students  spent  slightly  less  than  the  allotted  time  on  each  lecture.  There  are  good  textbooks  in  this 
field  including  my  own  (Matthews,  Freedland  & Meisfeld,  1997),  but  the  students  had  an  overwhelming 
preference  for  the  virtual  lectures.  Anonymous  student  evaluations  showed  strong  acceptance  of  the  virtual 
lectures.  The  full  evaluation  results  are  available  at  http://trc.ucdavis.edu/coursepages/bcm41Qa/evaluations/. 


Further  Development 

Three  steps  are  proposed  to  make  the  success  of  the  Web-plus-small -groups  approach  more  widely  available. 
First,  objective  evaluation  tools  will  be  developed  to  exploit  the  rich  data  in  the  database  in  order  to  determine 
how  students  used  the  virtual  materials  and  how  that  relates  to  their  outcomes.  Evaluation  will  also  be  extended 
to  better  assess  development  of  students'  critical  thinking  skills  and  to  include  focus  groups  as  well  as 
anonymous  surveys  and  analysis  of  objective  data.  Evaluation  will  exploit  the  existence  of  a closely  related 
course  that  has  been  taught — more  or  less  unchanged — together  with  the  present  course  to  the  same  students  for 
more  than  6 years,  providing  a control  for  each  individual  class.  Second,  the  system  will  become  "instructor- 
friendly",  by  providing  flexible  Web-based  forms  for  interacting  with  the  database  for  adding  and  organizing 
content  and  other  information  such  as  quizzes,  examinations,  assignments  and  student  data.  Third,  the  program 
will  be  used  with  another  course,  probably  at  the  undergraduate  level. 
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As  the  present  culture  becomes  more  and  more  graphic  in  information  presentation,  education  now  uses 
hypermedia  to  encourage  students  to  acquire  information  and  to  construct  knowledge.  Teachers,  trained  in  the 
traditional  text  methods,  need  to  expand  their  teaching  not  only  to  include  the  new  ways  of  presenting 
information  using  text,  graphics,  sound,  and  animation,  but  also  to  foster  learning  in  a non-linear  environment. 

The  Master  of  Arts  in  Educational  Technology  at  New  Jersey  City  University  has  as  one  of  its  required  core 
courses:  Hypermedia  Across  the  Curriculum.  This  course  up  until  Fall  1998  semester  had  always  been  taught 
in-person,  because  the  many  technological  skills  seemed  too  complicated  to  learn  through  an  asynchronous 
course.  However,  in  the  Fall  1998  semester,  the  course  was  offered  on-line.  This  paper  will:  a)  discuss  the 
differences  between  teaching/leaming  hypermedia  in-person  and  on-line,  b)  preview  the  web  site  of  the  course 
to  show  how  the  hypermedia  technological  skills  were  transferred  to  the  web  medium,  and  c)  assess  the 
learning  through  specific  examples  created  by  the  participants  in  the  course. 

BACKGROUND 

Hypermedia  is  defined  as  multi-media  technology  controlled  by  and/or  delivered  through  the  computer. 
HyperStudio™  was  the  course  software  because  of  its  non-linear  capabilities,  its  availability  on  both  platforms, 
and  it  is  being  used  extensively  in  K-12  classrooms.  The  technical  topics  included  manipulating  graphics, 
sound,  and  text;  cell  and  path  animation;  use  of  video,  CD's,  and  Laserdiscs;  scripting;  and  placing  a stack  on 
the  web  which  required  the  use  of  the  HyperStudio™  plug-in. 

The  students  in  the  class  were  all  educators:  either  teachers  of  a contained  classroom  or  a particular  curriculum 
area  or  computer  coordinators  who  were  responsible  for  more  curriculum  areas  than  just  computer  literacy. 
The  course  required  that  all  technology  be  expressed  within  the  participant’s  discipline  which  meant  that  all 
assignments  were  to  be  developed  using  the  participant's  curriculum. 

All  course  materials  were  available  on  the  web  at  the  beginning  of  the  course,  so  that  more  experienced 
participants  could  move  around  the  course  if  they  wished.  All  communication  was  done  through  private  e-mail 
notes,  public  listserv  notes,  and  through  a public  discussion  forum.  Participants,  placed  in  groups  of  four,  e- 
mailed  their  hypermedia  assignments  each  week  both  to  their  group  and  to  the  instructor.  The  listserv  became  a 
place  for  technical  help  questions.  The  public  discussion  forum  focused  on  different  learning  theories  and/or  the 
impact  of  a particular  technological  skill  (such  as  cell  animation)  on  learning. 

DIFFERENCES  IN  ON-LINE  AND  IN-PERSON  TEACHING/LEARNING 

Because  the  on-line  course  was  a first-time  experiment,  the  differences  discussed  here  are  not  the  results  of  a 
formal  study,  but  are  anecdotal. 

The  first  20  minutes  of  an  in-person  class  are  always  spent  in  a "show-and-tell"  of  the  week's  assignments  with 
each  stack  being  gently  reviewed  by  everyone  in  the  class.  The  on-line  environment  loses  the  class  response  to  a 
stack  and,  what  is  extremely  important,  the  class  experience  of  the  variety  of  stack  design.  However,  the  on- 
line environment,  in  which  the  instructor  becomes  a mentor  to  each  individual  participant  through  personal  e- 
mail  exchanges,  does  result  in  faster  learning:  more  modification  of  stacks,  more  quickly. 
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On  the  other  hand,  non-hierarchical  presentation  of  information  was  learned  more  slowly  by  participants  in  the 
on-line  course.  The  simple  use  of  a menu  card  to  allow  options  needed  to  be  addressed  repeatedly  in  the  on-line 
course,  partly  due  to  the  fact  that  the  web  environment  is  not  as  flexible  as  a HyperStudio™  stack. 

HYPERMEDIA  WEB  SITE 

An  on-line  hypermedia  course  must  demonstrate  on  its  web  site  all  of  the  technological  capabilities  that  the 
software  has.  However,  even  simple  transitions  between  cards  are  not  possible  yet  on  the  web.  Some  of  the 
graphic  and  text  object  manipulations  could  be  done  with  static  gifs,  but  all  path  and  cell  animation  were 
translated  to  animated  gifs.  The  section  on  sound  and  video  used  QuickTime  movie  format.  But  all  of  these 
examples  are  still  presented  linearly,  which  does  not  give  a complete  understanding  of  hypermedia.  So,  at  some 
point  in  the  course,  everyone  had  to  view  completed  stacks  and  hence  use  the  H>^erStudio™  plug-in. 

The  assignment  which  asked  everyone  to  use  the  HyperStudio™  plug-in  was  designed  as  a “win-win”  contest. 
The  stack,  to  be  viewed  with  the  plug-in,  was  about  a common  plant  with  various  clues.  By  identifying  the 
correct  name  of  the  plant,  every  student  could  win,  but  only  if  they  could  view  the  stack!  Although  it  was  a 
course  requirement  to  download,  install,  and  use  the  HyperStudio™  plug-in,  downloading  the  plug-in  from  a 
reputable  commercial  site  was  not  something  that  everyone  wanted  to  do  because  of  past  bad  internet  virus 
experiences.  Installing  and  troubleshooting  the  installation  added  another  level  of  technological  frustration. 

The  most  difficult  technological  assignment  was  the  posting  of  a stack  to  the  internet.  This  involved  learning  to 
"ftp"  stacks  to  an  anonymous  ftp  site,  and  creating  a small  HTML  document.  HyperStudio™  has  a web  page 
converter  command  to  create  the  HTML  code.  However,  since  this  code  produced  by  the  converter  was  not 
complete,  a template  code  was  placed  on  the  web  site  that  the  participant  could  then  copy,  paste,  and  modify. 
But  severe  limitations  exist  with  the  HyperStudio™  plug-in.  First,  all  techniques  in  HyperStudio™  stacks  are 
not  cross-platform,  which  leads  to  stacks  crashing  or  simply  not  functioning  correctly.  Second,  certain 
techniques  are  not  possible  to  view  with  the  plug-in.  Last,  size  is  essential!!!  No  compression  is  available  for 
HyperStudio™.  To  view  a stack  larger  than  500K  real-time  becomes  an  extremely  time-consuming  download! 

ASSESSING  THE  LEARNING 

Technological  skills  are  easy  to  assess.  A participant  would  send  a stack  with  a path  animation,  and  it  could  be 
easily  seen  whether  or  not  the  graphic  did  animate  in  a path.  But,  a course  on  hypermedia  requires  more  than 
just  a cluster  of  technological  skills.  The  course  offers  the  time  to  begin  to  look  at  presenting  information  in 
new  ways.  For  example,  HyperStudio™  has  now  added  the  dimension  of  direct  links  to  the  internet,  which 
allows  a stack  to  become  the  framework  or  context  for  viewing  the  variety  of  information  on  the  internet. 

One  of  the  problems  about  an  on-line  course  is  that  by  not  having  the  class  conversation,  an  on-line  student  can 
move  rapidly  to  fulfill  the  "tech"  part  of  the  course  without  participating  fully  in  the  discussions.  How  then,  is 
such  a student  slowed  down  to  begin  to  question  and  deepen  his/her  thinking  beyond  the  technology?  Not 
easily!  The  main  way  used  in  this  course  was  through  the  instructor’s  e-mail  exchange  with  each  participant,  by 
raising  questions  regarding  the  stack’s  information  designed  to  focus  the  participant  more  fully  on  curriculum 
and  methods  of  information  presentation. 

The  final  requirement  for  the  course  is  to  produce  a curriculum-based  extensive  hypermedia  project  through 
which  the  participant’s  knowledge  and  understanding  of  hypermedia  theory  may  be  assessed.  A comparative 
look  at  projects  developed  in  an  in-person  course  and  in  the  on-line  course  will  be  briefly  previewed  in  this 
presentation. 

CONCLUSION 

In  comparing  the  on-line  participants’  portfolios  to  an  in-person  class’s  portfolios,  the  assignments  and  projects 
demonstrated  quality  similar  to  an  in-person  course.  However,  the  level  of  frustration  learning  the  technological 
skills  at  home  alone  seems  to  be  higher  than  that  in  an  in-person  course.  Also,  the  web  site,  which  followed  a 
previously  designed  syllabus  for  an  in-person  course,  must  be  redesigned  to  incorporate  more  non-linear 
experience.  The  course  focus,  firmly  grounded  in  the  curriculum,  gives  the  course  its  strong  foundation,  but  a 
way  to  break  the  isolated  learning  of  multiple  technological  skills  in  an  on-line  course  must  be  found. 
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1.  Introduction 

This  study  main  objective  is  to  look  for  creative  educational  solutions  to  exploit  Internet  informationai, 
communicative  and  research  resources  to  improve  classroom  teaching  quality.  In  order  to  do  so,  we  had  to 
invest  in  updating  programs  for  our  faculty  and  also  in  a program  that  guarantees  the  development  of 
interactive  and  informational  interfaces  for  classroom  learning  integrated  with  IntemetAVWW.  We  are  taking 
measures  to  improve  the  quality  of  educational  strategies  in  our  institution  based  on  two  projects: 

Act-On-Line  project,  which,  besides  creating  short-term  on-line  courses,  develops  interface  and  interactive 
design  through  Web-activities  to  integrate  them  in  courses  taught  in  the  classroom. 

PADO  project,  which  develops  programs  (courses,  workshops,  didactic-technological  counseling),  aiming  at 
updating  university  teachers  on  the  use  and  exploitation  of  new  technologies  in  higher  education. 

These  two  projects  aim  at  the  following  goals: 

• Invest  in  methodological  research  to  create  distant  learning  environments,  emphasizing  the  exploitation 
of  interactive  tools  for  communication  and  information  to  give  support  to  courses  taught  in  the  classroom. 

• Stimulate  faculty  to  create  educational  projects  exploiting  Internet  resources:  virtual  experimental  labs, 
expert  systems  of  simulation,  data  banks  of  various  formats  (images,  slides,  graphics,  etc.) 

• Enable  transference  of  different  information  through  courses  and  interactive  repositories  available  at 
WWW,  in  which  the  information  being  searched  is  driven  by  activities  started  in  the  classroom. 

• Develop  course  methodologies  and  miscellaneous  assessment  processes,  preferably  exploiting  Internet 
interactive  tools. 

• Practice  formal  and  informal  distant  learning  simultaneously. 

• Assure  participation  and  contribution  of  faculty  members  through  their  ideas  to  exploit  Internet  resources 

• Research  pedagogic  effects  related  to  WWW  instructional  interface 

2.  IntemetAVWW  environment  solutions 

For  over  a year,  we  created  the  Act-On-Line  Project,  which  has  two  main  objectives: 

• Offer  courses  of  recognized  classroom  acceptance  in  on-line  format,  aiming  at  reaching  the  public 
outside  university  campus  in  general.  These  are  short-term  courses,  which  last  three  months,  in  average, 
for  continuous  professional  upgrading  purposes.  Presently  we  are  on  the  Fourth  Edition  of  these  courses, 
working  together  with  an  interdisciplinary  production  team  (Garcia,  M & Maia,  C.  1998). 

• Create  technological  and  didactic  conditions  to  enable  our  professors  to  develop  new  routes  between 
classroom  classes  and  distant  learning  to  get  access  to  different  contents  and  activities  related  to  the 
subject  taught  within  IntemetAVWW  environment. 

Initially,  we  offered  short-term  courses  aiming  at  researching  environments  and  interfaces.  We  presently 
managed  to  implement  two  programs  that  combine  classroom  learning  and  distant  learning  through  Internet. 
We  use  short-term  courses  and  WWW  activities  of  exercise  and  research. 

Standards  for  developing  mini-courses  and  Web-activities  providing  direct  access  interface  and  supporting 
tools  for  navigation  and  learning,  which  enable  page  reading,  guided  research  to  other  pages  outside  the 
course/activity  site,  participation  in  discussion  forums  and  chats,  reposition  of  information  in  conjunction  with 
other  students,  among  others: 

Interaction  tools  - direct  access  to  permanent  line  e-mail,  discussion  forum  opened  to  all  participants, 
teacher’s  voice  and  help  regarding  interaction; 

Learning  tools:  the  course  page  itself,  from  which  the  content  is  accessed,  external  links  for  guided  visit  to 
sites  of  interest,  suggestions  for  reading,  glossary  and  help  with  the  content; 

Navigation  tools:  guide  to  the  content  and  activities  within  the  course; 

Information  tools:  download,  hot-links,  bibliography  and  help  on  how  to  research  in  other  sites  of  interest; 
Educational  contents  shown  in  the  program  are  basically  of  two  natures: 


• The  ones  that  reflect  course  objectives; 

• The  ones  that  enable  self-study 

• The  ones  that  enable  deep  discussion  and  exchange  of  ideas  in  the  classroom. 

Activities  are  prepared  using  multimedia  elements,  such  as  sound  effects,  animated  images,  besides  text 
source.  Another  interesting  aspect  is  the  possibility  offered  to  the  students  to  discuss  their  difficulties  with 
other  participants  in  special  forums,  besides  receiving  answers  via  WWW  page  afterwards. The  pedagogical 
goal  is  trying  to  assure  a previous  reflection  and  construction  of  arguments  based  on  guided  research  to 
accomplish  the  activity. 

3.  Faculty  updating  program  solution  and  steps 

Universidade  Anhembi  Morumbi  faculty-updating  program  has  been  created  in  order  to  assure  that  projects 
aiming  at  improving  educational  strategies  quality  through  the  use  of  new  information  and  communication 
technologies  would  be  implemented. 

Initially,  our  goal  was  to  get  a teacher  who  had  good  performance  in  the  classroom,  take  advantage  of  his/her 
motivation  and  didactic  experience,  and  start  the  first  attempts  of  technological  and  didactic  updating. 

Our  goal  is  to  check  how  faculty  updating  programs  can  be  developed  and  implemented,  aiming  at  general 
and  specific  goals  and  how  to  maintain  and  follow-up  the  teacher  to  help  him/her  achieve  effective  results  in 
the  classroom,  creating  conditions  not  only  for  technological  updating  but  also  for  didactic  and  qualitative 
exploitation  of  WW/lntemet  for  classroom  work.  This  is  an  attempt  to  broaden  the  concept  of  distance 
education  in  WWWAntemet  environment  aimed  at  classroom  work. 

1.  Program  Phase: 

At  this  phase  we  emphasized  the  use  of  Word  to  create  handouts,  exercises  and  tests,  the  use  of  Power  Point 
to  create  presentations,  as  well  as  basic  notions  of  hardware.  Browser  basic  functions  and  fundamentals  to 
surf  and  do  research  in  Internet  have  also  been  explained.  This  step  comprised  24  hours  of  classroom  work. 

2.  Program  Phase: 

Deeper  research  notions,  as  well  as  exploitation  of  Internet  communication  resources.  Basic  principles  to 
create  html  pages,  which  can  be  proposed  in  an  educational  site,  have  also  been  shown.  This  step  comprised 
24  hours  of  classroom  work. 

3.  Program  Phase: 

How  to  create  Web-activities  and  on-line  courses.  The  course  aims  at  preparing  teachers  to  use  Internet  and 
produce  classes,  didactic  contents,  and  activities  for  WWW  environment.  The  course  consists  in  three 
Workshops,  each  one  of  them  working  on  a theme.  First  Workshop,  theme:  computer  mediated  teaching  and 
learning 

• Second  Workshop,  theme:  course  structure  and  on-line  activities. 

• Third  Workshop,  theme:  tools  to  create  on-line  course 

4.  Conclusions 

The  goal  of  creating  interactive  and  effective  learning  environments  is  the  main  purpose  of  these  actions. 
Thus,  it  is  our  task  to  design  educational  products,  within  WWW  context,  involving  data  transmission 
interfaces  and  information  that  lead  to  the  production  of  new  knowledge  from  discussions  and  considerations 
made  in  the  classroom. 

The  way  of  structuring  data  and  information  within  WWW  environment,  aiming  at  knowledge,  is  directly 
related  to  the  methodological  organization  that  must  comprise  content  transmission  resources  and  create 
opportunity  for  interactions  between  learners  and  their  facilitators. 

Thus,  our  pedagogical  model  integrates  educational  interface  development  to  the  human  potential  that  should 
exploit  it.  This  potential  is  made  of  learners  and  facilitators  of  the  process,  who  act  both  virtually,  by 
accessing  lists  and  discussion  forums,  and  in  the  real  world,  when  students  are  in  the  classroom. 

The  advantage  of  this  pedagogical  model  can  be  noticed  by  the  way  through  which  teachers  and  learners 
effectively  interact.  We  believe  that  the  production  of  knowledge  depends  on  people  behind  the  interfaces 
developed. 

Bibliography: 

Garcia,  M & Maia,  C (1998)  Act  On-Line  Project:  Networked  Learning  Practice  Intemet/WWW.  Networked  Lifelong 
Learning.  Proceeding  of  the  J998  International  Conference  held  at  the  University  of  Sheffield^  England. 

Fucks,  H.  et  all.(1998)  Clew  a Cooperative  Learning  Environment  for  the  Web.  Ed-Media  & Ed-TelecomJI998,  10  World 
Conference  on  Educational  Telecommunications.  Volume  2. 


1388 


Using  Computer-based  Visual  Mapping  Tools  to  Enhance  Pedagogical 

Practice. 


Ian  Brown  & Brian  Ferry 
University  of  Wollongong,  NSW  Australia 
ian_brown@uow.edu.au 


Rationale 

Visual  mapping  tools  are  computer-based  and  are  essentially  two  dimensional  and  /or  three  dimensional 
spatial  arrangements  of  text  and  diagrams.  They  have  shown  to  help  pre-service  teachers  to  efficiently  link 
curriculum  content  to  instructional  plans  in  Science  and  language  contexts  (Ferry  1996a,  Ferry,  1996b,  1997; 
Novak  and  Gavin  1984;  White  and  Gunstone  1992).  This  project  attempted  to  enhance  these  findings  by 
extending  the  process  to  another  subject  discipline,  Visual  Arts  education.  It  was  the  researchers'  belief  that 
visual  mapping  tools  could  be  used  with  other  subject  areas  but  difficulties  may  be  experienced  due  to  the 
unique  nature  of  the  particular  subject  discipline.  For  example,  the  reliance  on  visual  representation  in  Visual 
Arts  education  may  affect  the  efficient  use  of  the  tool.  It  was  predicted  that  existing  mapping  tools  may  have 
to  be  redesigned  or  modified  to  enable  for  efficient  use  to  occur.  As  a result  of  this  project  a number  of 
significant  results  and  areas  for  further  study  have  started  to  emerge. 

Aims  of  the  Project/  Areas  of  Investigation 

The  initial  aims  of  the  research  project  was  to  compare  the  use  of  the  computer  mapping  tool  Inspiration®  by 
pre-service  teachers  as  they  plan  instruction  in  Science  and  Visual  Arts  education.  A number  of  broad  areas 
for  exploration  were  developed  and  investigated  by  the  researchers.  For  the  purpose  of  this  paper,  the 
following  specific  questions  will  be  addressed:  Do  computer  based  visual  mapping  tools  assist  or  hinder  pre- 
service teachers  to  organise  curriculum  content  and  knowledge  to  be  effective  instructors?  When  using 
computer  based  visual  mapping  tools  to  construct  knowledge,  do  differences  exist  when  applying  subject 
matter  knowledge  in  Science  and  Visual  Arts  education?  Can  the  creative  process  in  Visual  Arts  education  be 
represented  and  deconstructed  as  a cognitive  framework  in  the  form  of  a visual  map? 

Methodology  and  Data  Collection 

During  the  research  study  twenty  four  volunteer  pre-service  teachers  in  their  final  year  of  elementary  school 
teacher  training  were  invited  to;  i.  compare  the  use  of  the  visual  mapping  tool  Inspiration®  to  plan  instruction 
in  Science  and  Visual  Arts  education  ii.  analyse  and  deconstruct  the  creative  process  with  the  use  of  the  visual 
mapping  tool  Inspiration®  Data  was  collected  through  three  main  sources,  ongoing  written  personal 
reflections,  analysis  of  the  visual  maps  produced  and  interviews.  Interview  transcripts,  reflections  and  visual 
maps  were  coded  and  analysed  for  trends  and  emerging  themes. 

Brief  Description  of  Study 

Students  were  given  four  specific  tasks  to  perform  over  a eight  week  period  which  involved  both  Science  and 
Visual  Arts  education.  They  were:  i.  to  plan  a short  program  of  work  for  an  elementary  class,  using  the  visual 
mapping  tool  package  Inspiration®,  to  conceptualise  ideas  and  discipline  specific  knowledge  ii.  to 
research/investigate  an  artist/artwork,  gathering  as  much  information  as  possible  in  relation  to  lifestyle, 
historical  background,  media  used  and  possible  influences,  iii.  Students  were  required  to  construct  a visual 
map  of  the  information/knowledge  gathered  in  (ii)  conceptually,  using  the  visual  mapping  tool  Inspiration®, 
iv.  plan  the  development  of  a personal  creative  response  (artwork),  using  the  visual  mapping  tool  Inspiration® 
to  conceptualise  thoughts,  ideas,  materials  and  processes.  Students  were  required  to  analyse  their  actions  and 
thoughts,  both  positive  and  negative,  through  personal  written  reflections  and  interviews. 

Preliminary  Results  and  Emerging  Themes 
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As  a result  of  the  analysis  of  preliminary  data  relating  the  areas  of  investigation  (relating  specifically  to  this 
paper)  the  following  can  be  reported:  All  students  agreed  that  the  visual  mapping  tool  package  Inspiration® 
clearly  assisted  them  with  their  organisation  of  curriculum  content  and  knowledge.  The  students  felt 
comfortable  in  a very  short  period  of  time  in  visually  representing  their  knowledge  and  conceptualising  in 
both  subject  disciplines.  They  commented  that  it  provided  a clearer  way  of  thinking,  it  was  more  efficient 
which  allowed  the  students  choice  in  changing  thoughts,  conceptualising  and  recording  their  ideas.  The 
students  unanimously  agreed  that  they  would  use  the  package  again  to  prepare  and  plan  their  work.  They 
believed  that  the  visual  mapping  tools  would  enhance  their  abilities  to  be  more  effective  instructors.  These 
results  are  congruent  with  earlier  studies  conducted  by  Ferry  1996,  1997  and  White  and  Gunstone  1992.  Note: 
a number  of  criticisms/annoyances  were  raised  by  the  students  about  specific  actions  relating  to  navigation 
which  occurred  when  using  the  package.  These  points  will  be  discussed  in  detail  later  in  the  paper. 

From  the  interviews  it  can  be  concluded  that  the  students  felt  that  some  differences  were  evident  between  the 
two  disciplines  when  using  the  visual  mapping  tool  package  Inspiration®.  Students  generally  commented  that 
initially  few  differences  occurred  when  planning  instruction  between  the  two  disciplines,  but  when 
conceptualising  knowledge,  greater  differences  were  apparent.  This  could  be  attributed  to  the  nature  of  the 
task.  In  the  Science  area  students  were  given  a topic  or  concept  from  which  the  knowledge  was  drawn  to 
develop  the  visual  maps.  While  in  one  activity  in  the  Visual  Arts  area,  the  students  had  to  work  in  reverse,  the 
concept  was  provided  (the  artwork  development),  then  categories  were  constructed  and  developed. 

From  the  visual  maps  provided  by  the  pre-service  students  in  this  study  and  the  analysis  of  the  results  it 
appears  that  the  creative  process  can  be  represented  and  deconstructed  as  a cognitive  framework  in  the  form 
of  a visual  map.  It  is  the  researchers'  belief  that  deconstruction  or  analysis  of  an  artwork  is  a common  practice 
in  art  education,  but  deconstruction  using  a visual  mapping  tool  is  a particularly  unique  phenomena.  It  is 
interesting  to  note  from  the  results  of  the  analysis  of  the  visual  maps,  both  the  deconstruction  of  the 
artwork/artist  and  the  construction  of  the  personal  response  (artwork),  is  that  it  appears  that  the  concept 
clusters  developed  by  the  pre-service  students  align  closely  to  the  New  South  Wales  Visual  Arts  Years  7-12 
syllabus  framework  for  conceptual  organisation,  (a  document  that  the  students  used  in  this  study  would  not  be 
familiar  with).  The  conceptual  organisation  evident  in  the  syllabus,  includes  areas  such  as  structural, 
subjective  and  cultural,  which  was  developed  from  postmodernist  and  deconstructionist  theories. 

Conclusion  and  future  directions 

This  paper  has  only  addressed  a small  area  of  the  total  research  project.  Other  areas  presently  under 
investigation  and  proposed  for  future  research  include:  the  relationship  between  learning  styles  and  the 
selection  of  visual  mapping  formats;  the  difficulties  experienced  in  navigation  by  the  pre-service  teachers 
when  using  the  visual  mapping  package,  such  as,  screen  size  and  printing  problems;  the  problems  of 
visualisation  and  the  need  for  a package  which  could  import  visual  or  photographic  images  to  enhance  the 
deconstruction  process;  and  the  issue  of  cross-cultural  applications  when  using  the  packages.  In  conclusion  a 
number  of  significant  findings  have  been  addressed  in  this  research  project.  Students  unanimously  agreed  that 
the  visual  mapping  package  has  enabled  them  to  plan  more  efficiently,  conceptualise  their  knowledge  and 
ideas  more  clearly  and  allowed  them  to  use  the  process  successfully  in  two  subject  disciplines.  As  one  student 
commented  when  using  the  package...  'her  learning  was  active  because  it  made  her  thinking  process  more 
visible  and  more  structured'. 
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In  virtual  learning  environments,  learners  accomplish  different  actions  in  different  media  spaces, 
interacting  with  different  tools  and  documents.  However,  they  need  to  keep  threads  between  their  actions  : 
logical,  structural  and  functional  threads.  Based  on  what  we  know  about  actors,  actions  and  situations  in 
virtual  learning  environments  (Bourdeau  et  al.  1998;  Bourdeau  et  al.,  1997),  we  recognize  a problem  in  the 
transitions  leaners  make  between  different  media  spaces  in  three  typical  situations  : individual,  team  and 
group.  In  an  individual  learning  situation,  actors  interact  with  their  tools  and  documents,  with  individual 
intentions,  goals  and  objectives.  The  types  of  individual  tasks  can  be:  exploration,  consultation, 
production,  playing;  the  basic  interface  is  a multimedia  computer.  In  team-based  situations  (2-6 
individuals),  actors  interact  with  each  other  and  with  tools  and  documents;  they  share  intentions,  goals, 
processes  and  tools.  Their  tasks  can  be:  exploration,  consultation,  production,  playing;  tools  can  be  shared 
applications  and  documents  with  video,  audio  or  text  conferencing.  Group  situations  (6-30  or  more)  are 
different  in  many  aspects:  the  media  space  consists  of  several  sites  equipped  with  videoconferencing 
rooms,  with  various  types  of  computer-based  facilities;  actors  may  interact  via  a floor-taking  protocol  or  via 
an  animator;  their  tasks  can  vary  from  interactive  presentation  to  discussion  or  collective  decision-making. 

In  looking  at  actions  rather  than  discourse,  a central  question  is:  how  can  learners  keep  track  of 
their  actions,  processes  and  products?  In  various  worlds  that  have  different  rules,  how  can  learners  find  the 
threads  they  need  in  order  to  achieve  their  goals?  What  happens  to  the  concept  of  seamlessness,  considered 
as  essential  in  human-computer  interface  design,  when  the  media  space  is  a room?  How  does  the  impact  of 
tele-presence  vary  in  the  three  different  situations,  from  inter-individual  text-based  asynchronous 
communication  to  videoconferencing  rooms  for  group  interactions?  If  changing  from  one  media  space  to 
another  one  presents  obstacles  to  learning,  or  a loss  of  time  and  attention,  should  instructional  designers 
recommend  that,  for  each  learning  scenario,  learners  remain  in  the  same  virtual  universe?  Should  designers 
search  for  new  design  principles  in  order  to  solve  or  to  reduce  this  problem?  How  can  pedagogical  agents 
be  designed  to  intelligently  support  the  learning  process,  when  learners  going  through  these  transitions? 

Given  three  learning  situations  (I,  T,  G),  a set  of  learning  tasks,  and  a set  of  technologies, 
transition  needs  and  problems  can  be  identified,  as  well  as  potential  solutions.  As  an  example,  it  appears 
that  Learner  II,  when  involved  in  the  task  of  exploring,  with  a navigator,  needs  a transition  process  to  Team 
T1  for  referring,  asking  questions,  displaying,  reporting,  or  receiving  feedback  on  his/her  learning  process 
or  product.  Transition  problems  can  be  the  lack  of  transferring,  of  conversation,  or  of  sharing  facility.  In 
another  example,  Tl,  when  reporting  to  the  group,  wants  to  refer  to  interactions  they  have  or  had  with  T2, 
T1  may  encounter  transition  problems  such  as  lack  of  display  or  recording  facilities;  they  can  also  face  a 
problem  of  technological  overload  if  they  try  to  work  with  different  media  spaces  at  the  same  time. 

Results  from  previous  studies  indicate  transition  problems  when  students  move  from  one  situation 
to  another  and  change  spaces  at  the  same  time.  In  the  case  of  a Web-based  business  simulation,  students 
shared  information  and  tools  for  their  teamwork  within  the  simulation,  but  found  themselves  away  from 
this  space  when  meeting  as  a group.  Coordination  analysis  pointed  to  critical  interdependencies  between 
actors  and  resources,  where  transition  could  be  supported  (Bourdeau  & Wasson,  1998).  In  a 
videoconferencing-based  group  situation,  students  enjoyed  a strong  tele-presence  through  shared  time  and 
multimedia  group  space,  but  with  no  transition  for  individual  or  team  situations  once  the 
videocommunication  was  cut  off  (Bourdeau  et  al.,  1998).  These  problems  can  be  anticipated  and  avoided 
in  a design  process  that  respected  principles  specific  to  these  situations. 

Coordination  theory’s  framework,  adapted  to  educational  situations  through  analysis  of 
interdependencies  among  actors  and  among  resources  (Wasson,  1999)  provides  a framework  to  identity 
potential  coordination  problems.  Bates  (Bourdeau  & Bates,  1996)  and  Collins  (1999)  provide 
classifications  for  new  media  based  on  their  attributes.  These  frameworks  can  be  used  in  the  design  process 
in  order  to  locate  potential  transition  problems  in  the  design.  Among  transition  problems,  some  will  soon 
find  a solution  through  technological  integration  in  network-based  environments.  In  other  cases,  bridging 
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processes  (capturing,  recording,  scanning,  reporting,  projecting),  or  gateways  can  be  built  into  the  learning 
scenarios.  Concepts  such  as  seamless,  transparent,  and  invisible,  should  guide  the  design  for  learning 
situations  so  that  technology  is  not  (too)  intrusive. 

In  an  attempt  to  find  a solution  to  this  problem,  we  suggest  a design  process  in  seven  steps  : 1) 
Analysis  of  the  three  situations,  2)  Design  of  learning  scenarios,  3)  Modelling,  4)  Virtual  experimentation, 
5)  Field-testing,  6)  Extraction  of  design  principles,  7)  Validation. 

An  analysis  of  the  three  basic  situations  (individual,  team,  group)  is  conducted  while  studying 
their  components  and  their  features  : space  (physical,  virtual);  channels  (audio,  video);  time  (synchronous, 
asynchronous,  quasi-synchronous),  duration  and  rhythm;  interactivity;  needs,  potential  and  limits  of  each 
situation  for  transitions  with  the  two  others. 

The  design  of  learning  scenarios  involving  the  threefold  siUiations  (individual,  team,  group)  gives 
specifications  for  the  learning  activities  with  their  spatio-temporal  dimensions. 

Modelling  is  the  activity  by  which  the  scenarios  are  dynamically  represented  with  a modelling  tool 
(Wasson  & Bourdeau,  1998). 

Virtual  experimentation  is  conducted  by  manipulating  variables  in  the  scenarios  and  assessing  the 
impact  of  the  variations,  using  the  modelling  tool. 

Field-testing  represents  a partial  implementation  of  the  scenario  with  a limited  number  of  subjects 
and  an  observation  protocol. 

Extraction  of  design  principles  is  oriented  toward  the  design  of:  a)  learning  scenarios,  b) 
technological  environments,  c)  pedagogical  agents. 

Validation  of  these  principles  is  done  by  a full  implementation  of  a collection  of  scenarios  with  a 
data  collecting  protocol. 

As  a conclusion,  a design  problem  has  been  identified  and  briefly  described.  A process  in  seven 
steps  has  been  proposed  in  order  to  better  understand  the  problem  and  its  dimensions,  and  to  propose 
directions  for  the  design  of  learning  scenarios,  technological  environments  and  pedagogical  agents.  The 
concepts  of  telepresence  and  of  seamlesness  receive  special  attention  in  this  approach,  as  well  as  the 
consciousness  of  time  and  space  by  learners.  This  view  underlines  the  necessity  of  analysis  (using 
frameworks  and  modelling),  and  of  observations  (through  virtual  and  real  tryouts)  when  designing  learning 
scenarios,  in  order  to  support  the  continuity  needed  in  the  learning  process. 
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The  Special  Problems  of  Computer-isased  i eaching 

Academia,  both  in  the  U.S.  and  abroad,  is  increasingly  adopting  new  instructional  delivery  methods.  Limited 
experience  to-date  shows  that  attrition  rates  for  computer-based  courses  often  runs  higher  than  for  traditional  courses. 
Adopting  savvy  strategies  can  offset  this  handicap  and  significantly  enhance  student  satisfaction  and  retention.  The 
secret  of  retention  in  this  new  world  is  to  help  students  feel  and  be  involvedB-with  faculty,  with  classmates,  and  with 
the  institution. 

This  year  I taught  three  consecutive  terms  of  an  online  introductory  interpersonal  communications  class,  I 
found  maintaining  high  levels  of  student  participation  and  retention  difficult  but  important,  especially  in  the  mediated 
instructional  setting. 

Because  distant  students=  primary  concern  is  losing  interactivity  with  the  faculty  or  other  students,  the  most  important 
adaptations  make  the  learning  experience  more  personal  and  less  A mediated  ,= 

I . Start  with  a good  foundation 


Require  new  student  orientation.  If  students  can=t  log  onto  the  World  Wide  Web,  use  e-mail 
or  post  assignments,  they  will  be  handicapped  throughout  the  course.  A grounding  in  Internet  basics 
assures  all  students  have  the  minimum  skills  for  success. 


Build  in  redundancy.  Not  everyone  follows  the  same  Alogical=  paths  to  information.  With 
hyperlinks  the  Internet  provides  a powerful  tool  for  building  a non-linear,  multi-stream  logical  structure. 
For  example,  hyperlink  the  grading  scale  to  the  syllabus  page,  the  assignment  page,  and  the  grade  page 
to  increase  the  odds  of  students  locating  key  information. 

Include  detailed  course  information.  Online  syllabi  should  provide  all  the  information  usually 
given  in  a traditional  syllabus  plus  everything  you  would  have  said  in  classroom  instructions.  Instructor 
expectations  should  be  especially  clear  to  foster  better  communication  and  higher  performance. 

Schedule  multiple  assignments.  Rather  than  a single  large  paper/project/exam  due  at  term  end, 
design  multiple  small  assignments  or  phases  of  a larger  assignment  due  periodically  during  the  term.  This 
keeps  student  involvement  constant,  and  allows  you  to  identify  immediately  when  a student  is  falling 
behind. 

Hyperlink  appropriate  Internet  resources.  Use  the  immediacy  and  breadth  of  information 
available  on  the  net  to  enrich  your  traditional  course  materials. 

Do  a complete  dry  run.  After  designing  your  online  course,  go  through  everything, 
personally.  Log  on  as  a new  student,  access  the  syllabus,  post  assignments,  take  exams  and  enter  the  chat 
room  — try  everything.  Then  ask  another  person,  with  no  familiarity  with  the  course  to  do  the  same. 
This  naive  subject,  like  a cold  reader,  can  often  identify  otherwise  invisible  problems. 


Help  students  feel  comfortable.  During  the  first  week  introduce  yourself,  and  invite  students 
to  introduce  themselves.  You  might  also  post  your  photo,  and  invite  students  to  do  so. 


Send  personal  e-mail.  Show  the  students  that  you  are  there  and  care  about  them.  Send 
answers  to  questions,  comments  on  individual  assignments,  or  just  contact  students  who  have  been 


2.  Develop  integrated  course  curriculum. 


3.  Create  a friendly  environment 


ERIC 


suspiciously  quiet.  Even  if  you  can=t  respond  immediately,  always  acknowledge  a message  or  assignment 
received.  Many  systems  automatically  generate  a return  response.  AGot  your  message  full  response 
follows.=  This  lets  students  know  their  messages  aren=t  lost  in  cyberspace. 

Be  accessible.  Log-on  frequently.  More  frequent  log-ons,  even  if  shorter,  give  a greater  sense 
of  faculty  presence  than  do  less  frequent,  but  longer,  log-ons.  In  addition  to  regular  e-mail  contacts, 
identify  real-time  office  hours  so  students  can  telephone  or  meet  you  online. 

Include  electronic  participation  in  grading  criteria.  Both  personal  experience  and  literature 
reviews  suggest  that  the  more  frequently  and  actively  students  participate,  the  better  their  retention  and 
learning.  Encourage  student  involvement  by  making  it  a part  of  their  grade,  or  by  requiring  a certain 
number  and  quality  of  interactions  per  week. 

4.  Treat  students  as  active  learners 

Facilitate  learning.  Most  online  students  are  working  adults,  use  to  taking  responsibility  for 
themselves.  Take  advantage  of  this  maturity  by  involving  students  as  much  as  the  first  shot  at  answering 
each  othei^s  questions  before  you  jump  in  as  the  Aexpert.=  Structure  assignments  requiring  student 
collaboration,  assign  students  to  create  exam  questions,  or  have  them  evaluate  each  other=s  work,  etc. 

Give  substantial  feedback.  Students  need  full,  concrete  feedback  on  how  they=re  doing, 
especially  with  initial  assignments.  Explicit  examples  of  what  they=ve  done  right  and/or  wrong  are  more 
helpful  than  general  comments  like  AGood  job!= 


>Unbundling=  Instructional  Tasks 

Academia  already  Aunbundles=some  traditional  instructional  activities,  shifting  them  from  overworked  and 
costly  faculty  to  skilled  helpers.  Teaching  assistants  grade,  academic  advisors  counsel,  and  departmental  secretaries 
post  grades.  Online  teaching  is  even  more  suited  to  unbundling. 

Technological  complexities  and  system  glitches  are  constants  in  mediated  teaching,  and  demand  a Ahand 
holdings  function.  Students  don=t  see  faculty  in  class,  so  they  must  find  answers  already  posted,  or  contact  someone 
to  get  the  information.  Even  if  you  post  and  students  are  able  to  find  a detailed  syllabus,  questions  inherent  in  the 
technological  medium  will  still  arise.  AIs  that  hyperlink  dead?=  AHow  long  will  the  system  be  down?=  AWhy  doesn=t 
my  password  work  anymore?=  require  a personalized  answer. 

A few  weeks  into  the  quarter,  I realized  I had  de  facto  unbundled  the  course.  I continued  the  traditional 
instruction  tasks,  but  my  secretary  became  the  class  manager.  Available  by  phone,  fax  or  mail  during  regular  business 
hours,  she  handled  day-to-day  class  operations,  answered  procedural  questions,  walked  students  through  processes, 
checked  on  system  glitches,  and  posted  grades  online.  She  also  monitored  student  log-on  frequencies.  If  a student  went 
inactive  for  a specified  period,  she  alerted  me  so  I could  make  special  contact  to  identify  problems  and  encourage 
reactivation. 


Conclusion 

The  demand  for  higher  education  continues  to  grow  exponentially  throughout  the  world.  Already,  mega- 
universities in  China  and  India  enroll  as  many  as  250,000  students,  many  taught  via  distance  education.  The  rapidly 
changing  face  of  education  brings  new  opportunities  but  also  the  demand  for  innovative  solutions.  Teaching  online 
presents  additional  challenges,  but  a creative,  problem-solving  attitude  and  artful  strategies  make  it  rewarding  for  both 
faculty  and  students. 
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Introduction 

DUKES  is  a computer  application  that  we  have  created  to  allow  developers  of  educational  multimedia  to 
produce  applications  that  include  real-time  variable-speed  speech.  Variable-speed  speech  is  speech  that  is  played 
back  in  more  time  (called  time-expanded)  or  less  time  (called  time-compressed)  than  it  was  originally  recorded. 
For  example,  a one  minute  speech  segment  played  back  in  40  seconds  is  time-compressed  by  33%.  Research  has 
shown  that  learners  can  listen  to  speech  that  has  been  time-compressed  by  up  to  50%  without  any  loss  of 
comprehension  or  retention  (Harrigan  1995;  Harrigan  & McLean  1995).  People  normally  speak  in  the  range  of 
125-175  words  per  minute  (wpm)  and  silently  read  at  275-300  wpm.  Thus,  when  listening  to  time-compressed 
speech  which  has  been  compressed  by  up  to  50%  learner's  are  listening  at  approximately  the  same  rate  at  which 
they  would  silently  read.  Listening  to  time-compressed  speech  can  produce  a significant  increase  in  learning 
efficiency  (learning  per  unit  of  time)  compared  to  listening  at  normal  speed,  without  any  known  negative  side 
effects. 


Producing  Variable-Speed  Speech 

There  are  various  methods  of  producing  variable-speed  speech.  Learners  do  not  like  the  "Mickey  Mouse" 
effect  that  is  produced  when  speech  is  time-compressed  by  simply  playing  it  fasterr,  so  researchers  have  create 
various  other  methods  of  time-compressing  speech  without  a significant  pitch  change.  Some  of  the  most  often 
cited  methods  are:  (a)  removing  pauses  in  the  speech,  (b)  removing  small  chunks  of  the  speech  at  regular  intervals 
in  time,  and  (c)  using  the  Synchronized  Overlap  and  Add  (SOLA)  (Roucos  & Wilgus  1985)  technique.  In  the 
implementation  of  DUKES  we  have  used  the  SOLA  technique  because  it  produces  the  best  quality  speech  of  all  the 
methods  (Arons  1994). 

The  SOLA  method  is  a selective  sampling  method.  With  this  method,  the  beginning  of  a speech  segment 
is  overlapped  and  averaged  with  the  previous  speech  segment  at  the  position  of  highest  cross-correlation.  The 
resulting  time-compressed  file  is  of  very  good  quality,  and  there  is  no  significant  change  in  pitch.  The  SOLA 
method  requires  a large  number  of  multiplications  and  additions  in  the  calculation  of  the  point  of  highest  cross- 
correlation but  our  implementation  has  been  optimized  to  run  in  real-time  on  a Pentium  90MHz  or  faster 
computer. 

Before  we  developed  DUKES,  we  used  Wave  for  Windows  (Turtle  Beach  Systems  1992)  to  create 
educational  multimedia  applications  in  which  we  included  multiple  versions  of  the  same  speech  file  which  was 
time-compressed  by  various  amounts  such  as  15%,  30%,  and  45%.  We  then  gave  the  learner  the  ability  to  change 
speech  files  at  any  time.  This  worked  quite  well  (Harrigan  1995)  but  it  was  limited  in  two  ways:  (a)  multiple 
speech  files  consume  an  enormous  amount  of  disk  space  and  (b)  the  learner  could  only  select  the  time-compression 
percentages  that  we  provided  rather  then  being  able  to  select  their  own.  We  developed  DUKES  so  that  we  could 
give  the  learners  more  control  by  allowing  them  to  control  the  speed  of  playback  in  real-time.  To  date  we  have 
only  concentrated  on  speeded-up  speech  but  our  application  has  the  ability  to  slow  down  speech  as  well. 
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The  DUKES  Software 


Figure  1 shows  screen  images  of  the  only  two  screens  in  DUKES  (note  that  DUKES  could  be  imbedded 
within  an  application  and  thus  the  learner  may  not  see  this  interface).  It  shows  that  the  input  can  be  from  a sound 
file  or  in  real-time  via  a sound  card.  Our  application  determines  the  characteristics  of  the  input  signal.  The  user  is 
given  control  over  the  default  parameters  that  control  the  quality  of  the  resulting  signal.  The  resulting  speech  can 
be  saved  as  a sound  file  or  played  on  the  sound  card,  whichever  is  appropriate.  When  the  user  selects  "Do  TSM"  to 
do  the  conversion  (known  technically  as  Time  Scale  Modification)  a second  screen  appears  to  give  the  user  the 
ability  to  change  the  speed  of  the  speech  as  the  speech  is  playing. 
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Figure  1.  Screen  shots  of  the  only  two  screens  in  the  DUKES  software  for  creating  variable-speed  speech. 

Conclusion 

We  have  used  time-compressed  speech  in  several  educational  multimedia  applications  and  have  received 
no  negative  feedback  from  our  learners.  We  feel  that  variable-speed  speech  is  a feature  that  designers  of 
educational  multimedia  applications  should  consider  using  in  the  products  that  they  produce.  Our  software  is 
available  as  freeware  from  the  author. 
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Key  to  the  growth  and  success  of  distance  education  as  an  alternative  delivery  system  for  university  courses  is 
faculty  confidence  in  the  effectiveness  of  distance  education  systems  for  the  delivery  of  curriculum,  and  student 
satisfaction  with  the  learning  experience  in  a distance  education  environment.  Yet,  too  often,  the  planning 
process  is  managed  by  persons  who  have  little  or  no  experience  in  teaching  in  a distance  environment.  And  the 
technology  issues  are  often  so  daunting  that  they  become  the  dominant  concerns  of  all  involved.  Lost  in  the 
discussion  are  the  needs  of  the  faculty  and  the  students.  These  needs  have  special  relevance  for  the  persons 
serving  as  the  link  between  the  students  and  faculty  - the  on-site  administrators  of  remote  site  facilities. 

The  University  Center  of  North  Harris  Montgomery  Community  College  District  (NHMCCD)  is  a collaborative 
partnership  of  the  community  college  district  with  six  publicly  funded  universities:  Prairie  View  A&M 

University,  Sam  Houston  State  University,  Texas  A&M  University,  Texas  Southern  University,  University  of 
Houston  and  University  of  Houston-Downtovm.  NHMCCD  covers  the  northern  one-third  of  the  Houston 
metropolitan  area.  Residing  within  the  district’s  boundaries,  which  cover  more  than  1400  square  miles,  are  1.2 
million  people.  No  university  is  located  within  the  district’s  boundaries.  NHMCCD  took  the  lead  in 
developing  The  University  Center  (TUC)  to  improve  access  to  higher  education  for  its  graduates,  residents,  and 
businesses.  The  six  university  partners  deliver  all  required  upper  division  and  graduate  courses  to  support  30 
undergraduate  and  23  masters  level  programs.  In  some  instances,  entire  major  programs  are  being  delivered 
using  compressed  video  interactive  classes  originating  at  the  main  campus  location.  Some  programs  are 
delivered  using  a combination  of  live-on  site,  compressed  video,  tape,  and  Internet. 

Approximately  170  sections  of  upper  level  and  graduate  courses  are  offered  each  long  semester  at  The 
University  Center.  Of  these,  approximately  40%  utilize  compressed  video  interactive  technology  for  delivery  to 
multiple  sites.  Four  of  the  six  partners  utilize  the  compressed  video  technology.  In  most  cases,  TUC  is  the 
“receive”  site.  However  each  semester  anywhere  from  4 to  8 sections  will  originate  on  a regular  basis  from 
TUC.  And  in  approximately  half  the  remaining  sections,  the  faculty  member  will  travel  to  TUC  at  least  once 
during  the  semester  to  broadcast  the  course  from  there.  Thus,  the  primary  focus  of  concern  for  The  University 
Center  is  providing  adequate  support  to  facilitate  delivery  of  compressed  video  interactive  classes. 

Because  The  University  Center  draws  from  several  universities  with  different  policies  and  processes,  it  provides 
an  excellent  laboratory  for  gathering  information  and  testing  support  strategies  to  achieve  a high  level  of  student 
and  faculty  satisfaction  with  distance  education.  This  paper  presents  the  results  of  two  surveys.  The  first  was 
directed  to  faculty  to  identify  their  expectations  related  to  types  of  support  that  they  perceive  necessary  for  their 
distance  education  courses  to  be  successful.  The  second  was  directed  at  students  to  determine  their  level  of 
satisfaction  with  the  compressed  video  classes  at  The  University  Center.  The  surveys  are  part  of  an  annual 
assessment  process  used  to  review  current  practices  at  The  University  Center  and  to  initiate  change  as 
appropriate. 

A brief  summary  of  current  classroom  operation  provides  the  framework  for  understanding  the  information 
gleaned  from  the  surveys.  In  the  typical  "receive"  classroom  situation,  a member  of  the  student  services  office 
meets  the  class  the  first  day  prior  to  the  beginning  of  class.  She  provides  a brief  explanation  of  using  the  push- 
to-talk  microphones  and  describes  to  the  students  the  process  for  calling  for  technical  assistance  using  the 
telephone  in  the  classroom  to  page  a technician.  She  hands  out  the  syllabus  for  the  class  and  any  other  material 
the  faculty  member  has  sent.  Additionally,  she  demonstrates  to  students  the  procedures  for  redialing  a class  in 
the  event  of  disruption. 


For  daily  class,  a technician  connects  the  class  approximately  15  minutes  prior  to  the  start  time  and  tests  the 
audio  as  well  as  video  transmission.  Students  arrive  and  take  their  seats  just  as  for  a live-onsite  class.  A student 
services  staff  person  brings  any  handouts  to  the  classroom  and  collects  any  papers  which  she  has  been  notified 
by  the  faculty  member  to  collect.  Students  are  unattended  during  the  class  period.  Exams  are  proctored  by  a 
contract  proctor,  if  requested  by  the  faculty  member.  When  only  a couple  of  students  are  at  the  "receive"  site, 
the  faculty  member  will  often  proctor  using  the  connection.  In  those  cases,  a student  services  staff  member 
distributes  the  exams  at  the  beginning  of  the  class  period  and  then  returns  at  the  end  of  the  period  to  collect  the 
exams.  Students  who  leave  early  place  their  exams  on  the  desk  if  they  do  not  want  to  remain  until  the  staff 
person  returns.  If  the  course  requires  a class  presentation,  then  the  student  services  support  staff  provides 
broadcast  training  for  the  students,  demonstrating  to  them  the  process  of  changing  from  camera  to  powerpoint 
to  document  camera.  Students  are  allowed  access  to  the  classroom  for  practice.  At  this  time,  only  one  college 
from  a partner  university  utilizes  web-based  classroom  management  software  to  support  delivery  of  its  courses. 

Overall,  the  types  of  support  that  faculty  perceived  to  be  important  for  their  compressed  video  courses  to  be 
successful  were  directly  reflected  in  the  students'  level  of  satisfaction  with  the  course.  The  two  areas  of  greatest 
convergence  centered  on  technology  and  getting  information/assignments/tests  to  and  from  students.  Faculty 
expected  the  technology  to  operate  in  a trouble-free  manner,  and  for  technical  assistance  to  be  quickly  available. 
Students  correspondingly,  found  interruptions  in  the  broadcast,  and  inferior  quality  in  such  things  as  audio,  to 
affect  how  they  felt  about  the  course.  Where  there  were  continual  technical  problems,  students  were  less 
satisfied  and  expressed  concern  about  taking  another  course  delivered  via  compressed  video.  In  response  to  this 
finding.  The  University  Center  technical  staff  instituted  a room  log  to  track  connection  problems.  This 
procedure  will  help  to  quickly  distinguish  general  problems  from  those  that  are  specific  to  connecting  with  a 
particular  institution.  Additionally,  to  speed  troubleshooting,  the  staff  developed  a "call  a tech"  procedure  using 
the  alphanumeric  paging  system  through  the  computer  to  page  the  technical  person  on  duty.  By  following  the 
instructions  taped  to  the  desk  by  the  computer  keyboard  (just  in  time  instructions)  the  student  can  send  a page 
with  the  room  number  and  brief  description  of  the  problem.  That  information  can  help  the  technical  staff 
determine  quickly  the  magnitude  of  the  problem  and  level  of  response  necessary. 

Faculty  want  and  need  to  get  information/assignments/tests  to  and  from  students  in  a timely  manner  to  derive 
maximum  pedagogical  value  from  the  activity.  And  students  indicated  on  their  surveys  a high  level  of 
frustration  at  the  amount  of  time  it  often  took  to  receive  back  graded  work.  The  highest  level  of  dissatisfaction 
existed  where  the  universities  were  using  an  informal  courier  system  (i.e.  whoever  happened  to  be  headed  to  the 
vicinity  of  TUC)  to  transport  materials.  In  that  case,  faculty  felt  they  were  not  being  supported  because  it  would 
sometimes  take  more  than  a week  for  materials  to  reach  the  students.  And  the  students  were  dissatisfied 
because  they  didn't  know  how  they  were  doing  in  the  course  or  felt  they  were  at  a disadvantage  when  compared 
to  the  students  who  were  physically  with  the  professor  at  each  class.  In  those  cases  where  overnight  express 
mail  service  was  used  to  send  materials  both  parties  were  somewhat  more  satisfied.  The  problem  was 
significantly  diminished  in  the  case  of  the  partner  university  using  course  management  software  to 
communicate  the  syllabus  and  transmit  assignments.  In  response  to  this  finding,  whenever  possible  TUC  now 
utilizes  express  mail  to  send  to  the  faculty  member  all  class  materials  collected.  Additionally,  the  TUC  staff  is 
evaluating  the  feasibility  of  providing  access  to  course  management  software  through  the  TUC  server  for  those 
faculty  who  do  not  have  such  access  at  their  universities.  The  information  gathered  in  this  section  of  the  survey 
demonstrates  the  value  of  systematic  assessment  in  identifying  needs  and  devising  solutions.  By  pinpointing 
the  relationship  between  high  satisfaction  and  the  efficient  transporting  of  materials  between  students  and 
faculty,  TUC  was  able  to  justify  the  budget  expenditures  related  to  using  express  mail  and  to  provide  incentive 
to  its  partners  to  seriously  consider  mechanisms  for  web  support  for  classes. 

The  U.S.  Department  of  Education  has  reported  that  more  than  50  percent  of  American  colleges  and  universities 
offer  or  expect  to  launch  distance-education  programs  in  the  near  future.  If  distance  education  is  to  prove  a 
successful  system  for  bringing  education  to  the  people  then  we  must  move  from  our  preoccupation  with 
technology  and  address  the  nuts  and  bolts  issues  of  support.  The  information  derived  from  systematic 
surveying  of  the  parties  most  affected  by  the  technology  - the  faculty  and  the  students  - provides  a basis  for 
beginning  to  address  this  issue. 


1398 


Early  Experiences  in  Broadening  the  Use  of  Web-Based  Learning  to 

Mainstream  Faculty 
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Dr.  William  Btimley,  D^ital  Media  Projects  Office,  Ryerson  Polytechnic  University 
Dr.  Rheta  Rosen,  Leamir^  and  Teaching  Office,  Ryerson  Polytechnic  University. 


The  history  of  instmctional  technology  is  littered  with  discarded  technological  innovations  that  promised  to 
revolutionize  the  way  teaching  and  learning  occurred.  This  paper  will  describe  the  efforts  of  the  st:^ff  of  the 
D^ital  Media  Projects  Office  at  Ryerson  Polytechnic  University  in  introducing  an  integrated  web- based  course 
delivery  packs^e  called  WebCT  to  Ryerson  faculty  and  students.  The  goal  was  to  provide  and  support  a 
software  package  that  would  enable  faculty  considered  to  be  non-technical  to  use  the  web  for  instmetion, 
extending  the  reach  of  this  instmctional  technology  beyond  that  of  the  typical  innovators  and  early  adopters.  In 
this  paper  we  will  report  on  our  efforts  to  ensure  that  these  web- based  tools  do  not  ultimately  contribute  to  the 
list  of  discarded  instmctional  technologies.  Specifically  we  will  reflea  on  the  techniques  used  to  promote  and 
support  WebCT,  our  experiences  on  the  successes  to-date  and  suggestions  on  howto  continue  to  reach  out  to 
faculty  not  already  using  web- based  tools  for  instmetion 

Ryerson  Polytechnic  University  is  an  undergraduate  university  in  the  heart  of  downtown  Toronto. 
Full-time  and  part-time  student  programs  are  divided  into  five  faculties  covering  37  program  areas.  Faculties 
include  Applied  Arts,  Arts,  Business,  Community  Services,  and  Engineering  and  Applied  Science.  In  addition 
to  14,000  day  students,  there  are  65,000  student  enrollments  in  continuing  education  courses  (both  on  campus 
and  throi^h  distance  education). 

In  1996  the  D^ital  Media  Projeas  Office  (DMP)  was  established  to  provide  support  for  faculty  with 
an  interest  in  using  multimedia  for  instmetion.  The  DMP  provides  a range  of  services  for  faculty  including, 
workshops,  drop-in  multimedia  suites,  and  consultation.  Faculty  receive  support  in  presentation  technologies 
such  as  PowerPoint,  web  authoring,  and  advanced  multimedia  authoring  (Btimley,  1998). 

The  orientation  of  the  DMP  is  to  provide  support  for  faculty  with  the  goal  of  helping  them  to 
develop  the  skills  to  create  their  own  multimedia  resources.  This  approach  has  been  very  successful  for  a small 
number  of  faculty  that  could  be  classified  as  innovators  and  early  adopters  according  to  criteria  suggested  by 
Moore  (Moore,  1991).  The  DMP  workshops  are  typically  attended  by  faculty  willing  to  learn  to  use  technology 
that  is  rapidly  changing,  that  may  be  in  a baa  or  development  phase,  or  that  requires  a significant  time 
commitment. 

In  the  summer  and  fall  of  1998  the  DMP  introduced  WebCT,  an  integrated  web- based  course  delivery 
software  package.  WebCT  is  just  one  of  many  new  like  products  becoming  available  to  higher  education 
institutions  in  the  past  few  years.  These  integrated  packages  combine  common  internet- based  tools  such  as 
threaded  discussion  groups,  real-time  chat,  web  page  delivery,  online  quizzing  and  student  management  in  one 
package.  The  instmetor  can  mount  and  maintain  a fully  functioning  web- based  course  with  litde  or  no  technical 
knoT^vdedge  of  the  intricacies  of  web  servers.  Javascript,  or  Peri,  etc. 

The  software  was  introduced  to  provide  an  accessible  set  of  tools  for  web- based  instmetion.  The  goal 
in  introducing  WebCT  was  to  continue  the  efforts  of  the  DMP  to  assist  faculty  in  becoming  self-sufficient 
when  using  technology.  An  effort  was  made  to  make  web- based  tools  accessible  to  a group  of  faculty  as  yet 
reluaant  to  use  the  web  for  teaching.  This  group  of  faculty  is  classified  as  the  early  and  late  majority  or  the 
mainstream  by  Moore  (Moore,  1991). 

In  his  paper  "What  Ever  Happened  to  Instmaional  Technology^',  Geoghagen  (Geoghagen,  1994) 
suggests  that  mainstream  facultyJiffer  from  their  innovator/early  adopters  colleagues  in  their  reasons  for 
adopting  any  new  technology.  Primary  among  these  reasons  is  a general  interest  in  the  technology  itself.  While 
innovators  and  early  adopters  are  "techies"  willing  to  explore  the  potential  of  the  technology,  the  mainstream  is 
not.  Faculty  falling  into  the  mainstream  are  interested  in  seeing  tried  and  tested  applications  and  do  not  want  to 
use  technology  that  is  not  congment  with  the  way  they  carry  out  their  daily  tasks. 

As  well,  different  methods  are  required  to  promote  and  support  new  technology  for  the  different 
groups  of  adopters.  The  standard  strategies  of  highlighting  the  technology  itself  and  teaching  people  how  to 
use  the  technology  may  be  appropriate  for  faculty  interested  in  technology,  but  this  is  not  effective  for 
mainstream  faculty.  The  mainstream  is  more  inclined  to  show  interest  in  a new  technology  if  it  is  easy  to  use 
and  does  not  require  a major  change  in  the  skills  they  already  possess. 


With  these  issues  in  mind,  the  DMP  set  out  to  select  and  implement  a web-based  course  delivery 
package  for  mainstream  faculty  Our  primary  assumptions  in  selecting  WebCT  were  that  the  software  should 
meet  the  instmctional  needs  of  faculty  using  the  web  for  instruction  and  that  litde  or  no  kno\\dedge  of  HTML 
should  be  required. 

As  much  as  possible,  faculty  were  involved  in  the  selection  and  implementation  process.  A committee 
of  faculty  and  support  representatives  was  formed  to  assist  in  the  final  selection  of  the  software  package  and  to 
advise  on  the  implementation  plan.  A short  survey  was  circulated  to  provide  the  committee  with  feedback  on 
the  key  features  required  by  faculty. 

All  faculty  were  invited  to  participate  in  the  pilot  project.  One  trainir^  session  was  arranged  and  two 
more  were  added  after  the  overwhelming  initial  response.  Duiii^  July  and  August  1998,  54  faculty  were  trained 
in  how  to  use  the  basic  features  of  WebCT  and  how  to  convert  standard  word  processing  files  to  HTML. 

An  effort  was  made  to  provide  as  much  a support  as  possible.  A srudent  was  hired  to  provide 
telephone  and  one-on-one  support.  In  addition,  a listserv,  an  e-rnail  address  and  a website  were  provided 
(http:// www.ryerson.ca/ dmp/ courseware)  to  communicate  with  faculty  participating  in  the  pilot.  At  the  start 
of  the  school  year  one  more  support  person  was  added  to  assist  faculty  and  when  necessary,  students  in  using 
WebCT. 

In  the  first  semester  there  were  25  faculty  using  WebCT  to  offer  a total  of  34  courses  with 
approximately  3,500  students  enrolled.  WebCT  is  beir^  used  across  the  five  faculties  and  for  all  four  years  of 
study.  In  its  second  semester  WebCT  was  used  to  offer  44  courses  by  32  faculty. 

It  would  appear  that  WebCT  has  attracted  some  faculty  that  had  not  yet  used  web- based  instmctional 
technology.  A small  group  of  faculty  who  are  interested  in  or  are  now  usir^  WebCT  know  nothing  about 
HTML  and  had  not  used  the  World  Wide  Web  for  instmction  prior  to  WebCT. 

In  an  effort  to  assess  the  effects  and  effectiveness  of  the  WebCT  pilot,  smdent  and  faculty  surveys  are 
being  used  to  gather  data.  Faculty  were  asked  about  their  previous  experiences  with  this  type  of  instmctional 
technology,  their  reasons  for  choosir^  to  use  WebCT,  how  they  are  usir^  WebCT,  etc.  Smdents  were  asked 
similar  questions  about  their  experience  with  the  Internet  and  the  computer  along  with  the  barriers  and  uses  for 
WebCT. 

It  is  perhaps  too  soon  to  reflect  on  the  reasons  for  mainstream  faculty  adoption  of  the  web  as  an 
instmctional  technology.  However,  it  may  be  possible  to  make  some  general  observations  about  the 
effectiveness  of  our  decisions  and  process  so  far. 

The  decision  to  involve  faculty  in  the  selection  and  implementation  process  was  a good  one.  Input 
provided  DMP  staff  with  ins^hts  into  the  day-to-day  needs  of  faculty  that  would  not  have  been  considered 
otherwise.  WebCT  has  proven  to  be  a tool  that  is  attractive  to  mainstream  faculty  because  it  is  flexible,  it  meets 
some  or  most  of  their  instmctional  needs,  and  because  it  is  an  aid  in  course  administration. 

Although  support  personnel  were  not  always  kept  busy,  faculty  appreciated  knowir^  that  there  was  a 
high  level  of  support  should  they  need  it.  As  faculty  take  small  steps  toward  full  course  implementation  and 
begin  to  push  beyond  their  current  WebCT  skills  the  number  of  support  calls  has  increased. 

There  are  factors  beyond  our  plannir^  and  procedures  that  contributed  to  the  pilot's  success.  These 
factors  include  increased  access  for  both  faculty  and  smdents  to  networked  computers,  a growir^ 
understanding  of  the  instmctional  applications  for  the  Internet,  and  the  availability  of  WebCT  and  integrated 
packages  like  it  that  are  des^ned  specifically  for  web- based  instmction  in  higher  education. 

As  we  begin  to  plan  to  move  out  of  the  pilot  st^e  we  are  refining  our  strategies  and  continuing  to 
look  for  ways  to  make  the  web  more  accessible  to  mainstream  faculty  for  instmction.  We  have  initiated  a 
"house  call"  service  that  provides  one-on-one  consultir^  for  faculty  usir^  WebCT.  Initial  faculty  and  smdent 
feedback  suggests  that  we  must  put  more  effort  into  supporting  smdents  to  eliminate  the  support  burden  for 
the  faculty.  We  will  also  continue  to  investigate  better  tools  for  creatir^  web  pages  from  word  processing  files. 
Finally,  by  focussing  on  the  instmctional  applications  of  this  technology  we  continue  to  explore  the  World 
Wide  Web  as  a means  for  effective  and  innovative  teaching. 
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Increasingly,  both  the  public  and  private  sectors  are  demanding  more  efficiency  from  public  Venezuelan  universities.  This 
pressure  for  efficiency  in  education  has  resulted  in  an  increasing  adoption  of  New  Information  Technologies  in  the  educational, 
research  and  administrative  areas  of  higher  education.  In  fact,  the  use  of  NIT  in  Venezuelan  universities  is  superior  to  averages 
elsewhere  in  the  Latin  America  and  in  the  world. 


Figure  1 : Universityes  Web  Sites 


The  above  chart  shows  that  while  60%  of  the  universities  in  the  world  have  a web  site  and  58,8%  of  the  South  American  and 
the  Caribbean  universities  have  a presence  on  the  web,  almost  all  public  Venezuelan  universities  (89%)  and  less  than  half  of  the 
private  ones,  have  a web  page  that  basically  fulfills  the  institution’s  information  needs. 

However,  in  analyzing  the  possible  uses  of  these  technologies,  it  is  clear  that  only  a few  universities  have  effectively 
incorporated  the  NIT  into  their  systems. 

For  example,  while  some  universities  such  as  the  Universidad  Simon  Bolivar,  the  Universidad  Central  de  Venezuela  and  the 
Universidad  Nacional  Experimental  Sim6n  Rodriguez  use  NIT  in  the  decision  making  process,  only  40%  of  the  public 
universities  with  on-line  systems  use  them  as  a support  for  the  research  activities. 

Mostly,  the  NIT  are  being  used  to  display  information  on  the  web  about  the  universities’  programs  and  mission.  However, 
without  a clear  picture  of  the  total  information  system  and  its  relationship  to  a communication  policy  statement,  it  remains  unclear 
how  NIT  are  being  incorporated  to  the  universities  policies  as  a whole. 

Despite  the  efforts  made,  only  three  universities  (Universidad  de  los  Andes,  Universidad  de  Carabobo  and  Universidad 
Simon  Bolivar)  have  an  effective  on-line  library  consultation  service.  The  others  struggle  against  the  generalized  financial  crisis 
and  poorly  trained  personnel  to  update  their  library  collections. 

E-mail  services  are  not  of  common  use:  i.e.  only  25%  of  the  professors  at  the  Universidad  del  Zulia  (the  second  largest 
Venezuelan  university)  has  and  uses  an  e-mail  address.  It  was  during  the  second  semester  of  1998  when  administrators  of  this 
university  began  using  e-mail  services  to  communicate  with  the  faculty. 

The  use  of  NIT  among  each  university  has  grown  at  a different  rate  and  speed,  thus  displaying  different  modalities.  This 
hinders  both  inter-operability  between  systems,  as  well  as  each  system’s  uses.  For  example,  private  university  students  produce 
100%  of  the  ’’unofficial  web  sites";  students  attending  public  institutions  have  made  none. 

Universities  like  Universidad  del  Zulia,  Universidad  de  Carabobo  and  Universidad  de  Los  Andes  develop  communication 
infrastructures  that  include  not  only  web  sites  and  e-mail  and  Internet  services,  but  also  Radio  and  Television  Stations  of  public 
service.  However,  these  services  do  not  collaborate  to  foster  and  coordinate  a common  communication  objective. 

The  architecture  to  build  the  different  systems  among  Venezuelan  universities  varies.  The  Universidad  Nacional 
Experimental  del  Tachira's  project  is  based  on  a joint  venture  with  IBM.  The  system  will  serve  the  administrative  and  academic 
sectors  and  coordinate  the  beginning  and  development  of  specific  data  pools  of  academic  information  (Regional  Information 
Center,  Research  Lab,  Libraries,  Dean's  Office  and  all  units  that  require  information  services). 

The  UNELLEZ  (Universidad  de  los  Llanos  Ezequiel  Zamora)  established  an  agreement  with  the  Universidad  de  Carabobo 
for  the  development  of  specialized  services  of  personalized  consultation  on  the  library.  This  will  allow  access  to  the  university’s 
databases  as  well  as  to  on-line  consultation  by  modem  at  all  four  of  its  campuses. 

The  Universidad  de  Carabobo  could  be  considered  the  biggest  virtual  library  of  the  central  area  of  Venezuela.  This  institution 
is  performs  highly  in  the  area  of  information  management. 

In  1987,  the  Universidad  Simon  Bolivar  began  an  internal  discussion  on  the  differences  between  information  industry  and 
knowledge  industry  concepts.  The  discussion  resulted  in  a small  team  of  information  and  technology  specialists  who  input  data  for 
the  Information  system  of  the  institution.  The  system  bases  its  performance  on  the  matrix  organization,  an  organizational  structure 
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that  characterizes  the  Universidad  Simon  Bolivar.  This  team  of  specialists  also  manages  the  web  sites  of  the  university. 

The  Universidad  Simon  Rodriguez  pursues  a project  considered  the  most  advanced  in  Venezuela.  It  consists  of  the  use  of 
networked  computers  to  enforce  the  curricula  development  model  of  the  university,  a model  based  on  the  Andragogy.  The  project 
also  includes  intercommunication  with  27  satellite  campuses  of  the  university  around  the  country. 

While  most  of  the  Venezuelan  universities  claim  that  they  use  NIT,  just  a few  have  their  entire  library  collections  available 
for  on-line  consultation.  Some  others  offer  workstations  with  basic  programs,  such  as  word  processors  for  at  least  half  of  their 
student  body,  but  very  few  can  guarantee  workstations  with  internet  access,  e-mail  service  and  on-line  data  search  capabilities  for 
research  purposes. 

Despite  the  evident  delay  in  the  full  application  of  NIT,  Venezuelan  universities  have  made  a great  effort  to  update  their 
technological  infrastructure  and  to  adjust  their  management  methods  to  become  more  efficient  through  the  uses  of  technology. 
Even  so,  the  major  challenge  they  face  is  to  overcome  the  financial  crisis  that  characterizes  most  underdeveloped  countries.  Such 
a barrier  must  be  surmounted  if  Venezuelan  universities  are  to  offer  the  top  quality  education  level  opportunities  necessaiy  for 
students  to  gain  a seat  at  the  banquet  with  those  who  are  euucalcu  in  the  wealthier,  tcchnolog)'-rich  countries  of  the  world. 

References 

- Journal  references: 

SALOM,  R.  (1998).  La  Gerencia  Universitaria  e Internet.  Revista  Venezolana  de  GerendOy  3 (5),  55-71 . 

- Proceedings  references: 

SILVIO,  J.  (1998).  La  Virtual izacion  de  las  Universidades.  Paper  presented  at  Visionaries  98,  April  22-24,  Caracas,  Venezuela. 


UNIVERSIDAD  CAT6liCA  ANDR6S  BELLO  (1998).  Pagina  Principal, 

http://www.ucab.edu.ve/UCAB/Facultades/Ingenieria/Derecho  (March  5^*^,  1 998). 

UNIVERSIDAD  CENTRAL  DE  VENEZUELA  (1998).  Pagina  Principal,  http://www.cosmopoima.ing.ucv.ve  (February  15’’’, 
1998). 

UNIVERSIDAD  DE  CARABOBO  (1998).  Fundacion  Centro  de  Informacion  y Documentacion,  http://www.cid.ve  (March  5’’’, 
1998). 

UNIVERSIDAD  CENTRO  OCCIDENTAL  LISANDRO  ALVARADO  (1998),  Pagina  Principal,  http://www.redisa.ucla.edu.ve 
(March  7’^  1998). 

UNIVERSIDAD  DE  LOS  ANDES  (1998).  Servicio  de  Documentacion  de  la  Universidad  de  los  Andes,  http://www.ula.ve 
(Februery  15’^  1998). 

UNIVERSIDAD  DE  LOS  LLANOS  EZEQUIEL  ZAMORA  (1998).  Enlace.  Pagina  Informativa  de  la  Oficina  de  Enlace, 
http://www.unellez.edu.ve  (March  9’’’,  1998). 

UNIVERSIDAD  DE  NUEVA  ESPARTA  (1 998).  Pagina  Principal,  httD://www.une.edu.ve  (March  5*,  1 998). 

UNIVERSIDAD  DE  ORIENTE  (1998).  Universidad  de  Oriente,  http://www.mochima.sucre.udo.edu.ve  (March  d*,  1998). 
UNIVERSIDAD  DEL  ZULIA  (1998).  Pagina  Principal,  http://www.luz.ve  (March  5*,  1998). 

UNIVERSIDAD  METROPOLITANA  (1998).  Pagina  Principal,  http://www.etheron.  net/etheron/iaz/unimet.htm  (March  T*, 
1998) 

UNIVERSIDAD  NACIONAL  ABIERTA  (1998).  Pagina  Principal,  http://www.memberes.Tripod.com/maguicha/postgrado.html 
(April  12'^  1998) 

UNIVERSIDAD  NACIONAL  EXPERIMENTAL  DEL  tACHIRA  (1998).  Centro  de  Estudios  de  Teleinformatica, 
http://www.unet.ve  (March  9’’',  1998) 

UNIVERSIDAD  NACIONAL  EXPERIMENTAL  FRANCISCO  DE  MIRANDA  (1998).  Universidad  Nacional  Experimental 
Francisco  de  Miranda,  http://www.unefin.edu.ve  (March  6*,  1 998). 

UNIVERSIDAD  NACIONAL  EXPERIMENTAL  POLIT^CNICA  ANTONIO  J0S6  DE  SUCRE  (1998). 

UNIVERSIDAD  NACIONAL  EXPERIMENTAL  POLIT^CNICA  ANTONIO  J0S6  DE  SUCRE,  http://unexpo.edu.ve  (March 
6'^  1998). 

UNIVERSIDAD  NACIONAL  EXPERIMENTAL  SIMON  RODRJGUEZ  (1998).  Interaccion  de  la  Universidad  con  su  Entomo, 
http://www.svr.edu/~imontova/unesr  (March  T*,  1998). 

UNIVERSIDAD  RAFAEL  MARIA  BARALT  (1998).  Programa  de  Investigacion,  http://www.civila.com/archivos/venezuela 
(March  1998). 

UNIVERSIDAD  RAFAEL  URDANETA  (1998).  Pagina  Principal,  httn://www.uru.edu  (April  10’’’,  1998). 

UNIVERSIDAD  SIMON  BOLIVAR  (1998).  La  Direccibn  de  la  Gestibn  de  la  Informacion,  http://www.usb.ve  (April  4”’,  1998). 
UNIVERSIDAD  TECN0L6GIC0  DEL  CENTRO  (1998).  Pagina  Principal,  http://www.unitec.edu.ve  (March  7‘\  1998). 

Acknowledgements 

This  research  has  been  possibly  by  a grant  from  CONDES  (Universidad  del  Zulia).  Thanks  to  Kathleen  Tyner  for  her  assistance 
with  the  manuscript. 


1402 


Remote  Control  of  Hybrid  Analog/Digital  Video  Networks  in  Distance 

Learning  Environments 


Anatoliy  Gordonov:  GORDONOV@POSTBOX.CSI.CUNY.EDU 

Roberta  Klibaner;  KLIBANER@POSTBOX.CSI.CUNY.EDU 

Computer  Science  Department,  Staten  Island  College,  City  University  of  New  York, 
2800  Victory  Blvd.  lN-215,  SI,  NY  10314,  USA 


Introduction 

Hybrid  Analog/Digital  Video  Networks  (ADViNet)  become  more  and  more  popular  on  the  market.  They  are 
employed  in  the  applications  that  require  fiill-screen  motion  video  transmission  in  the  local  area  networks  and,  at  the  same 
time,  cannot  afford  slowing  down  regular  network  activities.  Among  these  applications  are  real-time  control  systems, 
electronic  classrooms,  and  distance  learning  environments. 

The  typical  physical  architecture  of  ADViNet  includes  two  communication  media:  digital  and  analog  (Computer 
Video  Web  1996).  The  digital  medium  is  used  by  a regular  digital  network  according  to  its  type  and  topology.  The  analog 
medium  combines  a VGA  bus  with  several  additional  control  lines. 

ADViNet  allows  users  to  send  real-time  fiill-screen  motion  images  through  an  analog  communication  medium.  Main  video 
network  operations— Broadcast,  Multicast,  and  Monitoring—  cover  basic  instructors’  activities  such  as  lectures,  tutorials, 
individual  and  group  supervision.  Recently  developed  logical  level  intelligent  video  network  operations  make  teaching  in 
the  ADViNet  environment  more  productive  for  the  students  and  instructor  (Gordonov  A.  1997),  (Gordonov,  A.,  Kress,  M. 
1997).  However,  the  architecture  of  ADViNet  permits  its  use  only  within  a limited  area.  A distance  learning  environment 
includes  groups  of  network  nodes  dispersed  far  apart.  ADViNet,  being  very  useful  in  a local  classroom,  cannot  automatically 
be  used  if  several  local  classrooms  participate  in  the  same  session.  In  this  paper  we  present  our  current  development  of  a 
distance  learning  environment  that  includes  several  classrooms  equipped  with  ADViNet.  We  define  distributed  video 
network  architecture,  methods  of  video  transmission  between  sites,  and  methods  of  the  remote  control  of  a remote  site. 


The  Architecture  of  a Distributed  Video  Network 


First  of  all,  let  us  give  several  definitions: 

Cluster  - a set  of  computers  connected  to  the  same 
ADViNet  video  bus.  { 

Primary  Cluster  - the  main  cluster  in  the  distributed  \ 
distance  learning  environment  where  the  instructor  is  located. 

Secondary  Cluster  - any  remote  cluster  which 
participates  in  the  distance  learning  session. 

Primary  Teacher  station  - the  teacher  station  in  the 
primary  cluster. 

Secondary  Teacher  station  - the  teacher  station  in 
a secondary  cluster. 

A general  structure  of  the  distributed  ADViNet  architecture  is 
shown  in  Figure  1.  Video  information  may  be  transmitted 
between  nodes  inside  a cluster  and  between  the  clusters.  Each 
node  of  a cluster  can  use  ADViNet  to  transmit  images  from  one 
member  of  a cluster  to  some  others.  Technically,  digitized  video 


ADViNcI  Cluster  #3 


Figure  1:  General  Structure  of  Distributed 
ADViNet. 
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can  be  transmitted  from  one  cluster  to  another.  Virtually,  any  commercially  available  videoconferencing  system  ( such  as 
CU-SeeMe,  Intel  ProShare,  Tenberg,  etc.)  may  be  used  for  this  purpose.  Some  of  them  (such  as  CU-SeeMe,  Intel  ProShare) 
must  be  installed  on  the  teacher  stations  in  each  cluster.  In  this  case  video  information  is  distributed  inside  the  Primary 
Cluster  through  the  local  ADViNet  and  may  be  transmitted  to  the  Secondary  Teacher  stations  over  the  videoconferencing 
system.  In  the  Secondary  Clusters  local  ADViNet  sends  video  information  to  all  the  local  nodes  participating  in  a session. 
If  the  videoconferencing  system  is  a separate  device,  it  should  be  connected  to  a teacher  station  though  a video  card.  It 
allows  this  station  to  provide  exchange  of  video  information  through  the  videoconferencing  hardware.  The  quality  of  video 
transmission  depends  on  the  characteristics  of  the  digital  channel  between  clusters  and  is  very  often  limited  by  these 
characteristics,  yet,  it  could  be  acceptable  for  many  applications.  Inside  each  cluster,  ADViNet  relieves  the  burden  of  video 
distribution.  Using  ADViNet  in  any  cluster  insures  delivery  of  video  information  to  any  student  station  without  interfering 
with  other  network  users  connected  to  the  same  local  area  segment.  Although  technically  possible,  this  structure  suffers  the 
lack  of  remote  control:  a user  in  one  cluster  cannot  control  video  equipment  in  others.  In  a distance  learning  environment 
the  remote  control  is  a vital  necessity.  It  allows  the  instructor  to  implement  many  functions  ( such  as  remote  control  of  video 
equipment,  control  of  video  distribution  in  a remote  cluster  or  remote  ADViNet  configuration)  without  asking  for  assistance 
at  the  remote  site. 

For  control  and  management  purposes,  in  any  cluster  one  station  (control  station)  is  chosen  for  remote 
communication  with  other  clusters.  By  default,  this  is  a Teacher  station.  For  any  session,  one  of  the  control  stations  is  the 
primary  control  station,  and  the  others  are  the  secondary  control  stations.  Tlie  primary  station  communicates  with  all 
secondary  ones.  Any  secondary  station  may  gain  control  over  the  entire  video  network  (which  includes  several  ADViNet 
clusters)  only  after  it  gets  the  permission  from  the  primary  control  station.  The  primary  station  may  set  and  monitor  a time 
limit  for  the  current  session,  may  extend  this  time,  or  interrupt  the  current  session.  Any  control  station  which  is  authorized 
to  manage  a video  session  may  request  from  all  other  control  stations  the  information  about  the  current  status  of  the  ADViNet 
clusters  and/or  send  the  necessary  commands  to  change  the  local  mode  in  an  ADViNet  cluster.  It  may  also  issue  special 
commands  to  control  video  equipment  in  a remote  site  such  as  video  cameras  or  teleconferencing  equipment.  During  a 
session,  a cluster  may  drop  the  connection  or  connect  to  the  current  session.  By  default,  the  Primary  Teacher  station  controls 
the  distributed  ADViNet.  Any  Secondary  Teacher  station  may  gain  control  over  the  distributed  environment  after  getting 
permission  from  the  Primary  station. 

All  remote  control  commands  as  well  as  local  ADViNet  commands  are  included  in  the  Remote  ADViNet 
Management  Protocol.  This  protocol  allows  the  users  to  control  video  distribution  in  remote  clusters  in  the  same  way  as 
they  do  it  in  a local  area  network.  Most  local  operations  are  implemented  remotely.  Among  them  are  various  types  of 
broadcast,  multicast  and  management  operations. 

The  distributed  ADViNet  architecture  and  Remote  ADViNet  Management  Protocol  are  supported  by  specially  developed 
software.  This  software  provides  remote  programming  control  in  a number  of  ways.  Recently  we  have  developed  three 
different  variants  of  establishing  communication  between  Primary  and  Secondary  stations.  They  are: 

• communication  through  a local  area  network.  This  situation  may  happen  when  all  the  clusters  are  located 
inside  one  campus.  The  supporting  software  works  in  the  Windows  NT  environment  and  may  use  any 
available  protocols  for  the  communication; 

• communication  through  TCP/IP  protocol.  The  communication  may  be  established  between  any  nodes 
connected  to  the  Internet  or  inside  a local  area  network  with  appropriate  protocols  installed; 

• transmission  of  the  control  information  using  special  functions  of  the  videoconferencing  systems.  Tliis  method 
is  used  if  not  all  the  clusters  have  the  Internet  connection.  Intel  ProShare  videoconferencing  system  includes 
a special  function  that  supports  transmission  of  text  or  binary  information  concurrently  with  video.  This 
function  is  used  to  establish  communication  between  clusters  and  to  transmit  commands. 

The  proposed  paper  represents  an  ongoing  project.  The  implementation  of  the  proposed  distributed  video  network 
architecture  and  methods  of  the  control  of  a remote  cluster  allow  us  to  incorporate  ADViNet  in  the  distance  leaning 
environment  and  use  all  the  benefits  of  ADViNet  in  any  cluster. 
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A VIRTUAL  ENVIRONMENT  PROMOTING 
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While  browsing  the  Internet  we  may  find  various  types  of  educational  sites,  but  not  all  of  them  could  be 
addressed  as  an  environment  hosting  a learning  community.  In  this  article  we  would  deal  with  both  terms  in 
order  to  show  their  educational  advantages,  Likewise,  describe  Histomania  as  an  environment  hosting  a virtual 
community  of  people  interested  in  history. 

Virtual  communities  and  environments 

A survey,  that  explore  learning  communities,  defines  two  types  of  communities  existing  side  by  side,  even  in  our 
time.  One  type  refers  to  people  basically  sharing  a common  place  and  the  other  type  is  based  on  people  sharing 
interests.  Both  types  include  members  sharing  culture  (language  and  values  ) and  communication  means  (Office 
of  Learning  Technologies,  Canada,  1998).  By  that  definition  learners  and  teachers  in  school  form  a community 
of  the  first  type  and  subscribers  to  theater  or  music  programs  form  communities  of  the  second  type.  Members  of 
the  first  type  do  not  choose  their  membership  (the  place  and  the  subjects  studied),  but  they  share  a place,  use 
common  culture  and  feel  affiliated  to  the  school.  Members  of  the  second  type  are  volunteers  who  choose  to  be  part 
of  the  community  on  a ground  of  common  interests.  But,  in  order  to  fulfill  their  membership  they  also  have  to 
share  a specific  place  and  time.  The  main  feature  of  network  technology  is  the  communication  capabilities  which 
may  free  the  community  from  being  place  and/or  time  bounded.  The  main  question  is  how  do  we  use  these  net 
capabilities  as  a place  for  developing  a community  which  will  promote  the  study  of  history. 

When  analyzing  the  term  “virtual  community”  we  find  two  phases  in  the  community  development:  registration 
and  ongoing  existence.  At  the  registration  phase  people  check  in  voluntarily  and  look  around  to  find  out  whether 
they  may  gain  advantages  or  be  satisfied  socially.  At  the  ongoing  phase  membership  is  significant  if  the  member 
feels  that  he  contributes  to  the  community  and  that  he  is  gaining  advantages  from  his  visits.  Likewise,  he  has  to 
develop  a sense  of  identity  and  a feeling  of  affiliation.  These  will  emerge  only  if  social  bonds  will  be  established 
between  members.  We  assumed  that  given  a place  where  teachers  may  consult  each  other  and  hold  a dialog  with 
historians  and  history  instruction  experts  will  create  conditions  for  a community.  Likewise,  offering 
students  tips  for  learning  history  and  game  like  activities  might  do  the  same.  Moreover,  the  opportunity  of  a dialog 
between  students  and  teachers,  teachers  and  parents,  students  and  parents,  who  will  know  each  other  only  virtually, 
will  help  in  crating  a new  trust  between  populations  who  seem  to  be  in  an  ongoing  conflict. 

Virtual  communities  grow  and  function  in  virtual  environments.  The  environment  that  hosts  the  community 
represents  the  logical  framework  as  well  as  the  virtual  place  for  its  existence.  It  is  apparent  that  the  communication 
layer  of  a web  site  is  a main  factor  in  establishing  a virtual  learning  community  by  supplying  the  social  factor 
needed  in  the  learning  process.  Likewise,  we  propose  the  idea  that  conditions  needed  for  a community  to  emerge 
and  evolve  are  not  only  the  technological  capabilities  of  the  net  but  also  a meta  outlook  which  produces  not  a site 
but  an  holistic  wired  environment,  we  define  an  “holistic  learning  environment”  as  a set  of  web  sites  who  share  the 
same  meta  subject  (for  instance,  a discipline)  and  supply  a place  for  various  interested  visitors  (like,  students, 
teachers,  parents,  specialists)  etc.,  by  means  of  CMC.  We  argue  that  the  existence  of  a larger  structure  to  which  the 
discrete  sites  belong  gives  rise  to  a feeling  of  stability  which  creates  an  atmosphere  of  security  and  seriousness.  We 
assume  that  this  atmosphere  helps  the  visitors  in  developing  a sense  of  identity  with  the  environment,  similarity 
with  other  users  and  willingness  to  visit  once  more  and  participate  in  the  activities  proposed  there. 

Histomania 

The  environment  is  a tryout  to  implement  the  concepts  described  above. 

It  is  based  on  a holistic  approach  in  web  based  learning  where  the  environment  should  be  a combination  of 
information  handling  and  of  communication  tools.  Developing  a community  is  the  main  aim  of  the  environment. 
Like  in  virtual  places  the  membership  is  on  a volunteer  base.  Any  body  might  be  a guest.  In  this  position  he  may 
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look  over  the  information  sites  available  at  the  resources  center.  But  it  is  expected  that  guests  will  find  the  place 
interesting  enough  to  register  as  members. 

The  environment  consists  of  five  components: 

Following  Time 

A resources  center  comprised  of  several  sites,  most  of  them  build  as  data  base  including  a 
search  engine,  and  all  are  related  to  the  to  the  Israeli  history  curriculum.  We  think  that  the  focus  of  history 
instruction  should  be  the  primary  sources  - texts  and  visuals..  Therefore,  primary  sources  as  well  as  historical 
research  are  the  main  content  of  the  resource  center.  The  place  is  divided  into  four  parts  according  to  historical 
periods:  The  Classic  World,  The  Middle  Ages,  Modem  Western  Society,  Contemporary  History.  In  each  period 
Jewish  History  is  dealt  as  well.  The  center  is  intended  for  students  who  wish  to  prepare  papers  or  presentations 
as  school  projects  and  for  teachers  who  look  for  primary  resources  for  building  their  lessons. 

A Disciplinarium 

A center  which  serves  history  teachers  according  to  needs  defined  by  the  members.  Discussion  groups  enable 
teachers  to  discuss  pedagogical  questions  related  to  the  implementation  of  learning  materials.  Participants  are 
requested  to  contribute  ideas,  tips,  questions  etc.  to  each  other.  Likewise,  teachers  are  expected  to  send  to  the 
students’  center  tips  for  learning  history  or  participate  in  students  discussion  groups.  Members  may  also  meet  in 
a chat  room  where  they  may  talk  on  any  subject  they  like  and  develop  social  relations.  Parents  may  participate  in 
discussions  groups  of  teachers,  but  have  also  a place  of  their  own  where  they  may  discuss  other  subjects  as  they 
will  define. 

Tips  from  Clio 

The  students  center.  That  is  the  place  where  students  may  share  information  with  other  students,  discuss  subjects  in 
moderated  groups,  get  tips  for  history  learning  from  teachers,  students,  and  from  any  body  else  who  will  be  willing 
to  help.  At  the  same  section  where  learning  occurs  a chat  room  called  “Nirvana  Pub”  operates.  That  is  the  space 
where  students  may  talk  freely  and  develop  social  relations.  Being  members  of  the  community,  students  may  ask 
historians  questions  in  subjects  that  they  are  interested  in. 

Time  Machine 

It  is  an  effort  to  use  edutainment  concept  for  enhancing  motivation  towards  historical  issues. 

Riddles,  puzzles  and  crossword  puzzles  are  organized  in  a curricular  approach  and  are  related  to  the  Israeli 


The  “Seal  Defenders”  Club 

The  club  will  include  members  who  proved  themselves  to  be  as  active.  They  will  be  coaches  of  new  members, 
guides  to  guests,  participate  in  mingled  discussion  groups  (students,  teachers,  parents,  and  historians)  and  talk 
over  issues  in  history  and  in  education  as  well.  Club  members  will  propose  new  ideas  to  be  developed  in  the 
environment  and  evaluate  ideas  proposed  by  non-members. 

In  the  recent  months  almost  700  teachers  registered  voluntarily  and  400  students.  3 projects  based  on  the 
resources  embedded  in  the  environment  were  held.  Students  from  different  schools  used  primary  resources 
and  participated  in  discussion  groups.  Some  students  send  riddles  and  puzzles  they  solved.  Teachers 
participated  in  discussions  held  in  the  the  disciplinarium  and  some  of  them  sent  lesson  plans  and 
recommendations  related  to  history  sites  on  the  the  Internet. 

Still,  we  are  not  satisfied  with  the  activity  done  but  feel  that  we  are  in  the  right  direction. 


Syllabus. 
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1 Overview 

The  University  of  Ulster  has  approved  the  planning  of  a linked  series  of  postgraduate  awards  by  distance  education  that  will  focus  upon 
various  themes  within  the  Project  Management  discipline.  The  international  market  for  project  management  is  extensive  in  both  the 
developed  and  developing  world.  We  have  identified  students  in  Asia  who  wish  to  undertake  postgraduate  courses  at  an  overseas 
University  but  who  cannot,  for  various  reasons,  leave  their  job  or  family  to  study  abroad. 

The  Centre  for  International  Project  Management,  based  within  the  Faculty  of  Engineering  has  been  established  to  deliver  the  Portfolio. 
It  will  draw  upon  the  expertise  of  staff  from  the  School  of  the  Built  Environment,  the  School  of  Electrical  and  Mechanical  Engineering, 
the  Faculty  of  Informatics  and  the  Faculty  of  Business  and  Management.  Critical  factors  involved  in  the  success  of  this  venture  will  be: 

1.  The  formation  of  joint  ventures  and  strategic  alliances  with  other  institutions  and  colleges  based  in  each  country  where  the  courses 
are  to  be  delivered  and 

2.  World-wide  commercial  and  professional  organisations  that  can  introduce  state  of  the  art  practices  into  the  curriculum. 

Modem  organisations  are  effectively  introducing  change  through  the  use  of  business  projects  that  will  further  their  corporate  objectives. 
We  will  meet  the  requirements  of  prospective  part-time  students,  in  their  late  twenties  to  early  fifties,  who  have  been  chosen  by  their 
organisations  to  deliver  such  projects.  Initially,  it  is  anticipated  that  two  streams  or  pathways  will  be  available.  These  pathways  are 
Construction  and  Engineering  and  will  be  followed  by  Informatics,  Business  and  Leisure  and  Tourism,  dependent  upon  demand. 

University  of  Ulster  (UU)  academic  staff  will  deliver  the  Portfolio  via  a series  of  residential  courses  at  each  international  centre, 
supported  by  the  staff  of  the  local  institution,  independent  study  materials  and  dedicated  UU  administrative  staff.  The  role  of  the 
International  Partner  Institutions  is  extremely  important  to  the  successful  delivery  of  the  Portfolio.  We  expect  each  institution  to  send  a 
member  of  their  staff  on  our  existing  Masters  Programme  to  get  a level  of  expertise  in  the  discipline  and  form  links  with  our  staff,  with  a 
view  to  their  being  involved  in  the  local  co-ordination  of  the  Portfolio  on  their  return  home.  The  use  of  advanced  telecommunications 
facilities  would  be  incorporated  into  the  delivery  and  support  aspects  of  the  Portfolio.  Teaching  approaches  will  comprise  a combination 
of  conventional  methods  and  tutoring  via  the  Internet,  using  an  integrated  environment  specifically  for  this  purpose.  In  addition, 
supplementary  resource  material,  in  the  form  of  CD  ROMS,  videos  and  paper-based  materials  will  be  distributed  to  the  students. 

As  an  open  and  distance  learning  programme,  the  duration  may  be  varied  dependent  upon  the  requirements  of  each  country.  However 
the  preferred  model  of  delivery  is  that  the  taught  modules  will  be  completed  over  sixteen  months  and  the  Dissertation  and  accompanying 
Project  should  be  completed  in  a further  eight  months. 

2 Rationale 

The  rationale  for  developing  the  Portfolio  is  to 

• Meet  the  increasing  global  demand  for  qualified  project  managers 

• Develop  further  our  existing  Masters'  programmes 

• Provide  a focused  strategy  for  growth  in  teaching  and  research 

• Enrich  staff  development  via  the  direct  experience  of  design  and  implementation  of  distance  education  programmes 

• Further  the  achievement  of  the  University's  international  objectives 

• Indicate  the  University  of  Ulster's  commitment  to  widening  access  to  its  courses 

• Provide  an  opportunity  for  those  people  who  are  constrained  by  employment,  by  geographical  location,  or  who  are  unable  to  attend 
conventional  full  time  or  part-time  study  programmes  to  get  a University  of  Ulster  Postgraduate  award. 

3 Issues 

Currently  the  project  team  is  evaluating  various  systems  that  support  tutoring  over  the  Internet.  Based  on  evaluations  carried  out  in 
similar  projects,  there  are  features  that  will  be  considered  essential  for  inclusion  in  the  selected  computer  managed  instruction  software. 
3.1  Communication 

Given  that  the  Portfolio  is  being  designed  primarily  for  delivery  overseas,  we  consider  that  in  the  first  instance,  asynchronous  tutoring 
will  be  adequate.  Taking  into  consideration  time  differences  between  students  and  tutors,  it  is  unlikely  that  both  groups  will 
communicate  synchronously,  except  in  exceptional  circumstances.  However,  asynchronous  electronic  communication  tools  such  as 
electronic  mail  and  discussion  threads  are  perceived  to  be  crucial  to  the  success  of  the  project.  (Heywood  and  Cornelius  98)  have 


identified  that  such  communication  tools  promote  a feeling  of ’belonging'  among  students  who  are  registered  in  a virtual  class,  (Paul  and 
Brindley  96)  also  point  out  that  the  incorporation  of  good  support  mechanisms  is  crucial  to  the  success  of  distance  education  courses. 

Considering  electronic  mail,  a useful  feature  would  be  the  ability  for  the  student  to  email  a tutor  or  peer,  in  context,  from  every  section  of 
courseware  while  the  system  automatically  includes  the  Uniform  Resource  Locator  (URL)  for  that  particular  section  within  the  body  of 
the  email  message.  The  recipient  is  then  in  a position  to  click  the  URL  and  the  section  to  which  the  sender  referred  in  the  email  is 
launched  - a great  time  saving  feature. 

One  of  the  advantages  of  threaded  discussion  is  that  it  can  be  used  by  tutors  as  a means  of  eliciting  student  response  to  various  topics. 
Occasionally  the  material  generated  by  these  discussion  threads  may  include  aspects  that  the  tutor  had  not  considered  while  designing  the 
course,  and  thus  the  material  may  be  subsequently  incorporated  into  future  module  revisions.  In  addition,  this  form  of  communication 
would  require  little  effort  on  the  part  of  the  tutor,  and  could  be  a means  of  easily  immersing  students  in  the  learning  environment. 

3.2  Access 

Emphasis  will  be  placed  on  course  structure,  and  assessment,  using  text  and  graphics.  (Heywood  and  Cornelius  98)  have  also  identified 
that  students’  concerns  include  download  times  for  course  files,  and  actual  duration  of  on-line  study.  Bearing  this  in  mind,  initially,  the 
intention  is  to  develop  a course  with  small  file  sizes,  thus  facilitating  access  via  low  speed  modems.  Supplementary  traditional  material, 
such  as  video  files,  will  be  sent  to  the  student.  It  is  likely  that  such  concerns  will  fade  when  the  cost  of  technology  and  access  to  the 
Internet  drop.  However,  in  the  meantime,  the  use  of  software  such  as  Macromedia  Director  allows  the  development  of  animated 
interactive  sequences  that  can  provide  an  engaging  stimulus  while  requiring  considerably  less  download  times  than  video  files.  (Riley 
and  Hodgkinson  97),  in  early  studies  of  students  using  an  interactive  multimedia  package  incorporating  structured  student  guidance  as 
part  of  their  construction  course,  conclude  that  students  consider  such  interactive  systems  to  be  a motivating  and  flexible  means  of  self 
learning.  They  also  state  that  such  systems  should  be  comprehensive  in  order  to  provide  for  the  various  learning  styles  exhibited  by 
students. 

3.3  Learning  Environment 

The  learning  environment  created  for  the  student  will  not  only  draw  upon  a state  of  the  art  computer  managed  instruction  system,  but 
will  allow  direct  interaction  between  tutors  and  students  by  the  use  of  short  residential  courses.  The  latter  is  seen  not  as  an  information 
transfer  exercise,  but  one  that  promotes  a ’deep  approach'  (Entwistle  95)  to  teaching  and  learning  that  can  significantly  'add  value’  to  the 
quality  of  the  educational  experience. 

Considering  the  Internet  aspect  of  the  Portfolio,  (King  98)  makes  several  suggestions,  based  on  evaluation  of  current  applications,  for 
enhancement  of  predominantly  document  orientated  web  based  courseware.  It  is  important  that  such  courseware  is  not  seen  as  simply  an 
on-line  page  turning  exercise.  Students  benefit  from  being  engaged  with  the  system.  (Paul  and  Brindley  96)  stress  that  one  of  the 
lessons  to  be  learned  from  previous  forays  into  distance  learning  is  that  students'  individual  differences  should  be  taken  into 
consideration  when  designing  a course,  in  order  not  to  produce  an  assembly  line  type  of  learning.  To  this  end,  we  will  include  several 
self-assessment  exercises,  such  as  multiple  choice,  image  map  or  list  map  exercises,  which  are  automatically  graded  by  the  system. 
These  will  have  the  double  advantage  of  incorporating  interaction  while  providing  students  with  instant  feedback,  along  with  direction  to 
remedial  or  further  tutorial  material.  Based  on  previous  research  into  intelligent  tutoring  systems,  carried  out  in  the  University  of  Ulster, 
(Hannigan  et  al.  97)  significant  effort  will  be  expended  on  the  provision  of  an  individualised  feedback  feature.  The  software  system 
chosen  to  ’host’  the  Portfolio,  therefore,  should  be  capable  of  supporting  this. 

4 Future  Directions 

This  project  is  in  its  infancy,  and  our  current  goals  are  to  develop  course  modules  in  a manner  suitable  for  delivery  over  the  Internet,  to 
decide  on  suitable  software  for  the  delivery  of  such  modules,  and  to  positively  deal  with  the  issues  we  have  discussed  above.  Although 
our  first  ’intake’  of  students  is  not  due  until  March  2000,  we  anticipate  that  a good  proportion  of  the  Portfolio  will  be  in  place  over  the 
next  six  months. 

5 References 

Heywood  and  Cornelius  (98),  Heywood,  D.I.  & Cornelius,  S.C.  (1998),  Developing  a Virtual  Campus  for  UNIGIS,  HABITAT,  5 
(Spring),  27-29 

King  (98),  King,  T,  (1998),  Improving  the  Quality  of  Document-Based  Web  Courseware,  HABITAT,  5 (Spring),  24-26 
Hannigan  et  al,  (97),  Hannigan  C.,  Murphy  M.  and  Adamson  K.  (1997).  CELT:  An  Intelligent  Electrocardiology  Tutoring  Tool. 
Multimedia  Technology  in  Medical  Training,  Paper  and  Poster  Description  Proceedings  of  the  European  Workshop  RWTH,  Aachen, 
Germany.  82-93 

Riley  and  Hodgkinson  (97).  Riley,  M.  & Hodgkinson,  R.  (1997),  Integration  of  Multimedia  in  Construction  Technology  Education 
Programmes,  HABITAT,  3,  6-9 

Paul  and  Brindley  (96).  Paul,  R.  & Brindley,  J.  (1996).  Lessons  from  Distance  Education  for  the  University  of  the  Future.  In  Mills,  R and 

Tait,  A (Eds.)  Supporting  the  Learner  in  Open  and  Distance  Learning,  Pitman  Publishing.  43-55 

Entwistle  (95).  Entwistle,  N.  (1995).  Defining  Quality  in  Teaching:  The  Research  Perspective,  University  of  Edinburgh 


1408 


An  Analysis  of  CD-ROM  and  Print  Storybook  Reading  by  Parents  and 

Preschoolers 


James  P.  Van  Haneghan,  Dept  of  Behavioral  Studies  and  Educational  Psychology,  UCOM  3705,  University  of 
South  Alabama,  Mobile,  AL  36688,  USA,  email:  ivanhane@usamail.usouthal.edu 

Abigail  Baxter,  Dept,  of  Special  Education,  UCOM  3800,  University  of  South  Alabama,  Mobile,  AL  36688, 

email:  abaxter@usamail.usouthal.edu 


Despite  the  proliferation  of  Multimedia  CD-ROM  software  aimed  at  preschoolers,  there  is  little  data 
on  how  children  and  adults  interact  with  such  software.  Nor  is  there  data  concerning  educational  significance 
of  such  software.  Hence,  studies  of  how  children  and  parents  interact  with  multimedia  software  are  of  great 
importance.  The  goal  of  this  paper  is  to  examine  parent-child  interaction  with  CD-ROM  storybooks  and 
traditional  storybooks  to  determine  whether  the  interactions  between  parents  and  children  using  CD-ROM’s 
are  different  from  interactions  between  parents  and  children  in  joint  book  reading.  Joint  book  reading  has  been 
found  to  be  an  effective  context  for  the  development  of  pre-literacy  skills  (Bus,  Van  Ijzendoom,  & Pelligrini 
1995).  Hence,  comparisons  of  book  reading  and  CD-ROM  storybooks  could  provide  insight  into  how  CD- 
ROM  software  may  impact  literacy  development. 

Most  of  the  previous  studies  of  CD-ROM  storybook  reading  have  focused  on  school-age  readers 
working  with  CD-ROM  or  written  versions  of  storybooks  (e.g.,  Matthew,  1997;  Okolo  & Hayes,  1996). 
Findings  have  been  mixed.  Some  studies  found  that  CD-ROM  versions  led  to  better  comprehension  (Matthew, 
1997).  Matthew  found  that  students’  retellings  of  stories  were  better  when  they  worked  with  the  CD-ROM. 
Others  (e.g.,  Okolo  & Hayes,  1996)  have  found  that  comprehension  could  possibly  be  hurt.  Okolo  and  Hayes 
found  that  students  spent  a great  deal  of  time  clicking  on  animations,  rather  than  focusing  on  the  story. 

There  is  virtually  no  research  examining  preschoolers’  interactions  with  CD-ROM  storybooks,  even 
though  many  CD-ROM  developers  target  preschoolers.  Research  on  joint  storybook  reading  suggests  that  there 
are  several  benefits  to  such  activity  (Bus  et  al.,  1995).  Children  whose  parents  read  with  them  are  more 
advanced  in  language  and  pre-reading  skills  (Bus  et  al.,  1995).  Interventions  that  have  targeted  joint  reading 
of  books  in  children  at-risk  (e.g.,  Whitehurst  & Arnold,  1994)  have  found  that  joint  book  reading  improves 
reading  and  language  readiness.  We  were  interested  in  whether  CD-ROM  storybook  reading  had  the  same 
potential.  Our  goal  was  to  study  joint  (parent-child)  CD-ROM  storybook  reading,  much  like  investigators  have 
looked  at  joint  reading  of  storybooks  with  preschoolers.  We  use  examples  from  our  investigation  to  ask  what 
potential  benefits  there  might  be  to  preschoolers  and  parents  jointly  reading  CD-ROM  storybooks  versus 
traditional  print  storybooks. 

We  report  here  on  six  four-year-old  girls  and  their  mothers  as  they  worked  together  on  two 
storybooks.  The  books  were  part  of  the  Living  Books  Series  produced  by  Random  House/Broderbund.  The 
Living  Books  were  ideal  to  study  since  print  versions  of  the  stories  accompany  the  CD-ROM  versions.  Hence 
the  CD-ROM  and  written  versions  of  the  stories  were  comparable  except  for  minor  variations.  The  children 
and  their  parents  varied  widely  in  their  computer  knowledge.  Three  children  had  a computer  in  the  home,  and 
all  of  the  children  had  used  a computer.  The  children  and  parents  were  randomly  assigned  to  read  one  of  two 
stories  on  CD-ROM  and  one  in  a print  version.  The  order  of  reading  was  counterbalanced.  We  examined,  the 
children’s  comprehension,  the  time  on  the  task,  and  examined  differences  in  interaction. 

We  found  that  children  and  parents  spent  more  time  interacting  with  the  CD-ROM  than  the  book 
versions  of  the  stories.  However,  the  activities  afforded  by  the  CD-ROM  did  not  seem  to  enhance 
comprehension  or  vocabulary.  Children  and  parents  simply  did  a great  deal  of  clicking  on  each  page  of  the 
CD-ROM  to  see  the  effects.  For  the  most  part,  the  effects  were  visual,  unrelated  to  the  story,  and  were  done  to 
for  the  ”fun”  of  clicking.  One  child  spent  over  an  hour  clicking  on  every  single  thing  she  could  find  on  a page. 
The  typical  pattern  of  interaction  involved  the  playing  of  the  narration  of  the  text  on  a page,  and  then,  clicking 
on  things  until  either  the  mother  or  child  were  ready  to  move  the  next  page.  Reading  pages  of  the  books 
involved  the  typical  processes  of  book  reading.  The  mother  would  read  the  page  and  sometimes  elaborate  on 


the  story  or  vocabulary  relating  to  the  child’s  previous  experience.  Typically,  the  book  reading  took 
approximately  10  minutes,  whereas  the  CD-ROM  interaction  lasted  a minimum  of  30  minutes. 

Although  based  on  a small  sample,  these  findings  raise  some  important  issues  about  the  role  of  CD- 
ROM  storybooks  in  promoting  emergent  literacy.  One  element  of  the  interactions  with  the  CD-ROM's  was  the 
propensity  for  dyads  to  focus  on  clicking  to  see  ’’effects".  To  the  extent  that  clicking  might  introduce  the  child 
to  new  words,  help  facilitate  comprehension  of  the  story,  or  simply  create  a positive  shared  interaction  with  a 
parent,  there  would  seem  to  be  clear  benefits  to  enhancing  interest  in  stories  and  reading.  Alternately,  there 
were  times  that  the  clicking  became  the  primary  purpose  of  the  interaction.  While  all  of  the  four  year  olds  in 
the  study  could  correctly  remember  the  story  events,  younger  children  who  have  not  had  much  experience  with 
storybook  reading  may  comprehend  less  because  of  the  change  in  task  focus.  Additionally,  to  the  extent  that 
clicking  on  something  does  not  provide  anything  but  a visual  effect,  the  potential  for  learning  vocabiilary  is 
limited.  Two  examples  from  one  book  used  in  this  study.  The  Tortoise  and  the  Hare  (Random 
House/Broderbund,  1993),  illustrate  this  issue.  On  page  two  of  the  book,  clicking  on  various  animals  leads 
them  to  play  musical  instruments  or  sing.  The  animals  are  unrelated  to  the  story,  and  unless  a parent  works  on 
naming  of  the  different  animals,  there  is  little  in  the  way  of  vocabulary  learning.  For  example,  a parent  might 
point  to  the  squirrel  and  say  "let’s  click  on  the  squirrel".  Their  use  of  the  word  is  facilitative  of  vocabulary,  but 
because  it  is  not  really  relevant  to  the  story  context,  it  might  not  be  as  well  learned.  There  is  nothing  to  anchor 
the  squirrel  to  in  the  story.  Additionally,  the  animals  singing  and  dancing  have  the  potential  to  distract 
children  from  the  story  and  hurt  their  comprehension.  Page  six  contains  a more  functional  click  with 
references  to  the  Hare's  different  "moves"  as  he  runs  the  race.  The  page  lists  and  illustrates,  with  a click,  the 
various  ways  of  moving  (e.g.,  skipping,  sprinting,  etc.).  The  clicking  affords  links  to  vocabulary  that  are 
related  to  the  story  and  thus  seems  more  facilitative  of  interactions  that  could  increase  vocabulary  and 
comprehension.  Hence,  our  analysis  suggests  both  potentially  positive  and  negative  effects  of  using  CD-ROM 
animated  storybooks. 

An  additional  issue  concerning  CD-ROM  storybooks  is  their  potential  effects  on  the  motivation  to 
read  among  children.  Do  CD-ROM  storybooks  increase  or  decrease  motivation  to  read?  Children  and  parents 
working  together  on  a CD-ROM  version  of  a story  seem  to  spend  more  time  working  together  than  when  a 
parent  reads  a story.  The  children  spend  more  time  on  task,  and  therefore,  the  parents  have  a greater 
opportunity  to  enhance  literacy  development.  The  special  effects  and  animation  provide  a joint  context  that  can 
facilitate  parental  mediation  of  language  and  literacy  development.  However,  the  presence  of  animation  and 
special  effects  may  make  text  versions  of  stories  less  interesting  for  some  children.  The  only  evidence  we  have 
for  addressing  this  issue  is  our  analysis  of  our  own  daughters'  work  with  CD-ROM  storybooks  starting  at  about 
age  three  years  and  continuing  to  the  present  (they  are  now  six  years  of  age).  While  they  request  the  CD-ROM 
books  from  time  to  time,  they  are  far  more  interested  in  print  text.  Print  text  still,  as  many  have  pointed  out,  is 
far  more  convenient  and  portable.  However,  our  children  may  be  the  exception  rather  than  the  rule  in  the  sense 
that  both  of  their  parents  are  well  educated  and  began  reading  storybooks  to  them  very  early  in  their 
development.  Younger  children  and  children  whose  parents  have  put  less  emphasis  on  storybook  reading  may 
differ  in  their  motivation  to  work  with  CD-ROMs  or  print  storybooks.  Also,  the  sample  included  only  girls  and 
their  mothers.  Hence,  there  may  be  some  gender  differences  to  consider. 

Overall,  the  data  suggest  that  although  CD-ROM  story  books  can  be  used  like  print  storybooks  as  a 
tool  for  developing  language  and  literacy  by  parents,  there  are  some  differences  in  how  CD-ROM's  are  used 
and  some  drawbacks  to  CD-ROM  storybooks  as  they  are  presently  designed.  We  would  like  to  see  more 
functionally  designed  storybooks  that  take  include  more  functional  "clicks"  that  are  linked  to  the  story  rather 
than  clicks  that  are  simply  distract  and  entertain. 
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Introduction 

The  University  of  Alberta  (U  of  A)  Faculty  of  Education  has  undertaken  a restructuring  process  with  the  goal  of 
better  preparing  teachers  to  effectively  integrate  technology  into  their  teaching.  This  paper  describes  the  authors’ 
participation  in  one  project  which  is  a component  of  this  renewal  process:  the  development  of  a teacher  education 
course  offering  an  introduction  to  educational  media  and  instructional  technology.  This  course  emphasizes  a 
constructionist  approach  to  problem-solving  with  multimedia  tools. 

Although  it  is  still  important  to  understand  how  technology  can  be  used  to  assist  in  the  development  and 
presentation  of  instructional  material,  one  can  also  think  about  how  students  in  grades  K-12  can  use  technology 
themselves  to  construct  a learning  environment.  The  idea  is  simple,  children  construct  their  own  multimedia  material 
and  in  doing  so  "learn  by  doing."  The  children  become  multimedia  developers  and  in  the  process  they  gain  both 
content  and  process  knowledge.  This  approach  is  only  made  possible  by  the  continuous  development  of  more 
powerful  microcomputers  and  easy  to  use  tools  for  constructing  multimedia  software.  For  this  approach  to  work, 
teachers  themselves  must  be  familiar  with  the  current  technology  that  can  be  used  to  create  multimedia  material. 
Furthermore,  teachers  must  also  know  how  to  obtain  and  organize  content  material,  and  how  to  design  and  develop  a 
multimedia  application. 


Course  Design 

This  course  has  two  streams,  both  of  which  run  concurrently.  One  stream  emphasizes  a survey  of  the  existing 
technologies  that  can  be  used  to  create  multimedia  material.  This  is  the  pragmatic  side,  i.e.,  getting  your  hands  dirty. 
For  example,  students  learn  simple  things  like  how  to:  (a)  scan  a picture  into  the  computer,  (b)  record  audio  and 
video  into  the  computer,  (c)  use  graphic  tools,  (d)  use  Computer  Mediated  Communication  (CMC)  tools,  including 
the  course  website  (http://www.quasar.ualberta.ca/edpy485dev),  and  (e)  use  a multimedia  authoring  system 
(HyperStudio).  As  byproducts  of  learning  these  tools  they  discover  their  potential  for  the  development  and 
presentation  of  instructional  material.  The  second  stream  involves  the  student  teacher  acting  the  part  of  a K-12 
student,  representing  knowledge  by  developing  a series  of  small,  theme-based  multimedia  projects. 


Theoretical  Course  Design  Considerations 

Two  important  theoretical  considerations  concerning  the  role  of  computer  technology  in  teacher  education 
greatly  influenced  the  course  design.  The  first  of  these  is  the  importance  of  early  technology  learning.  Logan  (1995) 
postulates  that  there  are  five  languages  presently  being  used  to  communicate  and  process  information:  speech, 
writing,  mathematics,  science,  and  computing.  In  the  course  of  history,  each  language  evolved  from  its  predecessors 


due  to  the  need  to  address  more  complex  information  processing  problems  that  were  associated  with  economic  and 
cultural  advancement.  Currently,  the  first  four  levels  of  language  are  integrated  from  early  education  onward. 
However,  Logan  argues  that  most  students  do  not  have  the  opportunity  to  learn  the  fifth  level  properly  or  early 
enough. 

The  second  theoretical  consideration  was  that  computer-based  tools  and  learning  environments  (e.g. 
databases,  spreadsheets,  semantic  networks,  computer  conferencing,  hypermedia  construction,  or  microworld 
environments)  can  serve  as  cognitive  tools  or  extensions  of  the  mind.  With  the  use  of  such  "Mindtools"  (Jonassen, 
1 996),  learners  can  enter  an  intellectual  partnership  with  the  computer,  in  order  to  access  and  interpret  information, 
and  organize  personal  knowledge.  This  moves  in  a very  different  direction  from  traditional  computer  use  in 
education.  In  the  past,  the  emphasis  was  on  computer  literacy  (learning  about  computers)  and  computer  assisted 
instruction  (learning  from  computers).  The  application  nf  "Mindtools"  involves  students  learning  with  computers 
(Jonassen,  1996). 


Conclusions 

Multimedia  projects  can  improve  educational  outcomes  by  enabling  students  to  represent  information  using 
several  different  media,  to  link  and  organize  information  in  many  meaningful  ways,  to  draw  knowledge  and 
experiences  from  a wide  variety  of  sources,  to  practice  problem-solving  and  decision-making,  to  achieve  high  self- 
esteem and  to  become  more  self-directed.  Such  projects  include  a wide  variety  of  activities  and  skills  which  groups 
can  work  on  effectively  over  an  extended  timeframe.  They  involve  a substantial  amount  of  work,  open-ended 
assignments,  theme-based  activities  and  interdisciplinary  activities.  Multimedia  projects  require  and  reward  good 
planning  and  execution  skills  and  students  are  motivated  to  create  a quality  product  because  a wider  audience  may 
view  it  (Agnew  et  al.,  1996). 

As  student  feedback  indicates,  the  constructive,  collaborative,  student-centred  nature  of  this  course  has 
proved  to  be  a very  engaging  learning  environment.  Most  pre-service  and  in-service  teachers  indicate  that  they  feel 
more  confident  about  integrating  technology  into  their  teaching  after  completing  this  course.  Course  observations 
have  produced  support  for  the  use  of  information  technology  as  a cognitive  tool  in  today’s  classrooms.  Applications 
such  as  hypermedia  projects  or  CMC  appear  to  have  the  potential  to  more  powerfully  impact  student  learning  than 
traditional  uses  such  as  drill-and-practice.  "Mindtools"  elevate  educational  computing  from  addressing  only  simpler 
skills  such  as  memorization  into  the  realm  of  complex  reasoning  processes  such  as  synthesis  and  evaluation. 
Hopefully  this  course  furthers  the  "mindtools"  cause  by  providing  teachers  with  a glimpse  of  how  information 
technology  can  contribute  to  the  creation  of  engaging  and  relevant  learning  environments. 
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Introduction 

The  scope  of  the  definition  of  educational  technology  has  changed  along  with  the  development  of 
advanced  technology  over  the  years.  Many  educational  technology  centers  at  universities  were  only  used  to 
serve  as  warehouses  for  distribution  of  audio  and  video  media  in  the  old  times.  With  the  diversity  of  today’s 
computer  technology,  a modem  educational  technology  center  at  a university  setting  can  offer  services  to  cover 
ED-MEDIA  and  even  ED-TELECOM,  in  addition  to  its  traditional  roles.  The  purpose  of  this  paper  is  to 
reexamine  the  dimension  of  educational  technology  for  the  mission  a university  technology  center  can  carry. 

New  Role  of  a University  Technology  Center 

The  rapid  progress  of  the  use  of  technology  in  education  has  made  major  changes  in  the  ways  that 
students  learn  and  teachers  teach.  To  create  an  enriched  learning  and  teaching  environment  to  meet  the 
challenges  resulting  from  the  development  of  the  advanced  educational  technology,  a university  technology 
center  can  not  serve  solely  as  a distribution  center  for  teaching-aid  materials  but  need  to  extend  its  mission  to 
provide  the  necessary  organizational  infrastructure,  hardware  and  software  support  for  the  integration  of 
technology  across  the  curriculum  and  to  offer  pedagogical  and  instructional  training  for  faculty  and  staff 
members  to  use  new  technology  to  enhance  their  teaching  or  daily  operations  (Georgia  Southwestern  State 
University,  Technology  Committee,  1998).  Furthermore,  the  center  must  also  assist  faculty  and  staff  in 
developing  positive  attitudes  towards  new  technology,  add  new  strategic  visions  for  technology  to  the  campus 
strategic  plan,  and  provide  consultation  and  active  involvement  to  support  the  movement  as  an  active 
participant  in  the  global  electronic  community. 

Decentralization  of  Instructional  Technology 

Modem  instructional  technology  involves  a great  deal  of  communication  infrastructure,  computer 
hardware  and  peripherals,  communication  and  application  software,  and  presentation  equipment.  As  a result, 
the  complexity  of  providing  efficient  and  effective  services  to  faculty  and  staff  members  has  dramatically 
increased  and,  therefore,  approaches  to  establishing  a foundation  and  to  building  a technology  infrastructure 
must  be  carefully  studied.  There  are  basically  two  approaches:  a centralized  approach  and  a decentralized  one. 
In  a centralized  environment,  there  is  normally  a single  unit  in  the  university  that  manages  the  operations  and 
controls  the  access  to  the  resources.  This  approach  works  well  in  a traditional  setting  since  it  does  not  require 
a great  deal  of  diversified  expertise  to  provide  the  service  and  the  demand  of  flexibility  is  not  high.  In  a 
rapidly  changing  high-tech  field  such  as  the  current  instructional  technology,  however,  the  diversity  of 
expertise  and  the  demand  for  flexibility  in  providing  the  service  have  made  the  centralized  approach  less 
efficient.  Due  to  the  complexity  of  the  existing  and  forthcoming  new  technology,  we  feel  that  decentralization 
of  the  basic  technology  is  vital  to  the  success  of  the  operation.  To  achieve  this  goal,  basic  instmctional 
technology  stations  consisting  of  VCR’s,  TV’s,  monitors  and  projectors  should  be  distributed  and  put  directly 
in  the  hands  of  faculty  and  staff  to  make  the  use  of  the  technology  on  campus  flexible.  This  decentralization 
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can  also  be  applied  to  include  computers  and  networking  equipment  to  allow  for  easy  access  to  outside  world 
via  the  Internet. 

Use  of  Satellite  Technology,  TV  Broadcasting,  and  Distance  Learning  Programs 

Satellite  technology  has  found  several  important  applications,  including  television  distribution,  long- 
distance telephone  transmission  and  private  data  networks.  Currently,  there  are  many  national  and  state 
educational  programs  and  live  teleconferences  broadcast  via  satellites.  The  Public  Broadcasting  Service  (PBS), 
for  example,  provides  many  episodes  for  “Technology  in  the  Classroom”,  “Next  Steps  in  Technology”,  “GED 
Classroom”,  “Classroom  Contact”,  and  “Real  Science”,  just  to  name  a few.  It  is,  therefore,  very  desirable  to 
have  downlink  capability  to  receive  those  programs  over  the  air  to  enrich  educational  resources,  tc  assist 
teaching  in  the  classroom,  and  to  enhance  faculty  and  staff  development.  On  the  other  hand,  maintaining  and 
operating  a TV  station  on  campus  offers  the  ability  for  a university  to  provide  broadcasting  services,  including 
educational  and  entertaining  programs  downlinked  from  the  satellite  system,  to  not  only  the  students,  staff,  and 
faculty  on  and  off  campus,  but  also  the  neighboring  communities.  It  can  also  be  used  to  train  students  with 
mass  communication  major  to  gain  experience  in  the  production,  programming,  video  and  audio  editing,  and 
broadcasting  of  TV  programs. 

Made  feasible  by  the  availability  of  video  conferencing  networks  and  the  Internet  technology,  distance 
learning  has  several  advantages  over  traditional  classroom  learning,  such  as  flexibility,  convenience,  and 
adaptability.  It  is  flexible  because  learners  can  take  lessons  at  the  time  they  feel  most  comfortable.  It  is 
convenient  because  taking  lessons  is,  to  a great  extent  , geography-independent.  It  is  adaptable  because 
learners  can  adjust  their  learning  speed  and  path  based  on  their  progress.  These  advantages  are  especially 
apparent  if  the  learning  takes  place  on  the  Internet  using  the  WWW  technology.  Although  there  are  still 
arguments  on  the  definition  of  distance  learning,  distance  learning  has  already  had  a impact  on  the  way 
instruction  is  designed  and  delivered  and  will  continue  to  affect  the  filed  in  a significant  way  ( Reiser,  Ely, 
1997).  Accordingly,  it  has  become  essential  for  an  instructional  technology  center  to  establish  distance 
learning  programs  using  both  video  conferencing  networks  and  the  Internet.  This  involves  not  just  establishing 
distance  learning  classrooms  for  the  students,  staff,  and  faculty  to  use,  but  providing  the  capability  and 
environment  for  faculty  members  to  develop  their  course  material  for  distance  learning  and  to  conduct  online 
examinations,  online  survey,  and  online  enrollment  (Tu,  Babione,  and  Chen,  1998)  over  the  Internet. 

Conclusions 

Judging  from  the  rapid  progress  in  the  computer  hardware,  software,  and  applications,  in  the 
communications  networking  technologies,  and  in  the  development  of  WWW,  it  is  not  difficult  to  envision  that 
educational  technology  will  continue  to  grow  and,  very  likely,  lead  us  to  an  even  newer  world  in  the  future.  If 
we  are  the  practitioners  in  this  field,  what  we  need  to  do  is  to  open  our  eyes  for  the  new  trends  as  long  as  it 
opens  some  avenue  for  teaching  and  learning.  There  is  no  need  for  us  to  limit  ourselves  to  just  what  are 
available  today. 
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Introduction 

Most  of  the  recent  research  investigating  the  effects  of  multimedia  on  student  writing  has  converged 
on  the  computer’s  capability  to  process  text  (Hartley,  1993).  And,  to  date,  much  of  this  research  has  centered  on 
the  use  of  word  processors  to  transpose  text  or  on  the  use  of  style  checker  applications  such  as  Editor's 
Assistant  (Dale,  1990).  Although  new  programs  continue  to  be  developed,  development  and  use  of  multimedia 
or  computer  applications  have  not  kept  pace  with  emerging  cognitive  theories  about  composition  as  a goal- 
directed,  problem-solving  activity  (Flowers  & Hayes,  1981,  1977;  Scardamalia  & Bereiter,  1986). 

Most  of  these  modem  composition  theories  encourage  the  use  of  some  version  of  prewriting  activities 
such  as  planning,  translating,  and  reviewing.  Often  to  aid  the  students  with  these  pre writing  activities,  heuristics 
are  incorporated  into  the  instruction.  These  strategic  exercises  ask  students  to  envision  solving  specific 
communication  problems  (Flower  & Hayes,  1981).  However,  little  research  using  multimedia  for  teaching 
pre  writing  skills  as  a heuristic  can  be  found  in  the  literature.  Therefore,  these  researchers  speculated  whether 
teaching  such  heuristics  via  a multimedia  environment  would  be  successful  and  for  which  type  of  learner. 

These  learners  vary  according  to  learning  styles,  which  can  be  an  important  variable  in  cognitive 
processes  and  perceptions  and  use  of  heuristics  (Rasmussen  & Davidson-Shivers,  1998;  Davidson,  1990;  Kolb, 
1984).  Kolb’s  four  learning  styles  are  based  on  active  or  reflective  processes  and  concrete  or  abstract 
perceptions  when  learning.  Divergers  tend  to  use  concrete  conceptualization  and  are  reflective  processors; 
Accommodators  rely  on  the  concrete  concepts,  but  are  active  processors;  Assimilators  rely  on  abstract  concepts 
and  are  reflective  processors;  and  Convergers  also  use  abstractions,  but  are  active  processors  of  information.  In 
addition,  recent  studies  using  various  structures  within  multimedia  environments  for  various  learners  have 
yielded  mixed  results  (Rasmussen  & Davidson-Shivers;  Lin  & Davidson,  1996;  Kinzie  & Berdel,  1990). 
Rasmussen  and  Davidson-Shivers  reported  a significant  difference  in  the  interaction  of  structure  that  allowed 
various  learner  control  and  learning  styles  on  performance  as  did  Lin  and  Davidson  in  their  study  on  structure 
and  cognitive  styles.  However,  other  studies  have  not. 

Therefore,  using  a multimedia  lesson  on  pre  writing  heuristics,  the  purpose  of  this  study  was  to 
investigate  learning  styles  and  multimedia  structure  (random  and  hierarchical)  on  writing  performance  of 
undergraduates.  Of  primary  interest,  the  researchers  asked  does  either  structure  affect  composition  quality  and 
do  learning  styles  influence  writing  performance?  As  a secondary  focus,  the  researchers  collected  data  on 
student  demographics,  writing  and  computer  skills,  attitudes  as  well  as  time  spent  on  the  lesson. 

Methods 

Subjects 

Forty-seven  students  in  an  undergraduate  composition  course  at  a regional  university  in  the  Southeast 
participated.  There  were  equal  numbers  of  males  and  females  with  the  majority  of  them  being  freshmen. 

Materials 

The  researchers  designed  a multimedia  lesson  on  the  pre  writing  heuristics  of  brainstorming  and 
outlining  for  developing  a written  composition.  This  lesson  was  developed  using  Authorware  and  Windows- 
based  computers.  The  content  and  instructional  strategies  were  the  same  for  both  treatments.  However,  the 
lesson  had  two  versions  based  on  how  the  lesson  was  structured.  The  hierarchical  structure  (HS)  allowed 
students  to  move  through  the  lesson  in  a sequenced  manner  while  the  random  structure  (RS)  allowed  students  to 
choose  any  parts  of  the  lesson  they  wished  to  see  and  in  any  order.  Both  versions  of  the  lesson  allowed  the 
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individual  to  spend  as  much  time  on  any  given  section  as  they  desired.  The  multimedia  program  was  designed 
to  collect  data  on  the  amount  of  time  each  individual  spent  on  the  lesson  and  its  subsections  for  both  RS  and  HS 
treatment  versions  as  well  as  to  indicate  the  order  of  the  path  chosen  by  those  in  the  RS  treatment  version. 

Procedures 

Prior  to  the  study,  students  completed  a demographic  survey  and  an  attitudinal  questionnaire  about . 
their  computer  and  writing  skills.  They  also  completed  the  Kolb's  (1981,  1984)  Learning  Style  Inventory. 
Students  were  then  randomly  assigned  to  one  of  the  two  treatment  lessons.  At  the  beginning  of  the  lesson,  the 
researcher  read  written  directions  stating  that  three  in-class  assignments  would  be  given:  brainstorming  a list  of 
ideas  for  a composition,  outlining  based  on  the  brainstorming,  and  composing  an  essay.  Students  were  able  to 
work  at  their  own  pace  using  the  multimedia  lesson  during  the  regularly  scheduled  class  periods  for  one  week, 
and  were  told  their  compositions  would  be  due  at  the  end  of  the  last  class  period  and  would  be  graded. 

Results 

Researchers,  having  been  trained  on  a rubric  for  grading  quality  in  writing,  critiqued  and  scored  the 
compositions  (with  an  interrater  reliability  of  .70),  Results  of  the  study  indicated  no  significant  differences  in 
performance  between  the  two  treatments;  however,  the  RS  group  completed  the  lesson  faster  than  the  HS  group 
without  a qualitative  difference  in  their  compositions. 

Although  not  statistically  significant,  there  was  a trend  toward  the  Divergers  writing  better 
compositions  than  the  other  three  styles.  In  comparison,  Accommodators  did  less  well  on  performance; 
furthermore,  their  attitudes  about  the  lesson  and  the  writing  assignment  were  less  positive  than  the  other  styles; 
again  this  finding  did  not  reach  statistical  significance. 

Discussion  of  Results 

Although  the  results  of  the  study  were  not  as  anticipated  by  the  researchers,  they  may  be  of  importance 
with  regard  to  using  multimedia  for  teaching  heuristics  about  writing.  First,  even  though  type  of  structure  did 
not  achieve  significant  difference,  the  RS  subjects  completed  the  heuristics  lesson  faster  than  the  other  group. 
This  may  suggest  that  students  who  select  their  pathway  through  the  lesson  may  complete  it  more  quickly 
without  the  loss  of  writing  quality.  Additional  research  is  needed  to  support  these  findings  as  well  as  investigate 
the  relationship  between  prior  writing  experience  and  prewriting  skills  instruction.  Additional  research  is  also 
needed  to  verify  whether  learning  styles  influence  writing  performance  and  attitudes  toward  writing. 
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Introduction 

Southern  Cross  University  offers  a Bachelor  of  Multimedia  program  to  provide  students  with  comprehensive 
study  in  the  field  of  multimedia,  taking  advantage  of  specialisations  within  teaching  units  to  consolidate  the 
multi-disciplinary  nature  of  interactive  multimedia.  On  completion  of  the  course,  students  would  be  expected  to 
have  (a)  developed  comprehensive  skills  and  knowledge  in  the  foundations  of  interactive  multimedia,  (b) 
demonstrated  skills  in  the  design  and  development  of  multimedia  applications  incorporating  digital  media;  and 
(c)  undertaken  a major  study  in  one  area  of  the  multimedia  field.  The  locations  http://edmm.scu.edu.au/  and 
http://multimedia.scu.edu.au  identify  the  teaching  and  marketing  web-sites  of  the  multimedia  program.  There 
were  several  reasons  for  moving  towards  on-line  delivery  of  multimedia  content,  including  an  increased  focus  on 
international  delivery  of  courses,  a large  cohort  of  mature  and  part-time  students,  changing  socio-economic 
conditions  and  an  overall  commitment  towards  more  flexible  delivery  within  the  teaching  unit,  the  School  of 
Multimedia  and  Information  Technology. 

With  the  overall  goal  of  being  able  to  offer  the  full  undergraduate  multimedia  program  in  a flexible  mode  of 
study  from  2000,  a phased  program  of  preparation  for  on-line  delivery  was  commenced  during  Semester  1,  1998. 
The  initial  phase  of  this  process  involved  introducing  the  on-campus  students  to  on-line  access  of  materials  by 
providing  the  majority  of  unit  resources  and  teaching  materials  on  the  web,  even  though  classes  continued  to  be 
delivered  in  the  traditional  internal  (face-to-face)  mode.  Additional  preparation  for  the  task  of  adopting  an  on- 
line strategy  consisted  of  informal  interviews  with  students  to  discuss  the  structure  that  would  be  most  beneficial 
to  their  learning,  assessment  of  other  similar  implementations,  a review  of  similar  teaching  sites  already 
implemented  and  a review  of  the  overall  strategy  of  the  multimedia  program. 


Design 

A structure  for  the  site  was  designed  to  include  a news  area,  unit  statements,  staff  contact  details,  the  teaching 
timetable  and  links  to  each  unit  being  taught.  For  the  individual  units,  information  included  a semester  outline, 
links  to  web-based  readings  and  library  resources,  lecture  notes  and  weekly  exercises,  on-line  tutorial  material 
and  assessment  specifications.  One  of  the  design  parameters  was  to  restrict  access  to  those  students  enrolled  in 
the  program.  The  two  main  options  available  were  to  restrict  access  by  network  address  or  by  username  and 
password;  initially  the  username  and  password  option  was  ignored  because  of  the  potential  administration 
problem  of  maintaining  password  lists.  However,  after  determining  that  students  could  conceivably  wish  to 
access  the  site  from  anywhere  in  the  world,  making  network  address  restrictions  impractical,  the  second  option 
was  chosen.  Fortunately  the  web  server  being  used  (Novell  Web  Server  3.1)  allowed  the  securing  of  web  sites 
using  existing  IntraNetware  account  information,  which  has  to  date  worked  well  in  practice. 


1417 


The  design  of  the  web  pages  themselves  was  driven  by  several  main  goals,  chiefly  the  desire  to  make  a site  that 
was  easy  to  navigate  and  pages  that  were  fast  to  download  over  a modem.  The  wide  range  of  client  machines, 
operating  systems  and  web  browsers  that  could  be  used  to  view  the  site  meant  that  the  HTML  code  had  to  be 
simple  and  straightforward,  avoiding  browser-specific  tags  and  scripting  languages.  Many  of  the  conclusions  of 
Nielsen  relating  to  site  navigation  and  page  design  were  found  to  be  very  useful  in  achieving  these  goals 
(Nielsen  & Sano,  1994;  Nielsen,  nd).  The  resulting  site  features  minimal  use  of  graphics,  with  techniques  such  as 
table  background  colours  being  used  instead.  Pages  are  identified  with  the  name  of  the  author,  the  date  first 
published  and  the  date  of  the  most  recent  modification  to  indicate  to  students  when  material  has  changed.  An 
hierarchical  navigation  system  at  the  top  of  each  page  indicates  where  in  the  site  structure  the  student  is  currently 
browsing,  with  links  on  the  left-hand-side  to  the  top  of  each  significant  area  of  the  site.  Links  to  pages  that  are 
within  the  current  site  structure  are  marked  with  blue  arrows,  while  links  to  other  web  sites  are  marked  with  red 
arrows;  links  to  non-electronic  resources  are  marked  in  black. 


Development  and  Assessment 

Initial  production  of  the  site  was  implemented  using  a template-based  publishing  system  (Userland  Frontier), 
which  enabled  rapid  development  of  the  structure  of  the  site.  Because  the  overall  look  of  the  pages  is  governed 
by  these  templates,  and  the  content  of  the  pages  is  stored  separately,  changes  to  the  whole  site  can  be 
implemented  quickly  and  easily.  The  original  web  site  incorporated  most  of  the  features  of  the  original  design, 
the  major  omission  being  the  news  area.  As  electronic  mailing  lists  were  already  heavily  used  within  the 
multimedia  program  to  communicate  with  students  and  disseminate  news,  it  was  decided  not  to  create  yet 
another  place  for  news  to  be  published,  and  instead  to  use  the  web  site  to  archive  postings  to  the  mailing  lists. 
The  archival  feature  is  to  be  incorporated  into  the  next  major  revision  of  the  site. 

Initial  assessment  of  student  feedback  to  the  site  revealed  overall  positive  results.  In  a rating  scale  from  0-5,  all 
major  factors  were  rated  favourably,  with  the  possible  exception  of  the  On-Time  factor.  This  highlights  one  of 
the  major  issues  confronting  the  development  of  web-sites  in  general  and  on-line  learning  sites  in  particular,  that 
of  regular  and  up-to-date  maintenance.  Once  the  student  cohort  becomes  entrenched  in  using  on-line  materials, 
any  delay  in  access  is  noted  almost  immediately! 

Overall,  the  Multimedia  On-Line  site  has  been  used  as  the  primary  information  resource  for  most  of  the  twenty 
units  taught  during  the  first  and  seconds  semesters  of  1998.  While  there  were  some  difficulties  in  producing  and 
publishing  the  information  in  a timely  fashion,  many  of  these  were  related  to  a lack  of  suitable  writing  tools  for 
on-line  publishing.  The  traditional  software  packages  used  for  writing  have  many  problems  when  they  are  used 
to  write  material  intended  for  on-line  delivery,  but  as  this  is  an  area  of  significant  change,  it  is  anticipated  that 
such  tools  will  mature  over  time. 
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Abstract:  This  paper  describes  in  detail  how  an  IT-based  course  is  designed  and  delivered  using  the 
latest  information  technology  (IT)  in  Hong  Kong.  Based  on  Lotus  Notes,  an  electronic  learning 
environment  called  ELE  is  developed  and  used  in  an  IT-based  course  as  an  alternative  for  delivering 
distance  education.  The  ELE  runs  on  both  Windows  and  Macintosh,  caters  for  communication, 
collaboration  and  course  co-ordination  purposes,  and  acts  as  a transportation  vehicle  for  distributing 
course  materials.  Evaluations  of  the  ELE  are  conducted  throughout  the  course.  Very  encouraging 
results  are  obtained. 


Introduction 

The  Open  University  of  Hong  Kong  (OUHK),  formerly  the  Open  Learning  Institute  of  Hong  Kong  (OLI),  is  the  only 
distance  learning  university  in  Hong  Kong.  Currently,  most  OUHK  courses  are  print-based.  Communication  between 
students  and  tutors  is  carried  out  through  telephone  and  correspondence,  and  assignments  are  submitted  and  returned  by 
mail.  A number  of  initiatives  along  IT  strategy  in  distance  learning  were  started  at  the  OUHK  in  the  past  years.  This 
paper  reports  the  first  experience  of  online  course  teaching  and  learning  in  the  OUHK. 

The  Aims  of  this  Research 

In  1 997  an  OUHK  research  project  was  initiated  with  the  aim  of  designing,  developing,  delivering  and  evaluating  an 
electronic  delivery  system  for  distance  students  in  Hong  Kong.  It  is  expected  that  this  system  not  only  provides  an 
alternative  for  the  delivery  of  course  material  but  also  fosters  a better  environment  of  communication  and  collaboration 
between  students,  their  tutors  and  their  peers. 

Research  Methodology 

An  IT-based  course  is  selected  for  pilot  purposes.  Students  are 
first  asked  to  opt  for  the  ^ electronic^  or  "print'  version  of  the 
course.  Sixty  (out  of  120)  students  opted  to  take  part  in  the  pilot 
study  and  they  are  divided  into  two  tutorial  groups  with  30 
students  each.  During  the  course,  they  studied  the  course  online 
and  did  not  receive  any  printed  materials.  Three  electronic 
questionnaires  are  sent  to  students  on:  (a)  background  and 
initial  reaction  to  the  electronic  course  delivery  system,  (b) 
usage  characteristics  and  problems  encountered,  and  (c) 
summative  evaluation  at  the  end  of  the  course.  Interviews  are 
also  conducted  with  a sample  of  students  and  tutors. 

The  IT-based  Course 

The  course  selected  for  the  project  is  B32J  Advances  in 
Information  Technology,  which  aims  to  develop  students’ 
knowledge  and  understanding  of  current  and  emerging  IT,  and 
their  capability  to  identify,  evaluate  and  apply  these  IT  products 
to  appropriate  business  situations.  Topics  include  OA 
(Document  Management  and  Groupware);  DBMS;  00  Visual 
Development  Environments;  Networking;  Open  Systems  and 

C/S  Computing;  the  Internet  and  Electronic  Commerce;  and  Knowledge  Work  Support  Systems. 


The  ELE  Architecture 

The  electronic  course  delivery  system  developed  (based  on  the  Lotus  Notes  R4.5)  for  this  project  is  called  the 
Electronic  Learning  Environment  (ELE).  The  server  is  a Pentium-based  PC,  featured  with  64M  RAM  and  2GB  hard- 
disk space  (see  the  network  infrastructure  of  the  ELE).  The  backup  server,  also  configured  with  Lotus  Notes  R4.5  and 
connected  to  the  ELE  server  via  a fast  Ethernet,  runs  an  agent  program  to  replicate  all  the  ELE  databases  every  30 
minutes.  In  case  of  systems  failure,  the  latest  backup  databases  can  be  reloaded  to  recover  the  ELE  server.  The  ELE  has 
been  further  developed  to  allow  Internet  access  with  proper  authentication.  The  ELE  components  include  Course 
Materials,  Assignments,  Discussion,  Information,  Evaluation,  and  Personal  Folder. 
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Major  Findings 

Student  Background  - of  the  60  students,  46%  are  married.  The  majorities  of  the  students  have  sub-degree 
qualifications,  and  are  working  in  the  technical  (35%),  administrative  (18%),  and  business  (18%)  professions,  with 
monthly  salary  of  US$2,000  or  above  (68%).  They  have  access  to  a desktop  computer,  with  Pentium  or  higher 
processor  running  with  MS  Windows  95,  more  than  16  hours  per  week.  All  students  are  equipped  with  at  least  one 
telephone  line  at  home  and  30%  of  them  have  two.  Overall,  the  students  found  the  Lotus  Notes  and  ELE  installations 
easy  and  straightforward.  Some  minor  problems  were  encountered  initially  but  they  were  manageable. 

Usage  Characteristics  and  Problems  Encountered  — although  the  majority  of  students  found  it  more  effective  to 
print  out  the  course  materials  to  study,  a few  students  generally  preferred  to  study  on-screen.  Interestingly,  not  many 
students  posted  but  frequently  read  messages  in  the  Discussion  forum.  The  arrangement  of  the  assignment  submission 
electronically  is  frequently  praised.  Overall,  students  found  the  ELE  system  easy  to  use,  and  preferred  ELE  to  the  print- 
based  mode. 

— ..  - ..  .*  . 1 . t • 1-  . .1  .1.  .1 * ] -j onn/  ...uii... 

^iummanve  evaluation  — me  proporaon  or  siuuenis  wnu  iiiuicaieu  inm  mcy  cnjuycu  siuuyni^  me  la  o.^./o  wimu 

another  18%  of  students  indicated  that  they  are  neutral  or  of  mixed  option.  The  electronic  course  features  that  students 
liked  most  the  electronic  course  are  ‘Email’,  ‘Discussion  forum’,  ‘Electronic  assignment  submission’,  ‘Search  engine’, 
and  ‘Advances  in  IT’.  While  the  features  that  they  liked  least  the  electronic  course  are  ‘Computer  dependence’,  ‘On- 
screen study’,  and  ‘Printing  course  materials’  respectively.  There  are  5 assignments  and  a final  examination  for  this 
course.  The  electronic  mode  students  performed  as  well  as  the  printed  mode  students.  In  particular,  they  performed 
better  in  terms  of  the  number  of ‘Pass  1’  and  ‘Pass  3’,  and  the  overall  passing  rate  in  the  course. 

Discussion 

The  ELE  offers  better  assignment  submission  process  - it  is  validated  that  the  ELE  has  helped  a great  deal  in  the 
assignment  tracking,  submission  as  well  as  the  due  date  extension  approval  process.  This  shortens  the  assignment 
submission  cycle  from  2-3  weeks  to  a 1-2  hour. 

The  ELE  acts  as  a transportation  vehicle  for  distributing  course  materials  - as  an  extension  of  the  study  materials, 
it  provides  learners  with  pointers  (such  as  URLs),  presentations,  images,  audio/video  clips,  animations, 
exercises/workshops,  simulated  laboratory  environment,  etc.  which  are  almost  impossible  in  traditional  print-based 
materials.  As  an  information  delivery  mechanism,  it  facilitates  assessment,  discussion,  Q&A,  etc.  It  has  also  enhanced 
offline  communication  on  administrative  and  ad-hoc  matters. 

The  ELE  is  used  to  enhance  distance  education  - instead,  the  ELE  should  not  be  aimed  at  replacing  the  traditional 
print-based  study  materials  in  the  sense  that  the  current  display  technology  and  user  interface  do  not  seem  to  offer  a 
better  alternative  to  print-based  study  materials.  Besides,  the  traditional  mode  of  study  is  location-independent  while  an 
access  to  the  computer  (and  occasionally  the  network)  is  required  for  the  ELE.  In  addition,  psychological  barriers  need 
to  be  overcome,  as  the  ELE  does  not  satisfy  the  sense  of  possession  of  the  learned  knowledge  as  everything  exists  in  an 
intangible  form. 

Lotus  Notes  vs.  WWW  - Notes  offers  an  'offline'  communication.  However,  it  is  location-dependency  as  a user 
identification  file  {user. id)  and  the  study  materials,  which  are  usually  stored  inside  a predefined  machine,  are  required. 
WWW  which  offers  a 'real  time  online*  solution  is  also  a good  alternative  platform.  It  is  potentially  more  convenient 
as  no  set-up  is  required  and  students  can  access  the  ELE  server  from  any  location.  According  to  our  research,  students 
are  typically  equipped  with  only  one  telephone  line  at  home.  They  may  need  to  resolve  conflict  regarding  with  other 
family  members  using  the  telephone  line  for  a real  time  access. 

The  ELE  training  and  supports  are  necessary  - a tutor  has  to  know  more  than  the  students  on  the  use  of  the  ELE  as 
he  or  she  is  probably  their  first  contact  for  problems.  Training  for  tutors  becomes  mandatory.  A well-organised 
technical  support  mechanism  providing  support  at  night  time  is  critical  as  most  problems  occurred  at  night  that  is  the 
peak  time  for  students  and  tutors  using  the  ELE  system. 

Conclusion 

In  general,  we  found  that  the  objectives  of  the  research  project  were  achieved.  An  electronic  course  delivery  system  was 
designed,  featured  with  six  major  components:  Course  Materials,  Assignments,  Discussion,  Information,  Evaluation 
and  Personal  Folder.  An  IT-based  course  was  converted  into  Notes  database  document  format  and  delivered  to  students 
electronically.  Very  encouraging  results  are  obtained.  Experiences  gained  in  running  the  online  course  are  useful  in  the 
subsequent  offer  of  other  electronic  courses  in  the  University. 
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Story/Concepts 

A story /concept  is  a concept  that  involves  a situation  rather  than  an  object.  Many  concepts  in  the  social 
sciences  involve  more  than  the  examination  of  a particular  object— rather,  they  involve  complex  actions  and 
events  often  involving  human  actors.  These  concepts  are  the  basis  of  many  social  science  principles.  For 
example,  the  field  of  group  dynamics  uses  the  related  concepts  of  constructive  conflict  and  destructive 
conflict  in  the  analysis  of  group  interactions.  In  teaching  story/concepts,  many  teachers  will  use  "word 
problems"  that  explain  a situation  to  a student  and  asks  the  student  to  classify  the  word  problem. 

Such  a word  problem  might  look  like  following: 

Bill  and  Jim  are  working  on  a project  for  the  Chain  Division  of  General  Bicycle,  Inc.  Bill  suggested  that 
they  use  a titanium  chain  for  the  project,  but  Jim  was  against  the  idea.  Their  dialogue  went  like  this: 

Bill:  "I  think  the  titanium  will  be  more  powerfuU-provide  added  strength.” 

Jim:  ”No-a  thousand  times  no!  You  have  the  worst  ideas-always.  Can't  you  think?” 

Is  this  an  example  of  constructive  or  destructive  conflict?  Why? 

Story/concept  examples  are  time  consuming  to  create;  consequently  a teacher  may  provide  only  limited 
examples.  For  some  students,  a few  examples  may  be  sufficient,  but  others  may  need  additional  practice  in 
classifying  the  concept.  Therefore,  a tool  that  could  help  teachers  create  a large  number  of  story/concepts, 
quickly,  would  be  useful. 


Figure  1.  Components  of  the  Story/Concept  Generator 


The  Intelligent  Test  Toolkit  Story/Concept  Generator 

The  Intelligent  Test  Toolkit  Story/Concept  Generator  (SCG)  is  a natural  language  text  generation  tool, 
designed  to  take  a single  story/concept  example  and  to  generate  many  parallel  story/concept  examples. 
Specifically,  it  (a)  accepts  user  input,  in  the  form  of  a paragraph,  (b)  prompts  the  user  for  the  critical 
attributes  of  the  story/concept  and  the  degree  of  generalization  and  (c)  creates  as  many  parallel 
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story/concept  examples  as  possible,  given  its  current  knowledge  base.  These  activities  coordinate  with  three 
general  components  in  the  SCG  as  shown  in  Figure  1.  There  are  four  screens  within  the  SCG  that  represent 
the  functions  described  above. 

1.  Parser  Screen  for  User  Input 

The  user  enters  information  into  a Parser  Screen  that  allows  for  free-text  input.  When  the  user  has  finished 
entering  the  story/concept  example,  the  story  is  parsed  in  several  different  ways.  First,  the  parser  is  seeks 
the  verb  in  each  sentence  and  also  all  content  words.  Harris  (1985,  pg  322)  describes  content  words  as 
nouns,  verbs,  adjectives,  and  adverbs.  Verbs  are  used  to  recognize  the  conceptual  primitive  (see  Schank  & 
Abelson  1977)  involved  in  the  sentence.  Each  primitive  in  the  story  is  added  to  a discourse  structure  list, 
that  keeps  track  of  the  order  of  the  sentences. 

2.  Generalizabilty  Parameter  Screen 

The  Genralizability  Parameter  Screen  (GPS)  is  the  first  of  two  areas  that  are  used  to  limit  and  understand 
the  degree  of  generalizability  of  the  story/concept.  The  GPS  lists  each  content  term  in  the  story/concept.  It 
then  prompts  the  user  to  answer  questions  about  the  desired  degree  of  generalization  of  each  term. 

All  knowledge  for  the  SCG  is  contained  in  a knowledge  base  using  a semantic  network  representation.  The 
knowledge  base  is  initially  searched  for  hierarchical  data  (links  labeled  "a-kind-of ).  Because  terms  may  be 
members  of  several  categories  (a  car  is  a kind  of  vehicle,  is  also  a kind  of  gift,  and  is  also  a kind  of  status 
symbol),  users  must  clarify  the  "tangled"  induction  (Holland,  Holyoak,  Nisbett,  & Thagard  1986). 

3.  Semantic  Network  Screen 

Certain  key  elements  in  the  story/concept  have  relationships  that  must  be  preserved  while  the  hierarchical 
inference  occurs.  These  relationships  are  then  used  throughout  the  new  story/concept  example.  A semantic 
network  tool  allows  users  to  create  an  ad  hoc  semantic  net  of  the  relationships  that  must  be  preserved  as 
new  story/concept  examples  are  created.  A search  and  match  function  attempts  to  find  parallel  structures  in 
the  knowledge  base. 

4.  Sentence  Generator  Output  Screen 

The  Sentence  Generator  screen  uses  the  decisions  that  were  made  on  previous  screens  to  re-construct  a 
series  of  parallel  story/concept  examples.  The  number  of  possible  parallel  story/concepts  is  limited  to  the 
data  stored  in  the  knowledge  base.  Ideas  relating  to  discourse  structure  are  relatively  unexplored  in  this 
tool.  It  would  be  interesting  to  explore  an  "understanding  component"  to  the  SCG  that  could  attempt  to 
"understand"  the  story's  goals  with  a richer  internal  representation.  This  could  lead  the  system  to  create 
alternative  discourse  structures  to  convey  that  information. 
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intruiluciiori 

Currently  many  approaches  are  suggested  to  achieve  CAI  in  a networked  environment  such  as  the  Internet. 
They  all  try  to  enhance  and  make  more  efficient  the  physical  components  of  the  learning  environment  on  campus 
by  creating  a digital  infrastructure  that  has  come  to  be  known  as  the  virtual  university.  UNU  in  collaboration 
with  IIS  has  launched  the  "Virtual  University"  project  to  jointly  develop  new  technologies  for 
instructors/students  on  the  Internet.  In  this  paper,  we  focus  on  a sub-component  of  the  project  that  attempts  to 
provide  support  for  the  full  cycle  of  organizing  and  developing  education  on  the  Internet,  from  registration  to 
curricular  development,  and  from  learning  to  student  evaluation. 


The  problems  found  in  the  past 

Even  though  CAI  software  is  fabricated  very  well  these  days,  not  all  of  them  are  based  on  the  goal  of 
improving  the  quality  of  the  education  nor  on  the  methodologies  specially  studied  for  CAI  software.  Besides 
problems  of  quality,  not  all  instructors  are  happy  to  use  the  software  due  to  two  reasons.  One  is  a problem  of 
teaching  style  and  the  other  is  a course  material  problem.  CAI  software  does  not  always  fit  the  instructors’  needs 
nor  his/her  style  of  teaching.  Usually  the  software  is  produced  based  on  the  teaching  methods  of  one  particular 
instructor.  It  works  very  well  only  when  either  the  instructor  is  a programmer  or  the  instructor  has  his/her  own 
programmer  to  modify/update  the  program  for  his/her  needs.  This  scenario  rarely  happens  in  the  real  world. 
Some  software  developers  try  to  offer  authoring  tools  to  cope  with  this  problem.  However,  most  instructors  do 
not  have  enough  time  to  learn  how  to  use  the  tools,  and  then  collect  or  create  multimedia  contents.  It  is  not  cost- 
effective  even  for  skillful  instructors  to  create  multimedia  data  by  themselves,  and  multimedia  data  sources  on 
CD-ROMs  are  limited  in  quantity.  These  are  the  reasons  why  CAI  software  is  not  becoming  very  popular  among 
instructors. 

Here,  we  have  a proposal  that  leads  to  the  solution  of  the  problem  above.  The  Internet  is  a treasure  box  of 
multimedia  data  sources.  If  we  have  an  effective  tool  to  find  course  materials  in  an  intelligent  manner,  one  of  the 
problems  is  solved.  If  we  expect  the  software  to  be  a tool  to  help  instructors  teach  rather  than  a substitution  for 
conventional  textbooks,  we  can  see  the  solution  for  another  problem.  This  is  the  role  of  SmartSearch  in  the 
Virtual  University  project. 


CAI  with  SmartSearch 

SmartSearch  is  a web-based  search  engine  classified  as  “Meta-Search  Engine.”  The  well-known  example  of 
Meta-Search  Engine  is  the  Internet  search  feature  of  “Sherlock”  of  MacOS  8.5.  The  web-based  example  is 
“Metacrawler”  which  works  very  well.  Meta-Search  Engine  is  an  intelligent  state-of-the-art  search  engine  (SE). 
Conventional  SEs,  web-directory  service  SEs  and  robot-driven  SEs,  have  a lot  of  problems  due  to  their 
structure.  The  main  point  is  the  relevancy  of  the  result  with  users'  input.  The  Meta-Search  Engine  focuses  on 
the  improvement  of  this  relevancy  rate,  and  uses  these  conventional  SEs  by  sending  queries  to  them  in  the 
background.  This  system  eliminates  the  costs  of  building  up  a new  SE  while  achieving  a higher  relevancy  rate 
with  less  waiting  time.  The  effective  ranking/re-ordering  techniques  of  the  results  are  studied  in  Meta-Search 
Engines.  The  main  point  is  the  relevancy  of  the  output  and  not  the  size  of  the  main  database.  Therefore,  the  key 
to  “Meta-Search  Engine”  is  “re-ordering.” 
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Our  SmartSearch  claims  to  perform  better  than  these  examples  by  featuring  weight  adjustment  techniques.  The 
following  are  the  factors  we  count  to  retrieve  relevant  data: 

• The  number  of  keywords  appearing  in  the  title 

• The  number  of  keywords  appearing  in  the  main  text 

• The  number  of  keywords  appearing  in  the  URL 

• The  relevancy  information  returned  from  conventional  search  engines 

• The  number  of  URLs  which  share  the  same  web-server 

• Whether  or  not  the  keywords  are  jargon  which  can  specify  the  area  the  user  is  looking  for 

• The  type  of  search  engine  which  returned  the  link 

• The  host  type  (e.g.  edu,  com,  net,  etc.) 

• The  length  of  the  URL 

After  SmartSearch  retrieves  the  actual  HTML  from  the  site,  it  will  calculate  the  relevancy  score  for  each  link. 
Ordered  by  this  score,  the  new  list  of  relevant  links  is  shown  to  the  users.  This  system  works  very  well  with 
ordinary  keywords,  and  we  are  enhancing  its  features  especially  for  CAI  use  of  handling  multimedia  data. 

As  we  proposed,  SmartSearch  will  be  able  to  solve  one  of  the  big  problems  of  current  CAI  software. 
Instructors  will  have  fewer  troubles  using  CAI  software  and  finding  digitized  data,  as  we  all  hoped  for  in  the 
past.  This  idea  can  be  applied  in  two  ways.  One  of  them  is  to  prepare  course  materials:  a sub-component  for 
conventional  CAI  software  is  to  help  the  instructor  produce  his/her  own  digital  textbook.  Another  is  to  help 
instructors  actually  teach  in  class:  SmartSearch  can  be  used  online  as  a dictionary  or  as  an  encyclopedia  which 
has  accumulated  knowledge  of  everyone  in  the  world.  Also,  students  can  take  advantage  of  this  system.  In  the 
past  CAI  software,  all  system  components  including  the  authoring  tools  are  for  instructors  who  are  supposed  to 
be  the  only  people  who  create  materials.  Although  the  conventional  hardcover  textbooks  are  made  by  educators, 
the  students  of  the  next  generation  can  edit  their  own  references.  This  tool  is  also  useful  for  students  to  find 
relevant  information  for  self-studying. 


Conclusion  & Future  Work 

Again,  collecting/building  up  course  materials  is  not  a simple  task.  If  we  want  to  automate  this  task,  we  need 
something  intelligent.  We  can  use  SmartSearch  to  gather  information  in  order  to  skip  first  stage.  The  collected 
data  at  the  first  stage  may  need  modification  for  teaching  use,  but  should  help  instructors  considerably.  The 
Digital  Library  of  UNU,  where  the  digital  information  of  the  world  will  be  served  by  SmartSearch,  will  greatly 
facilitate  inquiry-based  learning.  For  digital  libraries  to  be  useful,  SmartSearch  will  allow  users  to  access  quality 
information  more  efficiently,  and  to  make  such  information  easier  to  search  for. 

Increasingly  many  universities  around  the  world  are  taking  advantage  of  the  wide  accessibility  of  the  Internet 
to  deliver  educational  contents  online.  Mobile  computing,  building  upon  the  global  connectivity  of  the  Internet, 
adds  a new  dimension  to  the  learning  process.  In  particular,  it  gives  users  the  illusion  that  their  computer- 
mediated  learning  environments  are  ubiquitous.  Users  can  continue  their  learning  applications  irrespective  of  the 
users’  physical  location  and  the  computer  being  used  for  the  task  at  anytime,  right  down  to  the  last  mouse-click 
as  when  last  accessed.  Under  this  model,  data  and/or  applications  follow  users  wherever  they  go.  However,  the 
past  approaches  of  virtual  classroom,  library,  and  laboratory  do  not  take  into  account  the  mobility  of  the  learner 
in  most  cases.  In  our  Virtual  University  project,  this  ubiquitous  learning  environment  is  an  important  element. 
SmartSearch  will  be  enhanced  by  the  feature  for  distant-learning.  Wherever  the  user  is,  the  software  will 
download  the  last  state  of  the  search  and  user's  interests/preferences. 
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Introduction 

In  our  workgroup  of  “Computer  Science  and  Society”  at  the  University  of  Paderbom  lectures  are  usually 
presented  with  Powerpoint  slides.  Afterwards  these  slides  are  made  available  to  the  students  of  this  lecture 
within  our  local  Hyperwave-Server  and  Teaching  Environment  (Engbring,  Schwolle  1997).  The  content  of  these 
presentations  is  highly  condensed  and  the  students  are  advised  to  make  their  own  notes  and  comments  during  the 
lessons.  Experience  showed  that  most  students  lack  the  key  qualification  to  follow  the  different  branches  of 
argumentation  given  by  the  professor  and  to  write  down  their  own  notes,  deciding  online  what  makes  sense  and 
what  is  nonsense. 

As  we  saw  the  problems  of  the  students  with  grasping  intention  and  contents  of  the  lectures,  we  had  to 
find  a way  to  give  them  a second  opportunity  to  understand  the  lecture  and  to  support  the  learning  process.  This 
required  a more  differentiated  form  of  course  material.  We  decided  to  record  the  whole  lecture,  audio,  video  and 
slides,  and  to  make  it  accessible  for  the  students.  We  thought  it  useful  for  understanding  the  intention  of  a lesson 
to  ''re-listen  ” to  the  spoken  word  of  the  lecturer.  But  in  our  opinion  it  was  not  necessary  to  view  the  video 
because  it  is  not  very  expressive  in  itself  The  experience  of  “Authoring  on  the  Fly”  (Bacher,  Ottmann  1 996) 
showed  that  video  did  not  improve  the  learning  process,  so  we  decided  to  use  only  audio.  For  fast  access  the 
audio-streams  had  to  be  compressed  and  linked  to  the  slides  of  the  original  course  material.  So  any  student  not 
quite  understanding  the  contents  of  one  slide  could  download  the  original  spoken  lecture  by  internet  and  replay  it 
at  home  or  at  a workplace  at  the  university. 

We  were  convinced  that  this  was  useful  because  our  professor  gives  the  students  many  different  examples 
during  one  lesson  for  better  understanding.  This  approach  to  expand  the  amount  of  information  available  on  a 
lecture  had  an  advantage  to  writing  down  long  text  passages,  because  the  recording  of  audio  is  done  on  the  fly. 
The  only  work  was  to  cut  it  up  in  usable  pieces  and  connect  these  semantically  as  well  as  physically  to  the 
existing  slides.  Here  it  seemed  reasonable  to  use  short  pieces  of  maximum  1 0 minutes  for  better  motivation  and 
precise  navigation. 


Development  of  an  Effective  Technical  Realization 

The  actual  recording  of  the  lectures  was  done  by  using  readily  available  tools.  We  used  a standard  8mm 
Camcorder  with  an  external  microphone.  For  archiving  purposes,  we  first  transferred  the  recordings  to  VHS 
videotape.  Afterwards  the  audio  of  the  recordings  was  digitized  by  an  audio  connection  to  a standard  sound 
device  of  a PC.  After  several  revisions  it  proved  to  be  most  efficient  to  permanently  install  all  the  needed 
equipment  in  one  transportable  compartment.  This  way  the  recording,  archiving  and  digitizing  was  all  done  “on 
the  fly”  during  the  lecture  in  one  go.  The  resulting  audio-files  still  had  to  be  remastered.  First  it  was  necessary  to 
erase  the  rather  high  noise  level  from  the  audio  files.  The  students  present  at  the  lectures  were  not  totally  silent 
and  noises  like  air  conditioning  and  chairs  and  the  like  were  not  to  be  dismissed.  After  that  the  volume  level  had 
to  be  adjusted,  because  we  could  not  place  the  microphone  directly  in  front  of  the  lecturer.  And  lastly  we  had  to 
reduce  the  files  in  size.  One  hour  of  lecture  resulted  in  about  1 60  MB  of  data.  After  some  considerations  about 
availability,  reduction  factor  and  performance  we  decided  to  use  MPEG  I Layer  3 (MP3),  which  allowed  for  data 
compression  of  about  1:14  (which  left  us  with  about  20  MB  for  one  90  minute  lecture  including  slides  and 
compressed  audio).  That  way  the  audio  files  became  possible  to  handle  by  internet.  All  this  development  was 
done  during  the  course  “Computer  Science  and  Society”. 


Evaluation 


After  finding  an  easy  way  to  record  a lecture  and  producing  the  improved  course  materials,  two  different 
approaches  to  use  these  were  developed.  One  was  to  apply  the  course  materials  to  a distance  learning  course  at 
Potsdam  University.  The  other  was  to  use  the  recording  technique  to  accompany  a second  lecture  course, 
“Softwareergonomie”  at  Paderbom  University, 

For  the  distance  learning  course  at  Potsdam  University  we  used  the  audio  files  in  addition  to  the  lecture 
slides.  The  contents  of  the  course  where  identical  to  those  of  the  on  campus  course  “Computer  Science  and 
Society”,  just  that  the  attendants  had  to  retrieve  all  their  material  by  internet.  Because  of  the  big  quantity  of  audio 
files  we  selected  only  parts  of  them  which  still  amounted  to  about  150  MB,  This  may  be  the  reason  why  an 
evaluation  (Hoff-Holtmanns  1999)  showed  that  from  the  36  students  only  30%  where  able  to  download  and  use 
the  audio-files.  Of  these  72%  made  effective  use  of  about  half  of  the  recordings.  The  other  28%  actively  used 
more  than  60%  of  the  recordings.  This  allows  the  conclusion  that  given  an  easier  opporiunity  ivj  access  the  files 
needed,  most  students  would  make  use  of  them  and  a satisfying  number  would  use  them  to  support  their  learning 
process.  One  solution  for  this  would  be  the  production  and  distribution  of  a CD-ROM  for  the  students  with  all 
the  course  material  beforehand. 

To  evaluate  the  use  of  the  audio  annotations  in  the  lecture  “Softwareergonomie”  we  chose  to  interview 
the  students  after  their  colloquium.  Unfortunately  on  writing  this  paper  the  evaluation  still  is  in  progress.  For 
more  information  see  (Grimm  1999).  Nevertheless  we  have  first  results: 

• The  majority  of  the  students  used  the  CD  we  produced  after  the  semester.  Nobody  listened  to  an  annotation 
during  the  course  period. 

• To  improve  the  usability  we  need  to  cut  the  lectures  into  smaller  pieces  than  our  maximum  of  10  minutes 
and  increase  the  audio  quality. 

• It  seems  the  annotations  are  of  great  interest  in  the  preparation  for  the  exams. 

• The  students  reviewed  most  of  the  slides,  but  they  only  listened  to  a small  part  of  the  audio  annotations, 
mostly  those  in  connection  with  graphics,  diagrams  and  summaries. 


Prospect 

Despite  the  problems  we  had  to  face  and  will  encounter  in  the  future,  this  approach  of  enhancing  course 
materials  by  audio  recordings  of  the  lecturer  is  promising.  The  process  of  creating  these  materials  is  easy  and 
readily  transferable  to  other  courses  and  different  lecture  situations.  It  remains  to  be  seen  if  the  experiences 
described  above  are  correlated  to  the  form,  content,  and  niveau  of  the  lecture,  the  lecturer  herself,  or  if  they  are 
specific  to  learning  with  audio  annotations.  Therefor  we  intend  to  record  lectures  in  other  disciplines  such  as  e.g. 
business  management. 

Future  work  will  also  concern  enhancements  in  quality  and  size  of  the  materials  as  well  as  further 
functionality.  This  being  especially  solutions  to  questions  like  what  about  links  ''inside”  the  audio-files,  or  links 
and  references  from  within  documents  to  just  parts  of  an  audio  annotation. 
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Using  the  advanced  WEB  and  the  Oracle  Database  technology,  I created  an  interactive  videoconference  and 
distance  learning  management  system.  This  system  has  been  used  for  3 years,  and  has  saved  a lot  time  and 
trouble  for  both  the  users  and  administrators  of  the  videoconference  rooms. 

Powerful  Features 

1.  Provides  searchable  and  automatically  updated  Video  Conference  Schedule  on  the  WEB 

A user  can  view  the  up-to-date  schedule  on  the  WEB,  and  can  search  the  schedule  by  date  and  by  the 
conference  organizer’s  name. 

2.  Enables  users  to  make  video  conferencing  reservations  on  the  WEB 

A user  can  send  a request  to  reserve  the  video  conference  facility  by  filling  out  a form  on  the  WEB.  The 
inputted  data  will  automatically  be  put  into  a database  and  a copy  provided  to  the  videoconference  room 
administrator  via  e-mail.  Remote  site  technical  information  is  also  inputted  this  way. 

3.  Generates  Video  Conference  Room  Usage  Report 

An  additional  feature  generates  useful  reports  to  answer  the  questions  such  as  who  had  videoconferences, 
how  many  times,  and  with  what  sites  the  conferences  were  held.  This  feature  also  provides  information 
such  as  how  many  were  classes,  administrative  meetings,  or  training  workshops.  It  can  provide  reports  on 
all  the  videoconferences  or  for  a specified  time  period.  For  example,  it  can  answer  a question  such  as  how 
many  training  workshops  have  been  held  from  September  1 till  December  31,  1998. 

4.  Complies  the  form  of  “Request  for  Off-NSHEC  Network  Calling  Assistance” 

To  do  an  “off-nef  ’ videoconference,  we  need  to  send  a special  request  form  to  a company  called  Tendberg. 
In  the  request  form,  we  need  to  provide  both  the  technical  and  contact  information  related  to  our  sites  and 
to  the  site  with  which  we  will  hold  the  conference.  The  system  can  compile  such  forms  automatically  by 
pulling  data  from  local  user  input  and  from  the  remote  site  administrator’s  input. 


An  Inside  Look 

The  system  was  created  using  WWW,  CGI,  and  Oracle  database  technology.  A user  first  looks  at  the 
videoconference  schedule  on  the  web  to  see  if  the  time  they  want  is  available.  Then,  the  user  inputs  the  data 
about  a conference  (such  as  when,  where,  purpose,  and  contact  person’s  name,  email  and  phone)  by  using  a 
reservation  form  on  the  WEB.  After  the  web  form  receives  the  data,  it  automatically  forwards  the  data  to  a 
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CGI  engine.  This  CGI  engine  then  does  three  things  with  the  data:  1)  generates  a web  page  that  sends 
confirmation  and  gives  further  instructions  to  the  user,  2)  generates  email  to  send  data  to  the  videoconference 
room  administrator,  3)  puts  the  data  into  an  Oracle  database.  (See  attached  diagram  below) 

When  the  users  click  on  the  link  to  the  videoconference  schedule  on  the  web,  the  system  sends  a request  to  a 
CGI  engine.  This  engine  pulls  the  data  from  the  database  accordingly,  and  generates  the  schedule  for  the  user. 
In  this  way,  the  schedule  presented  to  the  users  is  always  up-to-date  and  tailored  toward  users’  specific  interest 
(See  attached  diagram). 

To  generate  videoconference  room  usage  reports,  an  administrator  clicks  corresponding  buttons  on  the  web, 
which  sends  a request  to  the  CGI  engine.  This  engine  summarizes  the  data  from  the  database  and  shows  the 
results  on  the  WES.  In  a similar  way,  the  system  generates  the  form  of  “Request  for  Off-NSHEC  Network 


Diagram 
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Introduction 

Remember  when  you  were  a kid  how  the  above  taunt  reflected  youthful  humor  when  you  caught  someone 
by  surprise?  Remember  when  you  took  great  delight  in  such  antics  as  “Your  shoe’s  untied!”  “Ha,  ha,  I made  you 
look!  April  fool!”  Or  how  with  two  or  three  of  your  friends  you  stood  somewhere  in  public  all  gazing  up  at  some 
imaginary  thing-in-the-sky  and  laughed  like  crazy  when  passers-by  would  stop  and  try  to  see  what  you  were 
looking  at?  Imagine  my  surprise  after  many  years  in  the  classroom  suddenly  to  arrive  at  the  conclusion  that 
teaching  is  a lot  like  that!  And  to  realize  that  with  all  its  complexity,  the  real  trick  to  teaching  is  getting  students  to 
really  look  at  something. 

My  beginning  point  is  the  realization  that  in  order  to  learn,  a person  must  perceive  the  data  as  something 
new  — really  to  have  a new  "perception  of"  Humans  tend  to  see  what  they  expect  to  see  and  just  looking  is  in  itself 
boring  - unless  we  are  surprised  by  what  we  see!  Moreover,  our  system  of  education  seems  to  foster  the  notion 
that  study  is  boring  because  it  creates  the  assumption  that  we  won’t  be  surprised.  In  some  way  we  seem  to  have 
mistakenly  defined  classroom  orderliness  as  quiet  inactivity  and  classroom  attentiveness  to  be  a sort  of  dopey 
reticence. 


The  Problem 

In  an  effort  to  “bring  education  into  the  twenty-first  century,”  many  educators  have  adopted 
without  question  the  assumptions  of  industry  managers  who  are  not  themselves  educators. 

Walk  the  aisles  of  any  training  trade  show  and  you’d  think  high-tech  America  had  finally  found  the  magic 
bullet  that  would  solve  the  nation’s  educational  crisis.  If  you  can  cram  a course  down  an  electronic  pipe,  and  deliver 
it  to  the  desktops  of  your  employees, . . . well,  that  must  be  great  teaching,  right? 

Wrong.  Corporate  America  has  finally  awakened  to  the  need  to  teach  employees  how  to  do  not  Just 
today’s  jobs,  but  the  ones  they’ll  inevitably  wind  up  with  tomorrow.  Demand  creates  supply  - bringing 
multimedia  curricula  out  of  the  proverbial  woodwork.  But  do  these  training  programs  teach?  Today,  most 
don’t.  And  they  won’t,  without  sound  instructional  design  - design  that  engages  the  student,  simulates 
complex  problems,  helps  her  practice,  guides  her  to  learn  through  failure,  and  - depending  upon  your  point  of 


If  we  look  upon  applications  of  technology  in  education  as  oriented  towards  “a  learning  ‘with’” 
modeP,  and  if  we  can  use  this  “learning  ‘with’”  model  to  interrupt  student  expectations  about  learning;  and 
if  this  “interruption”  gets  their  attention  so  that  they  really  look  at  the  data,  the  possibility  of  turning 
information  into  knowledge  becomes  significantly  more  real. 


' Fryer,  Bronwyn,  “Powerhouses  of  Design,”  Inside  Technology  Trainings  April,  1998,  p.25. 

^ Reeves,  Thomas  C.,  Answering  Critics  of  Media  and  Technology  in  Education,  Proceedings  of  ED-MEDIA/ED- 
TELECOM  98  (Freiburg,  Germany,  June  20-25,  1998)  pp.  1182-1 183. 
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Without  [effective  instructional  design]  . . . , the  most  sophisticated  “training”  in  the  world  is 
simply  information.  And  information,  as  M.  David  Merrill  says,  “is  not  instruction.”  And  that’s  true  whether 
you’re  sending  a program  down  an  electronic  pipe  or  presenting  it  at  the  front  of  a classroom.^ 


The  Status  Quo? 

If  we  define  “culture”  as  simply  “the  way  we  do  things  around  here,”^  the  objective  would  be  to  use 
multimedia  presentations  to  interrupt  the  "business-as-usual”  culture  of  today's  student  thereby  facilitating 
their  involvement  in  classroom  rituals  engaging  them  in  the  process  of  education  rather  than  the  task  of 
gathering  information. 

“It  is  not  multimedia  resources  that  make  a difference  in  training,  it  is  how  they  are  used.”  Media  is  merely  a 
way  of  representing  subject  matter  content.  Merely  having  multimedia  objects  included  does  not  determine 
instructional  effectiveness.  If  the  media  objects  are  relevant  to  the  instruction,  if  they  facilitate  in 
implementing  effective  instructional  strategies,  then  the  effect  is  efficient,  effective  and  appealing  instruction. 
If,  on  the  other  hand,  the  media  objects  are  merely  decorative  and  serve  no  instructional  purpose,  then  they 
may  in  fact  interfere  with,  rather  than  facilitate,  learning."* 

Using  technology  to  manipulate  the  "rituals"  of  the  classroom  could  challenge  the  way  students 
perceive  their  roles  in  the  classroom  and  change  the  understanding  they  have  about  learning.  Along  with 
this  new  curiosity,  students  would  discover  that  they  have  the  ability  to  change  information  into  knowledge 
- and  can  do  so  with  greater  flexibility,  more  efficiency  and  less  pain! 

The  Strategies 

This  paper  presents  examples  of  nine  strategic  uses  of  technology  that  are  effective  in  teaching  an 
introductory  course  in  Religious  Studies.  By  using  varied  combinations  of  these  strategies,  student 
expectations  about  classroom  performance  are  “interrupted”  in  such  a way  that  the  subject  matter  of  each 
class  session  is  perceived  as  in  some  way  new.  These  strategies  would  be  useful  in  any  Humanities  class  and 
constitute  what  I would  call  “appropriate  use  of  technology.”  They  are  (in  no  particular  order): 

1 ) Visuals  and  animations  used  to  present  a graphic  definition  of  the  problem  or  problems,  which  are  the  focus  of 
each  particular  class. 

2)  A “language  resource  pool”  - a database  of  questions  and/or  definitions  of  terms  and  complex  ideas  which 
are  easily  accessible  in  class  to  stimulate  conversation. 

Computer  programming  that  maps  the  interrelationship  of  ideas  and/or  that  connects  layers  of  thought  or 
information.  Part  of  the  problem  with  humanities  courses  is  that  it  is  often  necessary  to  explain  complex 
experiential  events  and  to  connect  them  with  facts  or  hard  data. 

4)  Animated  presentations  used  to  illustrate  the  development  of  a logical  argument  or  an  historical  outline. 

5)  Interactive  programming  that  challenges  student  assumptions;  this  strategy  makes  student  responses  public  so 
that  they  can  be  perceived  as  being  part  of  the  problem. 

6)  “What  ifs”  — show  various  hypotheses  to  stimulate  debate  about  the  value  of  different  solutions  to  a problem  - 
“there  are  no  answers  to  questions,  just  hypotheses  that  suggest  solutions  to  problems  that  need  to  be  defined.” 

7)  The  display  of  obvious  bits  of  data  in  ways  (relationships)  that  may  not  be  obvious  (lateral  thinking)  --  seeing 
the  obvious  juxtaposed  in  creative  ways  can  create  new  questions. 

8)  Digital  Video  used  to  create  conflict  by  having  a teacher  talk  with  a virtual  self  or  aigue  a case  with  another. 

9)  Digital  Video  presenting  short  clips  of  “experts”/and  or  scholars  commenting  on  a question  or  problem  and 
used  to  illustrate  different/contradictory  points  of  view. 


^ Deal,  Terrence  E.  and  Kennedy,  Allan  A.  (1982).  Corporate  Cultures:  The  Rites  and  Rituals  of  Corporate  Life. 
Reading,  MA:  Addison- Wesley  Publishing  Company,  p.4. 

"*  Merrill,  M.  David,  “Instructional  Strategies  That  Teach,”  CBT  Solutions,  Nov/Dec  1997,  p.2. 
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During  the  1990*s  the  Internet  has  evolved  from  primarily  an  electronic  mail  system  for  academics  and  the 
military  to  become  the  avenue  of  access  for  millions  of  databases  worldwide.  Web  access  to  massive 
amounts  of  information  once  available  only  in  libraries  has  impacted  library  use  by  adult  learners.  One  sub- 
group of  adults  students,  distance  learners,  have  long  been  an  underserved  academic  library  population  and 
have  the  potential  to  benefit  greatly  from  the  resources  now  available  to  anyone  with  web  access.  The 
proliferation  of  learning  resources  is  now  making  it  possible  for  many  students  be  come  at  least  partial 
distance  learners  with  some  classes  and  resources  being  accessed  through  alternative  communications 
media.  These  changes  are  evidenced  in  the  fact  that  the  term  "distance  learning"  is  now  often  replaced  with 
"distributed  learning" 

In  1 998,  in  an  attempt  to  gain  some  insight  into  these  changes,  I developed  a survey  that  was  sent  out  by 
the  Library,  Instruction,  Systems  and  Technology  department  at  Arizona  State  University.  Surveys  were 
sent  to  800  students  registered  through  ASU's  College  of  Extended  Education.  These  students  had  all 
registered  to  take  classes  via  television,  correspondence,  CD-ROM,  the  Internet  or  at  remote  sites.  One 
hundred  thirty  nine  surveys  were  returned. 

My  goal  was  threefold:  gather  demographic  information  to  tell  us  who  these  distance  learners  are;  find  out 
what  library  services  they  use  and  need;  and  learn  how  they  use  the  Internet  for  their  information  needs. 

The  median  age  of  the  respondents  was  32  with  an  age  range  extending  from  1 8 to  53.  Fifty-six  percent 
were  undergraduate  and  66%  were  female.  Fifty-two  percent  were  fully  employed,  while  33%  worked  part 
time  and  1 5% were  not  employed. 

A wide  variety  of  majors  were  represented  with  the  largest  group  coming  from  Education  with  30%, 
followed  by  Public  Programs  (17%),  Social  Work  (15%),  Business  (1 1%),  Engineering  (10%)  and  Liberal 
Arts  (9%).  Five  percent  declared  no  major  and  Architecture,  Fine  Arts  & Nursing  were  each  represented  by 
1%  of  the  respondents. 

The  respondents  lived  an  average  of  39  miles  from  an  academic  library  and  10  miles  from  a public  library. 
Eighty-five  percent  of  them  have  Internet  access,  with  74%  of  them  using  e-mail  and  67%  having  used 
online  searches  for  course  related  information.  In  the  area  of  library  services  65%  have  used  library  web 
pages,  22%  have  used  interlibrary  loan  and  14%  have  used  document  delivery  services 

The  Internet  has  become  the  first  information  source  of  choice  for  32%  of  the  respondents  followed  by 
Academic  Libraries  (25%),  Friend,  Spouse,  Colleague  (16%),  Public  Library  (13%)  and  Home  Reference 
Books  (8%). 

While  the  Internet  has  become  a significant  resource  for  most  these  adult  learners,  with  81%  agreeing  that 
it  is  an  important  source  of  information  to  them,  libraries  are  obviously  still  critical  to  them  with  87% 
agreeing  on  there  need  for  library  services. 
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Sixty-seven  percent  agree  that  a 1 to  2 hour  class  on  Internet  search  techniques  would  be  valuable  to  them. 
Indicating  that,  although  the  vast  majority  of  them  use  the  Internet  to  some  extent,  most  of  them  also  feel 
they  need  to  learn  more  about  using  it  for  information. 

After  the  survey,  interviews  were  conducted  with  31  local  community  college  students.  This  group  was 
selected  for  three  reasons:  they  were  generally  younger  than  the  surveyed  group;  they  had  less  need  for 
scholarly  information  sources;  and  they  were  potential  ASU  students.  A few  rarely  used  the  Internet  but 
most  of  them  felt  that  it  was  the  easiest  way  to  find  the  information  that  they  were  looking  for.  One  quote 
from  these  interviews  is  typical  of  the  viewpoint  of  many  of  these  younger  students: 

/ used  to  always  go  to  tke  library’  to  look  up  things  in  the  encyclopedias  but  now,  on-line  service  is 
too  convenient.  I stay  at  home  and  use  the  library.  I can  access  any  of  the  periodicals  I want,  just 
about  and  magazines — / can  find  back-issues  I want  on-line.  The  Internet  has  pretty  much 
eliminated  my  library  use. 

Conclusion 

Searching  the  Internet  has  become  the  first  means  of  finding  new  information  for  nearly  one  third  of  the 
adult  learners  in  this  survey,  ahead  of  both  academic  and  public  libraries.  The  students  still  express  a strong 
need  for  libraries.  It  is  becoming  clear  that,  with  more  and  more  information  inundating  students,  the 
Internet,  at  least  at  this  point,  is  mostly  a supplement  to  libraries  and  not  a replacement  for  them.  Younger 
students  with  simpler  academic  needs,  however,  are  often  using  the  Internet  as  a library  replacement. 

The  importance  of  resources  available  through  most  academic  library  web  sites  is  the  value  added  to  the 
information  by  the  librarians:  the  fact  that  the  electronic  resources  have  been  selected  specifically  for 
academia  and  are  reliable  and  appropriate  sources  of  information  for  students.  Anyone  who  has  spent  any 
time  searching  for  information  on  the  Web  is  well  aware  of  the  massive  amount  of  information  available 
and  also  just  as  aware  how  unreliable  much  of  it  is.  Libraries  can,  any  many  are,  providing  network 
gateways  to  appropriate  academic  information  resources,  including  links  to  many  full  text  journal  and  news 
sources,  that  guide  students  to  the  material  they  need  from  either  a library  workstation  or  a personal 
computer  at  home. 

The  words  of  one  librarian  I interviewed  illustrate  the  impact  that  the  Internet  has  had  on  academic  library 
use  and  the  raised  expectations  it  has  created: 

They  (students)  tend  to  think  everything  is  on  the  Internet.  They  tend  to  think  that  everything  is 
available  the  way  they  want  it.  They  tend  to  think  everything  is  cost-free. . . You  will  find  that  a lot 
of  times  people  will  come  up  to  the  reference  desk  and  the  first  thing  they  will  say  to  us  is  7 looked 
on  the  Internet*  and  we  are  not  sure  they  actually  did,  we  are  not  sure  they  know  what  the  Internet 
is,  actually.  They  tend  to  want  that  more  and  more. 

There  is  a lot  of  information  out  there  and  it*s  a lot  easier  to  get  to  and  I think  it*s  more  confusing 
because  there  is  so  much  — it*s  like  this  overload  of  information  and  an  overload  of  ways  to  find  it. 
lVe*re  witnessing  that  just  being  at  the  reference  desk  with  hundreds  of  databases  and  the  scores  of 
ways  of  looking  for  things.... Some  students  are  expecting  instantaneous  packaged  information, 
quickly.  They  expect  that  they  are  going  to  be  able  to  find  this  quickly  without  any  problems,  so 
they  won*t  have  to  stay  in  the  library,  or  they  won*t  even  have  to  come  to  the  library. " 
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In  spite  of  claims  regarding  the  potential  benefits  of  using  hypermedia  in  education,  and  the  controversial 
issues  about  the  relationship  bebween  media  and  learning,  research  results  comparing  the  effects  of  hypermedia 
and  non-hypermedia  instruction  are  conflicting.  For  example,  Bain,  Houghton,  Sah,  & Carroll  (1992),  Barnes 
(1994),  Chen  (1993),  Delclos,  & Hartman  (1993),  Gretes,  & Green  (1994),  and  Toro  (1995)  all  report  significant 
gains  for  hypermedia  over  non-hypermedia  instruction.  On  the  other  side,  Azevedo,  Shaw,  & Bret  (1995),  Barker 
(1988),  D’lessandro,  Galvin,  Erkonen,  Albanese,  Michaelsen,  Huntley,  and  Tabar  (1991)  have  found  no  significant 
differences  between  hypermedia  and  non-hypermedia  instruction.  Yet,  owing  to  the  contradictory  evidence 
provided  by  existing  research  in  the  area,  it  is  important  to  conduct  a meta-analysis  to  clarify  the  research 
conclusions.  The  results  of  this  meta-analysis  may  also  shed  light  on  the  debatable  issue  regarding  the  relationship 
between  media  and  learning. 


PROCEDURES 

The  research  method  used  in  this  study  is  the  meta-analytic  approach  which  was  similar  to  that  described 
by  Glass,  McGaw,  & Smith  (1981),  The  studies  considered  for  use  in  this  meta-analysis  came  from  three  major 
sources  and  were  published  from  1986  to  1998.  One  large  group  of  studies  came  from  computer  searches  of 
Education  Resources  Information  Center  (ERIC),  A second  group  of  studies  came  from  Comprehensive 
Dissertation  Abstracts.  A third  group  of  studies  was  retrieved  by  branching  from  bibliographies  in  the  documents 
located  through  review  and  computer  searches.  Forty-seven  studies  were  located  through  these  search  procedures; 
14  studies  came  from  ERIC,  26  studies  were  retrieved  from  published  journals,  and  7 studies  were  from 
Comprehensive  Dissertation  Abstracts.  However,  one  study  reported  by  Liu,  & Reed  (1995)  had  an  Effect  Size 
(ES)  several  times  higher  than  mean  ES  of  other  studies  included  in  the  synthesis  (i.e.,  ES  = 6.54  for  Liu,  & 
Reed’s  study).  The  study  was  therefore  considered  as  outlier  and  excluded  from  this  meta-analysis. 

Several  criteria  were  established  for  inclusion  of  studies  in  the  present  analysis. 

1 . Studies  had  to  compare  the  effects  of  hypermedia  (HI)  vs.  non-hypermedia  instruction  (NHI)  on 
student’s  achievement. 

2.  Studies  had  to  take  place  in  actual  educational  settings.  There  was  no  restriction  on  grade  level. 

3.  Studies  had  to  provide  quantitative  results  from  both  hypermedia  and  traditional  classes, 

4.  Studies  had  to  be  retrievable  from  university  or  college  libraries  by  interlibrary  loan  or  from  ERIC, 
Dissertation  Abstracts  International,  or  University  Microfiche  International. 

5.  Studies  were  published  between  1986  and  1998. 

Outcome  Measures 

The  instructional  outcome  measured  most  often  in  the  46  studies  was  student  learning,  as  indicated  on 
standard  or  researcher-develop  achievement  tests  at  the  end  of  the  program  of  instruction.  For  statistical  analysis, 
outcomes  from  a variety  of  different  studies  with  a variety  of  different  instruments  had  to  be  expressed  on  a 
common  scale.  The  transformation  used  for  this  purpose  was  the  one  recommended  by  Glass  et  al.  (1981)  and 
modified  by  others  (e.g..  Hunter,  Schmidt,  and  Jackson,  1982),  To  reduce  measurements  to  a common  scale,  each 
outcome  was  coded  as  an  Effect  Size  (ES),  defined  as  the  difference  between  the  mean  scores  of  two  groups 
divided  by  the  pooled  standard  deviation  of  two  groups.  For  those  studies  that  did  not  report  means  and  standard 
deviations,  F values,  t values,  or  proportion  values  were  used  to  estimate  the  ES. 

Variables  Studied 

Sixteen  variables  (type  of  publication,  year  of  publication  , subject  area,  grade  level  , sample  size, 
instructor  bias,  instrumentation,  reliability  of  measure,  statistical  power,  statistics,  type  of  research  design,  type  of 
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delivery  system,  comparison  group,  type  of  instruction  for  treatment,  implementation  of  innovation,  duration  of 
treatment  were  coded  for  each  study  in  the  present  synthesis.  Each  variable  was  employed  as  a factor  in  an 
analysis  of  variance  (ANOVA)  to  investigate  whether  there  were  significant  differences  within  each  variable  on 
the  ES. 


RESULTS 

Of  the  46  studies  included  in  the  present  synthesis,  28  (61%)of  the  study- weighted  ESs  were  positive  and 
favored  the  HI  group,  while  17  (37%)  of  them  were  negative  and  favored  the  NHI  group.  Only  1 (2%)  of  them 
showed  no  difference  between  HI  and  NHI  groups.  The  range  of  the  study- weighted  ESs  was  fiori'i  -0.91  to  3.13. 
The  overall  grand  mean  for  all  46  study-weighted  ESs  was  0.41 . When  this  mean  ES  was  converted  to  percentiles, 
the  percentiles  on  students'  achievement  were  66  for  the  HI  group  and  50  for  the  NHI  group.  The  overall  grand 
median  for  all  46  study-weighted  ESs  was  0.15,  suggesting  that  percentiles  on  students'  achievement  were  56  for 
the  HI  group  and  50  for  the  TI  group.  The  standard  deviation  of  0.87  reflects  the  great  variability  of  ESs  across 
studies. 

For  ANOVA,  four  variables,  instrumentation,  type  of  research  design,  type  of  delivery  system,  and 
comparison  group  showed  statistically  significant  impact.  For  each  of  these  variables,  a post  hoc  (Fisher 
Protected  LSD)  test  was  performed. 

The  post  hoc  test  for  instrumentation,  (F(2,43)  = 3.241,  P<.05),  showed  that  the  mean  comparison  of 
studies  in  which  the  instrumentation  was  coded  as  unspecified  was  higher  than  the  studies  employed  local  or 
standardized  instruments.  There  were  no  significant  differences  found  between  the  mean  comparison  of  studies 
using  local  instruments  and  standardized  instruments. 

The  post  hoc  test  for  type  of  research  design,  (F(3,42)  = 6.998,  p<.001),  showed  that  the  mean 
comparison  of  studies  coded  as  one  group  repeated  measure  was  significantly  higher  than  studies  coded  as  pretest- 
posttest  control  group,  nonequivalent  control  group,  or  posttest-only  control  group  designs.  There  were  no 
significant  differences  found  among  the  mean  comparison  of  studies  coded  as  pretest-posttest  control  group, 
nonequivalent  control  group,  and  posttest-only  control  group  designs. 

For  type  of  delivery  system,  (F(2,  43)  = 3.951,  p<.05),  the  post  hoc  test  showed  that  the  mean  comparison 
of  studies  in  which  simulators  were  employed  was  significantly  higher  than  studies  in  which  interactive 
multimedia  were  employed  as  delivery  systems.  There  were  no  significant  differences  found  between  the  mean 
comparison  of  studies  in  which  computer-based  interactive  videodisc  and  interactive  multimedia  were  used.  In 
addition,  no  significant  differences  were  found  between  the  mean  comparison  of  studies  in  which  simulators  and 
computer-based  interactive  videodisc  were  employed. 

Finally,  the  post  hoc  test  for  comparison  group,  (F(5,  48)  =4.876,  p<.01),  showed  that  the  mean 
comparison  of  studies  with  no  comparison  group  (i.e.,  one  group  repeated  measure)  was  significantly  higher  than 
studies  in  which  the  comparison  groups  using  traditional  instruction,  CAI,  textbooks  only,  or  other  types  of 
instruction  (e.g.,  audiotapes).  In  addition,  the  mean  comparison  of  studies  in  which  the  comparison  group  using 
videotapes  was  significantly  higher  than  the  studies  in  which  the  comparison  groups  using  CAI. 


CONCLUSION 

The  results  from  this  study  suggest  that  the  effects  of  using  hypermedia  in  instruction  are  positive  over 
non-hypermedia  instruction  as  a whole,  however,  the  effects  may  be  varied  depends  on  what  type  of  instruction 
that  hypermedia  compares  to.  The  results  of  this  study  also  provide  some  evidences  which  disagree  with  Clark's 
viewpoint  about  the  relationship  between  media  and  learning.  While  many  educators  devote  tremendous  efforts 
with  great  expectation  that  technology  will  dramatically  increase  students*  academic  achievement,  the  results  of 
this  study  provide  to  classroom  teachers  an  accumulated  research-based  evidence  for  using  technology  in 
instruction.  Left  unanswered  is  the  question  of  what  factors  truly  affect  the  diverse  outcomes  for  different  types  of 
instructions.  Studies  of  this  question  will  require  further  clarification  of  the  distinct  attributes  between  hypermedia 
and  various  types  of  instructions,  and  their  relationships  with  learning.  This  meta-analysis  points  out  only  that 
improvements  of  students*  academic  achievement  are  possible.  That  information  by  itself  is  useful. 
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Although  many  studies  have  been  conducted  on  the  hypothesis  that  women  and  girls  are  more  likely 
to  hold  more  negative  attitudes  toward  computers.  (Blumer,  1987;  Campbell,  1988;  1989;  Chen,  1986; 
Koohang,  1989;  Loyd  & Gressard,  1986;  Meier,  1988;  Wallace  & Sinclair,  1995;  Woodrow,  1994),  the  results 
of  existing  studies  have  reported  a confusing  picture.  For  example,  Campbell  (1990),  Chen  (1986),  Colley, 
Gale,  & Harris  (1994),  Jacobson  (1991),  Loyd  & Gressard  (1986),  and  Woodrow  (1994)  all  reported 
significantly  higher  computer  attitudes  for  males  than  females,  while  Blumer  (1987),  Campbell  (1986;  1989), 
Cantrell  (1995),  Chu  & Spires  (1991),  Francis  (1994),  Koohang  (1987;  1989),  Loyd  & Gressard  (1985),  and 
.Robertson,  Calder,  & Jones  (1995)  have  indicated  that  there  is  little,  if  any,  differences  of  computer  attitudes 
between  males  and  females.  Owing  to  the  mixed  evidence  provided  by  existing  research  in  the  area,  this 
study  attempts  to  use  the  meta-analytic  approach  to  investigate  the  gender  differences  on  attitudes  toward 
computers.  The  results  from  this  meta-analysis  will  help  provide  clearer  conclusion. 


PROCEDURES 

The  research  method  used  in  this  study  is  the  meta-analytic  approach  which  was  similar  to  that  described  by 
Glass,  McGaw,  & Smith  (1981).  The  studies  considered  for  use  in  this  meta-analysis  came  from  three  major 
sources  and  were  published  from  1984  to  1997.  One  large  group  of  studies  came  from  computer  searches  of 
Education  Resources  Information  Center  (ERIC).  A second  group  of  studies  came  from  Comprehensive 
Dissertation  Abstracts.  A third  group  of  studies  was  retrieved  by  branching  from  bibliographies  in  the  documents 
located  through  review  and  computer  searches.  One  hundred  and  six  studies  were  located  through  these  search 
procedures;  19  studies  came  from  ERIC  and  conference  proceedings,  80  studies  were  retrieved  from  published 
journals,  7 studies  were  from  Comprehensive  Dissertation  Abstracts.. 

Several  criteria  were  established  for  inclusion  of  studies  in  the  present  analysis. 

1 . Studies  had  to  assess  the  differences  between  males  and  females  on  computer 
attitudes. 

2.  Studies  had  to  provide  quantitative  results  from  both  male  and  female  subjects. 

3.  Studies  had  to  be  retrievable  from  university  or  college  libraries  by  interlibrary  loan  or  from  ERIC, 
Dissertation  Abstracts  International,  or  University  Microfiche  International. 

Outcome  Measures 

The  outcome  measured  most  often  in  the  selected  studies  was  survey  data  from  participants,  as  indicated 
in  various  instruments  for  examining  participants’  computer  attitudes.  For  statistical  analysis,  outcomes  from  a 
variety  of  different  studies  with  a variety  of  different  instruments  had  to  be  expressed  on  a common  scale.  The 
transformation  used  for  this  purpose  is  the  one  recommended  by  Glass  et  al.  (1981).  To  reduce  measurements  to  a 
common  scale.  Glass  and  his  colleagues  coded  each  outcome  as  an  Effect  Size  (ES),  defined  as  the  difference 
between  the  mean  scores  of  two  groups  divided  by  the  standard  deviation  of  the  control  group.  For  those  studies 
that  did  not  report  means  and  standard  deviations,  F,  or  t,  values  were  used  for  estimating  the  ES. 

Variables  Studied 

Nine  variables  were  selected  for  coding  each  study  in  the  present  synthesis.  The  first  2 variables  (nation 
of  Subject,  population  Group)  were  coded  so  that  potential  different  effects  for  subjects  with  different  background 
could  be  detected.  The  following  2 variables  (i.e.,  type  of  publication  and  year  of  publication)  were  coded  because 
it  is  important  to  know  how  effects  are  related  to  sources  of  information  over  time.  The  next  4 variables  (sample 
Size,  reliability  of  measure,  statistical  power,  and  statistics)  were  coded  so  that  effects  related  to  characteristics  of 
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research  design  could  be  detected.  The  last  variable  (type  of  Attitude)  was  coded  because  it  is  critical  to  know  how 
effects  are  associated  with  different  types  of  attitudes.  Each  variable  was  employed  as  a factor  in  an  analysis  of 
variance  (ANOVA)  to  investigate  whether  there  were  significant  differences  within  each  variable  on  the  effect 
size. 


RESULTS 

Of  the  106  studies  included  in  the  present  synthesis,  85  or  80%  of  the  study-weighted  ESs  were  positive 
and  favored  the  male  subjects,  while  21  or  20%  of  them  were  negative  and  favored  the  female  subjects,  indicating 
that  males  had  lower  computer  anxiety  than  females-  "I  he  range  of  the  study-weighted  ESs  was  from  -0.85  to 
0.881.  The  overall  grand  mean  for  all  106  study-weighted  ESs  was  0.192.  When  this  mean  ES  was  converted  to 
percentiles,  the  percentiles  on  computer  attitudes  were  58  for  the  male  subjects  and  50  for  the  female  subjects. 

The  standard  deviation  of  0.286  reflects  the  small  variability  of  ESs  across  studies. 

Among  the  489  ESs  included  in  the  present  synthesis,  363  or  74%  were  positive  and  favored  the  male 
subjects,  while  121  or  25%  were  negative  and  favored  the  female  subjects.  Only  1 or  1%  of  the  ESs  indicated  no 
difference  between  male  and  female  subjects.  The  range  of  the  ESs  was  from  -0.85  to  0.881 . 

The  positive  means  for  liking,  belie f/usefulness,  confidence,  and  mixed  indicate  more  positive  attitudes 
favor  male  subjects;  however,  for  sex  stereotype  and  ability  stereotype,  the  positive  means  indicate  a higher  sex 
and  ability  stereotype  for  male  subjects. 

For  ANOVA,  1 variable,  type  of  attitude,  showed  statistically  significant  impact.  The  post  hoc  test  for 
type  of  attitude,  (F(6,232)  - 2.554,  P<.05),  showed  that  the  mean  comparison  of  studies  that  measured  sex 
stereotype  was  significantly  higher  than  studies  that  measured  liking,  anxiety,  belief/usefulness,  confidence,  ability 
stereotype  and  mixed  attitudes.  In  addition,  the  mean  comparison  of  studies  that  measured  belief  was  significantly 
higher  than  studies  that  measured  mixed  attitudes. 


DISCUSSION 

The  results  of  this  meta-analysis  indicate  that,  overall,  male  subjects  had  slightly  higher  computer 
attitudes  than  female  subjects.  An  effect  is  said  to  be  small  when  ES  - 0.2,  medium  when  ES  - 0.5  and  large 
when  ES  - 0.8  (Cohen,  1977).  Eighty  percent  of  positive  study-weighted  ES  values  and  74%  of  positive  ESs 
overall  also  confirm  the  gender  differences  on  computer  attitudes.  The  slightness  of  the  differences  must  be  keep 
in  mind,  however;  the  overall  study- weighted  mean  ES  of  0.192  only  indicates  8 percentile  scores  higher  than  the 
female  subjects.  The  percentile  scores  for  the  overall  grand  mean  and  median  were  identical,  reflecting  the  small 
variability  of  ESs  across  studies. 

The  significant  differences  found  among  the  mean  comparison  of  studies  that  measured  sex  stereotype  and 
studies  that  measured  liking,  anxiety,  belief/usefulness,  confidence,  ability  stereotype  and  mixed  attitudes  were 
quite  interesting.  It  is  possible  because  male  subjects  hold  higher  sex-related  stereotype  toward  computer  use,  and 
this  tendency  influences  female  subjects’  feelings  about  computers  that  results  in  females’  overall  more  negative 
attitudes  toward  computers.  In  short,  the  gender  differences  on  sex-related  stereotype  toward  computer  use  may  be 
the  key  factor  that  yields  the  gender  differences  on  overall  attitudes  toward  computers.  More  studies  need  to  be 
addressed  on  this  hypothesis. 


CONCLUSION 

Although  many  studies  have  been  conducted  to  examine  the  hypothesis  that  women  and  girls  are  more 
likely  to  hold  more  negative  attitudes  toward  computers  (Blumer,  1987;  Campbell,  1988;  1989;  Chen,  1986; 
Koohang,  1989;  Loyd  & Gressard,  1986;  Meier,  1988;  Wallace  & Sinclair,  1995;  Woodrow,  1994),  and  still  fail  to 
get  a clearer  picture,  the  results  of  this  study  provide  an  accumulated  research-based  evidence  to  support  this 
hypothesis.  Left  unanswered  is  the  question  of  what  factors  truly  contribute  to  the  differences.  Studies  of  this 
question  will  require  further  clarification  of  the  exact  characteristics  of  each  type  of  computer  attitude  (e.g., 
anxiety,  confidence,  and  sex-related  stereotype)  and  their  relationships  with  gender.  This  meta-analysis  points  out 
only  that  gender  differences  on  attitudes  toward  computers  exist.  That  information  by  itself  is  useful. 
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Introduction 


The  evaluation  of  educational  software  can  be  divided  into  two  phases:  1.-  the  formative  evaluation  which  is  done 
during  software  design  and  development,  its  aim  is  to  determine  the  capability  of  the  software  to  satisfy  user  learning  needs, 
and  2.-  the  summative  evaluation  which  is  done  once  the  software  is  implemented,  its  purpose  is  to  relate  the  context  of 
usage  to  learning  effectiveness.  In  both  phases,  educators  and  software  developers  have  relied  on  traditional  methods  of 
evaluation  like  interviews,  direct  observation,  pre-tests  and  post-tests.  During  the  past  few  years,  the  number  of  educational 
applications  that  is  published  on  the  World  Wide  Web  has  increased  considerably,  meaning  that  we  can  no  more  evaluatethe 
usage  of  software  in  a controlled  environment. 

The  purpose  of  this  project  is  the  development  of  a monitoring  program  that  automatically  generates  information  that  is 
useful  for  carrying  out  summative  evaluation  on  educational  software  that  is  used  on  the  World  Wide  Web.  The  program 
collects  data  (measures)  on  a group  of  variables,  several  of  the  results  are  expressed  in  terms  of  fixed  parameters  that 
represent  the  logical  or  recommended  student  performance.  The  monitoring  program  stores  the  measures  in  a database:  raw 
data  can  be  consulted,  and  statistics  are  produced  on  demand.  In  this  way,  the  results  are  available  for  analysis  and 
evaluation. 

Experts  in  design  and  usage  of  educational  software  introduce  the  parameters.  The  final  comparison  and  analysis 
should  be  done  by  experts  in  content,  usage,  and  educational  techniques,  technical  and  graphical  experts,  and  language 
communication  experts.  An  important  part  of  the  project  has  been  the  development  o a framework  for  analysis  that  relates 
measures  to  software  characteristics:  these  include  flexibility,  legibility,  consistency,  grammatical  correctness,  interface, 
content,  degree  of  interaction,  browsing  capabilities,  media  resources  and  technical  features.  For  each  characteristic  there  is 
a set  of  indicators  of  possible  software  flaws. 

The  implementation  of  variables  has  been  done  for  a prototype  tutorial  course  on  Local  Area  Networks  that  is 
representative  of  a hybrid  educational  application  (tutorial  & drill).  The  course  was  developed  in  HTML.  The  Variables 
Database  was  developed  in  the  Informix  DBMS.  The  monitoring  program  includes  code  in  both  JavaScript  and  Informix 
4GL  CGI  (Common  Gateway  Interface).  The  next  research  phase  will  be  the  analysis  of  data  collected  by  the  monitoring 
program  during  its  usage  by  a group  of  Systems  Engineering  Students.  Simultaneously,  data  will  be  collected  and  analyzed 
through  more  traditional  methods.  By  comparing  both  analysis,  the  monitoring  program  will  be  validated. 


Use  of  hypermedia  in  educational  applications 

Multimedia  becomes  Hypermedia  when  the  designer  provides  a structure  of  elements  through  which  the  user  can 
browse  interactively  according  to  his  interests  (Vaughan  94). 

The  prefix  “Hyper”  refers  to  the  methods  for  non-sequential  browsing  through  an  information  content.  The  suffix 
“Media”  is  related  to  the  availability  of  the  information  in  different  forms  such  as  audio,  video,  graphics,  animation  and  text. 
Two  terms  are  considered  fundamental  in  hypermedia  systems:  link  and  node.  A link  is  a connection  between  conceptual 
elements,  and  a node  contains  text,  graphics,  sounds  or  any  information  related  to  the  knowledge  base. 

The  hypermedia  concept  has  allowed  to  question  the  manner  in  which  information  is  structured  to  produce  learning 
(Rivard  95).  Traditional  instruction  intend  to  create  links  between  student’s  previous  knowledge  and  new  knowledge.  These 
links  are  created  step  by  step,  in  a manner  that  is  conceptually  sequential.  Hypermedia  structures  information  in  a non 
sequential  manner  through  associative  links,  causing  individualized  knowledge  acquisition  and  manipulation. 

Links  are  the  navigation  “guides”.  Navigation  tends  to  become  more  complicated  when  an  associative  link  connects  to 
a node  that  has  several  associative  links  as  well.  The  evaluation  of  Hypermedia  educational  applications  should  take  into 
account  the  usage  of  media  resources  and  the  navigation  path  followed. 


Monitoring  Program 

The  variables  established  for  the  evaluation  of  educational  software  were  determined  from  the  inherent  characteristics 
of  an  hypermedia  educational  software.  Each  variable  takes  values  as  a result  of  the  measurement  of  student  performance 
during  software  use. 


1437 


The  monitoring  program  takes  the  measures  of  each  variable  and  stores  them  in  a database.  The  interaction  with  the 
database  was  developed  with  the  Informix  CGI  Library  which  extends  Informix  4GL  capabilities  for  the  development  of 
Web  applications.Two  different  approaches  were  followed  for  programming  the  measures  taken  by  the  monitoring  program: 

• Directly  in  the  HTML  page  belonging  to  the  educational  software,  using  a routine  written  in  Javascript,  and  then 
stored  in  the  database  using  CGIs. 

• In  the  CGIs  themselves 

The 


Some  of  the  measures  are  calculated  using  data  already  stored  in  the  database.  For  example,  the  number  of  correct  and 
incorrect  answers,  number  of  times  a content  or  help  page  is  visited,  number  of  times  an  audio  or  video  resource  is  invoked, 
and  total  execution  time  of  the  application. 

The  parameters  establish  standards  or  a value  range  for  each  of  the  variables  against  which  the  results  can  be  analyzed 
and  measured.  A few  of  the  parameters  are  the  following: 

• Recommended  navigation  path 

• Estimated  total  time  for  each  section 

• Number  of  times  a resource  should  be  invoked 

• Number  of  times  the  Help  section  should  be  visited 

• Recommended  execution  time  of  each  session 

For  instance,  for  the  variable  Navigation  Path  some  of  the  results  are  the  following: 

Rl.  The  navigation  path  is  the  same  as  the  recommended  navigation  path. 

R2.  The  navigation  path  is  different  form  the  recommended  navigation  path. 

R3.  The  student  visits  several  times  the  same  section. 

R4.  The  student  did  not  visit  a certain  section. 

The  result  Rl  is  related  to  the  characteristics  Flexibility  and  Sequence.  Flexibility  is  the  degree  in  which  the  software 
permits  the  student  to  work  at  his  own  pace  or  according  to  his  interests,  and  Sequence  has  to  do  with  structure,  location 
indicators,  ability  to  stop  and  resume  in  the  same  place,  and  exit  options.  These  characteristics  could  lead  to  possible 
software  flaws. 


measures  are  the  following: 

Navigation  Path:  These  are  stored  in  the  database  as  a string  consisting  of  the  names  of  the  pages  visited  separated 
by  a special  character.  The  page  name  is  brought  from  the  page  to  the  CGI. 

Time:  The  moment  the  browser  loads  the  HTML  page  belonging  to  the  educational  software  the  time  is  stored  and 
then,  when  the  student  leaves  the  page,  the  CGI  script  is  called  and  the  difference  between  both  times  is 
calculated. 

Questions  & Answers:  The  answer  is  evaluated  by  the  educational  software,  and  then  sent  to  the  CGI. 


oiiuiiig: 
the  section. 


Conclusions 

The  design  and  development  of  educational  software  has  been  an  important  research  area  in  many  universities, 
specially  in  those  that  offer  Distance  Education;  however,  computerized  evaluation  has  been  an  area  little  explored.  The 
monitoring  program  described  in  this  paper  is  the  first  phase  of  a project  that  is  aimed  to  achieve  evaluation  in  a context 
where  other  evaluation  methods  are  not  available. 

The  monitoring  program  not  only  allows  evaluation  by  the  design  and  usage  experts  that  are  related  directly  with  a 
certain  educational  software,  but  also  by  any  professor  that  uses  the  World  Wide  Web  as  a means  of  instruction  for  his 
Distance  Education  Students. 

It  should  be  noted  that  the  time  parameters  can  be  affected  by  the  variability  of  connection  rates  due  to  factors  like  the 
telephone  network,  the  Internet  service  provider,  and  the  Web  Server  demand. 

It  is  recommended  that  pilot  tests  be  done  with  the  purpose  of  determining  the  parameters  needed  for  further  analysis 
and  evaluation. 
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The  Logo  language  (Harvey,  B.,  1997)  has  been  ideal  for  students  first  approach  to  computer  science:  today  the 
evolution  of  systems  and  programming  languages  has  caused  the  evolution  of  its  implementations  including 
multimedia,  graphics  and  concurrency  features  (Cistemino.  et  at.,  1 998)  (Resnick  1 990). 

In  this  paper  we  propose  a micro-world  based  on  the  Logo  programming  philosophy:  the  user  may  program 
interacting  entities  through  a language  that  integrates  the  Logo  turtles  primitives  with  constructs  to  co-ordinate 
and  synchronise  concurrent  agents.  The  power  of  this  system  is  allowing  a decentralised  way  of  modelling  the 
world:  a creature  has  “conscience''  of  itself  and  interacts  with  the  surrounding  environment  in  explicit  way  using 
communication  primitives.  This  approach  is  different,  for  instance,  from  that  of  the  StarLogo  system  (Resnick 
1990)  where  a turtle  is  view  as  part  of  a community. 

The  system  we  present,  is  an  evolution  of  the  Orespics  system  (Capretti.  et  al.,  1998),  an  environment  in  which 
students  guide  the  action  of  virtual  robots  through  the  Logo-  PL  language.  The  environment  includes  a friendly 
graphical  interface  by  which  students  can  use  Orespics  as  a game.  The  language  is  provided  with  traditionally 
sequential  constmcts,  constructs  to  control  the  concurrency  and  communication  primitives  (Hoare,  1978).  In  our 
experimentation,  we  have  noted  that  this  set  of  concurrent  constmcts  is  not  sufficient  to  solve  all  problems  that 
students  fantasy  suggests  them.  We  have  extended  Orespics  system  to  define  the  “Advanced  Orespics” 
environment,  a new  system  including  non-deterministic  features  and  other  forms  of  communication;  furthermore 
the  students  may  define  “creatures”  appearance.  This  new  type  of  interaction  among  creatures  allows  exploring 
the  world  in  a decentralised  way:  a child  has  to  modify  his/her  way  of  thinking  according  to  this  paradigm. 

We  present  the  spirit  of  Advanced  Orespics  through  an  example:  we  show  how  the  classical  problem  of  Hanoi 
towers  may  be  programmed  in  our  system  by  defining  the  behaviour  of  each  disk  in  a decentralised  way:  each 
disk  has  its  own  identity  and  may  interact  with  the  word  autonomously. 

The  well  known  Hanoi  tower  problem  is  the  following:  a monk  has  three  pegs  and  has  to  move  n disks  of 
increasing  dimension  from  a peg  to  another  one  with  the  constraint  that  a disk  may  be  put  on  another  one  if  and 
only  if  its  dimension  is  smaller.  The  problem  of  Hanoi  is  a typical  example  of  the  power  of  recursion  in  the 
classic  Logo  language.  We  exploit  the  same  problem  to  show  the  power  of  a decentralised  way  of  thinking.  We 
believe  that  an  agent  approach  is  intuitive  for  this  problem:  each  disk  is  an  agent  that  is  capable  to  move  from  a 
peg  to  another  peg  in  respect  to  the  problem  constraints.  The  disk  is  capable  to  receive  the  message  “move  to  peg 
jc”  and  knows  the  peg  where  it  is  located.  In  order  to  move  from  a peg  jc  to  a peg  there  are  two  possibilities:  the 
selected  disk  D is  on  top  of  x or  not. 

If  D is  on  the  top,  we  consider  two  sub-case:  if  the  topmost  disk  on  the  destination  peg  is  bigger  then  it,  D moves 
from  Xto  Y immediately,  otherwise  D asks  to  the  lowest  disk  smaller  than  it  on  the  destination  peg,  to  move  to 
the  third  peg  Z.  If  D is  not  on  the  top,  first  it  asks  to  the  disk  above  to  move  to  peg  Z in  order  to  become  the 
topmost  disk  and  then  it  behaves  as  describe  above. 

There  is  a starting  process  that  tells  to  each  disk  starting  from  the  biggest  dimension  one,  that  it  must  reach  the 
goal  peg. 

This  solution  consists  in  creating  a different  creature  for  each  disk  and  a monk  creature  that  activates  the  disk 
movement:  a different  process  is  associated  with  each  creature.  We  use  Orespics  PL  syntax  (Capretti  et  at.  1999) 
to  write  disks  and  monk  programs.  The  programs  are  the  following: 


Agent  monk 
i <-  1; 

while  ( i <=  n ) 
send&wait  C to  disk_i; 
wait&receive  OK  from  disk_i; 
i <“i+l; 
endwhl 1 e 
end 

Agent  disk_i; 
myPeg  <-  A; 
myDim  <-  n-i+1; 
repeat 

wait&receiveAny  peg; 

while  ( dimTcp  (peg)  < myDim  or  so^iUp  ()  <>  nil) 
if  (seeDpO  <>  nil  ) 
receiver  <-  seeUpi) ; 
else 

receiver  <-  Max_0f_5maller (peg,  myDim) ; 

endif 

send&wait  third (peg,  mypeg)  to  receiver; 
wait&receive  OK  from  receiver; 
endwhile ; 
l_go  (peg)  ; 

Figure  1 : The  first  movement 


send&wait  OK  to  Sender(peg); 
until  ( i s_My_Ri gh t_ Posi tion{)  ) ; 

end 

where  seeUp,  dimTop^  max_Of_Smaller,  third,  is_My_Right_Position  and  I_Go  are  macros  created  by  the 
teacher.  seeUpQ  returns  the  identification  of  the  disk  located  above  the  disk  invoking  the  function;  dimTop(P) 
receives  the  identification  of  a peg  P and  returns  the  dimension  of  the  disk  on  the  P top;  Max_Of_Smaller(P,  d) 
function  receives  in  input  the  identification  of  a peg  P and  a dimension  d and  returns  the  identification  of  the  first 
disk  on  the  peg  P whose  dimension  is  the  greater  of  the  smaller  than  d.  The  macro  third (x,  y)  returns  the 
identification  of  the  peg  different  from  X,  Y pegs  and  is_My_Right_PositionQ  returns  true  if  the  disk  is  in  the 
right  order  on  the  peg  C.  The  I_Go(P)  macro  manages  the  movement  of  the  disk  from  the  actual  peg  to  the  P one 
and  updates  the  value  of  the  my  Peg  variable  with  this  new  value. 

We  stress  the  fact  that  the  disks  are  numbered  from  1 to  n,  from  the  greatest  to  the  smaller  one. 

The  macro  defined  may  be  implemented  through  Orespics-  PL  language,  for  instance  the  teacher  may  implement 
a state  manager,  storing  how  the  disks  are  distributed  on  the  pegs.  This  means  that  the  student  is  able  to  learn  the 
laws  of  the  worlds  till  he  gains  the  skills  to  program  all  the  problems. 

In  this  program  we  use  the  primitives  send&wait  and  wait&receive  allowing  a synchronous  communication 
between  agents:  Orespics  system  includes  a rich  set  of  communications  to  describe  better  different  real 
situations.  For  the  first  time  these  features  are  included  in  a language  for  teenagers. 

Since  the  world  may  be  perceived  as  a set  of  interacting  entities,  we  think  that  it  may  be  naturally  modelled  in 
our  system  through  agents  and  communications. 

REFERENCES 

Capretti,  G.,  Lagana,  M.R.  & Ricci,  L.  (1998).  Microworld  to  learn  concurrency,  SSCCV8.  Durban,  SouthAfrica,  VA.  255- 
259. 

Capretti,  G.,  Lagana,  M.R.  & Ricci,  L.  (1999).  Learning  concurrency:  a constructionistic  approach.  Pact  '99.  St.  Petersburg, 
Russia. 

Cistemino,  A.  Lagana  , M.R.  YALL:Yet  Another  Logo  Language,  ISCEE  98,  Amiens,  France. 

Epistemology  and  Learning  Group  (1996).  StarLogo  reference  Manual.  Cambridge:  M.I.T.  Media  Laboratory. 

Harvey,  B.  (1997).  Computer  Science  Logo  Style.  Cambridge:  The  MIT  Press. 

Hoare  C.  A.  R.  (1978).  Communicating  Sequential  Process.  Comm,  of  the  ACM,  21  (8),  666-677. 

MicroWorlds,  http://www.microworlds.com/. 

Papert  S.(1980).  Mindstorm:  Children,  Computers  and  Powerful  Ideas.  New  York:  Basic  Books. 

Resnick,  M.  (1990).  Multilogo,  a Study  of  Children  and  Concurrent  Programming.  Interactive  Learning  Environment,  1 (3) 


1440 


Evaluating  On-line  Courses 


Ulric  Bjorck 

Goteborg  University,  Department  of  Education,  Goteborg,  Sweden 
ulric.biorck@ped.gu.se 


Introduction 

A broader  use  of  on-line  courses  has  become  more  widespread  in  recent  years.  Virtual  universities  are 
popping  up  almost  all  over  the  world  and  the  development  towards  more  on-line  education  in  higher 
education  is  continuing  (Brown,  1998;  Tomlinson-Keasey,  1998).  The  use  of  Computer  Mediated 
Communication  (CMC)  is  affecting  and  changing  the  society,  as  we  know  it.  A common  opinion  and  a 
frequent  one  in  a lot  of  research  is  that  the  computer  is  simply  a tool.  We  find  that  the  computer  indeed 
is  a tool,  and  a very  powerful  one,  but  the  computer  and  especially  the  use  of  the  computer,  is  much 
more  than  a tool.  In  fact,  the  use  of  computers  and  especially  the  Internet  is  changing  some  of  the 
fundamentals  in  the  world  today  and  it  definitively  affects  several  areas. 

Dowling  (1998)  stresses  that  the  use  of  CMC  should  be  studied  focusing  more  on  the  social  and 
relational  aspects  of  our  virtual  lives  since  they  may  very  well  prove  to  be  important  for  learning. 
Research  on  the  development  and  use  of  communication  conventions  in  CMC  has  been  going  on  for 
quite  some  years  now  (Murphy,  1997),  but  theoretical  foundations  of  evaluating  on-line  courses  have 
not  been  given  a lot  of  room  in  the  current  research  debate.  In  many  cases,  transcripts  of  asynchronous 
conferencing  in  instructional  environments  have  been  studied  (Harasim,  1990;  Henri,  1992). 
Analyzing  transcripts  has  been  established  as  a good  way  to  study  communication  and  cognition  in 
coordination  with  the  ideas  within  the  scientific  field  of  situated  cognition.  This  can  be  recognized  as 
guidelines  and  valuable  information  in  evaluating  on-line  courses  since  the  knowledge  built  and 
constructed  within  these  particular  frameworks  could  prove  useful  to  the  evaluator.  Theories  and 
writings  about  evaluations  in  a broader  context  are  quite  extensive  (Lander,  1987).  In  this  paper  I will 
elaborate  the  views  and  writings  that  have  proved  to  be  significant  in  evaluating  on-line  courses.  The 
focus  will  be  on  the  use  of  formative  evaluations. 

Formative  evaluations 

Cronbach  (1983)  describes  the  design  of  an  evaluative  investigation  as  an  art.  There  are  several 
decisions  to  be  made  and  the  choices  to  be  made  are  almost  innumerable.  The  choices  that  are  made 
will  undoubtedly  lead  to  particular  advantages  and  disadvantages.  It  is  therefore  important  to  focus  on 
considerations  and  choices  behind  the  design  of  the  evaluation.  An  evaluation  plan  is  constructed  to 
secure  that  the  resources  available  for  the  evaluation,  and  the  research,  is  used  in  such  a way  that  as 
much  useful  and  necessary  data  as  possible  are  collected. 

A single  definite  question  is  often  the  starting  point  of  an  evaluation,  but  this  has  begun  to  change  when 
it  comes  to  evaluating  CMC  (Kinzie,  1991;  Tessmer,  1994).  A single  question  assumes  that  the 
objective  of  the  course  has  been  fixed  and  that  the  aim  of  the  evaluation  is  to  assess  its  merit.  Scriven 
(1967)  christened  such  evaluations  summative;  to  contrast  them  with  the  formative  study  which  regards 
the  program  as  fluid  and  seeks  ways  to  improve  it.  The  formative  evaluation  method  could  be  described 
as  a more  diagnostic  evaluation  method  than  the  summative.  Cronbach  (1983)  stresses  that  in  a 
formative  evaluation  what  variables  deserve  close  attention  will  be  discovered  as  the  fieldwork 
proceeds  which  is  something  that  seems  to  suite  the  designers  and  evaluators  of  today’s  computer 
applications.  The  proper  function  of  evaluation  is  to  speed  up  the  learning  process  by  communicating 
what  might  otherwise  be  overlooked  or  wrongly  perceived.  The  evaluator,  then,  is  an  educator.  His 
success  is  to  be  judged  by  his  success  in  communication;  that  is,  by  what  he  leads  others  to  understand 
and  believe.  Payoff  comes  from  the  insight  that  the  evaluator's  work  generates  in  others  (p  8).  Scriven 
claims  that  the  “curriculum  builder  is  almost  automatically  engaged  in  formative  evaluation,  except  on 
a very  strict  definition  of  "evaluation".  He  is  presumably  doing  what  he  is  doing  because  he  judges  that 
the  material  being  presented  in  the  existing  curriculum  is  unsatisfactory”  (p  43). 


Scriven’s  writings  are  particularly  interesting  today  because  an  additional  unsatisfactory  part  in 
evaluating  on-line  courses  is  the  use  of  the  on-line  course  environment  in  itself  It  is  obvious  that  this 
environment  is  not  100%  satisfactory  and  as  the  unsatisfied  evaluator  it  is  therefore  important  to 
proceed  and  construct  new  solutions.  Scriven  holds  the  formative  evaluation  in  favor  of  a traditional 
summative  approach  by  the  basic  fact  that  the  “ordinary**  evaluator,  while  a professional  in  his  own 
field,  is  usually  not  a professional  in  the  field  relevant  to  the  curriculum  being  reformed.  Scriven’s 
description  is  often  accurate  today  when  computer  scientists  and  designers  are  preparing  new  on-line 
course  environments  for  students  in  other  courses  than  their  own  subject.  The  meetings  between  the 
evaluator  and  the  professional  in  the  field  often  leads  to  the  exchange  of  knowledge  and  a broader 
perspective  on  both  accounts. 

Results 

Formative  evaluations  provide  a sound  and  flexible  possibility  to  evaluate  on-line  courses.  The  on-line 
course  environment,  which  is  supported  by  the  Internet,  is  a continuously  changing  environment  and 
therefore  the  formative  evaluation  method  is  particularly  suitable  to  meet  higher  demands  on  quick 
changes  of  the  course.  An  additional  aspect  supporting  the  formative  evaluation  approach  is  that  a lot 
of  on-line  courses  are  not  in  a regular  educational  program,  they  are  often  done  within  a particular 
project  environment.  If  these  evaluations  were  summative,  any  suggestions  of  improving  the  courses 
would  be  too  late  for  the  current  project. 

The  formative  evaluation  method  has  been  given  a diagnostic  function,  rather  than  a function  of 
making  final  conclusions  and  standpoints.  The  curriculum  builder  is  almost  automatically  engaged  in 
formative  evaluation,  except  on  a very  strict  definition  of '*evaluation“  (Scriven,  1967).  The  evaluator  is 
presumably  doing  what  he  is  doing  because  he  judges  that  the  material  being  presented  in  the  existing 
curriculum  is  unsatisfactoiy.  One  unsatisfactoiy  part  of  on-line  courses  could  be  said  to  be  the  use  of 
the  on-line  course  environment.  It  is  obvious  that  this  environment  is  not  100%  satisfactory  and 
because  of  this  it  is  important  to  proceed  to  construct  new  solutions.  In  constructing  new  solutions  and 
communicating  them,  meetings  with  the  course  management  are  crucial  in  succeeding  as  a formative 
evaluator. 
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Introduction 

A Web-Based  Instruction(WBI)  approach  employs  a wide  range  of  instructional  strategies  using  the  resources  of  the  World 
Wide  Web  (Pemici  et  al.  1997),  (Reeves  & Reeves  1997).  The  Web  environment  promotes  active  knowledge  acquisition  and 
collaborative  learning,  allowing  students  to  interact  with  a variety  of  information  and  to  communicate  with  different  people.  In 
the  process  of  designing  a WBI  application,  we  can  explore  the  fact  that  instructional  design  does  not  have  a rigid  structure,  due 
to  its  multiple  paths  to  solutions.  Course  designers  traditionally  treat  constructs  as  regular  and  well-structured,  idealizing  a best 
solution  presented  in  a plan  of  action  to  be  strictly  followed  by  the  students.  A traditional  instructional  designer  believes  that 
there  is  a single,  best  way  to  conceive  knowledge;  in  other  words,  there  is  a single  schema  or  concept  that  best  describes  any 
object  or  event.  That  best  method  is  revealed  by  task  analysis.  However,  in  order  to  comprehend  concepts,  learners  should 
understand  and  personally  form  their  own  intentions  based  on  different  perspectives  of  content  and  action.  The  aim  of  this  work 
is  to  provide  controlled  cognitive  flexibility,  in  which  different  perspectives  are  explored  and  represented  in  a structure  that 
allows  distinct  plans  of  action  to  be  generated.  Students  should  be  able  to  follow  their  intentions,  forming  their  own  plans  of 
action  and  at  the  same  time  be  attracted  to  the  expectations  and  requirements  of  the  course.  The  students’  intentions  will  be 
influenced  by  their  instructor  and  by  other  students  in  order  to  improve  performance.  Students  should  have  relative  freedom  of 
action,  but  there  should  be  some  way  whereby  it  is  possible  to  detect  whether  they  are  moving  away  from  the  objectives  proposed 
by  the  course  instructor  and/or  below  the  performance  of  the  group  of  students  attending  the  course.  It  should  be  possible  to 
assess  each  student  from  time  to  time.  If  a student  has  a low  grade,  the  instructor  may  impose  some  kind  of  behavior  until  the 
next  assessment,  by  means  of  restricting  the  possibilities  of  creating  his/her  plan  of  action.  In  addition,  at  any  point  in  the  flow  of 
action,  a visualization  of  the  information  about  the  group’s  previous  interaction  and  the  different  possible  future  paths  of  action 
should  suggest  the  direction  to  be  followed  by  the  student. 

Workflow  Technology  and  Learningware 

The  subjacent  technology  used  to  pursue  the  proposed  objectives  is  Workflow  technology  (Abbot  & Sarin  1994),  (Benjamin 
et  al.  1995),  (Dourish  1996),  (Glance  et  al.  1996),  (Khoshafian  et  al.  1995),  (Miller  et  al.  1995),  (Walters  1995),  (Winograd  et  al. 
1992).  A Workflow  has  an  infra-structure  that  facilitates  the  execution  and  control  of  work.  A Workflow  system  automates  plans 
of  action,  by  means  of  a process  model  that  addresses  a sequence  of  tasks  to  be  rigidly  followed  by  its  users.  The  users’  actions 
are  restricted  to  valid  movements  predicted  in  a flow  of  action.  Workflow  aids  the  rationalization  and  explanation  of  action, 
however  its  flow  has  to  be  strictly  obeyed.  Therefore,  students  are  easily  presented  with  the  course  objectives,  however  they  are 
not  able  to  be  active  in  learning.  A fundamental  point  in  this  work  is  to  adapt  Workflow  technology  to  fit  an  appropriate  cognitive 
flexibility  in  an  educational  context.  The  adapted  Workflow  structure  must  maintain  advantages  and  eliminate  disadvantages.  The 
focus  is  to  use  a clear  structure  to  present  the  material  available  and  requirements  as  well  as  to  provide  multiple  representation  of 
content  and  different  paths  of  action. 

A central  point  in  this  work  is  the  elaboration  of  a flexible  Workflow  structure  related  to  learningware.  The  term 
learningware  overwhelms  the  term  courseware,  not  only  because  it  is  related  to  active  interaction  but  also  because  it  incorporates 
group  collaboration.  Characteristics  should  be  adopted  that  make  communication,  coordination,  cooperation  and  contributing  to  a 
group  synergy  feasible.  On  the  whole,  there  are  three  complementary  and  inter-dependent  aspects  that  form  the  starting  point  in 
our  general  problem  solution.  The  three  aspects  to  be  investigated  are  a flexible  Workflow  structure,  a coordination  structure  and 
a visual  interface  for  communication  and  analysis  of  plans  of  action. 

Firstly,  the  Workflow  rigidity  arises  from  viewing  work  processes  as  unfolding  along  a single  line  of  temporally  chained 
pre-fixed  activities.  A flexible  Workflow  structure  should  consider  the  possibilities  of  modifying  a workflow,  like  changing  the 
action  sequence  and  altering,  including,  and  excluding  actions.  We  intend  to  work  in  a Workflow  formalization,  in  which 
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students  will  have  active  but  predictable  behavior;  they  will  have  the  opportunity  to  change  the  action  sequence  at  some  pre- 
determined points  in  a workflow  and  to  alter  some  pre-fixed  actions.  For  example,  if  a causal  dependency  between  two  actions 
exits,  there  is  no  way  to  modify  it,  on  the  other  hand  if  a dependency  is  an  instruction  suggested  it  may  be  followed  or  changed 
by  the  student.  Students  may  alter  the  action  hierarchy,  to  decide  at  which  level  of  partonomy  or  specialization  to  act.  They  may 
navigate  through  different  levels  of  knowledge  taking  action  and  subtracting  detailed  action  or  not.  Our  flexible  Workflow 
approach  allows  users  to  control  the  way  in  which  sequencing  operations  are  carried  out  and  allows  some  activities  not  to 
proceed,  but  to  be  recognizable  in  the  system.  This  maintains  a correspondence  between  user  action  and  process  representation.  A 
mediating  between  the  active  user  interaction  and  the  work  representation,  rather  than  the  enactment  of  a process,  provides 
students  with  an  opportunity  to  enter  the  work  process  and  also  gives  students  and  instructors  control  over  the  way  the  work  is 
organized.  Our  Workflow  structure  may  be  considered  mobile,  because  it  incorporates  different  possible  plans  of  action.  By 
reaching  decisions  students  decide  which  learning  strategy  will  be  used.  They  have  autonomy,  independence,  strategic  initiatives 
and  guidance. 

Secondly,  a coordination  structure  has  been  developed  to  overcome  sources  of  rriisundcrstanding  among  learners  and  to 
promote  advanced  knowledge  acquisition,  avoiding  oversimplified  behavior.  Consider  a Workflow  instance  for  each  student: 
suppose  a student  starts  an  interaction  with  the  Workflow;  at  some  pre-determined  moment  during  the  course  an  assessment 
should  occur.  If  a student  is  not  doing  well,  he/she  will  receive  a new  Workflow.  This  may  contain  restrictions  of  flexibility  or 
changes  in  its  structure.  With  respect  to  coordination,  there  is  also  an  agent  called  Attractor.  The  Attractor  is  partially  responsible 
for  evaluating  a flow  context  and  signalizing  it  to  the  interface.  We  define  a flow  context  as  a flow  between  two  different 
moments  in  time.  Before  reaching  a decision  that  implies  action,  users  may  ask  the  Attractor  to  show  the  consequences  of  that 
decision,  in  other  words,  to  show  different  possible  future  flow  contexts  as  well  as  a partial  evaluation.  This  evaluation  may 
show,  for  instance,  whether  a flow  context  is  oversimplified.  Students  may  also  see  their  previous  flow  contexts  and/or  the 
previous  flow  contexts  of  other  students.  In  order  to  assess  learners,  instructors  require  context  information  from  the  Attractor.  A 
principal  concept  is  that  the  Attractor  suggests  instead  of  imposes. 

Thirdly,  there  is  a visual  interface  that  visually  presents  the  information  provided  by  the  Attractor.  The  interface  visually 
distinguishes  between  a good  or  bad  flow  context.  Moreover  the  interface  is  also  used  to  present  the  workflow  and  to  identify 
which  points  of  its  structure  could  be  modified  by  students.  By  means  of  the  interface,  students  are  able  to  visually  analyze  their 
future  perspectives  and  to  compare  their  cognitive  strategies  with  those  of  other  students.  Alternatively,  they  may  create  their 
plans  of  action  as  copies  of  other  students’  plans  of  action  or  as  a combination  of  various  students’  plans  of  action. 

Conclusion  and  Future  Work 

There  are  other  approaches  giving  more  flexibility  to  Workflow,  but  this  subject  has  not  been  explored  very  much  for 
educational  purposes.  There  is  no  Workflow  technology  that  is  suitable  for  every  domain.  Our  work  is  an  attempt  to  fit  this 
technology  into  an  educational  context. 
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Background  for  the  SULDAL'  project 

A recently  established  Norwegian  'NetUniversity'  (NU)  is  a formal  extension  of  a 5 years  long  collaborative 
project  between  four  Norwegian  institutions  (universities),  offering  nearly  100  different  ODL  courses  through 
Internet  to  nearly  5000  students  every  year.  An  important  characteristic  of  the  virtual  NU  is  the  mutual  ex- 
change and  joint  development  of  learning  materials,  where  students  can  cross  institutional  borders  in  their 
choice  of  subjects,  a flexibility  that  is  of  great  value  also  to  external  students. 

The  staff  at  teacher  training  institutions  needs  experience  from  field  trials  where  the  different  pedagogical  ideas 
and  technical  solutions  are  tested  in  real  educational  situations.  The  danger  is  prevalent  that  bright  ideas  and 
perfect  theories  are  borne  and  bred  in  academic  isolation,  well  shielded  from  practical  problems  and 
disturbances.  Only  in  real  situations  can  new  ideas  prove  their  true  value.  ICT  invites  for  new  learning 
situations,  changing  the  traditional  'teaching'  into  more  student  active  'learning'  environments. 

Some  local  communities  are  willing  to  invest  considerable  funds  and  up-to-date  infrastructures  in  order  to 
improve  their  local  school  systems  and  facilitate  the  lifelong  learning  process  of  their  inhabitants.  Suldal,  is  a 
scarcely  populated  municipality  with  7 small  schools,  separated  by  deep  fjords  and  steep  mountains.  A central 
objective  of  their  investment  in  iCT  is  to  strengthen  contact  and  unity  among  the  population,  fostering  collabo- 
ration and  exchange  of  skills  and  ideas.  The  schools  are  expected  to  have  central  roles  in  this  development. 

Project  methods  and  design 

Project  outline 

Problem  definitions  for  the  research  activity  concentrate  on  two  main  areas: 

1)  Organisation  and  infrastructure  for  integration  of  ICT  in  the  classroom,  in  the  school  and  in  the  total 
community  school  system.  What  are  the  goals,  and  will  the  chosen  structure  lead  to  reaching  the  goals? 
How  do  we  create  an  optimal  learning  environment? 

2)  Roles  of  the  different  actors  in  the  practical  work.  What  changes  in  roles  and  activities  are  observed  during 
the  project  period?  The  primary  actors  are  the  teachers  and  students/pupils,  but  the  roles  of  parents, 
employers  and  community  are  also  of  interest. 

Action  research  and  collaborative  learning 

Action  research  is  chosen  as  the  main  method.  It  is  based  on  close  relations  between  practice,  reflection,  and 
change.  The  framework  is  most  appropriate  for  participants  who  are  willing  to  adopt  some  initial  stance  in  re- 
gard to  a problem,  formulate  a plan,  carry  out  an  intervention,  evaluate  the  outcomes  and  then  develop  further 
strategies  in  an  iterative  fashion.  Action  research  is  by  nature  cyclical  or  iterative. 

Action  research  empowers  all  participants.  Involvement  is  of  a knowing  nature,  with  no  hidden  controls  or  pre- 
emption of  direction  by  the  researcher.  All  participants  negotiate  meaning  in  the  analysis  of  the  data  and 
contribute  to  the  selection  of  interventionary  strategies.  The  method  implies  a situation  where  informal 
evaluation  will  be  a continuously  ongoing  process,  bringing  about  feedback  of  many  kinds.  In  addition  more 

* SULDAL'  = 'SkoleUtvikling,  Laeranitdanning  og  Laeringsmiljo' 

= school  development,  teacher  training  and  learning  environment 


formal  evaluation  in  the  form  of  systematic  interviews  and  questionnaires  will  take  place  for  groups  of  relevant 
actors. 

In  traditional,  curriculum-based  learning  the  learners  work  individually,  without  sharing  their  learning  in  a 
public  forum,  and  work  largely  in  isolation.  The  method  of  collaborative  learning,  which  is  focused  here, 
engages  the  learner  in  thinking  about  why  they  are  learning,  and  how  the  learning  may  benefit  the  group.  This 
kind  of  learning  has  long  been  an  interesting  aspect  in  pedagogical  theory,  and  is  now  gaining  further  interest  in 
relation  to  new  ICT,  where  the  network  is  a major  tool  for  collaboration  (D.  McConnell;  1998). 

Who  is  learning  from  whom  in  this  project?  Naturally  the  students  should  be  learning  from  their  teachers. 
Equally  important,  however,  is  the  learning  that  takes  place  be i ween  siuderiis  - and  uetween  teachers,  where 
they  exchange  experiences  and  information  and  construct  their  own  knowledge  through  trial  and  error,  by 
sharing  the  learning  tasks  and  communicating  results  in  a positive  way.  Furthermore,  teachers  will  also  learn 
from  their  students,  particularly  when  it  comes  to  technical  skills  and  new  ways  of  finding  information  on  the 
web,  where  the  new  generation  seems  to  have  a much  higher  adaptability  than  their  parents. 

Collaboration  among  institutions  is  one  of  the  strategic  approaches  given  prominence  in  this  project.  The 
enormous  efforts  and  resources  needed  for  providing  learning  material  to  new  groups  of  learners  simply  require 
that  parallel  work  should  be  avoided.  The  participating  schools  will  benefit  from  co-operation,  exchange  of 
material  and  sharing  of  duties.  Internet  increases  the  functionality  and  facilitates  practical  activities  among  co- 
operating institutions. 

Practical  approaches 

The  practical  part  of  the  project  includes 

1.  Initiation  and  motivation,  with  installations,  infrastructure,  planning  and  basic  training  of  all 

teachers,  and  starting  further  training  of  some  interested  ‘resource  teachers’. 

2.  Integration  of  ICT  into  subject  matters  and  methods.  The  teachers  are  central  actors,  with  researchers 

and  other  resource  personnel  as  partners  to  help  find  the  most  rewarding  applications  of  ICT 

3.  Making  the  technology  transparent,  i.e.  working  with  the  learning  process  in  an  electronic  learning 

environment,  without  paying  any  attention  to  the  ICT  itself 

Obstacles  - and  how  to  overcome  them 

Major  difficulties  are  the  wide  variety  of  backgrounds  among  teachers  involved.  Some  of  them  are  ICT 
‘specialists’  with  a very  high  motivation,  eager  to  start  the  activities.  Others  are  more  moderate,  not  too  well 
prepared,  but  positive  to  trying  new  ways  and  methods  in  the  teaching  process.  A third  group  even  expressed  a 
negative  attitude  from  the  beginning,  but  gradually  seem  to  be  more  open  and  interested  in  the  new  possibilities. 

Technical  difficulties  are  not  yet  a story  of  the  past.  Despite  new  PCs,  new  networks,  professional  installations 
and  local  support,  there  are  still  problems.  A support  team  has  a vital  role,  and  an  on-line  helpdesk  is  available. 

Status 

Infrastructure  and  technical  installations  are  functioning  well 

Motivation  and  involvement  of  the  teachers  have  increased  throughout  the  period 

A training  programme  for  the  teachers  has  started,  differentiated  according  to  interests 

A common  web  site  is  available  to  all  actors:  http://hugin.hsh.no/lu/inf/suldal2/Suldal.htm 

Each  school  has  provided  a report  of  their  activities  and  intermediate  goals 

The  web  site  functions  as  an  evolving  resource  base  for  teachers,  students  and  researchers 

Action  research  is  carried  out.  Academic  staff,  MEd  and  PhD  students  participate  in  class  and  school 

activities,  and  revised  plans  and  actions  are  made 

Gradual  change  of  guidance  and  contact  from  physical  visits  to  netbased  communication  & exchanges 

References 

D.  McConnell;  1998 


1446 


A Nontraditional  Use  of  WebCT: 

Communication  Applications  in  an  Educational  Mentoring  Program. 


Sue  Elwood-Salinas,  M.  Ed. 
Texas  Tech  University 
Box  41071 

Lubbock,  Texas  79409-1071  U.S.A. 

pnnoil*  O pU*?nnrl/^t+il  f»Hll 


Annee  Daniel  Bayeux,  M.  Ed. 
Texas  Tech  University 
Box  41071 


Lubbock,  Texas  79409-1071  U.S.A. 
Email:  adaniel@ttu.edu 


Introduction 

The  purpose  of  this  paper  is  to  describe  communication  applications  within  the  program  Intercambios,  using  an 
integrated,  server  side  software  package  called  WebCT.  Intercambios:  A technology,  learning,  community  service 
and  career  awareness  program  provides  adolescent  mentoring  from  career  professionals  and  university  students. 
WebCT’s  low  learning  curve  and  sophisticated  communication  features  enhance  each  participant’s  ability  to 
collaborate  in  the  career  mentoring  process.  This  paper  will  briefly  outline  the  grant’s  objectives,  current  projects, 
and  future  developments  related  to  WebCT. 


The  Goal 

The  primary  goal  of  the  grant  is  to  decrease  the  juvenile  crime  rate  for  the  junior  high  school  participants.  Secondary 
program  goals  consist  of  increased  leadership  skills,  more  integrated  curriculum  projects  involving  the  Internet, 
greater  technology-related  field  experiences  for  university  students,  and  exposure  to  career  options  by  having  pupils 
work  with  career  and  academic  mentors.  Program  directors  looked  for  a software  program  that  would  help  them  to 
achieve  these  objectives. 


The  Problem 

The  Intercambios  program  consists  of  the  junior  high  pupils  and  teachers,  university  students  and  faculty,  and  local 
and  world  wide  career  mentors.  The  program  required  face-to-face  mentoring  combined  with  on-line  resources  to 
facilitate  communication  with  career  mentors.  Since  much  of  the  participation  involved  on-line  communication 
between  adolescents  and  adults,  it  was  vital  to  find  a software  application  that  could  provide  a monitored  and  secure 
communication  environment.  Another  important  need  of  the  program  and  feature  of  the  software  was  the  ability  to 
collaborate  on  a synchronous  and  asynchronous  level.  Email,  or  some  other  form  of  private  electronic 
correspondence,  was  a highly  desirable  feature  in  order  to  maintain  a supportive  and  confidential  relationship 
between  mentor  and  mentee.  Additionally,  discussion  groups  could  introduce  new  careers  and  mentors  to  all 
participants,  providing  a continual  renewal  of  fresh  ideas.  Therefore,  a password-protected  environment  including 
private  email,  discussion  lists,  and  chats  are  desired  features  in  the  software  package. 

Several  possibilities  exist  for  asynchronous  and  synchronous  communication  via  the  World  Wide  Web  today.  The 
project  needed  to  focus  on  applications  like  self-contained  email,  discussion  lists,  chat  sessions,  and  collaborative 
planning  tools.  Finally,  if  was  imperative  to  find  a single  software  package  with  a low  learning  curve  in  order  to 
avoid  the  training  and  retraining  of  participants.  Our  primary  challenge  is  to  fulfill  the  gap  created  by  the  needs  for 
Intercambios  and  the  current  state  of  innovative  educational  technologies.  We  believe  WebCT  can  fulfill  this  gap. 
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Current  Development 


WebCT  has  been  chosen  as  the  innovative  technology  tool  for  this  project,  based  upon  university  reviews  of  the 
software  in  comparison  with  other  related  courseware  applications.  After  reviewing  three  top  courseware 
contenders,  Wayne  Miller,  Manager  of  the  Humanities  Computing  Facility  at  UCLA  stated,  "It  became  clear  to  us, 
though,  that  WebCT  offered  by  far  the  best  ratio  of  cost  to  features  [sic]  and  flexibility  among  the  systems  that  we 
considered."  Brian  Morgan  of  Marshall  University  adds  the  observations  that  the  strongest  features  of  the  software 
are  its  on-line  testing  and  grading  features.  These  desired  features  relate  to  pupils'  preparation  needs  for  future 
Intercambios  program  development.  Morgan  also  noted  that  Marshall  University’s  top  three  features  sought  in  such 
an  application  were;  ease  of  use,  on-line  testing  and  grading,  as  well  as  excellent  technical  support  staff. 

Although  the  program  has  regularly  scheduled  hours  and  meeting  places,  scheduling  conflicts  due  to  work  and 
school  obligations  for  all  participants  are  common.  As  a solution  to  this  conflict  in  time  and  space,  career  mentors 
establish  initial  contact  via  a discussion  list  or  private  email  to  interested  pupils.  Pupils  will  then  be  prepared  to 
interview  the  career  mentor  in  their  choice  of  conferencing  format  and  time  schedule:  email,  discussion  group,  or 
chat  sessions.  The  combination  of  WebCT's  synchronous  and  asynchronous  communication  and  a closed 
environment  for  email  will  best  serve  all  schedules  involved  for  a safe  and  optimal  career  mentoring  experience. 

The  chat  rooms,  discussion  groups,  and  private  email  can  also  be  used  by  the  university  students.  Personal  emails  to 
and  from  the  academic  mentors  can  provide  a line  of  communication  between  junior  high  school  participants  and 
academic  mentors.  A self-contained  asynchronous  and  synchronous  email  system  is  the  only  option  for  pupil 
communications  within  this  program.  WebCT  has  the  three  types  of  asynchronous  and  synchronous  communication 
desired  for  this  program  within  a safe,  self-contained  environment  for  junior  high  students. 


Future  Prospects 

Other  features  desired  in  WebCT  include  the  calendar,  participant  homepage  creation,  and  quiz.  The  calendar  will 
allow  junior  high  school  teachers  and  university  students  to  collaboratively  research  and  plan  Texas  Essential 
Knowledge  and  Skills  (TEKS)  standard-based  lessons.  University  students  can  periodically  check  the  calendar  to 
find  project  themes  and  teachers'  email  to  collaboratively  plan  any  projects  that  can  be  used  in  the  university 
student's  classes  and  in  the  junior  high  school  teacher's  curriculum.  A discussion  group  will  be  reserved  for  teachers 
and  university  students  to  exchange  ideas  and  resources  in  the  development  of  such  pages.  Participants  will  also  be 
able  to  post  original  homepage  creations.  This  may  result  from  the  discussion  group's  shared  ideas  and  resources. 
Fully  completed  homepages  will  be  posted  to  the  main  homepage  of  the  web  site  for  other  Texas  educators  to 
review. 

Finally,  the  program  directors  have  begun  collaborative  efforts  with  the  university’s  college  of  education  to  develop 
a Virtual  Professional  Development  School  (VPDS),  which  places  preservice  educators’  lesson  plans  on  the  World 
Wide  Web  for  K-12  educator  use.  Future  goals  include  further  development  with  the  VPDS  environment  and  its 
inclusion  in  the  college’s  imminent  professional  development  school  (PDS)  and  the  preservice  education  program. 
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Introduction 

Our  interest  in  the  design  of  cases  is  motivated  by  the  need  to  develop  tools  for  elementary  teachers  that  will  help 
them  understand  how  students  think  about  forms  of  mathematics  typically  unfamiliar  to  teachers,  like  geometry, 
space  or  measure.  In  designing  case-based  hypermedia  tools,  our  goals  are  to  exemplify  not  only  student  thinking, 
but  also,  core  mathematical  ideas.  We  agree  with  others  (Merseth,  1996;  Barnett  & Friedman,  1997)  that  case-based 
methods  for  teacher  education  afford  opportunities  to  develop  knowledge  of  exemplary  practices  and  to  learn  how  to 
analyze  and  reflect  about  classroom  contexts.  Building  off  of  our  earlier  work  that  focused  on  the  design  of  mostly 
example-based  hypermedia  systems  that  helped  teachers  to  learn  (Koehler  & Lehrer,  1998),  we  have  developed  a 
small  number  of  design  principles  for  selection  and  development  of  cases  for  case-based  hypermedia  systems  for 
teachers.  These  design  principles  play  a prominent  role  in  our  current  work  in  progress  in  which  we  look  closely  at 
case-based  hypermedia’s  role  in  the  development  of  teachers’  epistemology  of  measurement  instruction  in  the 
primary  grades. 


Case  Design 

Following  over  five  years  of  work  in  professional  development  conducted  within  the  field  of  mathematics  education, 
we  have  developed  cases  of  student  reasoning  about  elementary  mathematics.  Within  this  general  locus,  we  have 
developed  a number  of  selection  principles  for  choosing  among  many  hours  of  classroom  video.  These  selection 
principles  were  informed  by  a triangulation  of  our  previous  years  of  experience,  by  literature  in  a variety  of  fields 
(cognitive  psychology,  education,  educational  technology,  etc.),  and  by  two  empirical  studies  (Horvath  1998, 
Koehler  & Lehrer,  1998).  This  triangulation  has  resulted  in  a set  of  design  principles  that  at  this  point  represent 
"lessons  learned"  that  now  guide  our  development  efforts  (Tab.  1). 


Assessing  the  Design  Principles 

While  our  thinking  about  design  principles  for  cases  has  been  helpful,  we  found  that  these  principles  were  of  little 
use  without  an  overarching  design  for  the  computer  environment  in  which  we  planned  to  implement  our  professional 
development  activities.  Our  previous  work  has  suggested  a small  set  of  design  principles  that  are  important  for 
creating  hypermedia  tools  for  teachers  (see  Koehler  & Lehrer,  1998).  The  set  of  case  design  principles  we  have 
proposed  here  and  elsewhere  (Koehler,  Petrosino,  & Lehrer,  1998)  are  considered  important  next  steps  for 
incorporating  cases  into  these  hypermedia  tools. 

Our  current  efforts  have  focused  on  creating  two  versions  of  a hypermedia  system  aimed  at  helping  elementary 
teachers  think  about  measurement  instruction  in  the  primary  grades.  The  first  system  is  built  around  an  earlier  set  of 
hypermedia  design  principles  (Koehler  & Lehrer,  1998).  This  system  uses  video  footage  primarily  to  exemplify  the 
bigger  ideas.  The  second  system  is  being  constructed  in  accordance  with  both  the  hypermedia  design  principles  as 
well  as  the  case  design  principles  we  have  proposed  here.  Both  of  the  systems  share  a large  percentage  of  text  and 
case  video  footage,  but  differ  in  the  ways  in  which  the  video  (cases)  are  used.  Practically  speaking,  compared  to 
video  examples,  video  cases  provide  longer  looks  at  classrooms,  are  accompanied  by  more  interpretation,  and  have 
hypermedia  links  directly  embedded  in  the  digitized  video  to  allow  readers  to  explore  ideas  as  they  arise. 

Studies  in  progress  are  aimed  at  investigating  how  participants  using  the  case-based  system  differ  from  those  using 
the  example-based  system  on  a variety  of  measures  including:  the  ways  in  which  they  interact  with  the  system  itself. 
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the  order  in  which  information  is  visited,  how  video  is  used,  what  is  learned  about  the  domain  of  measurement,  what 
is  learned  about  student  thinking,  and  what  is  learned  about  classrooms. 


Design  Principle 

Justification 

Benefits 

Cases  Should  be 
Grounded  in 
Practice 

Teaching  is  grounded  in  practice,  so  instructional 
cases  about  teaching  should  also  be  grounded  in 
practice.  It  is  for  this  reason  that  video  cases 
should  be  created  from  footage  that  comes  from 
real  classrooms  with  full-time  professional 
teachers 

• Cases  capture  the  richness  and 
complexity  of  practice 

• Grounded  cases  are  more  realistic  and 
relevant  to  practicing  teachers 

Cases  Should  Tell 

‘‘Mathematics 

Stories” 

We  dill  111  tiiai  cases  are  stones.  As  rnath  stones, 
the  actors  should  be  teachers  and  students.  The 
plot  of  the  story  should  revolve  around  children’s 
attempts,  with  the  help  of  teachers  to 
“mathematize”  some  aspect  of  experience  and 
make  sense  of  it.  These  stories  can  be  historical, 
showing  the  development  of  children’s  thinking 
over  time. 

• ot-o  r»m f-TO rri n rr  mnA 
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memorable 

• Stories  facilitate  the  “mathematization” 

of  experience 

• Stories  can  be  used  to  show  the  history 

and  development  of  student  thinking 

Cases  Should  be 
Annotated  with 
Interpretation 

Like  stories,  cases  need  to  be  interpreted:  Video 
footage  does  not  “speak  for  itself’.  Annotation 
should  highlight  the  big  ideas  in  the  case:  the 
encapsulated  mathematics,  the  models  of  student 
thinking,  and  the  decisions  that  teachers  make. 

• Interpretation  helps  teachers  to 
understand  the  case 

• Interpretation  helps  teachers  to  see  the 
big  ideas  contained  within  the  case 

Cases  Should 
Anchor  Further 
Exploration 

Cases  should  anchor  further  exploration  into  other 
episodes  of  learning  [CTGV  1990].  Episodes 
anchored  to  the  main  story  line  of  the  case  provide 
a more  varied  developmental  landscape  than  might 
be  expected  by  simply  following  the  main  lines  of 
the  case  narrative.  Thus,  cases  should  provide 
opportunities  to  explore  both  the  current  and 
related  topics  in  depth. 

• Cases  can  provide  both  the  details  of 
learning  as  well  as  the  big  picture 

• Cases  permit  connections  to  related 
ideas,  other  teachers,  and  other 
classrooms. 

Table  1:  Design  Principles  for  Cases 
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As  the  growth  in  the  number  of  institutions  offering  web-based  courses  continues,  faculty  members  are  becoming 
increasingly  challenged  by  the  need  to  balance  sound  pedagogical  strategy  with  the  dynamics  of  the  Internet 
environment.  Some  faculty  are  embracing  this  challenge,  volunteering  to  teach  web-based  courses  and  participating 
in  every  available  opportunity  to  learn  how  best  to  incorporate  technology  into  instruction.  Others  have  been 
assigned  to  teach  web-based  courses,  with  turnaround  times  ranging  from  6-months  to  only  several  weeks.  With  the 
exception  of  those  technology  early  adopters  who  have  themselves  developed  tools  and  systems  for  web-based 
education,  most  faculty  members  are  relatively  new  to  web-based  teaching  and  have  neither  the  time  nor  the 
inclination  to  undertake  an  in-depth  assessment  of  courseware  development  tools.  The  image  of  24  x 7 learning 
certainly  has  its  appeal;  however,  for  those  faculty  members  making  the  transition  from  the  traditional  classroom  to 
the  cyber-classroom,  the  road  is  often  rockier  than  was  anticipated.  This  paper  offers  some  suggestions  on 
facilitating  that  transition. 

Getting  Started 

A review  of  the  basics  of  instructional  design  is  a logical  starting  point.  Learner  goals  and  objectives,  content  and 
instructional  method  and  strategy  remain  the  key  drivers.  These  in  turn  raise  a sea  of  issues  in  terms  of 
implementation  via  the  web:  student  support,  faculty  support,  text,  lab  and  library  resources,  office  hours,  advising, 
and  copyright.  In  addition,  faculty  members  often  have  to  juggle  multiple  roles,  particularly  in  these  days  of 
premium  IT  resources.  Those  roles  include  subject  matter  expert,  instructional  designer,  media  specialist,  classroom 
manager,  and  even  first-line  technical  support.  Nevertheless,  the  objective  is  to  apply  pedagogical  best  practices 
regardless  of  the  method  of  course  delivery  [Spear  98a]. 

Criteria  for  Evaluating  Courseware 

There  is  hope  on  the  horizon,  however.  Published  guidelines  and  standards  offer  guidance  through  the  maze  of 
commercial  and  proprietary  software  products  for  developing  web-based  courses.  In  November  1997,  the 
Midwestern  Higher  Education  Commission  (MHEC)  Interactive  Courseware  Committee  noted  four  key  criteria  for 
evaluating  courseware  [HREFl].  First,  the  courseware  should  provide  well-organized  and  dynamic  interactivity 
between  the  student  and  the  course  content,  the  student  and  the  teacher/facilitator,  and  among  students  sharing  the 
learning  experience.  The  human-computer  interface  should  also  allow  the  instructor  and  the  students  to  navigate  the 
material  easily  and  effectively.  Second,  the  courseware  should  be  consistent  with  current  knowledge  on  learning 
theories  and  precepts,  providing  strong  support  for  instruction  and  the  instructor,  and  facilitating  instructor 
involvement  in  the  learning  process.  It  should  allow  varying  "time  on  task"  in  the  learning  process  and  allow 
variable  levels  of  faculty  mediation  based  on  the  student’s  progress.  Next,  the  courseware  should  provide  highly 
motivating  learning  environments.  Asynchronous,  non-linear  presentation  branching  in  response  to  interaction  with 
the  learner  and  independent  exploration  are  essential.  Finally,  the  courseware  should  be  faculty-friendly.  Faculty 
members  should  need  only  basic  computer  skills  and  focus  largely  on  the  content,  not  the  technology. 

Creating  Course  Content  and  Materials 

Creating  course  materials  is  the  most  time  consuming,  resource-intensive  portion  of  faculty  workflow  and  there  are  a 
variety  of  shrink-wrap  products  for  authoring.  However,  nearly  all  of  the  PC-based  authoring  programs  require  a 
fairly  robust  technology  skill  set  to  get  beyond  on-line  page  turning;  few  provide  development  assistance  and  none 
integrate  or  interface  with  the  institution’s  back  office  administrative  information  systems.  A good  authoring  tool 
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should  provide  step-by-step  creation  methods,  templates,  sample  Web  pages  that  can  be  adapted  for  individual  use, 
automatic  generation  of  CGI  scripts,  and  basic  instruction  for  creating  standard  components  [Hansen  and  Frick, 
1997],  But  the  authoring  tool  should  also  be  flexible  enough  to  provide  the  functionality  and  the  benefits  of 
multimedia  tools  and  integrate  seamlessly  into  whatever  course  management  system  the  institution  has  selected. 

Faculty  members  should  take  stock  of  the  materials  they  currently  have.  While  digitizing  dog-eared  lecture  notes 
and  putting  them  out  on  the  web  is  certainly  not  the  best  approach,  we  need  not  throw  the  baby  out  with  the  bath 
water.  Ideally,  faculty  would  like  to  be  able  to  gather  their  materials  and  place  them  in  a "treasure  chest"  of 
resources  that  could  be  organized  dynamically  for  use  in  web-based  teaching  and  learning.  Microcosm,  a windows- 
based  authoring  software  program,  allows  faculty  to  do  exactly  that  [HREF2].  Created  by  a university  consortium  in 
the  United  Kingdom  and  winner  of  several  international  awards.  Microcosm  offers  a repository  from  which  a faculty 
member  can  create  the  same  kind  of  multimedia,  interactive  courseware  that  a professional  developer  prodiices.  but 
without  the  developer.  The  software  allows  faculty  to  integrate  information  and  create  courseware  from  virtually  any 
resource,  without  having  to  learn  programming.  Instead  of  having  to  translate  resources  to  suit  the  proprietary 
formats  that  most  authoring  tools  use.  Microcosm  uses  a series  of  built-in  viewers  to  access  the  resource  files.  For 
example,  there  are  viewers  for  each  of  the  MS  Office  applications,  for  GIF  and  JPEG  files,  etc.  When  an  instructor 
creates  hypertext  links,  the  links  are  stored  separately  from  the  data  in  their  own  link  database.  This  means  that  the 
resource  materials  can  remain  in  their  native  formats  and  in  their  native  locations.  Moreover,  the  courseware  can  be 
created  and  utilized  in  a non-linear  fashion.  Instead  of  page-turning,  linear  instruction,  the  course  design  is 
interactive  and  learner-centered.  Task  cards  and  wizards  in  plain  English  step  the  faculty  through  the  process,  with 
lots  of  positive  reinforcement  along  the  way.  The  structured  "treasure  chest"  can  then  be  "dropped"  into  the 
institution  internet  course  management  system  via  a Publish-&-Go  wizard.  At  the  moment.  Microcosm  is  being 
evaluated  by  a group  of  colleges  and  universities  in  the  United  States. 

Summary 

To  navigate  the  maze  of  products  and  ideas  about  tools  for  web-based  course  development,  faculty  members  should 
keep  their  options  open.  Experimentation  often  reveals  unexpected  results,  not  the  least  of  which  is  the  lessons 
learned  about  a particular  product  or  approach.  Listservs  such  as  ITFORUM  and  WWWDEV  welcome  all  those 
interested  in  sharing  ideas  or  simply  seeing  help  in  coping  with  the  dynamics  of  education  and  technology. 
Importantly,  students  are  an  excellent  resource.  Computer  Science  faculty  members  have  long  known  that  getting 
students  involved  in  the  design  of  their  own  web-based  courses  is  a win-win  situation  for  both  parties.  Finally,  to  be 
called  a technology  novice  is  not  an  insult.  The  novice’s  open  mind  and  freshness  have  traditionally  been  the  engine 
behind  the  evolution  of  approaches  discovered  by  the  innovators  and  developed  by  the  early  adopters.  The  novice 
approaches  the  chalkboard  and  the  chatroom  in  a positive  fashion. 
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LogNet  - The  idea  of  a network  to  support  logistics  education 

Some  years  ago,  the  on-going  practical  use  of  computer-based  methods  for  planning,  controlling  and  managing 
processes  in  industry  and  business  as  well  as  the  availability  of  modem  information  and  communication 
technologies  at  universities  encouraged  the  development  of  an  educational  system  tailor-made  for  the  specific 
interests  of  university-level  logistics  education.  As  result  of  a long-lasting,  ineffective  and  discontinuous 
developmental  process  a first  small  module  was  implemented  in  English  and  German  language  which  was  of 
interest  to  logistics  educators  from  more  than  20  universities  and  colleges  from  all  over  Europe.  In  the  same 
time,  students  and  educators  who  had  tested,  used  and  evaluated  this  “product”  have  been  coming  up  with  lots  of 
comments,  ideas  and  wishes  for  improvement  and  extension.  But  as  we  learned  from  experience,  further 
developments  of  any  kind  (i.  e.  translation,  modification,  adaptation,  or  extension)  would  only  become  possible 
in  a collaborative  developmental  process  of  internationally  distributed  authors  on  the  basis  of  a joint 
communication  network. 

Realizing  this  tremendous  demand  for  collaboration  in  developing  educational  multimedia,  the  idea  of  an 
educational  network  of  persons  and  institutions  engaged  in  logistics  education  and  interested  in  supporting  it  by 
use  of  multimedia  and  information  technologies  (LogNet)  arose.  Since  logistics  educators  are  often  not  very 
familiar  with  those  so-called  new  technologies  and  their  potentials  or  limitations,  the  network  must  first  of  all 
bring  together  interested  people  and  provide  them  with  structured  information  about  the  technical  and 
pedagogical  state-of-the-art  in  educational  multimedia,  its  development  and  use.  So,  first  activities  concentrate 
on  organizing  teacher  training  on  these  aspects.  In  addition  to  this  LogNet  wants  to  make  multimedia  resources 
of  different  logistics  topics,  educational  targets,  complexity,  and  forms  of  knowledge  presentation  available  to 
logistics  educators  and  to  encourage  the  discussion  about  concepts  for  their  use.  As  the  overall  objective  the 
initiative  is  focused  on  initiating,  setting  up  and  establishing  a co-operative,  distributed  development  of 
educational  multimedia  in  the  field  of  logistics  and  enabling  logistics  educators  to  introduce  those  kinds  of 
technologies  to  their  educational  processes  as  knowledgeable  consumers  or  well  acquainted  supervisors  or  even 
to  become  enthusiastic  multimedia  developers. 

For  this,  the  network  follows  two  different  strategies:  In  the  old,  traditional  way  workshops,  user  group  meetings 
and  summer  schools  will  frequently  be  organized  to  enable  face-to-face  contacts.  In  the  new,  modem  way  an 
Internet-based  infrastructure  will  support  all  information,  communication,  collaboration,  and  exchange  processes 
by  forming  a virtual  community.  Especially  the  latter  requires  to  think  in  more  detail  about  how  to  organize 
those  kinds  of  distance  training  and  co-operative  developments. 

The  LogNet  server  - Logistics  behind  the  logisticians^  educational  network 

From  a logistics  point  of  view  this  educational  network  of  logisticians  can  be  seen  as  a structure  of  nodes  and 
links  (relationships)  between  them  (both  together  forming  the  logistics  system),  that  has  to  store,  transfer,  and 
handle  objects  (in  a logistics  process).  Following  fundamental  logistics  principles  the  network  then  must  work  in 
a way  that  it  comfortably  provides  users  with  information,  knowledge,  and  services  in  a purposeful,  customized 
manner  appropriate  to  the  users’  needs  and  intentions.  As  most  reasonable  solution  for  this  an  Internet-based 
infrastmcture  can  be  seen  which  should  have  a powerful  heart  for  the  purpose  mentioned  above  - the  LogNet 


server. 


The  LogNet  server  can  be  characterized  as  a forum  for  discussion,  communication,  and  exchange  in  the  form  of 
an  open  information  system.  It  is  closely  oriented  towards  the  providing  of  maximum  support  to  logistics 
educators  and  consists  of  three  major  sections:  Information,  application,  and  collaboration.  The  first  section 
provides  the  user  with  general  information  about  multimedia  in  logistics  education  and  scenarios  for  its  use,  but 
also  information  about  multimedia  applications  available  to  support  logistics  education  as  well  as  an  overview  of 
results  from  their  evaluation.  On  the  other  hand  it  serves  as  meeting  point  of  interested  persons  or  institutions 
and  news  stand  containing  latest  publications,  conference  reports  and  announcements,  or  other  links  of  interest. 

The  application  section  deals  with  all  aspects  relevant  for  using  multimedia  material:  This  touches  not  only 
administrative  information  about  the  conditions  for  using  and/or  evaluating  available  educational  systems,  but 
also  instructions  about  how  to  use  or  how  to  evaluate  them  including  corresponding  teacher  training  information 
and  material.  Last  but  not  least,  it  contains  a multilingual  collection  of  educational  multimedia  on  different 
logistics  aspects  and  multimedia  resources  of  vaiydng  complexity  available  in  a knowledge  pool  and  ready  for 
being  used. 

This  pool  of  resources  forms  also  a very  important  component  of  an  environment  for  supporting  collaboration  in 
multimedia  authoring,  which  is  the  third  main  section  of  the  LogNet  server.  The  structure  of  this  section  is 
similar  to  the  application  section.  It  provides  also  both  administrative  information  (e.  g.  possibilities  and 
conditions  for  translating  or  expanding  existing  educational  modules  or  creating  a new  one)  as  well  as 
instructions  and  training  material  (in  this  case  about  how  to  work  and  which  basic  concepts  and  general 
guidelines  are  suggested  to  be  followed).  Authors  could  also  get  access  to  a collection  of  multimedia  resources 
available  for  being  translated  or  adapted.  These  modifiable  resources  represent  a subset  of  the  usable  resources 
accessible  from  the  application  section.  And  in  the  other  way  round,  all  newly  developed  or  modified  resources 
have  to  be  described  in  a way  that  they  can  become  part  of  the  knowledge  pool  and  are  available  for  use. 

In  this  way  the  LogNet  server  is  not  only  a stock  of  information,  but  also  a framework  for  education  and  training 
as  well  as  for  enabling  collaboration  in  distributed  authoring  processes.  That  is  why  it  has  to  provide  multiple 
functions  within  a user-friendly  designed,  context-sensitive  operating  management  infrastructure. 

Results  so  far,  steps  to  do  next  and  problems  remaining  for  the  future 

Despite  it  was  growing  step-by-step  the  educational  network  is  still  in  its  infancy:  At  the  moment  its  nature  is 
changing  from  being  a user  group  to  becoming  a supportive  developmental  (authoring)  environment.  This 
mutation  is  directly  driven  by  the  LogNet  members  themselves  and  their  wishes,  because  the  availability  of  the 
existing  multimedia  module  in  further  languages  and  its  adaptation  or  extension  in  contents  becomes  more  and 
more  important  for  its  wider  spreading.  These  kinds  of  activities  are  organized  in  two  phases  to  separate  the 
modification  of  resources  from  changing  the  application.  For  this,  the  developers  are  provided  with  the 
appropriate  (original)  resources  via  file  transfer,  which  they  translate  or  adapt  according  to  general  guidelines. 
These  new  resources  are  sent  back  to  the  original  team  of  developers,  who  finally  embed  them  into  the  existing 
module  and  make  this  new  application  available  for  further  use. 

Next  steps  we  are  going  to  do  will  aim  on  setting  up  the  LogNet  server  and  bringing  some  life  into  it  by  creating 
the  initial  stage  of  a pool  of  multimedia  resources  including  the  required  management  infrastructure.  While  the 
LogNet  server  in  general  will  also  work  as  the  network’s  shop-window  to  the  world,  the  access  to  the  pool  of 
resources  will  be  limited  to  those  LogNet  members  who  play  an  active  role  in  filling  it.  At  the  conference  we 
will  demonstrate  this  progress  and  report  about  lessons  learned  from  the  server’s  implementation  and  use.  With 
this,  special  attention  will  be  paid  on  explaining  which  criteria  will  have  been  proved  as  most  suitable  for 
describing  and  identifying  resources  as  a particular  piece  of  software  containing  a particular  aspect  of  logistics 
knowledge  and  being  usable  for  a specific  category  of  students  as  well  as  a certain  educational  target. 

But  there  are  lots  of  additional  problems  remaining  for  future  work.  Some  of  them  like  e.  g.  the  development  of  a 
comfortable  environment  enabling  real  co-operative  authoring  in  the  Internet,  the  definition  of  assisting 
guidelines  from  the  pedagogical,  software  engineering  or  design  point  of  view,  or  the  creation  of  an  evaluation 
procedure  and  technical  framework  for  guaranteeing  high  quality  of  multimedia  teaching  and  learning  resources 
are  still  a matter  of  research.  Other  questions  like  e.  g.  the  handling  of  copy  rights  should  rather  be  the  subject  of 
more  global  agreements.  And  the  financial  perspective  which  is  of  special  importance  for  future  developments 
leads  to  the  vision  of  transforming  this  mainly  idealism-driven  server  into  a self-supporting  service. 
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The  Multimedia  Arts  Education  Program  at  the  Tucson  Pima  Arts  Council  in  Tucson,  AZ  is  a five- 
semester  after  school  arts  technology  program  for  low- income  middle  school  urban  youth.  This  paper  is 
based  on  the  first  year  of  a longitudinal  study  designed  principally  to  determine  the  effects  of  the  program  on 
the  high  school  careers  and  upon  graduation  rates  of  students  who  complete  the  program.  Most  started  the 
600-hour  program  as  sixth  graders.  They  finished  their  work  in  the  five  labs  within  two  years,  including  a 
summer  semester.  Graduates  earned  skills  and  experience  in  art  technology,  and,  on  graduation,  a computer 
of  their  own.  Along  the  way  they  write,  draw,  shoot  video,  and  use  computers  to  design  graphics,  publish 
stories  and  articles,  and  create  animations. 

Our  lens  for  this  study  is  activity  theory.  Activity  theory  allows  us  to  look  at  this  group  of 
teachers  and  learners  as  agents-in-a-context,  in  a Vygotskian  cultural-historical  sense,  who  are  engaged  in 
tool-mediated,  goal  directed  action.  Agency,  context  and  genetic,  or  historical,  development  are  the  focus 
of  this  look  at  an  apparently  successful  program  for  young  people. 

Population 

Eight  to  ten  middle  school  students  are  enrolled  in  each  of  the  five  labs  each  semester.  Most  are 
Hispanic  and  bi-lingual  to  some  extent.  Many  come  from  single  parent  homes  in  the  downtown  areas  of 
Tucson.  Their  families  must  qualify  for  free  or  reduced  lunch  program  at  their  schools,  and  not  have  a 
computer  at  home.  A balance  of  male  and  female  students  is  maintained  through  admissions  processes. 
Students  are  required  to  maintain  a “c”  average  in  school  in  order  to  continue  in  MAEP, 

Students  receive  small  educational  incentives  each  semester  for  attaining  the  required  skill  levels 
($25)  and  completing  their  independent  projects  ($25).  Upon  completion  of  the  program,  each  student 
receives  a desktop  computer  of  his  or  her  own. 

Activity  theory 

Activity  theory  as  a methodology  can  provide  an  integrating  framework  for  linking  a set  of 
principals  (Nardi,  1998)  such  as  the  concept  of  computers  as  mediating  tools  which  allow  us  to  interact  with 
our  environments.  Activity  theory  (Rubtsov  & Margolis,  1996)  proposes  that  a learning  environment  can  be 
best  understood  by  taking  activity  as  a unit  of  measurement.  Activity  includes  working  with  others  inside  a 
system  of  culturally  defined  tools,  signs,  and  symbols.  Activity  theory  takes  into  account  the  goals  and 
motives  of  the  learners  in  the  context  of  the  setting  and  its  social  aspects. 

“Activity  theory  holds  that  the  integral  units  of  human  life  - humans  interacting  with  each  other 
and  the  world  - can  be  conceptualized  as  activities  (Italics  in  original)  which  serve  to  fulfill  distinctive 
motives.”  (Scribner,  in  Tobach,  1996,  p.  231) 

Activities  have  a structure  consisting  of  motives,  goals,  actions  and  operations.  Activity  theory 
holds  these  to  be  the  structural  units  of  human  behavior,  and,  accordingly,  appropriate  units  of  analysis  for 
learning  environments. 

In  the  course  of  their  activities  in  this  setting,  students  engage  in  actions,  such  as  designing  and 
creating.  The  operations,  which  compose  actions,  may  be  both  mental  [learning  new  software]  and 
behavioral  [drawing].  Actions  are  carried  out  for  particular  purposes  or  motives.  Students  may  wish  to  learn 
more  about  art  technology,  learn  new  computer  skills.  They  may  wish  to  please  their  parents,  earn  money, 
or  get  their  own  computer.  Activities  take  place  under  particular  conditions:  in  the  multimedia  lab,  next  to 
an  art  gallery,  after  school,  and  so  on,  and  with  particular  technical  means.  In  our  case  computers,  cameras, 
application  programs  that  support  creativity  in  language  arts,  graphics,  animation,  etc.  Activities  are  goal- 
directed.  One  salient  for  the  students  is  the  personal  computer  that  will  be  theirs  to  take  home  when 
they  graduate.  Another  is  high  school  graduation.  People  engage  in  many  activities  concurrently.  Activity 
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theory  accounts  for  learning  in  that  as  a task  becomes  automatized  it  forms  part  of  an  operation  that  upon 
mastery  becomes  an  action  that  is  part  of  an  activity,  which  leads  to  goals. 


Data  collection 

Our  purpose  was  to  gather  information  about  the  program  in  the  contexts  of  the  MAEP  lab,  the 
home,  and  the  school.  University  students  worked  as  teaching  assistants  in  the  labs  doing  participant 
observation  focusing  on  activity  patterns  and  the  development  of  curriculum.  Data  is  being  analyzed  for 
evidence  of  aesthetic  response  (Vygotsky,  1971)  and  correlation  with  student  outcomes  in  terms  of  perceived 
self-efficacy  and  attitudes  about  art  and  technology. 

Exit  interviews  were  conducted  with  the  graduating  class  during  their  final  portfolio  activity. 
Parents  of  graduates  were  interviewed  to  aseci  taiii  their  awareness  of  the  program,  level  of  satisfaction  and 
influences  of  the  students’  new  skills  on  the  family.  Graduates  and  their  parents  will  be  interviewed  again 
about  eighteen  months  after  graduation.  English  teachers  at  the  high  school  were  contacted. 

Each  semester  students  took  pre-  and  post  skills  tests  developed  by  their  instructors.  They 
completed  questionnaires  about  their  perceived  self-efficacy  (Bandura,  1993)  and  attitude.  Students  report 
on  their  favorite  arts  activities  and  books  they’ve  read,  etc.  Questions  related  to  attitudes  about 
collaboration,  perceived  self-efficacy  in  regard  to  creativity  and  technology,  their  expectations  for  the 
program,  and  their  career  plans.  Each  semester  they  worked  with  their  teachers  to  fill  out  self-evaluations. 
Students  and  teachers  kept  reflective  journals.  In  addition,  students  are  evaluated  by  their  teachers  and 
evaluate  their  learning  experience  each  semester.  These  young  people  produced  high  quality  work  in  print 
and  digital  form,  including  graphic  designs  such  as  logos  and  story  animations,  poetry,  newsletters  and 
video  programs 


Summary 

Parents  reported  that  they  were  highly  satisfied  with  the  program  and  that  the  experience  had 
helped  their  children  mature.  Students  reported  that  they  participated  in  the  MAEP  activities  motivated  by  a 
new  computer,  a wish  to  learn  new  art  skills,  and  to  please  their  parents.  Students  learned  to  use  the 
software  programs  as  tools  to  design  and  produce.  Many  responded  to  the  arts-based  setting  and  enjoyed 
sketching  a drawing  by  hand.  The  conditions  of  the  class  presented  a unique  opportunity  to  interact  with 
peers.  Students  reported  that  they  saw  a benefit  from  the  Language  Arts  lab  in  their  English  classes.  The 
computer  skills  acquired  were  less  often  recognized  by  the  students  as  advantageous  in  school,  however,  they 
were  looking  forward  to  taking  more  courses  that  utilized  technology  in  high  school  and  were  more  likely  to 
select  those  courses  for  themselves. 

Of  course,  this  after  school  program  takes  place  amid  a great  many  other  activities.  Students  and 
faculty  each  brought  a history  of  educational  experience  and  technological  expertise  with  them  and  they 
each  contributed  to  the  culture  of  the  MAEP  and  of  each  lab.  The  history  and  development  of  the  students, 
the  evolution  of  the  program's  rules  and  policies,  the  volatility  of  the  technology  all  are  a part  of  this 
equation  and  all  are  dynamic.  This  study  hopes  to  shed  some  light  on  these  phenomena.  We  have 
established  the  procedure  for  the  longitudinal  study.  The  exit  interviews,  the  parent  interviews,  the  role  of 
the  teaching  assistants,  the  assembly  of  data  from  the  various  questionnaires  are  in  place,  and  we  have  begun 
a baseline  for  further  study. 
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Introduction 

Today,  educators  have  the  technology  to  create  an  exciting  student-centered  learning  environment  that:  a) 
accommodates  different  learning  styles;  b)  accommodates  different  rates  of  learning;  c)  provides  learning  in  context; 
and  d)  broadens  the  choice  of  learning  and  teaching  partners.  Yet  most  of  the  computer-based  and  computer- assisted 
courses  developed  to  date  are  little  more  than  electronic  textbooks  or  lectures.  Many  educators  are  only  using 
multimedia  and  information  technology  to  do  the  same  old  thing  - modify  print  or  lecture-based  courses,  which  are 
linear  in  their  design  - to  a computer-based  equivalent  enhanced  only  with  hyperlinks.  This  is  not  good  education, 
and  it  does  not  address  the  learning  style  of  many  of  today’s  youth. 

Through  computer  games  and  programs  like  ICQ,  today’s  student  is  accustomed  to  the  interactivity  that  the 
computer  and  computer  networks  provide.  They  are  accustomed  to  being  able  to  complete  a variety  of  tasks  on  a 
computer  without  referring  to  an  instruction  manual.  They  leap  into  games  and  scenarios  going  to  the  ‘help’  screens 
only  when  hopelessly  lost  and  when  the  program  is  interesting  enough  to  make  it  worthwhile.  Mature  distance 
learners  are  generally  very  busy  with  other  responsibilities  and  commitments.  They  commonly  have  some  level  of 
practical  experience  in  the  discipline  they  are  studying,  and  become  frustrated  by  vast  quantities  of  seemingly 
irrelevant  on-line  reading. 

Philosophy/Concept 

We  believe  that  the  real  power  of  the  computing  environment  in  teaching  is  the  ability  to  create  real-life  scenarios 
that  engage  the  learner.  Through  the  process  of  solving  problems  posed  in  a scenario,  the  mentor  ensures  that 
students  have,  or  will  acquire  a basic  set  of  skills  or  knowledge.  The  use  of  virtually  unlimited  navigation  paths, 
along  with  multimedia  presentation,  and  controlled  access  to  information,  allows  the  course  developer  to  create  a 
stimulating  and  positive  learning  experience  for  all  students.  This  environment  allows  for  learning  to  take  place 
within  a real-life  context,  and  accommodates  the  different  ways  and  rates  students  leam. 

Project  EdEx 

With  this  in  mind,  Acadia  University’s  Division  of  Continuing  and  Distance  Education  has  embarked  on  project 
EdEx.  The  goal  is  to  create  a single  prototype  course  which  harnesses  the  power  of  the  computing  environment,  and 
focuses  on  student-centered  learning.  Through  the  development  of  prototype  courses  we  can  develop  a template  that 
encourages  instructors  to  break  the  old  habits  of  "sage  on  the  stage",  and  become  the  "guide  on  the  side". 

The  development  of  these  courses  goes  through  stages.  First,  working  with  instructors,  we  establish  the  objectives 
for  the  entire  course,  and  for  the  individual  course  modules.  The  second,  and  most  important  phase  of  the  project,  is 
identifying  a series  of  problems  or  scenarios  that:  a)  are  relevant  and  current,  b)  are  interesting  and  captivating,  c) 


require  students  to  apply  a specific  knowledge  set,  and  d)  require  students  to  interact  with  learning  and  teaching 
partners  anywhere  on  the  planet.  Finally,  we  use  today’s  technology  to  present  the  scenarios. 

As  a prototype  we  are  using  a combination  of  HTML  and  Authorware  to  develop  an  environmental  geology  course. 
The  course  adopts  a modular  approach  with  each  module  incorporating  a major  topic  area,  e.g.  geological  hazards, 
earth  resources,  etc.  The  course  is  designed  so  students  begin  with  a particular  problem  or  scenario  - not  a lecture, 
and  then  access  information  as  required  to  complete  the  tasks. 

In  each  module,  students  face  simulations  of  ‘real-life’  problems  encountered  by  environmental  geologists.  For 
example,  the  student  may  be  faced  with  the  problem  of  building  a shopping  mall  in  southern  Quebec.  In  order  to 
make  an  informed  decision  about  whether  the  mall  should  be  built  and  the  best  location,  students  need  to  have 
knowledge  of  several  key  geological  concepts,  e.g.  rock  classification,  geological  environments,  mass  wasting,  soil 
processes,  etc. 

As  the  level  of  prior  knowledge  for  each  student  varies,  the  course  is  designed  with  all  of  the  learning  options  up 
front  so  students  may  choose  the  one(s)  they  need  to  pursue.  The  problems  are  designed  so  that  students  must  apply 
correct  knowledge  and  concepts,  but  may  enter  the  knowledge  windows  and  help  areas  of  their  choice.  Those  who 
know  how  to  identify  rocks,  for  example,  are  not  required  to  wade  through  the  information  presented  on  rock 
identification  before  continuing  the  scenario.  To  avoid  the  misconceptions  which  are  often  associated  with  prior 
knowledge,  students  will  be  required  to  periodically  demonstrate  a certain  level  of  understanding  before  advancing 
to  the  next  stage  of  the  problem. 

The  scenarios  and  knowledge  banks  all  take  advantage  of  multimedia  to  accommodate  different  ways  students  learn. 
They  can  ’’see  it",  "hear  it",  and  virtually  "try  it". 

In  order  to  develop  cross-platform  courseware  we  chose  to  use  HTML  as  the  front  end  of  the  course.  Students  can 
run  the  entire  course  through  their  browser  with  the  QuickTime  3.0  and  Authorware  Shockwave  plug-ins.  Taking 
this  approach  allows  us  to  deliver  the  course  via  the  Internet  and  on  CD-ROM,  the  latter  being  the  more  desirable 
approach.  Menu  options  and  navigation  will  be  conducted  through  the  HTML  interface,  with  the  interactive 
component  of  each  module  being  controlled  through  a combination  of  HTML  and  Macromedia  Authorware. 
Courseware  developers  without  these  tools  should  not  be  deterred  from  this  model.  Although  the  tools  are  an 
important  factor  in  constructing  the  course,  the  key  is  the  idea  that  students  need  to  have  ownership  and  control  over 
their  learning.  The  ‘repaired’  web  allows  courseware  developers  to  give  this  control  of  learning  back  to  the  student. 

We  are  very  excited  by  this  approach.  By  working  through  several  prototypes  our  end  goals  are  to  develop  a 
template  that  University  instructors  can  adapt  to  any  course,  and  to  support  a more  dynamic  learning  environment 
both  on  and  off  campus. 
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The  Eastern  Virginia  Telemedicine  Network  (EVTN)  is  a health  professions'  distance  learning,  teleconsultation, 
and  clinical  telemedicine  network  established  by  the  Eastern  Virginia  Medical  School  (EVMS)  and  the  Eastern 
Shore  Rural  Health  System,  Inc.,  with  major  funding  support  provided  by  the  Virginia  Health  Care  Foundation, 
EVMS,  and  the  Eastern  Virginia  Area  Health  Education  Center  (EVAHEC). 

The  network  consists  of  two  distinct  telecommunications  links  between  the  Norfolk  campus  of  EVMS  and  the 
greater  eastern  Virginia  community.  The  first  network  utilizes  a one-way  video  signal  transmitted  via 
microwave  from  EVMS  to  the  control  room  of  WHRO,  the  eastern  Virginia  public  television  station.  WHRO 
then  rebroadcasts  this  video  signal  over  its  wireless  cable  network  throughout  eastern  Virginia.  Participating 
institutions  receiving  this  signal  include  hospitals,  health  departments,  community  health  centers,  community 
colleges,  multi-physician  private  practice  groups,  and  others.  Viewers  who  would  like  to  ask  a question  of  the 
presenters,  during  a Grand  Rounds  session  for  example,  direct-dial  in  to  the  EVMS  studio  over  a toll-free 
telephone  line.  The  second  network  consists  of  a two-way  video,  two-way  audio,  videoconferencing  network 
linking  many  of  the  same  institutional  partners  over  digital  telephone  lines  that  allows  for  ad  hoc 
communications  to  encourage  teleconsultation  among  providers,  to  support  health  professions'  student  training 
in  medically  underserved  areas,  and  to  facilitate  clinical  telemedicine  activities  in  the  coming  years. 

The  Eastern  Virginia  Telemedicine  Network  was  established  for  several  reasons:  First,  to  improve  the 

recruitment  and  retention  of  primary  care  health  providers  in  medically  underserved  areas  of  eastern  Virginia  by 
reducing  their  sense  of  professional  isolation.  Second,  to  support  the  training  of  new  primary  care  health 
providers  in  these  areas  by  increasing  the  opportunity  for  health  professions'  students  to  train  in  those  remote 
areas  while  being  linked  to  their  educational  programs  for  didactic  sessions.  Third,  to  improve  the  quality  of 
care  provided  in  these  areas  by  providing  the  most  current  medical  information  to  providers  and  by  increasing 
the  amount  of  professional  consultation  among  community  and  academic  health  center  professionals.  Fourth, 
the  EVTN  will  ultimately  reduce  the  cost  of  health  care  in  remote  areas  by  eliminating  unnecessary  patient 
referrals  to  distant  treatment  centers. 

The  Eastern  Virginia  Telemedicine  Network  provides  a variety  of  programming  to  diverse  audiences.  Grand 
Rounds  sessions  at  the  academic  health  center  are  made  available  to  a wider  audience  of  health  professionals  in 
eastern  Virginia,  providing  them  with  the  opportunity  to  receive  continuing  professional  education  credits 
through  program  viewing.  Health  education  programming  will  also  be  made  available  over  the  network  to 
consumers  of  health  care  services  in  eastern  Virginia.  Quality  health  care  programming  produced  by  other 
entities  throughout  the  United  States,  and  indeed,  the  world,  have  been  downlinked  utilizing  EVTN  satellite 
facilities  at  EVMS  and  rebroadcast  to  eastern  Virginia  health  providers  and  consumer  viewers  over  the  EVTN. 

During  the  two-year  development  of  the  EVTN,  a variety  of  surprises  happened.  These  surprises  involved 
factors  such  as  the  challenges  posed  by  using  existing  space  originally  designed  for  other  purposes;  by  working 
with  limited  (but  adequate)  funding;  by  technical  integration;  by  long-standing  presentation  habits  of  faculty;  by 
accreditation  constraints;  and  by  divergent  concepts  of  need.  Yet,  the  EVTN  works.  This  presentation 
describes  the  development  process  and  the  lessons  learned  so  that  future  development  efforts  might  suffer  fewer 
surprises  - both  technical  and  human. 


1459 


EASTERN  VIRGINIA  TELEMEDICINE  NETWORK  "LESSONS  LEARNED" 


1 . Unless  you  are  a techno-whiz,  utilize  a technology  consultant  to  help  you. 

2.  Utilize  a consultant  who  is  unbiased  concerning  telecommunications  medium  and  hardware  selection. 

3.  Attend  national  conferences  on  telemedicine  and  distance  learning — also  good  hunting  grounds  to  find  a 
consultant. 

4.  Attend  a minimum  of  20  sessions  of  whatever  type  of  program — classes,  Grand  Rounds,  etc.,  you  want  to 
broadcast  before  talking  to  a consultant. 

5.  Write  down  for  the  consultant  the  functionalities  you  want  your  project  to  accomplish. 

6.  Make  sure  that  the  audio  portion  of  the  project  is  not  forgotten  due  to  the  usual  focus  on  the  video  portion 
of  the  project. 

7.  Schedule  construction  work  during  organizational  “down-times”,  such  as  summers  and  winter  breaks  for 
educational  organizations,  to  minimize  disruption. 

8.  Prioritize  project  tasks  so  that  the  most  time-consuming  tasks,  i.e.,  communications  licensing  issues, 
accreditation  issues,  are  handled  first. 

9.  Don’t  underestimate  the  impact  that  small  changes  in  audio-visual  equipment  will  have  on  presenters  using 
the  new  equipment. 

1 0.  Don’t  underestimate  the  impact  that  n on-technical  human  factors  can  have  on  the  success  or  failure  of  your 


project. 
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Background 

The  US  WEST  Montana  Teacher  Network  (hereafter  referred  to  as  the  MTN)  is  a project  funded  by 
US  WEST  as  part  of  its  Teacher  Networks  initiative.  Beginning  in  1995  and  ending  in  2000,  the  US  WEST 
Foundation  has  provided  funding  on  a competitive  basis  to  help  one  percent  of  the  teaching  force  in  each  of  its 
fourteen  state  region  prepare  to  use  telecommunications  and  to  train  an  additional  ten  percent  of  the  state 
teachers  through  peer  coaching  and  training.  States  were  brought  into  the  projects  on  a phased  basis.  The  lead 
states  which  served  as  pilots  and  received  the  first  wave  of  funding  in  1995  were  Colorado,  Idaho,  and  Utah. 
Montana  is  a part  of  the  second  group  of  states  to  be  funded.  The  third  group  of  states  received  its  initial 
funding  in  1998.  A goal  of  all  of  the  projects  was  to  prepare  eleven  percent  of  each  state's  teachers  to  use 
telecommunications  in  schools. 

State  affiliates  of  the  National  Education  Association  worked  with  state  US  WEST  teams  to  plan  and 
implement  the  networks.  Each  state  was  given  funds  to  provide  one  percent  of  the  teaching  force  in  each  state 
with  a notebook  computer,  a graduate  course  in  using  telecommunications,  and  online  support  and  networking 
to  facilitate  the  peer  coaching  of  the  additional  ten  percent  of  the  teaching  force.  Some  funding  was  also 
provided  for  administration,  project  evaluation,  and  participant  expenses.  The  grantees  and  fiscal  agents  in  the 
networks  were  to  be  schools  or  colleges  of  education  although  a few  states  were  able  to  run  the  networks 
through  a state  office  of  public  instruction  or  equivalent  by  partnering  with  schools  or  colleges  of  education.  It 
was  hoped  that  the  states  would  be  able  to  supplement  project  funding  through  other  sources  but  this  task 
usually  proved  difficult  as  many  of  the  states  are  economically  depressed  and  face  tremendous  geographical  and 
telecommunications  obstacles.  However,  the  project  teams  and  teachers  have  been  very  creative  in  delivering 
training  and  the  projects  have  been  successful. 

Teachers  have  a great  deal  of  subject  knowledge  and  classroom  expertise.  However,  they  do  not  often 
have  an  opportunity  to  share  what  they  know  with  colleagues  in  a networking  manner  when  geographical 
challenges  make  physical  meetings  so  difficult.  Educators  can  feel  very  isolated— especially  in  smaller  schools 
where  h/she  may  be  the  only  technology  "guru."  The  beauty  and  power  of  the  Montana  Teacher  Network  is  that 
in  a low-cost  manner,  it  is  helping  educators  access  a technology  which  will  allow  a far  greater  sharing  of 
professional  knowledge  and  allow  them  to  also  expand  their  personal  professional  development.  Through  its 
model  of  "peer  coaching"  in  small  groups  or  one  on  one,  over  1 300  Montana  educators  have  learned  to  apply 
telecommunications  technologies  in  their  classrooms.  Students  and  communities  are  the  beneficiaries  of  such  a 
model. 


The  Montana  Teacher  Network 

Montana  is  a large,  sparsely  populated,  economically  depressed  rural  state  with  a total 
population  of  @ 800,000.  This  population  translates  into  less  than  four  people  per  square  mile. 
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Geographic  centers  and  populous  areas  are  separated  by  vast  land  mass  which  is  often  mountainous. 
As  a comparison,  the  sister  city  in  Kumamoto,  Japan,  has  over  one  million  people.  Travel  in  Montana 
is  often  difficult  due  to  poor  weather  conditions  during  much  of  the  year  and  air  access  is  limited 
beyond  a few  cities.  To  provide  a perspective  of  the  size  of  Montana  which  is  the  fourth  largest  state, 
readers  are  informed  that  the  entire  New  England  states  in  the  United  States  could  fit  into  Montana 
physically  and  Montana  is  two  thirds  the  size  of  France.  Montana  Office  of  Public  Instruction  data 
have  indicated  that  the  public  school  population,  K-12,  is  @164,560.  More  than  two-thirds  of  the 
public  schools  enroll  fewer  than  two  hundred  students.  In  the  1996-97  school  year,  students  were 
enrolled  in  schools  ranging  in  size  from  1 student  to  2,400  students.  In  Montana,  there  are  56  counties 
and  900  public  schools  in  460  districts.  School  districts  are  located  in  an  area  the  size  of  New  Jersey, 
New  York,  Pennsylvania,  and  Ohio  combined.  These  comparisons  provide  some  perspective  as  to  the 
challenges  facing  teachers  to  remain  technologically  current  as  well  as  the  difficulties  in  providing 
training  opportunities  for  teachers  with  or  without  credit  attached.  Thus,  the  opportunities  and  peer 
coaching  approach  provided  through  the  Montana  Teacher  Network  have  filled  a needed  gap  for 
professional  development  and  training  in  Montana. 

Montana  State  University-Bozeman  is  the  fiscal  and  intellectual  property  agent  for  the 
Montana  Teacher  Network.  During  the  first  year  of  the  project  which  began  in  late  May,  1997,  132 
teachers  were  selected  through  an  application  process  to  be  the  first  wave  of  coaches.  Each  received  a 
notebook  computer  for  their  use  as  long  as  they  are  public  school  teachers  in  Montana.  The  teachers 
were  divided  into  five  workshop  groups  to  attend  a three  day  summer  workshop  which  provided  an 
overview  of  telecommunications  and  hands-on  Web  basics.  The  workshop  was  the  first  part  of  a three 
credit  course.  Teachers  could  choose  to  take  the  course  for  graduate  credit,  renewal  credits  or  not-for- 
credit.  However,  as  part  of  the  grant  award  participation  , the  teachers  signed  contracts  to  complete  the 
course  and  grant  requirements  of  "each  one  coach  10."  by  end  of  1999.  During  summer,  1997,  four 
workshops  were  held  at  the  Montana  State  University-Bozeman,  Bums  Telecommunication  Center  in 
Bozeman.  One  workshop  was  held  at  the  University  of  Montana  campus. 

The  online  course  component  began  in  September,  1997  and  finished  in  May,  1998.  It  was 
delivered  over  MSULINK,  a First  Class  Intranet  Client  computer  conferencing  software  housed  on  a 
server  at  the  Bums  Telecommunication  Center. 

During  the  second  year  of  the  MTN  project,  the  teacher  coaches  were  required  to  execute 
their  training  plan  for  coaching  at  least  ten  peers  in  using  telecommunications  in  education.  A web  site 
and  MTN  conference  was  established  on  the  state  telecommunication  network,  the  METNET  which 
uses  First  Class  Intranet  Client  computer  conferencing  software  with  which  the  teachers  were  already 
familiar.  The  coaches  and  their  ten  colleagues  had  access  to  a wide  variety  of  resources,  information, 
and  online  conversation  about  incorporating  online  resources  into  their  respective  curricula  and 
communities.  The  URL  for  the  Montana  Teacher  Network  is: 

http://www.metnet.state.mt.us/~MTN/).  The  teachers  also  have  access  to  all  the  information  and 
resources  available  at  the  METNet  website  which  can  be  accessed  at  the  following  URL: 
http://www.metnet.state.mt.us/MAIN.html.  Since  the  First  Class  Intranet  Client  software  is  easy  to 
learn  and  use,  this  vehicle  fulfilled  US  WEST  grant  requirements  of  building  on  existing  infrastructure 
and  networks  within  a state  since  the  METNet  is  housed  on  a server  at  the  Montana  Office  of  Public 
instmetion.  In  addition  to  serving  education  the  METNet  is  used  as  the  state  server  for  government 
and  other  entities  such  as  museums,  projects,  state  information,  and  distance  education  opportunities 
offered  by  several  higher  education  units  in  Montana. 

To  date,  the  Montana  Teacher  Network  has  been  a success.  Plans  are  underway  to  find 
additional  financial  support  in  order  to  expand  the  project  to  more  educators  and  local  communities. 
Teacher  participants  are  being  encouraged  to  present  sessions  at  state  and  local  educational 
conferences  and  meetings  and  to  continue  their  peer  coaching  efforts  and  remain  actively  involved 
with  the  MTN  website. 


14G2 


Technology  Transfer  and  Web  Course  Augmentation  Based  on  Learner  Attributes 

Dermis  Moore,  Ed.D.  and  Elizabeth  M.  Corbitt,  Ph.D. 

Wright  State  University 

Eileen  Wolkstein,  Ph.D.  and  Rebecca  Bausch,  M.P.H. 

New  York  University 


Introduction 

As  education  and  training  increasingly  move  into  the  realm  of  distance  learning,  research  is 
needed  to  determine  the  impact  of  computer-based  instruction  on  individual  learning  needs  and 
styles.  Whereas  the  ultimate  goal  for  some  settings  is  total  technology  transfer  and  electronic 
delivery  of  course  materials,  experiential  and  non-computerized  interactive  components  have 
been  shown  to  be  important  for  many  persons.  This  concept  may  be  particularly  applicable  in 
fields  demanding  that  active  professionals  upgrade  their  education  despite  the  lack  of  appropriate 
degree  granting  programs  immediately  available.  Thus,  despite  these  professionals’  potential 
difficulty  in  utilizing  computer-based  instruction,  they  may  be  constrained  by  circumstances  to 
attempt  this  kind  of  course.  Research  on  the  effectiveness  of  different  web  course  features  for 
persons  with  different  learning  needs  is  therefore  timely  and  of  great  potential  utility.  We  are 
reporting  the  results  of  preliminary  data  based  on  a web  course  utilized  as  one  component  of  a 
classroom  course,  hypothesizing  that  variables  associated  with  learner  characteristics  would 
differentially  influence  learners’  satisfaction  with  and  successful  completion  of  the  web  course. 

Description  of  Web  Course.  The  web  course  on  which  this  research  is  based,  entitled  Drug 
Abuse,  Disability,  and  Vocational  Rehabilitation,  has  evolved  from  the  work  of  the 
Rehabilitation  Research  and  Training  Center  (RRTC)  on  Drugs  and  Disability,  funded  by  the 
National  Institute  on  Disability  and  Rehabilitation  Research  (NIDRR).  Our  epidemiology 
research  has  indicated  a high  prevalence  of  unidentified  substance  abuse  among  customers  of 
Vocational  Rehabilitation  (VR)  services,  leading  to  higher  failure  rates  and  expenditures  of  funds 
without  successful  outcomes.  Through  meetings  and  training  sessions  with  VR  agencies  across 
the  country,  a pressing  need  has  emerged  to  inform  counselors  about  the  high  rates  of  substance 
abuse,  and  to  provide  training  in  assessment,  treatment  planning,  and  VR  counseling  with 
customers  with  substance  abuse  issues.  The  decrease  in  training  resources  combined  with 
advances  in  technology  and  the  need  to  get  information  to  more  remote  VR  offices  has  led  to  the 
creation  of  this  course. 

Method 

Participants  in  this  pilot  study  were  37  individuals  completing  the  web  course  in  two  separate 
classes  (New  York  University,  N = 20;  Virginia  Commonwealth  University,  N = 17)  who  filled 
out  questionnaires  before  and  after  participation.  Learners  were  graduate  students  in  university 
rehabilitation  departments,  but  had  a wide  range  of  backgrounds,  including  a number  of 
professionals  in  the  fields  of  substance  abuse  and/or  vocational  rehabilitation.  When  the  course 
structure  is  finalized,  learner  characteristics,  including  age,  gender,  educational  level,  years  post- 
matriculation, levels  of  computer  expertise,  and  learning  styles,  will  be  measured  on  initial 
registration  for  the  course.  Outcome  variables  will  include  course  grade,  percentage  of  course 
completed,  learner  satisfaction  with  the  course  and  with  different  aspects  of  the  course,  learner 
ratings  of  their  own  success  and  learning,  and  learner  ratings  of  the  utility  of  the  course  and  of 


different  course  aspects.  Additional  course  variables  will  also  be  measured,  including  time  and 
frequency  using  various  web  course  features,  and  level  of  participation  in  "real  time"  activities 
for  course  augmentation.  For  the  purposes  of  this  pilot  study,  outcome  variables  included  course 
grade,  and  knowledge-based  questions  regarding  course  content,  which  were  administered  before 
and  after  course  participation.  A sub-sample  of  participants  answered  questions  regarding  their 
age  and  gender,  as  well  as  their  satisfaction  with  the  course,  ratings  of  their  own  success,  and 
ratings  of  the  utility  of  face-to-face  interactions  in  contributing  to  their  course  experience. 

Results 

For  the  NYU  sample,  age  was  negatively  correlated  with  enjoyment  of  the  computer  at  Time  1 (r 
= -.75,  p=.021),  and  showed  strong  negative  relationships  to  enjoyment  of  the  internet  (r  = - 
.544,  g = .13),  comfort  with  using  a computer  (r  = -.597,  p = .09),  and  comfort  with  using  the 
internet  (r  = -.47,  p = .20).  These  correlations  were  no  longer  present  at  Time  2.  For  all 
participants  at  NYU,  regardless  of  age,  comfort  with  the  internet  increased  significantly  (mean 
Tl=6.89,  mean  T2=8.32;  t = 2.19,  p = .044).  Course  grade  was  not  related  to  comfort  with 
computers  or  the  internet.  For  the  group  analyses  performed  for  the  VCU  sample,  ratings  of 
enjoyment  of  and  comfort  with  computers  and  the  internet  increased  between  T1  and  T2,  with 
group  means  increasing  from  around  5 to  around  7 on  a 10-point  scale  for  each  of  the  four  items 
(see  Figure  1).  An  additional  set  of  items  related  to  knowledge  of  course  material.  For  the  NYU 
sample,  the  mean  score  of  participants  increased  significantly  from  T1  to  T2  (mean  Tl=6.6, 
mean  T2=7.9;  t = 3.15,  p = .005).  At  VCU,  the  mean  knowledge  score  increased  from  5.3  at 
pretest  to  6.6  at  posttest.  Knowledge  scores  were  not  related  to  comfort  with  computers  or  the 
internet  at  Time  1 or  Time  2.  Information  regarding  learner  opinions  regarding  the  utility  of  face- 
to-face  meetings  was  collected  for  a small  subsample  of  participants  (N=9)  at  NYU  on  post-test. 
On  a ten-point  scale  (where  l=extremely  unimportant  and  10=extremely  important),  these 
participants  were  highly  variable  in  their  responses,  ranging  from  1 to  10  (mode  = 9). 

Discussion 

There  currently  exists  a paucity  of  information  regarding  how  specific  elements  of  electronic 
learning  differentially  affect  learner  outcomes,  especially  as  mediated  by  demographic  and  other 
learner  attributes.  The  research  reported  in  this  paper  relates  to  a web  course  that  is  likely  to  be 
offered  to  a number  of  simultaneous  groups  of  traditional  and  non-traditional  students  in  the 
future,  and  results  from  this  study  will  be  utilized  in  tailoring  future  revisions  of  the  web  course 
to  groups  of  learners  based  on  the  modalities  and  approaches  proven  most  efficacious  for  learner 
characteristics.  These  preliminary  results  suggest  that  older  students  may  feel  initially  less 
comfortable  with  web-based  learning,  but  become  more  comfortable  with  exposure.  On  the  other 
hand,  comfort  with  the  computer  or  the  internet  did  not  appear  to  relate  to  course  grade  or  to 
knowledge  gained  from  the  course.  It  is  interesting  to  note  as  well  that  the  two  samples  of 
students  in  this  pilot  investigation  had  different  initial  comfort/enjoyment  levels  with  computers, 
suggesting  that  different  classes  will  be  quite  variable.  Although  information  regarding  learner 
opinions  regarding  the  utility  of  face-to-face  meetings  was  collected  for  only  a small  subsample 
of  participants,  the  high  variability  of  responses  was  somewhat  surprising,  with  answers  ranging 
across  both  ends  of  the  scale.  While  preliminary,  these  results  suggest  that  learners  do  indeed 
have  widely  varying  levels  of  comfort  with  computer-based  learning,  and  may  thus  react  quite 
differently  to  different  course  aspects. 
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Introduction 

At  the  department  of  business  education  and  human  resource  management  (University  of  Innsbruck;  Austria)  we 
offer  all  relevant  information  services  for  the  students  via  the  internet  with  a 3-D  interface.  Students  (but  also 
other  visitors)  can  choose  an  avatar  and  move  around  in  the  virtual  reality  environment.  They  cannot  only 
explore  all  the  relevant  information  for  their  study  in  an  interactive  way,  e.g.  knocking  at  the  door  of  a professor 
and  asking  for  the  outline  of  her  lecture  but  they  can  also  start  the  necessary  action  for  the  university 
administration,  e.g.  booking  the  lecture,  download  the  literature  etc. 

This  is  a pilot  project  not  on  the  technical  but  on  the  organizational  level.  Linking  all  the  relevant  information 
together  is  primarily  an  organizational  task  which  will  change  the  university  structure.  The  project  is  embedded 
in  a three  year  development  project  funded  by  the  Austrian  government  which  will  restructure  the  whole 
university. 

Background  information 

The  University  of  Innsbruck,  situated  in  the  center  of  Innsbruck  (130.000  inhabitants),  has  about  30.000 
students.  The  faculty  of  Social  and  Economic  Sciences  has  5.000  students  and  consists  of  five  major  fields  of 
studies:  Business  Administration,  Economics,  Business  Education,  International  Economics  and  Business 
Studies  and  a doctoral  program.  The  idea  to  build  a virtual  faculty  is  linked  to  three  major  university 
developments: 

Firstly  more  autonomy  for  Austrian  Universities.  With  a new  law  (University  Organization  Act  1993)  all 
Austrian  universities  will  get  more  autonomy  in  a transition  period  of  about  10  year.  Formerly  all  major 
decisions  like  budget,  areas  of  specialization,  appointment  procedures  were  directed  by  the  ministry  of  science  in 
Vienna.  Now  we  are  building  different  university  organs  that  will  overtake  these  responsibilities.  This  process  is 
very  complex  and  has  different  phases;  one  of  them  is  a kind  of  dual  control  of  the  old  and  new  bodies.  The  new 
self  governing  bodies  are  preparing  their  statutes  but  do  not  yet  have  the  decision  power  for  everyday  business. 
The  transfer  of  power  from  the  old  to  the  new  bodies  is  a crucial  process.  Currently  the  University  of  Innsbruck 
is  undergoing  this  difficult  transition  period. 

Secondly,  the  move  towards  greater  autonomy  is  accompanied  by  a strategic  initiative  of  the  senate  (the  self- 
governing  body  at  the  executive  level  at  the  university).  In  a three-year  project,  funded  by  the  ministry  of 
science,  the  members  of  the  university  are  discussing  organizational  changes  and  future  alternatives  with  regard 
to  the  content  of  services  and  performances.  The  main  idea  of  this  project  is  to  motivate  as  many  members  of  the 
university  as  possible  to  participate  and  cooperate  in  the  transition  process.  Within  the  project  ten  different 
subgroups  work  on  issues  such  as  “Teaching  & Research”,  “Personnel  Management”,  “Central  Information 
Services”,  “Financing  & Controlling  etc.  One  subgroup  “New  Media”  investigates  the  future  possibilities  and 
organizational  consequences  of  New  Media  for  teaching  and  will  provide  the  ruling  body  of  the  university  with 
all  the  necessary  information  for  strategic  decisions. 
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Thirdly,  in  February  the  complete  faculty  has  moved  to  a new  building.  This  building  was  designed,  constructed 
and  built  by  a more  than  ten  year  participatory  process.  The  new  university  building  has  not  only  already  won 
some  architectural  prizes  but  has  also  led  to  a certain  mood  of  awakening  of  faculty  members. 

Business  Education  as  a virtual  world 

In  this  atmosphere  of  change  our  project  tries  to  collect  and  concentrate  all  the  different  information  for  the 
studies  of  Business  Education  in  one  common  interface.  At  the  moment  we  have  different  approaches  (database, 
booklets,  internet  services,  introductory  information  by  the  faculty  and  student  union,  etc.)  coming  from  different 
sources  (ministry,  center  of  EDP,  initiatives  of  the  department,  individual  information  via  a personal  homepage, 
etc.).  Most  of  this  information  is  text-based  with  just  a small  amount  of  interactivity  and  time  control.  Redundant 
and  complicated  access  to  the  relevant  information  are  the  consequences. 

Virtual  worlds  are  three-dimensional  representations  of  objects  in  the  space  and  their  reproduction  on  display.  To 
program  virtual  worlds  the  Virtual  Reality  Modeling  Language  (acronym  VRML)  is  used  as  a standard  platform- 
independent  language  for  describing  interactive  3-D  objects  and  worlds  delivered  across  the  Internet. An 
extended  type  of  the  virtual  worlds  are  the  so-called  multi-user  worlds.  In  a multi-user  world  the  visitors  are 
shown  by  so-called  avatars,  so  that  visitors  of  multi-user  worlds  can  see  themselves  among  themselves. 

This  pilot  project  offers  service  performances  for  the  students  (in  particular  those  in  their  first-year),  the 
instructors  and  the  administration.  Beyond  that  it  should  also  promote  communication  by  using  multimedia 
between  instructors  and  students  and  among  the  students  themselves.  The  three-dimensional  reproduction  of  the 
faculty  building  and  selected  interiors  and  the  virtual  world  resulting  from  it  form  the  basis  for  the  planned 
supplies  and  service  performances.  The  information  will  also  be  callable  in  a pure  HTML  page  version  for  faster 
access  to  the  data. 

Study  plan:  The  compilation  of  the  complicated  study  plan  can  be  done  comfortably  from  home  by  the 
accessible  list  of  lectures  and  linked  cgi-based  HTML  forms. 

Virtual  information:  Students  can  receive  closer  information  about  lectures,  seminars  and  exercises  on-line  at 
the  virtual  information  boards.  At  any  date  modifications  can  be  placed  by  the  instructors.  This  up-to-date 
information  can  help  to  avoid  unnecessary  journeys. 

Meeting  allocation:  At  the  virtual  information  boards  the  students  can  themselves  into  the  lists  for  the 
respective  meeting.  This  must  not  be  done  anymore  at  the  real  faculty  in  Innsbruck. 

Email  pool:  By  the  collection  of  the  email  addresses  of  the  students  in  an  email  pool,  indicated  for  the  entry  to  a 
meeting,  the  instructors  and  the  students  receive  the  possibility  of  taking  up  with  all  users  of  the  appropriate 
meeting  by  email  contact. 

Virtual  self-manifestation  of  the  institute  and  its  areas:  Here  instructors  can  publish  up-to-date  information 
about  their  institute  e.g.  research  results,  new  book  publications  or  job  advertisements  for  projects,  tutors  or  a 
scientific  co-worker.  Over  an  appropriate  HTML  form  and  email  the  application  can  be  dispatched  on-line. 
Virtual  consulting  hour:  In  the  lecture-free  time  the  students  can  discuss  any  problems  after  previous  log-on  by 
HTML  form  with  the  instructors  in  the  virtual  consulting  hour. 

Virtual  office  for  examination:  Through  a link  to  the  examination  office  the  students  can  check  what 
documents  are  needed  for  the  examination  and  request  the  missing  ones  by  email  or  develop  at  least  a precise 
routing  plan  to  get  all  certificates. 

Virtual  communication:  By  the  implementation  of  the  virtual  faculty  as  a multi-user  world,  the  possibility  of  a 
communication  among  themselves  exists  for  the  students.  When  meeting  at  the  points  of  information  or  in  the 
cafeteria  experiences  can  be  exchanged,  seminars  be  selected  and  occupied  together. 

Virtual  ** black  boards":  Students  can  enter  themselves  at  the  virtual  black  boards  into  different  lists  (room 
search,  riding  along  opportunity,  teaching  materials  etc.)  and/or  look  up  relevant  information. 

The  administration  of  the  updateable  sections  in  the  virtual  faculty  is  made  in  each  case  by  the  responsible 
persons  (e.g.  office  for  examination,  secretariat,  instructors)  over  the  Intemet/Intranet  by  using  a simple  and  self- 
describing interface.  The  updated  virtual  world  is  available  directly,  without  further  maintenance  by  a system 
administrator  from  the  outside.  A virtual  information  world  developed  in  such  a way  can  be  administered  by  the 
users  (the  lecturers)  themselves.  When  the  system  is  accepted  by  students  and  faculty  members  it  can  be 
extended  to  other  major  field  of  studies,  faculties  or  universities. 
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There  are  a number  of  new  issues  that  science  educators  have  been  grappling  with  over  the  past  few  years.  Three  of 
these  issues  are  the  introduction  and  increased  use  of  educational  technology  in  the  classroom,  the  teaching  of 
science  in  an  integrated  format,  and  the  use  of  problem-based  learning  (Ramsey,  1997).  To  address  these  issues  we 
have  designed  a course  for  non-science  majors  that  is  based  on  the  very  real  problem  of  global  warming,  an  issue  of 
relevance  to  the  students.  To  understand  the  various  scientific  issues  involved  in  the  global  warming  phenomenon 
necessitates  a basic  understanding  of  physics,  chemistry,  geoscience  and  biology,  thus  requiring  an  integrated 
approach  to  the  sciences. 

Presently,  a student,  whether  a science  major  or  not,  receives  an  introduction  to  college-level  science  by  taking  a 
basic  science  course  in  one  discipline,  be  it  chemistry,  biology,  physics,  or  geoscience.  It  is  perhaps  only  after 
several  courses  that  students  begin  to  be  able  to  integrate  knowledge  across  the  sciences  and  see  the  larger,  holistic 
picture  - that  the  sciences  have  commonalties  such  as  the  scientific  method,  a need  for  quantifiable  and  verifiable 
data,  the  use  of  statistics  and  probabilities,  and  a healthy  skepticism  tied  to  the  process  of  critical  thinking.  Only 
after  several  courses  do  students  see  the  interface  and  overlap  between  chemistry  and  biology,  geology  and 
chemistry,  physics  and  math,  and  the  quantitative  skills  needed  for  good  science.  If  they  only  take  a couple  of 
courses,  they  may  never  see  this. 

Problem  based  learning  starts  with  the  premise  that  students  will  learn  better  if  they  learn  by  solving  ' rear 
problems  as  opposed  to  learning  by  rote  memorization  or  solving  artificial  problems  of  little  interest  to  the  student 
(Arumbula-Greenfield,  1996).  The  sciences  have  always  had  a strong  component  of  problem  based  learning  in  the 
traditional  laboratory  experiments.  Unfortunately,  two  problems  with  the  usual  science  laboratories  are  a lack  of 
relevance  to  the  student  and  little  connection  between  one  weeks  experiments  and  the  next,  making  it  difficult  for 
the  student  to  tie  everything  together.  Problems  where  the  student  wants  to  know  the  answer  and  needs  to  learn  the 
science  to  find  that  answer  are  in  very  short  supply. 

Technology  is  critically  important  in  almost  all  current  scientific  endeavors,  however,  little  of  this  technology  can 
be  found  in  the  typical  undergraduate  science  laboratory,  especially  in  classes  for  non-majors.  Two  explanations  for 
this  are  the  expense  of  the  technology  and  the  lack  of  time  and  motivation  for  the  students  to  learn  proper  use  of 
the  technology.  Current  scientific  problems  and  access  to  the  appropriate  equipment  for  studying  these  problems  is 
needed  to  teach  modem  science. 

With  funding  from  the  NSF  we  have  designed  a course  organized  around  the  scientific  controversy  of  global 
warming.  This  course  will  introduce  the  students,  both  science  majors  and  not,  to  the  scientific  method.  Emphasis 
is  placed  on  the  collection  and  analyses  of  data,  simple  statistical  evaluations  and  basic  mathematical  modeling 
techniques.  This  approach  encourages  the  evaluation  of  ideas  and  critical  thinking,  questioning,  hypothesis  testing, 
and  theories  vs.  hypotheses,  all  related  to  the  same  scientific  and  societal  problem.  By  focusing  the  course  on  an 
extensive  and  complex  issue,  basic  concepts  and  principles  of  science  will  arise  naturally  through  the  student's 
inquiry-based  investigations  and  attempts  to  understand  the  various  problems  involved  in  the  issue.  Using  a theme 
such  as  global  warming,  they  will  discover  that  they  need  to  learn  about  concepts  such  as  radiation,  photosynthesis, 
the  greenhouse  effect,  gas  absorption,  and  the  laws  of  thermodynamics.  They  will  also  have  to  consider  the 
implications  of  this  knowledge,  and  the  limits  of  our  current  knowledge.  An  added  benefit  of  studying  a complex 
problem  such  as  global  warming  is  that,  in  addition  to  teaching  some  basic  science,  it  will  introduce  the  students  to 
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some  of  the  techniques  for  dealing  with  complex  systems,  in  particular,  systems  thinking  and  the  use  of  dynamic 
models. 

An  introduction  to  systems  thinking  and  mathematical  modeling  of  global  warming  will  be  accomplished  with  the 
modeling  program  Stella®,  A team  of  students,  at  the  culmination  of  the  course,  will  demonstrate  and  justify  the 
final  form  of  their  model.  The  subject  area  of  each  scientific  discipline  would  be  substantially  represented  in  the 
simulation.  Chemistry  will  be  represented  by  information,  data,  and  interactive  demonstrations  on  carbon  dioxide, 
photochemical  reactions,  and  gaseous  exchange  between  air  and  water.  Physics  will  have  instructional  components 
dealing  with  radiation,  the  greenhouse  effect,  and  thermal  loading.  Geoscience  will  be  represented  by  information 
on  volcanic  activity,  precipitation  data,  satellite  images,  the  use  of  fossil  fuels,  and  long-term  climate  projections. 
Biology  would  have  data  on  photosynthesis,  the  effects  of  deforestation  on  the  atmosphere,  and  simulations  of  plant 
growth  in  varying  concentrations  of  carbon  dioxide.  Mathematics  will  be  lep resented  in  aii  of  these  areas  as 
students  learn  how  to  quantify  various  aspects  of  the  model  using  basic  mathematical  modeling  techniques. 
Students  will  access  data  from  satellite  photos  and  international  databases  available  over  the  web,  these  will  be 
integrated  into  their  models  along  with  their  own  experiments  in  the  laboratory. 

The  goal  of  the  course  is  to  stimulate  students  to  learn  science  by  becoming  involved  in  a large,  complex  scientific 
problem  that  requires  their  actively  learning  many  aspects  of  science  in  their  efforts  to  find  a solution  to  the 
problem.  The  specific  problem  for  the  students  will  be  to  determine  the  atmospheric  composition,  global  tem- 
perature, and  effect  on  the  planet  of  a fifty-year  period  of  carbon  dioxide  increases.  They  will  create  a model  of 
biogeochemical  cycles  and  weather  phenomena,  and  collect  information  on  fossil  fuel  use,  methane  generated  by 
cattle  feedlots  and  biomass  burning,  CFCs,  etc.  They  will  use  their  textbook,  their  instructor,  the  WWW,  the  results 
of  their  experiments,  and  specific  data  sources  posted  by  instructors  to  gather  information.  Then  they  will  use  the 
modeling  program  to  analyze  the  connections  between  these  different  components  and  to  make  predictions.  In  the 
process  of  solving  this  problem,  students  would  have  to  understand  and  interrelate  a variety  of  ideas  without  the 
typical  discipline-based  focus. 

Students  will  do  various  experiments  to  learn  about  the  scientific  method  and  how  experiments  can  be  designed 
and  used  to  quantify  the  parameters  and  relationships  that  interact  to  determine  the  earth's  temperature.  These  will 
include  black  body  experiments,  infrared  absorption,  solubility  studies  and  measurements  of  photosynthesis  and 
respiration.  Students  will  analyze  their  results  and  incorporate  the  experimentally  determined  parameters  into  their 
models  at  computer  workstations  that  will  be  part  of  the  laboratory.  A special  laboratory  is  being  constructed  for 
this  course  and  will  include  both  computer  workstations  and  traditional  lab  benches.  This  studio-type 
classroom/laboratory  is  similar  to  those  used  at  other  institutions  such  as  Rensselaer  Polytechnic  Institute  and 
California  Polytechnic  State  University,  San  Luis  Obispo. 

Our  eventual  goal  is  to  use  educational  technology  to  teach  science,  and  some  math,  in  an  integrated  way  by  tying 
together  several  courses,  faculty,  and  departments  into  an  integrated  curriculum  which  uses  computer  software 
simulations  and  models  to  help  interrelate  concepts,  ideas,  and  principles  across  the  science  disciplines  in  a 
problem  solving  format.  This  curriculum,  we  believe,  will  address  the  needs  of  a wide  variety  of  students  with 
diverse  backgrounds.  Whether  the  students  have  a strong  interest  in  math  and  science,  are  inadequately  prepared, 
or  have  little  interest  or  aptitude  in  math  and  science,  the  format  of  this  course  will  encourage  them  to  become 
deeply  involved  in  the  processes  of  science. 
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An  informal  poll  at  the  1998  EDMEDIA  conference  suggested  that  there  is  little  awareness  of 
disability  access  issues  among  AACE  members.  A report  by  the  US  National  Council  on  Disability  (1998) 
reported  a similar  lack  of  awareness  among  US  multimedia  developers.  The  NCD  report  also  found  that  most 
multimedia  developers  were  happy  to  comply  with  accessibility  guidelines  once  the  issues  were  explained.  This 
paper  is  intended  to  raise  awareness  of  disability  access  issues  and  point  readers  to  resources  for  more  specific 
information  on  achieving  accessibility. 

When  asked  about  disability  access,  developers  at  the  1998  EDMEDIA  conference  gave  two  consistent 
responses:  "People  with  disabilities  are  not  part  of  our  audience"  and,  "It  is  too  difficult  to  accommodate  them." 
These  objections  are  both  incorrect  and,  in  many  countries,  illegal. 

"Not  Our  Audience"  - The  Moral  and  Legal  Mandate 

The  US  National  Council  on  Disability  (1996a)  called  people  with  disabilities  the  largest  minority 
group  in  the  United  States.  There  are  an  estimated  30  million  US  citizens  with  functional  disabilities  that  limit 
their  access  to  multimedia  products  (Vanderheiden  1990).  The  abilities  and  interests  of  this  population  are  as 
diverse  as  those  of  the  "normal"  population.  It  is  unlikely  that  any  educational  topic  addressed  by  a multimedia 
product  will  not  be  of  interest  to  some  individuals  with  disabilities. 

The  proportion  of  the  population  that  experiences  functional  disabilities  rises  dramatically  with  age, 
from  10%  of  adults  age  25  to  34,  to  45%  of  adults  age  65,  and  72%  for  adults  aged  75  and  older  (Vanderheiden 
1990).  These  figures  suggest  that  the  number  of  people  with  disabilities  will  dramatically  rise  as  the  US 
population  ages.  These  figures  also  suggest  that  the  odds  of  any  particular  individual  (such  as  the  reader) 
needing  accommodations  increases  as  that  individual  ages. 

Different  disabilities  will  limit  in  different  ways  a person's  ability  to  interact  with  multimedia  software. 
Relevant  sensory  disabilities  include  low  vision,  blindness,  color  blindness,  reduced  hearing,  and  deafness. 
Relevant  physical  disabilities  that  limit  a person's  fine  muscle  control  and  hand-eye  coordination  include  spinal 
cord  injuries,  arthritis,  cerebral  palsy,  various  degenerative  nerve  and  muscle  conditions  and  other  disorders.  A 
variety  of  cognitive  disabilities  can  limit  how  well  an  individual  deals  with  rapidly  changing  situations. 

Several  researchers  have  pointed  out  that  accommodation  features  for  people  with  disabilities  often 
improve  a product's  general  utility  (Bergman  & Johnson  1995;  Newell  1995).  One  classic  case  is  sidewalk 
curbcuts  for  wheelchair  access.  Unexpectedly,  curbcuts  have  also  improved  access  for  people  pushing  strollers, 
shopping  carts,  movers'  dollies,  or  any  wheeled  device.  Another  example  is  books  on  tape  that  were  originally 
developed  as  an  accommodation  for  the  blind  but  are  now  widely  used  by  people  on  long  car  trips. 

The  latter  example  illustrates  that,  in  some  situations,  individuals  without  a disability  may  have 
situational  limitations  similar  to  individuals  with  a disability.  The  driver  of  a car  is  visually  limited  by  the  need 
to  watch  the  road.  Similarly,  accommodations  for  color  blindness  in  a computer  interface  will  also  solve 
potential  problems  of  users  with  black  and  white  monitors.  Alternative  text-based  versions  of  Web  pages  to 
accommodate  individuals  with  vision  problems  will  also  improve  access  for  individuals  with  slow  modem 
connections.  Captioned  video  clips  for  users  with  hearing  loss  will  also  improve  use  for  individuals  viewing  in  a 
quiet  library  or  a very  noisy  computer  lab.  The  provision  of  information  in  multiple  formats  (audio,  graphic, 
text,  etc.)  from  which  the  user  can  choose  is  useful  to  accommodate  both  users  with  different  disabilities  and 
users  with  different  learning  styles.  Because  of  these  general  utility  aspects,  some  researchers  call  computer 
accessibility  features  "electronic  curbcuts"  (Bergman  & Johnson  1995;  Campbell  & Waddell  1997) 

The  archetype  for  many  laws  regarding  disability  access  is  the  US's  Americans  With  Disabilities  Act 
(ADA)  passed  in  1990.  This  law  requires  (among  other  things)  that  all  public  facilities  and  services  be 
accessible  to  people  with  disabilities.  In  the  wake  of  this  law,  it  has  become  standard  in  many  US  universities  to 
include  on  course  syllabi  a statement  describing  the  right  of  any  student  to  accommodation  for  a disability.  This 
has  direct  relevance  to  educational  multimedia  products.  Any  US  educational  institution  that  purchases  or 
develops  a computer-based  educational  product  that  is  not  accessible  to  individuals  with  disabilities  is  open  to  a 
lawsuit  under  the  ADA. 

Over  the  last  decade,  many  countries  have  passed  similar  laws  requiring  access  and  accommodation 
for  people  with  disabilities,  including  Australia,  New  Zealand,  Canada  and  Britain.  There  is  no  single  list  of 
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countries  with  such  laws,  but  it  is  likely  most  countries  have  them  at  least  on  the  books  (personal 
communication,  Mark  Behrendt,  World  Institute  on  Disability,  February  1999). 

''Accommodation  is  too  difficult" 

The  ADA  and  similar  laws  do  not  require  expensive,  burdensome  accommodations  for  all  possible 
disabilities.  Instead,  the  ADA  requires  that  software  developers  make  a "reasonable"  effort.  Put  another  way,  if 
there  is  something  basic  that  is  not  too  hard  to  do,  you  need  to  do  it.  What  a developer  cannot  do  is  claim  that 
nothing  can  be  done.  If  access  issues  are  addressed  early  in  the  development  process,  solutions  are  generally 
simple  and  inexpensive  to  implement.  Accommodation  only  becomes  expensive  if  it  is  left  until  after  the 
product  is  finished. 

The  ideal  approach  to  accommodation  is  built-in  access  to  multiple  media  formats  within  the  product. 
This  "universal  design"  concept  is  most  appropriate  to  developers  working  "from  scratch"  on  entirely  new 
software  products  that  do  noi  depend  on  existing  software  platforms.  Approaches  to  universal  design  are 
currently  the  focus  of  more  literature  than  can  be  reviewed  here.  A list  of  resources  is  available  at  the  author's 
website  at  tiger.coe.missouri.edu/'^rcepT/access. 

A second  approach,  "Adaptive  accommodation, " depends  on  the  use  of  third  party  "accommodation 
software"  (National  Council  on  Disability  1 996b;  Bergman  & Johnson  1 995).  There  are  many  types  of 
accommodation  software  products,  ranging  from  screen  readers  and  Braille  readers  for  the  blind  to  alternative 
mouse  and  keyboard  controls  for  those  with  physical  disabilities.  Resources  for  such  software  are  also  at  the 
author's  website.  If  developers  provide  features  accessible  to  these  programs,  a variety  of  disabilities  can  be 
accommodated  to  some  extent.  Below  are  some  general  guideline  for  developers  and  educators  using  adaptive 
accommodation: 

1.  Provide  ASCII  text  (or  text-only)  alternatives  for  all  information,  ASCII  text  functions  as  a "presentation 
independent"  format  for  a variety  of  accommodation  software  products.  Many  individuals  with  blindness  use 
"screen  reader"  software  that  translates  on-screen  ASCII  text  to  spoken  language  or  Braille.  Unfortunately, 
multiple-column  text  layouts  can  confuse  screen  readers.  In  addition,  graphic  images  of  text  are  inaccessible  to 
screen  readers.  Developers  should  provide  simple  ASCII  text  files  that  accommodation  software  can  accurately 
process.  The  problem  of  layouts  has  become  especially  significant  in  developing  HTML  accessibility 
guidelines.  It  should  be  noted  that  PDF  files  are  not  currently  accessible  to  accommodation  software. 

2.  Provide  captioning  and  alternative  audio  tracks  for  video  clips.  Many  video  formats  (including 
Quicktime  and  SMIL)  now  allow  closed  captioning  of  the  narration  for  users  with  deafness.  In  addition,  these 
formats  allow  an  alternative  audio  track,  similar  to  closed  captioning,  to  provide  additional  descriptions  for 
users  with  blindness.  As  mentioned,  this  can  also  prove  useful  for  users  in  extremely  noisy  environments. 

3.  Provide  text  descriptions  of  all  graphic  and  audio  information.  Besides  text  versions  of  images  of  text, 
text  descriptions  of  all  images,  charts,  and  sounds  must  be  provided.  In  HTML,  use  ALT  labels  for  this. 

4.  Distinguish  interface  elements  with  more  than  one  type  of  coding.  Use  a combination  of  media  elements 
for  all  significant  features  of  a multimedia  product.  If  screen  elements  are  distinguished  only  by  different  colors 
(colored  parts  of  a map,  for  instance)  people  with  color  blindness  may  not  be  able  to  read  it.  This  problem 
would  also  arise  for  people  using  a black  and  white  monitor. 

5.  Be  aware  of  local  disability  access  resources.  Many  universities  have  disability  access  offices,  which  are  a 
good  source  of  information  on  accommodation  software.  The  staff  is  frequently  willing  to  demonstrate  software 
and  evaluate  multimedia  projects.  Most  educators  and  developers  will  find  it  enlightening  to  meet  with  the  staff 
of  a disability  access  office  even  before  a specific  need  arises. 
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Prior  research  on  live  classroom  teaching  in  higher  education  has  found  that  the  degree  of  closeness 
between  instructors  and  students,  or  teacher  immediacy  (Hackman  & Walker,  1990)  can  increase  affective 
learning  (or  student  attitudes  towards  instruction),  which  in  turn  increases  cognitive  learning.  Immediacy 
behaviors  that  are  easy  to  perform  in  class  are  difficult  to  even  imagine  in  a Web  course,  such  as  smiling  at 
students  and  calling  them  by  name.  Earlier  research  (LaRose,  Gregg  & Eastin,  1999)  showed  that  Web  courses 
can  match  the  immediacy  of  live  instruction  when  at  least  some  of  these  features  are  present.  Interactions  with 
other  students  and  interactions  with  computers  may  also  affect  outcomes,  we  refer  to  these  factors  collectively 
as  instructional  immediacy  (LaRose  & Whitten,  1999).  Here  we  examine  how  Web  courses  might  be  improved 
by  assessing  the  state  of  the  art,  adding  immediacy  features  to  web  courses  and  understanding  the  immediacy 
needs  of  learners  on  the  Web. 

Web  Course  Content  Today 

We  searched  the  World  Lecture  Hall  for  standalone  Web  courses  delivered  during  the  1998  calendar 
year  and  that  had  at  least  one  of  their  lessons  available  for  public  viewing.  We  found  48  and  performed  a 
content  analysis  of  their  immediacy  features  using  a standard  coding  guide  that  yielded  90  percent  inter-coder 
reliability.  Few  courses  employ  the  multimedia  capabilities  aside  from  providing  hyperlinks  to  additional 
information.  Only  a third  include  graphics  in  their  lessons  while  less  than  10  percent  used  video  or  animations 
and  none  of  the  courses  used  audio  lessons.  Aside  from  providing  an  email  address,  publication  of  the 
instructor's  phone  number  was  the  only  course-level  immediacy  feature  found  in  over  half  the  cases,  although 
discussion  areas  are  common  to  about  a third.  Few  courses  invite  student  contact  outside  of  scheduled  office 
hours.  About  two-fifths  attempt  to  convey  the  personality  of  the  instructor  by  posting  snapshots  or  personal 
information.  Opportunities  for  student  interaction  are  minimal.  There  was  only  a single  instance  of  a "students 
only"  communications  option.  The  most  common  immediacy  feature  in  the  lessons  themselves  is  content  that 
refers  to  the  class  in  the  second  person  (e.g.,  by  using  "we"  or  "our"),  a third  of  the  lessons  did  that.  Open-ended 
questions  and  (delayed)  feedback  on  student  quizzes  and  assignments  are  in  only  a fourth  of  the  lessons.  To 
quantify  immediacy,  we  computed  an  additive  index  of  24  immediacy  features  and  found  an  average  of  less 
than  six  per  course.  Our  sample  was  limited  to  publicly  accessible  material  and  perhaps  better  ones  are  hidden 
behind  password  protection  barriers.  With  the  permission  of  their  instructors,  we  peered  behind  the  password 
barriers  at  five  Web  courses  on  our  own  campus.  The  five  courses  averaged  13  immediacy  features  each  and 
one  contained  21  of  the  24  immediacy  cues  in  our  index.  Informal  comments  from  other  Web  instructors  with 
whom  we  shared  our  results  confirm  that  the  best  features  are  often  hidden  from  public  view. 

Improving  Web  Course  Immediacy 

We  asked  a sample  of  net-savvy  college  students  from  one  of  our  on-campus  courses  to  visit  three 
Web  courses  and  to  "think  out  loud"  as  they  surfed.  One  was  entirely  text  and  graphics,  a second  added  audio 
and  a third  had  video.  Then  our  surfers  joined  focus  groups  to  identify  characteristics  of  "best  courses"  in  the 
conventional  classroom  mode  and  to  compare  them  with  Web  courses.  Immediate  feedback  is  what  students 
crave.  They  want  instant  feedback,  even  if  limited  to  finding  out  the  answer  to  a multiple  choice  question  or  to  a 


"roll  call"  feature  that  acknowledges  them  by  name.  Email  is  insufficient,  responses  are  too  slow  and  it  requires 
opening  a separate  application.  One  virtual  instructor  encouraged  students  to  use  the  AOL  "instant  messenger" 
system  so  that  they  could  reach  him  any  time  he  was  on-line.  But  the  "Help"  button  needs  to  be  ever  present  in 
the  lessons  and  (ultimately)  lead  to  a real  person.  Many  comments  related  to  text-laden  lessons.  Ifs  hard  to  stay 
focused  on  text  or  static  graphics,  but  attention  wanders  even  with  Web  video  — especially  the  talking  head 
variety  common  today.  Changing  visual  course  outlines  add  to  immediacy  by  locating  learners  in  the  course. 
Animated  gifs  and  mouseover  events  can  help.  Failing  that,  students  want  an  easy  way  to  print  sections  of  the 
course  (as  opposed  to  each  individual  page)  so  they  can  read  from  paper.  Learners  also  criticize  marginally 
relevant  links  to  volumnious  text  resources.  Teacher  immediacy  comes  from  personal  information  about  the 
instructor  that  invites  learners  to  imagine  what  kind  of  people  instructors  are,  especially  candid  personal 
information,  snapshots  and  personal  comments  about  lesson  material.  Chatrooms  do  not  create  sufficient 
immediacy  with  other  students.  Providing  pictures  and  bios  of  students  is  a way  of  "getting  to  know"  others 
better,  but  some  of  our  female  students  had  a problem  with  that.  Navigation  issues  include  dead  and  misdirected 
links  and  secondary  browser  windows  that  completely  overlap  the  lesson  page.  An  on-site  search  engine  and 
basic  navigation  and  instructor  contact  buttons  on  every  lesson  page  are  also  desired.  Learners  also  like  to  save 
notes  on-line,  automatically  go  back  to  where  they  left  off  in  the  previous  session,  and  launch  the  next  page 
automatically  (i.e.,  instead  of  hitting  the  "next"  button). 

Reaching  Nontraditional  Learners 

We  sampled  email  addresses  at  random  from  Bigfoot  and  directed  respondents  to  our  Web  survey  site. 
We  received  25  responses  to  our  pretest  that  give  a preliminary  indication  of  the  factors  in  Web  course 
adoption.  Half  of  our  respondents  are  interested  in  taking  courses  over  the  Web.  The  convenience  of  taking 
courses  whenever  and  wherever  the  learner  might  want  are  two  important  advantages  of  Web  courses.  General 
convenience,  the  ability  to  work  at  one's  own  pace,  the  elimination  of  travel  to  campus  and  the  ability  to  take  the 
course  in  different  places  also  received  multiple  mentions.  Other  advantages  included  the  ability  to  concentrate 
and  organize  better  in  the  home  environment  and  to  get  better  feedback  than  in  the  conventional  classroom. 
Most  saw  the  loss  of  instructor  and  student  interaction  as  major  disadvantages.  Concerns  about  the  quality  of 
Web  instruction  and  the  acceptance  of  credits  by  employers  are  also  widespread.  Other  are  concerned  about  the 
difficulty  of  reading  text  off  the  computer  screen  and  typing  out  course  interactions,  overcoming  the  distractions 
of  the  home  environment,  finding  the  motivation  for  self-paced  lessons,  and  obtaining  technical  help.  According 
to  our  respondents,  providing  feedback  and  praise  to  students  are  the  most  important  teacher  immediacy  needs, 
followed  by  the  opportunity  to  hold  conversations  before  class,  asking  questions  of  students,  organizing 
activities  outside  of  class  and  smiling  at  the  class.  Some  of  the  instructor  behaviors  thought  to  increase 
immediacy  in  the  classroom  are  not  very  important  to  potential  Web  students,  including  inviting  contact  outside 
the  classroom,  displaying  a relaxed  posture,  gesturing  and  allowing  students  to  call  the  instructor  by  his  or  her 
first  name.  Interactions  with  other  students  are  not  as  important  as  interactions  with  instructors  but  two  stand 
out:  Gamering  praise  from  other  students  and  having  others  sit  near  you  in  class. 
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1.  Intruduction 

WebReCourse  [5],  the  Retargetable  Course  Generator  enables  creation  and  reuse  of  Web  courses.  It  is  a secure 
software  system  for  online  course  management,  allowing  instructors  to  increase  the  accessibility  of  online  course 
material  and  to  create  a convenient  means  of  communication  between  instructors  and  students.  It  provides  many  of 
the  enhancements  missing  in  the  World  Wide  Web  implementation  of  hypertext. 

Previous  papers  [6,7]  have  analyzed  student  use  and  response  to  these  tools. 


2.  Recourse:  the  Student  Perspective 

Recourse  requires  the  student  to  login  with  a username  and  password.  (Students  who  just  want  to  see  the  pages,  and 
don't  plan  on  using  the  tools  can  bypass  this  login.)  Once  entered,  the  student  sees  the  course  web  pages  and  icons 
representing  the  tools.  Although  the  instructor  can  turn  off  any  of  the  tools,  the  available  tools  are:  Add  New  User, 
Home  Page  Link,  Chat  Room,  Search  Tool,  Grade  Access,  Site  Map,  Bulletin  Board,  Quiz  Tool,  Password  Tool,  and 
Help.  For  further  details  on  these  tools,  see  [5]. 


3.  Recourse:  the  Instructor  Perspective 

We  characterize  faculty  characteristics  below: 

Faculty  Status  (!)  When  the  ReCourse  team  (students)  sent  an  email  invitation  to  faculty,  inviting  them  to  try  out 
and  then  use  the  system,  there  was  virtually  zero  response.  When  the  author  (faculty)  sent  the  same  message,  many 
people  responded. 

Faculty  Chaos  & Inertia  Of  the  faculty  who  did  respond  and  declined  to  use  the  system,  no  one  had  time  to  even 
look  at  it.  All  mentioned  wanting  to  use  it  "in  the  future."  All  appeared  overworked  - one  mentioned  being 
overwhelmed  that  term 

Missing  functionality  One  faculty  used  it  for  one  term  and  did  not  choose  to  use  it  again.  One  of  these  wanted  a 
feature  not  yet  available  (the  grading  system  does  not  operate  with  full  spreadsheet  features  yet). 

No  instructor  help  One  factor  is  that  some  instructors  have  no  teaching  assistants  to  help  them. 

Unknown  One  other  faculty  member  responded  to  our  request,  said  he  would  like  to  use  it,  but,  even  with  urging  and 
help  from  the  ReCourse  team,  failed  to  use  the  tools. 


3.  Author  Characteristics 

Although  we  continue  to  gather  information  and  statistics,  we  know  the  following: 
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• The  more  computer  literate  an  instructor/author  is,  the  more  apt  they  are  to  use  the  system,  and  the  more  apt 
they  are  to  use  more  of  the  available  tools. 

• The  more  helpful  the  assigned  ReCourse  team  member  is,  the  more  apt  the  author/instructor  is  to  use  the  tool  in 
its  entirety. 

• Newer  faculty  are  much  more  apt  to  respond  to  the  invitation,  to  use  more  of  the  tools,  and  to  return  to  use  it 
again.  This  is  due  to  more  experienced  instructors  having  (existing,  but  not  as  appropriate)  tools  they  are  used  to 
using  and  the  general  chaos  that  surrounds  faculty  members  as  they  find  themselves  with  more  service 
activities.  The  ReCourse  team  believes  as  they  hear  that  the  tool  is  an  asset,  they  will  make  the  (small)  effort  to 
use  it. 


4.  Conclusions 

Using  the  Web  to  enhance  or  as  the  sole  medium  for  teaching  has  produced  a new  range  of  problems  and 
opportunities.  Tools  that  facilitate  such  teaching  are  appearing  and  enhancing  the  delivery  and  activities  of  these 
courses.  Studies  have  been  made  on  how  readers  (usually  students)  use  such  material  and  tools.  Systems  have  been 
developed  that  analyze  the  student  profile,  adapting  both  the  pages  and  the  navigation.  Tools  such  as  chat  rooms, 
bulletin  boards,  cooperative  learning  techniques  have  all  been  designed,  implemented,  used,  and  studied  from  the 
student  perspective.  This  paper  explored  the  other  side  of  these  issues:  the  perspective  from  the  author  (usually  an 
instructor). 

ReCourse,  the  Retargetable  Course  Generation  system,  has  had  the  instructor  as  a focus  from  inception.  We  reported 
on  what  we  have  discovered,  what  we  have  done  as  a result  of  these  studies,  and  how  these  changes  have  improved 
and  will  continue  to  improve  the  system  for  students  as  well. 

ReCourse  is  documentably  popular  and  useful  for  web  courses.  Students  like  it.  But  students  have  grown  up  in  a 
computer  world  and,  now,  even  in  a web  world.  Instructor/authors  have  more  inertia.  We  are  continuing  to  gather 
statistics  from  both  the  users  and  non-users  in  an  attempt  to  encourage  use  of  a system  which  improves  the  delivery 
of  web  courses. 
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Background 

Teacher  education  programs  are  a catalyst  for  producing  quality  educators  for  the  preK-12  enviromnents.  It  is  through  the  classroom 
experiences  and  the  guidance  of  teacher  educators  that  can  assist  preservice  teachers  in  the  formulation  of  their  philosophical  beliefs 
regarding  teaching  and  learning  (Harrington,  1993).  New  teachers  are  quickly  confronted  with  many  classroom  and  administrative 
demands  which  requires  them  to  have  a firm  grasp  on  their  philosophical  beliefs  so  that  they  can  handle  educational  issues 
effectively.  Technology  is  an  increasingly  important  issue  forpreK-l2  schools  and  has  the  potential  to  have  immense  impact  on  the 
learning  and  teaching  environments.  Because  of  the  expedient  growth  in  technology,  it  is  critical  for  educators  to  be  cognizant  of 
technological  uses,  impacts,  and  creative  integrative  strategies. 

Teacher  educators  in  higher  education  are  finding  themselves  inadequately  prepared  for  technological  uses,  let  alone  having  the 
capacity  to  integrate  it  as  part  of  the  curriculum  (Panel  on  Educational  Technology,  1997).  Many  agree  that  integrating  technology 
into  the  curriculum  is  very  consequential  and  vital  to  the  survival  of  our  educational  institutions,  yet  it  continues  to  be  insufficient  for 
many  (OTA,  1996).  Fortunately,  there  are  quality  institutions  implementing  innovative  ways  to  use  technology.  No  longer  are  just 
televisions  and  VCRs  being  used,  but  various  technologies  are  surfacing  in  higher  education  learning  environments  creating  rich  and 
fertile  grounds  from  which  students  and  educators  are  constructing  and  building  knowledge  collaboratively  (Thomas  & Boysen, 
1984).  Using  technology  to  transform  classrooms  evolves  into  changing  the  philosophy  of  the  schools,  and  ultimately  the  overall 
view  of  education  (Sheingold,  1991).  Thomas  & Boysen  (1984)  contend  that  in  utilizing  computers  in  the  most  meaningful  capacity, 
there  is  the  potential  to  create  a context  in  which  students  are  learning  to  educate  themselves  through  the  use  of  computers. 


Objective/Purpose/Project  Description 

Since  teacher  educators  are  receiving  insufficient  exposure  to  learning  technological  uses,  preservice  teachers  are  denied  learning 
opportunities,  which  ultimately  impacts  preK-12  education.  There  are  periodic  workshops  offered  for  teacher  educators  in  higher 
education,  but  they  lack  the  situational  learning  that  is  needed  so  that  they  can  adapt  and  transfer  learning  to  their  own  environment. 
The  poignant  question  that  now  stares  at  us  is:  What  can  be  done  to  provide  concrete  examples  and  promote  situational  learning  on 
the  uses  and  integration  techniques  of  technology  for  higher  educators? 

It  is  our  contention  that  the  construction  of  knowledge  through  the  use  of  technology  may  be  the  most  feasible,  meaningful,  and 
integrative  method  in  which  to  learn  methods  of  technology  integration  in  the  classroom.  Educators  need  and  desire  concrete 
examples  of  what  technology  integration  looks  like  and  how  it  affects  learning. 

A research  team  at  Iowa  State  University  in  the  College  of  Education  in  the  Center  for  Technology  in  Learning  and  Teaching  (CTLT) 
has  begun  to  address  this  problem.  This  research  team  has  designed  a multimedia  program  with  video  case  studies  on  CD-ROM 
format  that  models  exemplary  uses  of  technology  in  various  learning  environments.  The  purpose  of  this  study  is  to  design,  develop, 
evaluate,  and  implement  this  program  in  a learning  environment  for  teacher  educators  to  provide  them  with  rich  examples  of  how 
technology  can  fertilize,  stimulate,  and  flourish  a learning  environment. 

The  video  cases  will  be  used  as  an  anchor  in  the  learning  environment  to  provide  a situation  where  the  learners  share  a common 
experience  (Hasselbring,  1996).  Within  this  shared  experience  among  the  participants,  the  learners  will  collaboratively  build 
knowledge  by  pulling  from  past  experiences  and  merging  that  with  new  information  received  from  the  video  cases.  This  approach  of 
using  the  conceptual  anchor  in  the  form  of  video  can  produce  a common  frame  of  reference  better  than  any  verbal  explanation 
(Blackhurst;  Morse,  1996).  The  video  captures  a rich  and  flavorful  environment  from  which  the  participants  can  see  the  emotion, 
energy,  and  body  language  (from  both  students  and  the  educator)  which  prospers  in  a learning  environment  interwoven  with 
technology. 


Procedure 

During  Phase  one,  teacher  educators  were  video  taped  in  the  College  of  Education  who  demonstrated  exemplary  technology  uses  in 
their  teaching  methods.  These  tapings  were  then  viewed,  edited,  and  constructed  into  video  vignettes  through  the  use  of  Adobe 
Premiere.  The  clips  for  the  vignettes  were  carefully  chosen  in  order  to  provide  the  richest  examples  that  would  be  meaningful  for  the 
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audience.  A portion  of  the  spectrum  of  video  examples  obtained  include  a math  class  using  the  Iowa  Communications  Network 
(ICN),  a language  arts  class  using  Powerbooks,  a math  class  using  the  Jasper  series,  and  a special  education  class  using  the  Discourse 
groupware  system.  Through  utilizing  a variety  of  content  areas,  we  will  be  able  to  more  adequately  reach  the  diversity  of  the  end- 
user. 

The  design  and  development  occurred  during  Phase  two  and  was  formulated  simultaneously  with  the  video  editing  portion  of  the 
project.  Many  hours  were  spent  within  the  participatory  design  team  to  create  the  most  suitable  interface  for  the  end  user.  Drawings 
were  initially  compiled  so  that  the  group  had  a conceptual  idea  of  the  overall  look  of  the  program  and  navigation  abilities  that  would 
be  provided  for  the  user.  An  initial  pilot  program  was  created  to  obtain  expert  appraisal  and  end-user  feedback  from  Iowa  State 
faculty  and  audience  participants  in  various  conferences.  Suggestions  were  taken  back  to  the  participatory  design  team  and  revisions 
were  made.  As  this  project  is  continually  under  revision  as  we  add  new  video  cases  and  involve  more  end-users,  this  feedback  is 
critical  to  the  continual  design  and  development  of  our  final  program. 

The  template  for  the  interface  of  the  multimedia  program  was  developed  in  Adobe  Photoshop  and  Authorw'arc  was  used  as  the 
authoring  tool.  The  nvultimedia  program  is  cousliucleu  in  such  a way  that  each  panicipant  can  navigate  according  to  his/her  interests 
and  have  the  opportunity  to  experience  realistic  examples  of  technology  integration  and  begin  to  reflect  and  construct  knowledge  for 
their  personal  usefulness. 

Our  intention  for  Phase  3 is  to  formulate  a workshop  for  higher  educators  where  the  video  cases,  accessed  through  the  multimedia 
program,  are  used  as  the  anchor  for  instruction.  The  combination  of  group  discussions,  individual  navigation  within  the  multimedia 
program,  and  individual  and  group  reflections  will  contribute  to  the  ability  of  the  participants  to  transfer  newly  acquired  knowledge 
to  their  present  situation.  Rather  than  just  using  their  imagination,  educators  will  have  the  ability  to  obtain  a realistic  picture  of  what  a 
learner-centered,  technology  rich  environment  looks  like. 


Data  Collection 

Research  members  will  conduct  detailed  observations  as  participants  interact  with  the  program,  noting  reactions,  comments,  interest 
level,  and  the  time  spent  navigating  through  the  program.  In  addition,  a follow  up  interview  will  be  completed  to  determine  if  any  of 
the  perceptions  and  uses  of  technology  of  the  participants  have  changed. 


Preliminary  Conclusions 

The  utilization  of  this  video  case  multimedia  product  will  not  address  all  of  the  problems  that  staff  development  sustains  for  teacher 
educators  in  higher  education.  However,  the  modeling  of  learner-centered  technology-rich  environments  will  procure  the  missing 
element  that  many  training  situations  desperately  crave.  This  type  of  learning  environment  allows  the  educators  to  gain  the 
knowledge  needed  to  integrate  technology,  but  is  presented  in  such  a way  that  each  learner  can  construct  how  technology  utilization 
can  fit  into  his/her  particular  classroom.  Through  the  rich  examples  of  various  technology  uses,  the  learning  environment  becomes 
more  meaningful  and  realistic  for  the  teacher  educator.  This  collaborative  effort  of  viewing  the  videos,  reflection,  problem-solving, 
and  construction  of  meaning  will  reinforce  that  the  technologies  used  can  improve  the  learning  and  transfer  of  knowledge  to  the 
participants  classroom  environment. 
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Introduction:  Learning  by  Doing 

The  Virtual  Cell  is  being  developed  by  the  NDSU  World  Wide  Web  Instructional  Committee  (WWWIC),  a 
multi-disciplinary  group  of  faculty  engaged  in  the  development  of  virtual/visual  worlds  for  science  education  (Slator 
et  al.  1999).  Other  WWWIC  projects  include  The  Geology  Explorer,  The  Visual  Computer  Program  and 
ProgrammingLand  MOO.  Each  of  these  projects  share  a common  objective:  to  teach  scientific  problem-solving 
skills  (deductive  reasoning,  experimental  design,  hypothesis  formation)  through  immersion  in  leam-by-doing  virtual 
environments.  These  projects  are  designed  to  support  discovery-based  learning  in  a self-paced,  role-based,  and 
goal-oriented  framework.  They  are  also  learner  oriented,  immersive,  and  exploratory  in  nature.  Each  project  aims 
toward  highly  interactive,  highly  graphical  systems  employing  software  tutors  to  guide  and  remediate  in  the  event  of 
student  failure.  In  addition,  each  project  individually  pursues  the  objective  of  teaching  discipline-specific  science 
content  through  the  achievement  of  authentic  problem-solving  goals,  but  using  uniquely  different  approaches.  The 
projects  are  designed  to  employ  consistent  elements  across  disciplines  and,  as  a consequence,  foster  the  sharing  of 
development  plans  and  development  tools.  More  information  about  WWWIC  and  related  projects  is  available  at 
http://www.ndsu.nodak.edu/wwwic/. 

One  goal  of  post-secondary  science  education  is  to  train  future  scientists.  One  problem  with  science 
education  is  the  standard  lecture/laboratory  format.  In  lecture,  the  professor  speaks,  and  the  student  passively 
listens.  In  the  worst  case,  lectures  are  no  better  for  learning  than  television:  a totally  passive,  non-interactive 
experience.  Meanwhile,  laboratories  are  intended  to  afford  students  with  an  interactive,  experimental  experience, 
but  in  reality  these  are  usually  rigidly  structured  by  the  laboratory  outline  where  the  intended  outcome  is  known  and 
the  procedure  is  inflexible.  These  are  not  experimental  experiences.  If  laboratories  were  to  be  truly  experimental, 
question-  and  hypothesis-driven  experiences,  then  students  would  need  access  to  modem  laboratory  equipment  and 
instrumentation,  an  array  of  expensive  laboratory  reagents,  and  a wealth  of  instructor  time  that  the  university  cannot 
possibly  afford.  Active  learning  can  be  expensive,  yet  the  value  of  active  versus  passive  learning  has  become 
increasingly  clear  (Reid,  1994). 


The  Virtual  Cell 

The  Virtual  Cell  is  a virtual,  multi-user  space  where  students  "fly"  around  and  practice  being  cell  biologists 
in  a role-based,  goal-oriented  environment.  The  Virtual  Cell  development  project  can  be  visited  at 
http://www.ndsu.nodak.edu/instruct/mcclean/vc/.  Working  individually,  or  with  others,  students  learn  fundamental 
concepts  of  cell  biology  and  strategies  for  deductive  problem  solving  through  their  experiences  in  the  exploratory 
environment,  where  they  can  make  observations,  manipulate  the  interactive  objects,  and  design  and  perform 
experiments.  This  pedagogical  approach  gives  students  an  authentic  experience  that  includes  elements  of  practical, 
experimental  design  and  decision  making,  while  introducing  them  to  discipline  content.  By  practical  applications  of 
the  scientific  method,  students  learn  how  to  think,  act,  and  react  as  cell  biologists  (Slator  and  Chaput,  1996). 

The  Virtual  Cell  is  populated  with  subcellular  components:  nucleus,  endoplasmic  reticulum,  Golgi 
apparatus,  mitochondria,  chloroplast  and  vacuoles.  Each  structure  is  rendered  as  a 3D  object  using  the  Virtual 
Reality  Modeling  Language  (VRML),  a computer  language  for  specifying  three-dimensional  worlds  that  can  be 
displayed  and,  most  importantly,  interacted  with  via  the  Internet.  The  Virtual  Cell  consists  of  a VRML-based 
laboratory  and  a cell.  In  the  laboratory,  the  learner  receives  a specific  assignment  (learners  are  always  assigned 
motivating  goals  in  our  leaming-by-doing  environments),  performs  simple  experiments,  and  learns  the  basic 
physical  and  chemical  features  of  the  cell  and  its  components.  The  virtual  laboratory  procedures  require  a voyage 
into  a VRML  cell,  where  experimental  science  meets  virtual  reality.  The  learner  is  supplied  with  a toolbox  of 


measuring  devices  that  assay  various  cellular  processes.  These  tools  include  an  O2  meter,  CO2  meter,  pH  meter, 
sugar  assay,  protein  assay,  various  stains,  and  enzyme  assays.  As  the  students  progress,  they  revisit  the  laboratory, 
bring  cellular  samples  back  for  experimentation,  and  subsequently  receive  more  assignments. 

The  virtual  cell  contains  3D  representations  of  all  the  components  and  organelles  of  a cell  (nucleus, 
mitochondria,  chloroplasts,  etc).  The  user  "flies"  among  these  organelles  and  uses  virtual  instruments  to  conduct 
experiments.  All  navigation  is  learner  directed;  there  is  no  predetermined  exploratory  path.  This  feature  empowers 
the  student  to  direct  their  own  learning.  The  student  is  also  able  to  travel  into  linked  VRML  worlds  that  represent 
the  interior  of  each  of  the  cellular  organelles.  Further  experimentation  inside  each  organelle  allows  the  student  to 
learn  about  their  specific  functions.  For  example,  the  learner  may  confront  the  nucleus  and  perform  several  simple 
experiments  via  touch  sensitive  points.  The  nucleus  is  not  consuming  or  generating  O2  or  CO2  and  it  has  a positive 
DNA  Synthesis  assay.  Putting  this  and  information  from  additional  experiments  together,  the  learner  should  deduce 
that  the  object  is  the  nucleus  and  that  DNA  is  contained  there.  Additional  pei  tinent  data  about  tlie  nucleus  and  otlier 
cellular  organelles  can  be  collected  in  the  same  manner. 

More  advanced  levels  of  the  Virtual  Cell  include  the  introduction  of  cellular  perturbations  and  the 
investigation  of  the  functions  of  various  cellular  structures  or  processes.  In  the  second  level,  a simulation  will 
change  the  cell  by  either  introducing  a mutation  or  adding  an  inhibitor  that  disrupts  a cellular  process.  An  alarm  will 
sound,  some  cellular  process  will  malfunction,  and  the  learner  will  be  given  the  goal  of  diagnosing  the  problem. 
Using  the  same  tools  as  in  the  previous  level,  the  learner  will  navigate  through  the  cell,  make  observations,  and 
perform  measurements  and  experiments.  The  learner  will  attempt  to  identify  the  affected  area,  the  perturbed 
process,  and  the  nature  of  the  mutation  or  inhibitor  that  is  causing  the  problem.  As  a result,  the  user  will  learn 
details  of  cellular  processes  and  functions  and  become  familiar  with  the  importance  of  various  mutations  and 
inhibitors  for  cell  biology  experimentation.  In  the  third  level  of  the  Virtual  Cell,  the  learner  will  be  given  a set  of 
goals  to  investigate  a specific  cellular  structure  or  process.  The  learner  will  have  at  their  disposal  the  tools  from  the 
first  level  and  the  mutations  and  inhibitors  from  the  second  level.  Using  these  in  various  combinations,  the  student 
will  form  hypotheses,  design  experiments,  and  employ  the  toolbox  items  to  perform  these  experiments. 

The  implementation  of  the  Virtual  Cell  depends  on  coordinating  three  technologies:  1)  VRML 
visualization,  2)  a text-based  MOO  server  (Curtis,  1992),  and  3)  Java  client  and  simulation  software.  Students  use  a 
standard  WWW  browser  to  launch  a Java  applet,  which  provides  a connection  to  an  object-oriented,  multi-user 
domain  (a  MOO;  see  below)  where  cellular  processes  are  simulated  and  multi-user  viewpoints  are  synchronized. 
The  Java  applet  also  launches  an  interface  to  the  VRML  representation  of  the  Virtual  Cell,  allowing  the  student  to 
explore  and  experiment  within  the  3D  representation.  The  object-oriented  multi-user  domain  (MOO)  simulation  on 
which  the  cell  is  established  allows  students  to  interact  directly  with  one  another  within  the  cell,  providing  advice  to 
one  another  or  even  working  together  to  achieve  particular  goals.  Because  the  MOO  is  internet-based,  students  from 
separate  and  distant  locations  can  simultaneously  interact  with  the  cell  and  with  one  another.  Unintrusive  but 
proactive  software  agents  act  as  tutors,  monitoring  students'  actions  and  "visiting"  the  students  as  the  need  arises. 
Tutor  agents  provide  advice  on  equipment  choices,  navigation,  and  scientific  conclusions.  Tutors  neither  mandate 
or  insist  on  student  actions  nor  do  they  block  or  prevent  student  actions. 
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Global  Perspectives  in  the  Political  Science  Classroom 
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/' 

In  attempting  to  get  my  students  to  engage  with  the  actual  conditions  of  the  countries  we  study  in  an 
introductory  comparative  politics  course,  I have  created  assignments  based  around  exchanges  between  my 
students  (mostly  from  the  US,  18-23  years  old)  and  students  in  our  case  countries.  The  exchanges  took 
place  over  in  3 different  ways  with  different  levels  of  instructor  participation.  The  first  exchange  was 
between  myself,  my  Comparative  Political  regimes  class  and  Professor  Nicolai  Petrov  who  was  teaching  at 
Novgarod  State  University  in  Russia  and  his  Political  Philosophy  class.  The  second  exchange  was  a one  to 
one  "keypal"  match  between  my  comparative  students  and  individual  students  from  an  ESL  class  in  the 
Ukraine  and  the  third  exchange  took  place  between  my  students  and  an  introductory  class  in  Foreign 
Affairs  taught  by  Greg  Poelzer  at  the  University  of  Northern  British  Columbia,  Canada. 

The  Carleton-Novgarod  Exchange 

I responded  to  a post  on  the  Political  Science  Teaching  e-mail  list  from  Professor  Nicolai  Petro  from  the 
University  of  Rhode  Island  who  was  on  a Fulbright  Teaching  fellowship  at  Novgarod  State  University  in 
Russia.  We  arranged  a class  to  class  exchange  where  students  composed  questions  as  groups,  which  he 
translated  from  English  into  Russian.  The  result  was  a series  of  interesting  question  and  answers  between 
the  students,  which  I posted  onto  our  web  site  along  with  additional  material  about  the  class,  and  pictures  of 
the  students.  Nicolai  Petro  sent  me  scanned  images  of  his  students  and  I put  them  up  on  our  site.  The 
exchange  fit  well  into  our  week  on  Political  Culture  since  1 asked  the  students,  not  only  to  think  of 
questions  but  also  to  reflect  on  the  Novgarod  students’  responses  in  their  "lab  assignment."  The  exchange 
that  we  undertook  with  the  Novgarod  students  was  a fairly  static  affair,  due  to  the  constraints  on  the 
situation  (the  fact  that  the  Novgarod  students  didn’t  have  individual  access  to  the  internet  and  the  need  for 
translation.)  Nevertheless,  the  students  enjoyed  the  interaction  and  I was  even  able  to  use  the  assignment  as 
a learning  tool  the  following  term  when  I had  my  students  evaluate  the  exchange,  focusing  on  the  questions 
that  were  asked,  how  they  were  answered  and  if  there  were  noticeable  differences  in  political  culture.  The 
students  found  that  there  were.  There  were  also  interesting  differences  in  the  way  my  students  perceived 
what  was  going  on  in  Russia  versus  what  the  students  perceptions  were,  living  there.  A great  deal  of 
attention  was  given  to  Ultra-Nationalist  figure  Vladimir  Zhirinovsky  in  the  US,  compared  to  his  actual 
draw  at  the  polls  in  the  1996  elections.  The  Russian  students  felt  that  Zhirinovsky  was  “stupid”  and 
“comical”  and  said  they  were  not  surprised  that  he  lost  the  elections.  The  Russian  students  also  looked  at 
the  effect  of  the  economic  crisis  from  their  own  perspective,  they  commented  on  how  their  instructors  were 
not  receiving  their  paychecks  regularly  and  the  students  were  not  getting  their  stipends  either. 

The  Carleton-Ukraine  Exchange 

After  this  positive  experience  with  an  exchange  I began  to  search  out  other  opportunities  for  exchanges.  A 
matching  service  at  by  St.  Olaf  College  (http://www.stolaf  edu/network/iecc/)  for  teachers  and  students 
seeking  "keypals"  (matching  students  with  e-mail  addresses  for  direct  correspondence)  although  the  process 
of  identifying  and  setting  up  potential  exchanges  proved  to  be  a frustrating  process  with  this  method.  The 
main  reason  I discovered  was  that  most  of  those  interested  in  exchanges  were  high  school  classes,  which 
were  not  at  my  students’  level  or  ESL  classes.  I was  eventually  successful  in  establishing  a "keypal"  link  up 
with  an  ESL  class  at  the  Technological  University  of  Podillia  in  Khmelnitskiy,  Ukraine.  I developed  some 
simple  questions  that  the  students  could  ask  about  living  under  economic  reform  in  the  Ukraine. 

Although  two  of  my  students  reported  positive  and  interesting  interactions  with  the  Ukrainian  students,  the 
rest  of  my  students  complained  that  they  had  been  disappointed  with  the  assignment  on  a number  of  counts. 
Students  that  initiated  the  contacts  late  discovered  that  some  of  the  Ukrainian  students  only  seemed  to 
check  their  e-mail  about  once  a week.  Some  never  seemed  to  check  their  e-mail  at  all.  When  I questioned 
the  teachers  about  this,  they  replied  that  a few  students  had  been  "sick"  a great  deal  or  that  they  really  had 
no  idea  what  the  problem  was  with  the  student.  Such  low  accountability  led  to  an  imbalance  in  the 
exchange  and  except  for  the  two  aforementioned  students,  the  exchange  was  not  a success.  The  University 


of  Podillia  also  apparently  experienced  a massive  failure  with  their  external  internet  connection  late  in  the 
term  which  further  complicated  the  exchange.  In  principle,  an  ESL  class  could  have  worked,  however,  ESL 
teachers  are  notoriously  overworked  and  underpaid  nearly  everywhere  in  the  world,  it  seems  and  thus,  the 
amount  of  effort  that  teachers  can  devote  to  such  a project  is  limited.  Of  the  two  students  who  did  exchange 
e-mail  with  the  Ukrainians,  it  must  be  said  that  the  exchanges  were  extraordinarily  beneficial.  One  student 
was  puzzled  that  her  keypal  didn’t  seem  to  be  suffering  much  from  the  Economic  situation.  After  several 
exchanges,  my  student  learned  that  this  woman’s  parents  were  both  business  people  in  the  new  capitalist 
economy,  and  were  relatively  successful.  From  this,  my  student  learned  that  in  the  economic  reform  in  the 
former  Soviet  Union,  some  people  are  doing  better,  even  though  many  are  doing  a lot  worse  in  the 
immediate  context  and  that  experiences  are  likely  to  be  different  across  society.  The  other  student  who  was 
successful  found  common  ground  with  his  partner  in  that  both  came  from  families  who  had  struggled.  My 
student  was  from  an  Hispanic  neighborhood  in  east  Los  Angeles  and  had  come  to  Caiieton  on  a 
Scholarship. 

The  Carleton-  UNBC  Exchange 

This  last  exchange  was  probably  the  most  effective  in  terms  of  the  quality  and  quantity  of  information 
traded  back  and  forth.  I put  my  request  for  an  exchange  on  the  political  science  research  and  teaching  list 
that  I had  discovered  Nicolai  Petro’s  request  on  and  received  a response  from  a Canadian  political  scientist, 
Greg  Poelzer  who  had  been  forwarded  my  request  by  a colleague.  Greg  took  the  initiative  to  set  up  a 
simple,  threaded  web  based  forum  in  which  both  of  our  classes  could  input  questions  and  responses  on 
topics  that  he  put  up.  We  put  up  political  culture  and  the  Quebec  issue  on  the  board  and  both  got  interesting 
responses.  My  students  were  perplexed  to  learn  that  the  British  Columbia  based  students  could  care  less 
about  Quebec  separatism  and  were  more  interested  in  issues  involving  their  own  province,  such  as  Native 
rights.  From  this  they  learned  about  Canadian  Federalism,  as  a cultural  and  institutional  separateness  that 
already  exists  in  Canada,  quite  apart  from  Quebec’s  bid  to  be  sovereign. 

Some  Concluding  Remarks  and  Suggestions 

Not  all  courses  benefit  from  this  kind  of  exchange,  but  such  interactions  can  be  extremely  worthwhile.  The 
role  of  the  instructor  is  transformed  to  some  extent.  I am  less  of  a gatekeeper  and  more  like  a guide. 
Although  I still  present  the  student  with  information  selected  and  "pre-packaged"  by  me,  with  these 
exchanges  they  are  also  faced  with  "raw  data"  (comments  from  their  "keypals"  or  classes  in  other  countries) 
that  they  must  attempt  to  evaluate.  The  three  different  formats  for  exchanges  provided  acceptable 
communication  conduits,  although  it  was  clear  that  the  human  component  had  to  be  committed  to  insuring 
a quality  exchange.  Exchanges  with  other  political  science  classes  worked  better  since  the  students  were 
already  primed  for  discussing  political  science  topics.  If  greater  attention  were  given  to  creating  specific 
types  of  discussion  exchanges  in  higher  education,  higher  quality  exchanges  involving  complex  topics  and 
cross-cultural  themes  could  result.  There  are  a number  of  ways  that  the  potential  for  the  internet  media 
exchanges  could  be  realized:  These  types  of  exchanges  could  benefit  students  at  various  types  of 
institutions  and  could  be  tailored  to  fit  the  needs  of  both  a small  liberal  arts  college  environment  like 
Carleton,  or  a large  university  setting.  Small  liberal  arts  colleges  focus  on  the  character  building,  rather  than 
training  in  a specific  discipline.  Exchanges  could  allow  the  liberal  arts  student  to  gain  knowledge  from 
external  perspectives.  The  large  university  student  who  typically  has  less  means  and  opportunity  to  travel 
abroad  for  enrichment  can  learn  much  from  a discussion  with  individuals  in  other  countries  that  could  be 
put  to  use  later  professionally  in  the  increasingly  global  work  environment.  Exchanges  based  around  an 
event  might  also  prove  to  an  excellent  way  to  mobilize  teachers  for  multi-lateral  exchanges.  Disciplinary 
Academic  Organizations  could  attempt  to  foster  classroom  exchanges  using  inter-institutional  contacts  that 
they  already  have.  This  would  help  them  fulfill  one  of  their  roles  as  a bridge  between  the  classroom  and  the 
research  environment,  which  all  too  often  remain  distant  from  each  other  in  higher  education. 
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By  way  of  a grant  and  a summer  workshop  on  information  technologies  sponsored  by  the  Innovative 
Technologies  Center  at  the  University  of  Tennessee,  Knoxville,  we  (Cary  Staples,  Art;  Russel  Hirst,  English) 
have  created  a class  that  we  call  “Verbal  + Non. Verbal  Communication.”  It  began  in  January  and  ends  in 
May,  1999.  The  students  are  Graphic  Design  majors  and  English  majors--about  half  and  half. 

The  purpose  of  the  class  is  to  provide  students  with  a highly  collaborative  and  technologically  enhanced 
classroom  setting  wherein  they  can  learn  how  to  combine  textual  and  visual  elements  for  effective 
communication  in  business  and  industry.  A special  concern  is  that  communications  be  as  “global”  as  possible- 
-that  is,  easily  understood  by,  or  easily  translatable  to,  other  cultures  and  countries. 

We  have  approached  this  rather  ambitious  goal  by  way  of  three  phases.  In  the  first  phase,  the  English  majors 
learned  something  about  the  technological  tools  at  their  disposal  (Quark  Express,  PhotoShop,  etc.)  and  about 
basic  principles  of  visual  design.  Design  majors  got  some  review  on  principles  of  mechanics  and  style.  Both 
sets  of  majors  were  encouraged  to  draw  upon  each  other’s  expertise  in  the  months  ahead.  In  the  first 
assignment,  writer/designer  teams  created  business  cards— at  this  point,  with  no  particular  attention  to 
international  communication. 

Phase  two  saw  us  ushering  in  the  international  theme  by  way  of  an  exercise  that  required  students  to  “create  a 
new  country”  by  merging  the  characteristics  of  two  real  countries.  The  idea  was  to  define  and  advertise  the 
new  country  so  as  to  attract  tourism.  Some  students  chose  to  create  web  sites  to  satisfy  the  assignment;  others 
produced  hard  copy.  Students  created  countries  such  as  “The  Enlightened  People’s  Republic”  (Egypt  and 
India)  and  “Tai won-on”  (pronounced  “tie  one  on,”  a country  bom  of  a combination  of  characteristics  evidently 
inspired  by  Tai  won  and  Lebanon— a haven  for  those  who  love  to  party).  We  asked  students  to  critique  each 
other’s  productions  as  we  ourselves  were  doing  for  them.  At  the  same  time,  we  began  to  ask  our  students  for 
contributions  to  the  class  web  site.  For  example,  each  student  was  asked  to  submit  three  annotated  URLs  of 
sites  that  they  consider  valuable  for  communicators. 

Phase  three  has  focused  on  documents  for  business  and  industry.  The  first  assignment  in  this  phase  was  to 
create  a set  of  instructions.  Some  students  produced  instructions  with  direct  business/industry  application,  but 
others  were  quite  playful,  coming  up  with  instructions  such  as  “how  to  juggle,”  “how  to  put  on  a pair  of 
pants,”  and  “how  to  avoid  conversion  into  fanatical  religious  organizations.” 

Our  final  assignment,  however,  has  left  no  room  for  anything  but  direct  application  to  business  and  industry 
on  an  international  level.  Though  our  original  idea  was  to  have  students  work  as  writer/designer  teams  to 
develop  a series  of  warning  labels  and  instructions,  we  actually  have  them  doing  something  even  more 
sophisticated  than  this:  they  are,  in  their  hypothetical  roles  as  industry  consultants  or  in-house  technical 
communication  managers  and  designers,  creating  policy-defining  handbooks  or  web  sites  that  show  fellow 
communicators  how  best  to  write  and  design  warning  labels  and  instructions  for  their  company’s  products. 
This  exercise  requires  them  to  define  company  communication  goals  and  policies  and  to  create  individual 
warning  labels  and  instruction  “posters”  by  way  of  illustration.  Their  concerns  are  to  communicate  clearly,  to 
create  the  best  possible  marriage  of  text  and  graphics,  to  communicate  so  as  to  avoid/reduce  legal  difficulties, 
and  to  be  culturally  sensitive  and  intelligent.  They  must  also  coach  their  fellow  communicators  about  how  to 
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create  warnings  and  instructions  that  will  avoid  or  reduce  litigation  for  their  companies.  We  invited  a 
colleague  who  has  doctorates  in  both  linguistics  and  law  to  speak  with  our  students  on  this  theme. 

This  current,  and  last,  assignment  involves  much  research  into  international  communication.  Once  again,  we 
have  asked  students  to  share  their  research  with  their  colleagues.  For  example:  each  student  has  sent  us 
information  about  a particular  country  or  culture  that  could  help  communicators  to  communicate  well  in  that 
country  or  culture.  This  involves  information  about  politics,  race,  history,  gender  issues,  etc.  as  well  as  special 
meanings  of  color,  gestures,  symbols,  and  so  on.  All  this  information  has  gone  onto  the  class  web  page  for  use 
by  all  the  students. 

Our  next  guest  speaker,  the  president  of  a company  that  makes  warnings  labels  for  international  companies, 
will  provide  an  important  “reality  check”  to  students—though  we  have  deliberately  delayed  his  arrival  until  late 
in  the  semester  lest  the  creativity  of  students  be  curtailed  in  any  degree!  Students  will  get  feedback  on  their 
work  not  only  from  our  speakers  and  from  us  and  from  their  fellow  students,  but  from  professionals  abroad. 
These  professionals  will  use  our  class  web  site  to  evaluate  our  students’  work.  This  evaluation  by 
internationals  will  be  very  important,  since  students  need  to  know  whether  their  products  really  do 
communicate  effectively  to  a variety  of  cultures. 

Each  student  team  will  compose  a “brief’  for  these  international  reviewers— that  is,  an  explanation  about  what 
they  are  trying  to  accomplish  in  their  communication.  However,  the  reviewers  will  post  their  initial  reaction  to 
the  work  generated  by  the  students  before  they  have  access  to  the  briefs.  Later,  after  reading  the  briefs,  the 
reviewers  will  be  able  to  post  additional  comments.  Students  can  then  make  modifications  based  on  both  sets 
of  feedback. 

Our  hope  is  to  initiate  an  ongoing  dialogue  between  educational  institutions  and  industries  in  developing  clear, 
understandable  warnings,  instructions,  and  other  documents.  We  want  to  teach  our  next  course  to  students  at 
a number  of  universities  via  distance  learning  technologies.  The  hope  is  that  the  later  classes  will  provide  a 
more  diverse  pool  of  images  and  terminology  and  will  allow  the  discussion  to  progress  further. 

To  summarize— our  class  has  several  purposes: 

•To  offer  insight  into  the  complexity  of  global  communication  problems. 

•To  allow  students  to  work  in  multi-disciplinary  teams  without  head-to-head  competition. 

•To  create  an  a technologically  enhanced,  non-traditional  classroom. 

•To  experiment  with,  and  review,  communication  efforts. 

•To  create  an  environment  that  will  live  beyond  the  framework  of  a traditional  semester  and  become  an 
ongoing  resource  for  education  and  industry. 

Previous  classes  that  used  a web  site  as  a class  outline,  and  that  used  student  research  to  complete  the  site, 
have  been  very  successful.  This  is  our  first  attempt  to  create  a site  that  is  both  cross  disciplinary  and 
international.  We  have  been  helped  tremendously  in  the  creation  of  this  site  by  the  superb  staff  of  UTK’s 
SunSITE  (http://www.sunsite.utk.edu).  The  website  we’ve  created  in  collaboration  with  them  is  at 
http://sunsite.utk.edu/neighborhoods/verbal. 

During  our  presentation,  we  will  give  you  a tour  of  our  web  site  and  display  and  discuss  examples  of  student 
work  and  feedback  they  received. 
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The  Context 

Open  School,  a service  of  the  Open  Learning  Agency,  develops  Kindergarten  to  Grade  12  distance 
education  courses  and  resources  for  approximately  48,000  students  located  throughout  British  Columbia  and 
Canada.  These  courses  have  traditionally  been  print-based,  supplemented  with  some  additional  media  (e.g.,  audio, 
video),  and  delivered  to  students  via  the  postal  service.  More  recently,  however.  Open  School  has  been  attempting  to 
develop  a new  distributed  learning  environment  which  would  allow  for  more  flexible  design  and  distribution  of 
course  materials  to  meet  the  various  and  ever  expanding  needs  of  our  learners.  Features  of  this  new  environment 
include  a learner-centred  approach  to  development  of  materials,  electronic  communications  between  students  and 
instructors,  a modular  course  development  process  based  on  structured  information,  choice  and  flexibility  of 
different  delivery  formats,  a resource-rich  environment  for  students,  different  media  being  used  in  parallel,  and 
synchronous  and  asynchronous  interaction  between  students  and  instructors. 


Using  SGML  as  an  Underlying  Framework 

The  framework  underlying  Open  School's  new  distributed  learning  environment  is  a technology  called 
SGML  (Standard  Generalized  Mark-up  Language).  SGML  is  an  international  standard  (ISO  8879)  published  in  1986 
and  has  been  used  extensively  for  producing  documentation  within  high  technology  and  publishing  industries.  It  is 
an  approach  used  in  the  design  and  creation  of  documents  where  the  focus  is  on  specifying  a standardized  approach 
to  identifying  the  components  of  a document  and  how  they  relate  to  each  other.  Documents  are  independent  of  any 
specific  hardware  or  software,  so  exchange  of  files  between  users  who  have  different  SGML  systems  is  quite 
seamless.  SGML  allows  for  the  separation  of  a document  into  three  layers:  structure,  content,  and  style — but  deals 
mainly  with  the  relationship  between  structure  and  content.  Structure,  which  is  at  the  core  of  SGML,  is  embodied  in 
the  Document  Type  Definition  or  DTD.  Content  is  the  information  itself  such  as  titles,  paragraphs,  lists,  tables, 
graphics,  and  audio.  SGML  does  not  try  to  store  information  about  the  way  a document  looks — instead  it  tries  to 
describe  a document  according  to  its  content  (Arbortext  1999).  The  separation  of  structure  and  content  of  a 
document  from  the  appearance  or  style,  enables  a variety  of  instructional  media  to  be  produced  using  the  same 
source  material;  for  example  one  can  simultaneously  produce  a print  instance  with  italicized  words  and  an  on-line 
version  of  the  same  content  with  hypertext  link  to  a glossary. 


SGML  was  selected  as  the  foundational  framework  for  Open  School's  new  distributed  learning  environment  because 
it  can  effectively: 

• provide  a structured  approach  to  the  development  of  course  materials, 

• enable  an  easy  customization  of  courses  for  different  student  needs. 
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• facilitate  the  re-purposing  of  content, 

• provide  flexibility  beyond  print-based  publishing  (e.g.,  multiple  formats — such  as  print,  WWW,  CD  ROM). 

The  intent  is  that  all  of  these  features  will  help  support  the  development  and  sustainability  of  the  Open  School  distributed 
learning  environment. 


Instructional  Design  within  a Structured  Information  Framework 

instructional  design  is  an  eclectic  integration  of  principles  and  procedures  from  many  theoretical  fields  historically 
based  on  hard  sciences  and  social  sciences.  At  the  heart  of  it,  however,  is  the  design  and  development  of  learning  experiences 
to  support  learners’  success.  Open  School’s  instructional  design  process,  as  based  on  structured  information  and  SGML, 
consists  of  four  main  stages  of  development;  Stage  1 learning  outcomes/content  outline,  Stage  2 resources,  Stage  3 
activities/assessment,  and  Stage  4 authored  course  [see  Figure  1],  The  first  three  stages  of  Open  School’s  development 
process  represents  core  features  of  instructional  design  which  include  extensive  planning  based  on  learning  outcomes  and 
learner  profiles  (Stage  1).  The  result  of  Stages  1 to  3 is  a very  structured  and  detailed  plan  for  Stage  4 that  comprises  of  a 
fully  written  and  fleshed  out  course.  This  process  allows  for  detailed  planning  of  content  to  be  delivered  using  different 
media  (e.g.,  print  and  the  Web)  while  allowing  for  flexibility  and  customization  in  order  to  the  needs  of  the  learners. 


Figure  1.  Open  School’s  Development  Process 

The  implementation  of  a structured  framework  based  on  SGML  has  prompted  a closer  examination  of  instructional 
design  and  specifically  instructional  design  plans  (e.g.,  Stages  1 to  3).  The  implementation  of  a structured  framework  has 
explicitly  enforced  a consistent  way  of  describing  learner  profiles,  learning  outcomes,  creating  subject/topic  associations, 
suggesting  instructional  strategies,  and  assessment  strategies  (e.g.,  via  instructional  design  document  type  definition — DTDs 
).  Structure  within  varies  courses  is  consistent,  but  the  content  will  vary  depending  on  the  outcomes  and  subject  matter.  Each 
instructional  design  plan  becomes  a foundation  for  all  subsequent  development.  Learning  outcomes,  content,  activities, 
assessment,  and  resources  are  attached  directly  to  the  plan,  incrementally  growing  the  resulting  document  set.  A “course” 
ultimately  becomes  the  complete  set  of  documents  surrounding  a given  instructional  design  plan.  The  result  is  a manageable 
base  of  course  materials.  This  entire  process  is  still  in  a prototype  stage  within  Open  School.  The  test  bed  for  implementing 
SGML  is  presently  on  going  with  the  development  of  thirteen  high  school  level  courses  within  the  structured  information 
model.  These  thirteen  courses  will  be  delivered  entirely  in  print  and  on  the  Web  in  the  near  future. 

Reference 

Arbotext  (1999)  Web  site 

ww\v.arbortext,com/Think_Tank/SGML_Resources/Getting_Started_with__SGML/getting_started_with_sgml.html 
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Background: 

In  Polytechnic  curriculum,  equal  emphasis  is  placed  on  both  practical  and  conceptuaf  aspects  of  a subject.  With 
regard  to  the  conceptual  aspects,  better  results  are  realised  when  practical  demonstrations  are  used  in  the 
facilitation  process.  This  paper  illustrates  the  use  of  several  simple  gadgets  in  the  classroom  to  help  the  students  of 
Business  Computing  (National  Diploma  in  Business  Computing)  get  a better  feel  for  complexities  of  the  modern- 
day  computer  data  communications.  The  students  in  this  group  have  no  engineering  background,  even  though  they 
have  been  exposed  to  the  basic  layers  of  OSI  model,  the  operation  of  asynchronous  communication  link,  use  Linux 
for  networking,  and  an  overview  of  network  management.  Without  the  understanding  of  the  engineering  aspects  of 
data  communications  and  networks,  these  students  are  handicapped  in  appreciating  the  complexities  of  modem 
computer  networks  using  Fibre  Distributed  Data  Interface,  Asynchronous  Transfer  Mode,  etc,.  Further,  they  fail  to 
get  a feel  for  the  benefits  - such  as  higher  signal-to-noise  ratio,  larger  bandwidth,  low  channel  noise,  data 
transportation  at  a constant  rate,  etc.  - offered  by  these  multi-service  wide-band  networks. 

Possible  Facilitation  Enhancements: 

Normally,  analytical  models  and  theoretical  explanations  are  used  as  part  of  the  facilitation  process,  when  dealing 
with  such  topics.  However,  such  techniques  are  not  very  appealing  to  the  students  who  have  no  exposure  to  the 
Communication  Theory  and  as  such  they  soon  lose  interest  and  motivation.  In  some  institutions  with  on-going 
engineering  curriculum,  one  might  consider  a visit  to  the  communications  engineering  lab,,  where  some  of  these 
expensive  high  technology  equipment  and  tools  are  available.  However,  the  authors  do  not  have  access  to  such 
facilities  and  hence  have  resorted  to  the  use  of  simple  commonly  available  electronic  gadgets  to  demonstrate  these 
concepts.  In  the  following,  an  illustration  is  provided  which  uses  a Laser  Pointer  (used  with  Over  Head 
Projections),  a couple  of  AM/FM  radios.  Headphones  and  a Light  Dependent  Resistor  (LDR)  coupled  to  a pre- 
amplifier. 

The  schematic  in  Fig.  1 shows  the  actual  arrangement  of  these  gadgets  for  demonstrating  some  of  the  key  concepts 
in  Data  Communications.  As  pointed  out  earlier,  the  objective  of  the  Data  Communications  Demonstrator  (DCD) 
is  to  provide  a feel  for  the  following  practical  aspects  relating  to  Data  Communications: 

• Laser  beam  as  a data  carrier 

• Wireless  point-to-point  communication 

• Modulation/demodulation  of  a carrier  using  a signal 

• Noise  pick-up  by  the  communication  channel  and  signal-to-noise  ratio 

• The  effect  of  bandwidth  limitation. 

Referring  to  Fig.  1 , the  headphones  with  a flat  polished  metallic  diaphragm  work  as  a reflector  for  the  laser  beam, 
(The  front  side  of  one  earpiece  of  a common  stereo  headphones  is  chopped-off,  to  expose  the  metallic  diaphragm.) 
The  Laser  Pointer  is  positioned  in  such  a way,  that  the  narrow  light  beam  is  reflected  (at  about  90  deg.)  from  the 
exposed  diaphragm  of  earpiece.  The  headphones’  jack  is  connected  to  one  of  the  AM/FM  radios,  and  the  radio 
tuned  to  a FM  channel  playing  music.  The  musical  note  causes  the  diaphragm  to  vibrate  and  thus  Amplitude 
Modulate  the  reflected  Laser  beam.  The  LDR,  functioning  as  a demodulator,  reproduces  the  electrical  signal 
corresponding  to  the  original  audio  signal.  This  received  signal  is  processed  through  a preamplifier  to  prevent  the 
loading  of  demodulator 
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Fig. - 1,  Schematic  of  the  Data  Communication  Demonstrator 


and  fed  to  the  audio  section  of  the  second  AM/FM  radio  for  reproducing  the  musical  notes.  The  circuit  schematic 
of  the  pre-amplifier  is  shown  in  Fig.  2.  This  is  constructed  with  two  LM324  type  operational  amplifiers  with  an 
overall  voltage  gain  of  1 0.  The  power  for  this  circuit  is  derived  from  the  second  AM/FM  radio. 
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Fig.  - 2,  Circuit  schematic  of  the  Pre-amplifier 

The  following  is  a description  of  how  the  DCD  setup  is  used  to  demonstrate  the  key  practical  aspects  of  computer 

data  communication  - 

• A laser  beam  is  used  as  a data  carrier.  At  the  headphones,  the  electrical  signal  (corresponding  to  the  music) 
modulates  the  carrier.  The  LDR  demodulates  the  carrier  to  extract  back  the  signal. 

• The  data  is  transported  with  a wireless  (light)  link.  As  the  light  can  not  bend,  the  transmitter  and  receiver 
must  have  line  of  sight.  This  arrangement  thus  illustrates  point-to-point  communications. 

• The  florescent  room  lighting  is  a source  of  noise  for  this  set  up.  For  a fixed  signal  strength  (i.e.,  keeping  the 
music  volume  constant),  switching  off  the  room  light  improves  the  quality  of  music  signal  received.  By 
varying  the  music  volume  (signal  strength)  the  significance  of  signaUto-noise  ratio  is  demonstrated. 

• The  environmental  noise  pick-up  is  demonstrated  by  moving  a clear  plastic  film  (even  reading  glasses)  in  the 
path  of  the  laser  beam.  The  minute  scratches  / impurities  in  the  clear  film  produce  a ‘scratching’  noise  in  the 
received  audio  signal. 

• Displacing  the  laser  beam  away  from  the  LDR  has  the  effect  of  reduced  bandwidth,  which  results  in  a poorer 
quality  of  the  received  signal. 


Conclusion: 

Following  the  practical  demonstration  using  DCD,  the  data  communication  concepts  were  discussed  in  the  class. 
After  session,  the  participants  were  given  a questionnaire  to  evaluate  the  usefulness  of  the  demo.  In  their  response, 
80%  of  the  students  considered  that  the  DCD  demo  had  contributed  significantly  to  their  learning,  compared  to  the 
theoretical  discussion.  Some  even  indicated  that  they  would  like  to  perform  these  experiments  themselves.  On  the 
whole,  50%  rated  good  (and  30%  average)  understanding  of  the  data  communications  concepts  contained  in  the 
DCD  demo.  Specifically,  the  aspect  on  wireless  was  appealing  to  about  80%,  media  / noise  aspect  to  about  60%, 
modulation/demodulation  to  50%  and  signal-to-noise  ratio  to  only  30%. 
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It  is  difficult  to  teach  people  to  be  good  online  courseware  developers  if  they  don't  know  what  it  is  like  to  be  on 
the  receiving  end  of  their  designs  - either  as  a student  or  an  instructor.  In  a graduate-level  course  on  Designing 
Web-based  Instruction  for  Distance  Education,  students  are  typically  involved  in  a number  of  design  and 
development  projects,  in  which  they  function  as  instructional  designers  hired  to  build  a Web-based  lesson, 
module,  or  course  for  a distance  learning  audience.  In  the  past,  students  would  excel  in  (1)  Web  site  and  page 
layout  and  functionality  and  (2)  the  distribution  of  content  and  activity/project  descriptions,  but  would  fail  to 
address  issues  of  social  interaction,  student  collaboration,  student  motivation  to  return  to  the  Web  site  over  and 
over,  etc.  In  fact,  it  was  difficult  for  them  to  understand  why  focusing  on  Web  teaching  and  learning  strategies 
was  necessary  for  them  in  their  roles  as  "WBI  designers".  Unfortunately,  the  class  did  not  have  access  to  pre- 
existing Web-based  courses  that  they  could  be  required  to  facilitate  in  order  to  gain  experience  in  Web  teaching 
and  learning.  Therefore,  students  are  required  to  host  a virtual  conference.  By  hosting  the  virtual  conference, 
students  are  exposed  to  the  triumphs  and  frustrations  of  creating  an  interactive  learning  experience  delivered  via 
the  Web. 

Thus  far,  two  Virtual  Conferences  have  been  hosted  by  two  different  groups  of  students:  one  in  Fall  1997  and 
one  in  Fall  1998  (the  3"^  Annual  Virtual  Conference  will  be  held  in  Fall  1999).  Comparing  students'  work  prior 
to  and  after  the  Virtual  Conference  assignment  revealed  that  WBI  students/designers: 

• moved  from  creating  professional-looking  "page  turning"  Web  sites  that  simply  delivered  textual/graphical 
content  to  developing  interactive,  student-centered  activities  (such  as  games,  case  studies,  and  problem- 
based  learning  activities)  for  learners  to  work  through;  and 

• utilized  communication  technologies  differently:  instead  of  simply  posting  discussion  questions  on  an 
asynchronous  threaded  discussion  forum  for  learners  to  respond  to,  both  asynchronous  and  synchronous 
communication  technologies  (threaded  discussions,  reflective  Joumals/forms,  chat  and  whiteboard)  were 
being  used  for  debates,  collaborative  project/product  development,  peer  teaching,  and  presentations. 

The  Virtual  Conference  assignment  has  been  so  effective  as  a learning  experience  for  WBI  designers/students  — 
as  well  as  providing  a good  learning  experience  for  the  virtual  conference  participants  --  that  the  Fall  Virtual 
Conference  is  now  an  on-going  staple  of  the  Distance  Education  track  of  the  Instructional  Technology  graduate 
program,  with  plans  to  host  a virtual  conference  every  Fall. 


The  Virtual  Conference  Assignment 

For  the  Virtual  Conference  assignment,  students  collaboratively  develop  and  deliver  an  online/virtual 
professional  conference  on  various  aspects  of  designing  distance  education  for  the  WWW.  Besides  the  actual 
conference,  documentation  (1)  describing  the  strategies  and  tactics  used  to  promote,  organize,  facilitate, 
evaluate,  etc.  the  conference  and  (2)  supporting  the  design  decisions  made  based  on  front-end  analysis  data  is 
required.  Results  of  the  conference  are  debriefed  in  class,  and  students  are  required  to  reflect  on  the  overall 
experience  and  on  what  they  learned  from  the  process;  their  reflection  was  structured  by  an  online  assessment 
(see  http://www.cudenver.edu/Hdunlap/5990/conferenceioumal.html  they  helped  to  develop  (see  an  example  of 
a student's  evaluation  of  the  virtual  conference  experience  at  http://clem.mscd.edu/^woodlevx/wbi/online.htm). 
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Description  of  Student  Activities 

The  objective  of  this  project  is  for  students  to  collaboratively  develop  and  deliver  an  online/virtual  professional 

conference  covering  issues  related  to  designing  distance  education  on  the  WWW.  To  make  this  happen 

students  have  to  take  responsibility  for  a number  of  activities  and  deliverables: 

• As  a class,  students  have  to  select  five  topics  that  represented  important  issues  of  Web-based  distance 
education.  They  then  have  to  determine  who  will  be  responsible  for  which  topic/s.  Responsibility  for 
topics  requires  students  to  research,  prepare  (includes,  but  is  not  limited  to,  developing  a Web-based 
presentation  and/or  paper),  manage,  and  facilitate  the  portion  of  the  virtual  conference  related  to  those 
topics.  The  conference  runs  for  four  days  during  each  Fa!!  semester;  topics,  activities,  and  facilitators  have 
to  be  scheduled  by  the  students  for  each  day  of  the  virtual  conference. 

• Asa  class,  students  have  to  decide  what  Web-based  communication  technologies  are  needed  to  conduct  the 
online  conference  and  create  or  contract  those  technologies. 

• Students  need  to  determine  what  strategies  and  tactics  should  be  employed  to  ensure  conference  attendee 
participation  throughout  the  duration  of  the  conference.  This  includes  lining  up  "guest  speakers"  and/or 
developing  presentations,  debates,  case  studies,  and  other  engaging  activities  that  will  keep  conference 
participants  involved. 

• Students  also  need  to  develop  an  evaluation  tool  so  participants  can  let  them  know  what  they  think. 
Students  must  analyze  the  evaluation  findings  to  determine  "lessons  learned". 

• Students  develop  and  upload  the  conference  site. 

• Students  promote  the  conference  to  classes,  students,  and  faculty  in  the  School  of  Education  and  any  others 
deemed  appropriate,  and  to  outside  educational  and  professional  groups. 

• Students  are  responsible  for  providing  technical  support  throughout  the  conference. 

• Students  document  in  writing  the  decisions  they  make  on  the  above  issues. 


1**  and  2”*^  Annual  Virtual  Conferences:  The  Students^  Conference  Sites 

There  have  been  two  virtual  conferences  hosted  by  graduate  students  in  the  Designing  Web-based  Instruction 
for  Distance  Education  course: 

1^*  Annual  Virtual  Conference:  Instructional  Strategies  and  Delivery  Issues  of  Web-Based  Instruction  - 
http://carbon.cudenver.edu/^idunlap/seminar/ 

2"^^  Annual  Virtual  Conference:  Beyond  Page  Turning  and  Lecture  Techniques:  Effective  Instructional 
Strategies  for  Web-based  Learning  Environments  - http://web-education.net/fall98conf/ 

[Note:  For  notification  of  future  Virtual  Conferences,  see 
http://ceo.cudenver.edu/Honi  Dunlap/A ACE/virtual examples.html  ] 


A Few  Examples  of  Students*  WBI  Projects  after  Hosting  a Virtual  Conference 

Web  Design  for  K-12  Teachers  — http://carbon.cudenver.edu/-lbatzel/Web  Design/home. htm 

Ethical  Decision-making  in  Business  --  http://clem.mscd.edu/-woodleyx/wbi/Content/ethics/index.htm 

Introduction  to  the  Internet  --  http://carbon.cudenver.edu/-slancton/frame.html 

Creating  Geographic  Cross  Sections  — http://www.cudenver.edu/-abol/Cross  Section/home. html 

Teaching  with  the  Web:  An  On-Line  Forum  for  K-12  Teachers  - http://bvsd.kl2.co.us/-mullerb/teaching/forum/ 

Case  Studies  in  Research  Ethics  — http://carbon.cudenver.edu/-mstephen/bioethics/homepage.html 
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Introduction 

The  rapid  advancement  of  the  World  Wide  Web  and  hypermedia  systems  puts  educators  more  and  more  under 
pressure  to  provide  some  of  their  course  material  on  the  web.  But  given  the  time  constraints  of  the  faculty  and 
the  necessary  skills  in  electronic  media  production,  it  cannot  be  expected  that  many  useful  systems  will  be  built 
and  used  by  them.  Most  available  systems  support  a great  deal  of  activity  that  goes  into  the  development  of  an 
electronically  enhanced  course  with  features  which  support  testing,  interfacing  with  other  university-based 
systems,  tracking  of  students,  assignments,  chat  rooms  and  some  instruction.  However,  explicit  support  for  the 
development  of  a web-based  environment  for  learning  is  rare  at  best.  One  of  the  exceptions  is  an  adaptive 
hypermedia  system  provide  navigation  support  that  adapts  to  the  user’s  action  and  learning  (Brusilowski  et  al., 
1997).  However,  the  provided  navigation  space  tends  to  constrain  the  learner’s  possibilities  for  exploration. 

Our  system  attempts  to  provide  hypermedia  sites  that  minimize  such  constraints  on  exploration,  yet  still 
providing  a flexible  site  structure  optimized  for  the  used  teaching  method.  A possibly  radically  different  site 
structure  is  necessary  to  support  various  teaching  methodologies.  But  who  creates  all  the  web  pages  for  the 
courses  and  keeps  them  also  up-to-date?  And  how  do  we  make  sure  that  the  students  exploring  the  site  don’t  get 
lost  and  frustrated  instead  of  learn  from  them? 

Reuse  and  Exploration 

Once  a reasonable  fraction  of  a course’s  content  is  put  on  the  web,  the  site  is  rather  difficult  to  adapt  to  new 
situations  due  to  the  static  structure  of  web  sites.  Thus,  the  reusability  of  web  sites  is  minimal.  The  organization 
of  the  pages,  navigation  support  and  links  to  related  issues  or  prerequisites  are  static  and  optimized  for  this 
course’s  topic,  time  frame,  emphasis,  and  the  educator’s  teaching  style.  However,  the  course  should  be  reusable 
in  many  different  ways:  when  moving  from  the  quarter  system  to  the  semester  system;  when  the  senior-level 
course  is  upgraded  to  a graduate-level  course;  when  the  focus  of  the  course  changes;  or  when  a related  course  is 
designed.  If  a different  faculty  member  teaches  the  same  course  it  probably  needs  to  be  restructured  due  to  a 
different  teaching  style  and  other  preferences.  Thus,  providing  the  same  course  material  for  everybody  and 
every  situation  is  in  general  not  feasible  because  it  does  not  support  what  the  educator  and  the  students  need. 
However,  changing  a web  site  to  accommodate  another  teacher  can  cause  a major  reorganization  of  the  web 
site,  which,  in  general,  is  not  feasible  either.  Current  tools  do  not  support  reuse  in  any  interesting  way. 

The  second  problem  addressed  by  our  approach  is  to  provide  a hypermedia  system  that  provides  a large 
exploration  space.  All  those  and  only  those  paths  that  are  consistent  with  the  pedagogical  method  used  by  the 
educator  should  be  available  to  the  learner.  The  user  is  supported  in  selecting  paths  consistent  with  the 
pedagogical  method  by  the  organization  of  the  site  and  by  dynamic  navigation  support. 

Our  approach  is  based  on  the  observation  that  the  order  in  which  content  is  taught  depends  on  both,  the  content 
and  the  teaching  methodology  employed.  Whereas  most  other  tools  have  a “correcf’  teaching  methodology 
built-in,  our  tool  provides  a simple  mechanism  to  describe  such  methodologies. 

The  educator  uses  the  tool  as  follows.  First,  she  represents  the  content  independent  of  what  teaching 
methodology  she  will  use.  Then,  the  teaching  methodology  is  formalized  in  such  a way  that  it  can  be  applied  to 
any  content  resulting  in  an  organization  of  this  content  supporting  the  selected  teaching  methodology.  The 
content  is  represented  in  a semi-formal  using  concept  maps.  The  claim  that  teaching  methodologies  can  be 
formalized  may  come  as  a surprise.  This  paper  will  describe  in  some  detail  how  we  formalize  those  (only)  parts 
of  a teaching  methodology  relevant  for  the  content  organization.  It  is  an  empirical  question  whether  the 
resulting  web  sites  are  indeed  educationally  well  organized. 
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Content  Description 

The  course  content  is  described  independent  of  how  it  will  be  organized  in  the  web  site,  allowing  the  user  to 
concentrate  on  content  only.  We  have  chosen  a semi-formal  representation  that  many  educators  are  quite 
familiar  with.  We  use  a hierarchical  concept  map  containing  descriptions  of  content  units  like  concepts,  skills, 
activities,  etc.  and  their  relationships  as  a graph.  The  interface  allows  an  incremental  development  of  the  content 
without  having  to  deal  with  the  constraints  of  the  course  and  the  teaching  methodology  employed.  Furthermore, 
content  units  can  also  be  assembled  from  other  content  units,  i.e.,  content  units  range  from  a single  idea  to 
topics  to  courses  and  even  to  whole  curricula.  These  units  can  be  related  to  each  other  with  relations  like  “a  is 
prerequisite  of  fe”  or  “a  is  part  of  b*\  We  are  using  semantic  zooming  and  fractal  display  techniques  to  deal  with 
the  potentially  large  concept  maps  ( Furnas  & Beueisen,  1995). 

Description  of  Organization 

The  teaching  methodology  is  represented  in  terms  of  rules  that  generate  the  temporal  organization  for  the  course 
content.  There  is  more  to  a pedagogical  methodology  than  ordering  the  content,  however,  organizing  the  content 
for  a course  can  be  viewed,  slightly  simplifying,  as  providing  all  the  educationally  appropriate  sequences  in 
which  the  course  material  may  be  traversed.  Most  often,  when  a faculty  prepares  a course,  he  or  she  chooses 
one  of  the  many  sequences  that  seem  to  be  adequate. 

Instead  of  providing  a number  of  built-in  methodologies,  the  user  can  describe  a methodology  reflecting  her 
own  teaching.  This  description  can  then  be  applied  to  any  content.  The  methodologies  are  described  with  simple 
if-then  rules.  In  a later  phase,  we  will  provide  a more  appropriate  interface  to  the  end  user.  Although  we  can 
only  express  the  temporal  relationships  before,  after  and  during  we  believe,  based  on  some  informal  tests  with  a 
prototype,  that  we  can  describe  a large  part  of  interesting  relationships  for  content  organization.  For  instance,  if 
a X and  y are  two  concepts,  then  the  rule  “if  x is  a prerequisite  of  y then  x is  before  y”  says  that  prerequisites 
must  be  visited  before  what  they  are  prerequisite  for.  Another  rule  could  rule  say  that  related  issues  should  be 
visited  parallel  to  the  main  topic  “ifx  is  strongly  related  toy  then  x is  during y”. 

These  rules  transform  the  concepts  into  an  and-or  graph  that  can  be  traversed  in  any  order  that  is  consistent  with 
the  rules  selected.  This  organization  is  then  transformed  into  web  site  and  navigation  support  in  form  of  an 
applet  makes  sure  that  exactly  those  links  are  accessible  consistent  with  the  selected  methodology.  In  this  way 
we  maximize  the  space  the  learner  can  effectively  explore.  For  instance,  given  above  rules,  the  student  can  look 
at  a part-whole  hierarchy  moving  from  the  whole  to  the  parts. 

Conclusions 

A prototype  is  currently  being  implemented  as  a Java  applet  with  a visual  interface  that  can  deal  with  large 
concept  maps  to  describe  the  content.  We  have  developed  an  efficient  graph  algorithm  that  accomplishes  the 
guidance  of  the  exploring  student.  Finally,  the  resulting  organization  is  mapped  to  a web  site.  This  part  will  take 
advantage  of  work  on  multimedia  roles  (Recker  & Ram,  1994).  Thus,  the  educator  is  required  to  deal  with 
content  and  pedagogical  issues,  but  not  with  HTML,  CGI,  and  other  details  irrelevant  for  him. 

Several  important  questions  need  to  be  answered  empirically.  First,  can  we  represent  large  and  complicated 
concept  maps  so  that  educators  can  use  them  effectively?  Second  and  more  central  to  the  topic  of  this  paper,  can 
all  interesting  pedagogical  methodologies  be  expressed  with  our  simple,  rule-based  language?  Third  and  most 
importantly,  is  the  resulting  organization  of  the  content  indeed  supporting  these  methodologies? 
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ResearchAid,  an  interactive,  dynamic  hypertext  system,  is  being  developed  to  help  students  become 
members  of  a working  scientific  research  community.  The  system  has  students  editing,  creating  and  researching 
content  using  a Web  browser.  ResearchAid  strives  to  make  students  become  active  members  in  a scientific  research 
community  through  aiding  the  following  activities:  it  helps  make  explicit  and  available  the  goals  of  a research 
community;  it  helps  provide  a structure  for  learning  and  using  the  communities'  discourse  and  reasoning  strategies; 
and  it  helps  make  explicit  some  of  the  distributed  knowledge  within  that  community. 

ResearchAid  started  with  the  goal  of  having  students  learn  to  operate  a complex  machine,  a meniscus 
coater,  in  a manufacturing  environment  from  a computer  system.  However,  a contextual  inquiry  into  a 
manufacturing  research  environment  at  Georgia  Institute  of  Technology  found  that  the  way  students  learn  to  operate 
the  machine,  a one-on-one  apprenticeship  with  a graduate  student,  is  entwined  with  their  learning  other  practices  in 
the  research  laboratory.  Through  this  apprenticeship,  students  get  to  know  other  members  of  the  research  community 
and  receive  advice  on  different  aspects  of  their  project.  Having  these  students  learn  how  to  operate  the  machinery 
from  a computer  tutorial  system  may  prevent  them  from  learning  the  other  practices  and  especially  from  learning 
how  the  practices  are  related. 

It  was  also  realized  during  the  contextual  inquiry  that  what  did  need  support  was  a different  learning 
activity  — the  planning,  running  and  reporting  of  independent  research  by  undergraduate  students.  Undergraduate 
students  were  told  that  they  would  be  doing  independent  research  projects  in  the  laboratory.  However,  they  were  not 
encouraged  to  design  their  own  experiments.  Instead,  they  were  told  roughly  what  they  would  be  doing,  given  the 
procedures  for  collecting  data  and  would  run  the  experiment  on  their  own.  It  was  not  until  they  were  analyzing  the 
data  that  they  got  a sense  of  what  the  design  of  the  experiment  they  were  running  was.  They  were  not  trusted  to 
design  a relevant  experiment  for  the  community,  because  becoming  a member  of  that  community  of  practice  is  a 
long  and  difficult  process  for  the  undergraduate  students. 

To  improve  undergraduate  students'  learning  experiences  in  research  laboratories  students  need  to  develop 
the  abilities  that  are  required  for  taking  a more  active  role  in  their  research.  As  Scardamalia,  Bereiter  et  al. 
emphasize,  "Educationally  irrelevant  burdens,  should  be  minimized,  but  not  in  ways  that  deprive  students  of 
occasions  to  develop  the  planning,  monitoring,  goal-setting,  problem-solving  and  other  higher-order  abilities  that  are 
important  objectives  of  education,"  (1989,  p.  53).  The  practices  seen  in  the  research  laboratory  studied  took  away 
the  students  ability  to  participate  in  these  higher-order  objectives. 

The  computer  can  help  develop  communities  of  practice  when  it  acts  as  a mediational  technology,  which 
"emphasizes  the  possibilities  of  using  the  computer  not  simply  as  an  individual  tool  but  as  a medium  through  which 
individuals  and  groups  can  collaborate  with  others,"  (Bannon,  1995  , p.  272).  ResearchAid  acts  as  a resource  and  aid 
for  collaborating  students,  but  does  not  itself  collaborate  with  students  or  create  problems  for  the  students  to  work  on 
collaboratively. 

The  students  in  the  research  laboratory  needed  help  in  becoming  a member  of  the  research  community 
more  than  they  needed  help  with  any  particular  machine.  As  Lave  and  Wenger  (1991)  have  argued,  becoming  a 
member  of  a research  community  requires  familiarity  with  its  current  goals,  strategies  and  knowledge.  Lack  of 
familiarity  with  these  goals,  strategies  and  knowledge  made  it  difficult  for  these  students  to  take  an  active  role  in 
their  research  projects  beyond  the  collections  and  analysis  of  data. 

ResearchAid  helps  scaffold  new  students'  participation  in  the  manufacturing  research  community  in  the 
following  ways: 

• It  presents  prior  research  (experimental  designs  and  reports),  so  that  students  can  understand  how  it  developed 
from  the  lab's  goals  and  can  use  it  as  a model  for  creating  new  research.  The  application  presents  the  research  in 
the  framework  of  research  problems  that  have  been  or  are  being  explored  by  the  research  community.  This 
helps  students  formulate  a problem  they  would  like  to  explore  that  fits  within  the  research  community's  goals. 
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• It  helps  students  practice  and  understand  the  discourse  strategies  of  the  community  by  designing  experiments  in 
an  environment  which  explains  the  elements  of  an  experiment. 

• It  provides  the  means  for  members  of  the  community  to  asynchronously  communicate,  monitor  each  others' 
progress  and  give  feedback  through  adding  commentary  to  the  experimental  design. 

• It  promotes  students  revising  and  refining  of  their  initial  experimental  design  and  promotes  reflection  about 
alternatives  to  their  design. 

• It  gives  students  a way  to  collect  data  on  the  flexible  database  accessible  by  the  World  Wide  Web  (WWW)  and 
makes  it  available  for  analysis.  This  way  others  can  research  the  data  collected  by  students,  examine  progress 
being  made  and  use  old  data  to  explore  different  interpretations. 

• It  helps  the  research  community  analyze  and  discuss  experimental  results. 

ResearchAid  uses  the  WWW  to  have  students  present,  collect  and  edit  information,  through  editable 
WWW  pages  and  forms.  Using  the  WWW  for  ResearchAid  helps  provide  students  with  access  to  distant  faculty  and 
students  for  advise  and  feedback.  The  research  laboratory  that  was  the  subject  of  the  initial  investigation  had 
influential  members  of  the  research  community  located  on  different  parts  of  the  campus  and  the  undergraduate 
students  had  little  access  to  these  people.  To  become  a member  of  a community  of  researchers,  a students  needs  to 
be  able  to  meet  with  other  students  to  work,  discuss  and  create  a shared  understanding  of  their  common  needs  and 
experiences  (Selfe  & Eilola,  1988).  Because  computer-mediated  communication  helps  peripheral  members  of  a 
community,  such  as  undergraduate  students  in  a university  research  laboratory,  become  more  active  in  that 
community,  it  is  hypothesized  that  these  members  will  feel  like  a more  important  member  of  the  organization  and 
will  be  more  committed  to  it  (Eveland  & Bikson,  1988;  Sproull  & Kiesler,  1990). 

Future  work  on  ResearchAid  involves  development  of  the  final  system  (only  a working  prototype  exists 
right  now)  and  evaluation  of  how  students  actually  use  it  in  scientific  research  laboratories  should  also  be 
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Introduction 

Our  university  has  offered  distance  learning  courses  for  several  decades.  Like  many  universities, 
ours  has  in  recent  years  encouraged  faculty  to  explore  uses  of  the  internet  for  instruction.  We  currently  have 
over  sixty  courses  which  use  the  internet  in  some  way  to  deliver  instructional  materials,  however  most 
courses  are  web-supported,  rather  than  fully  web-based. 

This  paper  provides  a brief  overview  of  the  development,  implementation,  evaluation  and 
implications  of  one  of  our  university's  first  fully  web-based  courses.  Teaching  with  Technology.  This  is  a 
graduate  course  for  teachers  which  includes  both  theory,  design  and  considerable  hands-on  components. 

The  presentation  will  describe  in  detail  how  the  course  was  designed  to  prepare  teachers  to  integrate  many 
forms  of  technology,  both  computer  and  non-computer-based,  into  their  teaching.  Thorny  development  and 
implementation  issues  will  be  described. 

We  have  been  conducting  ongoing  developmental  research  while  teaching  the  course.  We 
conducted  a case  study  evaluation  of  the  course,  and  compared  the  results  of  the  internet-only  section  with 
those  of  a campus-based  section  which  utilized  the  same  web  materials.  The  course  was  offered  again,  in 
internet-only  form,  with  refinements,  during  the  spring  semester  of  1999,  and  we  again  collected  qualitative 
and  quantitative  data  to  help  us  in  evaluating  and  revising  the  course  for  delivery  later  in  1999. 

The  Teaching  with  Technology  Course 

Development  of  the  course  began  in  the  spring  of  1998.  Although  we  have  taught  technology 
courses  for  teachers  for  many  years,  we  began  by  analyzing  the  technology  knowledge  and  skills  teachers 
need  currently  to  ensure  that  the  course  is  up  to  date.  We  then  determined  the  overall  goals  and  objectives  of 
the  course. 

The  university's  Distance  Learning  Division  of  the  College  of  Extended  Education  was  our  partner 
in  delivering  the  course.  Distance  Learning  in  1998  used  a "custom  programming"  approach  to  deliver 
internet  instructional  materials.  The  course  instructor  wrote  the  course  materials  and  worked  with  the  small 
Distance  Learning  programming  staff  to  implement  the  web-based  version  of  the  course.  The  staff  used  web 
programming  software  and  languages  to  build  custom  courseware  wrapped  around  a commercial  discussion 
software.  This  approach  had  its  advantages,  but  also  yielded  several  drawbacks  which  will  be  discussed  in 
the  presentation.  For  instance,  we  needed  to  develop  a supplementary  web  site  to  support  the  course,  and 
also  used  email  communications  considerably  to  ensure  student  learning  and  satisfaction.  The  course  was 
next  implemented,  spring  of  1999,  using  commercial  web  course  development,  with  some  custom  elements. 

The  course  is  primarily  designed  for  K- 12  teachers  and  community  college  or  university  instructors. 
Some  students  enrolled  in  the  course  are  typically  also  trainers  from  business  and  industry,  therefore 
examples  from  both  education  and  training  are  included. 

This  course  provides  students  with  an  overview  of  the  most  effective  ways  to  teach  using  various 
learning  technologies.  The  course  begins  with  a broad  view  of  learning  technologies  and  the  research  and 
theory  related  to  using  them.  Students  then  learn  how  to  apply  instructional  planning  techniques  to 
integrating  technology  into  their  instruction.  We  cover:  advantages  and  disadvantages  of  various  learning 
technologies,  planning  for  technology,  evaluation,  and  basic  principles  of  educational  technology 
development.  The  course  consists  of  quizzes,  short  papers  and  projects,  and  a hands-on  final  project. 

Course  Components,  Internet-Only  Section: 

The  fully  online  course  section  initially  included  assigned  readings  in  textbooks  to  provide  the  basic 
information,  email  communications  software  for  instructors  and  students  and  a web  site  that  is  part  of 
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Distance  Learning’s  ASU  ONLINE  offering,  which  includes  features  such  as  "News"  or  announcements,  an 
"Assignments"  posting  and  submission  area,  "Test"  features,  a "Discussion"  thread  component  for 
asynchonous  postings,  a "Chat  Room"  for  synchronous  discussions,  and  "Web  Resources"  and  a course 
"Calendar"  component. 

During  the  initial  course  offering  we  revised  components  on  an  ongoing  basis.  One  concern,  for 
example,  was  that  we  did  not  truly  know  the  needs  of  the  students  before  they  signed  on  for  their  first  class, 
so  we  conducted  a needs  assessment  with  the  students  and  revised  the  course  topics  somewhat  to  reflect  their 
professional  needs 

Another  particular  concern  was  the  need  to  have  hands-on  projects  for  teachers  who  have  many 
different  types  of  hardware  and  software,  as  well  as  resources  and  constraints,  represented  in  their  off- 
campus  work  settings.  We  have  tackled  this  issue  by  alio  wing  students  a large  degree  of  control  in 
determining  and  completing  projects,  while  providing  them  flexible  guidelines  and  clear  evaluation  criteria. 

We  developed  video  clips  included  in  a new  "Media"  component  of  the  course.  Clips  show 
demonstrations  of  applications,  technical  processes  and  procedures,  guest  speakers  and  "virtual  field  trips." 

We  also  developed  a supplementary  web  site  in  response  to  the  relatively  long  turn  around  time  for 
custom  programming.  The  site  is  used  to  provide  information  such  as  mini-lectures,  extra  text  materials  and 
handouts.  It  also  serves  as  a substitute  delivery  medium  for  directions  and  assignments  at  times  when  the 
online  course  site  components  are  not  functioning  as  expected. 

During  the  second  semester  of  the  course,  spring  1999,  we  began  to  use  Courseinfo  2.0  by 
Blackboard,  Inc.  This  software  provides  us  with  easily  updateable  components  and  faculty  can  do  these 
updates  themselves.  Another  new  feature  is  the  provision  of  student  home  pages,  allows  students  to  easily 
present  information  about  themselves  and  about  projects  they  would  like  to  work  on  collaboratively.  The 
Courseinfo  software  will  be  described  in  more  detail  in  the  presentation. 


Evaluation  Methods 

During  the  first  pilot  semester,  Fall  of  1998,  we  evaluated  the  effectiveness  of  web-based  online 
delivery  of  this  technology  course  for  teachers  using  various  methods. 

We  collected  data  on  student  learning  by  comparing  overall  student  grades  in  the  online  and  the 
campus  section.  We  reviewed  the  quality  of  students’  technology  integration  plans  and  final  development 
projects.  In  addition,  students'  ratings  of  how  much  they  perceived  they  learned  were  compared,  along  with 
their  evaluation  of  the  learning  value  of  the  course. 

Student  attitudes  were  measured  using  several  means.  Students  in  both  sections  completed  a survey 
at  the  beginning  of  the  semester  in  order  to  determine  their  needs  and  interests  and  incorporate  these  into  the 
course.  Students  were  asked  at  the  mid-semester  point  to  discuss  aspects  of  the  course  and  how  the  course 
could  be  improved  for  them.  Finally,  students  in  both  sections  completed  a detailed  course  evaluation, 
focussed  on  determining  the  factors  that  helped  them  learn  and  what  motivated  them  to  learn. 

Communication  among  students,  the  instructor  and  staff  were  analyzed.  The  discourse  in  the 
discussion  threads  was  captured  and  these  communications  data  were  analyzed  using  qualitative  methods, 
particularly  content  and  pattern  analysis.  Also,  patterns  of  communication  and  issues  that  arose  in  the  email 
communications  between  the  instructor  and  students  and  staff  were  analyzed. 

The  instructor  and  development  team  also  kept  ongoing  logs  of  the  issues  and  solutions  that  evolved 
during  the  course 

Conclusion 

Teacher  educators  are  struggling  with  determining  how  best  to  deliver  instruction  to  teachers  on 
how  they  can  integrate  technology  to  help  their  students  learn.  In  our  presentation  we  discuss  how  this 
method  is  providing  technology  education  to  teachers  who  could  not  otherwise  receive  it.  It  is  hoped  that  our 
discussion  will  provide  other  teacher  educators  with  guidance  for  providing  web-based  technology 
instruction  for  teachers. 
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Introduction 

In  South  Africa  under  Apartheid  educational  institutions  were  segregated  according  to  pseudoracial  categories. 
“White”  institutions  had  the  greatest  advantage  in  terms  of  funding  and  long-term  investment  opportunities, 
while  institutions  categorised  as  “black,”  “Indian,”  or  “coloured”  were  generally  established  much  more 
recently  and  lacked  the  funding  traditions  and  potential  to  develop  their  own  investments.  These  institutions 
continue  to  draw  their  student  population  largely  from  a school  system  that  was  deliberately  created  under 
Apartheid  to  be  inferior.  This  is  particularly  a problem  in  the  sciences,  an  area  that  was  previously  denied 
access  by  the  broader  South  African  population. 

We  believe  that  the  educational  needs  of  students  from  disadvantaged  backgrounds  can  be  addressed  by 
designing  interactive,  educational  multimedia.  This  will  allow  students  with  different  learning  abilities  to 
acquire  a body  of  skills  and  knowledge  at  their  own  pace.  In  this  context,  we  embarked  on  a programme  of 
using  computers  and  web-based  technology  within  the  context  of  mainstream  teaching-and-leaming. 


The  Educational  Principles  Followed 

All  course  materials  aim  to  promote  meaningful  learning  (Ausubel  et  al.  1978),  so  that  students  come  to 
exercise  the  higher  intellectual  processes  of  analysis,  synthesis  and  evaluation.  There  is  a tremendous  inertia 
among  students  at  UWC  to  focus  on  the  process  of  memorisation  (personal  observation).  Both  multimedia  and 
conventional  resources  for  the  course  were  designed  in  an  effort  to  move  students  away  from  rote  learning 
(Keats  & Boughey  1993).  The  meaningful  construction  of  learning  is  an  active  process,  involving  an 
interaction  of  students  with  a variety  of  learning  resources  (Jones  et  al.  1987,  Moust  et  al.  1987).  Knowledge 
construction  can  be  improved  through  the  use  of  preliminary  organisers,  such  as  statement  of  objectives  and 
background  knowledge  required,  as  well  as  an  overview  of  the  material  to  be  covered  and  the  problems  to  be 
addressed  (Melton  1984).  Hence,  web  and  multimedia  resources  are  designed  with  clearly  stated  objectives  as  to 
what  was  expected  of  the  learner. 


The  Technology  and  Development  Methods 

With  a broadband  network,  the  creation  of  deliverance  of  a full  multimedia  course  becomes  possible.  A fairly 
static  HTML  webpage  was  created  to  allow  basic  concepts  to  be  explored  from  workstations  outside  the  ATM 
network.  Multimedia  is  then  used  to  add  depth  to  the  resources  when  they  are  accessed  via  workstations  with 
higher  speed  connection  to  a network. 

A small  multimedia  lab  was  established  with  all  computers  having  a dedicated  1 0 Mb/s  of  Ethernet  bandwidth 
linked  to  a 155  Mb/s  ATM  network.  A multimedia  web  server  is  connected  to  the  same  Ethernet  switch  and 
the  ATM  network.  In  addition,  a computer  lab  is  available  via  the  campus  Ethernet  backbone.  For  the  first  time 
this  gives  us  the  ability  to  provide  students  access  to  both  web  and  multimedia  facilities. 

We  have  adopted  an  approach  of  developing  teaching-and-leaming  resources  that  have  the  following  goals: 

1.  At  one  level  resources  should  be  bandwidth-friendly.  One  of  our  goals  in  developing  this  course 
material  is  to  have  it  available  globally. 


2.  At  another  level  resources  should  exploit  the  full  potential  of  a high-speed  network,  and  provide  an 
exciting  learning  opportunity  motivating  students  to  break  out  of  the  bounds  of  conventional  learning. 

3.  Resources  should  exploit  the  full  potential  of  HTML  and  scripting  to  incorporate  interactivity,  thereby 
reducing  dependence  on  additional  software  wherever  possible. 

On  the  surface,  the  first  two  goals  seem  to  be  mutually  exclusive.  How  is  it  possible  to  minimise  bandwidth  use, 
while  at  the  same  time  maximising  the  potential  of  a high  bandwidth  network?  The  answer  that  follows  from 
these  competing  goals  is  a layered  approach,  whereby  learning  topics  are  covered  using  relatively  static  HTML, 
but  in  which  users  with  high-speed  access  can  go  to  another  level  with  animation,  sound,  video  and  greater 
interactive  control.  This  approach  then  provides  the  guidelines  for  a layered  development  of  web-based  learning 
resources. 


The  Research  and  Instruments 

The  work  is  being  carried  out  under  the  framework  of  Action  Research  which  is  qualitative  and  cyclical, 
leading  to  ongoing  improvements  in  action  (Davidoff  & Van  den  Berg  1991).  Thus  it  is  a suitable  mode  of 
research  to  inform  on-going  development,  such  as  this  project  involves. 

Two  main  research  instruments  are  being  used  during  this  project:  the  interview  and  the  questionnaire.  In  this 
project,  a semi-structured  interview  is  used  as  a means  to  gather  information.  Pre-formulated  questions  will 
guide  the  interview,  but  they  will  be  open-ended  ones  which  Cohen  & Manion  (1980)  defined  as  "those  that 
supply  a frame  of  reference  for  respondent  answers,  but  put  a minimum  of  restraint  on  the  answers  and  their 
expression”.  Interview  results  are  used  to  inform  the  Action  Research  in  that  the  data  and  recommendations 
will  be  subjected  to  evaluation  and  the  actions  within  the  project  (multimedia  content  and  activities)  subjected 
to  revision  accordingly.  The  lecturer  will  be  interviewed  to  determine  content  that  is  most  appropriate  to  be 
enhanced  using  multimedia  activities,  and  the  students  to  determine  their  area  of  greatest  difficulty  that  can  be 
addressed  by  incorporating  more  interactivity.  Once  the  students  are  using  the  facilities,  they  will  be  given 
structured  questionnaires,  and  will  be  observed  and  interviewed  as  they  use  the  multimedia  lab. 
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Introduction 

Educational  opportunities  are  reaching  populations  heretofore  left  out  of  the  equation  thanks  to  distance 
education  technologies  and  institutional  desire  to  expand  the  student  base.  Once  a component  of  only  the 
richest  universities,  distance  education  technologies  are  becoming  more  affordable,  enabling  incorporation  by  a 
significant  proportion  of  the  university  community.  One  of  the  most  popular  technologies,  and  the  one 
compared  most  directly  to  face-to-face  interaction,  is  interactive  video.  This  relatively  new  technology  has 
provided  instructors  with  the  ability  to  reach  new  student  populations  as  well  as  expose  traditional  students  to 
new  peer  groups.  Although  many  institutions  have  interactive  video  equipment  and  are  using  it,  the  focus  in 
training  remains  on  the  course  content.  The  interactive  dimensions  are  rarely  touched  upon  during  faculty 
training  for  those  using  interactive  video  to  conduct  courses.  As  with  other  forms  of  distance  education,  the 
acquiring  of  delivery  skills  associated  with  interactive  video  teaching  is  generally  through  practice,  or  trial  and 
error  might  be  more  descriptive.  In  essence,  the  classroom  becomes  the  testing  laboratory,  influencing  the 
learning  process  during  that  preliminary  stage.  To  facilitate  effective  learning  for  the  students,  and  to  alleviate 
unnecessary  aspects  of  teaching  and  credibility  problems  for  the  instructor,  faculty  training  must  be  targeted 
directly  and  appropriately  at  the  technological  and  social  dimensions  involved  in  interactive  video  instruction. 


Course  Integration 

When  using  interactive  video  in  teaching,  the  interactive  video  is  so  much  a part  of  the  course  that  it  influences 
the  course  content  and  structure  directly.  In  order  to  construct  an  effective  syllabus,  the  instructor  must  have  a 
working  knowledge  of  all  the  dimensions  of  the  influence  the  interactive  video  format  will  have.  Interactive 
video,  as  a mode  of  communication,  can  be  very  effective  if  it  is  a tool  that  has  been  mastered.  An  instructor 
who  does  not  have  an  understanding  of  the  technology  involved  will  find  it  a hindrance  that  can  actually  block 
the  educational  process. 

When  selecting  a person  to  teach  a course,  there  is  no  doubt  that  the  credentials  of  that  person  will  be 
thoroughly  examined.  That  person  will  be  expected  to  have  college  degrees,  generally  in  a field  containing  or 
directly  related  to  the  course  being  taught.  Previous  experience  in  the  classroom  is  examined,  including  sample 
student  and  peer  evaluations,  as  to  the  goodness  of  fit  for  the  applicant  and  the  institution  and  class.  However, 
when  selecting  the  person  to  teach  via  interactive  video,  where  did  the  inquisitiveness  go?  Generally  the  person 
assigned  to  the  interactive  video  course  is  the  person  who  happens  to  already  be  teaching  the  course  that  is 
desired,  the  person  who  has  the  desired  time-slot  available,  or  the  person  who  seems  to  mind  the  imposition  the 
least.  Once  that  person  begins  to  teach,  it  is  still  unclear  where  they  can  go  for  help — with  the  possible 
exception  of  technological  problems  that  can  be  asked  of  attendants  on  duty  during  the  class.  Thus,  the  students 
see  the  instructor  less  than  prepared,  asking  questions,  and  showing  inadequacies.  This  appearance  will  then 
lower  the  perceived  credibility  of  the  instructor,  even  though  it  is  based  on  non-content  issues. 


Student  Dynamics 

The  classroom  interaction  dynamics  are  greatly  effected  in  a class  delivered  via  interactive  video.  Although  the 
individual  compositions  may  vary,  the  “group”  of  students  is  generally  at  least  two  separate  groups,  sometimes 
three  or  four,  attempting  to  merge  into  one  class  group.  “The  primary  group  is  that  which  is  physically  with  the 
instructor.  The  secondary  group  is  that  which  perceptually  resides  in  the  television”  (Nance  & Strohmaier, 
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1998).  For  effective  interaction  in  the  classroom,  the  students  must  perceive  the  physically  separate  groups  as 
one  class  group.  Accomplishing  this  goal  can  be  difficult  for  an  instructor,  especially  if  he  or  she  is  unaware  of 
this  potentially  disastrous  dynamic  issue.  Faculty  development  could  address  this  issue. 

Another  major  area  of  impact  lies  in  the  visual  perception  engendered  in  interactive  video.  As  a member  of  the 
primary  group,  the  instructor  sees  fluid  movement  in  the  physically  present  group,  and  visual  compression  with 
the  secondary  group  images.  As  the  students  are  listening,  this  visual  compression  does  not  appear  to  have  a 
significant  impact.  If  the  instructor  was  to  trade  places  with  a student  in  a secondary  group,  he  or  she  would  see 
the  magnitude  of  the  impact  of  visual  compression.  The  visual  cues  of  the  instructor  are  not  fluid,  thus  dividing 
the  focus  of  those  students.  Other  interaction  components  are  also  effected  by  this  lack  of  visual  flow,  such  as 
with  tum-takirxg  behaviors.  “The  way  an  individual  know.s  that  it  is  his  or  her  turn  to  speak  in  a conversation  is 
through  pause  time,  eye  contact  changes,  altering  the  tonal  pattern  of  the  voice,  and  posture  adjustment. 
Although  the  students  can  note  the  pause  time  and  tonal  differences  in  an  interactive  video  setting,  the  eye 
contact  and  posture  adjustment  are  not  easily  read”  (Nance  and  Strohmaier,  1998).  The  vocal  aspects  of  the 
interaction  are  however  generally  in  real  time,  which  can  compensate  for  some  of  the  interaction  cues  that  are 
taken  away  in  the  visual  mode. 

Just  as  the  dynamic  effects  of  the  interactive  video  mode  have  far-reaching  consequences,  the  potential 
confusion  regarding  turn-taking  behavior  has  several  ramifications.  When  faced  with  a long  pause  time  people 
feel  uncomfortable  and  begin  to  get  restless.  To  alleviate  the  situation,  someone  will  end  the  pause  through 
verbal  interaction.  In  an  interactive  class,  the  primary  group  will  be  privy  to  all  of  the  pertinent  interaction  cues 
as  to  turn-taking  behavior.  They  will  have  the  insider  information  as  to  when  the  instructor  is  actually  looking 
for  an  answer,  versus  stating  a hypothetical  question.  The  students  in  the  secondary  groups  will  not  be  sure 
which  is  the  case,  and  will  therefore  choose  the  polite  path  and  wait  longer  before  answering.  In  this  situation, 
then,  the  students  in  the  primary  group  will  generally  always  be  the  ones  to  break  the  pause-time.  The  routine 
will  become  standard,  and  the  secondary  groups  will  be  both  perceptually  and  actively  removed  from  the 
discussion  core.  To  avoid  this  scenario,  the  instructor  must  overtly  draw  in  the  members  of  the  secondary  group 
through  directed  discussion.  Again,  faculty  development  should  address  these  issues  to  ensure  that  the  faculty 
member  is  adequately  prepared  to  handle  these  situations  and  facilitate  meaningful  interaction  between  groups. 


Conclusion 

Distance  education  is  a timely  concept.  Although  in  existence  for  decades,  technological  developments  have 
created  many  new  dimensions  and  possibilities  for  distance  education.  Much  research  is  being  done  on  the 
potential  and  use  of  distance  education,  with  a focus  improving  the  technologies.  Research  must  branch  out  and 
enhancing  the  capabilities  of  the  instructors  through  formalized  faculty  development  programs  as  well.  Rather 
than  instructing  faculty  based  on  crises  that  arise  during  interactive  video  classes,  faculty  development  should 
be  approached  in  a proactive  manner  to  avoid  the  crises  before  they  occur.  In  this  age  of  serious  budget 
considerations,  the  institution  would  benefit  financially  with  this  approach  as  well. 
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Section  1:  Introduction 

This  paper  reports  on  an  initial  and  on-going  study  into  the  use  of  haptic  (Schiff  & Foulke,  1982)  technology  in 
veterinary  education.  The  University  of  Glasgow  has  a commitment  to  the  implementation  of  non-invasive 
procedures  for  diagnostic  and  education  purposes  in  Veterinary  Medicine,  wherever  possible.  As  part  of  the 
continued  development  in  the  use  of  technology  in  veterinary  education,  Virtual  Reality  is  now  being  explored  as 
a possible  aid  to  teaching  and  in  particular  to  replace  invasive  examination  procedures. 

Section  1.1:  The  problems 

One  major  problem  in  the  education  of  veterinary  students  is  the  danger  faced  by  animals  when  being  examined 
by  inexperienced  students.  The  students  need  to  gain  experience  in  internal  examinations  (a  key  method  for 
diagnosing  problems  and  diseases)  but  it  can  be  dangerous.  The  animals  may  become  stressed,  be  injured  or  may 
even  die  because  of  unskilled  internal  examinations.  Large  classes  of  students  mean  that  each  person  may  only 
get  a very  limited  amount  to  time  learn  the  practical  examination  skills  required.  Another  problem  is  that  the 
students  must  learn  about  a whole  range  of  diseases  and  problems  as  part  of  their  education.  At  the  time  during 
their  training  when  they  are  learning  about  a particular  disease  there  may  be  no  animal  available  with  the 
particular  disease  in  question.  This  means  that  students  may  not  be  able  to  consolidate  their  learning  with 
practical  experience. 

Section  1.2:  The  solutions 

We  are  investigating  ways  to  solve  these  problems  by  using  haptic  devices  that  allow  users  to  feel  virtual  objects. 
There  are  three  main  benefits  from  our  solution: 

Safety:  The  use  of  haptic  models  to  simulate  problems  allows  the  students  to  learn  in  a safe  environment.  They 
can  learn  on  the  models  without  endangering  any  animals.  Once  they  have  gained  sufficient  skills  on  the  models 
they  can  then  move  on  to  the  live  animals  with  much  less  chance  of  doing  harm. 

Cost:  The  cost  of  education  is  also  reduced.  A large  number  of  students  can  interact  with  the  haptic  models  very 
quickly  and  cheaply.  They  can  also  do  this  much  more  frequently  than  would  be  possible  with  live  animals. 

Flexibility:  The  flexibility  of  the  models  is  great.  It  is  possible  to  simulate  a range  of  diseases  that  the  students 
would  not  normally  experience.  Different  stages  in  the  progress  of  a disease  or  condition  can  also  be  simulated. 
This  can  be  done  at  the  time  the  students  are  learning,  allowing  them  to  try  out  their  theoretical  knowledge 
immediately  in  a practical  setting. 

Section  2:  The  technology 

The  technology  to  feel  virtual  objects  is  just  now  becoming  available  (Massie  & Salisbury,  1994,  Bryson,  1995). 
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It  was  first  developed  so  that  users  could  feel  objects  in  virtual  environments.  Minsky  (in  Blattner  & 
Dannenberg,  1992)  describes  the  technology  thus:  ’’Force  display  technology  works  by  using  mechanical 
actuators  to  apply  forces  to  the  user.  By  simulating  the  physics  of  the  user’s  virtual  world,  we  can  compute  these 
forces  in  real-time,  and  then  send  them  to  the  actuators  so  that  the  user  feels  them”.  The  device  used  for  the  work 
described  is  a PHANToM.  This  is  a very  high  resolution,  six  degrees-of-freedom  device  in  which  the  user  puts 
his/her  finger  in  a thimble  or  holds  a pen  at  the  end  of  a motor-controlled,  jointed  arm.  It  provides  a 
programmable  sense  of  touch  that  allows  users  to  feel  textures  and  shapes  of  virtual  objects,  modulate  and 
deform  objects. 

Section  3:  The  haptic  models 

As  mentioned,  we  are  developing  haptic  software  models  that  will  simulate  the  feel  of  parts  of  a iiorse  under 
examination.  This  shows  two  horse  ovaries.  The  ovaries  feel  correct  and  they  move  in  appropriate  ways  (if 
pressure  is  applied  they  can  be  moved  in  three  dimensions  in  a realistic  way).  Both  ovaries  have  very  smooth, 
taut  surfaces.  The  left  ovary  is  softer  and  slightly  smaller  than  the  right  one.  A follicle  can  be  seen  on  the  left 
ovary  (indicating  ovulation  is  about  to  take  place).  This  sticks  out  from  and  is  much  softer  than  the  rest  of  the 
surface.  The  visual  representation  of  the  ovaries  is  built  in  OpenGL.  The  haptic  representation  is  built  using  the 
GHoST  software  toolkit  from  SensAble.  When  vets  are  doing  internal  examinations  they  cannot  see  the  ovaries, 
so  the  visual  images  are  mainly  for  illustrative  purposes.  However,  a visual  representation  is  beneficial  when 
learning  to  use  the  system.  Our  software  allows  the  visual  representation  to  be  removed  as  required.  The 
PHANToM  device  only  allows  the  user  to  feel  with  one  finger,  not  the  whole  hand.  This,  however,  is  not  a 
limitation  as  vets  will  only  use  their  thumb  when  feeling  the  ovary  in  a real  examination. 


Section  4:  Evaluation 

The  development  of  the  models  has  been  performed  using  an  iterative,  participatory  approach.  Computing 
scientists  have  been  working  closely  with  vets  to  ensure  that  the  models  are  correct.  This  has  involved  the 
building  of  initial  prototype  models  and  their  continual  refinement  via  expert  user  evaluation.  The  next  stage  in 
our  work  is  to  carry  out  evaluations  with  vet  students.  We  will  discover  if  the  models  are  usable:  can  the  vet 
students  interact  with  them,  use  the  device,  etc.  We  will  also  be  able  to  find  out  how  effective  they  are:  can 
students  correctly  identify  the  particular  diseases  and  conditions  we  have  simulated  in  our  models. 


Section  5:  Conclusions 

One  problem  in  veterinary  education  is  that  students  must  learn  how  to  do  internal  examinations  of  animals  but 
in  doing  so  can  be  dangerous  for  the  animals.  Another  difficulty  is  that  there  may  not  be  an  animal  available  with 
a particular  disease  when  the  students  are  studying  it.  The  availability  of  haptic  devices  means  that  these 
problems  can  now  be  overcome.  We  have  developed  a set  of  haptic  models  of  horse  ovaries  that  will  allow 
students  to  learn  in  a safe,  cheap  and  flexible  environment.  Once  they  have  learned  the  necessary  skills  they  can 
move  to  working  with  live  animals  with  much  less  risk  of  danger. 
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Introduction 

With  the  spread  of  Internet,  the  source  of  information  surrounding  us  diversifies.  As  a result  the  burden  for 
gathering  and  sharing  information  for  analyzing  problems  suddenly  increases  in  a real  business.  Such  a change  gives 
a big  influence  to  learning  environment  for  both  an  individual  and  a business  organization.  For  one  of  representative 
cases  we  can  point  out  the  trend  that  “Knowledge  Management”  attracts  many  people  in  the  field  of  business.  In 
other  words,  they  need  a continuous  collaborative  learning  that  is  attached  great  importance  to  experience  in  a real 
business  in  order  to  cope  with  such  a difficult  situation.  Teams  are  organized  according  to  interest  domains,  and  we 
expect  that  participants  in  teams  search  information  from  their  various  points  of  view.  By  exchanging  many  opinions 
with  members,  they  will  share  a hypothesis  and  information  furthermore,  and  through  collaborative  activities  they 
will  share  a lot  of  experience  and  learn  how  to  make  a new  concept.  Such  a learning  method  is  called  “Collaborative 
Learning”  or  “Action  Learning”,  and  many  people  trend  t think  that  this  is  a kind  of  team  learning  (or  team 
teaching),  that  attached  great  importance  to  such  a process.  The  purpose  of  this  paper  is  to  show  the  usefulness  for 
the  system  of  designing  user  interface  to  promote  and  support  a practice  of  them. 

Problems  of  “Collaborative  Learning”  or  “Action  Learning” 

Judging  from  examples  of  education  in  a corporation  as  before,  the  biggest  problem  is  that  “long  time  is 
necessary  in  order  to  get  effect”  when  we  practice  “Collaborative  Learning”  or  “Action  Learning”, 

The  result  of  our  experiment  clearly  shows  that  an  equal  characteristic  was  found  in  apportioning  time 
according  to  an  activity  process  in  inspection  experiment  of  two  sets.  The  graph  in  Figure  1 shows  that  in  both  sets 
the  ratio  of  a process  of  gathering  information  (including  Experiencing  and  Publishing)  is  very  high,  and  a ratio  of  a 
process  of  opinion  exchange  and  the  logic  construction  (including  Processing  and  Generalizing)  is  lower.  As  this 
reason,  when  learning  persons  accessed  the  various  source  of  information  that  they  hadn’t  used,  they  tended  to  have 
a long  time  to  gather  information  (including  grasp  of  characteristic  information  and  the  way  to  access  to  every  the 
source  of  information).  We  used  the  analysis  of  the  log  of  experiments  and  questionnaire  investigation  to  learning 
persons  in  order  to  report  this  result. 


0%  20%  40%  60%  80%  1 00% 


Set  1 


Set  2 


■ Experiencing 

□ Publishing 

□ Processing 

□ Generalizing 
D Applying 


Figure  1:  The  ratio  of  apportioning  time 
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Schemes  and  Effects  of  the  system 


In  this  study,  we  establish  the  following  point  of  view  of  solving  problems  of  these  learning  methods. 

• By  filtering  necessary  information  from  the  various  source  of  information,  several  learning  persons  and 
teacher  (facilitator)  build  the  structure  that  gathering  and  sharing  information  become  possible 
effectively. 

• By  re-organizing  much  information  collected  by  all  the  participants,  we  build  database  environment  to 
structuralize  the  information  flexibly  according  to  a purpose. 

According  to  basic  concept  of  a system,  we  build  a system  with  C/S  application  of  Web  base  shown  by  Figure 
2.  The  system  has  the  following  characteristic. 

This  sysiciii  collects  interest  of  learning  persons  with  questionnaire  system  beforehand.  Based  on  these  interest 
profile  databases,  this  system  can  automatically  form  interface  for  every  learning  person  and  a team.  In  next,  by  the 
key  word  they  prepare,  they  can  search  several  database  and  WWW  simultaneously  and  output  search  results.  And, 
on  the  basis  of  interest  profile  and  their  rating,  this  system  has  a function  to  recommend  information  that  other 
learning  person  collected  (utilized  by  push  technology  and  collaborative  filtering  technology).  Facilitators  can 
manage  monitoring  of  access  situation.  Finally  this  system  has  a database  including  input  interface  of  outline 
processor  type.  This  database  can  organize  it  accounting  to  own  participant’s  logic  for  all  the  information  that  they 
collected  again. 

In  the  whole  activity,  about  around  5 % ratios  of  gathering  information  did  decrease  as  a result  of  having 
introduced  this  system  in  practice  process  of  Action  Learning.  By  sharing  gathering  information  process  with  other 
participants,  reduction  of  about  around  10  % gathering  information  costs  was  more  effective  than  experiments  before 
introducing  system. 


Teacher  (Client) 


Figure  2;  Characteristic  of  the  system 


Conclusion 

So  far  we  have  outlined  the  concept  and  design  of  system.  In  concluding,  we  should  note  that  this  system 
principally  effective  for  shortening  an  apportionment  of  gathering  information  process  when  practicing  these 
methods.  The  Future  direction  of  this  study  will  be  to  extend  this  system  examine  structure  clearing  and  managing  a 
copyright  of  digital  content. 
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7.  Introduction 

In  the  sphere  of  health,  the  introduction  of  new  technologies  in  the  educational  context  is  taking  place  in  an 
international  scale,  with  the  development  and  adequation  of  computer  systems  that  deal  with  informations  in  an 
optimized  way,  as  relates  to  the  efficiency  of  access,  recovery,  presentation  and  dissemination,  at  the  same  time  that 
they  treat  in  an  adequate  way  the  psychopedagogical  issues  involved  in  the  learning  process.  This  is  certainly  not 
only  due  to  the  complex  and  diversified  nature  of  the  information  that  is  always  in  expansion  in  the  field  of  health, 
but  also  to  the  very  quantity  of  information  that  needs  to  be  dealt  with  in  this  field,  allied  with  the  advancements  in 
computer  sciences  and  telecommunications.  All  these  facts  result  in  the  need  of  development  of  products  related 
specifically  to  the  health  field,  carried  out  by  professionals  that  should  not  only  dominate  the  theoretical  and 
practical  knowledge  of  the  relevant  aspects,  but  should  also  be  in  close  contact  with  the  main  advances  in  this  field. 
The  Oswaldo  Cruz  Fundation  - FIOCRUZ,  well  known  for  its  experience  in  teaching  and  research,  with  highly 
qualified  professionals,  turns  its  interests  towards  the  education  and  training  in  the  field  of  the  health  sciences, 
developing,  distributing  and  evaluating  educational  products  based  on  multimidia  technology  ( hipertexts  ), 
focussing  on  the  theoretical-  practical  knowledge  of  the  targets  that  are  most  in  demand  from  the  scientific 
community,  including  graduate  and  undergraduate  students  from  the  Brazilian  institutions  of  teaching  and  research. 
The  spheres  of  development  of  these  systems  are  necessarily  multidisciplinary,  demanding  an  interdependence  of 
knowledge.  That  is  why,  besides  the  team  that  is  considered  the  basis  (biologists,  system  engineer,  visual 
programmers)  other  teams  composed  of  pedagogues,  psychologists,  computer  science  professionals  and 
fotographers  will  be  formed  in  accordance  with  the  subject  to  be  dealt  with.  The  encounter  of  the  teams  is  taking 
place  in  the  National  Public  Health  Scholl  -Secretary  of  Educational  Development  (SDE/ENSP),  a point  of  intra  and 
interinstitutional  integration,  where  efforts  are  joined  in  order  to  promote  a qualitative  leap  in  the  educational 
products  issued  by  the  society. 

2,Projects  in  Course 

A.  HIPERGIS:  a hypermedia  system  on  techniques  of  spatial  mapping  and  systems  of  geographical  informations  for 
the  health  field.  CASTILHO-BARILLI  ECV‘;  NOBRE  FF^;  CARVALHO  MS^  / ‘Nacional  Publuc  Health  School 
/FIOCRUZ; ^Biomedical  Engineering  Program  COPPE/UFRJ;^Nacional  Publuc  Health  School 
The  main  objective  of  this  work  is  to  favour  the  routine  employment  of  spatial  analysis  in  managing  processes  of 
public  health  with  the  usage  of  techniques  of  spatial  mapping  through  computerized  Geographical  Information 
Systems,  as  it  is  believed  that  this  type  of  analysis  facilitates  its  steps  in  relation  to  time  spent,  costs  involved  and 
also  in  what  concerns  the  broadcasting  of  its  results.  The  HIPERGIS  is  a hipermedia  system,  whose  aim  is  to  aid  in 
the  acquisition  of  knowledge  about  the  techniques  of  spatial  representation  of  data,  as  well  as  in  the  employment  of 
computerized  geographical  information  systems,  through  the  training  and/or  improvement  of  the  professionals  who 
work  in  the  health  field  and  undergraduate  and  graduate  students,  therefore  mending  a gap  in  the  knowledge  of  these 
people  who,  in  general,  do  not  have  access  to  these  subjects  in  their  graduation  studies.  The  data  basis  of  the 
hipermedia  is  composed  by  specialized  mainly  derived  from  textbooks  on  cartography  and  scientific  articles.  The 
so-called  “authorship  tool”  used  for  implementing  the  basis  of  knowledge  of  the  HIPERGIS  was  the  Multimedia 
software  Toolbool  4.0,  since  it  was  easily  handled  and  it  presented  a good  cost-benefit  relationship.  HIPERGIS  is 
divided  in  4 main  modules:  1)  Help-  subdivided  in:  a)  descriptions  on  its  navigation  and  b)  Hipertexts:  historical, 
definitions  and  applications;  2)  Basic  Concepts  - topics  on  cartography;  3)  Spatial  Representation  - descriptions  of 
the  spatial  mapping  techniques  with  graphic  examples  (maps)  and  4)  Systems  of  Spatial  Informations-  in  this  topic 
one  includes  the  GIS.  The  buttons  constitute  the  main  means  of  navigation  of  the  system.  They  are  disposed,  in  a 
pattern,  at  the  inferior  left  comer  of  each  screen,  being  divided  in:  Fluxogram  - presents  the  topics  and  subtopics, 
navegate??  - summary-screen  that  explains  the  function  of  each  botton,  return  to  the  beginning  button-  sends  the 
user  to  the  page  or  the  introduction  screen  of  each  topic  or  chapter;  Main  Menu-  presents  the  main  modules;  send 
page  forwards  and  backwards  and  exit  button-  ends  the  system.  The  system  still  offers  a screen  or  page  of  glossary, 
with  a list  of  words  in  alphabetical  order.  The  minimal  technical  requirements  for  the  HIPERGIS  use  with  a good 
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performance  are:  a microcomputer  PC-486,  with  16  Megabytes  of  memory,  12  Megabytes  of  free  space  in  hard  disk, 
SVGA  monitor  with  a 1 Megabyte  plaque  configured  for  256  colours,  as  well  as  an  MS-Windows  3.1  File  Manager 
( which  fits  perfectly  well  the  more  modem  versions,  like  Windows  95  ). 

B.  Development,  Application  and  Evaluation  of  Multimedia  Systems  on  the  Main  Human  Parasitic  Disseases  for 
Health  Education  . ‘Elomar  CHRISTINA  ViEiRA  CASTILHO  Barilli;  ^Luis  Key;  ^ AntOniO  J S de  Paiva; 
"^Luciano  de  Paula  /‘ENSP  -Oswaldo  Cruz  Fundation;  ^Tropical  Medicine  Department  - lOC/FIOCRUZ; 
^’"‘SDE/ENSP  The  main  HUMAN  PARASITIC  DISEASES  constitute  the  central  target  of  this  work,  which  is 
meant  for  graduate  and  undergraduate  students  of  the  national  institutions  dedicated  to  teaching  and  research.  The 
first  theme  to  be  dealt  with  is  Human  Malaria,  its  meaning,  characteristics,  ways  of  treatment  and  control  of  the 
disease.  The  level  of  information  to  be  followed  is  the  basic  one,  and,  as  information  source,  the  texts  of  the 
following  books  are  being  employed:  Parasitologia,  Bases  da  Parasitologia  Medica  and  Dicionario  de  Termos 
Tecnicos  de  Medicina  e Saude,  all  being  written  by  Dr.  Luis  Rey.  The  multimedia  is  divided  into  five  main  modules: 
Introduction-m^idQ  up  of  a video;  historicaX-  with  a slide  show  linking  an  audio  file  with  historical  photos  that  are 
presented  on  the  screen;  the  parasite-  morphology,  evolutive  cycle  and  physiology;  the  mosquitos-  morphology, 
biology,  collecting  and  control  and  the  disease-  pathology,  diagnostics  and  therapeutics,  epidemiology  and  control. 
The  system  contains,  besides  the  strong  textual  content,  schemes,  videos,  photos,  and,  animations.  The  basic 
programme  used  for  the  implementation  of  hte  system  is  the  AuthorWare  4.0.  It  will  be  applied  in  samples  students 
from  the  graduate  courses  of  FIOCRUZ,  undergraduate  Medicine  students  from  Rio  de  Janeiro  Federal  University, 
as  well  as  in  opinion  and  suggestion  surveys  with  teachers  and  researchers  linked  to  the  teaching  of  the  health 
sciences.  This  work  will  bring,  as  final  result,  a product  of  multimedia  software,  available,  initially,  under  the  form 
of  CD-ROM  and,  after  the  revision  step,  at  the  Internet.  This  work  is  being  developed  with  the  purpose  of  offering  a 
facilitating  means  of  cognition  that  could  be  applied  as  an  optimizing  factor  in  the  acquisition  of  specialized 
knowledge  in  an  individual  and  pleasant  mode,  thus  favouring  the  introduction  of  new  environments  of  teaching/ 
learning  and  recycling  in  Parasitology. 

C.  WARNING  SIGNAL:  an  integrating  signal  between  environment  and  health.  Danielli  Grynszpan;  ElOMAR 
Christina  Vieira  Castilho  Barilli/  Oswaldo  Cruz  Fundation 

This  work  seeks  to  present  a dynamic  and  interactive  pedagogical  product,  which  might  contribute  to  the  raising 
of  the  consciousness  of  the  intrinsic  relationship  between  ecology  and  health.  Through  education  and  reflection  on 
the  triad  : human  being,  health  and  environment.  WARNING  SIGNAL  will  be  put  at  the  general  disposal  in 
The  Biodiscovery  Space,  that  belongs  to  the  Science  and  Technology  Museum,  The  Museum  of  Life,  that  is 
being  implanted  at  the  campus  of  FIOCRUZ.  This  work  presents  an  effort  made  towards  offering  teachers,  pupils 
and  other  citizens,  that  should  come  to  the  Museum,  a starting  point  for  the  exploration  of  themes  linked  to  health 
and  environment  that  would  lead  to  a basic  principle,  linked  to  both,  that  is  the  constant  fight  for  improvement  in 
the  quality  of  life.  This  work  still  intends  to  develop  in  the  user  a critical  posture  in  relation  to  his  rights  and  duties 
as  an  individual,  of  fighting  for  good  life  conditions,  for  himself  and  the  community.  Summing  up,  one  intends  the 
Warning  Signal  to  develop  in  the  visitor  of  the  Biodiscovery  Space  of  the  Museum  of  Life/FIOCRUZ  a holistic 
perception  of  the  touchstones  of  education  in  health  and  ecology 

J.  General  Conclusion  and  Perspectives 

The  multimedia  technology  was  selected  as  the  adequated  environment  to  implement  the  systems  because  permit  its 
utilization  for  people  without  previous  experience  in  the  use  of  computers  and  its  efficient  manipulation  of  large 
volumes  of  text  and  images.  At  the  present  stage  the  system  has  been  implemented  only  in  Portuguese.  These 
products,  at  first,  will  be  implemented  in  CD-ROM,  and,  later  (after  the  evaluation  methodology),  made  available 
through  Internet.  The  methodology  evaluation  will  be  carried  out  with  professionals  whose  specialities  are  linked  to 
the  theme  dealt  with  by  the  system  in  relation  to  the  technology  employed,  as  well  as  the  target  public  of  the  system, 
according  to  these  steps.’  preliminary  questionnaire  ( that  seeks  to  collect  information  on  the  user  experience  within 
the  theme  to  be  presented,  computational  experience  etc  ),  “ and  evaluation  questionnaire  ( collecting  of 

opinions,  observations  and  criticism  ).  The  intention  of  the  professionals  from  SDE/ENSP  is  that  it  should  not  only 
be  a place  that  would  produce  tools  to  sustain  and  amplify  cognition,  but  also  contribute  for  research  on  the  efficacy 
of  its  very  function:  that  of  the  knowledge  communication  and  integration  of  FIOCRUZ  researchers  with 
professionals  from  other  institutions.  Our  hope,  in  fact,  is  that  it  may  contribute  both  for  technical  - scientific 
development  and  for  the  educational  development  in  the  field  of  health. 
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SCALING  INFORMATION  LITERACY  AT  THE  UNIVERSITY  OF  IOWA; 

WEB-BASED  APPROACHES 


Barbara  I.  Dewey,  University  of  Iowa  Libraries,  University  of  Iowa,  U.S.A.,  barbara-dewey@uiowa.edu 


This  short  paper  discusses  issues  and  strategies  related  to  scaling  information  literary  programs  in  the  large 
university  setting.  The  University  of  Iowa  Libraries’  multi-fcrmat  approach  in  attempting  to  reach  more  of  its 
28,000  students  will  be  described  with  a focus  on  web-based  delivery  systems  developed  in  partnership  with 
Colleges  and  academic  programs. 


THE  NEED: 

• The  information  environment  is  too  complex  and  changing  too  rapidly  to  expect  students  to  acquire 
information  literacy  without  a planned,  systematic,  cumulative  instructional  program. 

• Students  must  learn  critical  thinking  and  research  skills  as  preparation  for  a lifetime  of  changing 
information  needs. 

• Effective  learning  about  information  retrieval,  use,  and  analysis  is  tied  to  a particular  information  need, 
often  discipline-specific. 

• Students  have  different  learning  styles  and  acquire  information  in  different  ways.  Any  information  literacy 
program  must  accommodate  these  differences  by  using  a variety  of  approaches  that  provide  practice  in 
these  skills. 

• The  most  effective  way  to  reach  students  is  through  collaboration  between  the  Libraries  and  academic 
departments  and  faculty  in  integrating  information  skills  into  the  curriculum  and  evaluating  outcomes 
through  a variety  of  means. 


INFORMATION  LITERACY  LEARNING  OBJECTIVES: 

• Identify  and  articulate  needs  that  require  information  solutions; 

• Identify  appropriate  information  sources  and  execute  search  strategies  appropriate  for  each  resource; 

• Interpret  and  analyze  search  results; 

• Critically  evaluate  the  information  retrieved; 

• Organize,  synthesize,  and  apply  the  information; 

• Understand  the  structure  of  the  information  environment  and  the  process  by  which  both  scholarly  and 
popular  information  is  produced  and  disseminated;  and 

• Understand  the  ethical  issues  related  to  access  and  use  of  information. 

TIER  ONE.-  A tiered  approach  that  builds  on  knowledge  acquired  throughout  the  students’  career  has  been 

developed  including  strengthening  the  current  introductory  research  and  information  seeking  components  in  two 

products: 

• Online  Iowa  - updating/enhancing  library  section  of  a general  university  CD-ROM  orientation  program 
that  provides  a basic  understanding  of  Main  Library  service  points  and  examples  of  where  to  search  for 
information  resources  (http://www.uiowa.edu~online/). 

• Library  Explorer  - strengthening  the  existing  partnership  where  library  staff  provide  instruction  on  how  to 
integrate  the  use  of  Library  Explorer,  a computer-based  library  instruction  program  with  automated  “quick 
tests  of  knowledge”,  other  information  sources,  search  strategies,  and  development  of  effective  research- 
related  assignments  for  teaching  assistants  within  their  training  program.  Librarians  are  subsequently  paired 
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one-on-one  with  Teaching  Assistants  to  provide  customized  support.  Extensions  underway  for  Library 
Explorer  include  subject-specific  “chapters”  for  different  disciplines  and  special  CD-ROM  “cuts”  from 
Library  Explorer  to  use  with  distance  education  students  who  do  not  have  access  to  the  Internet 
(http://www.lib.uiowa.edu/libexp/). 

TIER  TWO  - UTRIPLE  I:  Librarians  work  with  faculty  to  determine  a desirable  combination  of  instructional 
formats,  assignments,  and  outcome  evaluation  methods  building  on  experience  derived  from  current  and  past 
partnerships  with  GER  and  upper  division  course  instruction  for  tier  two  components  in  an  initiative  called 
UTriplel  (University  of  Iowa  Information  Literacy  Initiative)  for: 


5 General  Education  Requirement  Courses  — introductory  subject-based  information  literacy  component 
developed  with  faculty  and  built  into  selected  GER  courses  selected  from  the  Departments  of  English, 
Geography,  History,  Political  Science,  and  Psychology  including  query  formulation,  information  seeking 
strategies,  basic  evaluation  of  information  sources,  and  a basic  understanding  of  copyright  and  intellectual 
freedom  issues. 

• Science  Information  Literacy  Initiative  Project  - development  of  specific  information  literacy  components 
related  to  the  special  needs  of  science  disciplines  and  their  courses. 

• Undergraduate  "Majors”  Component  — advanced  and  more  complex  subject-based  information  seeking, 
retrieval,  and  analysis  components  built  into  selected  courses  for  undergraduate  majors. 

TIER  THREE  - FACULTY  TRAINING  AND  SUPPPORT:  Efforts  to  scale  information  literacy  efforts 

through  faculty  training  include  programs  for  on  the  application  of  learning  technologies  in  their  courses: 

• nTITLE  (New  Technologies  in  the  Teaching  and  Learning  Environment  - website  noted  below)  is  a 
summer  faculty  training  program  taught  largely  by  librarians.  The  Center  also  provides  input  into  the 
University  Libraries’  TWIST  (Teaching  with  Innovative  Style  and  Technology)  project 
(http://www.uiowa.edu/~ntitle). 

• TWIST  (Teaching  with  Style  and  Innovative  Technology)  is  a three  year  grant  funded  program  whose  major 
goal  is  to  create  a model  program  for  training  faculty  to  integrate  networked  information  into  teaching  as 
well  as  information  literacy  components.  TWIST  project  staff  are  sponsoring  a series  of  late  summer 
workshops  for  faculty  and  staff  to  explore  ways  of  fully  using  information  technology  in  instructional 
settings.  The  sessions  focus  on  building  learning  environments  for  students  - how  to  help  students  learn  to 
use  electronic  resources  via  OASIS  (the  University  Libraries’  online  catalog),  the  Web  or  CD  -ROM;  how 
to  build  instructional  Web  sites  to  guide  students  as  they  learn  critical  thinking  skills.  One  “scaling”  factor 
in  this  program  is  the  development  of  web-based  tutorials  for  faculty  to  use  at  anytime.  Another  scaling 
factor  is  the  development  of  a TWIST  “template”  for  faculty  to  develop  course  web  pages  with  an  emphasis 
on  linkages  to  resources  and  information  literacy-based  lessons  (http://twist.lib.uiowa.edu). 


SUMMARY 

Issues  and  challenges  to  scaling  efforts  with  be  reviewed  including  the  need  to  “mine”  partnerships  with  faculty 
and  technologists,  the  need  for  improved  marketing  of  the  inherent  benefits  of  integrating  an  information 
literacy  component  into  one’s  course,  the  difficulty  of  measuring  and  evaluating  the  success  of  a “scaled” 
information  literacy  program,  and  the  challenge  of  putting  together  a coherent  program  or  curriculum  in  an 
easily  translatable  modular  form. 
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Multimedia  Communication  - A National  Program  for  High  Schools 
to  Develop  Professional  Multimedia  Skills 


Ricardo  Dal  Farra 
Multimedia  Communication 
National  Institute  of  Technology  Education  (INET) 
National  Ministry  of  Culture  and  Education  - Argentina 
dalfarra@clacso.edu.ar  and  dalfarra@inet.edu.ar 
http://music.dartmouth.edu/^ricardo/index.html 


The  New  Educational  System  of  Argentina 

In  Argentina  we  are  living  an  advanced  stage  of  what  we  call  “educative  transformation”.  This  embraces  from 
student’s  early  years,  when  them  goes  to  the  General  Basic  Education  (GBE)  right  after  the  kindergarten,  and  includes  also  the 
high  school  level,  both  technical  and  non-technical. 

Our  new  system  introduce  9 years  of  general  education  organized  in  3 stages  of  3 years  each  (GBE  I,  GBE  II  and  GBE 
III),  while  the  previous  one  had  only  7 years.  After  this,  the  project  includes  Polimodal  Education  (PE),  another  3 years  where 
students  receives  more  advanced  general  education  while  at  the  same  time  follows  one  of  five  basic  orientations  we  call 
“modalities”,  that  can  be  chosen  between:  Communications  and  Art,  Management  and  Economy,  Natural  Sciences,  Production, 
Humanities  and  Social  Sciences. 

To  receive  a technical  diploma  is  possible  to  take  (parallel  to  the  “Polimodal  Education”)  one  of  a series  of 
“Technical-Vocational  Pathways”  (TVP)  pointing  directly  to  develop  professional  skills  around  different  professional  areas  of 
the  work  world  of  today. 

Technical-Vocational  Pathways 

The  education  by  competences  is  on  the  basis  of  all  the  “Technical-Vocational  Pathways”  developed.  These  programs 
were  created  considering  the  present  situation  of  the  educational  system  in  our  country  and  the  neccesities  for  the  future.  We 
worked  consulting  permanently  both  to  the  different  levels  of  the  educational  system  as  well  as  the  productive  world. 

Each  “Technical-Vocational  Pathways”  (TVP)  focuses  on  a work  area  of  the  real  world  and  the  professional  skills 
needed  there.  Among  the  TVPs  developed  are:  Informatics,  Agriculture,  Industrial  Process,  Organizations’  Management, 
Electronics,  Health  and  Environment,  and  Multimedia  Communication. 

Each  TVP  proposes  a modular  organization  where  each  module  focuses  on  certain  competences  related  to  the  field  of 
work  involved,  developing  a coherent  body  of  competences  to  receive  the  diploma  on  the  specific  technical  area  of  interest. 

TVPs  are  between  1200  and  1800  hours  long,  and  the  modules  of  their  structure  can  be  organized  in  three  years, 
following  steps  of  progressive  complexity. 

A partial  option  to  the  technical  diploma  are  the  “Itineraries”,  when  students  takes  only  certain  modules  limiting  the 
general  field  of  study  to  develop  only  a group  of  competences  coherent  with  an  specific  trimming  of  the  field  focused  by  the 

TVP. 

Multimedia  Communication 

The  “Technical- Vocational  Pathway  on  Multimedia  Communication”  have  been  developed  by  a small  group  of 
multimedia  producers,  artists  and  pedagogues  working  since  1996  on  the  program. 

A national  program  on  multimedia  communication  for  high  schools  in  our  country  needs  to  solve  several  problems  to 
start,  being  faculty  training  on  new  areas  of  knowledge  and  work,  and  the  equipments  required  Just  some  of  them.  Also  must  be 
considered  the  social  situation  of  a very  large  country  where  almost  half  of  the  population  is  concentrated  on  the  capital  city 
(Buenos  Aires),  thinking  that  while  there  are  small  to  big  companies  based  there  around  the  wide  multimedia  market,  mainly 
small  groups  are  working  on  this  field  on  the  rest  of  the  geography.  So  related  areas  to  multimedia  should  be  considered  into  the 
project’s  structure  considering  the  employment  opportunities  in  small  cities  while  the  specific  multimedia  field  grows  there. 
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The  modular  structure  is  built  with  16  modules  of  96  hours  each,  plus  one  of  144  hours  (total  = 1584  hours).  These 
modules  are  organized  in  6 modular  areas:  Script,  Images,  Sound  and  Music,  Multimedia,  Management  and  Technologies  that 
groups  them. 

While  the  other  “Technical-Vocational  Pathways”  are  only  technically  oriented,  this  program  joins  art  and  sciences 
concepts  with  direct  experiences  on  new  technologies.  Not  all  modules  on  Multimedia  Communication  looks  for  specific 
competences,  some  of  them  on  the  first  level  of  complexity  (=  first  year)  serve  as  an  analytical  introduction  with  artistic 
elements  to  the  visual  and  the  sound/music  world  (Visual  Morphology,  Sound  Morphology),  while  others  (Science  and 
Technology  Elements)  put  the  basis  to  understand  many  processes  involved  on  modules  of  the  second  and  third  level.  Modules 
like  Script,  Desktop  Publishing,  Image  Synthesis  and  Animation,  Audiovisual  Realization,  Sound  Technology,  Production, 
Multimedia  Introduction,  Closed  Multimedia  Systems,  Open  Multimedia  Systems,  Management  and  Marketing  of  Micro- 
Enterprises  and  Equipments  Ivlainienaiiee  focuses  on  specifc  competences  of  the  field.  On  the  last  step  nf  complexity, 
Integration  Project  and  Training  on  Work  Centers  modules  looks  both  for  coordination  and  integration  of  efforts  as  well  as  for 
complex  experiencies  useful  on  the  real  world. 

About  the  modules,  there  are  no  breaks  between  theory  and  practice  on  them;  learning  is  taken  as  a process  where 
different  aspects  are  necessary  to  reach  the  knowledege  and  skills  to  work  on  the  multimedia  field,  Basic  science  concepts  are 
important,  as  well  as  practice  with  suitable  technologies  to  solve  real  world  problems  using  a creative  approach.  About  each 
module  itself:  on  the  first  level  of  complexity.  Script  starts  with  traditional,  linear  scripting  for  different  medias,  goes  through 
parallel  and  multiple  stories,  ending  on  hypermedia  scripting;  Visual  Morphology  takes  both,  fixed  images  and  moving  images, 
approaching  also  the  relationships  between  images  and  sound;  Sound  Morphology  focuses  on  sound  and  music  perception, 
analysis  and  creation;  Multimedia  Introduction  goes  from  the  perceptual,  aesthetical  and  technical  analysis  of  multimedia 
projects  to  the  production  of  basic  hypermedia  works;  Science  and  Technology  Elements  works  over  the  basis  of  light,  sound, 
electricity  and  electronics  to  understand  processes  learned  and  technologies  used  through  the  TVP;  on  the  second  level  of 
complexity.  Desktop  Publishing  is  focused  on  the  visual  articulation  between  text  and  fixed  images;  Image  Synthesis  and 
Animation  takes  both,  the  2 and  3D  worlds;  Audiovisual  Realization  works  over  the  full  process  of  capturing  and  editing  real 
moving  images  and  their  integration  with  the  sound  world;  Sound  Technology  is  focused  on  synthesis,  recording,  processing  and 
mixing  of  sounds  and  music;  Closed  Multimedia  Systems  points  to  multimedia/hypermedia  products  where  information  flows 
mainly  inside  closed  environments  of  use  (as  in  most  CD-ROMs  products);  Production  is  focused  on  the  management  of  each 
project  step;  and  on  the  third  level  of  complexity.  Open  Multimedia  Systems  points  to  multimedia/hypermedia  products  where 
information  flows  on  open,  distributed  environments  of  use  (WWW);  Management  and  Marketing  of  Micro-Enterprises  points 
specifically  to  those  wishing  to  start  their  own  business;  Equipments  Maintenance  is  planned  to  help  to  identify  and  solve 
simple  computer  and  AV  fails;  Integration  Project  is  where  the  more  complex  and  interdisciplinary  projects  happens;  and 
Training  on  Work  Centers  put  students  on  the  real  world  of  work  so  they  can  make  their  own  experiences,  using  this  to  analyze 
techniques,  processes,  products  and  situations  to  help  them  to  progress  from  there  and  complete  this  stage  of  learning. 

Together  with  the  16  modules,  students  needs  to  take  some  subjects  from  the  “Polimodal  Education”  too: 
Communication,  and  English  I-II-III,  It  must  be  considered  here  that  in  spite  is  mandatory  to  receive  a “Technical- Vocational 
Pathways”  diploma  to  complete  also  the  “Polimodal  Education”,  there  are  some  electives  subjects  on  this  system,  making 
neccesary  to  remark  those  two  areas  (Communication  and  English)  as  essentials  to  fufil  the  goals.  While  in  Argentina  the 
official  language  is  Spanish,  students  could  study  French,  Portuguese,  English,  German  or  Italian  as  foreign  language  on  the 
high  school,  thus  making  necessary  for  us  to  take  a decission  regarding  which  is  the  language  needed  at  present  for  those 
starting  on  the  multimedia  field:  (at  least,  three  levels  of)  English.  About  the  high  significance  of  Communication  for  this  TVP, 
it  must  be  remarked  that  settle  fundamental  (communication)  concepts  needed  for  most  of  the  modules  of  the  second  and  third 
level  of  complexity. 

As  a shorter  option  to  the  technical  diploma  on  Multimedia  Communication,  students  can  receive  a certificate  on 
Desktop  Publishing,  Image  Synthesis  and  Animation,  Audiovisual  Realization  or  Sound  Recording  and  Processing  approving 
only  certain  modules  to  develop  a group  of  competences  coherent  with  the  named  specific  fields  (“Itineraries”),  As  an  example, 
to  receive  a basic  certification  on  Image  Synthesis  and  Animation  (an  “Itinerary”  with  a total  of  576  hours)  students  must  to 
approve  the  following  modules:  Script,  Visual  Morphology,  Image  Synthesis  and  Animation,  Multimedia  Introduction,  Science 
and  Technology  Elements  and  Equipment  Maintenance;  in  addition,  is  highly  recommended  (not  mandatory  as  for  the  full 
technical  diploma)  to  take  the  Communication,  and  English  I-II-III  courses. 


This  program  starts  in  public  schools  of  Argentina  in  March  1999. 
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Introduction 


T171  You,  your  computer  and  the  Net.  is  a distance  education  course  presented  at  the  Open  University  in  the  UK.  The  course  was  piloted  in  1999  with  900 
students.  It  is  a foundation  course,  which  provides  a grounding  in  cuiiipuici'  skills  auu  knowledge  of  the  persona!  eomputer  and  the  Internet.  !t  is  a broad 
appeal  course  aimed  not  just  at  Technology  students,  but  students  of  all  Faculties,  as  well  as  a general  audience  who  may  only  take  this  one  course.  The 
course  is  presented  entirely  over  the  Internet,  with  most  of  the  material  being  web  based,  and  extensive  use  is  made  of  computer  conferencing. 


The  course  was  developed  to  reflect  a Kolbian  iterative  learning  model  [1],  which  had  been  used  successfully  in  other  OU  courses  [2].  The  model  has  the 
following  three  main  phases: 


• Conceptualisation  - the  learner  receives  information  about  other  people’s  concepts; 

• Construction  - the  learner  uses  such  concepts  to  perform  meaningful  tasks; 

• Dialogue  - the  learner  tests  existing  concepts  and  creates  new  ones  through  dialogue  with  other  learners. 

With  this  model  as  a basis,  and  the  following  aims,  the  course  structure  could  be  developed.  The  initial  aims  of  the  course  team  were  as  follows; 

1 . To  produce  a large  scale,  broad  appeal  ICT  course 

2.  To  make  the  Internet  the  central  delivery  mechanism 

3.  To  produce  a course  in  a relatively  short  time  scale 

4.  To  make  students  feel  comfortable  in  the  ‘wired  world’ 

5.  To  bring  the  OU's  traditional  standards  of  production  to  the  Internet. 

There  is  an  initial  face  to  face  session,  which  operates  as  an  introduction  to  the  University  as  a whole,  and  to  distance  learning.  This  is  not  course  specific, 
and  the  emphasis  on  face  to  face  was  kept  to  a minimum  in  order  to  utilise  the  advantages  of  computer  mediated  communication  (CMC)  tuition  for 
distance  learners  with  diverse  lifestyles. 


Course  structure 

T171  is  based  around  three  modules,  two  of  which  have  a set  book.  The  web  site  is  then  used  to  add  academic  material  exploring  issues  raised  in  a chapter 
of  the  book.  Students  also  have  access  to  a number  of  conferences.  The  modules  are  outlined  below: 


Module  1 You:  Computing  with  confidence 

This  starts  from  the  beginning  and  assumes  that  at  least  some  students  are  completely  new  to  computing.  It  covers  general  computer  skills  such  as:  the 
basic  operating  principles  of  a PC;  the  Windows  interface;  an  introduction  to  standard  office-type  programs  for  word-processing,  database  operations, 
spreadsheet  calculations  and  simple  charts  and  graphs;  the  use  of  modems,  electronic  mail  and  conferencing;  connecting  to  the  Internet;  using  a Web 
browser;  searching  the  World  Wide  Web  on  specific  topics;  planning,  designing  and  authoring  Web  pages;  Netiquette  and  principles  of  online  behaviour. 

Module  2 Your  Computer:  The  story  of  the  PC 

This  module  is  based  around  the  story  of  how  the  personal  computer  came  to  be  one  the  dominant  technologies  of  our  age.  It  starts  with  the  invention  of 
the  microprocessor  and  leads  up  to  the  present  dominance  of  Microsoft.  It  is  based  around  an  accessible  set  book  [3]  and  a web  site  containing  a large 
amount  of  additional  material,  study  guides,  links,  resources  and  assessment  material. 

The  module  looks  at  the  key  technologies  involved  with  the  PC  such  as  the  microprocessor  and  the  operating  system  and  explains  their  importance  and 
how  they  work.  It  also  looks  at  the  social  implications  of  the  IT  revolution,  the  nature  of  the  computer  industry  and  the  reasons  behind  the  success  and 
failure  of  computer  products. 

Module  3 The  Net:  Where  it  came  from,  what  it  is 

This  covers  the  evolution  of  the  Internet  from  its  origins  in  the  inter-war  years  through  to  the  explosive  growth  of  the  Woiid  Wide  Web  in  the  1 990s.  It  is 
again  based  around  a set  book  [4]  and  a web  site. 

The  Module  covers  the  history  of  the  Net  in  four  stages: 

• Prehistory:  the  evolution  of  ideas  about  computing  and  communications  from  the  1930s  to  the  1960s 

• The  ARPANET:  the  creation  of  the  first  packet- switched  wide  area  network 

• From  ARPANET  to  Internet:  the  metamorphosis  of  the  original  ARPANET  concept  into  the  'global  network  of  networks'  which  constitutes 
today's  Internet 

• The  World  Wide  Web:  the  evolution  of  the  Web  from  its  invention  in  1 989  to  its  current  form. 
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Module  1 is  largely  activity  based,  and  thus  incorporates  both  the  conceptualisation  and  construction  phases  of  the  learning  model  within  the  material. 
Modules  2 and  3 are  more  content  based.  In  this  respect,  the  web  based  material  represents  the  conceptualisation  phase.  In  order  to  make  it  active  for  the 
student  it  is  necessary  for  them  to  engage  with  the  material  in  a manner  which  personalises  it  for  them.  This  was  achieved  through  the  use  of  a study 
journal,  as  well  as  assessment..  All  modules  need  to  make  use  of  a dialogue  stage,  whereby  students  test  their  understanding  of  a concept  against  that  of 
others.  This  is  done  through  the  use  of  computer  conferencing.  The  course  thus  makes  extensive  use  of  conferences  throughout  the  material. 

The  Study  Model 

One  of  the  problems  with  any  material  which  is  studied  at  a distance  is  that  of  pacing.  In  order  for  students  to  gain  the  maximum  benefit  from  the  material 
they  need  to  work  through  all  of  it,  but  when  all  of  the  material  is  available  there  can  be  a temptation  to  rush  through  it.  In  the  traditional  distance  learning 
model  this  has  been  accommodated  with  regular  mailings,  timing  of  key  events  (such  as  assignment  submission  and  so  forth).  Similarly  distance  education 
must  deal  with  the  issue  of  ‘information  overload’.  Level  1 students  may  be  studying  for  the  first  time,  or  returning  to  study  after  a long  time.  They  thus 
need  to  be  made  to  feel  comfortable  with  the  material,  and  what  is  required  of  them.  These  two  considerations  led  us  to  develop  the  following  study 
model,  which  is  broadly  analogous  to  tlie  traditional  model. 

The  web  site  is  revealed  as  students  progress  through  the  course.  As  students  have  different  demands  on  their  time,  it  is  necessary  to  make  material 
available  which  is  ahead  of  the  current  study  schedule.  Thus  material  is  available  up  to  four  weeks  ahead  of  where  students  are  currently  recommended  to 
be.  This  pacing  is  reinforced  through  the  use  of  study  guides,  which  are  e-mailed  as  an  attachment  to  students  every  two  weeks.  These  study  guides 
provide  a detailed  breakdown  of  exactly  what  is  required  of  students  during  the  specified  study  period. 

Assessment 

The  assessment  of  T171  is  based  around  four  tutor  marked  assignments  (TMAs)  and  an  end  of  course  assessment.  All  of  these  are  submitted 
electronically.  The  first  two  TMAs  occur  in  module  one  and  are  based  on  collections  of  exercises  students  perform  during  the  module.  The  last  two  TMAs 
occur  at  the  ends  of  modules  two  and  three,  and  are  web-based  reports  which  explore  key  issues  raised  during  the  modules. 

The  final  assessment  is  based  around  a web  site  which  students  construct.  There  are  two  key  components  to  this.  The  first  is  a portfolio  of  material 
produced  during  the  course.  This  includes  examples  of  the  study  journal,  reflection  on  one  TMA,  conference  messages  and  so  forth.  It  thus  represents  a 
way  of  reinforcing  the  importance  of  these  skills  to  students.  The  second  component  is  a scenario  based  web  site,  which  requires  students  to  use  material 
from  all  three  modules,  and  with  modules  two  and  three  in  particular  examine  similarities  and  differences  in  the  two  stories.  This  encourages  them  to  use 
new  skills  they  will  have  developed  such  as  writing  for  the  web,  page  layout,  website  structure  and  use  of  links,  rather  than  the  more  traditional  essay  type 
assessment. 

Study  Skills 

As  a Level  1 course  within  the  OU,  T171  has  a role  to  perform  in  developing  student’s  general  study  skills  which  will  help  them  with  any  further  study.  As 
part  of  its  aims  to  produce  students,  who  are  comfortable  in  using  and  dealing  with  the  new  technologies,  it  also  needed  to  develop  skills  which  would 
allow  students  to  use  computer  tools  effectively.  Module  one  was  based  around  a generic  method  of  teaching,  that  is  it  was  not  specific  to  one  software 
package.  Students  are  thus  provided  with  information  and  activities  germane  to  all  brands  of  tlie  broad  software  type,  e.g.  spreadsheets.  The  skills 
developed  are  those  of  transferable  ones  between  different  software  packages. 

General  study  skills  are  developed  in  module  two  through  the  use  of  exercises  in  the  material.  These  include  skills  such  as  note  taking,  critical  reading, 
producing  effective  explanations  and  so  on.  The  requirement  of  the  end  of  course  assessment  is  that  students  provide  evidence  of  having  completed  at  least 
some  of  these  exercises.  These  exercises  represent  the  traditional  study  skills  which  any  Level  1 course  has  to  address  to  some  degree. 

Skills  which  have  not  been  developed  in  other  courses,  but  which  will  become  increasingly  important  for  students  are  developed  in  module  three.  These 
include  finding  and  evaluating  information  on  the  Internet,  organising  large  amounts  of  information  through  the  use  of  hyperlink  documents,  effective 
writing  for  the  web,  and  so  forth. 

Conclusions 

T171  was  produced  in  one  year,  compared  with  the  traditional  production  cycle  of  three  to  four  years  in  the  OU.  It  has  proved  extremely  popular  with  a 
broad  range  of  students,  many  of  them  new  to  the  University.  Estimated  student  numbers  for  2000  are  in  excess  of  5,000.  It  has  managed  to  combine  many 
of  the  successful  elements  of  traditional  OU  course  production  with  the  benefits  offered  by  the  new  technology. 

The  course  uses  a three  stage  learning  model,  which  encourages  students  to  engage  with  the  learning  material,  and  to  discuss  this  in  computer  conferences. 
The  course  uses  accessible  set  books  as  the  basis  for  teaching  and  adds  academic  material  to  these  in  the  form  of  a web  site.  Students  are  directed  through 
this  by  means  of  explicit  study  guides.  Their  contact  with  their  tutor  is  mainly  through  a tutor  group  conference. 

A number  of  study  skills,  both  new  and  traditional  are  developed  through  the  use  of  exercises.  These  are  reinforced  by  inclusion  in  an  end  of  course 
assessment. 

The  advantages  of  this  method  of  presentation  are  that  it  allows  course  teams  to  present  up  to  date  courses  in  a rapidly  changing  field.  It  gives  students 
experience  in  using  the  new  technologies,  and  allows  them  to  access  the  material  anywhere  they  can  find  an  Internet  connection.  Problems  of  attendance  at 
face  to  face  tutorials  are  also  removed  by  using  online  methods. 

Alternative  software  methods  are  currently  being  investigated.  A full  scale  evaluation  of  the  tutor  and  student  experience  is  being  performed,  which  will 
influence  alterations  to  the  course  material  for  presentation  in  2000. 
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The  magic  lantern,  film-strips,  tape  recorders,  16mm  films,  teaching  machines,  audio  labs,  television,  video- 
discs, and  teleconferencing  have  all,  at  one  time  or  another,  been  touted  as  technologies  that  would  have  a 
significant  impact  on  education  (Saettler,  1990).  The  history  of  technology  in  education  contains  extensive 
documentation  of  the  reasons  why  such  innovations  have  failed  or  have  had  limited  success.  These  reasons  are 
as  varied  as  the  technologies  themselves.  However,  there  are  common  threads  running  through  all  of  the 
attempts  to  transform  education  through  the  use  of  technology.  The  factors  related  to  success  or  failure  are  well 
known  and  include  administrative  support  and  understanding  of  technology,  teacher  training,  sufficient  funds 
and  release  time,  and  effective  technical  and  instructional  support.  Today  the  technology  that  is  receiving  the 
most  attention  is  the  computer.  The  question  is,  will  this  technology  achieve  the  success  that  other  high-profile 
innovations  of  the  past  have  not? 

This  short  paper  on  a work  in  progress  responds  to  the  lack  of  information  about  real-world  situations  where 
teachers  are  trying  to  apply  computer  technology  in  instruction.  It  is  based  on  a case  study  that  documents  the 
experiences  of  Dr.  John  Adams,  a community  college  English  instructor,  who,  over  a period  of  five  years, 
progressed  from  teaching  composition  in  a computer  laboratory  setting  to  teaching  seven  classes  on-line.  It  is  a 
record  of  the  experiences  that  he  had  dealing  with  the  challenges  of  applying  computer  technology  in  his  classes 
with  little  administrative  support,  almost  no  training,  little  technical  or  instructional  support,  and  no  release  time 
or  financial  assistance  — the  same  factors  that  limited  the  success  of  previous  educational  technologies. 

In  the  1993-94  semester,  his  first  at  his  present  position,  Adams  was  asked  by  his  division  chair  to  teach  one 
class  each  semester  in  a computer  lab.  The  lab  contained  24  PCs  placed  on  regular  straight  tables  arranged  in 
four  rows  with  six  computers  to  a row.  The  computer  screens  were  facing  away  from  Adams  (i.e.,  not  visible  to 
Adams);  the  room  was  also  used  for  storing  discarded  miscellany  from  other  rooms.  For  his  classes,  Adams 
used  the  tools  (handouts,  reading  book,  test/quizzes,  etc.)  and  methods  (lecturing,  answering  questions,  grading 
papers  outside  of  class)  that  he  used  in  the  traditional  classroom.  Adams  expressed  a high  degree  of  frustration 
with  the  computer  lab  setup.  He  felt  that  it  was  a major  hindrance  to  communicating  with  the  students,  and  he 
missed  the  conferencing  orientation  that  he  was  accustomed  to  in  his  other  classes. 

In  the  fall  of  1996,  Adams  began  putting  his  course  materials  on  the  Internet,  with  the  assistance  of  his  wife. 
These  initial  materials  were  copies  of  the  ones  he  was  using  in  his  classes.  Then,  in  the  fall  of  1997,  Adams 
negotiated  with  representatives  of  Old  Dominion  University’s  Teletechnet  program  to  teach  advanced 
composition  on  the  Internet.  For  this  class,  he  not  only  had  to  create  a completely  new  course  (for  him),  but  one 
that  was  appropriate  for  on-line  use.  In  addition,  that  same  semester,  he  gave  his  freshman  composition 
students  the  alternative  of  being  taught  in  a traditional  classroom  setting  or  via  the  Internet  (with  an  option  for 
one  class  meeting  per  week).  Most  students  opted  for  the  Internet-based  course.  Teaching  these  courses  on  the 
Internet  involved  revising  old  materials,  creating  new  materials,  and  getting  everything  organized  effectively 
and  put  on  the  Internet.  It  also  required  Adams  to  develop  a new  method  of  dealing  with  students,  students 


whom,  for  the  most  part,  he  never  met  face-to-face.  And  it  meant  learning  different  organizational  skills  to 
keep  track  of  the  e-mail  messages  that  he  was  receiving  from  his  students. 

With  the  1 998  summer  session,  Adams  taught  all  four  of  his  classes  via  the  Internet  only.  This  included  one 
section  each  of  college  composition  I,  college  composition  II,  advanced  composition,  and  short  fiction.  The 
first  three  courses  required  a number  of  revisions  and  additions  to  the  course  websites  to  make  them  effective  as 
Internet-only  courses.  For  the  short  fiction  course,  however,  Adams  had  to  create  and  organize  a completely 
new  set  of  materials  to  be  put  on  the  Internet.  During  the  fall  1998  and  spring  1999  semesters,  Adams  taught 
seven  sections  of  courses  on-line.  These  included  the  same  four  classes  that  he  had  taught  during  the  summer  as 
well  as  African-American  Literature  I (fall  1998)  & II  (spring  1999)  and  a technical  writing  course  (both 
semesters).  Once  again,  for  these  three  new  courses,  Adams  had  to  create  and  organize  the  materials  and  assist 
his  wife  in  the  process  of  creating  the  webpages  for  the  course  websites. 

Administrative  support  is  a critical  element  to  the  success  of  any  academic  endeavor.  In  Adams’  situation,  this 
support  has  been  negligible.  The  first  semester  that  Adams  taught  in  the  computer  lab,  he  felt  that  the 
administrative  position  was,  in  essence,  “There’s  the  lab.  Use  it.”  As  long  as  there  were  no  complaints  from 
students,  the  administration  seemed  to  be  satisfied.  Over  the  course  of  several  years,  the  administration 
continued  to  request  that  Adams  use  the  computer  lab  for  classes,  but  they  expressed  a strong  opposition  to 
Internet  classes,  Adams  felt  that  this  opposition  was  based  in  part  on  the  assumption  that  students  could  only 
receive  a quality  education  in  the  classroom.  Adams  felt  that  another  factor  was  a lack  of  knowledge  about 
computers  on  the  part  of  top-level  administrators.  Until  the  fall  of  1 997,  none  of  them  had  a computer  on  their 
desks,  and,  as  a result,  they  had  a limited  understanding  of  how  computers  operate  or  of  the  potential  of 
computers  in  education.  Because  of  their  opposition  to  Internet  courses,  Adams  established  his  Internet  classes 
surreptitiously,  with  only  spoken  approval  from  his  division  chair.  Not  until  the  summer  of  1998,  with  very 
positive  enrollment  results  from  his  on-line  classes,  did  they  begin  to  publicly  state  their  acceptance  of  such 
courses. 

Technical  and  instructional  support  is  also  critical  to  the  effective  application  of  educational  technology. 
Unfortunately,  this  has  been  lacking  in  Adams’  experience.  During  the  semesters  that  he  was  teaching  in  the 
computer  lab,  technical  support  was  minimal,  just  basic  maintenance.  When  he  moved  to  Internet  classes,  the 
situation  did  not  improve.  While  the  technician  on  staff  has  the  qualifications  to  handle  the  responsibilities  of 
computer  maintenance  and  those  of  web  master,  he  does  not  have  a background  in  education.  As  such,  he  is 
unable  to  provide  effective  instructional  support  on  the  applications  of  computer  technology. 

Adams  has  received  neither  release  time  nor  financial  assistance  from  his  college  for  his  work  related  to  the 
application  of  computers  in  his  instruction.  Since  the  fall  of  1 996,  Adams  has  developed  materials  for  seven  on- 
line courses,  and  with  his  wife’s  assistance,  has  created  and  updated  a comprehensive  website  with  these 
materials  and  links  to  other  resources.  Adams  purchased  his  own  laptop  computer  and  zip  drive  because  of 
problems  with  his  office  computer  (it  could  not  handle  the  workload  and  broke  down;  after  it  was  fixed,  it  was 
given  to  another  instructor).  He  also  purchased  critical  software  and  instructional  manuals. 

Year  after  year,  the  literature  on  instructional  technology  has  documented  the  problems  and  has  provided  the 
solutions.  However,  in  the  real  world  of  education,  the  problems  are  often  not  recognized  by  the  people  in 
positions  of  authority  and,  thus,  the  appropriate  solutions  are  not  applied.  At  the  end  of  one  interview,  Adams 
commented  that,  in  his  present  situation,  he  spends  all  of  his  time  maintaining  what  he  has  developed  so  far,  and 
that  he  has  no  time  to  keep  up  with  new  advances  in  technology.  As  a result,  he  feels  he  that  is  actually  falling 
behind  rather  than  progressing.  Unless  there  is  sufficient  administrative  support,  adequate  teacher  training, 
effective  technical  and  instructional  support,  and  appropriate  compensation  through  release  time  and  financial 
assistance,  those  who  are  trying  to  advance  the  effective  use  of  computer  technology  in  education  will  also  find 
themselves  falling  behind  rather  than  progressing. 
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Introduction 

Current  technology  is  now  available  that  can  remove  the  restriction  of  time  from  the  delivery  of  classroom  lectures. 
By  utilizing  Microsoft’s  PowerPoint  software  along  with  RealNetwork’s  RealPresenter  plug-in,  any  school  can 
easily  provide  a low  cost,  efficient  way  to  offer  recorded  lectures  over  the  Internet.  With  the  emphasis  on  life  long 
learning  and  the  increase  in  non-traditional  students  in  classrooms,  Internet  accessible  lectures  let  students  “attend”  a 
class  lecture  that  they  may  have  missed  due  to  shift  change,  unexpected  overtime,  or  an  out-of-town  business  trip. 
This  gives  them  the  flexibility  to  listen  to  any  recorded  lecture  at  their  convenience,  from  a computer  lab  on  campus 
or  from  any  Internet  accessible  computer. 


The  Web  Enhanced  Audio  System 

During  Winter  Quarter,  1999,  North  Central  Technical  College  offered  recorded  lectures  from  seven  different 
courses  using  the  Web  Enhanced  Audio  System.  Using  conventional  portable  audio  tape  recorders,  lectures  were 
recorded  and  then  processed  through  a computer  into  single-slide,  PowerPoint  presentations,  with  up  to  one  hour  of 
lecture  embedded  into  the  slide.  NCTC  already  had  a standard  web  server  and  network  on  campus  and  were  also 
using  Microsoft’s  Office  97  software.  Adding  to  this,  the  College  purchased  RealPresenter  software  ($39)  and  a 
number  of  portable  audiocassette  recorders  ($50  each).  The  only  other  hardware  used  was  an  audio  cassette  player 
for  playback  into  the  computer  ($70).  With  an  outlay  of  less  than  $500,  the  school  was  able  to  offer  asynchronous 
delivery  of  course  material  via  the  Internet. 

There  seems  to  be  two  major  roadblocks  that  hinder  the  development  of  Internet  accessible  course  material:  time  and 
money.  This  Web  Enhanced  Audio  System  was  designed  to  overcome  these  two  obstacles.  The  nominal  amount  of 
hardware  required,  assuming  a school  has  an  existing  web  server  and  Office  97  software,  eliminates  the  financial 
portion  of  the  problem.  And  the  other  roadblock  is  finding  the  time  and  expertise  to  create  or  develop  web 
accessible  material.  The  Web  Enhanced  Audio  System  solves  this  by  not  requiring  instructors  to  have  computer 
expertise,  nor  burdening  them  with  additional  class  preparation,  aside  from  wearing  a recorder  during  class.  A web 
course  coordinator  can  perform  all  of  the  processing  and  must  simply  have  a working  knowledge  of  PowerPoint  and 
be  familiar  with  standard  audio  equipment. 


The  Process 

Once  a lecture  is  recorded,  it  is  sent  to  the  web  course  coordinator  and  downloaded  into  a one-slide  PowerPoint 
presentation  using  the  “Record  Narration”  function.  The  single  slide  identifies  the  course  number,  date,  and  lecture 
subject.  The  software  captures  the  audio  lecture  as  large  wave  file  (up  to  70  megabytes  for  a typical  one-hour 
lecture).  The  RealPresenter  plug-in  software  then  compresses  the  lecture  (about  2 megabytes  per  lecture)  and 
creates  a web  page  suitable  for  publishing  onto  a standard  server.  A Table  of  Contents  web  page  is  created  for  each 
course  and  linked  to  the  instructor’s  home  page.  The  Table  is  then  updated  with  the  addition  of  each  new  lecture, 
acting  as  a directory  for  locating  class  meetings  by  date.  To  access  a lecture,  the  students  simply  go  to  their 
instructor’s  web  page,  click  the  appropriate  audio  course  button,  and  select  the  date  of  the  desired  lecture.  Students 
can  access  their  lectures  from  any  campus  computer  lab  by  plugging  head  phones  into  the  computer  speakers,  or 
from  their  own  home,  at  their  convenience. 


Benefits 


There  are  many  benefits  to  this  Web  Enhanced  Audio  System.  The  low  start-up  cost  in  hardware  and  software  for 
any  college  that  already  has  a website  makes  this  system  possible  without  causing  a major  impact  on  the  budget. 
The  level  of  expertise  for  instructors  is  limited  to  the  ability  to  push  the  record  button  on  a portable  tape  recorder. 
The  level  of  expertise  for  the  Web  Course  Coordinator  is  limited  to  knowledge  in  PowerPoint  software,  simple  web 
page  creation  and  revision,  and  some  audio  equipment  expertise.  In  addition,  the  impact  on  the  web  server  and 
network  infrastructure  is  minor  due  to  the  gradual  increase  of  storage  requirements  and  activity  as  the  quarter 
progresses.  And  the  benefit  to  students  is  the  removal  of  the  time  barrier  associated  with  traditional  class  schedules. 
Plus,  they  can  use  this  material  for  review  purposes  as  well. 


New  Advances 

One  drawback  to  this  system  had  been  the  slow  download  time  found  with  traditional  audio  tape  recording  media. 
With  the  availability  of  new  portable  digital  recorders,  even  the  transfer  of  recorded  media  has  been  reduced  for  a 
“real  time”  45 -minute  lecture  down  to  about  4 minutes.  Digital  recorders  are  now  available  that  can  record  up  to 
two  hours  on  a flash  memory  card,  which  can  be  handled  like  any  other  electronic  file,  and  can  be  converted  into  a 
wave  file  in  a matter  of  minutes. 

The  level  of  web  enhanced  lecture  can  also  be  upgraded  by  adding  pictures  or  text  to  the  web  pages  to  be  seen 
during  the  lecture.  In  addition,  instructors  already  using  PowerPoint  slide  shows  during  class  can  include  the  entire 
slide  show,  time-encoded  so  that  it  appears  on  the  web  just  like  it  was  given  in  class.  The  software  used  in  this 
system  could  even  be  used  to  embed  video  of  a process  or  procedure  into  the  slide  show  if  desired. 


Conclusion 

The  Web  Enhanced  Audio  System  is  a practical  and  low  cost  method  of  placing  lecture  material  onto  the  Web. 
Using  cutting  edge  tools  that  are  inexpensive,  coupled  with  a technique  that  is  time  efficient  and  extremely  user 
friendly,  any  school  can  develop  a web  presence  without  the  typical  obstacles  associated  with  new  ventures.  The 
Web  Enhanced  Audio  System  can  also  be  used  to  augment  independent  study  courses  and  can  become  the  first  step 
in  developing  complete  online  courses  at  any  college.  The  insignificant  impact  on  faculty’s  time  and  the  low 
hardware  cost  make  it  a practical  and  economical  method  of  converting  and  delivering  lecture  material  over  the 
Internet. 
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This  paper  outlines  the  context  and  rationale  for  a piece  of  curriculum  development  at  the  University  of 
Brighton  which  is  also  the  site  for  collaborative  research  by  a teacher  educator  and  a learning 
technologist.  The  development  relates  to  'Developing  Reading  and  Writing,’  a first  year  module  on  an 
English  education  degree  for  intending  high  school  English  teachers. 

The  aims  of  the  module  are:  i.  to  investigate  literacy  as  a sociocultural  practice;  ii.  to  investigate  the 
ways  in  which  technology  changes  both  the  nature  of  text  and  the  ways  texts  are  produced;  and  iii.  to 
explore  the  pedagogical  possibilities  that  would  enable  high  school  teachers  of  English  to  use  digital 
technologies  to  enhance  pupils’  learning.  The  context  for  the  development  includes  a commitment  to  a 
pre-existing  and  successful  model  of  a learning  environment  for  English/Language  Arts  education 
shared  by  the  development  team  and  the  recent  curriculum  prescription  contained  in  the  UK 
Department  for  Education  and  Employment’s  Initial  Teacher  Training  National  Curricula  in  Secondary 
English  and  the  use  of  Information  and  Communications  Technology  (DfEE  1998).  The  current 
development  is  an  evolution  of  previous  work  at  the  university  and  not  a reaction  to  this  prescription. 
The  aims  of ’Developing  Reading  and  Writing’  have  remained  consistent  for  the  last  two  years. 

The  shared  model  of  a learning  environment  for  English/Language  Arts  can  best  be  described  with  the 
metaphor  of  the  atelier,  a place  for  artisanal  activity.  This  metaphor  was  explored  by  one  of  the 
authors  in  previous  collaborative  work  with  Claire  Woods  and  Patrick  Dias  (Woods  et  al.  1996)  and,  in 
relation  to  learning  about  literacy,  is  useful  in  conveying  a sense  of  a shared  purpose  and  of  the 
dependence  and  interdependence  of  the  artisans  who  work  within  it.  The  metaphor  of  the  atelier  allows 
for  notions  of  expertise  in  a learning  environment,  expertise  that  is  shifting  and  dynamic  between 
teacher  and  students.  It  suggests  the  assumption  of  responsibility  on  the  part  of  the  learner.  It  also 
assumes  that  experience  - read  and  lived  - can  be  ’crafted’  by  language. 

An  important  implication  of  the  use  of  this  model  for  learning  about  literacy  is  the  positioning  of 
learners  as  ’ethnographers  of  their  own  situations.’  This  is  not  to  say  that  students  spend  much  time 
talking  about  ethnographic  research  but  they  do  talk  about  key  terms  and,  in  view  of  the  module’s  aim 
to  examine  literacy  as  a sociocultural  practice,  they  are  encouraged  to  refer  to  the  work  of  some  of  the 
more  celebrated  ethnographers  of  literacy  such  as  Brian  Street  (Street  1992).  This  is  not  a research 
methods  module,  however,  so  what  exactly  does  the  phrase  ’ethnographers  of  their  own  situations’ 
mean?  Woods  et  al.  suggested  that  this  involves  positioning  students  as  observers  and  researchers  of 
contexts,  recording  data  and  observations  and  making  decisions  about  the  selection  of  details,  leading 
students  to  a critical  experiencing  of  texts,  contexts  and  discourses  (Woods  et  al.  1996:  12).  This  was 
our  aim  in  immersing  our  students  in  the  digital  literacy  practices  of  the  internet  and  its  associated 
technologies. 

As  resources  for  internet-based  learning  in  UK  higher  education  evolved  in  the  political  climate 
following  the  comprehensive  review  of  higher  education  provision  conducted  by  Lord  Dearing,  it 
became  possible  to  extend  the  ’on-line’  dimension  of  this  module  in  line  with  current  theory  in 
computer-assisted  learning.  We  decided  to  embed  an  existing  on-line  discussion  forum  within  a World 
Wide  Web  environment  which  provides  contextualised  access  to  the  technologies  associated  with  the 
face-to-face  ’classes’.  In  line  with  theories  of  constructivist  internet-based  learning  (see,  for  example, 
Laurillard  1998),  the  module’s  web-based  environment  contains:  home-produced  materials,  suggested 
links  to  external  materials  and  search  engines  which  enable  students  to  seek  authentic  materials  on  the 
net.  There  are  also  synchronous  and  asynchronous  forms  of  communications  technologies  embedded  in 
the  site. 
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As  part  of  the  curriculum  development  process,  and  at  the  beginning  of  the  academic  year,  we 
conducted  a questionnaire  survey  with  the  cohort  of  students  who  were  to  take  the  module  in  the 
second  semester.  Our  questionnaire  revealed  a cohort  in  their  late  teens  and  early  twenties  with  very 
limited  skills  in  Information  and  Communications  Technology.  This  finding  influenced  a number  of 
design  choices. 

Firstly,  in  terms  of  genre,  we  chose  to  present  the  environment  as  an  on-line  magazine  with  one  'story' 
each  week  surrounded  by  links  to  resources,  tools  and  activities.  Second,  the  navigation  of  the  site  was 
initially  made  very  linear.  This  was  to  support  the  skills  development  of  the  students  with  the  intention 
that  the  navigation  would  become  increasingly  heterogeneous  with  external  linking  via  'spin-off 
windows.  Third,  among  the  tools  available  would  be  on-line  instructions  for  synchronous  and 
asynchronous  conferencing,  an  archive  of  previous  'stories',  additional  readings  and  the  facility  for 
students  to  publish  their  own  texts  on  the  site.  Through  the  use  of  various  'seed'  questions  dispersed 
throughout  the  site,  we  invited  the  students  to  comment  on  all  aspects  of  the  design  of  the  enviruruiieiii 
in  the  discussion  forum  and  to  speculate  on  the  pedagogic  reasons  for  these  design  choices.  This  was 
one  aspect  of  positioning  students  as  ethnographers.  Others  include  immersing  the  students  in  the  texts, 
contexts  and  discourses  of  fantasy  role-playing  MOOs. 

In  line  with  our  concerns  about  hype  and  the  new  technnologies,  our  environment  is  being  critically 
evaluated  with  our  students.  Their  reactions  will  be  reported  and  used  to  refine  our  subsequent 
approach  to  learning  about  literacy  in  a virtual  environment. 
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Opportunities  for  Collaborative  Learning 

The  opening  of  the  Internet  and  the  World  Wide  Web  (WWW)  to  businesses  has  introduced  an  entirely 
new  specialized  area  in  Management  Information  Systems  (MIS)  called  Electronic  Commerce  (EC).  This  paper 
illustrates  how  EC  provides  rich  pedagogical  possibilities  for  collaborative  learning  using  hypermedia,  active 
learning  by  doing,  and  constructivist  teaching  approaches.  Selected  universities  in  the  country  introduced  EC 
courses  in  their  MIS  programs  to  address  new  developments  taking  place  in  the  Internet-enabled  marketspace. 
Content  analysis  of  the  course  materials  put  out  on  the  WWW  by  these  selected  universities  in  the  ISWorldNet 
Web  site  was  the  primary  technique  used  in  writing  this  paper.  Kosiur  (1997)  defines  EC  as  "...a  system  that 
includes  not  only  those  transactions  that  center  on  buying  and  selling  goods  and  services  to  directly  generate 
revenue,  but  also  those  transactions  that  support  revenue  generation  such  as.. offering  sales  support  and 
customer  service,  or  facilitating  communications  between  business  partners." 

The  technological  platform  for  conducting  EC  in  the  marketplace  could  also  be  used  for  student 
instruction  in  recreating  business  Web  sites.  This  is  perhaps  one  of  the  course's  primary  strengths. 
Furthermore,  it  lends  itself  to  the  pedagogy  of  collaborative  learning  where  student  teams  take  an  active  role  in 
constructing  the  knowledge  they  gain.  Research  is  clearly  supporting  the  efficacy  of  this  technique  in 
effectively  engaging  student  attention  and  energies.  In  a cooperative  learning  environment,  the  instructor 
organizes  content  and  topical  activities  around  tasks,  problems,  and  projects  that  student  teams  with  a balanced 
mix  of  abilities  and  backgrounds  are  designed  to  undertake  (Adams  and  Hamm,  1998).  Team  members'  social 
skills,  also  called  "interpersonal  intelligence"  according  to  Howard  Gardner,  encompassing  interpersonal 
communication,  task  management,  group  interaction,  and  conflict  resolution  are  constantly  honed  throughout 
the  process. 


Class  Activities 

The  first  set  of  activities  involves  a cognitive  appreciation  of  the  potential  of  EC  for  generating 
innovative  Web-based  products  and  services:  assembling  Web-based  business  plans  used  to  justify  potential 
online  businesses;  designing  small  business  Web  sites;  writing  reports  on  such  topics  as  EC  technologies  or 
industries  that  lend  themselves  well  to  EC;  doing  case  analysis  on  firms  pioneering  the  use  of  the  WWW; 
creating  Web  pages  with  a variety  of  content,  and  undertaking  Internet  activities.  All  these  activities  are  to  be 
conducted  within  a context  that  is  meaningful  to  the  student  and  one  that  builds  on  their  specific  business  major, 
prior  experience,  and  knowledge  following  the  constructivist  pedagogical  approach  as  opposed  to  the  traditional 
objectivist  method  (Cheek,  1992).  Students  will  be  asked  to  choose  topics  to  explore  and  practical  hands-on 
projects  that  most  directly  connect  to  their  line  of  work  in  the  office  or  personal  interests. 
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Learning  by  Doing 


"The  yeast  of  knowledge,  openness,  and  enterprise  raises  the  need  for  a multiplicity  of  learning  media 
and  technological  tools  "(Adams  and  Hamm,  1996).  The  very  wealth  of  Web  site  building  tools  that  is  now 
available  allows  students  to  experiment  and  "play"  through  "learning  by  doing"  activities  and  significantly 
changes  the  whole  instructional  experience.  Students  have  more  control  over  the  end  product  via  the  software 
tools  and  they  assume  greater  responsibility  over  their  learning  process  (Adams  and  Hamm,  1996).  The 
"standard"  activity  for  the  beginner-level  EC  course  is  having  students  build  small  business  Web  sites  using 
publishing  software  like  HTML  editors,  Netscape  Composer;  Microsoft  Publisher  98,  FrontPage  Express,  or 
Frontpage  98;  Cold  Fusion;  and  other  such  tools.  Web  site  design  issues  appear  to  be  the  emphasis  of  courses 
at  this  level.  Courses  for  the  more  advanced  EC  course  could  look  into  Web  site  features  that  facilitate  the 
actual  sale  transaction.  Thus,  students  may  be  asked  to  perform  more  advanced  E-Commerce  activities  like 
developing  online  order  entry  forms  that  actually  accept  data  at  the  backend  via  a database  package  using  tools 
like  Symantec’s  Visual  Cafe  employing  visual  programming.  Another  small  project  is  using  CGI  scripts  for 
writing  shopping  cart  application  consisting  of  an  item  catalog  interface,  item  description  pages,  and  a main 
checkout  page.  There  are  Web  sites  such  as  those  of  Data  Transfer  Associates,  Inc.  (http://www.dtastore.com) 
and  iCat  (http://www.iCat.com)  where  demo  software  is  available  so  that  students  can  experience  building 
online  retail  stores  and  act  like  online  merchants.  Both  companies  allow  students  to  build  prototype  online 
stores.  The  site  is  customized  so  that  it  reflects  the  store  owner's  name,  products  sold,  their  descriptions,  prices, 
and  code  numbers.  The  store  is  set  up  so  that  there  are  different  departments  showing  the  appropriate  product 
line.  An  online  shopper  goes  through  these  departments,  chooses  products  to  buy,  and  then,  checks  out  desired 
items.  The  service  also  makes  the  online  merchant  capable  of  processing  shipping  and  billing  and  credit  card 
information.  The  server  of  Data  Transfer  Associates  runs  tests  to  verify  customer  addresses  and  check  credit 
card  approvals  using  CyberCash,  Secure  Socket  Layer  (SSL),  and  other  technologies. 

Students  could  also  be  asked  to  enhance  their  business  Web  sites  by  adding  features  that  promote  a 
"virtual  community"  using  software  packages  like  Virtual  Suburbia  (http://www.virtualsuburbia.com).  Hagel 
and  Armstrong  (1997)  define  "virtual  communities"  as  online  communities  allowing  customers  with  the  same 
interests  to  congregate,  create  content,  and  communicate  on  an  ongoing  basis.  Virtual  Suburbia  allows  students 
to  create  a collaborative  environment  for  groups  of  people  by  allowing  them  to  chat  online  and  to  show  any 
Web  location  in  the  world  for  comment  and  evaluation. 

Perhaps,  the  greatest  strength  of  the  EC  course  activities  is  that  they  lend  themselves  to  authentic 
student  learning  assessment  using  the  portfolio  approach  (Adams  and  Hamm,  1998).  Since  the  course  will  be 
heavily  project-  and  activity-based,  outcomes  of  student  learning  are  constantly  being  produced  and  collected. 
There  are  even  a few  instances  in  these  courses  where  student  teams  actually  went  into  business  using  their 
pilot  projects  for  the  course.  This  is,  perhaps,  the  best  measure  of  course  success  even  though  it  is  rare. 
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It  is  commonly  stated  that  we  live  in  an  age  of  “information  overload.”  Increasingly  advanced 
communication  tools,  including  the  Internet  and  digital  telecommunication  technologies,  allow  us  to  transiiiil  inore 
and  more  information  at  faster  and  faster  rates.  A discipline — ^known  variously  as  information  design  or 
information  architecture — has  evolved  to  meet  the  challenge  of  using  information  to  provide  meaningful 
communication.  The  discipline  embraces  such  diverse  fields  as  business  administration,  computer  science, 
cognitive  psychology,  graphic  and  typographic  design,  and  technical  communication. 

Information  architecture  is  the  structuring  of  data  to  meet  the  informational  and  management 
requirements  of  an  organization  or  group  of  people.  For  example,  in  their  work  on  designing  web  sites,  Rosenfeld 
and  Morville  (1998)  state  that  the  information  architect  is  responsible  for  clarifying  the  mission  and  vision  of  a 
web  site;  for  determining  the  site’s  content  and  functionality;  for  specifying  how  users  will  find  information  by 
defining  the  site’s  organization,  navigation,  labeling,  and  searching  systems;  and  for  mapping  how  the  site  will 
change  and  grow  over  time  (Rosenfeld  & Morville,  1 998,  p.  11). 

This  paper  briefly  examines  some  theoretical  underpinnings  of  information  architecture,  considers  the 
application  of  the  principles  of  information  architecture  to  instructional  theory  and  practice,  and  describes  ongoing 
research  to  evaluate  the  efficacy  of  such  an  application. 

Theoretical  Background 

Information  Theory 

Building  on  the  seminal  work  of  Shannon  (1948),  information  and  communication  theorists  have 
proposed  a model  that  describes  the  transmission  of  information  from  source  to  recipient.  Current  information 
theory  is  concerned  primarily  with  the  binary  coding  of  transmitted  messages  in  ways  that  minimize  “noise”  (Ash, 
1965;  Gallager,  1968).  Information  designers  have  drawn  on  the  theory  to  suggest  that  information  be  designed  in 
ways  that  reduce  confusion  and  enhance  the  transmission  of  meaning  to  the  recipient  of  the  information  (Senechal, 
1997). 

Perception  and  Cognition 

Stem  and  Robinson  (1994)  define  perception  as  the  gathering  of  information  through  the  senses  and  the 
organizing  of  that  information  to  create  meaning.  All  sorts  of  factors — such  as  personal  characteristics,  past 
experiences,  and  present  feelings — can  influence  how  sensory  information  is  interpreted.  Kosslyn  and  Chabris 
(1993)  demonstrate  the  use  of  principles  of  cognitive  psychology  to  improve  the  effectiveness  of  informational 
graphics,  such  as  charts  and  graphs.  With  an  understanding  of  cognitive  principles,  the  designer  can  create 
graphics  that  influence  how  viewers  will  perceive  graphical  elements  and  the  meaning  of  the  relations  of  those 
elements  to  each  other. 

The  Social  Construction  of  Knowledge 

Vygotsky  and  others  have  suggested  that  knowledge  is  socially  constructed:  a conversion  of  social 
relations  into  psychological  functions  (Driscoll,  1994,  p.  229).  Learning  takes  place  as  learners  are  moved,  through 
social  interaction  with  more  advanced  peers  and  adults,  to  higher  levels  of  development. 
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Constructing  an  Information  Architecture 

The  literature  suggests  the  following  guidelines  for  designing  an  information  architecture: 

■ Center  the  process  on  the  needs  of  the  users  of  the  information. 

■ Perform  a structured  user  task  analysis  to  determine  the  user’s  information  requirements. 

■ Involve  users  at  all  stages:  analysis,  design,  and  implementation. 

■ Map  specific  information  needs  to  information  elements  in  the  web  site  or  instructional  piece. 

■ Structure  information  so  that  meaning  is  conveyed  effectively.  Consider  whether  design  elements  could  detract 
from  meaning  without  providing  some  benefit  such  as  increasing  motivation. 

■ Organize  information  so  that  the  user  can  easily  find  it.  Create  labeling  and  navigational  systems  that  are 
logical,  consistent,  and  easy  to  use. 

■ Organize  information  in  ways  that  facilitate  the  user’s  learning  and  schema  creation. 

■ Consider  personal  or  cultural  factors  that  might  affect  perception  of  information. 

■ As  ideas  take  shape,  create  storyboards,  mockups,  and  flowcharts.  Eventually,  map  the  entire  site  with  detailed 
architectural  blueprints. 

Instructional  Applications:  Ongoing  Research 

Proponents  of  constructivism  state  that  learners  construct  their  own  knowledge  in  ways  that  are  personally 
meaningful.  Some  constructivists  also  suggest  that  the  constructivist  instructor  guides  learners  to  integrate  their 
personally-constructed  knowledge  into  the  larger  body  of  socially-constructed  knowledge.  I suggest  that  instructors 
can  use  information  architecture  to  allow  students  to  structure  their  learning  in  ways  that  foster  their  own 
construction  of  meaning. 

To  evaluate  my  suggestion,  I have  designed  a class  entitled  “The  Instructional  Applications  of  Information 
Architecture.”  The  class  will  be  offered  to  graduate  students  in  the  Curriculum  and  Instruction  department 
(Instructional  Technology  concentration  area)  at  the  University  of  Houston,  and  I will  co-teach  the  class  with 
Dr.  Sara  McNeil.  The  expected  population  will  be  mature  graduate  students  who  are  conversant  with  computer  and 
Internet  technology.  At  or  near  the  beginning  of  the  term,  we  will  conduct  an  exercise  in  which  students  will  map 
the  types  of  learning  they  wish  to  achieve.  Students  will  map  out  the  instructional  sequence  and  build  an 
information  architecture  for  the  course  using  the  guidelines  listed  above.  This  architecture  will  be  collaboratively 
negotiated  by  instructors  and  students.  Using  a web  site  management  program,  such  as  Microsoft  Frontpage  or 
Adobe  PageMill,  students  will  create  the  basic  architecture  of  the  course.  As  the  term  proceeds,  students  will  flesh 
out  the  web  site  with  articles,  assignments,  and  projects.  At  the  conclusion  of  the  course,  I suggest,  students  will 
have  designed  and  completed  an  information  architecture  that  gives  tangible  form  to  the  cognitive  maps  they  have 
built. 
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The  PASSENGER  CSCL  Tool  for  Distributed  Learning  in  a Multimedia 

Environment 


Introduction 

Nowadays,  a new  component  is  added  to  the  complex  task  of  industrial  Software-Engineering:  the 
development  in  worldwide  distributed  teams.  This  paper  describes  new  concepts  and  tools  based  on  this  scenario 
that  are  developed  for  the  Software-Engineering-Education,  This  tools  give  the  students  the  ability  of  shortening 
their  study-time,  but  also  they  have  the  chance  of  obtaining  knowledge  in  systems  known  as  Compuer  Supported 
Collaborative  Working  (CSCW)  Environments  (Kremar,  in  Salvendy,  Smith,  1991).The  practical  training  aims 
at  applying  subject  matters  which  were  imparted  in  the  lecture  in  a realistic  scenario.  During  the  semester  the 
teams  will  experience  the  entire  life  cycle  of  Software-Engineering,  Problems  given  in  the  past  were  amongst 
others  simulating  an  elevator  control  for  a high-rise  or  the  simulation  of  a car-production-plant. 


The  Passenger  Concept 

The  organization  of  the  Software  Engineering  lab  is  supposed  to  be  extended  by  aspects  of  distributed  teams 
in  the  future.  This  scenario  has  long  become  reality  in  the  Software-Engineering  industry  and  is  well-known  in 
the  literature  (Nakatani,  Nishida  and  Takeda,  in  Salvendy,  Smith,  1991),  Following  this  scenario,  the  Passenger- 
Concept  defines  four  modules  for  the  conduction  of  the  Software-Engineering  course.  In  this  paper  we  will  focus 
on  the  development  of  the  Passenger  Case  & Video  Tool.  A description  of  the  other  modules  can  be  found  in 
(Hunger  and  Werner,  in  Gil-Mendieta,  Hamza,  1998). 

The  implementation  of  the  concept  requires  efficient  CSCL-tools  to  transfer  the  knowledge  of  the  lecture 
and  to  conduct  the  practical  training  in  a distributed  manner.  A condition  for  conducting  the  practical  training  is 
the  availability  of  tools  for  a synchronous  communication  and  for  common  document  processing.  Altogether  the 
following  modules  have  to  be  realized: 

1 . Synchronous  communication  tools  for  text,  audio  and  video, 

2,  A session  handling  to  control  the  synchronous  sessions, 

3,  Protocol  stacks  to  meet  the  required  network  services, 

4.  A Software- Engineering-Case-Tool  for  common  document  processing, 

5.  A control  system  to  handle  different  kinds  of  permissions, 

6,  A version  control  system  for  the  common  documents. 

Commercial  available  videoconference  tools  do  not  offer  all  of  the  above  mentioned  functions. 
Therefore  those  tools  can  not  be  considered  for  the  conduction  of  the  practical  training. 

The  scenario  of  distributed  Software-Engineering  places  additional  demands  on  the  students.  Apart 
from  learning  the  actual  contents  and  how  to  work  in  a team,  the  following  three  aspects  will  lead  to  a steeper 
learning  curve, 

1)  The  students  are  not  used  to  applying  Software-Engineering  Case-Tools  (Boloix,  G,  & Robillard,  P,N.,  in 
IEEE,  1998), 

2)  For  the  students,  discussing  by  the  means  of  video  conferencing  represents  a completely 
new  form  of  communication  (Hunger  & Werner,  in  Verbraeck,  Al-Akaidi,  1998), 

3)  Common  document  processing  over  a network  is  new  to  the  students. 
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The  Passenger  Case  & Video  concept  sees  windows  based  systems  as  its  target  platform  and  the  global 
internet  as  its  transport  medium.  For  the  clients  a multimedia  data  interface  represents  the  interface  to  the  input 
and  output  peripherals  like  keyboard,  mouse,  video  camera,  speaker  and  monitor.  A Case-Tool  is  used  for 
common  software  development.  During  the  conceptional  considerations  of  the  Passenger-System-Architecture, 
different  possibilities  for  the  data  transfer  were  discussed.  As  a result  of  the  discussion  several  methods  of  data 
transfer  have  been  implemented:  several  unicast-connections  for  client/server-communications  and  several 
multicast-connections  for  interclient-communications. 

The  requirements  which  individual  connections  place  upon  a service  are:  Client/Server  connections 
require  reliable  services,  inter-client  connections  on  the  other  hand  only  require  unreliable  services,  but  a 
guaranteed  bandwidth  and  delay.  In  order  to  achieve  these  guaranteed  quality-of-service  we  make  use  of 
resource  reservation.  In  a first  step  a protocol  stack  based  on  the  Internet  Protocol  Version  6 (Deering  & Hinden, 
1998)  for  the  realisation  of  multicast  connections  in  the  Internet,  associated  with  the  Resource  ReSerVation 
Protocol  (RSVP)  (Braden,  Zhang,  Berson,  Herzog,  .Jamin,  1997)  for  quality-of-service  support  is  developed. 
Measures  to  increase  performance  arise  in  tliis  first  step  by  strictly  splitting  the  connection  types  in  reliable  and 
unreliable  services  and  implement  them  in  several  parallel  protocol  stacks.  Measures  like  adapting  certain 
protocols  should  be  considered  later. 


Summary  and  Outlook 

This  paper  presented  a new  concept  concerning  the  University-Education  on  the  subject  Software- 
Engineering.  Based  on  this  concept  new  tools  were  developed  and  implemented.  The  essential  innovations  of  the 
presented  idea  in  comparison  with  known  concepts  are  in  this  case  the  integration  of  different  tools  under  a 
common  user  interface,  the  way  in  which  face-to-face  discussions  are  taken  into  account,  the  development  and 
integration  of  case  tools  especially  for  the  usage  in  a practical  training  and  the  assembly  of  parallel  protocol 
stacks  with  usage  of  given  protocol  structures. 

The  first  version  of  the  communication  tools  is  expected  during  the  summer  of  1999.  Concerning  the 
practical  training  of  Software-Engineering,  the  tools  are  supposed  to  be  put  into  practice  during  the  summer- 
semester  2000 
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Introduction 

Discovering  Science  is  a new  introductory  science  course  from  the  Open  University  (OU)  that  makes 
extensive  use  of  inlciaciive  multimedia  activities  on  CD-R.OM  (The  Open  University  19^X).  Tt  was  first 
presented  in  1998  to  over  3000  students  in  the  UK  and  Western  Europe,  and  is  the  first  OU  science  course  that 
requires  students  to  have  access  to  a computer  in  their  homes.  25  CD-ROM  activities  were  produced,  requiring 
about  40  hours  of  study  time,  making  this  the  largest  single  educational  multimedia  project  in  the  world.  This 
paper  outlines  the  scope  of  these  materials,  how  they  are  integrated  with  the  course,  and  some  issues  arising  from 
their  production  and  presentation. 


The  CD-ROM  Materials 

The  role  of  the  CD-ROM  activities  is  to  teach  science,  and  the  range  of  topics  reflects  the  scope  of  the 
course,  which  covers  aspects  of  biology,  chemistry.  Earth  science,  physics  and  astronomy.  Activities  include: 

• an  interactive  computer  model  of  the  Earth’s  atmosphere  that  allows  students  to  modify  factors  such  as  the 
carbon  dioxide  level  or  the  cloud  cover  and  to  investigate  the  effects  of  these  changes  on  the  global  temperature; 

• a virtual  field  trip  to  the  Galapagos  Islands,  following  in  the  footsteps  of  Charles  Darwin,  to  investigate 
adaptation,  speciation  and  evolution;  and  a virtual  trip  in  a submersible  to  explore  the  Mid  Atlantic  Ridge; 

• use  of  molecular  viewer  software  to  visualize  the  3D  structure  of  molecules,  ranging  from  the  simple  methane 
molecule  to  complex  proteins; 

• interactive  maps  and  models  that  are  used  to  investigate  past  and  future  motion  of  the  Earth’s  tectonic  plates; 

• a virtual  telescope,  used  to  determine  the  distances  and  speeds  of  galaxies,  and  hence  the  age  of  the  Universe. 

All  of  these  CD-ROM  materials  are  highly  interactive  and  they  make  use  of  the  ability  of  a multimedia 
computer  to  present  a mix  of  video,  audio,  computer  simulation,  graphics,  text,  questions,  etc.  They  are 
integrated  with  the  other  distance-learning  components  of  the  course,  which  include  11  books  (1300  pages),  a 
loose-leaf  study  file  (500  pages),  five  hours  of  video  material,  ten  half  hour  TV  programmes  transmitted  by  the 
BBC,  a small  home  practical  kit,  and  a one-week  residential  school. 


Production  of  the  CD-ROM  Materials 

This  was  the  first  major  venture  into  production  of  interactive  multimedia  CD-ROMs  for  teaching  by 
the  Open  University.  Many  lessons  were  learnt,  and  a few  of  these  are  outlined  here. 

• There  were  long  discussions  about  whether  to  use  an  existing  interface/shell/navigation  system,  or  whether  to 
develop  a system  of  our  own  for  use  with  all  packages.  We  eventually  decided  that  both  of  these  routes  would 
restrict  the  scope  and  variety  of  the  materials  that  we  wanted  to  produce.  Broad  guidelines  for  the  interfaces  were 
drawn  up,  but  these  guidelines  were  interpreted  quite  flexibly,  so  the  different  activities  have  very  different  looks 
and  feels. 

• With  25  CD-ROM  activities  (plus  CAL  software  for  developing  mathematics  skills,  and  interactive  self- 
assessment  questions  covering  the  whole  of  the  course),  it  was  essential  to  devise  a simple  way  for  students  to 
access  the  computer-based  material.  We  developed  so-called  CD-ROM  Guides  that  students  installed  on  their 
hard  drive,  and  these  not  only  made  it  very  straightforward  to  access  each  of  the  activities  but  also  helped  to 
integrate  the  different  CD-ROM  activities  with  the  rest  of  the  course. 

• Production  of  alpha  versions  of  some  CD-ROM  activities  used  most  of  the  time  and  resources  that  had  been 
allocated  for  completing  the  activities,  leaving  limited  scope  to  make  changes  following  developmental  testing. 
In  future  projects  we  will  ensure  that  30%  of  the  time  and  resource  are  held  back  for  revisions  following  testing 
of  the  alpha  version  of  the  software. 


rr  o 4 
o 4 


• OU  students’  computers  have  a wide  range  of  hardware  specifications,  they  are  configured  in  a variety  of 
ways,  and  they  are  running  many  different  software  applications.  We  had  to  find  a compromise  between 
producing  state-of-the-art  multimedia  software,  which  would  only  run  successfully  on  high-specification 
machines,  and  producing  software  that  would  run  on  low-specification  machines  but  which  would  look  and  feel 
rather  dated.  The  software  went  through  thorough  QA  testing  on  a range  of  machines,  but  even  so  a small 
proportion  of  the  thousands  of  students  who  used  it  experienced  problems.  Our  telephone  helpdesk  sorted  out 
many  of  these  problems,  though  some  required  changes  to  CD-ROMs  that  could  only  be  made  for  the  second 
year  of  the  course. 

Studying  CD-ROM  Material 

Surveys  of  students  and  tutors  highlighted  a number  of  issues  relating  to  use  of  the  CD-ROM  materials. 

• Most  students  enjoyed  studying  the  CD-ROM  material,  and  found  it  interesting  and  motivating. 

• Studying  CD-ROM  material  is  very  different  from  studying  text,  watching  a video  or  listening  to  a lecture, 
and  many  students  found  it  more  difficult  to  learn  from  the  CD-ROM  material.  Students  have  little  experience 
of  learning  from  CD-ROM,  and  they  need  advice  about  note  taking  and  time  management  in  particular. 

• Students  like  to  be  able  to  browse  freely  through  a CD-ROM  activity,  to  dip  into  it  at  any  point,  and  to  move 
backwards  through  an  activity  as  well  as  forwards,  just  as  they  can  with  a book.  Some  of  our  CD-ROM 
activities  constrained  students’  freedom  of  navigation,  and  were  criticized  by  students  and  tutors  for  this  reason. 

• The  ability  to  specify  where  you  are  in  a CD-ROM  activity  is  extremely  useful,  particularly  when  you  are 
remote  from  a person  with  whom  you  want  to  discuss  the  activity,  e.g.  your  course  tutor.  Page/screen  numbers 
are  far  more  convenient  than  specifying  ‘the  third  screen  after  the  second  question  in  part  three’. 

• We  integrated  the  CD-ROM  material  with  the  other  components  of  the  course,  so  that  students  wouldn’t 
perceive  it  as  an  optional  extra.  However,  some  topics  were  only  covered  in  CD-ROM  activities,  and  it  was  then 
necessary  to  emphasize  the  importance  and  relevance  of  these  activities.  Assignment  questions  and  exam 
questions  were  an  effective  way  of  doing  this. 

Ongoing  Evaluation 

Student  surveys  in  1998  concentrated  on  identifying  any  problems  that  students  had  with  using  the  CD- 
ROM  material,  but  further  evaluation  is  now  underway.  One  survey  is  investigating  how  students  study  CD- 
ROM  material  and  how  their  study  methods  correlate  with  their  learning  style,  as  determined  by  responses  to  an 
‘Approaches  to  Study  Inventory’.  It  is  clear  from  data  obtained  in  the  first  year  that  a small  proportion  of 
students  dislike  using  this  medium  and  had  difficulty  learning  effectively  with  it.  Information  from  this  survey 
should  help  us  produce  advice  for  distance-learning  students  about  developing  appropriate  learning  strategies. 

A second  survey  is  investigating  cognitive  gain  from  these  CD-ROM  activities,  and  the  dependence  of 
this  on  the  level  of  interactivity  in  the  activities.  Our  assumption  is  that  interactivity  promotes  effective  learning, 
and  we  will  test  this  hypothesis  in  this  study,  and  compare  cognitive  gains  from  different  types  of  interactivity. 


Conclusion 

The  CD-ROM  activities  produced  for  Discovering  Science  demonstrated  some  of  the  benefits  of 
incorporating  interactive  multimedia  activities  in  introductory  science  courses.  The  lessons  learnt  during 
production  and  presentation  of  this  course  will  feed  through  into  the  production  of  CD-ROM  material  that  the 
Open  University  is  now  producing  for  its  future  science  courses. 


References 

The  Open  University  (1998).  Discovering  Science.  Milton  Keynes,  UK:  The  Open  University. 

Acknowledgement 

The  CD-ROM  activities  for  Discovering  Science  were  produced  by  a large  multidisciplinary  course  team,  which  included 
staff  from  the  Open  University  Science  Faculty,  Academic  Computing  Service,  Design  Studio  and  Institute  of  Educational 
Technology,  and  from  the  OU  BBC  Production  Centre. 


Discovering  Science  - with  Interactive  Multimedia  CD-ROMs 

Stuart  Freake 
Faculty  of  Science 
The  Open  University 
United  Kingdom 
s.m.freake@open,ac.uk 


Discovering  Science  is  a new  introductory  science  course  from  the  Open  University  (OU)  that 
makes  extensive  use  of  interactive  multimedia  activities  on  CD-ROM  (The  Open  University  1998).  It  was 
first  presented  in  1998  to  over  3000  students  in  the  UK  and  Western  Europe,  and  is  the  first  OU  science 
course  that  requires  students  to  have  access  to  a computer  in  their  homes.  25  CD-ROM  activities  were 
produced,  requiring  about  40  hours  of  study  time,  making  this  the  largest  single  educational  multimedia 
project  in  the  world. 

The  poster  and  demonstration  highlight: 

• the  wide  range  of  styles  used:  game  format;  resource-based  free  exploration;  linearly  structured  CAL; 
virtual  experiments  and  virtual  field  trips;  etc; 

• the  wide  range  of  topics:  global  warming;  evolution;  quantum  physics  and  cosmology,  cell 

biochemistry;  etc; 

• the  integration  of  CD-ROM  material  with  other  course  components; 

• the  CD-ROM  Guides  that  are  used  to  access  the  CD-ROM  material. 
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INTRODUCTION 

With  the  rapid  evolution  of  Internet  technologies,  and  the  advent  of  WEB  TV  and  hybrid  cable/modem 
systems  that  promise  ever-increasing  data  throughput.  Virtual  Reality  Modeling  Language  (VRML),  is  rapidly 
approaching  viability  as  a medium  for  educational  program  delivery  and  interactive  learning.  As  the  industry 
furthers  its  resolve  on  standards  for  VRML,  a few  stepwise  approaches  to  VR  for  exploring  virtual  learning 
environments  are  being  embraced  and  utilized  by  creative  netizens.  Apple  computer’s  QuickTime  VR  “virtual 
panorama  and  virtual  objects”  technology  is  an  intriguing  step  in  the  right  direction,  and  offers  VR  learning  on 
the  halfshell,  sort  of  There  are  hundreds  of  websites  employing  QTVR  already;  from  Apple’s  Store,  to  The 
Wrinkle  in  Time  Project’s  “day  in  the  life”  global  simultaneous  QTVR  galley,  to  numerous  online  catalogs 
portraying  virtual  (rotate-able)  sale  items,  etc....  But  what  are  the  uses  for  this  VR  steppingstone  technology 
for  education?  Or  for  that  matter,  what  educational  use  VR  in  general? 


VIRTUAL  REALITY  MODELING  LANGUAGE:  ROOTS 

Virtual  Reality  Modeling  Language  (VRML,  commonly  called  “vermel”)  is  an  evolving  Internet  and 
web-based  scene  description  language.  Since  its  inception  in  1994,  VRML  has  been  an  open  standard.  In 
August  of  1996  the  VRML  Consortium  Working  Group  (CWG)  was  formed  following  a meeting  at  the  AMC 
SIGGRAPH  annual  convention,  and  in  1997  the  Consortiums'  efforts  to  continue  honing  VRML,  increasing  its 
capabilities  and  streamlining  protocols  resulted  in  the  present  incarnation  specification  VRML  '97. 
<http://www.vrml.org/> 

Like  its  hypertext  sibling,  (HTML),  VRML  comprises  a scripting  strategy  for  describing  the  layout  or 
arrangement  of  objects  in  a virtual  environment;  scene  descriptors  which  in  most  cases  are  formulas  for 
geometric  solids,  lines  and  planes,  and  scene  metadata,  as  well  as  applied  surface  textures.  Unlike  HTML  pages 
VRML  documents  describe  a virtual  space  or  three  dimensional  (3D)  environment  in  which  objects  are  located 
and  around  and  through  which  the  user  can  navigate  by  pointing  and  clicking,  dragging  and  rotating  or  engaging 
controls  on  a navigation  bar.  In  other  words,  VRML  instructs  a user’s  computer  in  how  to  arrange  a file’s 
contents  in  the  web  browser  window,  but  that  window  is  really  a mndow  or  portal  into  a 3D  space  not  just  a flat 
surface  on  which  text  and  graphics  are  situated.  VRML  works  because  relatively  simple  geometric  shapes  can 
often  approximate  fairly  complex  objects  with  textures  mapped  onto  their  surfaces,  like  wallpapering  a simple 
box  to  simulate  a fully  decorated  room.  Once  the  size  and  shape  of  basic  objects  along  with  simple  texture 
maps  is  downloaded,  the  browser  builds  those  objects  in  3D  space  locally.  Moving  around  in  the  virtual  space 
is  then  just  a matter  of  the  local  computer  reshaping  and  changing  the  relative  positioning  of  objects  to  match 
changing  perspectives  of  the  viewer.  All  or  most  of  the  data  is  already  passed  off  to  the  local  machine  for 
rendering,  so  individual  scenes  can  hold  many  possible  perspectives,  and  new  information  is  retrieved  only 
when  needed  to  render  as  yet  unseen  areas  of  the  3D  environment,  or  ever  closer  detail  in  an  object. 

The  computer  script  and  code  of  VRML  is  still  evolving  and  as  with  HTML,  SHTML  DHTML,  and 
XML,  enhancements  and  additions  to  the  standard  are  being  proposed  continuously.  The  first  efforts  to 
standardize  a description  of  VR  environments  for  the  web  took  place  at  an  adhoc  meeting  in  the  spring  of  '94  at 
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the  first  International  WWW  Conference  in  Geneva,  Switzerland.  Shortly  thereafter  a www-VRML  mailing  list 
was  opened  and  a draft  version  of  the  specification  for  VRML  was  collectively  and  collaboratively  produced  for 
the  Fall  1994  WWW  Conference  (Pesce,  1994).  Over  a thousand  individuals,  researchers,  technicians  and  users 
remain  on  this  and  other  related  www-VRML  mailing  lists. 

VRML’s  roots  are  deeply  entwined  in  the  3D  design  and  architecture  world,  and  the  emergent  “modeling 
language”  may  be  described  as  only  a recent  means  of  streamlining  scene  layout  descriptions  for  distributing 
those  worlds  on  a network.  The  roots  of  "Virtual  Reality"  can  be  traced  to  the  spatial  design  and  rendering 
work  of  Ivan  Sutherland  and  others  in  the  late  ‘60s  (Sutherland,  1965).  Numerous  issues  of  the  Association  for 
Computing  Machinery’s  Special  Interest  Group  on  Graphics  (ACM  SIGGRAPH)  have  been  devoted  to  VR  and 
visualization.  Conn,  Lanier,  and  others  (1989)  describe  the  future  of  virtual  environments  and  interactivity  on 
the  Internet  as  an  exciting  and  powerful  opportunity  for  communication,  collaboration  and  participation  that 
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A perpetual  challenge  for  VR  and  VRML  is  in  the  rendering  and  display  of  increasingly  detailed  and  rich 
environments.  Scene  detail  is  highly  dependent  upon  the  computer’s  ability  to  quickly  describe  geometric 
primitives  (or  polygons),  and  arrange,  highlight  and  render  them  with  enough  speed  to  permit  successive 
adjustments  and  differing  perspectives  of  the  described  world  every  few  fractions  of  a second.  As  complexity 
in  a scene  builds,  e.g.  landscape  details,  increasing  numbers  of  objects  in  view,  and  degrees  of  transparency  and 
reflectivity,  rendering  times  of  individual  scenes  often  cause  even  the  fastest  of  PCs  to  grind  to  a halt.  Effective 
VRML  scene  navigation  depends  on  available  RAM,  VRAM  and  clockspeed  of  the  viewing  computer  and  the 
speed  and  bandwidth  of  the  connection  through  which  scene  parameters  and  object  descriptions  are  transmitted. 
Addressing  this  limitation,  ongoing  negotiations  among  Cable  TV  signal  carriers  and  Internet  providers,  and  a 
new  incarnation  of  WEB  TV  devices  and  hybrid  cable/modems  may  soon  make  a variety  of  virtual  experiences 
more  widely  available  than  otherwise  possible  (solving  part  of  the  bandwidth  and  throughput  problem). 


VRML  FOR  EDUCATION 

Aside  from  simulated  military  missions  and  video  games  (a  common  and  very  well  institutionalized  use 
of  virtual  reality  technologies),  VR  “worlds”,  broadcast  over  the  Internet  have  many  potential  uses  in 
instructional  and  educational  settings.  VR  is  showing  outstanding  promise  for  teaching,  demonstrating  and 
learning  in  the  medical  profession,  in  the  engineering  and  structural  design  fields,  as  well  as  in  projects  where 
“being  there”  is  all  but  impossible. 

VRML  in  education  will  likely  find  utility  as  a means  of  exploring  complex  mathematical  “terraines”, 
giving  unique  views  of  very  small,  very  large  or  otherwise  inaccessible  spaces  such  as  navigating  along  a DNA 
helix,  probing  inside  an  ant’s  brain,  a human  heart,  and/or  at  the  other  extreme,  exploring  virtual  astronomical 
spaces  like  solar  systems  and  galaxies.  Other  unique  roles  VRML  can  play  in  enhancing  education  at  first  may 
seem  trivial.  Some  educators  are  beginning  to  explore  its  use  in  the  classroom  in  the  form  of  MOOS  or  MUDs 
(Multi  User  Dimensions)  or  virtual  chatrooms.  Finally,  the  exploration  and  “mapping”  of  real  terraines  and 
physical  places,  is  a possible  use  of  VRML  that  in  conjunction  with  “chaf  ’ or  narrative  exchanges,  and  as  an 
adjunct  to  shared  web-portfolios  (especially  where  QTVR  panorama  and  objects  are  linked)  may  really  open 
doors  to  learning  as  topics  can  be  put  into  local  community  context  and  differing  strategies  for  dealing  with 
common  issues  can  be  shared  among  reporting  students.  In  describing  and  sharing  each  unique  community's 
sense  of  place  or  "view-from-here",  including  student-made  local  detailed  maps,  GIS  data  layers,  imagery, 
QTVR  and  narratives,  students  and  teachers  can  create  a very  dynamic  and  engaging  virtual  collaborative. 
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A teacher  of  English  in  Ukraine,  as  well  as  in  any  other  non-English  language  country,  is  constantly  facing  a 
number  of  problems  that  are  associated  with  his/her  professional  competency  support.  Taking  into  account 
the  complexity  of  this  competency  which  embraces  pedagogical,  linguistic,  sociolinguistic,  and 
communicative  subcompetencies,  we  realize  the  enormity  of  the  task  of  upgrading  or,  at  least,  supporting  the 
knowledge  and  skills  that  the  teacher  needs  to  possess  in  order  to  be  efficient  in  the  teaching  process  at 
school. 

In  order  to  develop  a system  that  could  provide  life-long  professional  support  for  the  teachers  of  English  as  a 
Second  Language  (ESL),  we  need  to  define  these  problems.  The  most  urgent  among  them  are: 

- Limited  opportunities  to  practice  live  English  and  develop  communicative  competency  in  the  target 
language  in  the  non-English-speaking  country; 

- Insufficient  inventory  of  up-to-date,  authentic,  effective  teaching  materials  and  aids  for  the  teachers 
(textbooks,  readers,  audio  tapes,  video  tapes,  computer  courseware); 

- Almost  total  lack  of  qualified  methodological  assistance  in  tackling  everyday  classroom  issues. 

To  a larger  extent  these  questions  concern  teachers  of  remote  urban  and,  especially,  rural  schools. 

Our  task  was  to  design  and  develop  a specific  support  system  that  could  offer  all  the  necessaiy  means  for  an 
ESL  school  teacher  to  upgrade  his/her  professional  competency  without  abandoning  the  teaching  process  and 
investing  unrealistic  time  and  funds. 

Analysis  of  specific  requirements  of  the  ESL  teacher  regarding  support  of  his/her  professional  competency 
reveals  that  the  teacher  needs  a complex  program  allowing  him/her  to  systematically  upgrade  his/her 
vocabulary  and  grammar,  reading,  writing  and  communication  skills;  to  communicate  in  English  with  his 
supervisor,  co-leamers,  colleagues,  native  speakers  of  English;  to  obtain  different  learning  materials  (texts, 
audio,  video  and  computer  courses);  to  use  informational  resources  like  reference  books,  dictionaries, 
encyclopedias,  journals  etc.  The  teacher  also  has  to  have  access  to  a bank  of  the  teaching  materials  he/she  can 
use  in  the  class  activities,  to  the  existing  methodologies  on  how  to  teach  specific  subjects  and  topics,  and  to 
the  expert  who  can  offer  qualified  assistance  or  advice.  This  system  must  also  meet  the  following  criteria: 
affordability,  accessibility  and  openness,  professional  orientation  and  functionality,  continuity,  flexibility  and 
self-sufficiency,  availability  of  feedback  and  of  the  communication  mode. 

It  is  evident  that  we  need  to  build  a non-traditional,  powerful  informational  and  learning  system  based  on 
contemporary  pedagogical  theories  and  advanced  computer  and  telecommunication  technologies.  This  system 
has  to  be  available  for  any  teacher  at  any  school  to  link  to  for  obtaining  the  necessary  support  in  any  form. 
Such  a system  is  being  currently  developed  as  a Ukrainian  national  distance  system  for  ESL  school  teachers’ 
professional  competency  support . 

The  system  includes  the  following  components: 

1 . Multilevel  subsystem  of  language  training  and  competency  development; 

2.  A complex  of  English  courses  and  other  learning  materials; 


3.  Reference  subsystem  with  dictionaries,  grammar,  cultural  and  other  data  bases; 

4.  Methodological  support  subsystem; 

5.  Internet  facilities  for  group  collaboration  and  communication. 

The  system  is  actually  an  informational  and  learning  environment  that  offers  the  ESL  teacher  all  the 
materials,  means  and  help  he/she  needs  for  successful  professional  teaching  activity  in  the  class.  The  basic 
English  course  that  is  one  of  its  main  components  is  closely  coordinated  with  the  current  Program  in  Foreign 
Languages  for  Ukrainian  Schools  and  includes  the  following  topics: 

1)  Education  (system  of  education,  education  in  Great  Britain  and  USA,  high  school); 

2)  Curriculum  (school  subjects,  lesson,  English  as  an  tool  of  international  communication); 

3)  People  (I,  my  family,  friends); 

4)  Life  (my  work  uay,  stuuies,  free  time,  vacations); 

5)  Work  (plans  for  the  future,  professions); 

6)  Country  and  city  (basic  facts  about  Great  Britain  and  the  USA,  London  and  Washington,  DC,  Ukraine, 
Kiev,  my  home  town); 

7)  World  (the  planet,  geography,  environment); 

8)  Arts; 

9)  Literature  of  the  English-speaking  countries; 

1 0)  Sports. 

The  course  is  composed  in  such  a way  as  to  offer  the  user  maximum  learning  materials  and  activities  together 
with  the  efficiency  and  comfort  of  navigation  and  use.  Modular  structure  is  the  main  organizational  principle 
of  the  course  according  to  which  it  is  divided  into  separate  but  interconnected  modules.  Each  module  is  self- 
sufficient:  it  comprises  all  the  necessary  texts,  exercises,  tests,  additional  materials,  vocabulaiy  and  reference 
materials,  recommendations  for  the  student  teacher  how  to  study  and  for  the  practicing  teacher  how  to  use  it 
in  the  class.  The  materials  assembled  in  the  module  are  supplied  with  such  essential  characteristics  as  the 
topic,  summary,  level  of  training,  volume  (the  number  of  texts  and  exercises  in  the  module,  the  number  of 
words  in  the  text,  the  number  of  questions  in  the  test,  etc.). 

Three  types  of  activities  are  recommended  in  the  system:  informational,  training  and  communication. 
Informational  activities  are  focused  on  the  search,  processing  and  purposeful  use  of  the  desired  information. 
Training  offers  various  cognitive  activities  - individual  and  group  solution  of  linguistic  and  pedagogical 
problems,  writing  and  editing  essays  and  projects,  creating  and  distributing  learning  materials,  submitting 
entries  for  the  on-line  dialogue  journal,  and  working  with  the  computer  courses  and  other  materials. 
Communication  activities  include  exchanging  information,  writing  letters,  participating  in  the  on-line 
dialogues,  group  discussions,  interest  group  activities  and  conferences,  and  peer  consultations.  Some  of  the 
activities  are  to  be  done  independently,  many  are  to  be  implemented  in  collaboration  with  other  members  of 
the  group,  a few  require  tutor  participation  or  guidance  and  supervision.  All  these  activities  are  aimed  at  the 
collective  construction  of  a particular  knowledge  and  development  of  a common  ESL  teacher’s  professional 
competency  in  a specific  distant  informational,  learning  and  social  format. 

The  system  for  professional  competency  support  of  ESL  school  teachers  described  in  this  paper  is  currently  in 
its  development  phase.  It  is  supported  by  the  Ministry  of  Education  of  Ukraine,  the  British  Council,  Institute 
of  Cybernetics  of  the  Ukrainian  Academy  of  Sciences  and  Kiev  State  Linguistic  University.  While  working 
on  the  project  we  provide  training  in  Educational  Technology  for  teachers,  and  conduct  research  in  the  course 
development  and  methodology  of  Distance  Learning. 
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This  work-in-progress  describes  a collaborative  effort  between  two  faculty  members  at  Rose-Hulman  Institute 
of  Technology  to  build  a web-supported  integration  of  technical  communication  into  a sophomore-level 
course  in  engineering  practices.  The  collaboration  was  begun  in  the  summer  of  1998.  While  the  integrated 
delivery  environment  is  still  in  its  formative  stages,  we  believe  this  presentation  is  appropriate  for  ED- 
MEDIA  because  it  covers  two  timely  issues  for  engineering  education:  (1)  the  efficacy  of  asynchronous 
modules  for  course  delivery,  and  (2)  the  meaningful  integration  of  technical  communication  at  early  levels  in 
the  engineering  curriculum. 

EC  260  - "Engineering  Practice"  --  is  a two-credit  experience  designed  to  prepare  Electrical  Engineering 
students  both  for  the  senior  design  sequence  and  for  the  world  of  work.  Course  goals  include: 

1 . To  begin  the  integration  of  knowledge  through  the  solution  of  an  industry-sponsored  engineering 
problem. 

2.  To  improve  oral  and  written  communication  skills. 

3.  To  gain  experience  in  working  as  a team. 

4.  To  become  proficient  in  using  both  traditional  and  information-age  resources,  enacting  a need-to- 
leam  approach. 

During  the  Fall  Quarter  of  1998,  portions  of  the  course  were  delivered  via  a web-site,  containing  assignments, 
course  maintenance  materials,  resource  and  supplemental  documents,  as  well  as  such  facilitating  features  as  a 
chatroom,  electronic  file  transfer,  and  the  potential  for  online  commentary  on  student-submitted  work. 

Much  of  the  student's  success  in  this  course  is  highly  dependent  upon  abilities  to  use  sophisticated 
communication  skills.  However,  at  this  point  in  the  students'  education,  most  will  not  have  taken  a course  in 
Technical  Communication  (which  is  offered  as  a junior-level  course  at  Rose-Hulman).  Thus,  having  access  to 
basic  instruction  in  the  oral  and  written  forms  of  communication  required  in  the  course  is  vitally  important. 

A series  of  five  web-delivered  modules  were  constructed  --  each  having  interactive/transactional  capabilities  - 
- so  that  students  can  have  on-demand  instruction  in  the  verbal  aspects  of  the  course.  The  five  modules  are 
listed  below.  Each  highlights  a set  of  "information  processing  skills"  (represented  on  the  left),  as  instantiated 
in  a specific  communication  form  (represented  below  on  the  right). 

1 . Document  Design  / Formal  Program  Report 
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2.  Interpreting  and  Presenting  Data  / Feasibility  Study 

3.  Integrating  Text  and  Graphics  / Technical  Description 

4.  Audience  Analysis  and  Persuasion  / The  Proposal 

5.  Information  & Audience  Engagement  / The  Oral  Report 

While  our  assessment  nf  the  Fail  Quarter  experience  is  currently  (4/1499)  in  progress,  we  believe  that  the 
basic  premises  of  the  cross-disciplinary  collaboration  were  satisfied  and  that  the  web-based  learning 
environment  meet  the  following  objectives: 

• Enhanced  active  learning  — students  had  an  active  role  in  problem-solving  and  in  discovery 
learning.  Scaffolding  and  interactive  feedback  helped  students  to  adjust  and  consolidate  gains  in 
learning. 

• Provided  opportunities  for  collaboration  — opportunities  to  share  information  artifacts  and  to 
enhance  abilities  to  work/communicate  with  peers  by  providing  commentary  and  accepting 
constructive  criticism. 

• Accommodated  transfer  of  learning  - course  skills  and  concepts  generalized  in  that  the  pedagogy 
encouraged  the  learner  to  make  connections  and  extrapolations. 
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Introduction 

Broadband  communication  systems  and  enhanced  tools  for  transmission  of  video/audio  and  multimedia  data  pave  the 
way  for  new  concepts  of  distance  teaching  and  learning.  Especially  ATM-based  networks  allow  online  teleteaching 
applications  of  high  quality.  In  various  countries  there  are  strong  efforts  to  establish  a broadband  communication  in- 
frastructure for  education  and  research,  e.  g.  the  Intemet-2  project  in  the  United  States.  In  Germany  the  Federal  De- 
partment of  Education,  Science,  Research  and  Technology  initiated  the  construction  of  the  so-called  B-WIN 
(Broadband  Scientific  Network)  which  started  working  in  1996  with  the  secretary  of  education  and  research  cutting 
the  virtual  red  ribbon  at  the  CeBit  fair  in  Hannover.  Adequate  teleteaching  environments  have  been  implemented  at 
several  universities.  Preliminary  evaluation  results  show  that  students  as  well  as  teachers  accept  the  technical  add-ons 
and  take  profit  from  the  added-values  of  ‘ virtual  lectures’. 


Broadband  Scientific  Network  in  Germany 

The  German  Scientific  Network  is  based  on  ATM  technology  and  connects  nearly  all  of  the  70  universities  of  unified 
Germany  as  well  as  several  research  institutions  (Fig.  1).  The  ATM  backbone  operates  at  155  to  622  Mbit/s.  Universi- 
ties are  hooked  up  by  at  least  34  Mbit/s.  Actually  a Gigabit  testbed  (2.4  Gbit/s)  is  being  run  between  Berlin,  Nurem- 
berg and  Munich.  By  the  year  2000  a nationwide  Gigabit  communication  platform  for  universities  and  other  institu- 
tions concerned  with  education  and  research  shall  be  at  hand.  There  are  also  links  to  the  public  ISDN  network  used  as 
a ramp  to  that  information  highway,  for  instance  by  high  schools.  The  German  Scientific  Network  is  linked  to  the 
European  ATM  backbone  called  TEN34  (Trans  European  Network,  34  Mbit/s)  as  well  as  to  networks  in  other  conti- 
nents like  North  America,  Asia  or  Australia. 


Experiences 

Due  to  high  installation  costs  for  teleteaching  lecture  theatres  and  multimedia  teacher  terminals  there  are  few  univer- 
sities in  Germany  offering  complete  ‘virtual  courses’  at  the  moment.  At  the  University  of  Erlangen-Nuremberg  pilot 
applications  have  been  run  since  1995.  Now,  teleteaching  is  in  routine  operation.  First  experiences  made  at  the  Uni- 
versity of  Erlangen-Nuremberg  are  similar  to  those  of  projects  dealing  with  distance  learning  in  virtual  classrooms. 
Powerful  and  stable  audio  and  video  tools  are  crucial  to  realise  an  online  teleteaching  environment  which  is  accepted 
by  teachers  and  students.  After  this  being  achieved  first  empirical  studies  have  been  completed.  Different  groups  of 
students  involved  in  the  teleteaching  applications  as  well  as  teachers  have  been  observed  and  questioned. 

Here  some  characteristics: 

Remote  students:  The  major  advantage  they  expressed  is  increased  flexibility  in  place.  Especially  this  means  saved 
travel  time  which  goes  together  with  saved  money  for  public  and  private  transportation.  A secondary  positive  effect  for 
the  students  is  the  opportunity  to  get  in  touch  with  innovative  computer-based  communication  and  cooperation  sys- 
tems. 

Local  Students:  They  don’t  feel  affected  by  the  virtual  presence  of  remote  students.  Neither  positive  nor  negative  influ- 
ences could  be  observed.  As  a side  effect  digitized  lecture  recordings,  stored  on  a video  server,  are  highly  appreciated 
for  the  preparation  of  exams  or  simply  to  refresh  knowledge. 

Teachers:  On  the  one  hand  the  handling  of  the  technical  equipment  and  the  coordination  of  multiple  audiences  is 
demanding,  on  the  other  hand  a positive  aspect  is  the  higher  mutual  incentive  for  local  and  remote  participants.  The 
zeal  for  learning  is  believed  to  be  better  than  in  conventional  lectures. 

The  on-going  evaluation  process  is  accordingly  divided  into  the  phases  design  (definition  of  goals  and  objectives), 
development  (analysis  of  effectiveness  and  quality),  and  operation  (analysis  of  costs  and  benefits).  Table  1 shows 
some  selected  results  coming  from  an  enquiry  to  characterise  the  students’  basic  attitudes  towards  the  teleteaching 
project.  A l-to-7  scale  was  used  (1  = very  bad,  7 = excellent,  M = mean,  D = standard  deviation). 
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Figure  1:  German  Scientific  Network  Architecture 


Question  # 

Aspect 

Rating  (M) 

Dev.  (D) 

3 

overall  impression  of  the  teleteaching  project 

6.21 

1.30 

5a 

improvement  of  flexibility  in  place 

6.73 

0.43 

5b 

savings  of  travel  / commuting  expenses 

6.51 

0.55 

6a 

ease  of  use  of  the  communication  and  software  tools 

3.23 

1.92 

6b 

contact  with  the  remote  teacher,  trainer,  or  tutor 

3.11 

1.00 

6c 

contact  with  other  remote  students 

3.57 

1.51 

16 

demand  for  remote  access  from  home 

5.00 

3.53 

19a 

learning  efficiency  compared  to  conventional  courses 

4.33 

0.47 

19b 

opportunity  and  realisation  of  interaction 

4.07 

1.04 

Table  1:  Statistics  of  Selected  Students’  Comments 
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subgroup  of  children  with  a home  computer,  20.1  % never  of  rarely  used  a computer,  15.3  % used  the  computer 
about  once  a month,  37.9  % on  a weekly  basis,  and  26.6  % daily.  The  older  the  respondent,  the  higher  the  frequency 
of  daily  use  (X^  = 13.9  , p < .05). 

Chi^  testing  revealed  significant  differences  between  gender  and  computer  presence  at  home  (X^  = 14.1  , p < .001). 
Of  the  males,  85.0  % had  a home  computer,  as  compared  to  only  76.9  % of  the  females.  Although  more  boys  owned 
a home  computer  in  all  age  groups,  the  gender  gap  was  only  statistically  significant  for  12-,  16-  and  17-year  olds.  So 
the  assumption  that  gender  disparities  are  increasing  with  age  could  not  be  confirmed.  Overall,  girls  had 
significantly  less  computer  experience  (29.3  months)  than  boys  (37.1  months)  (F  = 23.2  , p < .001).  When  controlled 
for  computer  presence  at  home,  this  association  remained  significant  (F=10.8,p<.01).  In  addition,  males  (32.2  %) 
were  found  to  make  more  use  of  a computer  on  a daily  basis  than  girls  (12.0  %)  (X^  = 78.8  , p < .001). 

Results  indicate  that  gender  disparities  depend  on  the  nature  of  tasks  being  performed  with  computers.  The  use  of 
computers  for  leisure  activities  is  clearly  reported  as  being  a masculine  aciivily.  Significantly  more  boys  (73.7  %) 
reported  to  use  a computer  for  leisure  activities  than  girls  (58.6  %).  This  gap  remained  significant  when  children 
without  a home  computer  were  excluded  from  the  sample:  86.3  % of  the  boys  versus  75.5  % of  the  girls  (X^  = 20.2  , 
p < .001).  The  situation  appears  to  be  different  when  considering  computer  use  for  school  activities.  Similar 
proportions  of  girls  (58.5  %)  and  boys  (58.1  %)  use  a computer  at  home  for  school  activities,  but  if  respondents 
without  home  computer  were  not  taken  into  account,  significantly  more  girls  (75.3  %)  reported  to  use  computers 
than  boys  (67.9  %)  (X^  = 7.17,  p < .01). 

The  presence  of  a home  computer  was  significantly  lower  among  children  of  non-Belgian  origin:  73.9  % versus  82.5 
% (X^  = 8.0,  p < .01).  The  association  was  only  apparent  for  the  oldest  among  the  respondents  (17  years  or  older). 
Overall,  Belgian  respondents  reported  to  have  six  months  longer  experience  with  computers  (F  = 6.8  , p < .01). 
However,  when  the  effect  of  computer  ownership  was  imported  as  a covariate  in  the  test,  no  significance  was  found 
between  the  two  groups  (F  = 1 .4  , p = .23).  Moreover,  no  differences  between  groups  could  be  demonstrated  for 
frequency  of  computer  use  variable.  Respondents  of  Belgian  origin  did  not  report  to  use  the  computer  more  often  on 
a daily  basis  than  non-Belgian  respondents.  This  finding  is  confirmed  for  both  males  and  females.  Comparing 
Belgian  and  non-Belgian  respondents,  in  general  Belgian  students  accounted  for  a wider  use  of  computers:  8 % 
more  made  use  of  a computer  for  school  activities  (X^  = 8.8  , p < .05)  and  12  % more  for  leisure  activities  (X^  = 12.3 
, p < .01).  The  presence  of  a computer  at  home  seemed  responsible  for  the  difference  in  use  for  school  tasks.  In  other 
words,  if  respondents  without  a home  computer  were  removed  from  the  sample,  no  significance  was  found.  This 
finding  could  not  be  confirmed  for  leisure  use:  non-Belgian  respondents  made  significantly  less  computer  use  for 
free  time  activities,  even  when  controlled  for  computer  presence  at  home  (X^  = 4. 1 , p < .05). 

Discussion  and  conclusion 

The  present  findings  confirm  other  studies  which  clearly  have  demonstrated  gender  differences  in  computing. 
Besides,  this  study  suggests  the  existence  of  cultural  differences  in  computing.  Although  there  are  many  social  and 
psychological  factors  responsible  for  gender  and  cultural  inequalities,  the  absence  of  a home  computer  seems  to 
accentuate  the  differences  between  groups. 
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Tq  evaluate  the  use  of  newly  developed  IT  course  material  in  the  domain  of 
acoustics,  (http://educinno.rug.ac.be/wetenschapsmuseum),  the  students  had  to  perform 
an  open  design  project.  The  way  engineering  students  at  Ghent  University  tackled  the 
problem  and  used  the  available  tools  to  do  so,  is  evaluated  through  their  results  and  their 
opinion  expressed  in  a survey.  At  the  same  time,  their  global  ideas  about  the  usefulness  of 
information  technology  in  education  are  explored. 


Introduction 

To  evaluate  the  learning  environments  from  the  project  “Distributed  Development  of  a Virtual 
Science  Museum  with  Guide  Applied  to  education  in  Acoustics”  and  test  their  effectiveness  in  improving 
students’  insight  in  complex  formulas  and  concepts,  a suitable  student  project  had  to  be  built.  A design 
project  seemed  the  ideal  option  considering  the  numerous  alternatives  and  various  possible  ways  to  achieve 
an  acceptable  result  and  the  different  learning  paths  and  styles  to  be  used.  The  specifications  of  the  project 
corresponded  to  what  an  engineer  can  expect  in  real  life  situations.  Students  were  instructed  to  design 
Helmholtz  resonator:  basic  acoustics  were  known  in  advance,  but  the  students  had  to  explore  more  specific 
information  on  the  Helmholtz  resonator  themselves.  The  IT  project  provided  them  with  learning  objects 
containing  illustration  of  concepts,  mathematics,  models,  model  limitation,  simulation  tools,  etc. 

To  solve  the  problem  the  students  could  follow  five  different  problem  solving  paths.  The  path  of . . . 

1 . the  rote  experimenter:  look  briefly  at  the  general  operation  principle  and  use  the  virtual  experiment  to 
accidentally  come  up  with  a reasonably  good  solution. 

2.  more  intelligent  trial  and  error:  derive  a mathematical  relation  between  a few  design  parameters  from 
the  general  operation  principle  and  investigate  only  valuable  combinations  experimentally. 

3.  a meaningful  experimenter:  start  with  some  trial  and  error,  realize  there  must  be  a reason  why  some 
parameters  do  not  seem  to  have  any  influence,  turn  to  the  theory  to  find  out  if  there  is  a theoretical 
background  for  this  and  eventually  return  to  experiments  to  determine  the  last  degree  of  freedom. 

4.  a theorist  (pure  mathematics):  combine  the  available  mathematics  to  come  to  a result  possibly  using 
some  numerical  solvers. 

5.  an  expert:  use  the  formula  to  relate  some  parameters,  investigate  more  detailed  theory  to  find  some 
physical  and  technical  restrictions,  solve  the  last  degree  of  freedom  using  mathematics  and  their  own 
software  or  the  applet,  finally  turning  to  the  derivation  to  find  out  if  their  design  is  well  within  the 
limits  of  applicability  of  the  theory. 


Survey  and  exam  results 

The  students  reported  their  ideas  about  the  importance  of  the  several  factors  for  solving  the 
problem:  the  virtual  experiment,  derivation  of  equations,  limits  of  the  theory,  other  Internet  pages, 
communication  with  peers,  literature,  and  their  perception  of  the  insight  they  gained  in  some  acoustic 
concepts.  Despite  a small  statistical  population  of  27  students,  some  interesting  conclusions  can  be  drawn: 

• Part  of  the  goal  of  the  design  project  was  to  urge  students  to  learn  the  theoretical  background  by 
exploration  of  additional  resources.  About  66%  of  the  students  state  that  they  gained  insight  into  the 
theoretical  background.  We  consider  this  a reasonable  success. 
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Students  focussing  on  experiments  affirmed  they  gained  insight  in  the  problem  at  hand  but  they  did  not 
learn  about  the  theoretical  background.  They  probably  followed  paths  1 or  2 mentioned  above. 

The  more  students  perceive  the  equations  and  underlying  derivations  as  important  (solution  paths  3,  4 
and  5),  the  more  they  get  the  impression  they  understand  related  theoretical  topics.  However  the 
correlation  with  their  impression  of  understanding  how  a Helmholtz  resonator  really  works  gets  lower 
and  even  becomes  negative,  although  this  tendency  is  not  reflected  in  the  students’  exam  result  of 
questions  related  to  the  topic.  Perhaps  students  only  realize  the  complexity  of  the  matter  when  they 
study  the  topic  thoroughly. 

There  is  a negative  correlation  between  the  student’s  global  result  and  his  perception  of  insight  gained. 
This  can  mean  several  things:  the  material  is  not  good  enough  and  better  students  are  left  somewhat 
unsatisfied,  or  better  students  do  not  easily  state  understand  something  completely,  or  the  students  who 
perform  well  in  the  current  educational  system  are  not  inclined  to  put  much  effort  ;n  an  alternative 
project.  One  thing  is  sure:  it  is  a question  of  perception  since  there  is  no  negative  correlation  with  their 
examination  score  on  the  question  related  to  the  subject. 


This  project  forced  students  into  using  IT,  After  this  (for  some  students  ‘first’)  IT-experience  for 
educational  purposes  some  general  questions  about  the  appreciation  of  IT  were  asked.  The  following  trends 
reflect  their  ideas: 

• there  is  a strong  negative  correlation  between  the  wish  to  get  more  interactive  multimedia  material  (as 
was  presented  to  them)  and  the  student’s  performance  in  the  current  system. 

• the  students  almost  unanimously  approve  the  usefulness  of  self-assessment  tools  to  evaluate  the  level 
of  understanding  they  have  reached. 

• applets  allowing  students  to  explore  the  influence  of  parameters  in  a mathematical  theory  are  highly 
appreciated,  as  are  additional  self-assessment  tools. 

• the  possibilities  of  hypertext  to  hide  unessential  information  or  to  cross-link  different  topics  is  not 
estimated  very  high  by  the  average  Flemish  engineering  student. 


Conclusions 

In  general  students  appreciate  the  collaboration  and  the  exploration  type  of  approach  of  a design 
problem  on  a subject  they  had  no  prior  knowledge  of  The  exploration  stopped  at  an  earlier  stage  than 
hoped  for.  This  could  be  due  to  the  way  the  problem  and  the  tools  were  presented,  but  “the  study  culture” 
of  Flemish  engineering  students  probably  explains  a lot.  It  is  not  a habit  of  these  students  to  consult 
literature,  search  the  web,  etc.  When  using  IT  in  education  one  has  to  be  considering  the  study  culture  of 
the  students,  which  is  hard  to  change. 

There  was  an  unexpected  negative  correlation  between  the  general  performance  of  students  in  the 
traditional  educational  model  and  their  appreciation  of  the  project  and  IT  in  education  in  general.  Since  one 
of  the  initial  goals  of  this  project  was  to  make  sure  “better”  students  had  the  opportunity  to  learn  more  than 
in  the  traditional  educational  approach,  some  tuning  seems  appropriate,  in  order  not  to  loose  those  students. 

As  we  expected,  there  is  some  fear  of  loosing  track  and  of  doing  things  that  are  not  really  essential 
in  order  to  be  able  to  pass  the  exam.  In  the  first  semester  of  1999-2000  we  try  to  respond  to  this  by 
structuring  the  software  agent  we  called  “the  guide”  in  such  a way  that  a clear  path  through  the  material  is 
outlined  and  that  the  structure  becomes  more  evident.  Hyperlinks  are  applied  in  a more  structured  way  by 
the  use  of  icons  indicating  which  type  of  hyperlink  can  be  expected:  example,  practical  application, 
additional  explanation,  more  background,  etc. 

References 

[1]  Botteldooren  D.,  Vandaele  P,,  Pottie  S,  (1999),  Distributed  Development  of  a Virtual  Science  Museum 
with  Guide  Applied  to  Education  in  Acoustics:  First  Year’s  Experience,  Proceedings  of  ED-Media  1999, 
Association  for  the  Advancement  of  Computing  in  Education,  Seattle,  VA.  1515-1516, 


Malaysian  Smart  School  - Vision  Vs.  Reality 


Vijaya  Kumaran  K.K.Nair 
Multimedia  University 
Cybeijaya,  63000  Selangor,  Malaysia 
vi  @hvpermedia.com.m  V 


Abstract:  The  Smart  School  Project  is  one  of  the  flagships  of  the  Multimedia  Super  Corridor.  The 
Malaysian  government  ha.s  established  partnerships  with  the  private  sector  through  the  Concept 
Request  for  Proposals  (CRr?)  process.  Tliis  paper  will  critically  analyze  the  concept  of  Smart 
School  in  the  Malaysian  environment  and  look  into  the  issues  relating  to  implementation  of  this 
concept  as  per  the  vision  of  the  Ministry  of  Education  Malaysia.  The  paper  will  go  further  to  provide 
empirical  proof  with  regards  to  the  mind  set  and  the  response  to  the  Teaching-Learning  component 
of  the  Smart  School  Concept,  from  the  view  point  of  in  service  teachers. 


Introduction 

In  July  1997  the  Prime  Minister  launched  the  Flagship  Application  documents  to  invite  proposals  for  solutions  from 
the  private  sector  within  and  from  outside  the  country.  The  Education  Ministry  based  on  critical  and  creative 
teaching-learning  approach  planned  the  Smart  School  concept.  Technology  as  an  enabler  became  an  important 
component  of  this  flagship.  The  exercise  is  to  help  the  Malaysian  student  community  to  cope  with  the  Information 
Age. 


The  main  components  in  the  Malaysian  Smart  School  are: 

1.  Teaching  - Learning  Process  - here  self-accessed  and  self-directed  learning  will  be  encouraged 

2.  Management  and  Administration  - the  school  Management  and  Administration  will  be  technology  driven  for 
efficient  and  effective  management  of  resources  and  to  support  the  teaching-learning  process  in  1 above. 

3.  Human  Resources,  Skills  and  Responsibilities  - active  role  play  by  Parents,  Community  and  the  private  sector  as 
Stakeholders 

4.  Processes  - review  of  input  and  output 

5.  Technology  - to  be  used  as  an  enabler  i.e.  as  a means  to  the  end  and  not  as  an  end  to  the  problem 

6.  Policies  - changes  in  existing  policies  and  regulations  over  and  above  new  policies  and  regulations 


Objectives 

The  objectives  of  the  Smart  School  Project  are: 

1.  To  produce  a thinking  and  technology  - literate  workforce 

2.  To  democratize  education 

3.  To  increase  participation  of  stakeholders 

4.  To  provide  all-round  development  of  the  individual 

5.  To  provide  opportunities  to  enhance  individual  strengths  and  abilities 

Implementation 

The  Ministry  of  Education  has  planned  for  a three-year  Smart  School  Pilot  Project  through  the  CRFP  process. 
Ninety  (90)  pilot  schools  have  been  picked  for  the  pilot  stage  which  will  involve  three  levels  of  technology,  i.e.  level 
A,  B+,  and  B. 
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'LeVel^^ 

SiFotilfNumbier  of  Computers?  I 

|Pescriptibni.|;J®S'  fllSii..-?'.  ■■ 

A 

Primary  - 406 
Secondary  - 479 

Schools  in  this  level  will  be  equipped  with  state  -of-the-art 
technology.  Each  school  will  have  two  computer  labs  and 
each  classroom  and  science  lab  will  be  equipped  with 
computers. 

B+ 

86 

This  level  is  based  on  classroom  model.  This  model  is  based 
on  group-based  activities  and  project  work.  Computers  are 
available  whenever  needed  for  information  access,  writing 
reports,  developing  presentations,  e-mailing  etc. 

B 

42 

The  majority  of  schools  will  implement  technology 
supported  learning  by  use  of  a computer  lab.  Students  will 
use  the  to  orr-pcc  infnrmatinn  iisfi  software 

application  for  various  purposes  and  work  on  available 
courseware  in  the  four  subjects. 

The  Government  intends  to  play  a role  as  an  architect  and  driver  for  the  Smart  School  Project,  The  Ministry  of 
Education  will  prepare  guidelines  and  provide  the  basic  amenities  to  schools  according  to  their  individual  needs. 
The  Ministry  is  also  actively  encouraging  schools  to  become  Smart  Schools  on  their  own  initiative  by  using  their 
own  financial  resources  and  expertise.  The  Pilot  Project  period  is  three  (3)  years  from  28th  July  1999  to  22nd  July 
2002,  It  is  projected  that  all  schools  in  Malaysia  will  be  Smart  Schools  by  the  year  2010, 


Reality 

The  Malaysian  government  is  spending  over  RM,  300  million  for  only  90  schools  to  be  wired,  of  which  RM, 
183,573,737  for  Capital  Expenditure  and  RM,  116,426,263  for  Operating  Expenditure,  As  per  the  time  line  set, 
from  13^^"  March  - 22"^^  March  2000,  the  installation  of  Release  1 software  (Teaching-Learning  Materials  and  the 
Smart  School  management  System)  is  to  take  place. 

However  on  20^^  February  2000,  The  Star  (National  Daily)  reported  the  Director  - General  of  Education  saying  that 
the  “smart  school”  project  in  90  schools  nationwide  will  be  further  delayed  because  of  hitches  in  the  software 
development  for  the  lessons.  He  had  said  that  the  “slight  delay”  was  due  to  the  fact  that  it  took  time  to  develop  the 
courseware.  The  pilot  project  was  suppose  to  take  off  on  Jan  l*’^  1999,  but  was  delayed  for  several  reasons  including 
lack  of  funds  and  courseware.  The  infrastructure  for  the  implementation  is  ready  in  80  of  the  90  schools  for  the  full 
implementation.  The  Education  Deputy  Director  General  stated  that  the  software  for  the  4 subjects  took  time  to 
write,  and  to  evaluate  them.  He  went  on  to  say  that  it  will  take  about  4 years  to  fully  develop  and  implement  the 
program. 

Among  the  reasons  for  the  delay  were: 

♦ lack  of  fully  developed  smart  school  curriculum 

♦ lessons  being  rejected  several  times 

♦ difficulty  in  contacting  evaluators(Ministry  of  Education) 

♦ lack  of  communications 

♦ rejection  after  many  rounds  of  evaluation 

♦ evaluators  not  exposed  to  technological  limitations 

Should  the  consortium  entrusted  with  the  development  of  the  courseware  implement  software  engineering  principles 
in  developing  the  courseware?  If  the  basic  software  engineering  principles  had  been  incorporated,  couldn’t  most  of 
the  above  reasons  for  rejection  be  done  away  with? 
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Introduction 

One  of  the  primary  aims  of  educational  software  is  to  be  used  in  real  school  classrooms.  However,  a lot 
of  educational  software  has  been  criticised  that  has  not  been  designed  to  meet  the  needs  of  real  schools 
settings.  The  above  criticisms  show  that  there  is  a need  to  introduce  research  results  of  educational 
technology  into  the  environment  of  real  classrooms.  A solution  to  this  problem  may  be  the  involvement  of 
school  teachers  in  the  development  of  an  ITS  (Virvou  & Tsiriga  1999a)  and  the  application  of  evaluation 
methods  on  every  stage  of  the  tutoring  system's  life  cycle.  In  this  way  we  can  ensure  usability  and  learning 
effects. 

For  the  purpose  of  this  research,  a multimedia  tutoring  system  was  developed.  The  system  is  called 
EasyMath  and  incorporates  intelligence.  EasyMath’s  model  of  life  cycle  has  been  based  on  Object  Unified 
Process  (Kruchten  1999),  an  object  oriented  process  that  suggests  multiple  iterations  of  the  software 
development  phases.  In  this  way,  we  have  ensured  multiple  iterations  of  evaluation  of  the  design  and 
executable  releases  of  EasyMath. 

Evaluation  of  the  Design 

Evaluation  of  the  design  of  EasyMath  was  considered  very  important  in  order  to  ensure  that  the 
expertise  incorporated  in  the  error  diagnosis  component  would  be  acceptable  by  the  majority  of  school 
teachers.  EasyMath’ s error  diagnosis  and  student  modelling  have  been  based  on  reconstructive  approaches 
which  have  been  used  in  many  ITSs  such  as  (Brown  & Burton  1978).  These  systems  reconstruct  the 
problem  solving  process  and  generate  mal-rules  from  hypothesized  faulty  solution  paths  which  are  used  for 
the  modelling  of  students’  misconceptions  or  procedural  bugs. 

At  this  stage  we  aimed  at  evaluating  the  error  diagnosis  component  in  terms  of  its  cognitive  principles 
and  completeness.  Therefore,  ten  school  teachers  were  given  a computer  based  questionnaire  with  questions 
that  had  been  answered  erroneously.  The  faulty  answers  were  produced  using  the  library  of  the  most 
common  misconceptions  that  were  encoded  in  EasyMath.  The  questions  covered  the  whole  range  of 
exercises  included  in  EasyMath.  The  questionnaire  consisted  of  ten  questions.  In  each  one  of  the  questions, 
each  teacher  had  to  provide  a justification  of  what  s/he  thought  the  underlying  misconception  was.  One 
problem  in  this  stage  of  the  evaluation  process  was  that  many  teachers  did  not  provide  any  explanation  and 
just  gave  the  correct  answer  to  the  question. 

Evaluation  of  a Primary  Executable  Release 

In  an  early  stage  of  the  development,  a prototype  of  the  tutor  was  ready  to  be  used  by  its  end  users. 
This  prototype  was  used  in  order  to  perform  a primary  evaluation  of  the  system.  In  the  phase  of  the  primary 
release  evaluation,  two  school  teachers  along  with  ten  students  were  involved.  The  evaluation  methods 
employed  comprised  qualitative  techniques,  such  as  direct  observation  and  questionnaires. 

The  main  focus  of  this  phase  was  on  the  evaluation  of  the  user  interface  and  the  several  types  of  the 
exercises  provided  by  the  system.  Teachers  and  students  were  asked  to  comment  on  the  working 
environment  of  EasyMath.  Furthermore,  they  had  to  express  their  opinion  about  whether  the  amount  of 
different  exercises  encoded  as  well  as  the  form  of  these  exercises  were  adequate. 

The  teachers  and  the  students  involved  in  this  phase  were  introduced  to  EasyMath  and  were  then  asked 
to  use  it  for  about  an  hour.  During  their  interaction  with  the  system,  an  evaluator  was  present,  trying  to  note 
down  the  difficulties  that  the  users  had  encountered  while  performing  predefined  tasks.  After  the  interaction 
with  EasyMath,  both  the  teacher  and  the  students  were  given  a questionnaire  to  answer.  The  questionnaire 
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was  designed  taking  into  account  a set  of  “usability  heuristics”  presented  in  (Nielsen  1994).  The 
questionnaire  included  questions  such  as: 

1 . Were  instructions  for  use  of  the  system  visible  or  easily  retrievable  whenever  appropriate? 

2.  Were  the  symbols  and  the  words  used  in  the  different  components  of  EasyMath  consistent? 

The  first  question  corresponds  to  the  need  for  recognition  rather  than  recall,  while  the  second  question 
is  related  to  the  need  for  consistency  and  standards.  The  comments  made  during  this  stage  of  EasyMath's 
development  were  used  as  the  basis  for  the  refinement  of  the  requirements  of  the  system  and  the 
construction  of  the  final  product. 

Formative  Evaluation  Using  a Set  of  “Learning  with  Software”  Heuristics 

The  involvement  of  school  teachers  in  the  phase  of  the  evaluation  of  an  ITS  was  considered  crucial.  In 
addition,  it  is  also  important  to  track  students'  use  of  resource  based  packages  very  closely  to  uncover  the 
problems  and  successes.  For  the  above  reasons,  in  the  phase  of  the  formative  eva-lua.tion  of  Ea.sy]S4ath.  1() 
school  teachers  as  well  as  240  students  were  involved.  Both  students  and  teachers  were  asked  to  evaluate 
EasyMath  in  terms  of  the  purpose  of  education.  In  addition,  teachers  were  asked  to  evaluate  the  overall 
performance  of  the  system  and  express  their  opinion  about  the  usability  of  such  an  ITS  in  a real  classroom. 

In  the  stage  of  the  students'  evaluation  of  EasyMath,  the  students  were  separated  in  two  portions.  The 
first  120  students  were  introduced  to  EasyMath  and  were  asked  to  use  it  for  an  hour.  Then,  they  were  given 
a questionnaire  to  complete.  For  a more  detailed  description  of  the  results  of  the  questionnaire,  the  reader  is 
referred  to  (Virvou  & Tsiriga  1999b).  As  for  the  rest  120  students,  they  were  taught  half  of  the  syllabus  in 
algebraic  powers  without  the  use  of  EasyMath  and  they  were  given  a written  test.  Then  they  used  EasyMath 
while  being  taught  the  rest  of  the  syllabus.  In  the  end  they  were  given  another  written  test  and  the  grades  of 
their  first  and  second  test  were  compared.  The  results  showed  that  46%  of  the  students  obtained  a better 
grade  in  the  second  test,  43%  obtained  the  same  grade  and  only  1 1%  obtained  a lower  grade  in  the  second 
test. 

The  evaluation  questionnaires  used  in  this  phase  were  constructed  based  on  a set  of  "learning  with 
software"  heuristics,  introduced  by  (Squires  & Preece  1999).  These  heuristics  are  an  adaptation  of 
“usability  heuristics”  (Nielsen  1994),  so  as  to  relate  to  socio-constructivist  criteria  for  learning.  "Learning 
with  software"  heuristics  have  been  suggested  by  their  inventors  as  a method  for  performing  predictive 
evaluation.  However,  we  have  used  them  to  construct  a formative  evaluation  questionnaire,  that  would  give 
us  insight  about  the  integration  of  both  usability  and  learning  issues. 

In  the  phase  of  the  overall  evaluation  of  EasyMath,  the  teachers  who  participated  in  the  evaluation  were 
asked  to  role  play  an  average  student  interacting  with  EasyMath.  Next,  they  were  given  a questionnaire  to 
fill  in.  The  questionnaire  was  carefully  designed  so  as  to  ensure  that  questions  are  related  to  as  many  of  the 
"learning  with  software"  heuristics  as  possible. 

Conclusions 

To  produce  efficient  educational  software,  there  is  a need  to  ensure  the  correctness  of  the  domain 
knowledge  as  well  as  to  evaluate  the  usability  of  the  product.  One  way  to  achieve  the  above  goals  may  be 
the  application  of  evaluation  methods  on  every  stage  of  the  tutoring  system's  life  cycle.  In  addition, 
evaluation  should  involve  both  human  teachers  and  students  in  order  to  maximise  usability  in  real 
classroom  settings. 
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Abstract  This  work  in  progress  paper  reports  on  whether  hybrid  CD-ROM  technology  used  to  train  or  supplement 
the  tTuirting  of  mentors  of  beginning  teachers  in  VoCttuonal  Education  and  Training  is  an  improvciucru  u^^on 
traditional  face  to  face  and  print-based  delivery.  It  argues  that  the  shifting  ground  in  pedagogical  models  can  confuse 
educational  software  developers  but  an  understanding  of  a theoretically  driven  level  of  explanation  of  learning  may 
assist  decisions  about  learning  approaches  and  choice  of  technology. 


Introduction. 

Faced  with  a climate  of  change,  educational  institutions  are  under  increasing  pressure  to  explore  new  ways  of  learning.  Emerging  trends  in 
Information  and  Communication  Technologies  (iCTs),  combined  with  national  governments  initiatives  to  use  ICT  and  widen  participation  in  lifelong 
learning  (in  the  case  of  the  UK,  the  influential  Dearing,  Higginson  and  Kennedy  reports),  have  created  a framework  for  supporting  an  increasing 
community  of  participants  in  education  and  training. 

The  development  and  introduction  of  ICTs  into  the  further  and  higher  education  and  training  sectors  has  stirred  debate  about  pedagogical 
performance  versus  educational  efficiency.  In  training,  the  debate  has  focused  on  issues  of  Return  on  Investment  (Hawkins,  et  al  1 997).  Factors  of  cost 
and  value  for  money  versus  amount  of  learning  and  quality  of  learning  characterise  educational  and  training  evaluations  A lack  of  separation  of  these 
factors  has  created  a cloud  of  confusion,  and  decisions  about  allocations  of  scarce  resources  are  not  always  made  in  a manner  that  maximises  the 
collective  benefit  of  the  community. 

Many  would  contend  that  ICTs  are  not  only  cost  efficient  (Boulet  & Sene,  1997;  Greenwood  & Recker,  1996)  but  they  also  promote  more 
effective  learning  (Resnick,  1987;  Fjuk  & Dirckinck-Holmfield  1999).  Others  call  for  more  caution  to  be  exercised  about  such  claims.  In  the  UK,  the 
push  towards  the  integration  of  computer-based  education  within  education  and  training  has  been  made  at  the  highest  levels  with  funding  to  match. 
The  premise  is  that  ICT  can  deliver  more  effective  learning  than  traditional  media. 

There  is  no  one  single  recognised  right  way  of  thinking  about  learning;  theories  constantly  change  and  tend  to  reflect  the  attitudes  and 
technologies  of  their  time.  The  literature  points  to  shifting  ground  in  pedagogical  models.(Reiguluth  1999;  Herrington  & Standen,  1999).  Currently 
the  debate  is  influenced  by  ICT. 

It  is  of  some  concern  therefore,  that  educational  software  developers  have  an  understanding  of  a theoretically-driven  level  of  explanation  that 
identifies  the  kinds  of  cognitive  activities  (such  as  learning,  memory  tasks  and  problem-solving)  that  could  take  place  through  the  physical  activities 
provided  by  an  interface.  To  produce  successful  tools  for  learning,  it  is  necessary  to  understand  the  nature  of  the  learning  process  and  the  impact  of 
context  within  which  learning  operates. 

Background 

There  is  a particular  need  at  the  University  of  Greenwich  each  year  to  train  up  to  250  new  and  existing  workplace  mentors  who  support  trainee 
teachers  on  the  Post  Graduate  Certificate  of  Education  (PGCE)  in  Post  16  Vocational  Education  and  Training  (VET)  during  their  teaching  practice. 

The  learning  demands  and  needs  of  these  mentors  are  relatively  different  to  the  more  conventional  learner  and  it  may  be  argued  that  the 
instructional  approach  is  different.  Firstly,  it  concerns  the  training  of  trainers.  Secondly,  there  is  no  set  curriculum  to  follow,  or  assessments  to  be  made; 
thirdly,  there  are  no  prerequisites  other  than  being  a teacher/trainer  within  the  selected  institution  and  lastly,  whilst  the  institution  is  paid  for  the  efforts 
of  their  staff,  the  mentors  personally  go  largely  unrewarded  either  by  time,  cash  or  qualification  inducements. 

Although  some  mentors  may  work  within  the  same  organisation,  many  others  do  not  and  generally  they  work  largely  independently  of  one 
another.  Institutions  span  a large  geographical  area  across  London  and  the  South  East  of  England  and  mentors  rarely  meet  with  one  another  to  discuss 
their  role. 

There  is  no  substantial  proof  to  suggest  that  a trained  mentor  is  more  effective  than  an  untrained  one  (Jowett  1998),  however,  training  does 
enhance  an  understanding  of  the  stages  of  a relationship,  the  mentoring  process  and  it  enables  critical  reflection  on  professional  practice.  Conventional 
mentor  training,  by  virtue  of  location,  time  and  resources,  can  only  address  a small  number  of  aspects  of  the  mentoring  process  - the  tip  of  the  iceberg. 
Malderez  and  Bodoezky  (1998),  take  the  image  of  the  iceberg  further  and  draw  a parallel  between  the  tip,  which  is  out  in  the  air  and  represents  the 
visible  ‘good  professional’  and  the  main  body  of  the  iceberg,  which  lays  unseen  and  represents  all  that  the  professional  engages  in  as  an  expert  in  their 
subject  and  as  a mentor.  The  submerged  part  represents  the  mentor’s  thinking,  planning,  and  engagement  in  process,  and  draws  upon  their  knowledge, 
understanding,  values,  feelings  and  attitudes.  The  mentor’s  ability  to  respond  to  the  student  teacher’s  professional  development  hinges  on  their 
understanding  of  the  role  and  on  their  own  development  as  mentors.  An  analysis  of  these  elements  forms  the  content  of  the  interactive  tool. 

Project  Objectives  and  Scope. 

In  its  current  form,  training  is  administered  through  a combination  of  print-based  distance  material  and  a single  workshop  session  for  some 
mentors  at  the  beginning  of  the  academic  year.  Our  initial  research  showed  that  the  majority  of  mentors  would  be  willing  to  use  IT  for  training 
purposes,  although  we  identified  a number  of  constraints;  some  were  institutional  such  as  the  limitations  of  the  technical  specification  of  equipment 


provided  by  colleges  (processor  speed,  CD-ROM  drives,  soundcards,  the  ability  to  play  video,  etc)  and  some  personal  constraints  such  as  knowledge 
about  ICT  and  confidence  in  using  the  computer.  The  pursuit  of  a technological  solution  to  provide  support  and  training,  independent  of  time  and 
location  was  compelling. 

In  1998  the  Mentor  in  cyberspace  project  undertook  to  produce  an  alternative  means  of  supplementing  the  existing  mentor  training  and  support. 
Interactive  multimedia,  it  appeared  to  us,  offered  the  best  chance  of  developing  a range  of  tools  and  resources  to  support  mentor  processes  such  as 
thinking,  creativity,  self-reflection,  problem  solving  and  the  development  of  professional  attitudes  & approaches  over  time.  At  the  start  of  the  project 
only  a small  number  of  academic  institutions  which  were  not  universities  had  organised  individual  internet  access  for  their  staff.  We  considered  the 
lead-in  time  for  the  project  to  be  sufficiently  long  to  utilise  the  hybrid  function  of  CD-ROM  technology  with  its  ability  to  connect  on-line.  Furthermore, 
the  University’s  on-line  Campus  (OLC)  was  under  construction  and  the  idea  of  extending  the  range  and  type  of  tools  available  to  mentors  was 
attractive. 

Learning  approaches. 

Scleeiiiig  insiruciionai  methods  on  the  basis  of  the  theories  has  not  easy  because  little  agreement  exists  between  the  main  protagonists.  The 
constructivists  attack  instructivist  design  approaches  as  formulaic  and  mechanistic.  Defenders  of  instructivist  design  have  attempted  to  re-establish  its 
authority  at  the  expense  of  all  other  approaches.  They  take  a staunch  position  against  the  infringements  of,  in  Merill’s  words,  “these  persons  who  claim 
that  knowledge  is  founded  on  collaboration  rather  than  empirical  science,  or  who  claim  that  all  truth  is  relative”  (Merill,  cited  in  Wild  and  Quinn 
1998,  p 74).  In  much  of  the  current  and  recurring  debate  about  the  role  of  learning  theory  in  educational  interactive  multimedia  there  appears  to  be  a 
readiness  to  assert  one  theory  of  learning  over  another  - to  present  one  as  being  deficient  and  the  other  as  the  only  credible  explanation  of  learning. 

The  important  issue  is  to  match  the  instructional  approach  to  learning  need  whatever  the  level,  whether  this  is  the  efficient  dissemination  and 
testing  of  knowledge,  the  creation  of  new  personal  knowledge  or  the  development  of  opportunities  for  networking  and  collaboration.  In  a professional 
context  where  the  goal  is  to  encourage  the  acquisition  of  reproductive  and  productive  skills  and  knowledge,  a balance  must  be  struck  between  those 
methods  that  appear  to  work  best. 

Our  approach  has  been  to  try  to  strike  a balance  between  these  two  ostensibly  opposing  theories  and  adapt  the  technology  to  suit  the  perceived 
learning  needs  of  the  mentors.  On  analysis  of  the  strengths  of  each  learning  approach  in  relation  to  our  purposes,  we  saw  that  instead  of  contradicting 
and  negating  each  other,  these  two  approaches  could  be  complementary.  We  perceived  that  the  use  of  hybrid  CD-ROM  would  combine  both  learning 
approaches.  CD-ROM  is  a relatively  closed  technology,  limited  by  its  content  and  design.  Our  design  here  draws  upon  the  strengths  of  Instructional 
Systems  Design  (ISD);  its  emphasis  on  presentation,  information,  structure  and  reinforcement  is  useful  for  the  reproduction  of  knowledge  (Boyle 
1998)  and  is  suited  to  interactive  multimedia  on  CD-ROM.  Web  technology,  on  the  other  hand,  is  characterised  by  an  open,  explorative  architecture. 
Accordingly  we  have  used  web  technology,  specifically  Computer  Mediated  Communications  (CMC)  which  supports  a more  constructivist  approach: 
it  is  able  to  provide  a set  of  tools  and  methods  for  the  mentor  to  construct  new  knowledge  within  ill -structured,  authentic,  real  contexts  (Brown,J.S., 
Collins,  A.,  & Duguid  1989).  Hybrid  CD-ROM  offers  the  possibility  of  combining  the  reproduction  of  knowledge  with  the  production  of 
‘knowledgeability’  (Guile  & Young  1998). 

This  work  in  progress  will  report  the  findings  of  a pilot  study  ending  in  June  2000  with  approximately  20  mentors  who  will  have  utilised  the 
various  tools  and  approaches  during  this  academic  year.  The  particular  focus  of  the  research  will  examine  whether: 

• mentors  preferred  to  use  the  hybrid  CD-ROM  or  the  print  based  material  for  learning  about  their  role 

• one  instructional  approach  or  technology  was  more  successful  than  the  other. 

I will  attempt  to  draw  conclusions  about  the  success  of  introducing  interactive  multimedia  and  whether  the  use  of  ICT  in  this  context  is  an 
advantage  for  training  and  supporting  mentors  of  beginning  teachers  in  VET 

Bibliography 

Boulet  M & Sene  L (1997)  Investing  in  Computerised  Training:  The  Interactive  Decision  Aid  System.  Conference  Proceedings-  Ed  Media/Ed-Telecom. 
Calgary  Canada  1998,  pi 86. 

Boyle  T.  (1997)  Design  for  Multimedia  Learning.  Prentice  Hall 

Brown,  J.  S.,  Collins,  A.,  & Duguid,  P.  (1989/  Situated  cognition  and  the  culture  of  learning  Educational  Researcher,  /^(l),  32-42. 

Dearing  (1997)  National  Inquiry  into  Higher  Education.  HMSO 

FEFC  (1996)  Report  of  the  FEFC  Learning  and  Technology  Committee  (The  Higginson  Report),  Further  Education  Funding  Council. 

Fjuk  A,&  Dirckinck-Holmfeld,  L:  “Articulation  of  Actions  in  Distributed  Collaborative  Learning".  (To  be  published  in  Scandinavian  Journal  of 
Information  Systems)  (online)  http://www.infostream.no/?  1 92 

Greenwood  J & Recker  M.  “Networked,  Asynchronous  Student  Evaluations  of  Courses  and  Teaching:  An  Architecture  and  F . Proceedings  of 

ED-MEDIA  96  & ED -TELECOM  96.  Boston,  USA,  1996 
Guile  & Young  (1998)  Journal  of  Education  and  Training  50  (2)  (online)  http://www.triangle.co.uk/vae/index.htm 

Hawkins  C,  Gustafson  K & Nielsen  T.  “Return  on  Investment  for  Electronic  Performance  Support  Systems:  A Case  Study”.  Conference  Proceedings-  Ed 
Media/Ed-Telecom.  Calgary  Canada  1998,  pl519. 

Jowett,  V 1998.  lurking  for  a degree  — a mentoring  project  in  work  based  learning  Facets  of  mentoring  in  higher  education  2.  SEDA  paper  103. 19  —34. 
Malderez,A.  1998.  Addressing  tensions  in  the  mentoring  relationship  - interpersonal  skills  training.  Facets  of  mentoring  in  higher  education  2.  SEDA 
paper  103.49  -57. 

Herrington  & Standen  1999.  Moving  from  an  Instructivist  to  a Constructivist  Multimedia  Learning  Environment,  in  Ed  Media  99  Proceedings.  Seattle, 
USA  99  AACE  Virginia. 

Kennedy,  H {\991)  Learning  l^rks:  Widening  Participation  in  Further  Education  Coventry;  FEFC 

Reigluth  C (Ed).  {\999)  Instructional-Design  Theories  and  Models  Volume  2 . Lawrence  Erlbaum  Associates,  London 

Resnick,L  1987  “Learning  in  School  and  Out".  Educational  Researcher{i6),  13-20 

Wild  M.  & Quinn  C,  (1998)  “Implications  of  Educational  Theory  for  the  Design  of  Instructional  Multimedia”.  British  Journal  of  Educational  Technology 
29(1)73-81. 


3EST  COPY  AVAILABLE 


Visual  Lab  - A Multimedia  Virtual  Experiment  Environment  on  WWW 


Chao-Chiu  Wang 

Department  of  Information  and  Computer  Engineering 
Chung  Yuan  Christian  University,  Chung  Li,  32032  Taiwan 
mfc42@ms24.hinet.net 

Maiga  Chang 

Department  of  Information  and  Computer  Engineering 
Chung  Yuan  Christian  University,  Chung  Li,  32032  Taiwan 
maiga@ms2.hinet.net 

Chang-Kai  Hsu 

Department  of  Information  and  Computer  Engineering 
Chung  Yuan  Christian  University,  Chung  Li,  32032  Taiwan 
ken@mcsl.ice.cycu.edu.tw 

Jia-Sheng  Heh 

Department  of  Information  and  Computer  Engineering 
Chung  Yuan  Christian  University,  Chung  Li,  32032  Taiwan 
jsheh@ice.cycu.edu.tw 


Abstract:  This  paper  proposes  an  open  architecture  of  Visual  Lab  on  WWW  and  focuses  on 
scientific  experiment  for  the  asynchronous  mode  of  distance  learning.  Visual  Lab  is  a virtual 
experiment  environment  by  using  video-media.  A complete  scientific  experiment  in  Visual 
Lab  includes  four  essential  stages:  phenomenon  observation,  experiment  process  observation, 
data  measurement  and  experiment  recording.  In  Visual  Lab,  students  can  carry  out 
experiments  remotely  with  the  toolbox  designed  on  Internet.  A system  architecture  built  on 
World-Wide  Web  demonstrates  the  feasibility  and  flexibility  of  our  idea. 
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1.  Analysis  of  Scientific  Experiment  in  Visual  Lab 


For  a full  scientific  experiment,  there  are  several  essential  stages:  phenomenon  observation,  experiment 
process  observation,  data  measurement  and  experiment  recording.  These  essential  stages  are  analyzed  details  in 
the  following. 

Phenomenon  Observation:  With  such  knowledge,  phenomenon  observation  can  be  divided  into  two 


categories 

(a)  Static  Observation . 

(b)  Dynamic  Observation. 

Experiment  Process  Observation:  It  is  very  important  to  practice  with  one’s  own  hands.  After 


experimenting  themselves,  students  can  gain  much  impression. 

(a)  Traditional  Process  Observation. 

(b)  Video  Observation. 

Data  Measurement:  Data  measurement  is  the  key  to  prove  a physics  theory. 

(a)  Equipment  Usage  in  a Traditional  Lab. 

(b)  Usage  of  Simulation  Tools. 

Experiment  Recording:  The  formats  of  experiment  records  have  to  be  designed. 

Problems  Analysis  of  Four  Essential  Conditions  on  WWW:  The  formats  of  experiment  records  have  to 
be  designed. 


(a)  Video  Format 


Compress 

Network  Transfer 

MPEG 

No 

No 

Real  Video 

No 

Yes 

Optional  |No 

Table  1:  Video  formats  comparison. 


AVI 


(b)  Component  Format; 


Download 

Security 

Control  Operation 

Browser  Support 

Java  Applet 

Light 

No 

Can  not  access  local  files 

Both  IE  and  Netscape 

Real  Video 

Heavy 

No 

Can  not  control 

Both  IE  and  Netscape 

Window  Object 
embedded  in  Web  page 

No 

No 

Can  not  control 

IE 

ActiveX 

Heavy 

Yes 

Full  control 

IE 

Application 

Heavy 

No 

Full  control 

Both  IE  and  Netscape 

Table  2:  Comparisons  table  of  web  technologies. 

(c)  Experiment  Record:  Students  can  record  data  what  they  want  to  measure  and  send  out  these  data 
to  teacher.  Then  teachers  can  gather  all  the  student  learning  information  via  the  fanction  provided 
by  experiment  record. 

2.  Design  the  Platform  of  Visual  Lab 

In  this  section,  several  toolboxes  will  be  discussed.  Each  measurement  and  observe  components  must 
applied  these  toolboxes  in  Visual  Lab  platform. 

Measuring  and  Observing  Tools:  For  a Visual  Lab  System,  measuring  and  observing  tools  are  helpful  to 
both  students  and  teachers 

(a)  Video  Controller:  To  play  some  experiments  video. 

(b)  Scale:  In  order  to  measure  the  actual  length  from  the  experiment  video. 

(c)  Ruler:  Students  can  adjust  the  length  of  this  ruler  tool  to  can  get  the  length  what  they  adjust  when  they 
saw  in  the  video. 

(d)  Protractor:  Students  can  measure  the  angular  magnitude. 

(e)  Calculator:  Students  can  input  the  data  and  get  the  result  just  like  a calculator. 

(f)  Timer:  A timer  is  needed  to  measure  time  in  scientific  experiment. 

Components  of  Visual  Lab:  Each  learning  work  is  implemented  by  an  independent  component,  and  our 
architecture  can  be  illustrated  as  (Fig.  1)  shown. 

3.  Implement  the  Visual  Lab  on  WWW 

We  designed  a platform  using  open  architecture  technology.  Our  purpose  is  that  all  of  the  physics 
experiments  can  apply  in  Visual  Lab  without  any  modification.  As  long  as  teachers  film  the  experiment  and  join 
in  Visual  Lab,  students  can  experiment  in  Visual  Lab  environment  and  record  with  Microsoft  Excel  2000,  as 
show  in  (Fig.  2). 

4.  Conclusion 

This  paper  proposes  an  open  and  reusable  architecture  of  Visual  Lab  on  WWW.  All  of  the  scientific 
experiments  can  apply  to  Visual  Lab  based  on  the  architecture  of  Visual  Lab  had  been  accomplished.  Learners 
can  learn  knowledge  by  experiment  in  the  Visual  Lab  even  they  are  not  at  school.  Solution  for  transferring  video 
stream  through  network  should  be  the  next  research  direction  to  enhance  Visual  Lab. 
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Abstract.  Two  groups  of  teachers,  50  K-12  History  teachers  from  California  and  20 
K-12  teachers  in  a Masters  program  in  Massachusetts,  participated  in  an  online 
experience  to  develop  instructional  materials  which  integrate  technology  into  the 
curriculum.  Each  groups’  level  of  participation  in  online  communication  activities 
and  the  method  that  their  instructional  design  process  was  developed  predicted  the 
quality  of  the  final  materials.  Recommendations  for  structuring  online  collaborative 
activities  for  teachers  to  develop  instructional  materials  are  given. 


Introduction 

This  is  a comparison  of  two  on-line  collaboratives  with  the  same  purpose  — to  have  teachers  develop  K-12 
technology-integrated  instructional  materials  — but  significantly  different  approaches.  The  two  projects  we  are 
using  for  this  comparison  are  a California  statewide  professional  development  project  in  K-12  History  (CH- 
SSP)  and  an  on-line  course  in  Project  Based  Learning  for  a Masters  degree  program  in  Educational  Technology 
at  Framingham  State  College  in  Framingham,  Massachusetts. 

Key  Factor  - Participation  in  Communication  Activities 

For  all  participants,  developing  instructional  materials  was  a process  that  involved  participant  to  participant,  and 
participant  to  facilitator/instructor  communication  using  an  online  space  and  several  forms  of  electronic 
communication.  There  were  six  different  forms  — online  chat,  face-to-face,  online  submission  of  original 
written  material,  online  response  to  others  written  material,  participation  in  the  online  discussion  forums  and  e- 
mail.  The  greater  the  participation  in  online  activities,  the  more  probable  that  the  lesson  plan  was  completed  and 
the  more  likely  that  those  lesson  plans  were  of  high  quality.  There  were  significant  differences  between  the 
groups  in  their  participation  in  on-line  activities.  The  professional  development  group  participated  significantly 
less  than  the  online  course  group.  There  were  two  primary  reasons  for  this.  Although  both  groups  were 
required  to  participate,  there  was  no  tangible  consequence  for  not  participating  for  the  professional  development 
group.  Second,  technical  resources  and  technical  help  were  more  readily  available  for  the  online  course 
participants  than  the  professional  development  participants. 

The  greatest  disparity  between  the  groups  was  in  the  discussion  area.  The  postings  for  the  professional 
development  group  were  sparse  to  non-existent.  No  discussion  ever  emerged.  For  the  online  course,  this  was 
unquestionably  the  most  successful  communication.  The  discussions  were  interesting,  in-depth,  meaningful  and 
at  the  Masters  level.  The  major  difference  between  these  groups  is  that  the  discussion  was  participant  driven  for 
the  professional  development  group  and  instructor  driven  for  the  online  course  group.  We  find  this  to  be 
problematic,  as  one  of  the  goals  of  the  professional  development  is  to  have  the  participants  drive  the  interaction. 
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Key  Factor  - Instructional  Design  Process 

An  important  component  of  the  CH-SSP  project  was  the  development  of  the  instructional  design  process. 
CHSSP  was  interested  in  having  the  different  grade  level  groups  develop  their  own  process  for  the  development 
of  the  lesson  plans.  In  the  Project  Based  Learning  course,  conversely,  the  instructor  strictly  imposed  the  process 
and  the  students  were,  in  part,  evaluated  on  how  they  used  this  imposed  process.  We  refer  to  this  difference  as  a 
self- generated  process  (CHSSP)  vs.  an  imposed  process  (Framingham). 

The  different  process  experience  led  to  significant  differences  in  the  final  product.  The  self-generated  process 
group  spent  a great  deal  of  time  developing  the  process.  As  this  was  a new  experience  for  them,  there  was 
confusion  regarding  what  was  being  asked  of  them.  This  detracted  from  the  application  of  those  processes  to  the 
instructional  materials  development.  We  still  feel  strongly  that  we  would  like  to  see  the  process  of  materials 
development  arise  from  the  teachers  that  are  using  the  materials.  We  feel  changes  need  to  be  made  to  focus 
more  on  the  development  of  the  materials  themselves. 

Recommendations 

We  have  several  recommendations.  There  were  many  difficulties  with  the  professional  development  institute, 
most  of  them  centered  around  participants  actively  engaging  in  the  online  communications,  and  participants 
understanding  facilitator  expectations  in  relationship  to  the  process  of  instructional  materials  development.  We 
would  prefer,  encourage  and  work  towards  that  process  being  developed  by  the  participants  themselves. 
However,  many  of  the  participants,  from  both  groups,  felt  the  need  for  more  structure  and  more  direction  in 
how  to  go  about  developing  the  materials.  We  do  not  know  if  this  was  because  they  were  using  a new 
environment  and  felt  unsure  about  how  to  proceed,  or  if  they  would  have  difficulty  developing  instructional 
materials  without  an  imposed  process  in  any  environment.  In  the  future.  In  the  professional  development 
project,  some  direction  needs  to  be  provided  for  the  development  process,  however,  this  process  should  remain 
fluid  for  the  participants.  In  the  online  course,  we  would  recommend  that  a portion  of  the  beginning  of  the 
course  be  spent  with  the  participants  examining  their  own  practices  of  instructional  materials  development. 

The  professional  development  institute  needs  to  provide  a more  concrete  reason  for  the  participants  to  actively 
participate  in  the  online  environment.  Providing  participants  with  a 'reward'  at  the  end,  such  as  a stipend. 
University  Extension  credit  towards  salary  points,  or  the  like  may  be  an  important  component  to  motivate 
participants.  Technical  support  is  also  essential.  It  may  be  best  that  technical  support  be  pledged  from  the 
district  as  that  is  frequently  the  ISP  for  the  teacher.  It  also  provides  a district  commitment  to  the  project. 

In  general,  online  instructional  materials  development,  in  a group,  collaborative  setting,  needs  a structure  to 
support  it  which  includes  facilitator  support  for  the  multiple  forms  of  online  communication,  technical  support, 
and  clear  guidelines  for  the  participants  process.  In  addition,  participants  need  a tangible  motivation  for 
completing  the  project,  such  as  salary  points  or  a stipend.  These  online  development  projects  differ  from 
traditional  development  projects  in  that  they  can  be  conducted  asynchronously  and  between  participants  who 
are  at  a geographic  distance.  For  the  future,  we  hope  that  the  communities  that  are  formed  can  continue  on, 
develop  further  materials  and  expand  to  provide  online  support  for  other  teachers  in  their  subject  areas  or  grade 
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Abstract:  This  short  paper  illustrates  a framework  for  examining  the  application,  and  misapplication,  of  media  types  and 
contexts  in  educational  settings.  It  illustrates  a relationship  between  form,  format  and  function  that  is  useful  in  determining  the 
effectiveness  of  new  media  technologies  in  educational  contexts. 


Yoiii'  liOiiic:  has  a drawer  or  box  in  some  convenient  place  where  you  can  always  reach  the  most  useful  tools;  a flashlight,  a ruler,  a roll  of  tape. 

But  there  have  been  occasions  where  rummaging  around  in  that  drawer  didn't  get  you  the  desired  tools.  You  want  to  hang  a picture  on  the  wall 
but  you  can't  find  a nail,  only  some  screws.  You  realize  that  a screw  would  work  for  hanging  the  picture,  only  to  find  you  don't  have  a 
screwdriver.  You  rationalize  that  the  picture  isn't  all  that  heavy  so  you  could  just  pound  the  screw  into  the  wall  and  take  care  of  it,  but  there  isn't  a 
hammer  to  be  found  either,  only  a wrench.  This  tool,  clearly  not  designed  for  the  task,  is  pressed  into  service  and  with  a little  effort,  hopefully 
without  broken  fingers,  you  indeed  hang  the  picture.  You  may  not  have  used  the  right  tool  for  the  job  but  the  job  was  done.  Because  the  picture 
now  hangs  over  and  covers  the  screw  you  are  also  able  to  hide  this  odd  choice  of  a mounting  device  from  visitors  who  would  admire  the  picture. 

This  short  paper  is  meant  as  a starting  point  for  examining  educational  effectiveness  of  technology  applications  by  matching  format,  function,  and 
context.  In  far  too  many  cases  today  the  wrong  tools  are  being  used  in  the  wrong  context,  supporting  the  educational  process  in  ways  that  are  left 
unexamined  by  educational  researchers  and  instructional  designers.  We  arc  getting  the  job  done  with  whatever  we  have  available  and  no  one  is 
looking  at  the  inventory  of  tools  in  the  drawer.  The  efficiency  and  effectiveness  of  technology  in  educational  settings  cannot  be  measured  solely 
through  the  final  output  of  an  educational  process.  The  latest  studies  of  the  use  of  computers  for  educational  purposes,  now  focusing  on  the  role 
of  the  Internet,  have  provided  a confusing  mix  of  contradictory  results.  Most  of  the  studies  are  hopeful  in  tone  but  unclear  about  the  benefits,  or 
even  the  overall  value,  of  using  the  Internet  as  a medium  of  education.  It  can  be  used,  so  it  is  being  used. 

Educators  have  a long  history  of  adopting  and  adapting  to  new  communication  technologies  as  they  enter  society—  sometimes  with  great  effect. 
Other  communication  technologies  have  been  misused  in  specific  applications  by  teachers  and  learners  and  abandoned  for  all  other  uses.  The 
awareness  that  a communication  tool  can  be  used  in  an  educational  setting  is  related  to  the  need  that  could  potentially  be  satisfied  by  the  new 
technology.  If  the  tool  does  not  not  immediately  satisfy  the  need,  it  runs  the  risk  of  being  abandoned.  If  the  tool  can  satisfy  an  educational  need 
even  through  a use  that  was  unintended,  even  inefficiently,  it  will  be  adapted  to  suit  the  purpose.  The  effectiveness  of  this  use  can  be  improved 
through  an  understanding  of  the  educational  process,  its  context,  and  the  effects  of  communication  tools  on  teaching  and  learning. 


Communication  formats  and  settings 


Linking  communication  environments  with  educational  settings  is  a valuable  starting  point  for  comparing  the  operational  effectiveness  of  a new 
technology  tool  in  context,  the  following  six  educational/communication  environments  are  based  on  mass  communication  theories  that  grew  from 
research  efforts  through  the  1940s  and  50s  . 


Figure  /.  Formats  & Settings 


Figure  /.  If  the  effectiveness  of  educational  technology  is  to  be  measured  it  must  take  place,  necessarily,  at  one  of  these  levels,  and 
perhaps  be  used  as  a tool  at  several  levels.  The  understanding  that  different  types  of  communication  contexts  offer  differing  levels  of 
interaction  with  educational  material,  and  thus  provide  a variable  basis  for  learning,  is  essential  for  measuring  the  effectiveness  of  the 
application  of  technology  tools  at  each  level  of  interaction. 


Figure  2. 
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Figure  2.  The  quality  of  educational  interaction  breaks  down  when  at  lower  levels  of  activity,  and  cannot  be  applied  at  all  at  higher 
levels  of  interactivity.  Interpersonal  communication  cannot  take  place  over  mass  broadcast  technology,  but  lectures  can  be  misapplied 
to  interpersonal  tutorials.  Media  technology  is  most  effective  for  education  when  context  and  type  match  interactivity. 

Media  types  and  technology  tools 

The  historical  development  of  multimedia,  the  combination  of  moving  images,  sound,  and  graphic  information,  begins  with  the  development  of 
motion  pictures  and  extends  to  today's  subscription  television  cable  systems.  The  impact  of  these  multiscnsory  media  on  viewers  in  the  immersive 
environment  of  a darkened  theater  or  near  the  electronic  hearth  of  individual  homes  is  complex  and  impressive.  Multimedia  technology  has 
become  more  sophisticated  and  multilayered,  allowing  for  multiple  languages,  commentary,  text,  and  audio  input  and  feedback.  This  is  all  the 
more  remarkable  in  view  of  the  almost  total  lack  of  feedback  in  early  silent  movie  production,  mass  broadcast  television  monopolies,  and  linear 
videotape  media. 

Coalcscent  media  is  the  result  of  both  technology  and  media  convergence.  It  represents  the  development  of  a new,  digital,  component-based 
medium.  Coalcscent  media  have  emerged  from  the  development  of  computational  and  data  management  tools,  publishing  software,  digital  audio 
and  media,  hypertext,  and  hypermedia.  The  linkage  of  different  components  occurs  in  contexts  provided  by  the  user  in  ways  determined  through 
navigational  choices  made  on  the  spot  by  the  user.  Today  web  pages  can  be  built  by  relational  databases  on  demand.  Users  select  options  and  the 
database  engine  places  the  requested  information  in  an  HTML  document  using  preselected  parameters.  The  limitations  of  current  database 
technology  combine  with  bandwidth  restrictions  to  limit  the  potential  of  this  early  technology  to  combine  and  recombine  data,  in  numerous 
forms,  based  on  searches  of  worldwide  resources.  Coalcscent  media  will  be  first  relational,  drawn  together  through  user  demand  and  user  selected 
interaction  parameters.  Later  developments  may  find  hypertextual  hypermedia  coalescing  into  forms  of  communication  that  extends  the 
capabilities  of  text,  audio,  or  info-graphics  by  extending  the  interactions  users  have  with  new  combinations  of  that  media.  This  hyperlinking  of 
other  media  types,  in  constantly  shifting  referential  and  contextual  frames,  is  creating  a new  type  of  medium  that  demonstrates  the  characteristics 
of  each  level  of  interaction  and  uses  each  previous  type  of  medium.  As  technology  escapes  the  limitations  of  keyboards  and  screens  we  will  see 
new  types  of  interactions  take  place  in  contexts  wholly  created  and  invented  by  the  user,  not  producer,  of  the  content.  New  technologies  being 
developed  arc  allowing  different  types  of  interactions  with  content  in  contexts  not  imagined  by  the  author.  Wc  are  only  beginning  to  see  the  curve 
rise  in  the  growth  of  computational  efficiency  and  the  rise  of  new  computational  technologies. 

Educational  institutions  are  currently  setting  learning  requirements,  and  designing  the  curricula  to  reach  them,  through  interactions  with 
government,  professional  and  industry  interests.  These  may  be  anything  from  international  professional  societies  to  locally  elected  school  boards. 
What  will  education  look  like  with  information  age  learning  requirements  and  Icamer-de fined  knowledge  building?  If  the  individualization  of 
economic  activity  continues,  what  role  will  educational  institutions  play  in  defining  or  providing  education?  Does  the  personalization  of 
education  erode  or  enhance  the  role  of  educational  institutions? 

Conclusion 

Driven  by  a need  for  improved  commercial  skills  and  abilities  learners  are  rejecting  the  constraints  of  semesters  and  grades  for  the  realities  of 
deadlines  and  competencies.  The  growth  of  asynchronous  distant  learning  systems  has  been  dramatic,  reflecting  both  the  new  availability  of 
learning  through  new  technologies,  and  the  growing  demand  for  training  and  education.  Knowledge  workers  need  to  respond  to  demands  for  new, 
more  valuable,  knowledge  on  an  accelerating  time  scale.  Traditional  educational  institutions  entering  the  distance  education  marketplace  are  now 
faced  with  a similar  dilemma;  the  multifaceted  role  of  information  provider,  curricular  coordinator,  skill  assessor,  and  content  producer. 
Institutions  are  now  faced  with  competition  not  from  other  institutions  or  organizations,  but  from  individuals  operating  outside  of  traditional 
knowledge  production  frameworks.  Without  the  capital  investment  needed  to  create  a scriptorium,  library,  or  research  lab  entrepreneurs  can 
respond  to  the  demands  of  a new  economy  in  ways  that  slow  moving  institutions  cannot.  Research  into  the  improvement  of  learning  using  new 
media  technologies,  in  different  contexts,  is  not  just  important.  It  is  vital  to  the  improvement  of  educational  systems  that  take  advantage  of  each 
medium. 
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Introduction 

A poorly  planned  and  developed  distance  education  program  will  lead  to  negative  learning  outcome.  For  example,  when  a Texas 
college  began  to  offer  on-line  courses  taught  by  instructors  without  the  skills  and  experience  to  teach  on  the  Internet,  only  about 
40%  of  students  who  enrolled  in  the  on-line  courses  completed  them  (Simpson,  McCann,  & Head,  1999).  From  the  same  logic,  a 
poorly  designed  Web  course  interfece  will  have  a negative  impact  on  the  entire  distance  education  program. 

This  paper  discusses  the  roles  of  instructional  design  and  interface  design  in  the  development  process  of  a Web  course.  Although 
the  principles  of  both  instructional  design  and  interface  design  emphasize  the  importance  of  formative  evaluation,  formative 
evaluation  (which  should  be  conducted  to  assess  the  quality  of  the  Web  course  instruction  and  interface)  is  often  overlooked  or 
ignored  in  this  process.  Formative  evaluation  involves  (1)  collecting  a third  party  content  expert’s  feedback  to  the  instruction  and 
target  learners’  experiences  with  the  instruction  and  interface  and  (2)  taking  these  evaluation  results  into  consideration  when 
revising  the  Web  course.  The  purpose  of  conducting  such  formative  evaluation  is  to  further  improve  the  Web  course  instruction 
and  interface  to  maximize  learner  comfort  and  thus  their  learning  outcome. 

Instructional  Design  and  Interface  Design 

In  the  ideal  context  of  Web-based  instruction  development,  instructional  design  is  a process  in  which  a content  expert  (the 
instructor),  a distance  learning  instructional  designer,  and  a course  production  team  (that  composes  of  an  editor,  a Web 
programmer,  and  a multimedia  developer)  work  together  developing  a pedagogical ly  sound  Web  course.  According  to  the  Dick 
and  Carey  systems  approach  model  (Dick  & Carey,  1996),  the  instructional  design  process  involves  determination  of  the 
instructional  goal,  analysis  of  the  instructional  goal,  learner  and  context  analysis,  development  of  performance  objectives, 
development  of  assessment  instruments,  development  of  instructional  strategy,  development  and  selection  of  instruction, 
formative  evaluation,  revision  of  instruction,  and  summative  evaluation. 

In  this  instructional  design  model,  if  the  instructional  strategy  has  been  determined  to  be  Web  delivery,  then  the  next  stage- 
development  and  selection  of  instruction— will  focus  on  the  instructional  content  level  of  the  Web  course,  such  as  choosing  the 
appropriate  instructional  materials  for  the  intended  learners,  modifying  the  existing  format,  structure  and  sequence  of  these 
materials  to  which  learners  are  presented,  and  adding  links  to  Web-based  instructional  resources. 

The  literature  indicates  that  an  important  factor  to  the  success  of  Web-based  instruction  is  the  integration  of  interface  usability 
design  into  the  development  process  (Henke,  1997).  A Web  course  interface  design  defines  the  surface  level  of  the  Web  course 
(that  is,  the  visible  structure  that  helps  learners  accomplish  the  goal  of  content  learning).  Web  courses  produced  by  using  HTML 
coding,  Web  Course  in  a Box,  WebCT,  or  other  tools  appear  differently,  due  to  their  unique  interfaces.  An  interface  includes  the 
following  features: 

(1)  what  students  will  see  and  hear  on  a web  page  within  the  Web  browser  (e.g.  hyperlinks,  text,  images,  animations,  audio  and 
video  clips,  streaming  audio  and  video  media,  frames),  and 

(2)  any  responses  (e.g.  feedback,  change  of  web  page,  error  message)  to  students'  actions  such  as  clicking  on  a hyperlink  on  a 
Web  page  or  the  "Back"  button  in  the  Web  browser. 

Regardless  which  Web  course  production  tool  one  uses  (HTML  coding  or  Web  course  delivery  software  like  Courseinfo,  Lotus 
LeamingSpace,  Web  Course  in  a Box,  or  WebCT),  it  is  crucial  that  a learner-centered  Web  course  interface  should  address  the 
goals  of  user- interface  design:  time  to  learn,  speed  of  performance,  rate  of  errors  by  users,  retention  over  time,  and  subjective 
satisfection  (Shneiderman,  1998).  As  mentioned  earlier,  Wedman  is  concerned  of  students'  "subjective  satisfaction"  of  his 
School’s  on-line  master's  program. 

Formative  Evaluation 

In  the  Web  course  development  process,  an  in-house  review  of  a Web  course  that  can  be  called  an  alpha  testing  (in  software 
engineering,  the  first  test  that  is  often  performed  only  by  users  within  the  organization  developing  the  software)  consists  of 
instructional  designer's  initial  review  and  feedback  to  faculty,  faculty's  revision,  editor's  editing,  instructional  designer's  review 
and  feedback  to  editor,  editor's  re-editing,  programmer's  HTML  coding,  instructional  designer's  review  and  feedback  to 
programmer,  programmer's  revision,  faculty's  final  review,  programmer's  final  revision.  What  is  usually  missing  is  the  beta 
testing  which  requires  a group  of  learners  who  are  representatives  of  the  target  learners  due  to  practical  difficulty.  Beta  tests, 
usability  tests,  or  formative  evaluations  (in  the  instructional  design  process)  have  one  thing  similar— the  evaluation  of  the  use  of 
instruction  and  interface  by  actual  learners  or  users. 


A formative  evaluation  can  be  conducted  to  assess  the  quality  of  the  Web  course  interface  that  includes  the  Web  pages  of  general 
course  information,  and  the  first  two  lessons  (if  time  permits,  the  entire  course).  The  following  is  an  example  of  an  ideal 
formative  evaluation  of  a Web  course  based  on  Dick  and  Carey  systems  approach  model  of  instructional  design  (Dick  & Carey, 
1996):  (1)  subject  matter  expert  evaluation,  (2)  one-to-one  evaluation,  (3)  small-group  evaluation,  and  (4)  field  trial. 

Data  Collection  and  Analysis 

Technology  plays  a number  of  roles  in  the  data  collection  and  analysis  processes  during  the  formative  evaluation.  First  of  all,  for 
the  subject  matter  expert  evaluation,  the  subject  matter  expert  may  live  in  a remote  town  away  from  the  institution,  so  frequent 
travel  to  the  institution  will  not  be  a feasible  practice.  As  long  as  the  subject  matter  expert  has  access  to  the  Internet,  he  or  she 
can  review  and  evaluate  the  course  content  on  the  web,  and  make  comments,  corrections,  and/or  questions  in  an  e-mail  message 
sent  to  the  instructor  who  developed  the  course.  Through  e-mail  exchange  between  the  subject  matter  expert  and  the  instructor 
when  discussing  possible  revisions  in  the  course,  the  instructional  designer  will  eventually  be  notified  on  actual  changes  needed 
for  the  course  based  on  these  two  parties  consensus. 

Second,  for  the  one-to-one  evaluation,  the  technologies  used  are  similar  to  an  interface  usability  test.  The  entire  evaluation  will 
be  taped  by  using  a video  camera,  and  the  video  tape  will  be  reviewed  and  studied  to  observe  the  learner's  facial  expression, 
emotion  changes,  body  language,  and  spoken  language  (while  the  think  aloud  protocol  will  be  used)  which  will  be  also 
transcribed  into  words  for  further  qualitative  text  analysis.  The  text  analysis  will  make  use  of  a software  called  HyperQual  or  a 
similar  text  coding/analysis  software. 

Third,  for  the  one-to-one  evaluation,  small-group  evaluation,  and  field  Trial,  a web-based  testing  software  (which  can  be  used 
for  survey  purpose)  like  QuestionMark  Perception  will  be  used  to  collect  both  quantitative  data  and  qualitative  data.  If  a 5-point 
Likert  scale  of  attitude  questionnaire  is  used,  the  questions  can  be  entered  in  Perception.  After  learners  complete  the 
questionnaire.  Perception  is  capable  of  conducting  basic  statistical  analyses  of  the  quantitative  data  collected  such  as  mean 
scores,  medium  scores,  standard  deviations,  and  frequencies.  Such  data  can  be  also  transferred  to  a statistical  software  (e.g.  SPSS 
for  windows)  for  further  statistical  analyses  or  a database  software  (e.g.  Microsoft  Access)  for  data  storage  and  possible  future 
retrieval  for  further  analyses.  Likewise,  if  the  lessons  contain  some  form  of  objective  posttests.  Perception  can  be  used  to 
administer  these  tests,  and  thus  results  of  student  performance  can  be  obtained  by  looking  at  the  test/measurement  analyses 
Perception  conducted  and  by  further  analyzing  the  data  in  other  software  packages  mentioned  earlier. 

Perception  can  also  allow  learners  to  enter  open-ended  information  when  answering  attitude  questions  or  answering  subjective 
lesson  assignment  questions.  The  answers  will  be  either  kept  on  the  Perception  database  on  the  web  or  sent  to  the  instructor  or 
the  instructional  designer  for  grading  or  evaluation.  Again,  for  responses  to  attitudinal  questions,  the  responses  can  be  sorted  or 
coded  in  a text  analysis  that  will  make  use  of  a software  called  HyperQual  or  a similar  text  coding/analysis  software. 

Fourth,  for  the  one-to-one  evaluation,  small-group  evaluation,  and  field  Trial,  the  instructional  designer  can  use  e-mail  or  phone 
to  follow  up  with  the  learners  to  check  on  the  accuracy  of  the  designer's  interpretation  of  their  comments,  suggestions,  criticisms, 
and/or  questions. 

Conclusions 

In  order  for  the  Web-based  instruction  design  department  to  process  the  same  number  of  courses  with  the  inclusion  of  formative 
evaluation,  more  staff  members  will  need  to  be  employed,  and  the  Web-based  instruction  design  department  will  need  to  identify 
sources  that  can  allocate  such  money  for  additional  staff  recruitment.  If  it  is  impossible  to  secure  adequate  funding  for  more 
staff,  a compromised  formative  evaluation  will  have  to  be  tailored  (that  is,  to  cut  portions  of  the  evaluation  as  outlined  above)  to 
fit  the  existing  monetary  and  human  resource  structure. 
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Introduction 

As  noted  by  Speziale  and  Kachurick  (1996),  "advances  in  technology  have  provided  numerous  opportunities  for  educational 
integration;  however,  inadequate  professional  development  programs  have  seriously  limited  teachers’  knowledge  of  techniques 
for  integrating  instructional  technologies."  In  an  effort  to  solve  this  problem,  instructional  technology  professional  development 
programs  have  been  founded  in  many  major  universities  and  large  community  colleges  in  the  United  States  (Stoloff,  1999). 

Recognizing  the  emerging  needs  for  distance  learning  and  the  iniegralioii  of  liistractional  technolog’y,  Southwestern  Illinois 
College  has  launched  a professional  development  program  in  instructional  design  and  technology,  an  innovative  service  to  its 
faculty,  in  the  summer  of  1999.  The  new  program  is  operated  by  the  Instructional  Technology  and  Distance  Learning  department 
under  the  Learning  Resources  Division,  with  a mission  to  assist  members  in  planning,  developing,  implementing,  and  evaluating 
their  instruction  and  to  facilitate  innovative  integration  of  instructional  technology  into  their  on-campus  and  on-line  courses.  This 
institution-specific  case  study  describes  the  implementation  process  of  this  new  program  and  reports  its  recent  performance 
assessment. 


Program  Services  and  Facilities 

The  professional  development  program  is  led  by  the  instructional  technologist/design  specialist  with  assistance  from  and 
collaboration  with  other  staff  members  (including  the  associate  director  of  distance  learning,  the  Internet  systems  manager,  a 
Web-based  instruction  coordinator,  a Web  programmer,  an  Internet  support  technician,  and  a network  technician)  in  the 
Instructional  Technology  and  Distance  Learning  department. 

A.  Facilities:  The  long-term  goal  of  the  program  is  to  implement  a faculty  development  facility  called  the  “Instructional 
Design  & Technology  Center”  that  consists  of  a computer  lab  for  training  purpose  and  a multimedia  work  area  for  production 
purpose.  To  get  things  started,  a computer  workstation  with  a Pentium  II  dual  processor  has  been  built  and  is  available  for 
faculty  use.  Additional  hardware  includes:  a video  capture  card,  a flatbed  scanner,  a CD  writer,  a ZIP  drive,  a JAZ  drive,  two 
digital  cameras,  a color  printer,  and  a laser  printer.  Software  titles  installed  are  Microsoft  Windows  NT  4.0  Workstation, 
Microsoft  Office,  Microsoft  FrontPage,  Adobe  PhotoShop,  OCR  Software,  and  other  multimedia  applications.  In  addition,  a 
Macintosh  G3  computer  has  been  recently  included  in  the  Center  with  some  general  purpose  software  such  as  Microsoft  Office. 

B.  Consultation:  The  program  offers  one-to-one  consultation  to  faculty  with  tailored  information  about  theory  and  practice 
in  technology  applications  for  instruction.  A customized  small  group  session  can  be  arranged  when  there  are  two  or  more  faculty 
members  interested  in  a certain  design  and  technology  issue. 

C.  Training:  Hands-on  instructional  technology  workshops  on  how  to  use  various  instructional  software  are  offered  to 
faculty,  in  addition  to  pedagogical  seminars  for  faculty  to  facilitate  information  sharing  and  discussion  of  current  issues  on 
design,  development,  implementation,  and  evaluation  of  instructional  technology  applications. 

Program  Performance 

A.  Instructional  Design  Consultation:  To  increase  the  visibility  of  the  innovation-the  new  professional  development 
program,  a small  but  concise  Web  site  was  built  in  the  summer  of  1999  to  publicize  the  services  and  facilities  being  offered  to  the 
faculty.  As  words  about  the  innovation  regarding  the  professional  development  program  spread  out,  the  number  of  faculty  who 
came  to  the  program  for  consultation  on  instructional  design  and  technology  has  increased: 

June  1999:  1 faculty  on  1 occasion 
August  1999:  2 faculty  on  2 occasions 
September  1 999:  2 faculty  on  2 occasions 
October  1999:  7 faculty  on  14  occasions 
November  1999:  4 faculty  on  9 occasions 
December  1999:  1 faculty  on  2 occasions 
January  2000:  2 faculty  on  3 occasions 

The  average  length  of  a consultation  meeting  is  about  two  hours.  It  shows  faculty's  enthusiasm  and  willingness  to  devote 
their  time  commitment.  Issues  faculty  members  have  brought  to  the  consultation  meetings  are  diverse.  The  following  list 
demonstrates  the  variety  of  such  topics: 

- Developing  web  pages  for  a campus  course  in  FrontPage 

- Working  with  WebCT  for  an  online  course 

- Discussing  design  issues  on  Social  Science  departmental  website 

- Discussing  and  planning  a Faculty  Web  Directory 

- Designing  a welcome  template  for  an  online  course 

- Designing  a telecourse  support  Web  site  in  Frontpage 

- Working  with  PowerPoint  to  develop  a student  orientation  session 
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B.  Training  to  Meet  Faculty  Needs:  In  order  to  understand  faculty  members'  needs  in  instructional  design  and  technology 
and  to  prioritize  offerings  of  workshops  and  seminars,  a needs  assessment  was  conducted  in  the  form  of  a brief  survey  (Dick  & 
Carey,  1996)  to  130  full-time  faculty  members  in  August,  1999.  The  survey  included  questions  asking  faculty  to  rate  (on  a 1-3 
scale,  with  3 being  highly  interested,  2 being  interested,  and  3 being  not  interested)  their  interest  in  a range  of  workshop  and 
seminar  topics  derived  from  the  text  by  Newby,  Stepich,  Lehman,  and  Russell  (2000).  68  faculty  members  returned  the  survey. 
The  response  rate  of  the  survey  is,  therefore,  52.3%.  The  top  rated  topics  are  listed  below  in  the  descending  order  of  the  mean  of 


Rating 

Topic 

2.27 

Using  presentation  software  in  the  classroom 

2.24 

Creating  instructional  web  pages 

2.1 

Integrating  instructional  technology  in  the  classroom 

1.96 

Using  Internet  communication  tools 

1.87 

1.83 

Lessons  learned  from  teaching  an  on-line  course 

1.83 

Creating  an  electronic  gradebook 

1.69 

Fair  use  of  copyrighted  materials  in  on-line  instruction 

1.36 

Introduction  to  distance  education  technologies  . 

Meeting  the  faculty  needs,  the  program  developed  and  delivered  four  hands-on  workshops  and  two  pedagogical  seminars  till 
February  2000.  Workshop  training  materials,  with  step-by-step  instructions  and  screenshot  illustration  on  how  to  use  the 
software  applications,  were  produced  by  the  workshop  trainer  (who  is  the  instructional  technologist^design  specialist)  and  pilot 
tested  by  other  staff  in  the  department.  With  the  intention  to  reduce  faculty's  computer  anxiety  level  and  not  to  overwhelm 
faculty  by  teaching  them  every  single  feature  of  the  software  applications,  the  workshops  and  training  materials  were  tailored  to 
invite  faculty  members  to  practice  some  the  most  commonly  used  functions  to  get  started.  During  the  hands-on  workshops, 
pedagogical  issues  such  as  design  principles  were  discussed  before  the  hands-on  practice  sessions. 

The  following  table  illustrates  a brief  evaluation  of  the  training  delivered  from  September  1999  till  February  2000.  The 
‘‘No.”  column  is  the  number  of  faculty  who  participated;  the  “Overall”  column  is  the  participants'  average  rating  of  the  overall 
evaluation  of  the  workshop  based  on  a 1-5  scale  (with  5 being  the  highest  rating);  the  “Usefulness”  column  is  the  mean  of  their 


Title  of  Training 

No. 

Overall 

Usefulness 

Workshop  - Creating  Instructional  Web  Pages  with  Microsoft  FrontPage  98:  An  Introduction 

30 

4.77 

4.73 

Workshop  - Designing  Classroom  Presentations:  Basics  of  Microsoft  PowerPoint 

23 

4.87 

4.96 

Workshop  - Designing  Classroom  Presentations:  Advanced  Microsoft  PowerPoint 

13 

4.85 

4.77 

Seminar  - Integration  of  Instructional  Technology  into  Teaching  & Learning:  An  Overview 

15 

4.13 

4.40 

Seminar  - Interaction,  Instruction,  and  Research  Via  the  Internet 

12 

5.00 

4.91 

Workshop  - Image  Digitizing  and  Editing 

26 

4.92 

4.77 

More  workshops  and  seminars  are  under  development  and  will  be  soon  offered  to  faculty  members  in  the  future  so  as  to 
address  one  of  the  commonly  complained  drawbacks  of  a professional  development  program  according  to  national  findings 
regarding  to  professional  development:  infrequency  of  training  (Speziale  & Kachurick,  1996). 

Summary 

Instructional  technology  are  great  tools  that  offer  possibilities  for  improving  the  learning  process,  but  such  possibilities  will 
not  be  realized  automatically  unless  the  teacher  will  have  to  integrate  these  tools  into  the  curriculum  in  a way  to  facilitate 
teaching  and  learning  (Heinich,  Molenda,  Russell,  & Smaldino,  1999).  The  professional  development  program  in  instructional 
design  and  technology  fills  this  gap  between  the  tools  and  the  people  by  helping  faculty  use  the  tools  to  teach  in  a more  effective 
and  efficient  manner. 
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Abstract:  Many  recent  web-based  educational  systems  couldn’t  provide  individualized  instruction 
or  interactive  problem  solving,  since  they  are  mostly  built  upon  static  hypertext.  One  possible 
solution  for  this  problem  could  be  adapting  existing  proven  techniques  from  the  stand-alone 
Intelligent  Tutoring  Systems(ITS)  into  the  web-based  educational  systems.  Recent  web-based  ITS 
research  shows  this  direction  by  employing  the  existing  ITS  techniques  minimally.  And  this  needs 
to  be  studied  fiirther  to  support  more  adaptive  instruction.  In  this  paper  we  describe  the 
development  of  a Web  based  Adaptive  programming  Language  Tutoring  System(WALTS)  which 
is  designed  by  ITS  structure  primarily,  and  adapting  many  successful  ITS  techniques  into  the 
system.  In  addition,  the  system  adapted  CORBA  infrastructure  to  support  the  user  more  consistent 
and  reliable  performance.  Together,  the  system  behaves  more  adaptive  and  interactive  than  the 
existing  static  web-based  educational  systems. 


Introduction 

Many  recent  web-based  educational  systems  could  not  provide  an  individualized  instruction  or  interactive 
problem  solving,  since  they  are  mostly  built  upon  static  hypertext.  One  possible  solution  might  be  adapting  the 
existing  techniques  from  the  stand-alone  Intelligent  Tutoring  System(ITS).  However,  most  of  the  recent  web-based 
ITS  research  show  the  efforts  by  employing  the  techniques  selectively(Kay  & Kummerfeld,  1994;  Brusilovsky  et  al. 
1996;  Nakabayashi  et  al.  1997).  And  this  needs  to  be  studied  further  to  enhance  the  overall  capabilities  of  the  system 
at  the  previous  stand-alone  ITS  level.  In  this  paper  we  will  describe  development  of  Web-based  Adaptive 
programming  Language  Tutoring  System(WALTS).  The  main  goal  of  the  development  of  the  system  is  to  provide 
the  first  year  computer  science  student  a comprehensive  instruction  for  learning  C programming  language,  by 
porting  many  successful  ITS  techniques  into  the  system.  In  addition,  the  system  is  designed  to  support  the  user  more 
consistent  and  reliable  performance  by  employing  CORBA(Common  Object  Request  Broker  Architecture)  structure. 
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Figurel:  The  main  Architecture  of  WALTS 


Basic  architecture  of  the  system 


The  basic  architecture  of  WALTS  is  designed  by  typical  ITS  structure  primarily,  including  expert  module,  the 
student  modeler  module,  and  the  instructional  planning  module. 

The  expert  module  consists  of  the  object-oriented  knowledge  base  and  the  problem  solver.  Since  the  domain 
of  the  system  is  the  C programming  language  concepts,  which  may  not  require  any  complex  causal  relationships  and 
only  requires  mostly  simple  concepts,  we  used  the  frame  knowledge  representation.  The  object-oriented  approach  of 
designing  the  knowledge  base  make  it  easy  to  understand  and  can  provide  more  flexibility  for  updating  or 
manipulating  tutoring  strategy  in  the  construction  of  ITS.  The  problem  solver  creates  problem  table  by  referencing 
the  problem  template  from  the  current  lesson  unit 's  content  table,  which  is  the  form  of  hashtable.  The  solver  stores 
the  correct  answer,  and  then  it  presents  the  generated  questions  to  the  user  in  appropriate  HTML  form  through  the 
KTivIL  geiiei’ator.  This  ivicthod  can  provide  differeriL  styles  of  questions  for  different  users  even  though  they  arc 
accessing  the  same  lesson  unit.  Since  the  column  name  of  the  table  is  object’s  name,  the  planner  can  reply  to  the 
user’s  request,  such  as  hint  or  help,  by  referencing  this  table. 

The  method  of  building  the  student  modeler  is  the  simple  overlay.  The  modeler  keeps  all  the  necessary 
administrative  information  on  the  server,  such  as  initial  student’s  ID,  password,  e-mail  address,  the  access  time.  And 
the  information  regarding  the  student’s  learning  process  is  also  stored  in  the  student  model.  And  each  parameters  has 
unique  meanings;  for  example,  the  HelpCount,  and  HintCount  can  be  updated  only  when  the  unitLesson  is  quiz, 
increase  of  ReferenceCount  means  the  user  is  weak  at  the  current  unitLesson,  the  lessonLevel  stores  information 
about  how  many  times  the  user  visited  the  current  unitLesson,  and  so  on. 


The  Instructional  Planning 

The  existing  web  based  educational  systems  mostly  employ  the  hypertext  techniques,  which  is  hard  to  make 
hyperlink  in  every  HTML  pages,  such  that  it  may  be  inappropriate  for  adaptive  learning  environment.  Therefore  we 
need  to  devise  a method  that  can  generate  adaptive  lesson  unit  dynamically.  The  instructional  planning  of  the 
WALTS  can  be  further  divided  into  3 steps,  a curriculum  planning,  a lesson  planning,  and  a delivery  planning.  The 
curriculum  planning  of  WALTS  generates  a curriculum  in  tree  structure;  the  curriculum  planner  extracts  information 
from  the  knowledge  base  and  creates  a curriculum  hierarchically  in  the  order  of  prerequisites.  Then  the  lesson 
planning  sets  up  the  lesson  sequence  within  a single  lesson  unit.  The  role  of  delivery  planning  is  limited  to 
presenting  the  selected  lesson  content  to  the  user. 

Conclusion 

The  main  goal  of  this  paper  is  the  adaptation  of  the  existing  ITS  techniques  to  the  web  platform.  Therefore, 
we  have  designed  and  implemented  the  system  based  on  the  major  ITS  architecture,  and  this  brings  us  several 
advantages  over  traditional  HTML-based  educational  systems.  First,  the  main  knowledge  base  is  created  as  an 
object-oriented  concept,  which  can  provide  more  flexibility  for  updating  or  manipulating  tutoring  strategy.  Second, 
the  system  can  generate  a problem  dynamically  by  the  problem  solver  and  also  can  solve  the  problem  intelligently. 
Third,  the  instructional  planning  mechanism  can  generate  an  instructional  plan  dynamically.  The  secondary  goal  is 
the  designing  of  the  system  as  a distributed  infrastructure  using  CORE  A as  backbone.  This  structure  solves  the 
bottleneck  problem  of  previous  CGI  dependent  systems,  and  also  gives  some  benefits  of  better  performance  and  also 
gives  flexibility  in  the  case  of  further  enhancement  of  the  system. 
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Abstract:  Online  learning  communities  are  an  excellent  means  of  integrating  collaboration  in  online  learning 
environments.  After  a brief  introduction  we  [the  authors]  define  collaboration  and  learning  communities  and  address 
their  theoretical  foundations.  We  then  discuss  the  benefits  of  collaboration  and  learning  communities  in  online  learning 
environments.  Finally,  we  provide  a case  study  of  a new  online  learning  community  at  Western  Governors  University. 

Deflnitions  and  Theoretical  Foundations 

Learning  communities  are  environments  that  encourage  mutual  exchange  between  the  community  members  to 
support  their  individual  and  collective  learning.  Learning  communities  are  founded  on  the  aspect  of  social 
negotiation  of  meaning.  Collaboration  is  the  key  tenet  of  constructivist  and  small-group  theories  (Springer  Springer, 
Stanne,  & Donovan,  1999;  Grabinger,  1995;  Duffy  & Jonassen,  1992)  that  explains  the  social  component  of  learning 
and  demonstrates  that  “conceptual  growth  comes  from  sharing  perspectives  and  modifying  our  internal 
representations  in  response  to  that  sharing’’  (Grabinger,  1995,  p.  669).  Online  learning  communities  provide  the 
collaborative  means  for  achieving  the  desired  “shared  creation”  and  “shared  understanding”  (Shrage,  1991,  p.  40). 

Benefits  of  Online  Learning  Communities 

Collaborative  learning  and  learning  communities  benefit  learners  because  they  encourage  shared  ways  of  knowing, 
promote  active  participation,  improve  achievement,  contribute  to  the  creation  of  knowledge,  and  challenge  learners’ 
cognitive  abilities  (Springer,  Stanne,  and  Donovan,  1999;  Moller,  1998;  Lave  & Wenger,  1991;  Slavin,  1990). 

Through  new  Internet  technology,  synchronous  and  asynchronous  online  collaboration  now  provides  many  of 
the  benefits  that  exist  in  traditional  face-to-face  collaborative  and  learning  community  environments.  Our  recent 
literature  review  produced  evidence  that  online  collaboration  and  online  learning  communities: 

• Allow  learners  and  instructors  to  work  together  to  create  knowledge  constructions  from  their  combined 
experience  (Jonassen,  1998;  Reeves,  1997) 

• Encourage  learners  to  evaluate  complex  issues  from  multiple  perspectives  and  change  their  perspective  based 
on  others’  input  (Sherry,  1998;  Stone,  1996;  Honebein,  Dufry,  and  Fishman,  1993) 

• Hold  learners  individually  accountable  while  striving  toward  group  goals  so  that  students  help  one  another  and 
assess  one  another’s  learning  (Springer,  Stanne,  and  Donovan,  1999) 

• Provide  opportunities  for  learners  to  reflect  on  their  learning  experiences  and  upon  others’  input  (Stone,  1 996) 

• Demonstrate  that  meaning  and  reality  is  socially  constructed  and  negotiated  through  the  exchange  of  ideas, 
information,  and  feelings  among  the  members  of  the  community  (Hiltz,1998) 

• Improve  student  achievement  through  increased  motivation,  peer  support  and  communication,  and  commitment 
to  participate  and  finish  (Hiltz,  1998;  Sherry,  1998;  Stone,  1996) 

The  above  evidence  demonstrates  that  online  collaboration  and  online  learning  communities  can  impact  and  enhance 
the  learning  process.  We  have  personally  witnessed  the  impact  of  a self-initiated  online  learning  community  at 
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Western  Governors  University.  Although  the  WGU  online  learning  community  is  relatively  new,  we  have  already 
seen  positive  effects  ripple  through  the  community  members. 

Case  Study  of  an  Online  Learning  Community  at  Western  Governors  University 

In  August  1999,  Western  Governors  University  (WGU)  began  enrolling  students  in  its  new  competency-based 
Master  of  Arts  in  Learning  and  Technology  program  (MLT).  A cohort  of  50  students,  primarily  K-12  teachers, 
geographically  dispersed  throughout  Utah  were  some  of  WGU’s  initial  MLT  students.  In  October  1999,  a small 
group  of  15  cohort  students  created  their  own  collaborative  online  learning  community  utilizing  a listserv  since  all 
students  had  ready  access  to  e-mail. 

The  WGU  online  learning  community  provides  a fascinating  case  study  because  it  was  initiated  by  the  students 
themselves  who  by  and  large  had  no  previous  experience  working  together.  Of  the  15  original  members  of  the 
community,  only  five  had  previous  experience  working  with  each  other  on  previous  projects;  the  majority  of  the 
learning  community  members  had  only  met  each  other  briefly  at  a WGU  reception  in  September  1999. 

As  we  watch  the  WGU  online  learning  community  develop,  we  are  paying  attention  to  the  ways  that  the 
community  members  negotiate  roles  and  establish  rules  and  standards  for  communication,  interaction,  and  entrance 
into  the  community.  We  are  eager  to  compare  learning  community  members’  performance  to  the  performance  of 
students  who  do  not  participate  in  the  online  learning  community.  We  are  also  interested  in  determining  why  some 
learning  community  members  remain  inactive  or  decide  to  leave  the  community. 

Conclusion 

Salomon  and  Perkins  (1998)  state  that  “learning  to  learn...  fundamentally  involves  learning  to  learn /row 
others,  learning  to  learn  with  others,  learning  to  draw  the  most  from  cultural  artifacts  besides  books,  learning  to 
mediate  others’  learning  not  only  for  their  sake  but  for  what  that  will  teach  oneself,  and  learning  to  contribute  to  the 
learning  of  a collective”  (p.  17).  Collaborative  online  learning  communities  provide  community  members  the 
chance  to  learn  from  and  with  others  and  to  contribute  to  others’  learning.  As  a result,  they  are  important  means  of 
integrating  the  social  aspect  of  learning  into  online  learning  environments.  Our  literature  review  and  personal 
research  of  the  WGU  online  learning  community  provide  valuable  insight  into  issues  surrounding  online 
collaboration  and  online  learning  communities’  development,  communication  patterns,  member  roles,  rules, 
flexibility,  sustainability,  and  reproducibility. 
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Abstract:  Encouraging  effective  collaborative  learning  supported  by  computer 
technology  requires  coordination  among  faculty,  administrators,  technical  and 
instructional  support  staff,  and  library  staff.  This  paper  describes  the  roles  of  each  of 
these  groups  in  the  design  and  use  of  computer  technology  to  support  collaborative 
learning.  This  paper  draws  on  collective  experience  from  implementing  Project  Vision 
and  Project  Empower,  two  computer-supported  collaborative  learning  programs  in 
Commonwealth  College,  Penn  State  University. 


Too  often  the  discussion  of  collaborative  learning  and  use  of  computer  technology  to  support  instruction 
focuses  solely  on  the  classroom;  in  fact,  successful  efforts  in  the  classroom  rest  on  a foundation  of 
collaboration  and  support  from  administrators,  technical  and  instructional  support  staff,  and  library  staff. 
Over  the  last  several  years  Penn  State  University  Delaware  County  has  developed  teaching  methods  that 
incorporate  the  best  features  of  distance  education  and  collaborative  learning  in  undergraduate  education. 
The  two  projects  most  directly  responsible  for  the  development  of  collaborative,  technology-enhanced 
instruction  were  Project  Vision  and  Project  Empower,  both  programs  for  freshmen  and  sophomores  in 
general  education  courses. 


Administrators 

Important  administrative  support  issues  include  motivating,  training,  and  supporting  faculty  to  take  on  new 
teaching  roles  and  new  responsibilities,  since  collaborative  teaching  and  use  of  computer  technologies 
differ  significantly  from  the  standard  lecture/discussion  format.  Administrators  must  make  sure  that  the 
reward  system  for  faculty  recognizes  the  effort  necessary  to  change  from  the  old  to  the  new  model;  we 
provided  release  time  and  training  for  faculty  to  learn  new  roles.  Administrators  are  also  responsible  for 
making  sure  that  appropriate  equipment  and  software  for  courses  is  installed  properly  and  works  efficiently. 
Persuading  faculty  to  work  cooperatively  with  librarians  and  with  technical  and  instructional  support 
personnel  is  also  an  administrative  role.  Administrators  may  also  be  responsible  for  the  selection  and 
orientation  of  technical  and  instructional  support  personnel,  who  must  understand  the  academic  mission 
and  the  instructional  process.  Finally,  administrators  must  be  well  versed  in  designing  and  conducting 
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collaborative  learning  in  technology-enhanced  classrooms,  because  they  are  responsible  for  the  evaluation 
of  the  quality  as  well  as  the  economic  viability  of  these  efforts. 


Faculty 

Graduate  training  seldom  includes  a component  on  teaching  collaborative,  technology-enhanced  courses. 
Hence,  faculty  must  learn  how  to  design,  manage,  and  evaluate  collaborative  assignments,  as  well  as  how 
to  choose  appropriate  technologies  to  enhance  learning  in  their  specific  venues.  Faculty  must  be  expert 
users  of  the  technology  in  order  to  explain  the  technology  to  inexperienced  students  who  are  struggling 
concurrently  with  course  content.  Faculty  always  balance  the  advantages  of  new  instructional  media  against 
many  constraints,  including  the  students’  abilities  and  interests,  classroom  facilities,  and  the  quality  of 
technical  support.  They  must  also  balance  the  time  they  spend  in  these  activities  against  other 
responsibilities  of  teaching,  service,  and  research.  Computer  technologies  are  constantly  changing; 
therefore,  faculty  who  are  committed  to  these  pedagogies  know  that  the  process  of  learning  new 
technologies  and  the  balancing  act  are  never-ending. 


Technical/Instructional  Support  Staff 

Technical  and  instructional  support  staff  work  with  faculty  from  a wide  variety  of  disciplines,  as  well  as 
with  administrators  and  students.  Math,  social  sciences,  humanities,  and  business  all  have  different  content 
and  different  pedagogies.  Technical  and  instructional  support  staff  must  understand  a wide  variety  of 
technologies  and  software  programs  appropriate  for  specific  disciplines  to  help  faculty  make  informed 
choices.  Support  staff  often  advise  faculty  in  designing  assignments  that  make  the  best  use  of  collaboration 
and  computer  technologies.  Technical  and  instructional  support  staff  must  understand  students’  abilities 
and  interests  in  order  to  recommend  the  most  appropriate  technologies  for  each  particular  instructional 
situation,  keeping  in  mind  budgetary  constraints.  They  train  faculty  and  students  and  trouble-shoot  daily 
problems  which  inevitably  arise.  Close  collaboration  with  faculty,  administrators,  and  students  requires 
superior  communication  skills.  A sense  of  humor  doesn’t  hurt  either. 

Library  Staff 

The  new  technology-enhanced  and  collaborative  classrooms  are  changing  the  roles  of  librarians  as  well  as 
faculty.  Library  staff  must  constantly  keep  up  to  date  with  emerging  technologies,  a not  inconsiderable 
task  with  the  rapid  development  of  electronic  media.  Librarians  are  responsible  for  designing  systems  for 
accessing  information  that  make  sense  to  users.  Increasingly,  librarians  find  themselves  acting  as  coaches 
and  collaborators  for  instructors  who  are  learning  to  teach  in  collaborative  and  technology-enhanced 
modes.  Library  staff  also  train,  coach,  and  support  students  in  learning  to  use  search  technologies  to  find 
information.  Library  staff  share  responsibility  with  faculty  for  teaching  critical  thinking  skills  as  they  teach 
students  to  evaluate  the  quality  and  usefulness  of  the  information  students  are  using. 


Conclusions 

Commonwealth  College  of  Penn  State  University  created  Project  Vision  to  encourage  faculty  to  redesign 
courses  to  incorporate  collaborative  and  active  learning  strategies  supported  by  appropriate  technology  in 
teaching  undergraduate  education  courses.  The  administration  offered  release  time,  extra  compensation, 
and  training  to  prepare  selected  faculty  for  the  pilot  program  which  was  designed  by  faculty.  Over  the  four 
years  of  this  program,  the  faculty  tried  several  models  for  offering  courses.  Once  all  the  students  took  only 
Project  Vision  courses  for  the  first  year  as  a cohort;  in  other  years,  students  took  some  Project  Vision 
courses  and  some  regular  courses.  For  two  years,  the  faculty  offered  courses  coordinated  across  two 
different  locations.  Usually,  the  courses  were  offered  as  a package.  For  example,  students  took  Speech 
Communication  to  learn  how  to  work  in  groups,  Library  Studies  to  learn  how  to  do  research,  and  Health 
Science  as  the  content  of  the  research  project.  The  technologies  included  FirstClass  computer 
conferencing,  Eudora  e-mail,  PowerPoint,  and  video  conferencing. 
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Introduction 

We  proposed  a method  using  digital  class  video  for  review  and  self-studying  and  applied  it  to  some 
subjects.  We  believe  our  method  is  successful  in  actual  education.  Students  will  make  their  own  notes 
watching  digital  class  video  in  the  same  way  they  make  their  own  notes  during  ordinary  classes.  If  students 
watch  digital  class  video  for  their  study  many  times  over,  their  notes  will  be  essential.  In  addition  to  digital 
class  video,  we  propose  synchronized  notes  for  digital  class  video. 

Digital  Class  Video 

In  many  educational  organizations,  networks  are  used  for  educational  support.  On  the  other  hand,  there  are 
many  traditional  classes  that  have  no  educational  support  using  computer  networks.  When  one  intends  to 
utilize  computer  networks  or  the  Internet  for  traditional  classes,  he/she  will  suffer  from  various  problems 
such  as  time  for  investigating  teaching  materials,  manpower,  arrangements  of  equipment  and  preparation  of 
presentation  data. 

To  solve  those  problems,  we  proposed  a method  using  digital  class  video  for  introducing  educational 
support  utilizing  computer  networks  for  traditional  classes.  Our  five  goals  of  the  proposed  method  are:  the 
purpose  of  the  method  is  review  and  self-studying;  the  method  could  preserve  the  various  teaching  style  of 
teachers;  the  method  should  be  simple  so  that  the  number  of  staffs  and  a budget  should  not  be  large;  the 
method  is  able  to  be  adopted  to  traditional  lectures  and  computer  exercises;  more  than  one  simultaneous 
classes  should  be  supported  using  the  method. 

To  achieve  above  goals, 

• The  scene  of  a class  is  recorded  on  video  tape  as  it  is. 

• The  camera  for  recording  teacher's  movements  has  fixed  view  and  never  follows  of  teacher's  movements. 

• A written  material  or  a computer  screen  of  the  teacher  operates  is  also  recorded  on  another  video  tape. 

These  two  videos  are  synthesized  to  a single  image  for  publication. 

As  experiments,  we  applied  our  proposed  method  to  five  subjects  in  1998.  The  experiments  are  still 
continued  in  1999.  The  number  of  total  access  times  to  the  video  was  1300  in  1998.  We  believe  that  our 
educational  assistance  method  is  successful. 

Synchronized  Notes 

To  study  subjects,  just  watching  class  is  not  enough  for  studying.  Students  ordinarily  make  notes  on  class 
with  a notebook  and  a pen.  (Nowadays,  some  students  use  their  own  lap-top  computer  for  their  notes  in 
classes.)  When  students  watch  class  video  on  the  computer,  it  is  preferable  for  students  to  make  notes  on 
the  computer.  To  make  notes  on  the  computer  while  watching  digital  class  video  is  simple  because  they  can 
easily  use  another  window  for  a text  editor  or  something  like  that.  In  that  way,  students  are  able  to  review 


their  lessons  again,  by  watching  video  and  their  notes.  As  Notes  is,  however,  "static"  information,  students 
should  synchronize  notes  and  video  by  themselves  during  watching  video. 


To  synchronize  notes  and  video,  we  implemented  a telop  function  to  our  MPEG  video  viewer  software. 
Users  are  able  to  put  comments  or  notes  to  video  as  a telop  just  when  they  are  needed.  Once  users  put 
comments  or  notes  to  video,  the  telop  come  up  on  the  screen  at  the  designated  time  whenever  they  watch 
video.  Fig.  I shows  an  example  of  telop  function. 


Figure  1,  An  example  of  synchronized  notes 


If  user  put  comments  to  video,  a mark  is  shown  on  the  scroll  bar  so  that  users  are  able  to  find  where  they 
put  their  comments.  Scroll  bar  of  the  MPEG  viewer  is  used  to  forwarding  and  rewinding  video. 

The  mark  on  the  scroll  bar  is  also  used  to  find  important  points  of  the  video.  In  our  method,  the  entire  class 
is  recorded  on  the  video.  Usually  teachers  use  a lot  of  time  for  thinking,  joking  or  something  like  that.  The 
mark  is  used  to  avoid  overhead  of  watching  unnecessary  parts  of  video. 

In  the  same  way,  not  only  users  (students)  but  also  teachers  are  able  to  put  comments  to  video.  After  the 
class,  teachers  sometime  notice  there  were  mistakes  or  insufficient  explanation  during  the  class.  In  such 
case,  teachers  are  able  to  add  additional  explanation  or  corrections  to  video  as  a telop. 


Conclusion 

We  implemented  synchronized  notes  to  MPEG  viewer.  Users  are  easily  able  to  add  comments  and  notes  to 
class  video  for  their  self-studying.  After  the  class,  teachers  are  also  able  to  add  additional  comments  or 
explanation  to  their  class  video.  The  softwares  are  implemented  as  WindowsNT  and  UNIX  (FreeBSD)  X- 
window  applications  so  that  students  are  able  to  study  on  any  plathome. 
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Abstract:  In  recent  years,  popularized  technologies  such  as  “Multimedia**  and  “Internet"  came  to  be  used  in 
various  fields.  The  field  of  education  is  not  an  exception  either,  and  these  technologies  are  adopted  by  CAL 
(Computer  Assisted  Learning).  Especially,  the  successful  introduction  of  these  technologies  into  Japanese 
education  is  being  looked  forward  to.  Since  Japanese  as  the  2nd  language  is  said  to  be  high  in  the  degree  of 
difficulty  in  acquisition,  an  improvement  in  the  study  efficiency  Lliiough  adopting  such  technologies  is  expected. 
This  paper  focuses  on  Kanji  of  Japanese  language  education  and  discusses  the  multimedia  Kanji  dictionary. 


Introduction  & Background 

At  present,  there  are  many  non-Japanese  who  are  learning  Japanese  in  the  world.  However,  with  a limited 
time  allocated  for  study  in  classes,  the  present  condition  does  not  offer  sufficient  education  for  mastering  Japanese,  For, 
not  only  are  there  problems  related  to  the  basic  grammar  such  as  the  word  order  in  Japanese  is  sometimes  different  and 
the  use  of  the  auxiliary  verb  is  more  complicated  in  comparison  with  other  languages  but  also  there  are  three  kinds  of 
characters  used:  "Hiragana,"  "Katakana"  and  "Kanji."  Especially,  quantity  in  Kanji  is  voluminous  and  the  shapes  are 
complicated.  Even  for  Japanese,  it  takes  6 years  to  learn  over  1000  Kanji  characters.  Furthermore,  so  much  information 
is  contained  in  Kanji  (for  example,  reading,  meaning,  radical  and  the  stroke  count,  etc.).  Therefore,  the  more  efficient 
Japanese  education  system  is  necessary  because  it  is  excruciating  for  people  in  the  countries  where  Kanji  are  not  used  to 
learn  them  in  a short  period  of  time.  When  people  in  the  countries  where  Kanji  are  not  used  learn  their  mother  tongue, 
they  emphasize  "conversation,"  Therefore,  even  when  they  learn  Japanese,  they  emphasize  "conversation."  In 
comparison  with  Japanese,  a one  byte  code  language,  in  the  first  place,  has  a fewer  number  of  characters  but  has  a 
number  of  types  of  pronunciation.  Therefore,  such  a language  can  be  mastered  mainly  by  improving  conversation 
ability. 

From  such  a reason,  for  people  of  the  countries  where  Kanji  are  not  used  inevitably  neglect  learning  of  Kanji, 
To  effectively  learn  Kanji,  the  multimedia  Kanji  dictionary  was  developed.  Since  the  electronic  dictionary  facilitates 
the  understanding  of  Kanji  by  using  animation  and  sound,  it  can  improve  learning  efficiency  much  more  than  a 
conventional  paper  dictionary.  Also,  the  characters  of  the  index  are  so  small  that  we  sometimes  become  fed  up  with 
looking  up  Kanji  in  the  paper  dictionary.  However,  people  will  become  able  to  search  Kanji  more  easily  by  just 
clicking  if  they  use  the  multimedia  Kanji  dictionary.  This  electronic  dictionary  is  functioning  as  the  dictionary  module 
of  an  integrated  Japanese  learning  system  called  "Terakoya," 

URL:  http://www.narita.elec.waseda.ac.ip/~fuiita/terakova.html 


Kanji  Dictionary 

This  dictionary  system  can  be  divided  into  the  Kanji  searching  part  and  dictionary  database  part.  In  the 
searching  part,  there  are  three  kinds  of  functions  "Read,"  "Stroke  count"  and  "Radical,"  Here,  we  can  find  a Kanji 
character  for  which  we  are  searching,  and  we  can  read  out  detailed  information  from  the  dictionary  database  that  is 
stored  as  a CSV  file.  On  the  one  hand,  this  system  contains  detailed  information  such  as  radicals  and  the  stroke  count 
for  people  who  want  to  learn  detailed  information.  On  the  other  hand,  this  system  facilitates  easy  learning  of  Kanji  by 
including  items  for  children  such  as  pictures  and  figures  that  look  similar  to  Kanji,  This  system  has  become  an 
interactive  system  that  has  more  mobile  capability  by  using  animation  and  sound,  which  is  in  contrast  to  the 
conventional  dictionary  system  that  emphasized  text  data.  Animation  is  used  to  display  the  order  of  strokes  of  a Kanji, 
and  voice  is  used  to  let  the  learner  hear  the  data  of  "Reading"  and  "Examples."  Furthermore,  in  order  to  have  the  users 
learn  a large  quantity  of  Kanji,  this  system  understands  Kanji  through  graphic  means  by  recognizing  each  Kanji  as  a 
picture  or  a figure  to  improve  learning  efficiency. 

And,  because  this  system  is  using  WWW,  it  is  not  dependent  on  a particular  type  of  platform.  A learner  can 
use  it  through  the  Internet  from  PC  regardless  of  whether  the  person  is  at  home  or  in  a class.  Even  if  there  was  no  such 
environment,  in  such  case  information  of  Kanji  can  be  read  by  printing  the  contents  out  on  paper  as  a conventional 
dictionary.  Although  this  system  is  developed  with  Shockwave  of  Macromedia  Company,  the  browser  is  not  able  to 
function  as  a printer  when  the  movie  that  was  made  with  Shockwave  is  loaded  into  the  browser.  Therefore,  a page 
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where  Kanji  information  is  transformed  into  an  HTML  has  been  made.  If  the  hard  copy  of  Kanji  information  can  be 
made,  it  can  be  used  at  places  where  there  is  no  computer  nor  a network  environment.  This  function  comes  from  the 
requests  of  the  Japanese  language  teachers  and  also  of  Japanese  language  learners. 


Future  Plans 

Although  this  dictionary  system  has  searching  functions  of  "Read,"  "Stroke  count"  and  "Radical,"  these 
searching  methods  are  used  conventionally  in  the  conventional  paper  dictionary.  However,  there  must  be  cases  where  a 
beginner  of  Kanji  learning  has  no  idea  about  the  reading,  the  stroke  count  nor  the  radical  of  the  Kanji  that  he/she  wants 
to  find  out.  To  this  end,  we  are  studying  the  functions  by  which  a Kanji  can  be  searched  through  handwritten  letter 
recognition.  If  this  searching  method  is  created,  a learner  can  search  for  the  Kanji  by  the  action  that  is  similar  to 
drawing  a picture  of  the  Kanji  whose  detailed  information  is  not  known  to  the  learner.  Also,  although  the  present 
system  has  registered  80  Kanji  characters  in  the  database,  in  oider  to  maintain  coordination  with  the  "Terakoya”  system, 
it  is  necessary  to  classify  Kanji  by  the  degree  of  difficulty  or  by  the  use  frequency  in  daily  life  to  offer  data  to  the  user  in 
an  easy-to*understand  manner.  Furthermore,  because  there  are  some  cases  where  the  requests  of  teachers  differ  from 
the  contents  of  a dictionary,  we  should  add  such  function  that  enables  the  teachers  to  freely  select  the  words  and 
pronunciations,  etc.  that  he/she  wants  to  use  in  the  lesson.  If  there  is  such  a function,  a lesson  will  become  very 
effective.  At  last,  we  need  to  evaluate  this  Kanji  dictionary.  This  system  will  be  tested  by  the  Center  for  Japanese 
Language  of  Waseda  University  and  the  Richmond  school  in  Portland.  And,  we  want  to  advance  to  further 
improvement  by  asking  advices  from  teachers  and  students. 
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Abstract:  The  purpose  of  this  study  was  to  identify  the  similarities  and  differences  of  online  and 
traditional  foreign  language  courses  and  to  determine  the  advantages  and  disadvantages  of  the 
online  courses.  Using  a random  sample  of  20  courses,  ten  on-line  and  ten  traditional,  this  study 
examined  the  online  and  traditional  courses  in  terms  of  five  constructs  and  compared  the  methods 
and  strategies  used  in  online  courses  with  regular  classroom  instruction.  With  a rubric  developed 
for  the  study,  the  online  and  traditional  course  were  rated  in  a 4-point  scale  and  analyzed  with 
independent  t-tests.  The  findings  of  the  study  indicated  that  online  and  traditional  courses  had  about 
the  same  weight  in  teaching  foreign  language  and  suggested  that  they  each  had  its  advantages  and 
disadvantages.  The  study  concluded  that  traditional  courses  had  better  defined  and  more  specific 
goals  and  objectives  for  both  teaching  and  learning  and  was  more  effective  in  teaching  conversation 
and  listening  comprehension,  while  in  online  courses,  students  could  learn  at  varying  rates  and 
explore  material  to  whatever  depth  they  desire,  and  Internet  technology  made  it  possible  for 
students  in  different  areas  and  from  different  countries  to  access  online  courses. 


Introduction 

With  the  rapid  development  and  fast  growing  adoption  of  technology,  the  use  of  technology  in  education  has  been 
dramatically  increasing  over  the  past  few  years.  In  the  area  of  language  education,  various  technologies  are  being  utilized  by 
language  teachers  and  learners  in  and  out  of  the  classroom  (Warschauer,  1996).  These  uses  would  include  the  traditional  use  of 
audio  and  films;  however  they  are  now  being  supplemented  by  computer-assisted  instruction  and  interactive  media  technologies 
(Richards,  1996).  Computer-based  programs,  as  well  as  other  computer-related  interactive  technologies,  have  been  specifically 
developed  to  provide  students  with  visual  and  audio  support.  These  applications  have  shown  potential  for  helping  students 
develop  their  language  abilities  and  improve  classroom  performance  (Pennington,  1996). 

Among  a variety  of  instructional  delivery  media,  the  Internet,  especially  the  World  Wide  Web,  has  been  commonly 
adopted  by  many  educators  in  higher  education  (Khan,  1997;  Porter,  1997).  However,  little  research  results  have  shown  to 
support  claims  for  the  effectiveness  of  Web-based  instruction  (Nunan,  1999).  The  focus  of  this  study  was  an  analysis  of  existing 
web-based  foreign  language  courses  and  a comparison  of  the  methods  and  strategies  used  in  online  courses  and  regular  classroom 
instruction  to  determine  the  common  and  unique  methods  and  strategies  used  to  achieve  the  educational  objectives  in  each 
environment. 

Purpose  of  the  Study 

The  purpose  for  conducting  this  study  was  to  identify  the  similarities  and  differences  of  online  and  traditional  courses 
of  teaching  foreign  languages  and  to  determine  the  advantages  and  disadvantages  of  the  online  courses.  Methods,  strategies,  and 
technologies  used  in  online  foreign  language  courses  were  examined  and  compared  with  those  of  the  traditional  foreign  language 
courses.  In  order  to  achieve  the  purpose  of  the  study,  the  current  study  addressed  the  following  research  questions: 

1.  Based  on  the  courses  examined,  what  methods,  techniques  and  strategies  were  used  for  teaching  online  foreign  language 
courses  and  traditional  foreign  language  courses? 

2.  What  were  the  differences  between  traditional  foreign  language  courses  and  online  foreign  language  courses  in  terms  of 
goal/objectives,  strategies,  interaction,  feedback,  and  resources? 

3.  What  were  the  advantages  and  disadvantages  of  the  online  courses? 

Method 

The  sample  for  this  study  consisted  of  twenty  foreign  language  courses,  with  ten  on-line  courses  and  ten  traditional 
courses.  Thirty  online  courses  were  first  identified  directly  from  foreign  language  courses  available  on  the  Internet,  which 
included  listening,  speaking,  reading,  and  writing  courses.  The  following  systematic  procedure  was  used  to  randomly  select  ten 
courses  from  the  thirty  previously  identified  online  courses:  First,  a random  list  of  the  thirty  online  courses  was  generated.  Then 


a start  point  for  the  selection  was  randomly  chosen  on  the  list.  From  that  point  every  third  course  was  selected  until  a total  of  ten 
online  courses  were  identified  for  the  study.  Among  the  ten  courses  that  were  chosen  were  two  listening  courses,  one  speaking 
course,  two  reading  courses,  two  grammar  courses,  and  three  writing  courses. 

With  the  same  procedure,  the  ten  traditional  courses  used  for  this  study  were  randomly  selected  from  foreign  language 
courses  offered  by  the  Intensive  English  Programs  and  foreign  languages  departments  at  various  universities.  The  course 
materials  of  these  ten  courses  were  available  at  the  University  of  Louisville,  the  University  of  Alabama,  Southeast  Missouri  State 
University,  and  North  Carolina  Wesleyan  College.  They  were  three  listening  courses,  two  speaking  courses,  two  reading  courses, 
one  grammar  course,  and  two  writing  courses,  which  comprised  the  traditional  foreign  language  course  group  for  data  analysis. 
With  a rubric  developed  for  the  study,  the  online  and  traditional  course  were  rated  on  a 4-point  scale  and  analyzed  with 
independent  t-tests. 

Results  and  Discussion 

The  comparison  by  means  of  the  rubric  indicated  that  the  online  courses  and  the  traditional  courses  had  about  the  same 
weight  in  teaching  foreign  languages  in  general.  Online  courses  had  a total  score  of  147,  while  the  score  of  the  traditional 
classroom  teaching  was  153.  However,  the  independent  t-tests,  which  were  performed  to  identify  any  possible  significant 
difference  between  the  online  and  traditional  courses,  found  that  significant  differences  existed  between  the  two  types  of  courses 
in  two  of  the  five  constructs  measured. 

Conclusion 

A foreign  language  acquisition  process  consists  of  listening,  speaking,  reading,  and  writing  skills.  In  order  to  facilitate 
the  success  of  second  language  teaching  and  learning,  modem  technologies  have  been  used  in  the  classroom  to  promote  teaching 
performance  and  student  learning.  Online  courses,  which  have  been  developed  for  that  purpose,  have  proved  their  advantages  in 
foreign  language  teaching  and  learning.  However,  when  comparing  traditional  classroom  teaching  with  online  courses,  the  results 
of  this  study  suggest  that  each  had  its  advantages  and  disadvantages.  For  traditional  courses,  better  defined  and  more  specific 
goals  and  objectives  were  given  for  both  teaching  and  learning.  Delivering  course  material  through  the  Web,  on  the  other  hand, 
afforded  some  advantages  over  traditional  classroom  teaching.  In  online  courses,  students  could  learn  at  varying  rates  and 
explore  material  to  whatever  depth  they  desire.  There  were  new  ways  to  guide  students  through  tutorials  and  to  provide  them 
with  instant  feedback.  Using  Internet  technology  made  it  possible  for  students  in  different  areas,  different  countries  to  use 
varying  computer  types  to  have  access  to  the  course. 

However,  as  far  as  a foreign  language  is  concerned,  it  is  obvious  that  traditional  classroom  teaching  was  better  in 
teaching  conversation  and  listening  comprehension  courses  when  students  needed  face-to-face  interactions  with  the  instructor  and 
instant  verbal  feedback  from  the  instmetor.  Computer-mediated  communication  had  been  criticized  as  lacking  the  human  social 
interaction  necessary  for  human  learning  (Gilbert,  1996)  although  Lemke  (1993)  claimed  that  "cyberspace  will  be  a virtual  place 
FOR  human  social  interaction."  It  seems  that  web-based  courses  might  not  be  able  to  provide  as  effective  or  efficient  a means  of 
learning  a language  as  one-on-one  human  interaction.  Without  such  non-verbal  cues  as  body  language  and  the  tone  and  pitch  of 
one’s  voice,  it  is  possible  to  misconstrue  messages  (Berge  & Collins,  1995;  Gilbert,  1996). 

In  summary,  neither  of  the  two  types  of  courses  is  absolutely  good  or  faulty.  Each  had  its  own  advantages  and 
disadvantages.  As  an  instructional  designer,  we  should  be  aware  of  the  advantages  and  disadvantages  of  both  types  of  courses 
and  make  full  use  of  their  advantages  in  the  design  and  development  of  the  courses,  based  on  different  student  needs,  student 
characteristics,  and  skill  requirements,  to  achieve  the  goals  of  our  education. 

References 

Berge,  Z.  L.  & Collins,  M.  P.,  Eds.  (1995).  Computer-mediated  communication  and  the  online  classroom.  Cresskill,  NJ:  Hampton 
Press. 

Gilbert,  K.  K.  (1996).  Grief  in  a family  context:  a for-credit  university  course  delivered  on  the  World  Wide  Web.  Paper  presented  at 
the  Annual  Meeting  of  the  National  Council  on  Family  Relations,  Kansas  City,  MO. 

Khan,  B.  H.  (1997).  Wed-based  instruction.  Englewood  Cliffs,  NJ:  Educational  Technology  Publications. 

Lemke,  J.  L.  (1993).  Education,  cyberspace,  and  change.  The  Arachnet  Electronic  Journal  of  Virtual  Culture.  1 (1)  [On-line  serial]. 
Available:  http://www.monash.edu.au/joumals/ejvc/lemke.vl  nl 

Nunan,  D.  (1999).  A foot  in  the  world  of  ideas:  Graduate  study  through  the  Internet.  Language  Learning  & Technology,  3(1)  [On-line 
serial].  Available:  http://polyglot.cal.msu.edu/llt/vol3numl/nunan/index.html 
Pennington,  M.  C.  (1996).  The  power  of  the  computer  in  language  education.  In  M.  C.  Pennington  (Ed.),  The  power  of  CALL,  (pp.  1- 
14).  Houston:  Athelstan. 

Porter,  L.  R.  (1997).  Creating  virtual  classroom:  Distance  learning  with  the  Internet.  New  York:  Wiley. 

Richards,  J.  C.  (1996).  Preface.  In  S.  Fotos  (Ed.),  Multimedia  Language  Teaching,  (pp.  ix-xi).  Tokyo:  Logos  International. 

Warschauer,  M.  (1996).  Computer-assisted  language  learning:  An  introduction.  In  S.  Fotos  (Ed),  Multimedia  Language  Teaching. 
(pp.  3-20).  Tokyo:  Logos  International. 


1566 


The  Road  to  Hell  Or  Wrong  Turns  on  the  Path  to  Cyberlearning 


Stephenie  Yearwood 
Lamar  University 
P.O.  Box  10023 
Beaumont,  TX  77710 
Steffiyl@aol.com 

Paula  Nichols 
Lamar  University 
P.O.  10034, 

Beaumont,  TX  77710 
pnichols@tenet.edu 

Abstract:  The  paper  proposes  and  explains  five  common  mistakes  which  institutions  often 
make  in  implementing  distance  education:  using  packaged  courses  and  low-level  instructors, 
using  one  technology  to  deliver  all  distance  education,  not  monitoring  student  satisfaction  and 
retention,  assuming  that  all  courses  can  and  should  be  offered  via  distance  technology,  and 
focusing  marketing  only  on  the  technology-literate. 


Racing  along  the  Road 

The  shape  of  distance  education  is  changing  daily  as  more  and  more  institutions  embrace  the  technologies 
now  available  to  deliver  remote  instruction.  One  estimate  by  Robert  Tucker  (1997)  quoted  recently  in 
Forbes,  is  that  about  90%  of  US  four-year  institutions  will  have  technology-based  distance  learning 
offerings  by  2000.  Even  in  1996,  over  a million  students  took  distance  learning  courses  of  the  14  million 
attending  post-secondary  educational  institutions.  And,  although  these  courses  use  many  different 
technologies,  there  is  rapidly-increasing  focus  on  internet  delivery.  In  this  rush  to  get  on  the  bandwagon, 
the  authors,  who  have  been  involved  with  emerging  distance  education  in  Texas  for  the  last  ten  years,  have 
observed  several  common  assumptions  which  cause  great  concern.  The  focus  of  this  paper  will  be  to 
identify  and  call  into  question  some  frequent  phenomena  which,  we  believe,  threaten  to  undermine  the 
success  of  cyberleaming. 

Wrong  Turn  One:  Buy  a packaged  course  and  hire  a TA  or  two  to  run  it 

One  of  the  driving  forces  behind  the  charge  to  cyberleaming  is  the  common  perception  that  cyberleaming 
will  be  a cash  cow  for  traditional  institutions,  allowing  them  to  cut  costs.  In  a rush  to  milk,  institutions  are 
sometimes  choosing  to  dispense  with  home-grown  courses  run  by  local  professors.  This  choice  threatens  to 
result  in  a world  of  cyberleaming  where  there  are  many  institutions  to  choose  from,  but  all  are  teaching  the 
same  course.  Losing  the  personal  enrichment,  the  personal  mentoring  and  the  constant  updating  which  an 
experienced  teacher  puts  into  a course  is  equivalent  to  losing  the  personal  voice  or  identity  of  an  institution. 
Cyberleaming  should  increase  the  diversity  of  the  educational  ecosystem,  not  decrease  it. 

Wrong  Turn  Two:  Buy  one  technology  and  use  it  for  everything. 

Because  the  costs  for  establishing  technology-based  distance  learning  can  be  large,  institutions  are  prone  to 
make  an  initial  commitment  to  a technology  (VHS  with  two-way  audio,  or  TV,  or  compressed  video,  or 
web)  then  force  all  course  offerings  to  fit  themselves  to  that  technology.  Institutions  have  been  known  to 
simply  buy  whatever  was  pitched  to  them  most  intensely  in  the  hopes  of  not  having  to  face  the  bewildering 
array  of  choices  again  for  quite  some  time.  However,  close  analysis  of  student  needs,  course  content,  and 
educational  effectiveness  has  led  us  to  believe  that  hybrid  technologies  evolved  over  time  and  respondent  to 
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individual  courses  and  student  audiences  produce  superior  educational  results.  Educational  needs  and  goals 
should  drive  technology,  not  vice  versa. 

Wrong  Turn  Three:  Don’t  worry  about  student  satisfaction;  the  learning  is  fine 

Indeed,  much  past  research  on  distance  learning  using  all  kinds  of  different  delivery  modalities  from 
correspondence  courses  to  web  has  indicated  that  students  seem  to  learn  as  much  from  distance  courses  as 
they  do  from  face-to-face  instruction  (Russell  1996).  However,  we  cannot  ignore  that  cybercourses  vary 
enormously  in  learner-satisfaction  and  quality.  In  an  ideal  educational  cyber-marketplace,  learner-centered 
course  design  will  eventually  prevail.  Courses  which  use  technology  well  and  flexibly  to  deliver  their 
course  content  and  achieve  course  goals,  and  courses  which  attend  to  a wide  variation  in  student  learning 
styles  will  have  better  sueeess.  Convenience  of  access  is  not  enough;  quality  matters,  too. 

Wrong  Turn  Four:  Move  everything  onto  the  cyber-track. 

With  the  emergence  of  virtual  universities  and  entire  degree  plans  available  by  remote  technology,  the 
instinct  of  many  institutions  is  to  think  of  distance  learning  offerings  as  a substitute  for  face-to-face 
instruction,  rather  than  an  adjunct  to  it.  However,  a closer  look  shows  quite  clearly  that  cyberleaming  has, 
thus  far,  leaned  heavily  in  the  direction  of  engineering,  business,  computer  science  and  health  sciences. 
(Miller  1998).  According  to  Miller,  all  other  subject  matters  account  for  only  10%  of  cybercourses.  In  spite 
of  the  pronouncement  of  Peter  Drucker  (1997)  that  "Universities,  as  we  know  them,  won't  survive,"  almost 
all  lower-division  and  humanities  courses  continue  to  be  offered  in  traditional  fashion.  One  reason  for  this, 
we  believe,  is  the  need  for  greater  flexibility  in  adapting  to  the  goals  of  such  courses  and  the  needs  of 
students.  Cyberleaming  should,  we  believe,  offer  additional  choices  to  students  and  teachers— not  force 
them  into  a course-delivery  system  alien  to  them.  In  fact,  we  go  so  far  as  to  argue  that  cyber  courses  should, 
in  most  cases,  have  live,  face-to-face  versions  running,  too. 

Wrong  Turn  Five:  Just  market  the  courses  and  the  students  will  come 

Certainly  convenience,  accessibility,  and  low  cost  are  strong  incentives  for  students  to  engage  in 
cyberleaming.  However,  issues  of  equity  and  market  penetration  will  become  ever  more  pressing  as 
institutions  seek  to  draw  increasing  numbers  of  students.  Access  to  computers  is  increasing  daily,  as  is 
internet  access;  however  individuals  without  home  computers  or  internet  access  may  be  precisely  the  ones 
who  most  need  access  to  "anytime,  anywhere"  post-secondary  education.  Institutions  must  attend  to 
insuring  access  at  public  libraries,  K-12  schools  and  community  centers  in  order  to  insure  equity  for  all 
ranks  of  society  and  to  avoid  widening  the  digital  divide.  Moreover,  some  segments  of  the  market  will  need 
additional  technology  training  in  order  to  effectively  use  the  access  which  is  available  to  them,  and 
institutions  must  provide  appropriate  training  in  technology  if  they  are  to  draw  high-school  dropouts  and 
retired  persons  into  cyberleaming. 

In  a dynamic  growth  period  such  as  we  face  now,  the  drive  to  implement  cyberleaming  will 
benefit  many  students  and  institutions,  but  only  if  decision-makers  tread  the  path  thoughtfully  and  with 
their  eyes  open  for  wrong  turns. 
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Abstract:  This  paper  sought  to  identify  learner-demonstrated  learning  patterns  when  undergraduate  students  were  learning  to  use  a computer-based  presentation 
program  in  a multimedia  learning  environment  without  time  constraints.  Using  analysis  of  patterns  in  time  (APT),  a methodology  that  can  code  event  changes  over 
time,  the  frequencies  and  the  amounts  of  time  of  the  five  learning  patterns  were  calculated.  The  data  derived  from  the  APT  scores  indicated  two  major  findings:  (1) 
The  amount  of  time  used  by  the  participants  ranged  from  20  to  87  minutes.  The  amount  of  time  spent  did  not  predict  mastery  of  the  post  test.  (2)  ‘Following 
through  error  corrections’,  ‘confirming  actions’,  and  ‘trying  new  steps’  were  the  patterns  that  did  appear  to  predict  mastery,  and  ‘hand  on  mouse'  and  ‘keeping  up 
step-by-step  with  the  video  instruction’  were  indicators  of  learners’  task  engagement.  The  findings  suggest  that  it  is  important  to  design  flexible  instruction  to 
facilitate  repeated,  persistent,  and  successful  practice  in  order  to  achieve  mastery  in  a self-paced,  computer-based  learning  environment,  regardless  of  the  amount  of 
time  spent. 


lotroduction 


This  study  sought  to  investigate  the  existence  of  any  significant  temporal  patterns  of  learner-demonstrated  sequencing  as  learning  styles  that  are  related  to  the  mastery  of  leamiug  a piuccduiul 
coiiipuicr-bascu  iiiulitmcuia  task. 

The  impact  of  how  different  learning  styles  affect  human  learning  has  concerned  educational  researchers  for  decades  (Bruner,  1966;  Gagn6,  1967;  Kolb,  1984;  Wang  and  Walbcrg,  1985;  Tobias, 
1981,  1987).  According  to  Cronbach  and  Snow  (1977),  learning  style  can  be  used  to  predict  what  kind  of  instructional  methods  or  strategies  would  be  most  effective  for  a given  individual  and  learning  task. 
Such  attempts  to  discover  the  predictors  for  content  mastery  were  typified  in  Aptitude  Treatment  Interaction  (ATI)  research.  In  ATI  studies,  aptitude  was  defined  as  any  learner  characteristic  related  to 
instructional  outcome,  and  treatment  was  defined  as  any  variation  in  instruction  to  the  learner  (Cronbach  and  Snow,  1969).  However,  research  to  date  on  this  problem  has  not  identified  many  robust 
relationships  between  learning  style  and  instructional  method  that  are  needed  in  order  to  generalize  the  assignment  of  instruction  to  different  learners  (Snow,  1989).  Could  this  lack  of  robustness  be  caused  by 
overlooking  the  means  in  favor  of  the  ends  of  learning?  Or  perhaps  the  notion  of  aptitude  needs  to  be  defined  differently  when  the  means  of  learning  (or  learning  process)  is  considered  as  a critical  component 
in  a learning  experience. 

Frick  (1990;  1992)  indicates  that  one  of  the  fundamental  problems  of  ATI  is  that  the  linear  model  approach  (LMA)  employed  by  ATI  cannot  explicitly  demonstrate  the  different  means  of 
learning  because  the  LMA  was  designed  to  estimate  parameters  of  mathematical  models  and  is  not  adequate  in  dealing  with  occurrences  and  relative  frequency  of  temporal  events.  LMA  is  best  for  explaining 
the  relation  between  two  or  more  independently  measured  variables,  but  it  cannot  measure  the  frequency  or  duration  of  occurrences  of  a temporal  path  (Frick,  1983).  This  observation  directly  implies  that  there 
is  a need  for  a methodology  other  than  LMA  when  we  investigate  the  temporal  patterns  in  a learning  process,  especially  with  a concurrent  attempt  to  examine  aptitude  in  a different  way. 

Learning  takes  time.  Learners  take  different  amounts  of  time  to  perceive  information  in  order  to  acquire  knowledge.  Knowledge  is  actively  constructed  through  temporal  sequencing  of 
meaningful  interactions  between  the  individual  and  the  learning  context  (Piaget,  1973;  Dewey,  1916;  Bruner,  1973;  von  Glasersfeld,  1991).  But  how  can  we  move  the  learning-styles  research  away  from  the 
instruction-centered  “how  one  should  learn”  to  the  Icamer-ccntcred  ‘‘how  one  docs  Icam”?  In  order  to  make  instructional  systems  more  adaptive,  researchers  need  to  put  more  emphasis  on  Icamcr-dcmonstrated 
learning  styles,  where  the  relationships  between  the  sequencing,  time  duration,  and  nature  of  the  media  should  be  carcfUlly  examined.  This  need  is  especially  important  now  that  multimedia,  productivity 
software  programs,  and  the  explosive  growth  of  the  Internet  have  entered  our  educational  settings  and  homes  (Nielsen,  1995).  and  the  technological  capabilities  have  caused  us  to  reconccptualizc  the  learning 
process  and  to  design  new  instructional  approaches  (Duffy  and  Jonassen,  1992). 

To  understand  the  need  to  redefine  learning  styles,  it  is  necessary  to  comprehend  the  nature  and  limitations  of  conventional  learning  styles.  Sewall  (1986)  compares  four  popular  instruments  to 
measure  learning  styles:  the  Myers-Briggs  Type  Indicator  (Myers,  1962),  the  Kolb  Learning  Style  Inventory  (Kolb,  1976),  Canfield’s  Learning  Style  Inventory  (Canfield.  1980),  and  Gregorc’s  Type  Indicator 
(Grcgorc,  1984).  He  finds  that  all  of  these  instruments  fall  short  generally  in  four  areas:  (1)  no  appropriate  normative  base  for  the  valid  interpretation  of  scores;  (2)  limited  validity  due  to  construction  problem 
with  the  instruments;  (3)  reliability  estimates  arc  unstable,  though  one  can  argue  that  the  dynamic  nature  of  learning  style  makes  high  test-retest  reliability  unnecessary;  (4)  the  normative  frame  of  reference 
makes  the  interpretation  of  scores  very  difficult.  Among  his  suggestions  for  future  research,  one  suggestion  particularly  interests  the  present  writer: 

“Future  research  could  also  explore  the  issue  of  whether  an  individual’s  preferred  learning  style  is  modified  by  the  educational  environment.  For  example,  do  adult  students  leam  better  when  instruction 
is  adapted  to  their  learning  style  preferences?  Can  people  be  trained  to  adopt  a particular  learning  style?  Do  learning  styles  remain  stable  over  time  in  the  adult  population?  Does  a significant  change  in  life 
situations  result  in  changes  in  learning  styles?”  (p.61.) 

It  is  interesting  to  notice  that  all  four  of  the  above-mentioned  learning-style  instruments  use  lists  of  verbal  statements  or  different  visual  forms  prepared  for  people  to  choose  or  rank.  This  isolated 
“one  size  fits  all”  approach  is  dccontcxtualizcd  and  artificial,  and  apparently  biased  on  language  skills  and  cultural  backgrounds.  The  predefined  assessment  tools  require  people  to  fit  in  the  limited  and  selective 
choices  with  “1  think  this  is  who  I am”  or  “1  think  this  is  what  1 will  do  or  how  1 will  react,”  which  may  not  predict  how  one  actually  reacts  to  or  interacts  with  a problem  in  a specific  learning  situation.  This 
predetermined  and  convergent  assessment  approach  may  be  the  fundamental  reason  that  only  limited  reliability  and  construct  validity  have  been  found  in  existing  learning-style  studies. 

Maybe,  as  Sewall  suggested,  human  learning  styles  are  constantly  modified  by  the  educational  environment  and  social  context  and  are  too  dynamic  to  measure  on  a predetermined  and  predictable 
scale.  This  notion  leads  the  present  writer  to  believe  that  to  survey  fully  the  dynamic  nature  of  learning  styles,  rather  than  using  artificial  instruments,  we  can  observe  and  faithfully  record  the  event  changes  of  a 
leamer-demonstrated  process  over  time.  In  other  words,  the  result  of  this  stu^  may  not  verify  that  learning  style  is  a stable  trait  (Bateson,  1979). 

Research  Questions 

1.  In  the  midst  of  the  information  explosion,  multimedia  has  become  the  new  learning  environment.  Learning  how  to  interact  with  multimedia-based  information  or  instructions  has  become  our  daily 
experience.  It  is  important  to  note  that  multimedia  is  a combination  of  three  forms  of  content:  (1 ) abstract  (e.g.,  verbal  — either  text  or  audio  — explanation  in  a computer  program  window),  (2)  iconic 
(e.g.,  the  screen  capture  of  a computer  program  window  printed  on  paper),  and  (3)  concrete  (e.g.,  the  actual  computer  program  windows  displayed  on  a monitor  screen).  The  main  questions  addressed  in 
this  study  were  as  follows:Gtvcn  the  choice  of  instructional  materials  in  different  forms  of  content,  what  learning  sequences  are  selected  by  the  learners? 

1.1.  If  the  sequences  differ,  how  do  they  differ? 

1.2.  Arc  any  of  these  sequences  especially  likely  to  result  in  overall  mastery? 

1.3.  Is  there  a difference  in  the  amount  of  time  spent  learning  for  those  learners  who  achieved  mastery  and  those  who  did  not? 

In  order  to  increase  both  reliability  and  validity  in  identifying  the  learning  patterns  (Denzin,  1970;  Foreman,  1948),  a triangulation  method  was  used  that  involved  obseivation  of  the  learning 
process  and  think-aloud  interview.  “Triangulation”  is  to  use  multiple  methods  in  data  collection  (Denzin,  1970).  Mciriam  (1988)  indicates  that  “methodological  triangulation  combines  dissimilar  methods  such 
as  interviews,  observations,  and  physical  evidence  to  study  the  same  unit.”  An  observation  system  designed  and  based  on  the  analysis  of  patterns  in  time  (APT)  approach  was  used  to  code  how  and  how  long 
students  use  and  switch  the  instructional  materials,  and  their  engagement  during  the  processes  of  learning.  Both  the  learning  processes  and  the  think-aloud  interviews  were  recorded  on  videotapes  for 
examination  and  analysis. 

Twelve  participating  students  from  a midwest  regional  university  were  selected  through  a screening  process  using  Kolb’s  Learning  Style  Inventory  (Kolb,  1976)  where  three  students  were 
selected  extreme  cases  in  each  of  the  four  learning  styles  identified  by  Kolb  (converger,  accomodator , diverger,  and  assirailator).  The  participants  were  expected  to  create  three  electronic  slides  using  Astound, 
a multimedia  presentation  software,  during  the  learning  process.  They  were  told  that  there  was  no  time  constraint  and  that  they  could  freely  use  the  printed  Jumpstart  instructions,  the  video  demonstration 
parallel  to  the  printed  instructions,  and  the  user  manual  and  tutorial.  They  then  created  a fourth  slide  without  instruction  as  the  post  test.  Learning  sessions  were  recorded  on  video,  and  participants  subsequently 
watched  their  own  videos  and  were  interviewed  using  think-aloud  methodology.  Multiple  observers  watched  the  videotapes  of  the  learning  performances  for  patterns  of  interaction,  which  were  illuminated  by 
the  think-aloud  interviews. 

In  this  study,  the  interview  was  semistructured.  The  interviews  followed  the  observations,  hence  the  interviews  supplement  and  complement  the  outcomes  of  the  observations,  rather  than 
forming  a different  set  of  data  not  related  to  that  from  the  observations.  The  questions  asked  were  open-ended  so  the  students  could  explain  fully  and  interpret  any  specific  events  that  occurred  in  their  learning 
performance. 

As  soon  as  the  raw  data  from  the  interviews  were  collected,  the  data  gathered  from  both  the  observation  and  the  interview  on  each  student  were  arranged  into  one  case  for  analysis.  As  Merriam 
(1988)  suggests,  the  “simultaneous  analysis  and  data  collection  allows  the  researcher  to  . . . develop  a data  base  that  is  both  relevant  and  parsimonious.” 

Next,  the  data  collected  from  both  the  observation  and  the  interview  were  sorted  into  any  repeated  patterns  that  arc  common  to  more  than  one  person  or  identifiable  categories  or  themes.  Some 
patterns  and  their  variations  were  spotted  faster  than  others.  Some  unexplainable  events  or  sequences  from  the  observation  were  explained  when  data  from  the  interview  were  reviewed  and  compared.  This  step 
was  repeated  until  all  12  cases  were  analyzed.  Then,  an  analysis  to  rcduee  and  refine  the  categories  or  patterns  among  the  cases  was  conducted. 

Result 

The  narrative  temporal  descriptions  of  each  of  the  12  cases  were  eompiled.  The  descriptions  included  answers  to  the  following  questions:  Who  was  the  person  (age.  gender,  race,  academic  major, 
Kolb’s  learning  style,  mastery  of  the  post  test,  prior  computer  experience).  What  did  s/he  do  with  the  instructional  materials  during  different  parts  of  the  learning  session?  What  comments  (direct  quotes  from 
the  think-aloud  videos)  did  the  person  make  about  what  s/he  was  doing?  These  thiek  descriptions  (Lincoln  and  Guba,  1985)  illustrated  how  the  participants  actually  interacted  with  the  instructional  materials 
during  the  learning  sessions.  These  thick  descriptions  (Lincoln  and  Guba,  1985)  illustrated  how  the  participants  actually  interacted  with  the  instructional  materials  during  the  learning  sessions.  To  demonstrate 
the  uniqueness  of  these  participants,  the  following  is  a complete  set  of  descriptions  recorded  from  one  of  the  12  students. 

Student  F 

Student  F was  a twenty-two-year  old  white  male.  He  majored  in  Secondary  Education  and  had  had  four  years  of  computer  experience.  He  was  a converger  (with  a tendency  toward  active 
experimentation  and  abstract  conceptualization),  according  to  Kolb’s  learning  style.  He  was  judged  a master  of  the  post  test.  He  spent  a total  of  42.5  minutes  in  completing  the  learning  tasks  and  the  post  test. 

Student  F spent  nine  minutes  on  Task  #1.  He  began  his  learning  session  by  playing  the  video  demonstration  while  reading  the  instructions.  Student  F is  rather  reserved  but  well-mannered.  Of 
his  choices  of  instructional  materials,  he  said,  “I  didn’t  mind  it.  I like  to  have  something  to  follow  along  with  [the  printed  instructions},  and  I can  stop  [the  video]  whenever  I want,  so  that  makes  it  a lot  ^icr 
for  me.”  When  asked  how  he  used  the  instructional  materials,  he  answered,  “I  started  by  reading  the  [printed]  instructions  and  watching  the  videotape  step-by-step.  But  after  a while,  I was  not  even  watching  it, 
‘cause  I know  pretty  much  what  to  do  by  listening  to  it.”  He  said  of  his  reading  strategy.  “After  a while,  1 was  just  reading  ahead,  and  make  sure  that  I wa 
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insmictions,  “I  could  have  gone  faster”,  he  said,  “but  I just  wanted  to  cover  all  three  [video,  computer,  and  printed  instructions]  to  make  sure  I am  not  doing  anything  wrong." 

Student  F spent  eight  minutes  on  Task  #2.  He  continued  to  follow  the  video  demonstration  while  reading  the  instructions.  He  noted  that  he  did  not  use  the  manual,  ”I  didn’t  use  the  book  [the  user 
manual].  It  was  kind  of  complicated.  Your  language  may  not  add  up  to  the  technical  language  they  use.  It  is  better  to  have  these  kind  of  materials  [the  printed  instructions]  that  are  more  specific.”  He  said  of 
recovering  from  a mistake,  “When  1 made  that  mistake,  1 was  not  watching  the  video.  1 just  remember  how  to  put  a ‘Next’  button  and  I just  did  it."  On  how  he  checked  with  the  instructional  materials,  he  noted 
"After  a while,  I was  just  checking  [with  the  instructions],  though  1 wouldn’t  go  word  for  word.’’ 

Student  F spent  nine  minutes  on  Task  #3.  He  continued  to  follow  the  video  demonstration  while  reading  the  instructions.  He  said  he  had  his  hand  on  the  mouse  because  “I  think  I was  just 
getting  ready  for  the  next  step.”  When  asked  what  if  he  were  not  given  the  printed  instructions,  he  replied  “1  wouldn’t  have  read  the  instructions  if  1 didn’t  have  it.  Since  I had  it,  I used  it  anyway  for  checking." 
When  asked  which  instructional  format  he  liked  better,  he  answered,  “I  would  recommend  the  video  because  you  can  see  the  actual  mouse  movement,  for  example,  to  click  on  a certain  push  button  or  menu 

Student  F spent  16.5  minutes  on  the  post  test.  He  explained  how  he  created  the  forth  slide  without  the  instructions.  "Now  without  the  video  1 am  going  a little  quicker.  Basically  when  1 use  the 
computer  1 just  do  it;  if  1 mess  up,  1 can  always  go  back.”  On  trying  new  things  and  recovering  fitjm  error,  he  said  "There  1 picked  the  frog  [an  animation  object]  and  made  the  frog  box  too  small,  but  I know 
how  to  make  a box  bigger  so  1 made  the  frog  the  size  1 want.” 

When  Student  F was  asked  to  describe  his  own  learning  style,  he  answered,  “I  like  it  being  hands-on,  and  being  able  to  see  what  I am  doing.  I don’t  like  just  to  ‘sponge  in’  and  dump  it  on  paper 

In  a careful  review  of  all  the  think -aloud  interviews  in  which  the  learners  answered  open-ended  questions  to  explain  their  learning  performances  while  watching  the  video  recording  of  their 
learning  Astound,  five  learning  patterns  emerged:  (1)  following  through  error  correction,  (2)  confirming  actions.  (3)  hand  on  mouse,  (4)  keeping  up  step-by-step  with  the  video  instruction,  and  (5)  trying  new 
steps.  Using  analysis  of  patterns  in  time  (APT),  a methodology  that  can  code  event  changes  over  time,  the  frequencies  and  the  amounts  of  time  of  the  five  learning  patterns  were  calculated.  The  data  derived 
from  the  APT  scores  indicated  two  major  findings:  (1)  The  amount  of  time  used  by  the  participants  ranged  from  20  to  87  minutes.  The  amount  of  time  spent  did  not  predict  mastery  of  the  post  test.  (2) 
‘Following  through  error  corrections’,  ‘confirming  actions’,  and  ‘trying  new  steps’  were  the  patterns  that  did  appear  to  predict  mastery,  and  the  other  two  patterns  were  indicators  of  learners'  task  engagement. 
The  findings  suggest  that  it  is  important  to  design  flexible  instruction  to  facilitate  repeated,  persistent,  and  successful  practice  in  order  to  achieve  mastery  in  a self-paced,  computer-based  learning  environment, 
regardless  of  the  amount  of  time  spent. 

Suggestions  to  Practitioners 

First  and  foremost,  the  implications  these  dynamic  learning  patterns  brought  to  the  instructional  Hesigners  and  teachers  arc: 

(1)  Ideally,  thcic  should  not  be  any  time  constraints  in  a self-paced  computer-based  learning  environment.  However,  in  classroom  settings  and  even  in  corporate  training  settings,  there  arc  time 
limitations.  As  teachers  and  instructional  designers,  we  should  try  to  mitigate  the  way  the  timed  classroom  hour  shapes  learning.  For  example,  in  a traditional  classroom  hour.  Student  H,  a master  who  took  87 
minutes  to  finish  the  task,  would  have  failed  to  master  the  task  if  she  had  had  to  stop  after  50  minutes.  We  need  to  develop  new  ways  of  structuring  the  time  of  the  class  so  that  such  failures  would  not  be  built 
into  the  classroom  framework.  Nevertheless,  the  immediate  value  of  self-paced  learning  will  be  outside  of  the  classroom  where  the  learners  or  trainees  can  determine  or  have  control  over  the  time  they  use  to 
leam  the  tasks. 

(2)  Those  learners  who  have  difficulty  with  following  through  error  correction  may  have  problems  with  a lack  of  lower- level  computer  skills.  The  present  writer  recommends  that  under  such 
circumstances  that  the  instructional  designer  needs  to  develop  a mechanism  to  detect  the  problem  quickly  and  refer  the  learner  to  simple  optional  lessons  on  the  conventions  of  the  operating  system  (c.g., 
Windows  95  in  this  study)  and  the  basic  features  of  the  application  software  program  (e.g.,  the  tools  featured  in  Astound  in  this  study). 

(3)  If  those  learners  who  choose  to  use  exclusively  the  text-only  printed  instructions  and  have  difficulty  finding  the  features,  tools,  push  buttons,  or  have  difficulty  finding  where  they  are  from 
just  a verbal  description,  then  if  space  and  format  allow,  the  related  graphical  icons  of  the  tools  and  the  appropriate  screen  captures  should  be  added  to  the  printed  instructions. 

(4)  Making  the  printed  (verbal)  and  the  video  versions  of  the  instructions  as  well  as  the  computer  with  the  software  program  available  as  options  at  the  same  lacatian  can  accommodate  most 
learners’  preference  in  terms  of  forms  of  content  and  sensory  modalities. 

(5)  For  those  learners  who  like  to  try  new  steps,  the  learning  lessons  should  be  flexible.  However,  for  those  who  do  not  like  to  explore  new  things,  more  structured  jumpstart  lessons  at  the  same 
or  at  different  levels  of  complexity  should  be  provided. 

(6)  Many  participants  in  this  study  strongly  suggested  that  more  "context  sensitive”  hints  or  notes  should  be  made  available  on  both  the  printed  and  video  instructions  because  the  just-in-time 
hints  or  notes  given  immediately  before  or  after  the  “problematic  spots”  helped  some  learners  recover  from  errors  quickly  without  getting  frustrated.  Therefore,  it  is  important  to  observe  where  in  the  lesson 
learners  tend  to  have  problems  with  something,  then  design  and  position  the  “anticipatory  fee 

Discussion 


In  this  study,  there  were  five  leamcr-demonstrated  patterns  identified  in  a self-paced  environment  in  learning  to  take  part  in  a computer  software  program.  Although  generalizations  cannot  be 
made  for  all  people  in  all  kinds  of  learning  because  of  the  relatively  small  and  selected  sample,  the  trends  found  in  the  data  do  suggest  that  a number  of  these  learning  patterns  would  be  predictive  of  mastery  in 
procedural  learning.  Future  studies  with  a larger  randomly-selected  sample  size  will  further  test  the  character  of  these  trends. 

This  present  study  used  Kolb’s  Learning  Style  Inventory  (LSI)  as  the  screening  tool  to  select  samples  because  the  LSI  is  a popular  instrumc 
weaknesses  as  a learner”  (Kolb,  1976)  and  used  widely  for  career  predictions.  Comparing  the  activities  from  the  emerging  learning  patterns  identified  in  this  study  with  Kolb’s  four  learning  abilities,  we  can 
draw  some  parallel  relations  between  them:  reading  printed  instructions  is  an  example  of  abstract  conceptualization  (AC);  watching  a video  demonstration,  an  example  of  reflective  observation  (RO),  doing  the 
steps  as  described  in  the  instructions,  an  example  of  concrete  experience  (CE);  and  trying  new  steps,  an  example  of  active  experimentation  (AE). 

Kolb’s  learning  styles  arc  generated  from  two  combination  scores  (AC  - CE  and  AE  - RO)  that  indicate  the  extent  to  which  an  individual  emphasizes  abstractness  over  concreteness  and  action 
over  reflection.  However,  the  similarities  between  Kolb’s  learning  styles  and  the  identified  learning  patterns  end  here.  One  fundamental  difference  between  these  two  is  that  Kolb  assumes  the  learning  styles 
arc  stable  personality  traits,  whereas  the  learning  patterns  identified  in  this  study  are  dynamic  in  nature.  For  example,  while  student  F was  designated  as  a converger  with  tendency  to  be  abstract  and  active, 
according  to  Kolb,  he  actually  demonstrated  in  his  learning  performance  that  he  engaged  in  all  four  learning  abilities  over  time.  Student  f was  watching  the  video  demonstration  (reflection)  and  following  each 
step  05  it  was  described  on  the  video  (concreteness).  He  also  frequently  checked  with  the  printed  instructions  to  see  if  he  did  the  steps  right  (abstracmess).  After  getting  more  familiar  with  the  computer 
program,  he  started  to  try  new  steps  that  were  not  included  in  the  instructions  (action).  Over  all.  Student  F showed  a dynamic  learning  sequence  that  involves  several  learning  patterns  that  even  he  himself  could 
not  have  predicted  or  repeated. 

In  the  present  study,  the  major  finding  is  that  regardless  of  the  amount  of  time  spent,  repeated,  persistent,  and  successful  practice  tends  to  lead  to  the  mastery  of  the  post  test.  This  finding  echoes 
the  result  found  by  Szc  (1988)  that  repeated  masteries  during  practice  strongly  affect  student  learning  achievement  and  attitude  in  an  adaptive  computer-guided  practice  environment.  Also,  it  concurs  with  the 
Academic  Learning  Time  (ALT)  research  reported  by  Berliner  (1979)  that  when  students  spend  more  time  engaging  in  a particular  curriculum  content  area,  they  are  more  likely  to  attain  higher  academic 
achievement.  It  is  important  to  note  that  the  above-mentioned  findings  strongly  support  Carroll’s  (1989)  argument  that  aptitude  is  primarily  a measure  of  time  required  to  learn.  This  argument  is  in  contrast  with 
the  classic  models  (based  upon  theories  of  intelligence)  in  which  all  learners  arc  given  the  same  amount  of  time  to  leam,  and  the  focus  is  on  differences  in  ability. 
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AuSu  aci:  Hypermedia  learning  material  often  makes  broad  use  of  links  to  information  which  goes 
beyond  the  learning  objectives.  But  instead  of  being  additionally  motivated  some  students  rather 
feel  too  much  asked  of  Within  the  Lilienthal  project  (www.pilotschool.net)  links  that  don’t  follow 
the  linear  navigation  are  labeled  with  respect  to  their  relevance  for  the  learning  objectives.  First 
evaluations  prove  that  the  students  welcome  this  assistance. 


Introduction 

“Information  overflow**  or  *‘lost  in  hyperspace”  are  keywords,  which  also  are  frequently  used  to  describe  the 
difficulties  of  students  using  hypermedia  for  learning  purposes.  Indeed  they  give  a good  impression  of  what  may 
hinder  students  from  enjoying  the  new  technical  possibilities  offered  by  the  internet  in  contrast  to  their  teachers 
expectations.  When  creating  webpages  instructors  believe  they  can  motivate  their  students  by  offering  many  links  to 
further  interesting  or  related  aspects  on  the  specific  subject  matter.  But  e.g.  many  lecturers  report,  that  rather  the 
opposite  effect  is  shown  by  their  students:  learners  tend  to  complain  about  the  number  of  webpages  they  are 
supposed  to  visit  (Dunlap  1999). 

The  problem  faced  is,  that  while  the  lecturer  wants  to  offer  the  students  the  best  chances  to  learn  the  subject  matter 
as  well  as  possible,  the  student  also  has  in  mind,  that  he  needs  to  achieve  certain  learning  objectives  within  a certain 
amount  of  time,  e.g.  to  be  prepared  on  time  for  an  examination.  Consequently  students  feel  overwhelmed  and 
frustrated  by  the  amount  of  information,  because  they  cannot  differentiate  between  more  and  less  important  links  as 
their  teachers  can.  One  possible  solution  to  this  problem  might  be  the  labeling  of  the  links  according  to  their 
relevance  for  the  required  learning  objectives,  e.g.  by  the  use  of  small  icons  in  front  of  links. 

Labeling  of  links  according  to  relevance 

This  directly  leads  to  the  first  question:  How  to  decide  on  the  relevance  of  a link?  The  answer  can  be  easy,  if  the 
creator  of  the  webpages  and  the  corresponding  examination  responsible  are  the  same  person.  But  in  many  cases  the 
learning  material  is  programmed  by  a team  whose  members  can  have  different  opinions  on  this  question.  As  this 
also  affects  the  pages’  content  in  general,  it  is  necessary,  that  the  complete  team  agrees  on  a set  of  well  formulated 
(e.g.  Mager  1978)  detailed  learning  objectives  before  the  actual  programming  takes  place.  Afterwards  the  links  can 
be  labeled  with  respect  to  these  detailed  learning  objectives. 

In  order  not  to  confuse  the  students  within  Lilienthal  it  has  been  decided  to  restrict  the  labeling  to  three  icons: 
-mandatory,  e.g.  a link  leading  to  a formula  which  the  student  must  know  by  heart; 

-recommended,  e.g.  a link  leading  to  an  interesting  example  which  illustrates  the  effects  of  the  formula  in  a certain 
context;  or 

-nice  to  know,  e.g.  a link  leading  to  an  abstract  about  how  this  formula  had  been  discovered  historically. 

Of  course  it  still  remains  the  student’s  decision  whether  he  follows  a link  and  for  how  long  he  stays  on  the 
corresponding  pages. 

Research 

Hypermedia  material  for  roughly  80  hours  of  learning  time  has  been  distributed  to  pilot  students  of  four  European 
flightschools.  The  material  had  been  programmed  after  a Delphi-process  (Linstone  & Turoff  1978)  which  lasted  one 
year,  during  which  several  subject  matter  experts  of  all  involved  flightschools  had  discussed  and  finally  agreed  on  a 
set  of  detailed  learning  objectives,  each  describing  5 minutes  of  learning  time.  On  this  basis,  the  links  within  the 
material  have  been  labeled  as  described  above.  The  number  of  links  vary  per  module  (9  DLO  in  average). 


At  this  point  in  time  (March  2000)  evaluation  of  the  various  data  (tests,  questionnaires,  protoquols)  has  just  started. 
A first  evaluation  of  208  questionnaires  on  1 1 different  modules  (average  45  minutes)  not  only  proved  that  the  big 
majority  of  students  found  the  labeling  of  links  useful  (compare  figure  1),  but  also  that  there  is  a strong  correlation 
(p  < 0.01)  to  the  statement  “the  module  is  easy  to  operate”.  A reasonable  interpretation  of  this  correlation  is  that 
those  students  who  made  use  of  the  additional  info  of  the  labeling  icons,  also  felt  they  had  a better  control  in  general. 


The  labeling  of  links  (m,  r,  n)  is  useful! 
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Figure  1 : Evaluation  of  208  student  questionnaires  on  1 1 different  modules  for  the  statement  The  labeling  of  links  is  useful! 

Nevertheless  further  investigations  are  required,  e.g.  because  in  the  pre-evaluation  (in  total  19  students)  14  out  of  all 
17  students  who  marked  “(strongly)  agree”  almost  immediately  added  “...but  still  you  click  on  every  link  anyway”. 
This  shows  a well  known  behavior  of  hypermedia  users,  who  only  have  little  navigation  experience.  The  assumption 
is,  that  the  labeling  will  have  a stronger  influence  on  the  ‘click  or  no  click’-decision  in  the  long  run. 

In  this  context,  although  in  our  case  the  pilot  students  certainly  form  a rather  homogenous  group  (typically  male,  age 
19-23,  open-minded  for  technically  related  matters,  good  marks  in  physics  and  mathematics),  it  is  still  expected,  that 
students  are  influenced  differently  by  this  labeling,  E.g. 

A students  still  click  on  every  link, 

B students  select  recommended  and  in  particular  nice-to-know  links  according  to  the  indicated  topics, 

C students  avoid  most  recommended  and  all  nice-to-know  links. 

In  addition  it  might  well  be  possible,  that  a student  changes  his  attitude  in  the  courses  process.  As  has  been 
mentioned  before,  in  particular  novices  might  rather  show  A behavior,  but  after  a while  they  are  expected  to  realize, 
that  they  need  to  work  more  efficiently  and  become  B or  C students. 

But  the  labeling  should  not  only  influence  the  decision  whether  to  click  but  also  the  navigation  on  the  following 
pages.  A student  should  be  more  likely  to  really  explore  a webpage  if  he  knows  its  content  belongs  to  the  mandatory 
level.  In  contrast  he  might  show  scanning  navigation  (Canter,  Rivers  et  al,  1985)  if  he  knows  that  he  needn’t  follow 
the  link  before  at  all.  Since  so  far  almost  no  experiences  are  reported  in  this  context,  these  are  just  reasonable 
assumptions,  which  need  to  be  investigated. 
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When  is  an  Interactive  Learning  Scenario  a Matter  of  Interface  Design? 


Christian  M.  Adriano,  Adriane  M.  Adriano,  Ivan  L.  M.  Ricarte 
State  University  of  Campinas,  Brazil 

Interaction  is  an  important  feature  a learning  activity  must  promote.  Parker  (1999)  outlines  benefits  pro- 
vided by  interaction:  “Understanding  and  long-term  internalization  can  only  be  acquired  by  communication,  recon- 
struction and  reconciliation  of  information;  interaction  enables  socialization,  promotes  motivation,  and  learning 
commitment;  interactive  processes  foster  comprehension  and  consensus  on  universal  truths’*. 

In  previous  research  (Adriano  et  al.l999),  we  implemented  a computer  supported  learning  scenario  by 
producing  content,  modeling  evaluation,  and  choosing  a proper  pedagogical  orientation.  Our  effort  resulted  in  the 
first  version  of  the  CALM  platform,  which  implements  a learning  scenario  inspired  by  the  Goal  Based  Scenario 
approach,  provides  an  intelligent  tutor,  and  hosts  material  for  learning  Java.  In  order  to  guide  further  effort  on  in- 
teraction, the  following  problem  was  posed:  When  is  an  interactive  learning  scenario  a matter  of  interface  design? 
We  propose  two  situations  in  which  developing  interactive  learning  scenarios  incurs  in  interface  design.  First, 
when  concern  fits  on  one  of  the  items  proposed  by  Marcus  (1994):  Mental  and  Navigation  Models,  Appearance, 
Interaction,  or  Metaphor.  Second,  when  the  learning  scenario,  implemented  by  the  resulting  interface  design,  must 
accomplish  complementary  definitions  for  the  concept  of  "interaction". 

Metaphors  are  fundamental  concepts  communicated  through  words,  images,  and  sound.  Mental  models 
are  data  structures,  functions,  tasks,  roles,  jobs,  and  people  in  organizations  of  work.  Navigation  is  the  process  of 
moving  through  the  mental  models.  Interaction  is  all  input-output  sequences  and  means  for  conveying  feedback. 
Appearance  relates  to  visual,  verbal  and  acoustic  information. 

The  designed  metaphors  (Adriano  et  al.  2000)  were  annotation,  document,  and  annotation-place.  The  de- 
sign of  the  interaction  item  resulted  in  three  annotating  paradigms  represented  by  specific  annotation-places: 
Bridge,  Inline,  and  Tree.  The  Bridge  place  represents  the  idea  of  annotating  as  adding  links.  The  Inline  place  sup- 
ports the  paradigm  of  writing  over  the  text  and  having  the  annotation  content  visible.  The  Tree  place  organizes 
annotations  in  a newsgroup-like  fashion.  The  mental  model  comprises  the  annotation-place  model  and  organiza- 
tion of  roles.  In  the  former,  a document  may  contain  many  annotation-places  and  an  annotation-place  may  contain 
many  annotations.  The  organization  of  roles  depends  on  each  learning  scenario  specification.  The  navigation 
model  comprises  resources  to  manipulate  metaphors.  Such  resources  correspond  to:  annotation  editing  interfaces,  a 
search  tool,  a document  composition  tool  (annotation-places  are  implemented  as  applets  within  Web  pages),  a tool 
for  repositioning  annotation-places  due  to  changes  in  the  original  Web  page,  and  a tool  to  generate  workbook-like 
compilation  of  annotations.  The  appearance  item  relates  to  types  of  annotations  and  annotation-places.  The  anno- 
tation metaphor  has  four  visually  different  types:  doubt,  suggestion,  commentary  and  mark. 

The  discussion  scenario  comprises  theme  presentation,  individual  annotation,  group  annotation,  and  dis- 
cussion summarization.  Roles  are  “moderator”  and  “commenter”.  The  theme  presentation  is  a notification  that  the 
Web  page  hosting  the  text  for  discussion  is  available  for  commenting.  After  notification,  starts  the  individual  an- 
notation phase  in  which  commenters  read  and  make  personal  annotations  on  the  text.  In  the  next  phase  com- 
menters  may  request  the  annotations  of  their  peers  and  discuss  opinions  and  acknowledgements,  which  involves 
commenting  on  each  other  annotations.  When  discussion  is  finished,  due  to  a deadline  or  a common  agreement, 
the  moderator  compiles  annotations  into  a second  text  elaborating  a summary  of  the  discussion. 

The  co-authoring  scenario  includes  goal  definition,  authoring,  reviewing,  voting,  and  consolidation.  Roles 
are  “co-author”,  “reviewer”,  and  “author”.  Co-authors  annotate  using  inline  annotation-places  and  may  define 
when  an  annotation  is  ready  to  be  seen  by  others.  Reviewers  may  comment  on  annotations  made  by  co-authors.  As 
a co-author  decides  that  an  annotation  should  become  part  of  the  underlying  text,  he/she  requires  the  author  to  con- 
solidate the  annotation.  If  the  author  agrees,  the  annotation  is  incorporated  to  the  text.  The  author  can  also  ask  for 
co-authors  to  vote  whether  an  annotation  should  be  consolidated.  An  annotation  has  states  such  as  ready,  removed, 
in  consolidation,  being  voted,  and  not  available. 

Some  interaction  definitions  were  collected  from  Vigotsky,  Habermas,  de  Certeau,  and  Chartier.  Vigotsky 
(1978)  deals  with  the  cultural  dimension  of  learning.  Apprenticeship  is  result  of  interactions  among  historical 
subjects  and  the  world.  Signs  and  symbols  are  the  condition  for  these  interactions.  People,  instruments  and  signs 
mediate  relations  between  a subject  and  physical/social  environment.  Signs  correspond  to  a representation  of  real- 
ity and  refer  to  remote  times  and  spaces.  Interaction  is  conceptualized  as  relations  among  historical  individuals. 
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signs,  and  instruments.  Such  definition  is  satisfied  by  interfaces  that  relate  individuals  as  role  players,  signs  as 
metaphor  representations  and  instruments  as  tools  of  the  navigation  model. 

Certeau  (1974)  states  that  interaction  is  based  on  a metaphor  of  landmark  and  on  distinctions  between 
place  and  space.  In  short,  place  is  static  and  permanent,  while  space  is  dynamic  and  transient.  Landmarks  delimit 
regions  (spaces  or  places),  being  also  classified  as  dynamic  or  static.  Static  landmarks  are  frontiers,  like  rivers  and 
fences,  delimitating  places.  Dynamic  landmarks  are  bridges,  like  doors  and  windows,  delimitating  spaces.  Bridge- 
like landmarks  represent  encounters  of  action.  Moreover,  there  will  be  so  many  bridges  as  the  number  of  ‘‘inter- 
actions”. Interaction  is  defined  as  a bridge  separating  spaces  of  action.  Such  concern  inspired  the  design  of  anno- 
tation-places, whose  types  provide  different  bridging  between  the  document  space  and  the  annotation  space. 

As  quoted  by  Prestes  (1997),  Habermas  argues  that  the  communicative  action  has  a kind  of  anchor  con- 
stituting background  knowledge.  Such  knowledge  fosters  interpretations  for  participants  of  an  interaction.  The 
background  knowledge  relates  to  a woiiu  of  life  {Lebensweii),  the  horizon  on  which  we  move.  It  is  pre-scientific, 
intuitive,  not  expressed  and  not  questionable.  It  constitutes  itself  not  only  by  cultural  convictions,  but  also  by  insti- 
tutional organizations  and  structures  of  personality.  Habermas  uses  the  daily  communicative  praxis  to  guarantee 
the  symbolic  reproduction  of  three  spheres;  culture,  society,  and  personality.  Culture  refers  to  knowledge  acervus, 
over  which  we  formulate  interpretations  to  understand  the  world.  Society  refers  to  legitimated  organizations  that 
regulate  our  bindings  with  social  groups.  Personality  refers  to  competencies  that  enable  subjects,  capable  of  lan- 
guage and  action,  to  take  part  in  an  understanding  process.  The  following  citation  of  Habermas  clarifies  these  three 
aspects:  "In  respect  to  the  functional  aspect  of  understanding,  the  communicative  action  promotes  tradition  and 
cultural  renovation.  In  respect  to  coordination  aspects,  the  communicative  action  promotes  social  integration  and 
creation  of  solidarity.  In  respect  to  socialization,  the  communicative  action  promotes  formation  of  individual  iden- 
tities". Interaction  is  a space  of  conflict  between  a pre-comprehension  of  world  of  life  and  the  communicative  ac- 
tion. Therefore,  when  and  where  interaction  happens,  the  stability  of  the  world  of  life  is  disturbed  by  arguments  of 
the  communicative  action.  Annotation  interfaces  enabling  discussion  was  a mean  to  accomplish  such  vision. 

Following  a historical  point-of-view,  Chartier  (1997)  provides  a study  of  how  mankind  has  been  interact- 
ing with  and  by  means  of  the  written  letter.  Medium  has  changed  from  papyrus,  to  codex,  to  Gutenberg  press,  and, 
nowadays,  to  digital  text.  Each  medium  required  a proper  interaction  paradigm  suggesting  that  interaction  be  also 
defined  by  the  enabling  technological  support.  In  other  words,  medium  defines  interaction.  Such  concept  for  inter- 
action was  accomplished  by  supporting  audio  and  written  media  for  annotation. 

In  addition  to  the  five  components  proposed  in  Marcus  (1994),  we  suggest  a careful  study  of  the  interac- 
tion concept.  The  presented  discussion  stems  from  project  meetings  between  educators  and  software  developers.  A 
recurring  problem  faced  was  the  lack  of  consensus  on  concepts,  accompanied  by  concerns  of  presupposing  things 
that  needed  better  understanding  before  implementation.  Interaction  conceptualization  is  one  these  concerns.  We 
hope  the  ongoing  research  will  collect  further  insights  to  the  design  of  interactive  learning  scenarios  as  a coopera- 
tive entrepreneurship  of  educational  and  computer  scientists. 
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Abstract 

Although  there  are  strong  reasons  for  purchase  of  an  off-the-shelf  system  for  online 
course  delivery  (including  the  competitive  advantages  of  reliability,  the  comfort  of  a 
large  client  base  and  scalability),  many  universities  develop  a system  themselves.  These 
home-grown  systems  are  often  more  responsive  to  specific  needs.  When  James  Cook 
University  undertook  to  investigate  available  systems  for  university  wide  online  course 
delivery  it  was  important  to  explore  costs  of  building  a system  using  university  resources. 
To  this  end  a system  model  was  designed  as  an  interactive  diagram  reflecting  each  part  of 
the  system  in  prototype  http://www.tsd.icu.edu.au/icudevelop/icuproto/icuonline 
The  model  is  both  an  innovative  example  of  organisational  design  method  and  a useful 
means  of  testing  user  pathways  through  the  system,  (use-case),  which  can  assist  costing 
and  project  management  of  the  development  of  such  a system. 


As  part  of  the  search  for  the  most  suitable  system  for  use  at  JCU  we  set  out  to  determine  the  costs  of 
building  a system  using  university  resources: 

• there  was  a high  level  of  interest  in  the  JCU  community; 

• the  possibility  of  economies  of  production; 

• a collaborative  venture  of  this  kind  with  the  potential  for  cross-institution  partnerships  could  be  of 
value  to  the  university  in  the  future. 

To  this  end  a system  model  was  designed  as  an  interactive  diagram  reflecting  each  part  of  the  system,  in 
prototype  (at  http://www.tsd.icu.edu.au/icudevelop/icuproto/icuonliney 

The  model  is  both  an  innovative  example  of  organisational  design  method  and  a useful  means  of  testing 
user  pathways  through  the  system,  (use-case),  which  can  assist  costing  and  project  management  of  the 
development  of  such  a system. 

Discussions  were  held  with  the  schools  of  IT  and  Computing  Science.  The  interactive  system  model  was 
felt  to  be  helpful  in  this  process  in  that  it  provided,  by  visible  and  testable  means,  a common  understanding 
of  the  scope  and  direction  of  the  exercise.  Costing  was  based  in  staged  development  from  base  level, 
adding  functionality  as  we  saw  a need. 

There  was  found  to  be  no  case  for  developing  DIY  for  immediate  implementation: 

• the  real  costs  of  DIY  infrastructure  are  high  for  a one-off  development  which  is  not  amortised  over 
other  products; 

• time  of  development  has  to  be  compressed  to  keep  pace  with  user  needs  and  technical  innovation:  to 
compress  time  requires  commitment  of  personnel  which  increases  costs; 

• when  a university  implements  its  own  system,  support  for  users  becomes  an  internal  budget  decision 
not  externally  (market)  driven:  support  is  therefore  subject  to  internal  budgetary  pressures  rather  than 
the  wider  driving  client  base. 
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Validating  a Theory-Based  Design  for  Online  Instruction:  Aligning  Tool 

Use  and  Learning  Outcomes 


Abstract:  This  paper  provides  a theoretical  framework  for  the  design  of  a successful 
web-based  course  in  Personal  Finance.  Using  an  open  learning  model  based  on 
constructivist  principles  that  draw  from  the  literature  in  cognition  and  the  social 
sciences,  this  paper  sketches  an  instructional  design  model  and  specifies  online  tools 
that  mediate  between  course  objectives  and  desired  learning  outcomes.  In  order  to 
validate  the  model,  data  generated  from  tool  use  and  from  student  performance  will  be 
analyzed  using  confirmatory  factor  analysis. 


Mixed  Messages  with  Multimedia 

A growing  body  of  research  is  exploring  the  learning  effects  of  multimedia  designed  environments.  Some 
of  this  research  has  attempted  to  incorporate  educational  concepts  in  the  design,  process  and 
implementation  of  course  production  in  technology-rich  platforms.  However,  the  results  are  mixed. 
According  to  The  No  Significant  Difference  Phenomenon  (Russell,  1999)  which  is  based  on  355  research 
reports,  summaries  and  papers,  the  conclusions  mainly  indicate  that  the  learning  outcomes  of  students 
using  technology  enriched  courseware  are  similar  to  those  of  students  who  participate  in  conventional 
classroom  instruction.  Yet,  universities  are  allocating  substantial  resources  towards  the  development  of 
distance  education  courses  and  programs  (Brahler,  Peterson  et  al.,  1999)  to  increase  access  to  learning, 
with  the  hope  that  the  quality  of  education  will  be  damaged  little,  if  at  all  (Ehrmann,  1 999)  Questions  and 
concerns  relating  to  technology  integration  are  being  raised,  but  as  yet  have  had  little  systematic  impact 
on  the  policies  that  guide  educators,  administrators  and  practitioners  (Merisotis  and  Phipps,  1999;  Welte, 
1997)).  Faculty  realize  that  there  are  certain  benefits  of  technology-based  distributed  learning 
environments,  but  put  up  considerable  resistance  to  the  development  of  such  environments  (Rickard, 
1999). 

Mixing  Cognitive  and  Social  Science  Approaches 

This  working  paper  presents  a theoiy-based  design  of  an  undergraduate  web-based  course  in  Personal 
Finance  with  the  aim  of  addressing  some  of  the  issues  addressed  above  in  the  context  of  better 
understanding  student  learning.  Concerns  relating  to  technology  integration  in  educational  practice,  and 
technology's  intersection  with  learning  theoiy  are  addressed  in  a design  framework  that  aligns  learning 
objectives,  course  activities  and  assessment  methods.  This  alignment  simultaneously  considers  the 
selection  of  online  learning  tools  that  mediate  between  design  components  for  Personal  Finance.  The 
selection  of  learning  tools  also  plays  an  important  role  in  terms  of  how  they  will  be  meaningfully  used  by 
students,  and  how  they  will  affect  student  performance. 

Web-based  instruction  that  is  designed  on  accepted  theoretical  principles  in  cognitive  psychology  opens 
new  doors  to  designers,  learners  and  practitioners.  This  is  simply  because  we  know  very  little  about  best 
practices  arising  from  the  design  of  courseware  that  primarily  uses  this  medium.  However,  in  contrast, 
there  is  an  abundance  of  theoretical  studies  in  the  cognition  literature  that  have  increased  our 
understanding  of  competence  in  understanding  a given  discipline  (See  for  example,  Anderson,  Corbett  et 
al.,  1995;  Ausubel,  1968;  Hatano,  1996;  Means,  1993;  VanLehn,  1996;  and  Winn,  1990).  Similarly, 
studies  from  the  literature  in  social  sciences  have  proposed  interactive  socio-cultural  contexts  as  a basis  for 
understanding  how  learning  can  be  constructed,  scaffolded  and  facilitated  (See  for  example,  Bandura, 
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1986;  Brown,  Collins  et  al.,  1989;  Derry  and  Lesgold,  1996;  Lave  and  Wenger,  1991;  Moore,  1994; 
Shuell,  1993;  and  Richey,  1994.  In  his  1999  AERA  Presidential  Address,  Alan  Schoenfeld  describes  the 
split  between  cognitive  and  social  science  research  with  references  to  studies  in  cognition  that  excel  in 
fine-grained  data  analysis  but  suffer  from  tunnel  vision  and  points  to  as  many  sociological  studies  that  are 
rich  in  scope  but  are  weak  with  respect  to  validity  and  reliability  constructs.  His  suggestion  to  bridge  the 
gap  was  to  “build  something  to  see  if  it  works  and  then  study  the  hell  out  of  it”. 

This  study  seeks  to  validate  a non-traditional  design  based  on  theoretical  findings  from  the  cognitive  and 
social  sciences  literature.  The  world- wide- web  is  an  obvious  choice  as  the  platform  for  building  a design 
prototype  because  learners  recognize  that  it  is  more  accessible  and  convenient  than  traditional 
opportunities  afforded  in  the  classroom.  Combining  the  web  platform  and  a theoretical  basis  into  a 
prototype  design  becomes  meaningful  when  specific  learning  goals  arc  specified  with  evidence  to  link 
these  goals  with  performance  outcomes  that  are  assessed.  These  links  are  critically  explored  and  are  the 
focus  of  this  study. 

Challenges  in  the  Design  Process 

The  term  technology  is  used  broadly  to  refer  not  only  to  tools  but  also  processes  and  is  closely  identified 
with  the  Greek  form  techne  (art,  craft,  or  skill),  which  was  closely  associated  with  episteme  (systematic  or 
scientific  knowledge).  Its  educational  meaning  is  historically  derived,  bound  by  specific  psychological 
theories  and  philosophical  underpinnings  (Saettler,  1990).  Visions  of  technology  look  very  different 
depending  on  the  explicit  theories  of  learning  that  guide  their  design  and  implementation.  According  to 
The  Cognition  and  Technology  Group  at  Vanderbilt  (1996),  the  challenge  for  instructional  design  based 
on  significant  technology  integration  is  that  learning  theory,  educational  practice  and  technology  interact 
with  each  other.  To  understand  the  role  of  technology,  one  must  examine  a simultaneous  intersection  with 
student  learning  and  educational  practice.  This  challenge  is  increased  since  all  three  areas  have  seen 
dramatic  change  in  the  past  ten  years.  Thus,  changes  in  theories  of  learning  affect  uses  of  technology,  but 
new  technologies  such  as  the  world  wide  web  also  make  new  kinds  of  interactions  possible  and  hence 
affect  theories  of  learning  (Kozma,  1994;  Saloman,  1993).  Knowledge  acquisition  in  constructivist 
environments  engages  learners  in  drawing  meaning  from  multiple  perspectives  and  through  collaborative 
efforts.  Engagement  in  real  world  or  authentic  tasks  provides  a context  for  the  learner  to  construct 
meaning  from  his/her  experiences.  Thus,  learning  is  situated  in  a community  of  learners  who  construct 
personal  meaning  through  dialogue  and  socialization  (See  Brown,  Collins  et  al.,  1989;  Jonassen,  1998;  & 
Wilson,  1995). 

Sketching  the  Design  Components 

Given  these  perspectives,  different  views  of  how  humans  learn  and  how  designers  approach  instruction 
are  incorporated  under  the  umbrella  of  knowledge  acquisition  and  knowledge  construction.  In  addition, 
aspects  of  situated  cognition,  which  emphasizes  social  and  ecological  interactions  is  also  included.  Parts 
of  the  design  relies  on  an  information  processing  view  emphasizing  course  content  that  hierarchically 
structures  the  sequence  of  information.  Using  Hannafin,  Land  et  al.’s  (1999)  open  learning  model  which 
emphasizes  diversity  and  choice,  course  material  was  selected  to  reflect  a variety  of  print-based, 
multimedia  and  interactive  web-based  tools  that  could  best  enable  the  learner  to  attain  the  learning  goals 
set  for  the  course.  The  importance  of  constructive  alignment  between  different  design  components  is 
emphasised  by  Biggs  (1999);  they  become  inextricably  linked,  so  that  the  course  becomes  greater  than  the 
sum  of  its  parts.  These  components  include  a textbook^  short  vignettes  with  practionners  who  introduce 
each  topic,  simulations  that  forecast  your  net  worth,  or  your  tax  bill,  interactive  practice  tests  including 
pre-tests,  review  exercises,  short  cases,  etc  and  communication  software  that  allows  for  threaded 
discussion,  chats,  critiquing,  posting  and  other  activities  consistent  with  best  practices  in  effective 
learning  communities.  Technology  has  been  nested  within  a constructivist  framework  to  raise  the 
intellectual,  social  and  ethical  standards  of  learners,  by  empowering  them  to  question,  participate,  and 
practice  authentic  issues  in  personal  finance.  Jonassen,  Davidson  et  al.  (1995)  summarize  this 
approach:**...  learning  is  necessarily  a social  dialogical  process  in  which  the  communities  of  practitioners 
socially  negotiate  the  meaning  of  phenomena”  (p.9).In  addition,  feedback  opportunities  are  provided  by 
experts  (the  instructor  as  well  as  practionners)  to  ensure  accurate  knowledge  acquisition.  Finally, 
communication  strategies  are  designed  to  facilitate  elaboration  of  content  and  solicitation  of  responses  that 


reflect  what  students  are  thinking.  Here,  the  tools  employed  include  interactive  tests,  e-mail  and  bulletin 
boards,  as  well  as  audio  conferencing. 

Data  reflecting  student  use  of  designed  tools  and  their  performance  on  assigned  tasks  will  be  collected  to 
confirm  the  validity  of  the  design  model.  Evidence  of  learning  using  Bloom’s  (1956)  taxonomy  will  reveal 
the  general  factors  associated  with  the  model.  Path  analysis  will  be  used  to  confirm  the  relationship 
amongst  the  design  components  as  they  fit  within  the  instructional  design  model. 
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Abstract:  Multimedia  instruction  must  follow  instructional  design  principles  that  are 
based  on  learning  theories.  This  session  will  share  an  effective  model  for  designing 
learning  materials  and  will  describe  the  strategies  that  should  be  used  when  designing 
multi-media  learning  materials. 


Pre-instructional  Activities 

Before  teaching  new  materials  to  students,  the  lesson  must  overview  the  materials  so  that  students  can  form 
a mental  set  for  the  materials  and  become  motivated  to  learn  the  materials.  The  different  types  of  pre- 
instructional  activities  include  content  map,  advance  organizer,  pre-test,  learning  objectives,  and  rationale. 

Prepare  Learning  Activities 

Conventional  instructional  methods  tend  to  use  passive  instructional  strategies.  It  is  important  to  use 
strategies  that  will  keep  students  active  so  that  they  will  be  motivated  and  will  process  the  information  in 
greater  depth.  To  encourage  active  learning,  it  is  important  to  use  discovery  strategies  so  that  students  can 
find  out  things  for  themselves.  Instead  of  teaching  a large  amount  of  information  to  students,  we  should 
give  students  the  task  and  then  guide  them  to  find  and  apply  the  information.  Multi-media  and  on-line 
instruction  are  excellent  methods  to  promote  active  learning. 


Elaborative  Strategies 

To  make  the  instructional  process  more  effective  and  efficient,  the  lesson  must  provide  activities  for 
students  to  process  the  content  at  a deep  level.  The  deeper  the  information  is  processed,  the  better  it  is 
learned  or  remembered.  As  a result,  the  instruction  should  contain  elaborative  strategies  to  help  facilitate 
deep  processing.  Some  common  elaborative  strategies  are  asking  questions  during  the  instruction,  ask 
students  to  summarize  segments  of  the  instruction,  requesting  students  to  make  their  own  notes,  and  asking 
students  to  underline  or  highlight  important  information.  The  elaboration  process  facilitates  efficient 
storage  in  long-term  memory.  This  is  important  since  the  ease  of  retrieval  will  depend  on  the  way  the 
information  is  stored  in  long-term  memory. 

It  is  known  that  when  new  learning  materials  are  encountered,  they  are  attended  to  and  transfer  to  short- 
term memory  storage.  After  entry  into  short-term  memory,  the  information  must  be  transferred  to  long- 
term memory  immediately  since  short-term  memory  has  limited  capacity.  As  a result,  students  should  be 
given  the  opportunity  to  elaborate  on  small  segments  of  instruction  so  that  the  materials  can  be  transferred 
to  long-term  memory  immediately. 


Presentation  Strategies 
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Most  instruction  is  done  in  an  expository  fashion  where  the  learning  material  is  presented  in  its  final  form 
to  students.  The  most  common  expository  methods  are  rote,  inductive,  and  deductive  learning. 


The  rote  method  consists  of  presenting  the  materials  to  students  and  then  asking  them  to  recall  the  materials 
as  presented.  For  example,  the  lesson  may  present  a rule  and  then  ask  students  to  recall  it.  The  deductive 
method  occurs  when  students  are  given  a rule  followed  by  examples  to  explain  the  rule.  The  inductive 
method  involves  giving  examples  on  a certain  rule  followed  by  the  rule.  However,  to  make  learning  more 
efficient  and  effective,  discovery  strategies  must  be  used.  With  the  introduction  of  new  instructional 
technologies  and  multi-media,  it  is  possible  to  design  instruction  that  contains  discovery  strategies. 

Practice  Exercises 

Another  important  component  of  instruction  is  practice  with  feedback.  After  the  presentation  of  the 
materials,  students  should  be  given  the  opportunity  to  practice  what  they  have  learned  so  that  they  can  get 
feedback  on  how  they  are  performing  and  at  the  same  time  prepare  them  for  the  final  test  in  the  lesson.  In 
addition,  the  practice  test  acts  as  a good  learning  strategy  since  test  questions  usually  consist  of  general 
ideas  which  act  as  cues  to  recall  specific  details  and  skills. 

Summarize  the  Lesson 

After  the  lesson  is  presented  to  students,  they  should  be  given  a summary  so  that  they  can  get  a sense  of 
closure  for  the  lesson  and  gives  a general  review  of  the  lesson. 
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Abstract:  Training  preservice  education  students  on  the  Internet  requires  the  course  to  be  more 
structured  than  would  be  required  to  teach  the  same  course  without  an  electronic  component 
(Gillette,  1996).  Education  students  have  consistently  expressed  the  need  to  see  technology  used 
by  their  university  instructors  if  they  are  to  use  these  tools  effectively  in  their  own  teaching  (Larson 
& Clift,  1996).  Due  to  both  a lack  of  suitable  instruments  and  the  importance  of  being  able  to  assess 
web-course  effectiveness,  the  authors  developed  the  Web-Based  Survey  of  Instructional 
Effectiveness  (WebSIE)  to  meet  this  need.  Forty-one  students  enrolled  in  a web-based  course  were 
given  the  WeBSIE  prior  to  and  after  the  completion  of  the  course.  Pretest  to  posttest  comparisons 
showed  students  improved  in  their  self-reported  web-based  skills  and  attitudes  in  the  use  of  the 
internet,  integration  of  the  internet  into  instruction,  general  computer  use/attitude  and  using  a 
computer  for  communication.  Future  studies  need  to  replicate  these  findings  and  investigate  the 
effects  of  the  integration  of  technology  into  more  and  varied  teacher  training  courses. 


The  effective  use  of  technology  includes  rethinking  traditional  forms  of  instruction.  Instructors  teaching  online  need 
to  maintain  the  perspective  that  we  learn  best  “with”  technology  rather  than  “from”  it  (Johnson,  1996).  This 
perspective  allows  instructors  to  engage  students  in  active  (Copley,  1992;  Reeves,  1996;  Yakimovicz  & Murphy, 
1995),  authentic  (Copley,  1992),  challenging  tasks  (Copley,  1992;  Reeves,  1996),  and  collaborative  efforts  (Reeves, 
1996;  Yakimovicz  & Murphy,  1995). 

A computer-specific  introductory  course  is  usually  insufficient  in  modeling  for  preservice  teachers  the  use  of 
technology  as  a tool  for  teaching  and  learning  (Topp  et  al.,  1994).  Education  professors  from  various  disciplines 
must  begin  to  use  technology  as  a cognitive  tool  to  mediate  and  transform  the  training  experiences  of  teacher 
education  students  (Larson  & Clift,  1996). 

Instructional  Design 

Training  preservice  education  students  on  the  Internet  requires  the  course  to  be  more  structured  than  would  be 
required  to  teach  the  same  course  without  an  electronic  component  (Gillette,  1996).  The  structure  that  is  embedded 
and  remains  consistent  throughout  each  unit  of  study.  The  structure  of  an  introductory  special  education  course 
taught  online  will  be  elaborated  on  in  this  section  by  clustering  the  components  into  categories  and  providing  a brief 
description  of  each. 

1 . Instructional  Design  Framework 

(a)  Each  unit  of  study  is  divided  into  six  instructional  steps. 

(b)  Each  of  the  six  steps  are  divided  into  instructional  components. 

(c)  Reference  numbers  are  assigned  to  each  of  the  instructional  components.  The  reference  numbers 
are  depicted  using  three  digits  separated  by  decimals  (e.g.,  2.3.4  represents  Unit 
2. Step3. Activity 4).  The  reference  numbers  are  used  when  discussing  activities  via  e-mail, 
newsgroup,  and/or  chat. 

(d)  Buttons  are  located  at  the  bottom  of  each  of  the  six  instructional  step  pages  so  that  students  can  go 
to  the  “next  step”  or  return  to  the  “unit  index”,  which  provides  links  to  all  six  steps. 

2.  Curriculum  Development 

(a)  The  curriculum  is  built  upon  the  principles  of  active  learning,  cooperative  groups,  peer-mediated 
instruction,  and  content  experts 
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(i)  Active  learning  is  integrated  into  the  course  through  the  use  of  interactive  forms  and  case 
studies  found  on  the  Internet. 

(ii)  The  class  is  divided  into  cooperative  groups  of  two  based  on  where  students  live  and  their 
educational  major, 

(iii)  Peer-mediated  instruction  is  used  through  the  creation  of  student  homepages  and  peer 
expertise  pages.  These  pages  allow  students  to  share  their  work  and  ideas  with  each  other, 

(iv)  Content  experts  are  utilized  by  incorporating  parents  of  students  with  disabilities  and 
personnel  in  special  education  field  in  local  public  schools  into  the  class  discussions  (e,g,, 
e-mail,  chat,  and  newsgroup), 

(b)  All  activities  taught  in  the  traditional  course  on  campus  are  incorporated  into  this  online  course 
through  the  use  of  e-mail,  chat,  newsgroup,  electronic  handouts,  forms,  and  web  sites. 

Teaching  in  this  virtual  classroom  in  comparison  with  teaching  in  the  traditional  classroom  requires  much  more 
learning  and  much  less  teaching  (Peterson  & Facemyer,  1996),  The  dynamic  learning  environment  available  online, 
which  is  expanding  at  exponential  rates,  provides  opportunities  for  students  to  become  explorers  and  discoverers 
(Gates,  1996), 

Education  students  have  consistently  expressed  the  need  to  see  technology  used  by  their  university  instructors  if  they 
are  to  use  these  tools  effectively  in  their  own  teaching  (Larson  & Clift,  1996),  All  students  enrolled  in  this  online 
course  had  completed  at  least  one  required  computer-specific  course  prior  to  the  beginning  of  this  online  course.  The 
development  of  a survey  of  instructional  effectiveness  was  developed  by  the  authors  to  measure  the  effects  of 
learning  educational  content  through  the  use  of  web-based  technology. 

Web-Based  Survey  of  Instructional  Effectiveness 

Due  to  both  a lack  of  suitable  instruments  and  the  importance  of  being  able  to  assess  web-course  effectiveness,  the 
authors  developed  the  Web-Based  Survey  of  Instructional  Effectiveness  (WebSIE)  to  meet  this  need.  Expert  opinion 
from  individuals  with  both  academic  credentials  (Ed.D,  or  Ph,D,)  and  web-based  course  experience  were  sought  to 
develop  an  original  pool  of  58  items.  This  pool  of  items  was  subjected  to  factor  analysis.  A five-factor  model  was 
adopted  retaining  30  items:  Use  of  the  Internet  (8  items);  Integration  of  the  Internet  into  Instruction  (8  items); 
General  Computer  Use/Attitude  (4  items);  Using  a Computer  for  Communication  (5  items);  and  web-based  Course 
Effectiveness  (5  items). 

Using  this  30  item  survey  authors  administered  313  to  preservice  education  students  and  practicing  teachers. 
Cronbach’s  alpha  and  split-half  reliabilities  were  calculated  for  the  five  scales.  Alpha  for  the  whole  30  item 
instrument  was  .803,  and  the  five  scales  were  Use  = .557;  Instruction  = .531;  Computer  Use/Attitude  = ,415; 
Communication  = .653;  and  Course  Effectiveness  = .419,  Split-half  reliabilities  for  the  five  scales  were  Use  = ,762; 
Instruction  = .750;  Computer  Use/Attitude  = .726;  Communication  = .823;  and  Course  Effectiveness  = .672. 

Forty-one  students  enrolled  in  a web-based  course  were  given  the  WeBSIE  prior  to  and  after  the  completion  of  the 
course.  Single-sample  Dependent  T-tests  were  performed  on  all  five  scales.  An  alpha  level  of  .05  was  used  for  all 
statistical  tests.  Four  of  the  five  scales  were  significant:  Use,  t4i  = 4.873,  p_<  .001;  Instruction,  t4j  = 4,143,  p_<  .001; 
Computer  Use/Attitude,  t4]  = 3.797,  p_<  .001 ; and  Communication,  t4i  = 2.401,  p_<  .02. 

Pretest  to  posttest  comparisons  showed  students  improved  in  their  self-reported  web-based  skills  and  attitudes  in  the 
use  of  the  internet,  integration  of  the  internet  into  instruction,  general  computer  use/attitude  and  using  a computer  for 
communication.  These  statistically  significant  gains  in  web-based  skills  and  attitudes  were  noted  after  one  education 
methods  course  that  modeled  the  use  of  the  Internet  in  instruction.  Future  studies  need  to  replicate  these  findings 
and  investigate  the  effects  of  the  integration  of  technology  into  more  and  varied  teacher  training  courses.  Additional 
effects  of  the  integration  of  technology  into  teacher  training  programs  needs  be  studied  after  these  preservice 
teachers  become  practicing  teachers,  focusing  on  how  often  and  in  what  ways  they  integrate  technology  into  their 
own  teaching. 
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Abstract:  This  paper  reports  the  findings  of  a study  conducted  with  an  experimental 
undergraduate-level  class.  It  investigated  the  differences  between  learners  studying  with  the 
teacher  face-to-face  and  learners  studying  at  distance  in  terms  of  three  dimensions;  © learners' 
sense  of  distance,  ® interaction,  and  (D  achievement. 


Introduction 

One  of  the  most  important  innovations  resulting  from  the  remarkable  advances  in  communication  and 
Internet  technologies  is  ^face-to-face  at  distance  education ',  which  is  forcing  us  to  adapt  new  definitions,  theories 
and  practices.  Physical  distance  is  no  longer  the  limiting  factor;  psychological  distance”  is.  Drawing  on 
Moore’s  work  redefining  “distance”,  “interaction”  and  “transactional  distance”  (Sewart,  1988  & Moore  1991), 
the  concept  of  psychological  distance  (PSD)  refers  to  how  close  the  participants  within  the  educational 
environment  feel  to  each  other  in  terms  of  emotional  comfort  and  satisfaction. 

As  Holmberg  (1988)  observes  “real  learning  is  primarily  an  individual  activity  and  attained  through  an 
internalization  process;”  thus,  learning  is  the  product  of  internal  interaction  between  the  learner’s  previous 
experiences  and  new  information  obtained  from  external  interaction  with  the  other  components  of  the  educational 
situation,  including  contents/media,  other  learners  and  the  teacher/instructor/coach. 

The  present  study  focuses  on  this  interaction  within  the  educational  environment,  in  terms  of  the  learners' 
sense  of  distance'"'  (SoD)*,  as  a direct  indicator  of  PSD.  Specifically,  the  main  objective  of  the  experiment  is  to 
examine  the  relationships  that  exist  between:  a)  “the  learner’s  sense  of  distance”,  b)  “interaction  within  the 
educational  environment”  and  c)  “learners’  achievement”. 

The  Study 

Twenty-two  (only  the  data  from  19  participants  was  usable)  Japanese  undergraduate-student  volunteers, 
from  a women’s  university  attended  four  seminars  entitled  “Introduction  to  Public  Speaking”  during  spring, 
2000.  The  participants  were  divided  into  two  equal  groups,  matched  for  age  and  scores  for  English  and  Japanese 
language;  a “/oca/  group"  (LG),  which  studied  with  the  teacher  face-to-face  and  a ""distant  group"  (DG),  which 
studied  with  the  same  teacher  at  distance.  An  ISDN-based  TV  conferencing  system  was  used  to  mediate 
communication  between  the  two  groups,  so  that  they  can  study  in  different  places  at  the  same  time.  Achievement 
was  evaluated  by  pre-tests  and  post-tests.  The  participants  were  asked  to  evaluate  their  “sense  of  distance”  from 
others  (i.e.,  the  teacher,  other  learners  in  the  same  group  and  other  learners  in  the  other  group)  on  a 5-point  scale, 
with  1 indicating  very  close  and  5 very  far.  The  participants  also  recorded  both  how  often  they  actually  interacted 
with  others,  and  how  often  they  wanted  to,  as  an  indication  of  the  interaction  within  the  educational  environment. 
The  classes,  as  well  as  short  interviews  after  each  class  to  regarding  technical  aspects  such  as  sound  and  picture 
quality,  were  also  videotaped  for  further  analysis  and  to  improve  the  seminar  contents. 


^ Sense  of  distance  is  contrasted  with  “real  distance,”  as  a measure  of  the  distance  learners  feel  to  exist  between  themselves 
and  other  components  of  the  educational  setting,  irrespective  of  the  actual  distance. 
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Findings 


The  differences  between  pre-test  scores  concerning  the  participant’s  knowledge  of  public  speaking  for  the 
two  groups  were  not  significant  (average  scores  were  4.5  and  5.22  for  the  LG  and  DG  conditions  respectively). 
The  scores  for  the  post-test  were  16.4  for  the  LG  and  18.66  for  the  DG,  indicating  significant  improvements  in 
knowledge  concerning  public  speaking  for  both  groups  (p  < 0.0001).  The  average  scores  for  both  groups  also 
indicate  that  there  were  no  significant  differences  between  the  two  groups  in  terms  of  achievement  at  the  end  of 
the  experiment. 

In  terms  of  learners’  SoD  to  the  teacher,  the  average  evaluation  rating  for  the  LG  condition  fell  from  3.166 
after  the  first  class  to  1.7  after  the  last  class.  In  the  DG  condition,  this  fell  from  4.7  to  3.00.  Although  analyses 
(Wilcoxon  Signed  Ranks  tests)  indicated  that  these  changes  were  significant  (p  <0.025  and  0.015  for  LG  and  DG 
respectively),  further  analyses  (Mann- Wiiimey  U test)  also  indicated  that  there  were  significant  differences 
between  the  two  groups  after  both  the  first  and  last  classes.  In  other  words,  although  the  learners’  sense  of 
distance  to  their  teacher  declined  significantly  for  both  groups,  those  in  the  LG  condition  felt  much  closer  to  the 
teacher  then  those  in  the  DG.  Reviewing  the  videos  for  both  sites,  we  found  that  although  the  teacher  almost  had 
the  same  amount  of  interaction  with  both  groups  during  the  class,  he  did  spend  more  time  talking  and  explaining 
to  the  LG  both  before  and  after  the  classes,  and  that  he  was  able  to  give  more  individualized  attention  to  the 
learners  in  the  LG  condition,  which  may  have  contributed  in  making  them  feel  closer  to  him. 

The  data  indicated  no  significant  changes  in  the  learners’  sense  of  distance  to  other  learners  in  the  same 
groups  at  both  sites,  which  might  be  due  to  the  fact  that  the  students  came  from  the  same  university.  Examining 
the  learners’  sense  of  distance  to  the  learners  in  the  other  group,  the  evaluation  rating  fell  from  4.62  after  the  first 
class  to  3.00  after  the  last  class  for  the  DG  condition  (significant  at  4.6%  level).  However,  there  were  no 
significant  changes  in  the  SoD  for  learners  in  the  LG  condition.  This  might  be  due  to  the  initial  setting  at  the  LG 
site,  which  did  not  allow  learners  at  the  DG  site  to  see  their  counterparts  at  the  LG  site  during  the  first  class, 
which  was  adjusted  for  later  classes. 

The  correlation  between  the  learners’  SoD  to  their  teacher  and  the  records  of  interactions  was  -0.287  (sig. 
at  the  .05  level).  Although  no  significant  correlation  was  found  between  the  learners’  SoD  to  their  teacher  and 
achievement,  the  learners’  SoD  was  highly  correlated  (0.636  sign,  at  the  .01  level)  to  how  important  the  learners 
regarded  the  contents,  and  how  much  they  felt  they  had  learnt  (0.496  sign,  at . 1 level). 

Conclusions 

The  results  of  this  study  suggest  that  it  might  be  very  difficult  to  acquire  the  same  quality  of  interaction 
with  physically-distant  learners,  and  that  it,  therefore,  takes  more  effort  in  order  to  reduce  their  SoD  compared  to 
learners  studying  face-to-face.  Despite  this,  however,  the  more  learners  interact  with  their  teachers  the  closer  they 
feel  to  them  regardless  actual  physical  distances.  Moreover,  as  learners  feel  closer  to  their  teachers,  they  tend  to 
regard  the  study  as  more  important  and  to  feel  that  they  understand  it. 
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Abstract;  Pedagogical  agents  offer  great  promise  to  improve  the  learning  experience  of  students 
interacting  with  educational  software  systems.  This  paper  describes  work  in  progress  on  a generic 
pedagogical  agent  architecture  and  an  authoring  system  to  support  the  creation  of  custom  agents  by 
teachers  and  other  domain  experts  using  a natural,  conversational  metaphor. 


The  field  of  pedagogical  agents  is  a recently  emergent  one  which  draws  from  previous  work  done  in  intelligent 
tutoring  systems,  autonomous  agents,  and  educational  theory  (Johnson  et  al.  2000).  Pedagogical  agents  have  the 
potential  to  offer  a wide  spectrum  of  educational  interactions  with  students,  including  opportunistic  instruction,  and 
have  been  shown  to  have  positive  affective  as  well  as  cognitive  influences  on  students  (Lester  et  al.  1997). 

Several  successful  pedagogical  agents  have  been  designed  for  a variety  of  learning  environments,  ranging  from 
skill-based  training  systems  to  more  open-ended  constructionist  environments  (Johnson  et  al.  2000).  Currently, 
however,  the  task  of  building  pedagogical  agents  is  a non-trivial  one  which  falls  within  the  domains  of  researchers 
and  professional  programmers  — not  those  of  teachers,  curriculum  designers,  and  other  non-programmers. 
Authoring  tools  designed  for  non-programmers  to  create  customized  content  for  particular  agents  are  under  way,  but 
little  attention  has  been  devoted  to  the  question  of  authoring  tools  for  creating  customized  agents,  given  a particular 
computer-based  learning  environment.  It  is  to  this  question  that  we  turn  our  attention  in  this  paper.  We  will 
describe  work  in  progress  on  a project  which  aims  to  allow  teachers  and  other  non-programmers  to  author 
customized  pedagogical  agents  for  interactive  learning  environments.  We  first  discuss  the  design  of  a generic 
pedagogical  agent  architecture,  suitable  for  use  in  constructionist  learning  environments.  Secondly  we  discuss 
techniques  we  are  developing  to  allow  teachers  to  author  agents  based  upon  this  architecture.  We  require  that  the 
architecture  be  suitable  for  constructionist  learning  environments  due  to  their  open-ended  nature,  which  raises  a 
greater  challenge  for  the  agent  designer. 

Our  generic  agent  architecture  (see  figure  at  http://www.cs.washington.edu/homes/Jbaer/pedagent/agentarch.Jpg)  is 
built  around  the  Agent  Control  Module.  This  module  is  responsible  for  processing  events  received  from  the  learning 
environment,  updating  its  student  and  session  models  based  on  those  events,  initiating  interactions  with  the  student, 
and  potentially  sending  commands  back  to  the  learning  environment.  The  appearance  and  user  interface  aspects  of 
the  agent  have  been  separated  out  into  a distinct  module.  This  allows  for  maximum  flexibility  in  the  appearance  of 
the  agent.  Either  the  agent  can  appear  in  its  own  window,  or  the  learning  environment  itself  can  implement  this 
module,  allowing  the  agent  to  appear  within  the  main  window  of  the  environment,  as  do  many  of  the  existing 
pedagogical  agents  described  by  Johnson  and  Lester  (Johnson  et  al.  2000)  do. 

The  interaction  between  the  agent  and  the  learning  environment  consists  of  the  agent  receiving  events  from  the 
environment  and  requesting  state  information  from  the  environment  or  sending  commands  to  the  environment.  It  is 
up  to  the  environment  to  define  the  set  of  events  that  will  be  sent  to  the  agent,  as  well  as  the  state  and  control 
commands  that  it  wishes  to  expose.  A distinguishing  feature  of  this  agent  model  is  that  we  do  not  require  the  agent 
to  have  complete  access  to  the  state  of  the  learning  environment.  While  this  can  limit  the  information  which  the 
agent  has  to  work  with,  it  allows  for  interoperation  with  a wider  variety  of  learning  environments,  all  of  which  may 
represent  their  state  in  different  ways. 

While  the  Agent  Control  Module  is  the  center  of  operations  for  the  agent,  it  does  not  contain  any  built-in 
intelligence  pertaining  to  any  specific  learning  environment.  Instead,  it  relies  on  the  Agent  Behavioral  Control 
Description  (ABCD)  for  this  information.  The  ABCD  is  divided  into  four  types  of  objects:  potential  student  actions. 


±0 


88 


agent  interventions,  pedagogical  goals,  and  student  concepts.  Potential  student  actions  are  actions  or  classes  of 
actions  which  the  student  might  take  within  the  environment,  and  which  the  agent  should  recognize.  These  actions 
are  comprised  of  a series  of  events,  each  of  which  may  contain  a parameter  sent  from  the  learning  environment  to 
the  agent.  The  sequence  of  events  and  their  parameters  can  be  generalized  using  regular  expressions.  Agent 
interventions  may  be  of  three  distinct  types  - information  presentation  using  text  and  images,  agent-initiated  dialog 
with  the  student,  or  agent-initiated  intervention  of  arbitrary  complexity  by  loading  and  executing  an  externally 
authored  program.  Pedagogical  goals  represent  general  learning  objectives,  for  example,  what  the  most  important 
concepts  are  for  a student  to  learn,  what  interaction  styles  to  prefer,  etc.  Student  concepts  represent  correct  or 
incorrect  conceptions  that  students  may  hold.  Our  notion  of  student  modeling  is  based  upon  the  work  of  Minstrell  et 
al  on  educational  facets  (Minstrell  1992),  and  our  student  concept  database  is  a facet  base,  in  their  terminology.  The 
student  model  the  agent  constructs  consists  of  a collection  of  student  concepts;  the  agent  infers  these,  and  each  is 
weighted  with  a confidence  factor. 

Since  it  is  the  Agent  Behavioral  Control  Description  (ABCD)  which  makes  a particular  agent  unique,  we  must  allow 
non-programmers  to  author  it.  Its  rules  are  similar  to  rules  in  a declarative  programming  language.  However,  we  do 
not  want  to  require  teachers  to  think  like  programmers.  Our  solution  is  to  have  the  agent  guide  the  author  through 
the  process  in  a natural,  conversational  manner.  The  agent  directs  the  authoring  process  by  asking  a series  of 
questions.  As  the  author  responds  to  these  questions,  the  agent  modifies  the  conversational  flow  and  builds 
components  of  its  ABCD,  based  on  the  author's  input.  In  addition  to  direct  question  and  answer,  the  agent  utilizes 
generalization  techniques  from  machine  learning  to  allow  for  authoring  by  demonstration  in  some  circumstances. 

This  authoring  technique,  which  we  call  "conversational  authoring",  bears  a resemblance  to  the  command  "wizards" 
which  are  found  in  a variety  of  applications.  However,  there  are  a number  of  factors  that  distinguish  this  authoring 
paradigm  from  application  command  wizards.  One  factor  is  that  the  interaction  between  the  author  and  the  system 
may  be  long-lived  and  span  multiple  authoring  sessions.  Because  of  this,  a need  exists  for  the  agent  to  maintain 
continuity  between  sessions  by  being  able  to  summarize  the  previous  session  and  describe  its  current  state  to  the 
author  in  a conversational  manner.  This  self-description  is  also  important  for  purposes  of  agent  validation.  The 
conversational  authoring  tool  runs  concurrently  with  the  agent  and  learning  environment,  so  the  author  can  test  the 
behavior  of  the  agent  at  any  point  in  the  authoring  process  by  using  the  environment  from  the  perspective  of  a 
student.  However,  simply  testing  the  agent  may  not  provide  enough  validation  that  the  desired  ABCD  structure  has 
been  created.  The  ability  of  the  agent  to  conversationally  summarize  its  state  at  any  given  point  in  the  authoring 
process  adds  another  tool  that  the  author  can  use  to  validate  the  authoring  outcome. 

The  work  described  in  this  paper  is  currently  in  progress.  The  generic  agent  architecture  has  been  implemented  in 
Java  and  integrated  into  a constructionist  learning  environment  called  PixelMath.  PixelMath  is  a recent  outgrowth 
of  research  at  the  University  of  Washington  designed  to  teach  mathematical  concepts  through  digital  image 
processing.  Additionally,  a custom  agent  has  been  built  on  top  of  the  generic  architecture  by  hard  coding  the  ABCD 
in  Java.  We  are  currently  in  the  process  of  implementing  the  conversational  authoring  tool,  which  will  be  tested  for 
effectiveness  by  teachers  familiar  with  the  classroom  use  of  tools  like  PixelMath.  A complete  description  of  this 
project,  including  current  status,  is  available  at  http://www.cs.washington.edu/homes/jbaer/pedagent/index.html. 
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Abstract:  Video  films  are  particularly  suitable  as  a way  of  experiencing  the  world 
because  they  provide  a dynamic,  although  vicarious,  encounter  through  sound  and 
vision.  However,  the  use  of  video  films  in  their  present,  mostly  passive,  fomi,  do 
not  succeed  in  contributing  enough  to  the  learning  process.  SYNOPSUS  was 
designed  to  cause  a shift  from  the  passive  “leaming-by- vie  wing”  model  to  the 
*1eaming-by-doing”  model  while  studying  with  educational  films.  By  utilizing  a 
simple  user-friendly  interface,  the  tool  allows  students  to  engage,  on  a computer, 
in  active,  reflective,  and  interactive  learning  with  existing  video  films.  The  tool 
and  some  preliminary  results  of  an  evaluation  study  will  be  presented. 


Introduction 

Video  films  are  particularly  suitable  as  a way  of  experiencing  the  world  because  they  provide 
a dynamic,  although  vicarious,  encounter  through  sound  and  vision  (Collis,  1996).  Moreover,  they  use 
a number  of  technical  devices  to  manipulate  that  experience.  Salomon  (1979)  called  these  devices 
“supplantations”,  in  the  sense  that  they  supplant  a cognitive  process. 

However,  the  use  of  video  films  in  their  present,  mostly  passive,  form,  do  not  succeed  in 
contributing  enough  to  the  learning  process  (Clark  & Craig,  1992).  In  active  learning  situations 
students  are  able  to  reflect,  construct  and  interact  (Laurillard,  1993,  Bates,  1997).  In  such  learning 
environments,  students  are  expected  to  comprehend  the  structure  of  the  lesson,  integrate  its  parts  and 
construct  their  knowledge  through  experienc-based  actions.  Present  video-film  learning  environments 
do  not  succeed  in  providing  such  a model. 

SYNOPSUS  was  designed  to  improve  the  passive  “leaming-by-viewing”  model  by 
reinforcing  a shift  to  “leaming-by-doing”  while  studying  with  educational  films. 


SYNOPSUS  - An  Educational  Tool 

By  utilizing  a user-friendly  interface,  the  tool  allows  students  to  engage,  on  a computer,  in 
active,  reflective,  and  interactive  learning  with  existing  video  films.  By  allowing  students  to  actively 
summarize  a video  film,  the  tool  encourages  them  to  engage  in  active  learning.  While  summarizing 
the  film,  students  are  creating  their  own  personal  model  of  the  movie,  its  structure,  and  its  contents. 

When  the  observer  decides  to  summarize  the  film  (which  is  seen  on  the  left  side  of  a 
computer  screen),  he/she  begins  typing  the  summary.  Immediately,  the  film  stops  running;  the  written 
summary  appears  at  the  right  side  of  the  screen  and  the  last  video  frame  is  attached  as  an  icon  to  the 
beginning  of  the  text.  Pressing  the  Esc  function  resumes  the  film,  and  the  summary  with  its  icon  are 
framed  in  a box.  Each  time  the  observer  begins  typing,  the  film  stops  running  and  another  summary 
appears  with  its  relevant  icon  on  the  right  side  of  the  screen.  At  the  end,  the  student  has  a list  of 
blocks  in  each  of  which  there  is  an  icon  representing  the  place  of  summary  in  the  film,  and  a written 
text  which  can  be  printed.  This  list  of  blocks  can  be  arranged  according  to  their  order  in  the  film,  or 
according  to  the  order  in  which  they  were  created. 

By  automatically  attaching  relevant  video  icons  next  to  a summary  block,  the  tool  provides 
an  easy  way  of  visually  remembering  and  keeping  track  of,  and  returning  to,  important  points  in  the 
film.  This  capability  encourages  students  to  engage  in  reflective  thinking  since  relevant  sections  of 
the  film  can  be  found  and  seen  by  clicking  on  an  associated  video  icon  that  immediately  moves  the 


film  to  the  intended  section.  Reflection  is  also  aided  by  allowing  immediate  access  to  relevant  sections 
in  the  film  via  simple  word  searches  on  the  textual  summaries. 

The  tool  also  allows  instructors  to  prepare  interactive  questions  and  assignments  pertaining 
to  the  film  watched.  The  questions  and  assignments  pop  up  when  the  student  reaches  a pre-specified 
point  in  a the  film. 

SYNOPSUS  can  be  used  for  distance  learners  and  for  face-to-face  teaching  during  frontal 
teaching  sessions.  By  using  the  presentation  mode  of  the  tool,  instructors  can  show  predefined 
segments  of  a film  along  with  their  own  headers  and  annotations. 

An  Evaluation  Study 

The  Open  University  of  Israel  (OUI)  is  a distance  learning  institution.  Some  of  the  couiscs 
offered  to  students  are  telecourses  which  include  as  learning  material  a textbook,  26  video  films  and  a 
student  guidebook.  A telecourse  in  psychology  was  chosen  for  a study  of  the  new  tool.  Out  of  400 
registered  students,  100  volunteered  to  try  the  software.  They  received  two  CDs:  one  contained  the 
software  and  the  other  contained  a thirty-minute  educational  film.  The  first  60  students  who  were 
willing  to  participate  in  the  study  (after  experimenting  successfully  with  the  software),  were  divided 
into  two  groups  of  30  students:  In  group  1 students  received  films  1 to  6 on  CDs,  and  in  group  2 
students  received  films  7 to  12  on  CDs.  Both  groups  received  all  other  films  on  video  cassettes.  The 
two  groups  were  similar  in  average  age,  percentage  of  males  and  number  of  previous  course  credits. 
This  design  allows  for  between  and  within  comparisons  regarding  the  experience  with  the  software 
and  CDs  on  the  one  hand  and  video  films  on  the  other  hand.  A third  group  of  eight  students  with 
learning  disabilities  served  for  an  exploratory  investigation  into  the  possible  advantages  of  the 
software  for  students  with  such  difficulties.  They  received  the  same  material  as  group  1 . 

A questionnaire  will  be  sent  to  all  three  groups  at  three  points  in  time:  (a)  after  the  allocated 
learning  period  for  the  first  six  films  (CDs  for  groups  1 and  3 and  videos  for  group  2);  (b)  after  the 
allocated  learning  period  for  films  7-12  (CDs  for  group  2 and  videos  for  groups  1 and  3)  and  ( c)  after 
the  allocated  learning  period  for  films  13-18  (videos  for  groups  1,  2 and  3).  Group  3 subjects  will  be 
interviewed  individually  regarding  their  experience  with  the  software  and  its  possible  help  in 
overcoming  some  of  their  special  learning  difficulties.  At  the  end  of  the  semester  students  will  be 
asked  to  send  their  summaries  to  the  research  team. 

The  purpose  of  the  experiment  was  to  test  learners’  attitudes  towards  the  learning  tool  as 
well  as  their  learning  processes  while  using  it.  The  design  of  the  experiment  permits  absolute 
evaluations  as  well  as  comparisons  with  learners  who  watch  the  films  on  the  TV  screen  without  the 
tool.  The  following  hypotheses  will  be  tested: 

1.  SYNOPSUS  facilitates  the  process  of  summarizing  the  film. 

2.  Subjects  summarize  more  and  better  with  SYNOPSUS  than  without  it. 

3.  SYNOPSUS  facilitates  concentration  while  watching  the  film. 

4.  SYNOPSUS  facilitates  the  process  of  locating  a specific  subject  in  the  film. 

5.  Subjects  go  backward  and  forward  in  the  film  in  order  to  access  specific  segments  of  it  more  often 
while  learning  with  SYNOPSUS  than  while  learning  without  it. 

6.  Subjects  enjoy  the  film  more  on  a television  screen  without  SYNOPSUS  than  on  the  PC  screen  with 
SYNOPSUS. 

7.  Subjects  feel  they  learn  more  from  a film  they  watched  on  a PC  screen  with  SYNOPSUS  than  from 
one  they  watched  on  a TV  screen  without  it. 

SYNOPSUS  will  be  presented  along  with  some  of  the  experiment’s  results  that  will  be 
available  at  the  time  of  the  conference. 
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Censorship  or  self-discipline  on  the  Internet? 

Strategies  for  limiting  access  to  morally  undesirable  Internet  resources. 


Abstract:  The  purposes  of  the  Internet  are  many;  commercial,  recreational,  informational, 
educational,  commiinicational,  relational.  The  Internet  is  a paradoxical  universe  of  structure  and 
anarchy:  producing  some  good  and  some  bad,  some  beautiful,  and  some  ghastly.  There  are  many, 
educationally  relevant  Internet  resources,  “by  the  year  2000,  anyone  in  western  civilization  will  be 
able  to  get  the  answer  to  any  question  that  has  an  answer”  according  to  influential  Byte  columnist, 
Jerry  Poumelle.  In  the  1997  Current  Population  Survey  by  the  US  Census  Bureau  computer  use 
was  surveyed.  The  study  showed  that  three  quarters  of  the  children  in  the  USA  have  access  to 
computers  at  home  or  school  - significantly  more  at  school  than  at  home.  Children  most  frequently 
used  home  computers  for  educational  purposes  (93.3%)  and  games  (83%).  One  fifth  of  the  children 
who  have  home  computers  used  them  to  access  the  Internet.  For  adults,  18  and  over,  92  million 
people  used  the  computer  at  home  or  work.  More  than  half  these  adults  accessed  the  Internet  from 
home,  with  80%  of  them  using  the  web  to  access  government,  business,  health  or  educational 
information.  Significant  increases  on  the  order  of  5-15%  were  recorded  for  every  category  of 
questioning  over  the  5 years  span  between  surveys.  In  all,  one  of  five  Americans  (ages  3 and 
higher)  used  the  Internet  in  1997®. 

This  presentation/essay  concerns  controlling  a kid’s  access  to  “morally  undesirable”  information. 
But  before  we  can  get  to  that,  it  is  important  to  consider  the  need  for  such  control.  In  the  schools  of 
our  nation,  our  children  are  using  the  Internet  to  reach  out  of  the  walls  and  experience  the  universe. 
Free  ranging  access  to  the  wonders  of  science  and  humanity  can  be  had  through  simply  searching 
the  Internet  using  search  programs.  Any  teacher  or  student  deserves  these  resources  and  they 
deserve  to  follow  the  curiosities  that  are  a contingent  element  of  human  intelligence.  The  question 
that  arises  involves  the  various  interests  of  the  purveyors  of  these  archives.  Some  of  these  interests 
are  illegal  and  even  more  are  morally  undesirable. 


The  purposes  of  the  Internet  are  many;  commercial,  recreational,  informational,  educational,  communicational, 
relational.  The  Internet  is  a universe  of  anarchy  and  chaos:  producing  some  good  and  some  bad,  some  beautiful,  and 
some  ghastly.  There  are  many,  educationally  relevant  Internet  resources,  “by  the  year  2000,  anyone  in  western 
civilization  will  be  able  to  get  the  answer  to  any  question  that  has  an  answer”  according  to  influential  Byte 
columnist,  Jerry  Poumelle.  In  the  1997  Current  Population  Survey  by  the  US  Census  Bureau  computer  use  was 
surveyed.  The  study  showed  that  three  quarters  of  the  children  in  the  USA  have  access  to  computers  at  home  or 
school  - significantly  more  at  school  than  at  home.  Children  most  frequently  used  home  computers  for  educational 
purposes  (93.3%)  and  games  (83%).  One  fifth  of  the  children  who  have  home  computers  used  them  to  access  the 
Internet.  For  adults,  18  and  over,  92  million  people  used  the  computer  at  home  or  work.  More  than  half  these  adults 
accessed  the  Internet  from  home,  with  80%  of  them  using  the  web  to  access  government,  business,  health  or 
educational  information.  Significant  increases  on  the  order  of  5-15%  were  recorded  for  every  category  of 
questioning  over  the  5 years  span  between  surveys.  In  all,  one  of  five  Americans  (ages  3 and  higher)  used  the 
Internet  in  1997®. 

This  essay  concerns  controlling  a kid’s  access  to  “morally  undesirable”  information.  But  before  we  can  get  to  that,  it 
is  important  to  consider  the  need  for  such  control.  In  the  schools  of  our  nation,  our  children  are  using  the  Internet  to 
reach  out  of  the  walls  and  experience  the  universe.  Free  ranging  access  to  the  wonders  of  science  and  humanity  can 
be  had  through  simply  searching  the  Internet  using  search  programs.  Any  teacher  or  student  deserves  these  resources 
and  they  deserve  to  follow  the  curiosities  that  are  a contingent  element  of  human  intelligence.  The  question  that 
arises  involves  the  various  interests  of  the  purveyors  of  these  archives.  Some  of  these  interests  are  illegal  and  even 
more  are  morally  undesirable. 
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Introduction 


Illegal  interests  are  pursued  and  prosecuted  in  the  United  States  by  state  and  local  law  enforcement,  the  FBI,  the  US 
Marshall  or  the  Federal  Trade  Commission  depending  on  which  jurisdiction  the  activity  entails.  The  international 
nature  of  the  Internet  may  bring  other  law  enforcement  agencies  into  the  investigation  and  prosecution.  In  the  US 
there  are  specific  police  resources  devoted  to  cyber-crime. 

On  the  morally  undesirable  side,  there  is  no  such  concerted  effort  - in  fact  there  seems  to  be  a fair  level  of 
disagreement.  While  there  is  much  media  attention  given  to  (thus  far  ineffective)  efforts  aimed  at  outlawing  these 
undesirable  Internet  activities,  there  is  a strange  avoidance  of  discussion  into  exactly  which  information  should  be 
restricted.  Consensus  has  arisen  around  some  pseudo-illegal  activities  such  as  distribution  of  child  pornography. 

But  aside  from  that... almost  certainly  the  literature  simply  starts  off  with  the  assumption  that  children  need  to  be 
protected  from  whatever  morally  reprehensible  activity  the  writer  is  imaging  but  not  really  describing.  And, 
honestly,  aside  from  acknowledging  this  vagueness,  this  writer  is  not  much  different. 

The  generally  agreed  philosophical  rationale  is  that  children  from  ages  5 to  18  must  be  protected  from  this 
information  because  they  are  not  ready  to  contextually  understand  these  darker  fantasies  and  features  of  our  culture. 
(In  extreme  cases  — I am  wondering  who  is?)  Kids  are  ignorant  to  the  dangers  that  may  be  encountered  through  their 
venturing  into  knowledge  unconditioned  by  experience.  A loud  debate  rages  about  “how  to  stop  access”  to  these 
dark  media  resources.  When  the  next  kids-killing-kids,  school  shooting  incident  occurs,  our  attention  will  be  brought 
sharply  back  to  the  studies  that  widely  implicate  media’s  influence  on  a child’s  moral  development.  This  research  is 
not  clear  on  the  nature  of  the  developmental  harm  associated  with  a premature  familiarity  of  the  darker  side  of 
humanity.  We  could  wish  for  more  specifics  as  to  which  type  of  media,  how  it  specifically  affects  development  and 
when  in  the  child’s  development  are  they  vulnerable  to  such  influence. 

For  purposes  of  this  essay,  it  will  be  assumed  that  kids  need  protection  from  morally  undesirable  elements  of  our 
culture  even  if  adults  do  not.  That  further,  the  special  media  circumstance  of  Internet  delivery  of  these  morally 
undesirable  archives  directly  into  our  schools  and  homes  deserves  our  special  attention.  We  will  assume,  as  do  many 
others,  that  an  inadvertent  curriculum  can  formulate  around  these  dark  and  undesirable  interests. 

Since  these  fascinations  with  the  darker  elements  exist  in  our  culture,  there  is  necessarily  a developmental  time 
when  youth  should  gain  controlled  experiences  in  them.  The  school  however  is  excluded  from  that  objective  - it  is 
left  to  parents  to  control  this  curriculum.  There  are  logical  flaws  in  this  plan  (e.g.  that  parents  often  do  not  address 
these  legitimate  needs)  will  also  be  excluded  from  this  essay. 

This  essay  will  assume  that  someday  these  “morally  undesirable”  graphical  and  information  archives  can  be 
identified  through  public  discourse  and  reasoning.  Something  beyond  the  “we  will  know  them  when  we  see  them” 
criteria.  Though  there  is  no  theory  on  whether  it  really  is  undesirable  for  children  to  be  exposed  to  any  of  the 
following  areas... and  though  there  is  no  sharp  delineation  of  the  level  at  which  information  becomes  damaging. . .the 
general  content  areas  are  well  elaborated  in  the  literature  and  involve  concern  about  the  following  issues. 
Inappropriate  sexual  displays. 

There  is  a well-publicized  and  very  active  pornographic  sector  accessible  through  the  Internet^.  These 
resources  may  involve  text  or  graphical  displays  that  are  developmental ly  inappropriate  for  children  (and 
frankly  adults!).  This  area  receives  much  attention  by  the  naysayers  of  open  Internet  access.  It  is  said  to 
represent  a significant  but  small  proportion  of  Internet  informational  traffic  (I  have  seen  figures  of  less  than 
10%  - I cannot  seem  to  find  the  source  right  at  this  moment  though)  Additionally,  these  sites  often  involve 
unscrupulous  schemes  to  charge  the  viewer  — while  claiming  to  be  free  of  charge.  It  is  a common  family 
dilemma  to  have  a surprise  encounter  with  the  financial  consequences  of  a child’s  experimenting  with 
Internet  sex  sites.  Again,  research  is  disappointing  here,  but  in  my  circle  of  friends  and  family,  children 
have  spent  thousands  of  dollars  in  this  way.  Anecdotally,  costs  associated  with  these  episodes  are 
confabulated  with  similar  costs  charged  by  cable  and  phone  sex  experimentation. 

Predatory  persons 

There  is  a danger  that  children  may  be  lured  into  meeting  some  predatory  person,  whose  desire  is  to  do 
them  physical  harm.  While  this  danger  is  theoretically  real,  there  is  no  frequency  data  available  in  the 
literature.  It  can  be  assumed  though  from  reports  of  adults  being  lured  out  to  meet  predatory  persons,  that 
children  would  be  also  similarly  vulnerable  to  such  risks^^. 
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Financial  Scams 

There  are  a number  of  financial  schemes  communicated  through  the  Internet.  Pyramid  schemes  are  popular 
— so  are  offers  “too  good  to  be  possible.”  Low  interest,  poor  credit  credit  card  offers  are  also  frequently 
offered  on  the  Internet.  It  can  be  assumed  that  the  limited  experience  of  kids  would  make  them  targets  for 
these  deceptive  or  illegal  activities.  Again,  there  is  no  data  readily  available  on  the  incidence  of  children 
being  victims.  It  is  sufficient  to  say,  that  there  are  many  adults  that  get  scammed.  We  can  be  sure  that  if 
there  were  unlimited  access  to  children  for  the  scam  artists  and  a source  of  credit  for  the  children,  there 
would  be  a significant  level  of  risk  to  the  financial  well  being  of  families  and  children^^. 

Privacy  concerns 

The  issue  here  is  that  children  might  give  out  information  that  could  physically  target  them  for  predators. 
Another  concern  may  be  that  the  child,  not  being  aware  of  the  risks,  may  innocently  or  out  of  bravado 
arrange  a connection  witli  an  unsavory  character^*’^^. 

System  integrity  and  security  issues 

There  might  be  risks  to  the  stability  of  the  network  due  to  downloading  programs  that  originate  from 
certain  Internet  sites.  There  is  a fair  amount  of  Internet  traffic  visible  on  lists  associated  with  “hacking.” 
(search  keyword  “crackers”)  The  substance  of  these  messages  often  reflects  an  interest  in  illegally  obtained 
software  and  passwords  to  various  nefarious  sites.  An  easy  and  more  popular  invasion  of  networks  involves 
reflecting  junk  e-mail  (“spam”)  from  the  users  domain  so  that  it  will  appear  to  be  from  your  computer 
system.  I work  for  a women’s  college  in  the  upper  Midwest  and  we  discovered  that  advertising  for  a 
pornographic  site  was  using  our  domain  as  a reflector. ..to  the  extent  that  in  one  night  60,000  emails  had 
been  distributed!  This,  of  course,  was  unacceptable  to  our  network  administrators.  Protection  from  this  type 
of  hacking  has  been  installed.  Another  concern  is  viruses  and  virus  hoaxes.  Aside  from  the  damage  an 
actual  virus  can  do  to  networks,  the  volume  of  messages  that  virus  hoaxes  can  generate  is  in  itself,  a plague. 
Harassment 

It  is  possible  for  a predatory  person  to  stalk  and  harass  a person  via  the  Internet.  Especially  if  used  in 
combination  with  other  modes  of  information  accessible  through  the  Internet  or  other  information 
providers.  Phone  numbers,  addresses,  maps  may  assist  the  stalker  to  find  their  target  repeatedly.  It  is  more 
and  more  difficult  to  “disappear”  if  one  needs  to.  One  source  listed  the  costs  for  information  on  the  a 
person  as  $15  for  social  security  number,  to  $150  for  a complete  background  check. 

Internet  ^^addiction” 

While  dubious  as  a real  disorder,  some  people  spend  inordinate,  obsessive  amounts  of  time  on  the 
Intemet^’'*’^^’^^’^^’^'*’^^.  Sometimes  this  interferes  with  normal  activities  of  daily  living.  The  fear  is,  that 
introducing  Internet  use  in  schools  give  tacit  approval  for  obsessive  use  of  the  Internet.  There  is  no 
empirical  data  available  in  the  literature,  but  based  on  anecdotal  evidence  (again  from  families  and  friends) 
teenagers  are  known  to  sacrifice  duties  and  sometimes  sleeping  in  order  to  use  the  Internet.  Recently,  the 
American  Psychological  Association  gave  some  context  to  the  question  during  a symposia  on  the  topic. 
They  concluded  that  pathological  internet  use  was  possibly  an  addiction,  a disorder  or  a symptom  of 
depression. 

Abuse  of  credit 

A problem  not  exclusive  to  the  Internet,  there  cannot  be  any  doubt  that  the  Internet  is  a good  place  to  spend 
a lot  of  money.  Many  Internet  retail  and  service  sites  exist  and  accept  credit  cards.  It  is  easy  to  rack  up  an 
impressive  bill  in  a few  hours  of  Internet  shopping.  Amazon.com  for  books,  records,  presents;  Quill  for 
office  supplies;  LandsEnd  for  clothing;  EggHead  for  software;  Dell  for  a computer;  Auto.com  for  a car; 
ebay  for  a boat;  pretty  soon  you’ve  spent  some  bucks! 

Depression 

There  have  been  articles  in  the  psychology  literature  about  a relationship  between  Internet  use  and 
depression^’^.  The  debate  rages  on  with  both  side  positing  credible  pros  and  cons.  The  crux  of  the  issue  here 
is  the  chicken/egg  problem.  That  is  did  the  depression  precede  the  use  of  the  Internet  or  was  depression  a 
result  of  Internet  use?  Currently  the  theory  that  seems  most  favored  is  that  use  of  the  Internet  is  more  a 
symptom  than  a cause  of  depression. 

Academic  integrity 

A virtual  industry  has  evolved  selling  term  papers  to  students.  Keyword  search  on  “term  papers”  revealed 
100s  of  sites  devoted  to  free  and  for  sale  papers.  Many  services  offered  custom  services.  According  to 
“Paper-Masters”  their  custom  written  term  papers  are  meant  to  be  research  to  assist  you  writing  your  own 
paper.  They  conclude  that  it  is  not  in  any  way  unethical  to  use  their  services. 


Identity 

The  relative  anonymity  provided  by  the  Internet  is  a challenge  to  individual  identity.  This  feature  also 
allows  people  to  fabricate  an  identity  that  is  quite  different  from  their  own.  The  concern  is  that  this  fantasy 
identity  may  interfere  with  the  identity  forming  process  that  is  an  important  in  the  child^"^. 

Community 

There  are  fears  that  rather  than  encouraging  community  and  multicultural  exchange  the  Internet  actually 
encourages  isolation.^ 

Intellectual  property 

Many  teachers  report  a cut  and  paste  form  of  writing  where  the  students  simply  search  out  information  on 
the  Internet  and  then  copy  it  directly  into  their  papers  without  quotation.  Additionally  copyright 
infringement  rules  are  very  difficult  to  understand  within  the  context  of  the  Internet. 

Social  accountability 

The  anonymity  and  2 dimensionality  of  the  Internet  does  not  foster  authentic  relationships.  This  issue  is  the 
converse  of  the  safety  afforded  by  the  electronic  distance  of  the  Internet.  One  concern  for  kids  is  the 
negative  role  model  that  this  demonstrates.  The  superficiality  of  complex  individuals  as  represented  on  the 
Internet  is  also  an  impression  that  is  unrealistic.  On  the  Internet,  as  with  other  media,  the  distinction 
between  real  an  imaginary  is  difficult  to  detect.^^ 

Review  of  the  literature 

While  it  this  essay  is  focused  on  educational  issues  in  American  schools,  it  is  interesting  to  observe  that  in 
Europe  several  countries  (GDR,  GB  and  others)  have  also  sought  to  control  certain  types  of  Internet 
communications.  Great  Britain  actually  has  a complaint  bureau,  The  Internet  Watch  Foundation*,  where 
consumers  can  expect  to  see  prosecution  of  child  pornography,  and  other  harmful  activities.  The  bureau 
expressly  stays  clear  of  controversies  into  items  of  political  or  social  question  though.  Internet  access  is 
severely  limited  at  the  source  - the  Internet  access  hub,  in  several  Middle  Eastern  countries  (Saudi  and 
Egypt).  All  accessible  sites  are  government  approved  even  email  is  scrutinized. 

In  the  United  States,  a first  Amendment  disagreement  has  arisen  over  Internet  control.  The  US  Supreme 
Court  has  made  four  determinations  in  favor  of  free  Internet  access  for  anyone  able  to  get  there.  Though 
there  are  plenty  of  details  left  to  interpretation,  the  court  has  made  sweeping  statements  about  the  freedom 
of  the  Internet.  Congress  continues  to  contemplate  legislative  action,  intended  to  circumvent  the  Supreme 
Court’s  First  Amendment  rulings  and  control  some  types  of  Internet  communications.  As  stated  before, 
illegal  activities  are  prohibited  and  prosecuted  in  multiple  ways,  the  debate  concerns  pseudo-legal  and  legal 
but  controversial  Internet  resources  such  as  child  pornography,  bomb-building  directions,  racism,  slander, 
misinformation. 

Strategies  for  limiting  the  access  of  kids  to  morally  undesirable  Internet  archives 
Free  access  to  all: 

It  would  seem  that  this  is  what  we  have.  This  is  a common  misconception  of  the  Internet.  Certain  types  of 
information  cannot  (currently)  be  transmitted  via  the  Internet.  The  interface  is  restricted  to  materials  that 
are  comprised  of  an  audio,  visual  and/or  digital  nature.  Certainly  this  covers  a lot  of  ground,  but  in  reality 
this  is  a significant  limitation  from  the  start.  Not  everything  the  human  has  a capacity  to  sense  can  be 
transmitted  by  the  Internet. 

Additionally,  and  of  more  relevance  to  the  theme  of  this  essay;  illegal  activities  are  prohibited  on  the 
Internet.  As  stated  above,  several  governmental  policing  agencies  monitor  and  in  fact  utilize  the  Internet  to 
investigate  and  prosecute  criminal  activities*.  Email  threats  are  taken  quite  seriously  - the  appearance  of 
treasury,  secret  service  officers  at  a local  high  school  last  year  to  arrest  a student  for  having  sent  a death 
threat  to  President  Clinton  is  lesson  enough  to  prove  this  point.  Drug  and  firearm  enforcement  agencies  also 
monitor  the  Internet  for  signs  of  criminal  activity. 

Direct  adult  supervision: 

There  is  a tacit  rejection  of  direct  adult  supervision  as  a control  strategy  in  almost  all  the  literature^®.  It  is 
felt  unrealistic  that  an  adult  mentor  could  monitor  a kids  Internet  adventures,  or  help  them  gain  context  if 
they  bump  into  something  morally  undesirable.  The  literature  in  this  area  occasionally  mentions  that  this  is 
an  option,  but  that  then  goes  no  farther  than  to  state  it  is  simply  not  feasible  for  presumably  practical 
reasons.  There  is,  however,  agreement  that  this  method  of  control  would  be  desirable  and  effective.  This 
writer  is  not  so  sure  that  this  is  the  contingent  difficulty  with  the  idea.  Challenges  here  may  be  more 
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associated  with  the  problems  of  communicating  with  our  youth.  Adults  are  really  hesitant  to  talk  about 
morally  challenging  topics  - a sudden  view  of  human  sexual  interaction  might  initiate  questions  - and  that 
would  lead  to  a difficult  conversation.  Teachers  should  be  allowed  to  have  this  kind  of  conversation  with 
our  youths,  but  the  onus  of  responsibility  is  on  the  family  where  it  <dismay>  rarely  occurs. 

User  Self  Policing  - Moral  development  and  the  Internet 

It  could  be  left  to  the  user  to  restrict  their  own  access  by  virtue  of  their  own  experiences^^’^^.  This  highly 
improbable  suggestion  centers  on  development  of  a personally  held  and  derived  dislike  for  morally 
undesirable  Internet  resources.  It  can  happen,  and  most  probably  does,  after  the  thrill  is  gone  so  to  speak. 

The  impractical  part  of  this  suggestion  is  not  the  philosophically  sound  grounding  that  exists  in  allowing 
people  to  morally  develop,  but  rather  the  political  impossibility  of  implementing  such  a plan  in  a country 
founded  by  the  Puritans.  Additionally,  there  is  much  to  be  said  for  the  idea  that  a staged  approach  with 
contextulization  would  need  to  be  implemented  to  coincide  witii  the  developmeniai  capacity  of  each 
individual  kid.  Given  the  performance  record  of  our  public  schools.  ..it  is  utopian  to  imagine  such  a strategy 
could  be  harmlessly  implemented. 

Legislation: 

Communication  Decency  Act  (CDA)  and  Son  of  CDA  - overturned  by  the  US  Supreme  Courts.  Ruled  as 
unconstitutional  by  virtue  of  their  violation  of  First  Amendment  rights^®.  Politicians  continue  to  bring  up 
legislation  that  the  American  Civil  Liberties  Union  defeats  in  court  challenges.  Literally  millions  of  dollars 
have  been  expended  in  the  quest  for  legislation  that  can  get  through  the  first  amendment  tests.  Often  the 
problem  is  that  the  definitions  for  morally  undesirable  materials  cannot  be  specifically  measured,  and  could 
thus  be  applied  too  widely. 

Acceptable  Use  Policies  (AUP): 

As  an  aide  to  self  control,  a signed  statement  of  appropriate  internet  use  AND  the  consequences  of  abuse  of 
privileges  is  used.  Samples  of  AUPs  are  available  on  the  Internet.  AUPs  are  used  in  nearly  all  networked 
computer  labs.  It  is  necessary  that  some  person  observe  the  Internet  use  in  such  situations.  This  can  be  done 
by  walking  around  the  lab  or  by  looking  at  the  log  of  Internet  addresses  that  has  been  accessed  on  a 
particular  computer.  It  would  seem  very  possible  to  automate  this  observation  of  a lab  by  sharing  the  log 
with  a central  teachers  workstation,  but  there  was  no  mention  of  this  strategy  in  the  literature.  See 
references^’  for  a link  to  an  example  provided  by  Rice  University. 

Rating  Systems: 

A strategy  familiar  to  moviegoers  is  common  of  the  Internet  - warning  the  consumer  of  adult  content.  Most 
commercial  (even  free)  adult  sites  require  the  user  to  physically  confirm  select  their  intent  to  utilize  the 
morally  undesirable  archives.  Some  Internet  businesses  have  arisen  that  claim  to  restrict  the  access  of 
minors  to  their  sites  by  virtue  of  credit  card  credentials.  Of  course,  a credit  card  has  little  to  do  with 
verifying  age  - these  sites  are  more  interested  in  connecting  interested  parties  (and  their  credit  cards)  with 
their  cadre  of  pornographic  vendors.  The  intent  is  to  protect  the  inadvertent  viewing  of  these  resources,  but 
of  course,  many  curious  15  year  olds  are  undeterred  by  these  warning  and  click  right  through. 

Screening  Software: 

It  would  be  nice  if  software  could  be  developed  to  filter  morally  undesirable  Internet  content,  but  thus  far 
efforts  have  been  only  partially  successful.  Current  filtering  software  can  be  grouped  based  on  the 

strategy  used  to  filter.  One  group  uses  keywords,  taken  from  a ready  made  list  of  dirty  words,  to  eliminate 
access  to  undesirable  Internet  archives.  Often  the  list  can  be  modified  by  the  administrator  to  add  new 
words  or  to  add  in  a miss  spelling  of  the  particular  dirty  word  of  concern.  This  is  effective  but  suffers  from 
the  problem  that  sometime  dirty  words  also  have  morally  desirable  alternative  uses.  For  instance  “breast” 
as  a keyword  will  also  eliminate  sites  concerned  with  breast  cancer.  Another  approach  is  to  keep  a list  of 
either  acceptable  and  unacceptable  sites.  For  instance  sites  ending  with  the  “.gov”  suffix  are  approved, 
Disney.com  is  approved,  a list  of  sites  that  are  not  approved  is  provided  and  updated  by  the  software 
company  and  can  be  modified  by  the  administrator.  Unfortunately,  the  undesirable  element  of  the  Internet 
moves  around  a lot,  and  new  morally  undesirable  sites  are  launched  every  day,  so  it  is  easy  to  imagine  that 
something  will  get  past  the  filter. 

Network  Filter  programs: 

Similar  to  screening  software,  more  sophisticated  filters  can  be  installed  at  the  network  hub  . This  is  the 
strategy  used  by  those  Middle  Eastern  countries  to  prevent  access  to  certain  internet  resources. 

Intranet  strategies: 

One  strategy  taken  by  some  schools  has  been  to  simply  offer  only  internal  versions  of  Internet  available 
resources.  It’s  a sort  of  smaller,  internally  stored  version  of  the  Internet.  Web  Whacker^®  whacks  entire  sites 
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out  of  the  Internet  and  stores  them  on  your  computer  network.  This  strategy  is  fine  as  long  as  the  learner 
only  asks  questions  available  within  the  limits  of  the  stored  information.  An  encyclopedic  resource,  and 
other  informative  resources  are  made  available.  Often  access  to  the  external  Internet  is  possible  for  special 
situations,  and  can  be  more  effectively  monitored. 

Internet  watch  groups: 

Another  way  used  in  concert  with  screening  software  is  to  have  a group  of  evaluators  monitor  the  moral 
permissibility  of  Internet  sites^^’^^.  What  happens  is  that  a software  company  or  a school  enlists  the  help  of 
a screening  committee.  Their  function  is  to  expand  the  list  of  forbidden  and  permitted  Internet  resources. 

Free  market  pressure  and  industry  self  regulation: 

Especially  during  the  legislative  debate  on  CD  A,  many  Internet  Service  Provider  companies  became 
concerned  over  the  trouble  it  might  be  if  they  could  be  held  responsible  for  Internet  content.  America 
Online,  the  largest  Internet  Service  provider,  initiated  a program  of  parental  controls  that  allow  various 
levels  of  access  control  over  the  Internet  and  AOL  network  resources'’*  Also,  they  developed  a system 
of  “moderators”  to  monitor  kid  chatrooms,  as  well  as  a responsive  complaint  system.  In  other  words,  they 
felt  it  was  good  for  their  business  interests  to  allow  the  user  to  determine  limits  on  access. 

Conclusions 

So  what  is  the  big  picture  here?  First,  this  is  a new  experience  to  all  of  us,  second  the  Internet  needs  to  be  a 
vital  part  of  the  learning  experience  for  kids.  Third  we  will  not  be  able  to  bring  ourselves  to  restrict 
anything  but  illegal  activities  from  the  Internet  - and  even  here  it’s  a post  hoc  analysis. 

Clearly  the  best  choice  is  to  design  a system  where  Internet  use  is  closely  monitored,  but  not  for  punitive 
purposes,  but  for  contextualizing  by  adults  who  are  skilled  and  vested  in  helping  kids  see  our  world,  all  of  it 
at  the  right  time.  Since  this  is  so  impossible  given  our  current  factory  model  schools,  I would  recommend 
the  strategy  where  much  of  the  information  needed  by  students  is  anticipated  and  locally  stored  on  school 
friendily  servers  — a second  network  soto  speak.  When  kids  need  to  venture  onto  the  Internet  let  them,  and 
be  there  for  them. 

There  is  much  research  that  needs  to  be  done,  but  some  of  the  important  questions  IMHO  (in  my  humble 
opinion)  have  to  do  with  cultural  changes  that  will  come  as  a result  of  this  new  dimension.  Questions  about 
moral  development,  community  and  diversity  seem  largely  unstudied.  Philosophical  and  psychological 
issues  such  as  what  really  is  inappropriate  and  at  what  developmental  age  are  barely  disturbed.  Finally,  I 
wonder  what  the  future  will  hold  for  new  ways  to  communicate  human  sensations  and  feeling  will  be. 
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Using  Speech  Recognition  to  Support  Online  Learning  of 
Second/Foreign  Languages 


Francis  Bonkowski,  IC  Education,  Inc.,  Canada 
Quinn  Cheung,  IC  Education,  Inc.,  Canada 
Liam  Cheung,  IC  Education,  Inc.,  Canada 


This  paper  presents  IC  Education’s  state-of-the-art  pedagogy  and  technology  to  support  online  language 
learning.  IC  Education  lias  developed  speecli  recognition  over  the  Internet  using  a new  client/'server 
application  that  recognizes  the  voice  of  user’s  without  having  to  install  complicated  software  on  a personal 
computer.  The  system  entails  minimal  software  installation  by  the  user  and  provides  centralization  of  the 
speech  recognition  engine.  This  innovative  technology  is  a powerful,  albeit  simple  tool  used  to  teach 
learners  proper  pronunciation  and  new  vocabulary. 


A BUILDING-BLOCK  METHOD  FOR  COURSEWARE 
DEVELOPMENT  FOR  LESS-EXPERIENCED  DEVELOPERS 
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TNO-Human  Factors,  Soesterberg,  The  Netherlands 
Kampweg  5,  P.O.  box  23,  3769  ZG  Soesterberg 
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Abstract;  Although  coursew^are  has  the  potential  to  increase  the  efficiency  and 
effectiveness  of  courses,  until  now  the  results  of  its  application  is  often  disappointing.  In 
our  research  concerning  courseware  development  within  the  Royal  Netherlands  Army 
(RNLA),  we  investigated  some  of  the  underlying  reasons  for  this,  and  specified  an 
approach  to  promote  its  applicability.  One  of  the  main  problems  is  the  high  cost  of 
courseware  development,  resulting  from  the  long  development  time  and  the  specific 
expertise  required.  We  constructed  a 'building-block'  approach,  which  enables 
instructors  with  limited  multi-media  expertise  to  rapidly  develop  courseware  for  simple 
learning  objectives,  but  with  the  use  of  advanced  multi-media  technology,.  This  approach 
consists  of  templates  that  incorporate  sound  didactical  principles,  lower  the 
technological  complexity  of  courseware  development,  and  can  be  used  within  modern 
authoring  tools.  Currently  we  are  implementing  the  method  and  are  studying  how  to 
enlarge  the  scope  of  the  method  towards  more  advanced  learning  goals. 


The  use  of  courseware 

Recently,  the  interest  for  the  use  of  courseware  within  many  educational  and  corporate 
organizations  is  increasing  in  a rapid  pace,  as  it  has  the  potential  to  increase  the  efficiency  and 
effectiveness  of  courses  significantly.  Organizations  want  to  develop  courseware  in-house,  as  in  most 
settings  Commercial  Of  The  Shelf  (COTS)  courseware  is  not  specific  enough  and  currently  modem, 
powerful  authoring  tools  are  available.  However,  in  many  cases  such  attempts  have  been  disappointing 
in  terms  of  contributing  to  efficiency  and  effectiveness  of  (the  development  of)  training  programs.  For 
instance,  in  the  last  few  years,  the  Royal  Netherlands  Army  (RNLA)  has  spent  considerable  time  and 
effort  in  developing  courseware  for  its  Training  Centers.  This  courseware  mainly  directed  at  concept 
learning  and  procedural  tasks.  However,  the  application  of  courseware  has  not  yet  lived  up  to  its 
expectations.  It  has  never  been  very  cost-effective  and  (therefore)  organizational  acceptation  or  broad 
adaptation  of  courseware  within  training  programs  is  still  not  accomplished. 

Bottlenecks  in  developing  courseware 

TNO-HF  has  been  asked  to  investigate  this  problem  and  to  formulate,  implement  and  test 
solutions  for  it.  We  found  a number  of  problems  concerning  courseware  development  (Boot  & Van 
Rooij,  1999).  The  development  of  courseware  is  mostly  done  by  Subject  Matter  Experts  (SME’s)  and 
instructors,  using  modem  authoring  tools.  Despite  the  efforts  and  promises  of  these  tools  to  reduce 
development  time  and  the  level  of  required  expertise  (e.g.  Asymetrix,  1998),  it  still  takes  considerable 
time  (and  therefore  costs)  to  produce  courseware.  We  found  that  considerable  programming  knowledge 
is  required  for  anything  more  that  very  simple  courseware.  Also,  much  time  is  required  to  familiarize 
with  the  tools  and  its  many  features  and  the  translation  of  didactical  models  into  courseware  is 
complicated.  In  the  RNLA,  the  SME’s  and  instructors  lack  the  didactical-,  multi -medial-  and 
programming  expertise  to  make  optimal  and  efficient  usage  of  these  tools.  One  can  conclude  that  the 
development  of  anything  more  advanced  than  very  simple  courseware  programs,  remains  costly  and 
will  be  only  cost-effective  for  large  target  groups. 


The  building-block  approach 


If  one  aims  at  courseware  with  advanced  technological  features  (in  terms  of  multi-media 
usage  and  interactivity),  developed  by  people  with  limited  programming  expertise,  a so-called 
‘building-block’  approach  can  be  a solution.  This  approach  pre-structures  the  development-process  by 
offering  templates  within  authoring  tools.  Templates  consist  of  empty  courseware-structures  to  be  filled 
with  content.  These  structures  incorporate  sound  didactical  principles  and  advanced  technological 
features.  Once  connected  to  each  other  (hence  the  name  building-block  approach)  and  filled  with 
content  (the  learning  materials),  they  constitute  a complete  courseware  program.  We  investigated  some 
build-in  pre-structuring  features  of  modem  authoring  tools  that  make  the  same  claims,  but  they  are  too 
limited  for  our  purposes.  Therefore,  our  templates  should  be  more  advanced  and  comprehensive,  in 
order  to  promote  proper  implementation  of  didactical  models  and  lower  the  technological  complexity. 
As  standardization,  re-usability  and  exchangeability  of  course ware(elements)  is  a key  issue,  these 
templates  should  be  embedded  within  COTS  authoring  tools. 

Specification  and  implementation  of  the  approach 

We  specified  a building  block  approach  for  the  RNLA,  with  templates  for  four  levels  of 
courseware-structure  (Boot  & Van  Rooij,  2000); 

1 . At  the  top  level,  templates  provide  different  kinds  of  navigation  through  a number  of  lessons. 

2.  At  the  second  level,  templates  provide  different  kinds  of  navigation  of  learning  activities 
within  lessons. 

3.  At  the  third  level,  templates  provide  interactivity  within  learning  activities. 

4.  At  the  bottom  level,  templates  provide  the  presentation  of  multi-media  elements  within 
learning  activities. 

Altogether,  they  contribute  with  standardized  screen-elements  to  the  complete  user-interface 
of  the  program.  The  didactical  functionality,  embedded  in  the  templates,  is  directed  at  relative  simple 
learning  goals;  learning  facts,  concepts,  procedures  and  principles.  The  instructional  strategies  that  are 
embedded  are  based  on  theories  like  Gagne’s  Nine  Events  and  Merrill’s  Component  Display  Theory  as 
found  in  Reigeluth  (1983).  Technically,  the  templates  are  build  using  Knowledge  Objects®  within  the 
authoring  tool  Authorware®  Version  5,  and  are  provided  with  Wizards  for  guiding  the  interaction  with 
the  developers.  For  reasons  of  acceptance  and  commitment  of  the  future  developers,  we  have  involved 
them  during  the  specification  process  of  the  building-block  approach.  We  found  that  it  is  important  and 
difficult  to  balance  well  between  how  much  templates  should  prescribe  and  which  freedom  remains  for 
the  developers. 

Currently,  we  are  in  the  implementation  phase;  external  software  developers  are  building  the 
templates,  after  which  they  will  be  used  by  some  developers  within  the  RNLA.  We  will  start  a number 
of  pilot  projects  to  evaluate  the  application  of  the  approach  and  the  resulting  courseware.  Based  on 
these  results,  the  templates  can  be  improved  and  the  building  block  approach  will  be  applied  on  a wider 
scale. 

Future  developments 

One  important  aspect  of  our  research  is  yet  unresolved;  how  to  specify  a building-block 
approach  that  is  also  directed  to  more  complex  courseware?  During  specifying  the  described  approach, 
we  found  that  finding  a sound  theoretical  basis  is  troublesome.  It  is  a challenge  to  embed  didactical 
models  and  instructional  principles  into  software-templates  that  are  content-empty.  Our  future  research 
will  be  directed  at  embedding  didactical  models  for  advanced  learning  goals,  using  principles  from 
learning  theories  like  constructivism  and  situated  cognition.  Our  goal  is  to  formulate  concrete 
guidelines  for  the  problem-based  or  case-based  approaches  from  such  theories,  and  implement  and  test 
these  within  an  advanced  version  of  our  current  building-block  approach. 
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Abstract:  A concept  has  been  developed  to  achieve  various  forms  of  collaborative  teleteaching, 
such  as  sharing  ideas  about  teaching,  using  learning  scenarios,  and  engaging  in  joint  teaching 
actions.  Professors  play  a key  role,  since  they  are  instrumental  in  integrating  new  technologies  into 
teaching  An  environment  is  being  developed  to  encapsuiate  an  instructional  design  technique  to 
facilitate  sharing  processes.  A database  technology  provides  distributed  services  over  the  Internet. 


From  Teaching  to  Teleteaching 

University  teaching  has  traditionally  been  defined  as  an  on-campus  activity  where  all  players  (1  professor  and  X 
students)  are  in  the  same  place  at  the  same  time,  with  or  without  any  media.  In  fact,  all  teaching  has  been  based  on 
these  parameters.  New  technology  (NT)  brings  many  potential  changes  to  these  parameters,  since  it  creates  an 
opportunity  for  innovative  distributed  teaching  modes.  This  potential  has  increased  the  need  for  instructional 
design,  and  consequently  the  progressive  transformation  of  traditional  teaching  modes  into  instructional  design- 
based  teaching.  Integrating  NT  into  teaching  implies  changing  teaching  practices.  Integration  has  taken  various 
forms  in  higher  education,  such  as:  Web-based  instruction,  online  education  (www.teleleam.ca);  videoconference- 
based  teaching  (Bourdeau  & al.,  1998);  the  virtual  campus  (www.tecfa.unige.ch),  or  integrative  distance  learning 
environments  (www.licef.teluq.uquebec.ca).  Satisfaction  levels  in  students  and  faculty  vary  from  enthusiastic  to 
negative.  Faculty  is  being  pressured  to  digitize  material  in  the  form  of  electronic  books,  or  add  electronic 
communication  to  their  classroom  teaching.  However,  teaching  continues  to  be  based  on  the  synchronization  of 
professor  and  students  in  a given  space.  There  is  room  for  integration  modes,  which  could  be  better  anchored  in 
current  practices,  while  simultaneously  fostering  greater  faculty  involvement. 

PETRUS:  The  Concept 

Petrus  is  a new  concept  developed  to  integrate  new  technology  into  university  teaching.  The  fundamental 
characteristic  underlying  the  idea  is  to  consider  professors  the  driving  force  behind  educational  integration,  since 
they  are  the  agents  who  design  teaching  specifically  for  their  own  students.  The  integration  of  new  technology 
derives  its  meaning  (direction  and  semantics)  from  their  teaching  and  personal  interpretation.  The  second  one  is  to 
consider  new  technology  a catalyst  rather  than  a tool:  the  spirit  of  the  Internet  is  all  about  sharing  and  cooperation. 
A good  number  of  professors  are  clearly  prepared  to  integrate  this  spirit  into  their  teaching.  And  lastly,  the  third 
characteristic  of  Petrus  is  to  provide  professors  with  a computerized  environment  where  the  groundwork  for  an 
instructional  design  technique  has  already  been  laid  through  scenario-building.  Depending  on  their  preferences  or 
needs,  professors  can  use  the  ScenarioBuilder  in  Petrus  to:  1)  assemble  dynamic  instructional  scenarios,  2)  gain 
shared  access  to  comprehensive,  validated  scenarios,  3)  select  individual  components  from  scenarios. 

Implementation  Plan  and  Work  in  Progress 

Implementing  sharing  practices  requires  sharing  processes  and  an  environment  designed  to  support  these  processes. 
Scenario-building  skills  can  be  acquired  through  a high-quality  system  where  instructional  principles  have  been  built 
into  an  expert  environment.  Professors  can  access,  consult,  edit  or  assemble  scenarios  in  the  most  flexible  ways. 
They  also  have  the  option  of  designing  their  own  teaching  environment  and  learning  interface  for  students. 
Teleteaching  services  naturally  have  to  be  anchored  in  current  teaching  practices  and  jointly  designed  with 
professors.  The  following  plan  was  designed  for  a course  on  learning  theories:  1)  Collect  and  analyze  a series  of 
syllabi  on  learning  theories;  2)  Extract  and  classify  items  according  to  objectives,  content,  leaming/evaluation 
activities  and  material;  3)  Conduct  a needs  assessment  of  useful  teleteaching  services;  4)  Choose  software 
engineering  solutions;  5)  Develop  and  test  a prototype  for  one  course;  6)  Develop  and  test  a prototype  for  five 


courses  on  distributed  sites;  7)  Develop  a full  environment  including  rights  and  privileges.  The  plan  is  currently 
under  way  after  a team  of  professors  agreed  to  jointly  design  and  test  the  scenario-builder.  Steps  1 through  5 have 
been  completed,  and  a Web-based  prototype  environment  is  being  developed.  The  environment  will  be  served  by  a 
database  that  dynamically  feeds  information  into  the  system.  First  results  will  provide  information  on  the  quality  and 
level  of  interest  in  collaborative  teleteaching  for  a course  on  learning  theories  in  teacher  education  programs. 

Related  Projects 

To  the  best  of  our  knowledge,  the  concept  that  comes  closest  to  Petrus  is  ARIADNE,  which  stands  for  Alliance  of 
Remote  Instructional  Authoring  Distribution  Networks  for  Europe  (http://ariadne.unil.ch/).  The  project  focuses  on 
the  development  of  tools  and  methodologies  for  producing,  managing  and  re-using  computer-based  educational 
components  and  curricula.  Collaborative  approaches  are  encouraged,  as  is  the  share-and-reuse  principle  (Duval, 
1999).  Petrus  is  radically  different  in  that  professors  are  at  the  centre  of  all  processes;  they  are  the  agents  through 
whom  everything  passes,  and  both  their  personality  and  teaching  style  are  incorporated  into  the  design,  Petrus  is  not 
looking  to  set  standards,  but  seeks  diversity  instead  in  terms  of  intellectual  perspectives  and  teaching  styles. 
Secondly,  Petrus  contains  a scenario-builder  that  incorporates  teaching  strategies  which  can  also  be  shared.  The 
STAR.Legacy  project  (http://peabody.vanderbilt.edu/ctrs/ltc/)  at  Vanderbilt  University  has  some  features  similar  to 
those  of  Petrus:  the  learning  cycle  resembles  the  Petrus  Scenario-Builder  and,  and  the  “multiple  perspectives” 
activity  in  the  STAR  Legacy  learning  cycle  is  characteristic  of  a team-teaching  situation.  The  working  hypothesis 
underlying  the  Petrus  project  is  as  follows:  the  successful  integration  of  digital  technologies  into  university  teaching 
must  go  through  the  professor  — an  expert  in  a particular  domain  who  is  responsible  for  conveying  information, 
inspiring  and  coaching  students  to  help  them  achieve  their  personal,  academic  and  career  goals.  Professors  need  to 
customize  the  learning  environment  for  their  students,  and  change  it  as  needed.  Professors  would  ultimately  be 
engaging  in  team  teleteaching.  Videoconferencing  and  video  servers  would  provide  a multitude  of  perspectives  for 
students,  thus  enabling  them  to  acquire  greater  discernment  and  critical  skills. 

Sharing,  Collaboration  and  Teleteaching 

In  the  context  of  computer-supported  collaborative  work,  sharing  is  seen  as  a sub-process  of  collaboration,  which  is 
itself  a combination  of  communication  and  coordination.  Coordination  science  defines  coordination  as  managing 
interdependencies,  and  this  concept  can  be  extended  to  the  management  of  interdependencies  among  actors 
(Bourdeau  & Wasson,  1997).  In  the  Petrus  project,  sharing  processes  can  be  identified  together  with  their  associated 
interdependencies.  The  software  environment  would  be  built  around  these  interdependencies  and  provide  the  means 
to  manage  them.  Examples  of  sharing  processes  include:  sharing  ideas,  scenarios,  teaching  or  coaching  activities 
and  lessons  learned.  Mentoring  (at  the  novice  or  expert  level)  can  also  be  considered  a sharing  process.  Examples 
of  sharing  principles  include:  reciprocity,  privileges,  task  distribution,  resource  allocation,  and  so  on.  Software  to 
manage  and  support  sharing  processes  will  be  designed  when  sharing  processes  are  identified  and  validated,  and 
means  for  managing  interdependencies  are  developed.  Ethical  and  legal  issues  will  also  be  taken  into  account. 
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Abstract:  This  paper  describes  the  work  of  the  Carlos  III  University  in  the  project  SEBASTIAN 
(Educational  System  based  on  Internet  Technology).  The  aim  is  to  develop  a tele-educational  tool, 
providing  to  educators/trainers  with  a simple  mechanism  that  makes  possible  the  use  of  the  network 
for  the  broadcast  of  educational  content  and  enabling  the  creation  of  multilingual  courses  based  on 
XML  (Bray,  1998)  and  SMIL  (Stephan,  1998)  technologies. 


Introduction 

As  soon  as  web  technologies  first  became  available,  they  were  adopted  for  educational  purposes.  Hypertext 
technologies  had  already  been  used  in  learning  systems,  and  the  Internet  facilitated  the  overcoming  of  distance. 
Higher  education  is  now  undergoing  structural  changes  in  terms  of  the  composition  of  student  populations,  learning 
paradigms  and  curricula.  As  distance  education  becomes  an  integral  part  of  higher  education,  student  bodies  are 
expanding  to  include  more  non-traditional  students  and  the  contents  of  courses  are  richer  than  before,  and 
incorporate  more  interactivity. 

In  this  new  world,  students  participate  more  actively  in  class  and  collaboration  becomes  a more  important 
component  of  the  learning  process.  These  changes  have  stimulated  the  development  of  new  ways  to  represent 
information  (for  example:  animations,  simulation,  hypertextual  navigation  and  so  on).  In  particular,  multimedia 
content  has  been  introduced  via  the  medium  of  the  WWW.  However,  there  are  problems  associated  with  the  variety 
in  present  day  formats.  Facing  up  to  this  situation.  The  Carlos  III  University  of  Madrid  is  working  on  a project 
named  SEBASTIAN. 

System  Architecture 

The  system  serves  to  integrate  multimedia  learning  experiences  into  the  network  enabling  it  to  be  visualized 
according  to  the  preferences  and  characteristics  of  the  student.  The  courses  generated  are  described  in  EML  2.0 
(Educational  Markup  Language  2.0).  This  language  is  based  on  XML  and  includes  spatial  information  (multimedia 
objects  place,  presentation  size,  etc.),  temporal  information  (duration  of  multimedia  objects,  scenes,  etc.),  and 
information  about  students  profiles  (learning  styles).  The  professor  will  eventually  have  one  tool  with  three 
integrated  functionalities  for  the  creation  of  courses,  see  [Figure  1].  The  functions  are: 

• LG  (Layout  Generation):  Allowing  the  description  of  the  spatial  distribution  of  multimedia  elements  that 
comprise  each  scene  of  the  presentation.  A description  language,  SLML:  Space  Layout  Markup  Language 
(based  on  XML),  has  been  designed  for  this  purpose.  This  Rinctionality  will  provide  the  creation  of 
multimedia  templates.  Initially,  some  predefine  templates  will  exist,  but  the  object  is  that  professor 
generated  its  own  templates. 


H C:  r\  n 


• STG  (Synchronous  Text  Generation):  The  files  generated  are  based  on  a new  language  STML: 
Synchronization  Text  Markup  Language,  and  provide  textual  content  in  several  languages.  These  files 
contain  synchronization  labels  which  synchronize  the  text  with  audio  or  video,  as  in  a karaoke  session. 


• CG  (Course  Generator):  Integrates  multimedia  files  and  files  generated  by  the  two  previous  applications. 
The  result  is  a presentation  composed  of  a sequence  of  "abstract  scenes".  Each  scene  is  a compound  of  the 
different  presentations  that  may  be  presented  to  different  students.  The  view  presented  to  each  student 
depends  on  their  preferences  and  pedagogic  profile.  These  profiles  are  generated  automatically  in 
accordance  with  the  Learning  Style  Model  (Richard  Felder,  1996),  the  PAPI  (Public  and  Private 
Information,  2000)  and  the  LOM  (learning  Object  Metadata  1999)  specifications. 
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Abstract:  This  paper  describes  an  on-going  project  that  involves  the  construction  and  evaluation  of 
an  Internet-based  Knowledge  Integration  System  (IKIS)  developed  to  support  a Computer-assisted 
Instruction  (CAI)  graduate  course.  The  IKIS  integrates  cognitive  theories  of  learning  and 
instruction,  computer  networking  technologies,  and  face-to-face  classroom  teaching  in  order  to  (a) 
help  students  build  coherent  knowledge  representations  of  a great  variety  of  knowledge  and  skills 
that  are  required  to  design  and  produce  instructional  software  prototypes  and,  (b)  create  a 
community  of  practice  to  aid  students  in  making  the  decisions  concerning  the  design  and 
production.  Our  discussion  will  be  focused  on  two  important  system  features  of  the  IKIS  (i.e.,  the 
information  architecture  and  communication  facilities)  that  enable  us  to  achieve  such  goals.  A plan 
for  evaluating  the  IKIS  will  also  be  briefly  described. 

In  this  paper,  we  will  describe  an  on-going  project  that  involves  the  construction  and  evaluation  of  an  Internet- 
based  Knowledge  Integration  System  (IKIS)  developed  to  support  a Computer-assisted  Instruction  (CAI)  graduate 
course  in  which  students  will  develop  knowledge  and  skills  for  designing  and  producing  instructional  software 
prototypes.  Understanding  of  various  design  concepts  and  approaches,  specification  techniques,  authoring  tools,  and 
development  methods  is  emphasized  within  this  course.  Whereas  the  instructor  of  the  course  provides  face-to-face 
lectures,  demonstrations,  discussions,  and  laboratory  work,  the  complex  nature  of  the  subject  can  still  pose  a 
challenge  for  students  to  assimilate  and  integrate  such  knowledge.  Furthermore,  traditional  classroom  instruction 
remains  an  inadequate  method  in  providing  timely  instruction  to  each  individual  student.  The  IKIS  for  (CAI)  was 
constructed  to  (a)  provide  students  with  a rich,  well-structured  knowledge  base  in  order  to  ensure  their  active 
assimilation  and  integration  of  a variety  of  knowledge  and  skills  to  be  learned,  and  (b)  create  a community  of 
practice  to  aid  students  in  making  the  decisions  concerning  the  design  and  production  of  software  prototypes. 

The  Design  of  the  IKIS 

The  IKIS  integrates  cognitive  theories  of  learning  and  instruction,  computer  networking  technologies,  and 
face-to-face  classroom  teaching  to  enable  students  to  use  the  well-structured  knowledge  base  and  diverse  tutorial 
resources  to  build  coherent  knowledge  representations  of  the  subject-matter  knowledge  to  be  learned.  The  design  of 
the  IKIS  is  based  on  the  cognitive  theories  of  text  comprehension  that  emphasizes  the  importance  of  structured 
knowledge  presentations  in  the  construction  of  coherent  knowledge  representations  (Kintsch,  1988;  1998;  Mayer, 
1999;  1997),  and  on  situated  theory  that  views  social  interaction  and  collaboration  as  crucial  factors  in  the 
development  of  scientific  understanding  (Lave  & Wenger,  1991;  Resnick,  Levine,  & Teasley,  1991).  In  addition,  the 
cognitive  apprenticeship  model  proposed  by  Collins,  Brown,  and  Newman  (1989)  also  informed  the  design  of  the 
IKIS.  Five  principles  are  articulated  to  guide  the  design  of  the  IKIS:  (a)  providing  a rich  knowledge  base  to  reflect 
expertise  in  the  domain  of  instructional  software  design;  (b)  providing  a well-organized  and  transparent  knowledge 
structure  to  help  students  construct  a coherent  conceptual  representation  of  the  subject-matter  knowledge,  (c) 
employing  an  effective  model  of  instruction,  (d)  building  a community  of  expert  practice  and  providing  multiple 
perspectives  towards  the  solutions  and,  (e)  facilitating  collaborative  learning  among  students.  The  theoretical 
foundation  and  design  principles  of  the  IKIS  have  been  described  in  details  elsewhere  (Chen,  1999;  Chen,  2000).  In 
this  paper,  our  discussion  will  be  focused  on  two  important  system  features  of  the  IKIS,  namely  the  information 
architecture  and  communication  facilities  that  reflect  the  principles  described  above.  In  addition,  an  evaluation  that 
we  will  conduct  in  the  near  future  will  also  be  proposed. 

The  information  architecture  incorporated  into  the  IKIS 

The  CAI  graduate  course  is  focused  on  the  user-centered  design  for  Computer-supported  Collaborative  Learning 
(CSCL).  A series  of  topics  were  identified  as  important  for  successful  design  and  implementation  of  instructional 
software  prototypes.  In  order  to  help  students  construct  coherent  knowledge  representations,  the  IKIS  incorporates 
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an  information  architecture  (i.e.,  a hierarchical  organizational  scheme)  to  represent  two  “layers”  of  the  knowledge  to 
be  learned:  the  first  layer  consists  of  various  topics,  which  range  from  learning  theories  to  web  authoring  techniques 
whereas  the  second  layer  is  a breakdown  of  knowledge  on  each  topic  which  includes  concept,  rationale,  goals, 
methods,  example,  related  issues,  trouble  shooting  and  tips,  and  reflective  questions  and  exercises.  Such  a 
knowledge  base  is  expected  to  enable  the  students  to  answer  questions  concerning  what,  why,  how,  to  evoke 
reflective  thinking,  and  ultimately,  to  create  a rich  context  for  students  to  engage  in  productive  group  discussions  on 
the  topics  via  computer-mediated  conferencing.  Furthermore,  some  Internet  resources  and  lab  tutorials  that  are 
relevant  to  specific  topics  are  also  provided  to  extend  the  knowledge  base  of  the  IKIS.  Students  can  also  access  a 
perquisite-course  website  to  acquire  background  knowledge  if  they  wish.  The  information  architecture  of  the  IKIS 
enables  the  students  to  learn  the  topics  in  a logical  manner,  meanwhile  this  architecture  is  flexible  enough  to  allow 
each  individual  to  take  a different  navigation  path  based  on  his  or  her  own  selection.  There  is  a navigation  bar 
located  on  the  ton  of  the  screen  that  allows  students  to  maneuver  through  the  course  topics  and  brcakdowTi  by 
following  either  a pre-determined  sequence  or  a selected  path.  In  addition,  a bottom  navigation  bar  is  available  all 
time  to  enable  the  students  to  access  a computer-mediated  conferencing  facility,  newsgroup  discussions,  and  e-mail 
addresses  in  the  context  of  reviewing  the  course  topics  and  breakdown.  This  bottom  navigation  bar  also  provides 
accesses  to  the  resources  and  lab  tutorials  outside  of  the  instructional  content  that  is  defined  in  the  IKIS.  An  on-line 
help,  site  map,  and  search  facility  are  also  provided  by  the  bottom  navigation  bar  to  enable  students  to  navigate 
efficiently  through  the  site. 

The  communication  facilities  utilized  in  the  IKIS 

There  are  numerous  decisions  that  one  must  make  in  the  processes  of  designing  and  producing  instructional  software 
prototypes.  No  matter  how  rich  the  knowledge  base  is  and  how  well  such  knowledge  base  is  structured,  students 
may  still  have  unanswered  questions.  The  purpose  of  proving  the  accesses  to  a computer-mediated  conferencing 
facility,  newsgroup  discussions,  and  e-mail  addresses  is  to  enable  students  to  discuss  such  unanswered  questions 
with  their  peers  or  experts  outside  of  their  physical  environments.  Three  communication  channels  are  established  to 
support  students’  knowledge  construction  and  decision-making:  (a)  a channel  for  integrating  with  the  peers',  the 
FirstClass^^  software  program  is  used  to  provide  computer-mediated  conferencing  and  chat  facilities  to  facilitate  the 
discussions  among  students.  Each  student  is  required  to  provide  answers  to  the  reflective  questions  that  are  proposed 
at  the  end  of  each  topic.  Students  are  also  encouraged  to  bring  up  issues  that  they  wish  to  clarify  and  respond  to  each 
other’s  questions.  Students  can  also  the  FirstClass^^  to  exchange  electronic  files  that  they  produced  for  their  group 
work.  The  instructor  and  teaching  assistant  will  monitor  the  discussion  and  provide  directions  and  clarifications  if 
needed;  (b)  a channel  for  participating  in  the  communities  of practice'.  The  IKIS  provides  students  with  access  to  the 
Internet  newsgroups  that  are  most  relevant  to  the  topics  discussed  in  the  course.  These  newgroups  are  organized  in  a 
way  that  is  sensitive  to  the  course  topics  so  that  the  students  can  be  exposed  to  multiple  perspectives  on  a given 
issue.  Students  will  also  have  the  opportunities  to  share  their  experience  and  expertise  with  people  who  face  similar 
problems  and  understand  the  concerns  of  the  practitioners  in  the  community;  (c)  a channel  for  communicating  with 
experts:  In  addition  to  the  communication  with  the  instructor  of  the  course,  students  can  ask  questions  to  some 
design  experts  who  will  serve  as  the  mentors  during  the  course  of  the  semester.  These  experts  will  help  the  students 
diagnose  some  "unsolvable"  problems  and  provide  suggestions  via  e-mail.  In  addition,  a list  of  e-mail  addresses  of 
the  experts  in  the  fields  relevant  to  CAI  design  will  also  be  provided  to  allow  students  to  contact  these  experts.  The 
communication  with  these  experts  may  provide  students  with  insights  into  how  experts  think  and  practice.  Thus,  The 
communication  facilities  offered  by  the  IKIS  will  break  the  boundary  of  time  and  location  that  is  inherited  in 
traditional  teaching. 

The  Evaluation  of  the  IKIS 

The  IKIS  will  be  evaluated  both  formatively  and  summatively.  In  the  formative  evaluation  phase,  a few 
representative  students  will  be  asked  to  “walk  through”  the  environment  and  provide  “think  aloud”  protocols  for 
making  diagnosis  of  the  system  in  terms  of  its  usability  and  leamability.  Following  the  necessary  modifications,  a 
summative  evaluation  will  be  conducted  to  determine  the  effectiveness  of  the  IKIS.  Two  groups  of  students  will  be 
compared:  one  that  use  the  IKIS  and  another  that  use  a print-based  version  of  the  same  material.  An  attitude 
questionnaire  will  be  administered  before  and  after  the  course  to  determine  the  changes  in  student  attitudes.  In 
addition,  students’  on-line  discussions  and  course  work  will  also  be  analyzed  to  determine  the  levels  of  expertise 
developed.  The  ultimate  goal  of  the  project  presented  here  is  to  understand  the  ways  that  computer  technologies  can 
be  used  to  enhance  student  learning  of  complex  subject-matter  knowledge  and  to  develop  a model  of  successftilly 
integrating  technology  into  the  classroom  teaching. 
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1.  introduction 

Although  the  increasing  sophistication  of  computer-based  learning  systems  has  enabled  gains  to  be  made  in 
terms  of  their  pedagogical  effectiveness,  an  undesirable  consequence  of  this  sophistication  has  been  a significant 
increase  in  development  cost.  The  high  cost  of  these  systems  has  been  identified  as  a principal  barrier  to  their 
widespread  deployment.  Increased  complexity  in  learning  systems  has  also  led  to  lengthier  development  cycles, 
which  has  constrained  the  rate  of  design  evolution  and  innovation.  As  a result,  there  is  a pressing  need  for 
instructional  designers  to  focus  on  the  efficiency  of  their  development  processes  as  well  as  the  effectiveness  of  the 
outputs  that  are  produced  by  them. 

A possible  strategy  for  reducing  the  time  and  cost  of  system  development  is  to  create  a context  wherein 
whole  systems  would  not  have  to  be  designed  largely  from  scratch.  A key  aspect  of  such  a strategy  could  be  the 
establishment  of  cross-organizational  standards  that  would  allow  developers  to  reuse  content  components  from  other 
systems,  both  from  within  their  own  organization  and  from  external  sources.  In  essence,  this  would  represent  an 
attempt  to  leverage  the  productivity  gains  from  reusability  that  have  been  achieved  in  software  engineering  into  the 
domain  of  instructional  content  development.  Over  the  past  year,  coordinated  proposals  have  rapidly  emerged  from 
several  bodies  (e.g..  Advanced  Distributed  Learning  Initiative,  2000;  Anderson,  T.,  McArthur,  D.,  Griffen,  S.  & 
Wason,  T.,  1999)  as  candidate  standards  for  this  purpose.  As  these  proposals  are  vetted  through  their  initial  trials  in 
actual  learning  systems,  it  will  be  important  to  observe  how  standards  for  content  reusability  affect  instructional 
content  development  methodologies.  Will  processes  primarily  developed  in  the  context  of  authoring  content  in 
small-scale  closed  systems  evolve  smoothly  into  processes  for  architecting  large-scale  distributed  learning 
environments  where  the  number  of  distinct  content  components  may  be  in  the  hundreds  or  thousands?  Conversely,  if 
new  processes  must  be  developed,  which  ones  will  be  required,  and  on  what  basis  do  we  start  their  development? 

A fundamental  issue  that  needs  to  be  addressed  in  this  regard  is  the  broad  function  of  instructional  design 
(ID)  in  the  development  of  systems  that  have  a substantial  information  technology  (IT)  component.  Where  does  ID 
stop  and  IT  start?  Consider  the  development  of  a set  of  interactive  practice  exercises  that  include  variable  remedial 
feedback  based  on  learner  response  patterns.  A design  decision  could  be  made  to  use  a database-driven  approach 
wherein  feedback  is  assembled  from  low-level  content  components  that  are  sourced  from  a database.  If  these  low- 
level  components  include  subject-matter  terms  and  their  definitions,  the  same  database  might  be  used  to  drive  a 
glossary  function  in  the  learning  environment,  serving  the  objective  of  content  reusability.  In  designing  such  a 
system,  we  would  likely  not  expect  an  IT  professional  to  author  the  subject-matter  definitions  that  lie  within  the 
database,  just  as  we  would  not  expect  the  ID  professional  to  design  the  software  that  allows  the  database  to 
communicate  within  a client-server  environment.  Appropriate  allocation  of  responsibility  for  many  other  design 
issues,  however,  is  not  as  clear-cut.  For  example,  is  one  of  these  professionals  primarily  responsible  for 
decomposing  the  feedback  into  suitable  lower-level  components,  or  is  this  a joint  responsibility  - and,  if  so,  how  do 
these  professionals  collaborate  on  this  task  effectively?  Of  specific  concern  are  the  pedagogical  implications  of 
successful  technical  convergence  (e.g.,  the  use  of  database-driven  content  systems,  or  the  adoption  of  XML  as  a 
markup  language):  does  this  form  of  standardization  support  research  on  and  development  of  pedagogical  strategy, 
or  does  it  sacrifice  flexibility  at  this  level  to  achieve  the  goal  of  technical  convergence?  The  absence  of  a clear 
articulation  of  the  interface  between  pedagogy  and  technology  in  practical  terms  may  prove  problematic. 

2.  Common  architectural  issues  in  the  design  of  computer-based  learning  systems  using 
modular  content 


To  begin  assessing  the  viability  of  current  content  development  process  models  in  the  context  of  large-scale 
distributed  learning  environments,  we  can  examine  common  categories  of  design  decisions  that  occur  in  smaller- 
scale  development  projects  and  assess  the  possible  ramifications  of  a significant  change  in  scale  on  them: 

■ Modularity:  What  is  a module,  in  terms  of  instructional  content?  What  are  the  quality  criteria  for  the 
definition  of  ‘good’  modules?  Do  these  criteria  change  as  scale  increases?  How  do  we  use  partitions 
between  modules  to  manage  risk  in  content  development?  For  example,  how  do  we  partition  a system  in 
order  to  increase  our  ability  to  test  numerous  content  components  as  they  are  being  developed? 

■ Communication  and  information  flow:  What  types  of  information  get  communicated  between  modules  in  a 
single  system?  Where  are  the  architectural  boundaries  in  an  open  content  system?  How  does  the 
information  flow  in  a learning  environment  instantiate  a pedagogical  theory  of  control? 

■ Object-  and  component-based  approaches  to  the  development  of  instructional  materials:  How  do  we 
determine  the  right  grain  sizes  for  content  development?  What  is  the  relationship  between  grain  sizes  for 
authoring  and  the  units  of  analysis  employed  during  front-end  analysis?  How  can  the  choice  of  content 
units  facilitate  collaboration  with  software  developers  during  design  and  production? 

■ Reusability:  How  is  the  design  of  a single  reusable  content  component  different  from  the  design  of  an 
instructional  product  out  of  reusable  components?  What  parts  of  a learning  environment  can  be  reused? 
How  do  we  move  from  ad-hoc,  opportunistic  reuse  to  systematic  re-use?  What  are  the  tradeoffs  involved? 
Can  there  be  too  much  reuse? 

■ Interoperability:  At  what  levels  is  it  feasible  to  make  content  interoperable?  What  goals  are  achieved  by 
making  content  interoperable?  What  is  the  relationship  between  reusability  and  interoperability? 

■ System  maintenance  and  evolution:  How  do  we  design  system  architecture  so  that  it  can  survive,  and 
indeed  facilitate,  bottom-up  evolution  of  the  system?  What  kind  of  evaluation  data  should  the  system 
collect,  and  how?  How  do  we  document  system  architecture  so  that  it  is  understandable  by  those  who  will 
be  responsible  for  its  maintenance  and  evolution? 

■ Coordination:  What  happens  when  we  move  from  managing  individual  content  development  projects  in 
relative  isolation  to  managing  integrated  content  development  programs?  Which  design  assumptions  are 
challenged  when  discrete  learning  products  become  parts  of  continuous  learning  systems? 

3.  The  identification  and  categorization  of  design  decisions 

When  a design  domain  is  relatively  stable,  the  explicit  articulation  of  design  decisions  is  less  important, 
since  decisions  and  their  consequences  are  reasonably  predictable  based  on  past  experience.  Process-level  issues 
can  generally  be  resolved  on  a case-by-case  basis  in  such  circumstances.  As  the  transition  is  made  to  developing 
instructional  content  components  for  use  in  distributed  learning  systems,  instructional  designers  may  face  a period 
of  domain  instability  where  novel  and  complex  interactions  force  many  tradeoffs  to  be  made  unwittingly. 
Reassessing  and  evolving  processes  in  light  of  these  challenges  will  require  an  understanding  of  the  new  design 
space  that  instructional  content  developers  are  working  within.  In  part,  the  opportunities  and  constraints  associated 
with  this  new  context  will  have  to  emerge  through  hard  experience.  A viable  approach  for  articulating  the 
boundaries  and  substance  of  this  space  in  advance,  however,  is  to  identify  and  categorize  the  macro-  and  micro- 
decisions that  are  made  in  developing  small-scale  modular  content,  and  examine  these  decisions  in  terms  of  their 
scalability  to  larger,  more  integrated  systems.  This  is  the  subject  of  my  current  work-in-progress.  By  examining  the 
development  process  from  the  perspective  of  decisions  that  are  made  during  design,  a top-down,  principle-based 
approach  to  learning  system  architecture  can  be  combined  with  a bottom-up  awareness  of  the  individual  decisions 
designers  make  on  actual  projects.  This  is  a key  stepping-stone  in  the  analysis  of  the  cohesiveness  and 
comprehensiveness  of  a given  set  of  instructional  development  processes.  The  design  decisions  can  also  serve  to 
frame,  in  tangible  terms,  the  relationship  between  a given  pedagogical  strategy  and  its  technical  implementation. 
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Abstract:  Concerned  with  the  relationship  between  interaction  and  learning,  this 
paper  aims  at  exploring  relations  between  interactive  flows  in  an  EFL  course 
offered  on  the  Internet  and  language  learning.  The  course  was  produced  and  is 
run  by  the  research  team  of  EDULANG,  an  ongoing  research  project  in 
language  learning  on  the  Web  and  teacher  development  for  the  Web.  In  this 
research  context,  interactive  flows  are  being  used  to  record  and  analyze  the  rate 
and  type  of  interaction  in  the  course,  identify  units,  tasks  and  course  spaces  that 
do  not  promote  expected  levels  of  interaction  and  plan  and  promote 
improvement  in  course  structure,  course  content  and  teaching  strategies. 

Aims  and  Context 

This  paper  derives  from  ongoing  research  in  the  context  of  EDULANG,  a team  project  that  aims  at  investigating 
multiple  aspects  of  language  learning  and  teaching  in  courses  on  the  Internet.  The  paper  is  more  specifically 
concerned  with  the  relationship  between  interaction  and  learning  and  aims  at  exploring  relations  between  interactive 
flows  and  language  learning.  An  interactive  flow  is  here  defined  as  a longitudinal  profile  depicting  amount  (no.  of 
messages  or  interactive  turns)  and  direction  (sender  and  recipient)  of  synchronous  and/or  asynchronous 
participation  in  an  online  course.  The  context  of  the  research  is  an  EFL  course  offered  on  the  Internet 
(www.cogeae.uol.com/sal).  The  course  was  produced  and  is  run  by  the  research  team  of  EDULANG.  Aimed  at 
adults  with  a basic  command  of  English,  its  general  social  aim  is  to  offer  students  the  opportunity  to  make  new 
acquaintances  on  the  Internet.  This  general  aim  unfolds  into  specific  social  and  language-specific,  communicative 
functions.  The  course  officially  lasts  8 weeks  but  can  in  fact  reach  12  to  13  weeks.  The  interactive  spaces  make  it 
possible  for  participants  to  a)  communicate  through  messages  posted  to  a virtual  Bulletin  Board  (BB)  subdivided 
into  three  forums  (Classroom,  Cafe  and  Technical  Support),  b)  display  (and  re-edit)  their  interactive  production  to 
course  peers  in  a Production  Panel,  c)  send  progress  tasks  and  tutorial  exercises  to  the  teacher  through  mail -to 
devices  on  the  course  pages  and  d)  participate  in  online  Chats.  All  communication  is  carried  on  in  English,  except 
for  messages  left  in  the  Technical  Support  forum.  As  an  essential  element  of  learning  and  knowledge  construction, 
interaction  must  be  evaluated  for  type,  effect  and  efficiency  in  courses  on  the  Internet.  In  this  research  context, 
interactive  flows  are  being  used  to  record  and  analyse  the  rate  and  type  of  interaction  in  the  course,  identify  units, 
tasks  and  course  spaces  that  do  not  promote  expected  levels  of  interaction  and  plan  and  promote  improvement  in 
course  structure,  course  content  and  teaching  strategies. 

Method 

Data  was  collected  from  indicators  about  sender,  recipient,  date  and  subject  of  6 groups  of  students  and  their 
respective  teachers,  in  1998  and  1999,  and  include  a)  messages  posted  in  the  Bulletin  Board,  b)  messages  with 
ongoing  tasks  sent  to  the  teacher,  c)  texts  displayed  in  the  Production  Panel  and  d)  participation  in  online  chat 
sessions.  All  indicators  were  available  within  the  course  system. 

Some  Findings 

Results  were  organized  in  order  to  reveal  graphic  flows  of  interaction  involving  students  and  peers,  students  and 
teacher,  teacher  and  students.  These  flows  reveal  trends  within  each  group  as  well  as  across  groups.  Interpreted  in 


the  light  of  the  course  context,  the  graphic  flows  below  have  offered  support  for  pedagogical  and  managerial 
decisions,  as  well  as  for  changes  in  design.  Figure  1 indicates  that  in  this  course,  the  amount  of  student  talk  tends  to 
be  equivalent  to  or  higher  than  teacher  talk.  This  may  mean  that  in  an  interactive  internet  course,  students  engage  in 
more  interaction  than  students  in  traditional  face-to-face  EFL  courses.  Figure  2 shows  that  different  groups  present 
different  types  of  flows,  either  with  a focus  on  teacher-student  and  student-teacher  interaction  or  with  a focus  on 
communication  among  students.  In  groups  where  the  teacher  talks  less,  interaction  among  students  tends  to  be 
higher.  In  this  sense,  the  amount  of  teacher  interaction  with  students  influences  the  amount  of  students'  interaction 
with  their  peers.  Additionally,  Figure  2 shows  that  groups  A,  B,  C and  D (taught  in  1 998)  communicate  a lot  less  in 
the  BB  than  groups  C and  D (taught  in  1999).  This  suggests  a development  of  familiarity  and  comfort  with 
educational  contexts  on  the  Web  for  both  students  and  teachers.  However,  relatively  close  averages  of  participation 
in  the  1 998  groups  (1 1 .07,  5.53,  8.45  and  1 1 .95)  and  equivalent  participation  in  the  1 999  groups  (28.28  and  28.38), 
suggest  that  students  seem  to  have  a quantitative  communicative  quota,  that  they  strive  to  use,  regardless  of  the 
interlocutors  they  choose  to  have  (the  teacher  or  a peer).  Figure  3 allows  an  interpretation  of  the  interactive  flows  in 
the  BB  in  the  light  of  course  content  distribution  (weeks  1-3  for  unit  1,  weeks  4-5  for  unit  2 and  weeks  6-8  for  unit 
3).  A higher  state  of  motivation  seems  to  be  associated  to  specific  phases  of  the  course  when  participants  are  all 
getting  to  know  each  other  (the  beginning)  or  students  are  trying  to  close  their  participation  successfully  (the  official 
end,  the  8^^  week).  Mid  course  activities  have  not  been  able  to  sustain  adequate  levels  of  interaction  (notice  the 
relative  drop  both  in  the  98  and  99  groups)  and  are  now  being  thoroughly  reformulated.  Figures  4 and  5 offer  data 
for  interesting  observations.  Groups  A and  B seem  to  reinforce  the  quantitative  communicative  quota  hypothesis, 
with  task  completion  and  BB  message  flows  in  a complementary  relationship  (lower  averages  in  one  flow  type 
complement  higher  averages  in  the  other).  Groups  C and  D,  on  the  other  hand,  present  opposite  results  in  both  types 
of  flows  and  suggest  that  teacher  management  has  a big  influence  on  task  completion  and  student  participation  in  the 
BB.  Figure  6 reveals  that  among  students  who  completed  at  least  50%  of  tasks  there  are  more  poor  “chatters”  (23) 
than  good  “chatters”  (13).  However,  among  the  best  “chatters”  (more  than  30  turns)  were  the  best  task  completion 
rates.  This  suggests  that  completing  tasks  (a  traditional  activity)  is  still  favored  over  other  web-typical  types  of 
participation  and  that  an  intensive  participation  in  chat  sessions  is  usually  associated  to  intensive  task  completion. 


Fig  1:  Interactive  Flows  In  the  Bulletin  Board  between 
Teachers  and  Students  in  6 Groups 
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Figure  1 : BB  Messages  between  Teachers  and  Students 


Fig  3:  Interactive  Flows  of  6 groups  in  the  Bulletin  Board 
during  15  weeks 
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Figure  3:  Flows  of  6 groups  in  the  BB  during  15  weeks 
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Fig  2:  Interactive  Flows  in  the  Bulletin  Board  between 
teachers  and  students  and  among  students  in  6 groups 
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Figure  2:  BB  - teachers/students  and  student/student 


Figure  4:  Task  Completion  Flows  in  4 groups  (averages) 


Figure  S:  BB  Messages  Flows  in  4 groups  (averages) 
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Figure  6:  Chat  and  Task  Flows 
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Abstract:  This  paper  describes  a study  LhaL  is  being  deveioped  to  explore  online  collaboration  between 
undergraduate  journalism  students  at  universities  in  Australia,  Canada,  New  Zealand,  the  United 
Kingdom  and  the  United  States.  The  premise  of  the  study  is  that  by  bringing  students,  instructors  and 
experts  together  to  examine  different  perspectives  of  international  news  stories,  learners  will  develop  a 
stronger  understanding  of  stories  that  get  international  coverage.  The  goal  of  the  research  is  to  develop  a 
consensual  and  collaborative  module  that  explores  international  perspectives  on  global  news  and  to 
present  this  module,  online  ,to  teams  of  students  at  the  partner  universities.  Data  will  be  gathered  from 
online  interviews  and  questionnaires  given  before  and  after  the  module. 


Introduction 

In  the  past  decade  there  has  been  a growing  amount  of  research  in  journals,  books  and 
conference  papers  in  the  area  of  online  learning.  Online  learning,  for  the  purpose  of  this  research, 
refers  to  study  that  is  done  using  computer  technologies  with  the  learner  at  the  same  location  or  at  a 
distance  from  the  instructor. 

Many  established  universities,  such  as  Stanford  in  the  United  States,  Deakin  in  Australia, 
University  of  Twente  in  the  Netherlands,  offer  online  classes  to  students  who  live  at  a distance  from 
campus,  to  learners  studying  part-time  and  to  professionals  engaged  in  life  long  learning.  These  classes 
may  have  an  international  reach  but  are  run  and  controlled  entirely  by  individual  universities.  The 
Open  University  in  the  United  Kingdom  and  the  University  of  Phoenix  in  the  United  States  are  two 
examples  of  universities  that  are  completely  online.  All  students  study  from  a distance  but  students  are 
still  studying  at  one  university  and  working  within  the  structure  and  policies  of  that  system. 

Most  of  the  online  classes  described  above,  involve  students  working  alone  online  with  some 
support  from  instructors  or  tutors  via  e-mail  but  with  little  collaboration  amongst  students  themselves. 
Collaborative  learning,  as  defined  for  this  study  is:  learning  in  an  educational  program  created  by  a 
group  of  partner  universities  and  in  which  students  and  instructors  get  together  to  examine  course 
content  issues.  This  sort  of  interactive,  online  learning  is  becoming  more  widely  practised.  Medical 
schools  throughout  Germany  (Friedl,  99)  share  online  resources  to  study  cardiac  surgery.  An  online 
training  for  trainers  course  involved  learners  in  22  European  countries  (Lei no,  99).  Organizations  such 
as  the  United  Nations  are  using  online  modules  to  give  professionals  around  the  world  a chance  to 
study  copyright  law  (Durant,  99). 

So  far,  in  this  researcher’s  literature  search,  most  examples  of  online  collaboration  involve 
engineering,  science,  medical  and  business  programs  and  most  of  the  collaborations  are  developed  at 
one  university.  Telg  (96)  noted  that  in  the  United  States  “few  collaborative  distance  education  efforts 
have  taken  place  between  American  universities  and  universities  in  other  countries”. 

In  this  researcher’s  area  of  interest,  broadcast  journalism,  there  is  little  documentation  in  the 
literature  of  any  programs  using  online  collaboration  as  a learning  tool. 


It  is  becoming  easier  and  easier  for  journalists  to  quickly  gather  information  from  any  part  of 
the  world  and  disseminate  it  to  any  other  part  of  the  world  but  McGregor  (98)  asks  is  “the  news  is 
better  reported  with  more  understanding  as  a result  of  the  new  technologies”.  We  are  sending  young 
broadcasters  into  a world  of  instant  information,  instant  analysis  and  instant  conclusions  about  global 
news  issues.  One  of  the  questions  raised  by  this  study  is:  do  those  young  journalists,  and  the 


Study 


experienced  story  tellers  for  that  matter,  have  the  enough  understanding  of  global  issues  to  provide 
clear,  non-biased  reports? 

The  premise  of  this  study  is  that  by  linking  students,  instructors  and  broadcast  experts  in 
online  discussion  and  by  working  in  international  teams  on  assignments  and  presentations  concerning 
global  news,  the  learners  will  develop  a stronger  understanding  about  stories  that  gain  international 
media  attention.  The  researcher  is  working  in  consensus  with  partner  instructors  in  Australia,  Canada, 
New  Zealand,  the  United  Kingdom  and  the  United  States  to  develop  a collaborative  module  that  will 
allow  students  to  examine  and  discuss  international  perspectives  on  global  news  stories.  Students  will 
gather  at  the  module  web  site  that  includes:  structured  and  unstructured  forums  for  discussion  and 
debate,  a location  for  students,  instructors  and  industry  professionals  to  interact,  links  to  broadcast 
networks  worldwide  plus  visual  and  written  content  that  takes  the  learner  through  the  module  and  its 
assignments  step  by  step. 

Data  will  be  gathered  using  a quiz  to  examine  students’  knowledge  of  global  news  issues, 
before  and  after  the  module  is  run.  Learners  and  instructors  will  respond  to  questionnaires  online 
before  and  after  the  module  and  will  participate  in  structured  and  informal  taped  interviews  to  elicit 
personal  and  anecdotal  information.  At  least  ten  students  from  each  partner  university  will  participate 
in  the  first  run  of  the  module  in  April  2001. 

The  result  of  this  study  will  be  to  establish  an  international  circle  bringing  learners  with 
different  backgrounds  and  understandings  of  international  news,  into  an  ongoing  discussion.  The 
timing  is  right  to  examine  how  online  collaborative  learning  functions  within  the  untested  area  of 
broadcast  journalism  and  to  determine  if  there  is  a benefit  for  students  in  using  this  new 
teaching/leaming  method.  This  study  aims  to  bring  issues  of  online  learner  effectiveness  to  light  to 
provoke  further  discussion  research  and  practise  in  this  area. 
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Over  the  next  decade,  there  will  be  an  increasing  digital  gap  between  higher  education  institutions  offering  and  not 
offering  online  courses.  This  will  result  from  the  lack  of  planning  by  offline  institutions  in  information  technology 
policies,  networking  and  technical  infrastructure,  and  access  to  remote  electronic  library  resources.  This  is  not  an  all 
or  nothing  situation.  Some  offline  institutions  have  laid  plans  for  a digital  learning  future.  But  compared  to  online 
institutions,  they  have  less  extensive  plans.  It  takes  several  years  of  laying  down  the  technical  infra stnictiire  before 
we  see  the  results  in  suites  of  online  courses.  An  institution  cannot  decide  one  year  to  start  putting  in  place  such  an 
infrastructure,  and  then  expect  to  see  the  results  the  following  year  in  a dramatically  increased  online  learning. 


We  mailed  8 questionnaires  to  310  colleges,  CGEPS,  and  universities  in  English  and  French  Canada.  134 
institutions  returned  at  least  one  questionnaire  from  June,  1999  to  April,  2000,  for  a response  rate  of  43%.  The 
response  rate  was  good  among  the  60  universities  (89%),  but  poorer  among  colleges  (35%)  and  CGEPS  (25%). 

This  is  due  to  the  inclusion  of  many  very  small  colleges  and  CGEPS  that  had  difficulty  filling  out  complex 
questionnaires.  The  questionnaires  were  completed  by  8 senior  officials  responsible  for  policy  and  implementation 
of  information,  communication,  and  learning  technologies  (questionnaires  returned  are  in  parentheses):  VP  or  most 
senior  officer  responsible  for  information  and  communication  technology  policy  (82/134  =61%);  manager  or 
director  of  networking  and  technical  infrastructure  (72/134  = 54%);  head  of  computer  support  (84/134  = 62%); 
director  of  audio-visual  services  (68/134  = 51%);  head  librarian  (68/134  = 51%);  head  registrar  (69/134  = 52%); 
director  of  instructional  development  or  academic  or  teaching  technologies  center  (75/134  = 56%);  and,  president  of 
the  faculty  or  teachers'  association  (47/134  = 35%).  The  following  comments  and  analyses  are  based  on  these  data. 


INFORMATION  TECHNOLOGY  POLICIES 

VPs  of  Information  Technologies  at  institutions  offering  online  courses,  compared  to  those  not  offering  online 
courses,  placed  more  emphasis  over  the  next  3 years  in:  instructor  training  in  teaching  technologies;  recruitment  of 
more  technical  staff;  cooperation  with  other  institutions  in  software  licensing;  encouragement  or  requirement  of 
student  computer  ownership,  and  provision  of  open  access  computers  to  students.  If  we  expect  the  information 
technology  gap  to  close  between  online  and  offline  institutions,  institutions  not  offering  online  learning  should 
emphasize  these  policies  at  least  to  the  same  degree,  if  not  more,  than  institutions  offering  online  courses.  But  this  is 
not  so.  Over  the  long  term  this  will  lead  to  a growing  gap  between  online  and  offline  institutions. 


NETWORKING  AND  TECHNICAL  INFRASTRUCTURE 

We  asked  Managers  of  Networking  and  Technical  Infrastructure  how  much  they  emphasize  servers,  networking, 
student  computers,  and  wireless  devices  in  their  plans.  The  data  on  servers  reflects  the  declining  use  of  UNIX 
(despite  its  history  and  stability),  and  the  rise  of  Linux  and  Microsoft  NT  servers.  But  institutions  offering  online 
courses  are  more  likely  to  have  all  three  server  types  in  their  plans  than  institutions  not  offering  online  courses.  This 
may  reflect  the  greater  demand  on  servers  among  institutions  that  offer  online  learning.  However,  online  courses  are 
not  the  only  demand  for  servers;  they  are  used  for  a wide  variety  of  administrative  and  educational  functions.  Our 
question  did  not  tie  the  server  type  to  a particular  online  learning  use.  The  data  probably  reflect  a more  general 
difference  in  information  technology  configurations  between  online  and  offline  institutions.  Online  institutions  are 
also  much  more  likely  to  expand  wired  and  wireless  networking,  fast  ethnemet  connections,  notebook  and  other 
mobile  computing  devices,  and  student  computer  lab  capacity.  The  differences  are  so  large  that  this  kind  of  weak 
infrastructure  in  offline  institutions  will  likely  undermine  their  inability  to  offer  future  online  learning. 


DIGITAL  LIBRARY  RESOURCES 

If  institutions  not  offering  online  courses  expect  to  catch  up  to  institutions  offering  online  courses,  they  should 
reduce  their  expenditures  on  hard  copy  print  library  resources  as  much  as  or  more  than  institutions  that  are  heavily 
into  online  learning.  In  fact,  that  is  not  the  case.  Offline  higher  education  institutions  plan  to  spend  more  on  hard 
copy  print  library  resources.  A greater  percentage  of  head  librarians  of  institutions  offering  online  courses  plan  to 
expand  electronic  library  resources  over  the  next  three  years  in  contrast  to  offline  institutions.  This  reinforces  the 
view  that  institutions  already  offering  online  learning  are  forging  ahead  with  additional  plans  to  develop  and  extend 


their  information  technology  infrastructure,  in  this  case  the  library  infrastructure,  in  contrast  to  institutions  not 
offering  online  learning.  Our  data  also  suggest  that  most  libraries  are  moving  quickly  into  the  electronic  age  on 
almost  any  information  technology  question  we  asked.  But  only  about  one-third  of  head  librarians  plan  to 
emphasize  student  e-mail  access  over  the  next  three  years.  Libraries  do  not  view  themselves  as  centers  for  providing 
students  with  e-mail  access;  such  services  and  access  are  available  elsewhere  on  campus  and  from  off-campus. 

SUPPORT  FOR  ELECTRONIC  COURSEWARE 

Directors  of  Academic  Technology  and  Instructional  Development  Centers  at  institutions  offering  online  courses 
plan  to  emphasize  the  following  between  2000  to  2003  in  contrast  to  institutions  not  offering  online  courses: 
funding  for  teaching  support  and  teaching  technology  support  units;  funding  for  free  instructor  workshops  in 
academic  technologies;  training  instructors  in  computer  literacy  and  web  design;  funding  for  instructional 
technology  projects  proposed  by  instructors,  academic  programs,  and  departments;  web  course  outlines;  interactive 
multimedia  web  courseware  with  full  instructor  participation  to  teach  the  courses;  standalone  interactive  web 
courseware  with  minimal  instructor  intervention;  e-mail  in  courses;  computer  conferencing  in  courses;  and, 
computer-supported  collaborative  learning  (CSCL)  in  courses.  However,  those  items  receiving  the  greatest 
endorsement  over  the  next  three  years  among  institutions  offering  online  courses  are:  course  outlines  on  the  web; 
computer  conferencing  in  courses;  computer  supported  collaborative  learning  in  courses;  and,  training  instructors 
in  web  design.  The  least  emphasis  in  plans  over  the  next  three  years  are  in  user-pay  instructor  workshops  in 
academic  technologies,  and  binding  for  instructor  release  time  to  work  on  teaching  technologies. 

REGISTRAR  PLANS  FOR  SUPPORTING  ONLINE  LEARNING 

Among  head  registrars  at  Canadian  higher  education  institutions,  there  is  a moderately  positive  association  between 
the  number  of  online  courses  offered  and  plans  over  the  next  three  years  in:  student  web  registration;  electronic 
delivery  of  student  academic  counseling;  web  course  evaluations  and  course  outlines;  courses  geared  to  job  skills; 
distance  education  courses;  computer  conferencing,  video-conferencing  and  e-mail  in  courses;  computer-based 
training;  and,  linking  electronic  classrooms  between  institutions.  There  are  no  plans  among  registrars  in 
institutions  with  many  online  courses  to  move  into  general  paperless  office  procedures  and  web  calendars  over  print 
versions  (already  done);  student  computer  competencies  (these  skills  are  assumed  or  picked  up  as  students  take 
online  courses);  instructor  online  access  to  class  lists  and  online  submission  of  student  grades;  and,  information 
technology  in  face-to-face  courses  . There  are  three  areas  in  which  registrars  with  low  numbers  of  online  courses 
plan  to  emphasize  over  the  next  three  years,  while  registrars  at  institutions  with  many  online  courses  will  definitely 
not  emphasize:  part-time  student  enrollment;  continuing  education;  and,  instructor  access  to  student  academic 
records.  It  is  often  assumed  that  distance  online  learning  goes  hand-in-hand  with  continuing  education  and  part-time 
studies.  This  may  have  been  the  world  of  traditional  distance  education.  Online  learning  has  moved  out  of 
continuing  and  part-time  studies  into  full-time  studies.  We  are  probably  witnessing  here  a transition  in  the  academic 
programming  of  distance  education  as  it  moves  from  offline  print  correspondence  to  online  learning. 

DEVELOPMENT  AND  DELIVERY  TOOLS  OVER  NEXT  THREE  YEARS 

We  asked  directors  of  Instructional  Development  and  Teaching  Technology  Centers  the  emphasis  that  their 
institutions  will  place  on  development  and  delivery  software  over  the  next  three  years.  Institutions  offering  online 
courses  indicate  that  they  will  move  more  strongly  in  the  direction  of  the  top  software  in  the  market.  Over  three 
quarters  (78%)  of  institutions  offering  online  courses  indicated  that  they  will  move  even  more  strongly  in  the 
adoption  and  implementation  of  courses  using  Web  CT.  Even  among  institutions  not  offering  online  courses,  almost 
one-third  (30%)  plan  to  adopt  WebCT.  No  other  software  comes  close  to  WebCT  in  the  future  plans  of  institutions. 
However,  there  is  significant  growth  potential  in  Lotus  Notes,  Centrinity’s  FirstClass,  Macromedia  Director  and 
Authorware,  Asymmetrix’s  Toolbook,  and  the  Web  Board.  There  does  not  seem  to  be  much  growth  potential  in 
Virtual-U,  Question  Mark,  Allaire  Forum,  Norton  Connect,  and  Web  Course  in  a Box. 

CONCLUSION 

There  is  little  evidence  to  suggest  that  institutions  not  offering  online  courses  will  soon  catch  up  to  institutions 
currently  offering  online  learning.  They  have  not  prepared  their  technical  infrastructure  for  offering  online  courses. 
Their  main  hope  might  be  closer  collaboration  with  neighboring  institutions  that  have  prepared  their  own 
infrastructure  for  a digital  future  in  online  learning. 
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Abstract:  In  this  paper  we  focus  on  the  use  of  computer  supported  collaborative  learning 
(CSCL)  in  an  organizational  setting.  The  program  we  use  is  Web-Knowledge  Forum. 
Although  there  is  a lot  of  experience  in  using  Knowledge  Forum  in  a classroom  setting,  little 
is  known  about  how  to  implement  and  work  with  this  program  in  an  organization  other  than 
schools.  The  problem  investigated  in  this  pilot  study  is  how  a community  of  practice  uses 
Web-Knowledge  Forum  as  an  online  learning  environment  to  discuss  work-related  problems. 
This  study  focuses  on  role  of  the  worker  as  a learner  in  an  unstructured  process  of  negotiation 
of  meaning  in  a community  of  practice.  In  the  study  eight  police  officers  participated  by 
forming  a heterogeneous  group  initiated  by  the  Information  Management  Approach  of 
Criminal  Investigation  in  Police  Education  (ABRIO)  a subdivision  of  the  Dutch  police  force. 


Introduction 

A learning  organization  stimulates  the  ability  to  learn  individually,  in  groups  and  to  learn  by  the 
organization  as  a whole.  The  aim  in  this  is  to  support  continuous  learning  to  combine  and  develop  knowledge  in 
the  organization  in  order  to  respond  flexible  to  changes  that  occur  in  the  market  (Bolhuis  and  Simons,  1999). 
There  is  a tendency  to  organize  this  learning  as  close  to  the  workplace  as  possible  instead  of  sending  people  to 
courses  given  outside  the  workplace  (Van  der  Krogt,  1995;  Nonaka  & Takeuchi,  1997;  Bolhuis  & Simons, 

1999).  Revans  argued  for  action  learning.  He  formed  learning  teams  to  work  on  real  organizational  problems  and 
to  structure  their  experience  in  such  a way  that  both  useful  solutions  to  these  problems  emerge  and  substantial 
learning  occurs  for  participants  (Vaill,  1996).  So  the  employees  are  recognized  as  an  important  resource  to  the 
organization.  According  to  Wenger  (1999)  people  in  organizations  form  communities  of  practice  by  helping  out 
each  other  and  discussing  the  latest  developments.  These  communities  of  practice  are  bound  by  a shared  practice 
related  to  a set  of  problems.  In  these  communities  they  share  and  create  knowledge;  they  learn  through 
participation  (Wenger,  1998).  Membership  to  these  communities  of  practice  is  voluntary,  these  communities  are 
not  bound  by  organizational  affiliations. 

In  large  organizations  like  the  Dutch  police  force  online  communities  have  an  advantage  in  bringing 
people  together  independent  of  time,  space  and  local  cultures.  CSCL  makes  it  possible  for  people  to  participate 
in  communities  of  practice  and  work  at  their  own  pace  and  time.  One  of  the  programs  that  support  this  kind  of 
collaboration  is  Web-Knowledge  Forum.  Web-Knowledge  Forum  is  a discussion  program  designed  to  form  a 
learning  community  over  the  Internet.  It’s  a product  of  the  Computer  Supported  Intentional  Learning 
Environments  (CSILE)  family,  developed  at  the  Ontario  Institute  for  Studies  in  Education  (OISE)  to  support  the 
collaborative  construction  of  knowledge  (Scardamalia  & Bereiter,  1992).  The  participants  operate  in  a shared 
workspace  in  which  they  read  and  write  notes.  A note  is  a contribution  that  can  contain  text,  pictures  and  links  to 
documents,  html  pages  or  other  notes  in  the  shared  knowledge  workspace.  Working  with  this  program  stimulates 
the  participants  to  talk  about  the  subject,  read  relevant  resource  materials,  pose  questions,  offer  theories,  conduct 
experiments,  and  work  together  to  make  sense  of  new  ideas.  By  working  together  participants  develop  greater 
competence  in  a particular  subject  area,  using  what  group  members  already  know  as  an  important  component 
and  co-constructing  plans  of  action  to  extend  that  knowledge  (Hewitt  & Scardamalia,  1998).  The  creation  of 
knowledge  therefore  is  seen  as  a social  product. 

Originally  this  program  was  designed  for  use  in  the  classroom  to  support  the  construction  of  knowledge 
in  a social  context.  The  aim  in  the  development  was  to  support  an  environment  that  will  make  it  possible  for 
schools  to  function  as  knowledge  building  communities  (Scardamalia  & Bereiter,  1992).  Within  knowledge 
building  communities  the  focus  is  on  knowledge  construction.  It’s  a knowledge-centered  community  of  practice. 

The  problem  we  face  is  how  to  facilitate  the  creation  and  support  of  communities  of  practice  in 
organizations  that  work  in  a Web  based  environment  and  make  knowledge  building  the  core  of  their  activity.  In 
the  first  place  there  has  to  be  identified  a ‘real’  and  meaningful  problem  that  exists  in  the  organization.  A 
problem  that  is  owned  by  the  participants  and  in  which  solution  they  are  willing  to  put  effort.  Second  the 
members  of  the  organization  who  feel  affiliated  by  this  problem  have  to  form  a community  in  which  they  can 
participate  on  a voluntary  and  functional  basis.  In  this  paper  we  focus  on  the  possibilities  of  Web-Knowledge 
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Forum  to  serve  as  a meeting  place  for  such  communities.  A place  where  participants  can  work  together, 
undertake  collaborative  learning  activities,  sharing  their  knowledge,  aimed  at  deepening  their  expertise  in  the 
problem  to  be  solved. 

Our  question  is:  Can  groupware,  e.g.  Web-Knowledge  Forum,  support  the  negotiation  of  knowledge 
and  understanding  in  communities  of  practice  in  an  organization? 

We  focus  on  the  role  of  the  worker  as  a learner  in  an  unstructured  process  of  negotiation  of  meaning  in  an  online 
community  of  practice. 


The  study 

This  study  was  conducted  to  gain  experience  with  participants  who  were  engaged  in  an  online  learning 
community.  The  community  consisted  of  eight  participants  who  voluntary  joined  in  the  community.  They 
responded  to  a letter  lliat  was  sen  I by  Liic  ABRiO  to  several  police  departments  explaining  the  problem  that  has 
to  be  solved.  The  problem  was  about  how  to  identify  and  describe  general  work-processes  used  in  the  field  of 
criminal  investigation.  As  a whole  the  participants  formed  a heterogeneous  group  (policy  makers,  criminal 
investigators  and  experts). 

During  a period  of  two  months  they  worked  together  using  Web-Knowledge  Forum.  There  was  much 
uncertainty  about  how  to  identify  work-processes,  therefore  the  participants  agreed  to  start  with  an  open 
discussion  on  the  subject  ‘work-processes’,  instead  of  following  a structured  plan  of  action  to  tackle  the  problem. 
Web-Knowledge  Forum  played  a central  role  in  supporting  the  discussion  because  all  the  written  contributions 
are  stored  as  notes  in  a shared  database  available  to  all  the  participants.  The  discussion  was  divided  into  certain 
subjects  called  views  in  which  the  participants  contributed  a note  or  comment  on  a note  they  had  read,  by  writing 
a build-on  note. 


Instruments 

The  way  people  participated  and  interacted  with  each  other  provides  information  about  the  activities  of 
such  a community.  Web-Knowledge  Forum  is  provided  with  an  analytic  toolkit  (ATK)  that  analyses  the 
activities  of  the  members  of  the  community  in  the  database.  It  creates  log  files  of  all  the  users  about  how  many 
times  they  have  read,  write  or  edited  a note.  How  many  notes  are  linked  to  each  other  and  how  many  ‘build-on’s 
there  are  made. 

Web-Knowledge  Forum  is  designed  to  facilitate  cognitive  and  metacognitive  activities  by  providing 
opportunities  to  give  your  opinion,  to  give  a comment,  or  by  making  suggestions  or  providing  new  information. 
Because  of  these  possibilities  it  is  important  to  know  more  about  the  nature  of  the  content  of  the  material  the 
community  has  created  together  and  what  kind  of  activities  the  participant  undertake.  Are  they  trying  to  advance 
their  knowledge?  Veldhuis-Diermanse  (1999)  developed  a coding  scheme  to  gain  information  about  the  content 
of  the  written  notes  based  on  the  constructivist  view  of  learning  in  an  educational  setting.  This  coding  scheme  is 
still  in  a design  stage  and  it  is  the  first  time  this  scheme  is  applied  to  an  organizational  community.  This  coding 
scheme  consists  of  three  main  categories:  1 Cognitive  activities,  2 Metacognitive  activities,  and  3 Affective 
activities. 

1 Cognitive  activity:  Cognitive  activities  are  used  to  process  and  acquire  insight  in  the  information  being 
discussed.  Veldhuis-Diermanse  recognized  three  subcategories  of  cognitive  activities:  1 debating,  the  accent  is 
on  arguing,  presenting  new  ideas  or  thoughts  about  the  subject;  2 using  external  information,  the  accent  is  on 
referring  to  information  found  in  other  sources  than  the  database;  and  3 linking  or  repeating  internal  information, 
the  accent  is  on  referring  to  information  found  in  the  shared  database. 

2 Metacognitive  activities:  Veldhuis-Diermanse  (1999)  describes  metacognitive  activities  as  activities 
undertaken  to  regulate  each  other’s  learning  process  or  to  regulate  the  goals  and  direction  of  the  discussion. 

3 Affective  activities  are  used  to  cope  with  feelings  occurring  during  the  discussion  among  the  participants 
(Veldhuis-Diermanse,  1999). 

At  the  end  of  the  pilot-study  we  gave  the  participants  a questionnaire  to  gain  information  about  their 
experiences  working  with  Knowledge  Forum. 


Results 

The  log  files  generated  by  ATK  gives  a description  of  the  activities  that  has  taken  place  in  the  database 
(Tab.  1).  The  participants  contributed  98  notes  in  the  database.  That’s  an  average  of  12,25  notes  per  participant. 
56%  percent  of  the  notes  have  been  read.  This  means  the  amount  of  notes  that  have  been  opened  by  the 
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participants.  So  this  might  exaggerate  the  actual  reading  that  has  been  done.  83%  percent  of  these  notes  are 
linked,  also  called  build-on.  The  log-file  only  records  the  activities  of  the  participants,  a build-on  activity 
therefore  does  not  have  to  be  content  related.  Table  1 shows  that  there  are  substantial  differences  between  the 
participants,  both  in  writing  and  reading.  Notice  that  build-on  notes  are  also  a part  of  the  written  notes. 


Written 

Build-on 

Read 

N 

8 

8 

8 

N 

8 

8 

8 

Mean 

12,25 

10,75 

232,62 

Mean 

12,25 

10,75 

232,62 

Std. 

9,54 

8,80 

147,51 

Deviation 

Std. 

9,54 

8,80 

147,51 

1./V;  viauuii 

Minimum 

1,00 

1,00 

113,00 

Minimum 

1,00 

1,00 

113,00 

Maximum 

30,00 

26,00 

552,00 

Maximum 

30,00 

26,00 

552,00 

Table  1.  Participation  in  the  database 


The  content  analysis  of  the  notes  reflects  the  following  cognitive 
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Figurel.  Activities  in  the  database 


These  results  indicate  that  the  participants  discuss  a lot  about  the  subject.  The  participants  present  a lot 
of  ideas  of  there  own  and  start  discussing  about  that  in  the  community.  The  participants  do  not  bring  in  much 
new  information;  only  seven  times  they  referred  to  information  that  can  be  found  outside  the  database  (e.g.  book 
pages).  Also  they  did  not  make  many  content  related  references  to  the  contributions  of  other  participants  in  the 
database.  Although  according  to  the  ATK  83  % of  the  notes  are  linked.  This  difference  is  explained  by  the  fact 
that  the  coding  scheme  only  refers  to  written  comments  to  other  contributions  in  the  database.  The  participants 
show  quite  a lot  of  regulative  activities,  but  in  fact  two  participants  are  responsible  for  66%  of  the  regulative 
activities.  There  are  quite  a lot  of  affective  contributions  in  the  database.  The  explanation  for  it  is  that  this  way  of 
working  was  new  to  them  and  therefore  they  were  regularly  asking  for  feedback  and  gave  a lot  of  general 
reactions  to  the  other  participants.  This  category  shows  that  they  did  not  use  this  medium  for  chatting  or  sending 
‘e-mails’  to  each  other,  that  are  not  related  to  the  discussion.  Most  contributions  even  the  affective  ones  were 
subject  related. 

The  questionnaire  provided  information  about  the  way  participants  worked  with  Web-Knowledge 
Forum,  five  out  of  eight  were  returned.  60%  of  the  participants  agreed  on  the  question  if  they  were 
collaboratively  building  new  knowledge  about  ‘work-processes’,  but  they  pointed  out  that  they  need  to  grow 
more  into  building  upon  the  ideas  of  others.  Also  they  mentioned  that  there  was  a lot  of  confusion  about  the 
concepts  being  used  and  that  they  need  to  clarify  the  goal  of  their  study,  to  give  more  direction  to  the  discussion. 
80%  of  the  participants  indicated  that  they  were  satisfied  with  the  opportunities,  provided  by  the  program,  to 
discuss  the  subject  together.  The  participants  (80%)  notified  to  have  enough  information  to  be  able  to  take  part  in 
the  discussion.  Answers  to  the  question,  what  they  do  if  they  lack  certain  information,  are  searching  for  relevant 
information,  consulting  colleagues  at  work,  and  trying  to  stimulate  the  other  participants  to  explain  certain 
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issues.  Results  of  the  question  about  what  they  thought  of  the  quality  of  the  written  notes,  varies  from  good  to 
reasonable.  In  general  the  quality  is  good  but  the  discussion  became  more  silent  later  on.  “There  is  to  little 
structure  to  guide  our  discussion,  the  notes  contain  valuable  information  but  what  does  it  bring  to  us?”  40%  of 
the  participants  indicated  that  there  was  to  less  coordination  during  the  discussion.  60%  of  the  participants 
pointed  out  that  a more  structured  or  goal  directed  approach  is  necessary.  They  argue  that  this  will  help  them  to 
achieve  agreement  and  build  on  to  that. 


Conclusion  and  Discussion 

Summarizing  the  results  it  can  be  seen  that  the  participants  show  a lot  of  activity  in  the  database 
although  they  read  much  more  than  they  contribute.  Related  to  the  content  analysis  of  the  contributions.  It  seems 
that  the  activities  the  participants  undertook  are  more  discussion  oriented  than  building  knowledge  upon  already 
contributed  or  new  information.  However  in  their  perception  of  working  with  groupware  they  appreciate  the 
possibility  of  knowledge  sharing.  On  the  first  glance  people  are  willing  to  work  in  a groupware  environment  and 
share  knowledge  together.  This  is  promising  for  the  support  of  working  in  communities  of  practice  in 
organizations.  The  analysis  also  shows  that  sharing  of  knowledge  does  not  happen  automatically.  Although  there 
are  a lot  of  regulation  activities,  it  turns  out  that  they  were  mostly  carried  out  by  two  participants.  The 
questionnaire  reveals  that  the  community  members  desire  more  structure  and  support  to  direct  the  knowledge 
building  activities  of  the  community  as  a whole.  The  use  of  groupware  in  communities  of  practice  in 
organizations  seems  promising.  The  participants  report  that  this  tool  is  useful  for  knowledge  sharing  but  that  the 
actual  knowledge  building  activities  needs  to  increase.  They  had  trouble  clarifying  the  goals  and  direction  of  the 
discussion.  Also  they  discovered  that  there  was  quite  a lot  of  confusion  about  the  concepts  used  by  the 
participants.  The  main  concern  was  lack  of  coordination  during  the  discussion.  In  an  unstructured  process  of 
negotiation  nobody  feels  direct  responsibility  to  organize  and  structure  the  content  of  the  problem  studied.  The 
lack  of  the  learning  ability  in  the  sense  of  regulating  the  content  and  community  processes  seems  to  be  crucial 
for  people  to  become  used  to  share  knowledge,  deepening  their  own  and  common  understanding  and  creating 
further  insights.  To  compensate  this  lack  one  can  structure  the  negotiation  of  meaning  by  making  a learning 
agenda.  By  which  participant  express  their  goals,  divide  certain  tasks,  and  divide  responsibilities  (e.g. 
coordinator  of  the  content,  someone  who  keeps  the  community  together  or  invites  new  participants  when  needed, 
technical  assistance).  One  can  also  introduce  a didactic  approach  of  inquiry  analogue  to  phases  of  the  scientific 
enquiry  process  or  problem  solving. 
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Abstract:  Federal  mandates  enacted  on  August  7,  1998  revealed  that  ’’Effective  2 years  after  the  date  of 
enactment... any  individual  with  a disability  may  file  a complaint  alleging  that  a Federal  department  or  agency  fails  to 
comply  with  subsection  (aXU  in  providing  electronic  and  information  technology.  (Section  508.  "Electronic  and 
Information  Technology)  'The  new  rules  will  apply  within  a few  months  to  all  web  sites  operated  by  government 
agencies.. .By  August  7,  2000,  they  will  extend  to  vendors  doing  business  with  the  federal  government.  "Possibility 
soon  afterward  they  may  extend  to  every  Web  site  posted  in  the  U.S."  (Adam  Clayton  Powell  III).  Are  ready  for 
August  7,  2000?  If  not,  this  paper  can  provide  ideas  to  help  you  get  started. 


Introduction 

In  PC  Week  Online,  Vaas  (2000)  notes  that  between  95  and  99  percent  of  web  sites  are  inaccessible  to  the  visually, 
hearing  -and/or  mobility  impaired.  Many  web  developers  are  ignoring  the  accessibility  issue  or  just  do  not  know 
how  to  making  their  sites  universally  accessible.  They  are  bewildered  in  the  face  of  the  vast  information  available  in 
terms  of  how  to  make  it  applicable  to  their  situation.  Three  areas  can  help  focus  the  journey  to  understanding. 

What’s  What 

The  law  is  the  driving  force  behind  the  need  to  have  universal  web  sites.  There  are  already  been  law  suits  against 
AOL  and  Universities  for  not  providing  accessible  web  access.  Equally  access  to  information  is  the  key  reason  that 
the  focus. is  on  web  sites.  Individuals  cannot  fiilly  participate  in  today's  complex  society  based  on  technology-based 
information  if  they  do  have  access  to  the  same  information.  There  are  several  references  to  the  law  at  the  following 
web  site: 

http://www.swt.edu/'^dfl  2/summit/LAW.HTM 

Who’s  Who? 

This  paper  will  mention  just  a few  of  the  leaders  in  the  field  who  are  trying  to  make  electronic  information  globally 
accessible.  The  first  is  W3C.  Their  mission  states  "The  W3C's  commitment  to  lead  the  Web  to  its  full  potential 
includes  promoting  a high  degree  of  usability  for  people  with  disabilities.  The  Web  Accessibility  Initiative  (WAI), 
in  coordination  with  organizations  around  the  world,  is  pursuing  accessibility  of  the  Web  through  five  primary  areas 
of  work:  technology,  guidelines,  tools,  education  & outreach,  and  research  & development." 
rhttp://www.w3.org/WAIA  W3C  focus  is  for  experienced  programmers  and  guidelines  can  be  found  at  the  following 
address: 

http://www.w3.orgyTR/1999AVAI-WEBCQNTENT-19990505/ 

Accessible  Web  Authoring  Resources  and  Education  (AWARE),  mission  is  to  serve  as  a central  resource  for  web 
authors  for  learning  about  web  accessibility.  The  AWARE  Center  was  launched  in  April  1999  as  part  of  the  HTML 
Writers  Guild’s  (HWG)  annual  Web  Accessibility  Month,  a special  focus  on  the  importance  of  designing  for 
universal  accessibility.  The  Center  is  supported  by  the  Guild's  staff  and  volunteers,  and  is  designed  as  a resource  for 
all  web  authors.  Ken  Barlett  co- founded  the  HTML  Writer's  Guild  with  Martin  Bayne.  The  HWG  is  the  world's 
largest  international  organization  of  Web  authors  with  over  130,000  members  in  more  than  130  nations  worldwide. 
The  HGW  exists  to  assist  its  members  in  developing  and  enhancing  their  capabilities  as  web  authors,  to  compile  and 
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publicize  information  about  standards,  practices,  techniques,  competency,  and  ethics  as  applied  to  web  authoring, 
and  to  contribute  to  the  development  of  the  web  and  web  technical  standards  and  guidelines. 

Disability  Access  Information  and  Support  (DAIS).  Jane  Jarrows,  President,  states,  " But  after  we  got  the  ramps  and 
elevators  installed,  we  could  think  about  how  to  make  the  teaching  that  had  always  gone  on  in  those  classrooms 
fully  available  to  all  students.  The  parallel  ends  here.  While  we  are  scrambling  to  get  the  technological  ’’ramps  and 
elevators”  in  place,  we  are  actively  creating  new  barriers  to  access  on  a regular  basis.  At  the  risk  of  stretching  the 
analogy  a little  too  thin,  if  we  don’t  be  careful  we  will  find  that  when  those  technological  ramps  and  elevators  we 
installed  are  finally  in  place,  they  only  allow  for  a better  view  of  the  opportunities  passing  by  us.  We  cannot  wait  to 
get  the  technology  in  place  before  we  start  trying  to  impact  on  the  application  of  that  technology 

Microsoft.  The  Microsoft  Senior  Initiative  is  a program  aimed  at  bridging  the  digital  and  generational  divides  and 
ensuring  that  seniors  are  not  left  behind  on  the  information  superhighway.  By  providing  access  to  information 
technology  and  PC  literacy  training,  the  Microsoft  Senior  Initiative  introduces  the  exciting  possibilities  of 
technology  to  senior  citizens.  The  Seniors  and  Technology  Web  site  (http://www.microsoft.com/seniors/)  is  a 
resource  for  seniors,  their  families  and  communities  about  the  exciting  possibilities  that  can  be  realized  through  the 
use  of  technology. 

How  to?  For  people  who  are  just  learning  how  to  develop  web  pages,  the  easiest  way  is  make  a page  using  a 
’’what  you  see,  what  you  get  editor,”  then  save  the  web  page  as  text  and  htm.  The  problem  with  this  method  is  that 
that  many  people  do  not  do  an  adequate  job  of  keeping  both  pages  updated.  Software  companies  are  starting  to 
make  it  easier  to  make  web  pages  accessible  through  what  is  called,  style  sheets,  which  control  the  presentation  of  a 
documeint.  It  is  often  easier  to  work  in  pairs  or  teems  of  two  to  learn  this  process.  There  are  often  many  free 
workshops  on  the  web  and  in  your  local  area.  Perhaps  partner  with  someone  who  can  pay  to  have  a consultant. 
Design  is  the  first  part  of  ”how  to."  The  second  part  is  getting  your  page  validated  by  a reliable  source.  Sites  can  be 
easily  validated  using  on-line  features  which  require  that  you  paste  your  URL  name  in  the  box  and  almost 
immediately  you  will  learn  if  you  have  accessibility  problems.  The  top  two  sources  of  validation  are: 

Bobby  Accessibility  Validator 
http://www.cast.org/bobbv/ 

W3C  HTML  4.0  Validator 
http  ://val  i dator.  w3 . org/ 

The  complexity  of  implementation  of  this  mandate  can  not  be  fully  address  with  this  in  progress  paper;  however, 
there  will  a much  greater  discourse  at  the  Universal  Web  Accessibility  Symposium  2000  (at  WebNet  2000)  --  San 
Antonio,  Texas,  October  31,  2000. 
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Introduction 

Many  educators  are  experimenting  with  flexible  on-line  learning;  consequently  academics  have  to 
adapt  their  delivery,  preparation  and  course  administration.  This  study  shares  staff  and  student 
experiences  of  participating  in  a completely  remote  on-line  course,  compared  to  a flexible  on-line 
course  where  students  can  attend  classroom  sessions. 

The  issues  we  encountered  included  ones  related  to  hardware  and  software,  submission  of  assessments, 
student  communication  with  each  other  and  the  lecturers,  as  well  as  student  and  staff  motivation  over 
the  holidays. 

Existing  research  documents  the  pitfalls  of  teaching  on-line  and  we  used  these  lessons  to  anticipate 
problems  as  we  planned  our  on-line  course.  This  study  documents  the  problems  we  encountered  but 
hadn’t  anticipated. 

THE  COURSE 

Internet  and  Web  Design  (IWD)  is  a popular  first  year  introductory  course  in  the  Bachelor  of 
Computing  Systems  degree.  It  is  normally  taught  flexibly  with  lectures  and  practical  classes  as  well  as 
a self-paced  on-line  course  comprised  of  notes,  exercises,  self-assessment  questions  and  a discussion 
forum  (1).  In  the  1999-2000  Summer  School,  the  course  was  offered  completely  remotely  and 
condensed  from  14  to  11  weeks.  There  were  no  face-to-face  sessions  except  an  orientation  session,  the 
project  presentations  and  final  exam.  If  students  had  problems  they  were  requested  to  post  their 
question  on  a discussion  forum  and  had  no  option  to  ask  the  lecturers  face-to-face. 

DISCUSSION 
Student  issues 

No  time  for  a Summer  break,  condensed  time  frame,  lack  of  personal  tutor-student  contact,  no  option 
of  attending  classroom  sessions  for  personal  contact  with  other  students  and  no  opportunity  to  take  on  a 
vacation  job.  Perhaps  because  of  the  above  issues,  there  was  a higher  level  of  drop  outs  than  normal. 
Reasons  for  students  dropping  out  included:  getting  a job,  workload  too  heavy  and  no  computer  access 
at  home 

Staff  issues 

Lack  of  a gap  between  semesters  to  attend  to  the  course  administration  for  the  previous  course.  We 
undertook  the  course  as  an  extra  to  our  workload,  meaning  no  summer  break  to  recharge  and  prepare 


for  the  next  semester.  Most  of  the  workload  consisted  of  administration  and  marking  and  we  felt  the 
remuneration  we  received  did  not  reflect  the  associated  workload. 

Bulletin  board  issues 

If  students  had  problems  they  were  requested  to  post  their  question  on  the  discussion  forum  rather  than 
e-mail  the  lecturers,  (so  that  we  could  avoid  duplication  of  lecturer  responses  and  attempt  to  mirror 
classroom  communication  by  allowing  students  to  answer  each  others*  questions  when  lecturers  were 
not  around).  Despite  our  attempts  to  encourage  participation,  the  use  of  the  bulletin  board  was  fairly 
low,  with  only  140  posts  over  the  whole  Summer  School.  Many  students  continued  to  email  the 
lecturers  rather  than  use  the  bulletin  board  for  questions.  Perhaps  this  was  because  they  didn’t  want  to 
look  unknowledgeable  in  front  of  their  peers,  or  because  we  didn’t  assess  the  students’  contributions 
(2,3).  Lecturers  accessed  the  bulletin  board  every  other  day  to  check  for  questions  to  answer.  The 
lecturer’s  often  interrupted  their  vacations  to  make  special  trips  to  cyber-cafes,  only  to  find  that  no 
students  were  seeking  help. 

Assignment  submission  issues 

Students  didn’t  email  their  assignments  to  the  correct  address,  often  sending  them  to  our  home  email 
addresses  and  multiple  addresses.  Issues  resulting  from  this  included:  lack  of  software  at  lecturer’s 
home;  lecturer’s  vulnerable  to  the  risk  of  virus  infection;  and  duplication  of  administration  effort.  There 
was  a significant  administrative  burden  for  the  lecturers  who  had  to  confirm  receipt  by  email  and  print 
out  the  assignments.  This  was  exacerbated  by  the  often  inappropriately  large  file  sizes  of  the  email 
attachments,  (e.g.  one  Storyboard  Word  document  was  3.5Mb). 

Group  work  issues 

Generally,  students  find  it  difficult  to  work  in  groups.  This  is  exacerbated  in  an  on-line  course  by 
having  to  use  the  printed  word  rather  than  face-to-face  interaction.  The  condensed  timescale  of  the 
course  made  this  even  more  of  an  issue  and  one  of  our  groups  suffered  a complete  breakdown  in 
communication.  During  a Semester  course  it  is  easier  to  identify  problems  with  group  dynamics  and 
deal  with  them  before  they  escalate. 

CONCLUSIONS 

The  students  had  a generally  favourable  impression  of  on-line  learning,  but  with  reservations  about  the 
lack  of  personal  contact  and  the  high  workload  due  to  the  collapsed  course  time-scale.  Reasons  for  the 
course’s  success  included:  anticipating  possible  problems  and  devising  strategies  to  deal  with  them; 
and  the  considerable  amount  of  user-testing  that  took  place  in  the  prior  flexible  on-line  versions  of  the 
course,  when  semantic  problems  could  be  cleared  up  easily. 

However  unexpected  problems  were:  inter-group  conflicts;  a higher  than  normal  drop-out  rate  that 
impacted  on  other  group  members;  higher  than  normal  administrative  burden  on  the  lecturers;  and 
serious  practical  problems  with  the  publishing  of  student  web-sites  using  FTP  (4). 

It  is  important  to  spend  time  forming  project  groups,  to  let  the  students  get  to  know  each  other’s  skill- 
set  and  interests  thereby  avoiding  problems.  This  is  perhaps  even  more  important  in  a fully  on-line 
course. 
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Abstract:  Offering  video  and  other  rich  media  content  over  the  Internet  is  an  emerging  growth  area  and 
represents  a new  model  and  opportunity  for  education.  As  broadband  access  increases  and  advances  are  made  in 
network  caching,  streaming  servers,  video  and  audio  compression,  and  multimedia  databases,  delivering  streaming 
video  claims  unprecedented  potential  for  education.  Universities  and  corporations  will  continue  to  look  to 
technology  to  deliver  distance  education  and  web  based  courses.  Determining  the  pedagogical  applications  of 
streaming  video  is  an  important  area  of  research.  Identifying  the  strengths  and  weaknesses  from  an  educational 
point  of  view  provides  an  important  context  to  the  practical  matters  of  technical  implementation.  This  research 
intends  to  investigate  both  the  pedagogical  implications  and  the  technical  implementation  of  streaming  video. 


Offering  video  and  other  rich  media  content  over  the  Internet  is  an  emerging  growth  area  and  represents  a 
new  model  and  opportunity  for  education.  As  broadband  access  increases  and  advances  are  made  in  network 
caching,  streaming  servers,  video  and  audio  compression,  and  multimedia  databases,  delivering  streaming  video 
claims  unprecedented  potential  for  education.  Universities  and  corporations  will  continue  to  look  to  technology  to 
deliver  distance  education  and  web  based  courses.  Determining  the  pedagogical  applications  of  streaming  video  is 
an  important  area  of  research.  Identifying  the  strengths  and  weaknesses  from  an  educational  point  of  view  provides 
an  important  context  to  the  practical  matters  of  technical  implementation.  This  research  intends  to  investigate  both 
the  pedagogical  implications  and  the  technical  implementation  of  streaming  video. 

Often  when  new  technologies,  like  video,  are  applied  to  education  they  are  seen  as  having  the  ability  to 
make  us  all  learn  better  and  faster.  Some  of  the  questions  that  need  to  be  answered  regarding  pedagogy  include: 

• Does  streaming  video  over  the  web  offer  any  more  promise  than  standard  videotape,  the 
Laser  Disc,  or  the  CD-ROM? 

• Can  the  metadata  applied  to  streaming  video  offer  learning  strategies  and  information  that 
was  previously  unavailable? 

• Does  the  distribution  medium,  the  Internet  and  a computer,  improve  the  quality  or  access 
to  instruction? 

• What  are  the  ideal  tools  and  methods  to  develop  content  and  sufficient  metadata  that 
make  streaming  video  effective  in  teaching  and  learning? 


Overview 


In  addition  to  the  pedagogical  perspective,  there  is  a highly  technical  aspect  of  streaming  video  that  affects 
price,  availability,  usability,  performance,  and  quality.  The  following  technical  components  will  be  investigated: 

• Copyright,  Security,  and  Intellectual  Property 

• Multimedia  Databases 

• Servers 

• Clients 

• Compression 

• Network  Caching  and  Bandwidth  Issues 

Finally,  partnerships  with  industry'  suppliers,  distributors,  faculty,  and  technology  solutions  will  be 
investigated  in  an  effort  to  provide  business  models  that  can  recover  the  costs  of  implementing  and  delivering 
streaming  video  content  for  universities. 

Currently,  the  pedagogical  potential  of  streaming  video  has  not  been  demonstrated,  the  technology  is  so 
new  that  it  remains  largely  untested  in  a learning  environment,  and  content  development  tools  are  crude  or  non- 
existent. Answering  the  various  pedagogical  and  technical  questions  are  important  outcomes  of  this  research 
project.  Performance  indicators  of  this  research  will  include  the  application  of  streaming  video  technologies  to 
multiple  content  areas  - including  campus  and  off-campus  users.  In  addition,  the  partnerships  developed  with 
industry,  such  as  Akamai  or  other  industry  partners,  will  be  measured  against  the  benefits  to  the  institution. 

Detailed  Description 

Applications  for  streaming  video  are  various.  They  include:  webcasts  of  special  lectures;  access  to  a rich 
educational  video  library;  video  and  audio  conferencing;  the  integration  of  video  into  course  web  pages;  and  the 
creation  of  video-based  educational  objects.  The  effectiveness  of  this  technology  to  teaching  and  learning  is  largely 
untested,  but  none  the  less  the  demands  to  incorporate  it  into  teaching  and  learning  are  starting  to  be  met  by  the 
technology.  Only  recently  has  a distribution  medium  become  available  that  allows  full  text  searching  of  video,  the 
application  of  metadata,  interactivity,  and  other  media  in  ways  that  make  the  use  of  video  in  education  more  flexible 
than  ever  before.  Research  into  the  tools,  usability,  educational  effectiveness,  and  various  solutions  benefit  current 
and  future  teaching  and  learning  at  our  post-secondary  institutions. 

The  focus  is  on  three  main  aspects  of  streaming  video.  These  are  quality  and  performance,  pedagogy,  and 
business  models.  Compression  algorithms,  the  end  user  software,  or  client,  networking  technologies  and  server 
solutions  impact  quality  and  performance.  In  order  to  study  quality  and  performance,  various  partners  around  the 
world  are  providing  and  testing  content.  Other  partners,  that  can  provide  performance  enhancements  to  streaming 
video  including,  cable  companies,  phone  companies,  and  broadband  access  providers  is  under  investigation. 
Various  solutions’  quality  and  performance  will  be  compared  by  individual  reviewers  and  focus  groups  in  several 
locations.  Quality  and  performance  will  be  tested  given  different  conditions,  including  connection  speed,  time  of 
day,  distance  from  server,  end-user  platform,  server  characteristics,  compression  algorithm,  and  client.  Akamai’s 
Free  Flow  server  technology  will  be  tested  to  compare  performance  enhancements  at  a regional  and  international 
level.  Campus  and  off-campus  user  tests  will  determine  quality  and  performance  standards  for  the  main  streaming 
video  solutions. 

Streaming  video,  in  itself,  is  like  any  other  video  format  from  a pedagogical  point  of  view.  Video,  in 
general,  is  considered  linear  and  lacking  interactivity.  If  putting  learners  in  front  of  a video  feed  was  effective,  we 
would  learn  from  home,  in  front  of  our  televisions,  watching  a video  taped  lecture.  Clearly,  streaming  video  of 
lectures  over  the  web  does  not  represent  any  educational  advantage  over  distributing  VHS  copies  of  lectures. 
However,  the  potential  for  the  video  and  audio  to  contain  additional  information  may  provide  an  educational 
advantage.  Using  video  taped  scenarios  that  demonstrate  procedures  and  principles  can  be  an  effective  use  of  video. 
Marking  up  that  scenario  with  additional  information,  or  metadata,  may  demonstrate  clear  advantages  for  teaching 
and  learning.  Creating  interactivity,  allowing  the  learner  to  make  decisions,  search  text,  navigate  non-linearly  , and 
interact  with  the  video  may  allow  a learner  more  effective  access  to  learning  using  video  than  previously  possible. 
Demonstrating  the  possibilities  is  not  simple.  Tools  that  develop  the  metadata,  and  procedures  that  accommodate 
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advanced  content  development  for  video  are  crude  and  difficult  to  use.  In  fact,  although  the  need  and  technology 
exist,  there  no  cost  effective  methods  for  creating  a transcript  of  a video  without  the  use  of  an  individual  typing  what 
they  hear  into  a computer.  The  expense  of  adding  this  metadata  discourages  its  inclusion.  The  goals  here  are  to 
demonstrate  its  potential  and  to  discover  processes  and  tools  that  permit  the  inclusion  of  metadata  for  streaming 
video.  It  is  anticipated  that  this  research  will  more  clearly  specify  the  need  for  content  development  software  to  be 
developed. 


Associated  with  content  development  for  the  metadata  of  video  are  the  data  stores  for  the  video  and  its 
metadata.  There  are  several  commercial  database  solutions  currently  available  and  establishing  the  criteria  that 
matches  sound  pedagogy  for  the  database  is  an  important  outcome  of  this  study.  One  of  the  issues  is  extensibility  of 
the  database.  Scalability,  compatibility  and  reusability  are  all  related  to  extensibility.  The  momentum  of 
educational  objects  and  IMS  metadata  standards  for  education  dictate  the  need  for  the  open,  flexible  standards  and 
an  object  oriented  approach.  A solution  where  costs  do  not  prohibit  university  access  to  necessary  features,  while 
offering  security  and  intellectual  property  protection  will  be  sought. 


The  third  aspect  of  this  research  focuses  on  the  costs,  partnerships,  infrastructure  requirements,  and 
opportunities  of  streaming  video  for  institutions.  In  essence,  a business  model  that  can  sustain  the  delivery  of 
streaming  video  for  the  institution.  Costs  associated  with  video  production,  editing,  and  compression  are  high, 
adding  in  metadata  and  storing  in  it  all  in  large  database  applications  can  be  very  high.  Pricing  varies  widely 
between  suppliers  and  solutions  and  an  investigation  into  the  details  will  lead  to  recommendations  for 
implementation.  Other  costs  associated  with  streaming  video  include  servers,  licensing  fees,  and  Internet 
infrastructure.  A detailed  comparison  of  these  costs  against  the  performance  and  quality  testing  will  lead  to 
additional  recommendations. 


In  addition  to  the  cost/benefit  analysis,  the  opportunities  for  collaboration  with  the  private  sector  is  being 
investigated.  Opportunities  exist  that  allow  the  university  and  its  faculty  to  recover  royalties  to  its  intellectual 
property  and  charge  for  access  to  its  servers.  Streaming  video  over  the  web  could  prove  to  become  the  best 
distribution  medium  available  for  the  thousands  of  hours  of  video  already  produced  and  owned  by  our  institutions. 
Developing  the  types  of  partnerships  and  business  models  that  could  help  support  delivery,  provide  new  content, 
build  infrastructure,  reach  broader  audiences,  and  maintain  hardware  and  software  is  being  prototyped  at  the 
University  of  Calgary. 


Summary 

The  various  complex  issues  from  network,  to  compression,  multimedia  databases  and  usability  require 
research,  development,  and  testing  to  determine  the  best  solutions  for  the  institution.  Guided  by  sound  pedagogy, 
the  framework  suggested  above  is  meant  to  get  to  some  of  the  most  important  and  practical  issues,  enabling  the 
recommendation  and  adoption  of  solutions  for  the  use  by  a post-secondary  institution. 
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Abstract:  The  availability  of  computers,  the  emergence  of  new  visualisation  and  authoring  tools 
and  techniques,  together  with  the  expansion  of  web  based  learning  environments  and  CD-based 
multimedia  applications,  offer  chemistry  teachers  new  possibilities  for  bridging  the  gap  in  students' 
knowledge  and  understanding  of  abstract  and  concrete  chemical  concepts  and  processes.  This  paper 
shows  by  example  how  the  advantages  of  both  the  Internet  and  the  interactive  CD-ROMs  can  be 
used  to  improve  students'  learning  at  both  macro-  and  microscopic  level.  We  present  results  of  a 
study  of  the  effects  of  interactive  multimedia  teaching  units  on  the  quality  of  knowledge. 


Introduction 

The  old  Chinese  saying,  "a  picture  is  worth  a thousand  words",  has  been  proven  many  times  by  modem 
neuropsychological  studies.  Human  beings  can  memorize  20%  of  what  they  read,  and  30%  of  what  they  hear,  but 
40%  of  what  they  see,  55%  of  what  they  say,  and  60%  through  active  involvement;  however,  when  all  of  the  above 
sensory  channels  are  used  simultaneously,  this  results  in  the  memorization  of  90%  of  the  content  (Rose,  1993). 
According  to  the  Gardner  theory  of  multiple  intelligences  (Veenema  & Gardner,  1996),  we  possess  a number  of 
relatively  independent  types  of  intelligences:  linguistic,  logical,  musical,  kinesthetic,  spatial,  inter-  and  intrapersonal. 
Each  individual  has  the  natural  ability  to  use  some  of  his  sensory  systems  better  than  others.  This  shows  that  many 
different  communication  channels  have  to  be  used  to  achieve  similar  objectives  with  most  of  the  learners.  Spatial 
intelligence  is  commonly  linked  to  visualisation  and  can,  like  all  intelligences,  be  improved  by  various  activities 
(e.g.  in  chemistry  through  use  of  models  of  chemical  stmctures),  which  should  be  increasingly  incorporated  in  the 
teaching  and  learning  process. 

In  chemistry,  three  levels  of  visualisation  are  defined,  which  we  explain  by  the  example  of  chemical  experiments.  At 
the  macro  level,  the  real  world  is  representable  by  photographs,  drawings  or  video  clips  of  the  laboratory 
experiments.  The  3D  models  of  chemical  structures  or  animations  of  chemical  changes  represent  the  micro  level.  At 
the  symbolic  level,  chemical  change  is  visualised  by  chemical  equations  or  graphs.  In  order  to  achieve  the  learning 
objective,  all  three  levels  have  to  be  correlated. 

In  this  paper  we  present  our  work  on  the  design,  development  and  evaluation  of  interactive  multimedia  teaching 
units  and  web  based  teaching  resources  to  be  used  at  the  primary  and  high  school  level.  They  can  be  used  either  for 
individual  learning  or  in  parallel  with  the  traditional  teaching  tools  in  the  classroom  situation. 

Development  of  multimedia  chemistry  teaching  materials 

In  the  design  and  implementation  phase  of  the  underlying  CD-ROM  applications  of  teaching  materials  all  three 
different  levels  of  visualisation  in  chemistry  are  taken  into  account.  By  using  video,  animations,  and  simulations 
coupled  with  interactive  questions,  games  and  problem-solving  tasks,  the  users  have,  e.g.  in  the  case  of  a chemical 
reaction,  the  possibility  to  observe  chemical  changes  in  reactants  and  products  as  well  as  energy  changes,  to  follow 
them  step-by-step,  and  then  evaluate  their  observations.  We  try  to  simultaneously  activate  as  many  of  their  receptor 
channels  as  possible,  and  the  use  of  nonvisual  and  static  elements  is  kept  to  the  minimum;  the  learning  and  testing 


segments  are  tightly  interlaced.  Through  communication  - either  live  discussion  with  a teacher  or  via  guided  online 
discussion  groups  - the  concepts  learned  are  further  clarified  and  relevant  information  is  shared  among  the  users. 

The  use  of  existing  multimedia  teaching  materials  in  schools  is  supported  by  the  intensive  in-service  training 
seminars  for  chemistry  teachers.  Teachers  become  prepared  for  active  involvement  in  the  development  of  teaching 
materials,  produced  by  the  University.  To  facilitate  the  needs  of  teachers  and  students,  a specialized  chemistry 
website  named  Keminfo  was  established.  It  includes  a module  on  the  visualisation  of  chemical  structures  and 
processes,  together  with  tutorials  for  existing  high  quality  educational  multimedia  CD-ROMs.  It  also  provides  the 
users  with  original  interactive  teaching  materials,  prepared  either  by  teachers,  through  small-scale  projects,  or  by  the 
university  lecturers.  The  teaching  materials  vary  in  size  and  complexity,  the  emphasis  being  on  the  applications 
using  visualisation  techniques  (e.g.  interactive  maps  of  chemical  reactions)  and  interactivity  (collections  of 
interactive  exercises  and  games). 

Evaluation  of  multimedia  chemistry  teaching  materials 

In  spite  of  the  rich  assortment  of  educational  CD-ROMs  for  chemistry,  very  little  has  been  published  regarding  the 
impact  of  multimedia  on  the  knowledge  and  motivation  of  students,  let  alone  the  cost-effectiveness  of  these 
products.  In  a research  project,  our  own  interactive  multimedia  CD-ROM,  "Light  and  Chemical  Change",  was 
evaluated.  The  impact  of  multimedia  on  cognitive,  motivational  and  motoric  development  of  students  was 
investigated  in  a qualitative  study.  The  research  involved  a sample  of  50  third-year  secondary  school  students,  who 
were  divided  into  two  proportional  groups,  namely  the  experimental  and  the  control  group.  Both  groups  were  pre- 
tested a week  prior  to  the  experiment  through  multiple-choice  and  open-answer  written  tests,  so  that  the  initial 
difference  between  the  groups  could  be  established.  The  experimental  group  then  worked  with  the  interactive  CD- 
ROM  in  pairs,  and  they  were  given  three  hours  to  learn  a specific  content,  presented  by  video  clips,  animations  and 
simulations,  accompanied  by  interactive  questions  with  intelligent  feedback.  This  group  did  not  use  any  other  source 
for  knowledge  consolidation.  Students  from  the  control  group  were  also  given  three  hours,  with  several  course 
books  available,  starting  with  the  teacher's  explanation  of  the  subject  topic.  A week  after  the  experiment  both  groups 
took  a post-test  focused  on  understanding  the  concepts  at  the  microscopic  level  of  visualisation.  The  performance  of 
both  groups  before  and  after  the  learning  session  was  compared.  Whereas  the  groups  did  not  significantly  differ  at 
the  beginning,  the  results  of  the  post- test  show  significant  difference  in  favor  of  the  experimental  group,  which  can 
be  largely  attributed  to  the  visualisation  elements  included  in  the  CD-ROM  and  its  interactivity.  In  our  ongoing 
research  project,  a study  aimed  at  improving  the  efficiency  of  visual  elements  used  in  multimedia  teaching  units  is 
being  conducted.  We  are  investigating  the  correlation  between  the  spatial  abilities  of  the  individual  students  and 
their  ability  to  correctly  interpret  different  chemical  processes  visualised  by  application  of  the  diverse  tools. 

Conclusion 

Active  involvement  of  teachers  in  developing  high  quality  multimedia  teaching  materials  is  the  key  factor  for  their 
regular  use  in  schools.  The  research  on  the  effects  of  such  materials  is  needed  to  provide  the  developers  with  the 
feedback.  The  use  of  visualisation  elements  at  all  three  levels  is  crucial  in  chemistry  teaching,  however  the  learning 
process  has  to  be  supported  by  discussion  and  testing. 
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Abstract: 

A unique  service  organization,  consisting  exclusively  of  student  employees,  exists  within  the  University  of 
Wisconsin’s  Information  and  Media  Technologies  Division,  Known  as  Student  Technology  Services  (STS),  this 
student  managed  organization  is  tasked  with  the  operation  of  various  computer,  media  and  technology  related 
campus  services.  STS  employees  work  in  all  areas  of  I&MT  including  the  Campus  Computer  Labs,  Classroom 
Support,  Desktop  Support,  Help  Desk,  Network  Services,  Peer  Training,  Photo  Services,  Printing  Services, 
Shortcourses,  Technical  Resources,  Video  and  Multimedia  Production  Services,  TV  Engineering,  Distance 
Education  Technologies,  Visual  Design  Services,  and  Web  Maintenance  Services,  More  than  300  students  are 
expected  to  be  employed  in  STS  by  Fall  2000, 

Student  Technology  Services  (STS): 

Student  Technology  Services  (STS)  is  a student  staffed,  student  managed  organization  responsible  for  delivering 
technology  and  media-related  services  to  the  campus  community  of  the  University  of  Wisconsin-Milwaukee,  While 
empowered  student  IT  workers  are  employed  to  provide  a service,  they  are  at  the  same  time  engaged  in  a learning 
experience.  They  are  preparing  themselves  for  their  future  careers  through  "experiential  learning". 

Student  employees  of  STS  come  from  virtually  every  academic  major.  Whereas  many  universities  employ  primarily 
technical  majors  under  the  mistaken  belief  that  only  they  can  satisfy  their  campus  community's  technology  needs, 
STS  instead  looks  for  interpersonal  skills,  drive,  and  willingness  to  learn.  STS  has  discovered  that  customer  service 
and  management  skill  sets  have  become  as  important  as  technical  skills  in  the  delivery  of  information  and  media- 
related  technology  services.  Because  most  STS  student  employees  do  not  have  technical  back^ounds,  a carefully 
tailored  training  curriculum  was  developed  and  implemented.  The  purpose  of  the  curriculum  is  to  empower  STS 
student  employees  by  giving  them  the  technical  skills  they  will  need  to  excel  in  their  jobs  as  well  as  important 
professional  and  life  skills  they  will  need  to  excel  in  their  future  careers. 

The  STS  training  curriculum  becomes  an  education  when  the  learning  is  demonstrated  through  experience  (i.e,, 
implemented  on  the  job)  and  when  the  experience  is  reflected  upon  (i.e.,  in  an  extensive  oral  and  written  evaluation.) 
Because  completing  the  multiple  levels  of  the  curriculum,  using  the  skill  sets  on  the  job  and  participating  in  the 
evaluation  process  are  all  mandatory  conditions  of  employment,  STS  provides  the  setting  for  experiential  learning  at 
institutions  of  higher  education.  Many  of  the  classes  in  the  STS  curriculum  are  developed  and  taught  by  STS 
students  themselves.  These  STS  Peer  Trainers  do  not  impose  upon  the  institution's  academic  teaching  mission,  yet 
are  an  integral  part  of  adding  value  to  the  learning  experience  of  STS  students.  STS  Peer  Trainers  are  evaluated  after 
each  class  they  teach  and  are  charged  with  making  the  necessary  adjustments  to  their  lesson  plans. 

In  addition  to  the  experiential  learning  provided  by  the  curriculum,  STS  students  are  encouraged  to  transfer  to  other 
departments  each  year  in  order  to  learn  and  practice  new  skills.  The  exposure  they  gain  in  these  other  areas 
strengthens  their  learning  experience  and  enables  them  to  identify  their  personal  and  professional  strengths  and 
weaknesses.  STS  students  are  required  to  maintain  a portfolio  of  their  diverse  work  experience  and  training.  The 
portfolio  serves  as  the  reflection  piece  to  the  classroom  and  on-the-job  experience,  thereby  further  adding  value  to 
TS'  philosophy  of  experiential  learning, 

STS'  focus  on  experiential  learning  has  been  met  with  welcomed  enthusiasm  at  UW-Milwaukee.  STS  student 
employees  are  sought  after  not  just  for  their  learned  technical  skills,  but  also  for  the  maturity  and  professionalism 


O 
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that  they  acquire  through  on-the-job  experience  and  reflection  pieces.  UWM  department  heads  specifically  request 
STS  student  employees  to  be  placed  in  their  departments  to  troubleshoot  and  to  maintain  department  web  pages. 

STS'  example  of  experiential  learning  is  now  being  introduced  to  the  26  other  campuses  within  the  University  of 
Wisconsin  System.  Wisconsin's  governor  recognized  the  benefits  of  experiential  learning  as  a means  of  preparing  a 
qualified  workforce  in  the  state  that  can  remain  competitive  even  in  the  fluid  environment  of  information 
technology.  The  Governor's  Office  recommended  increased  funding  for  training  IT  student  workers  throughout  the 
entire  University  of  Wisconsin  system.  His  decision  was  based  solely  on  the  UWM  STS  model  and  its  success.  The 
biennium  budget  was  signed  into  law  in  October,  1999  thus  propagating  the  STS  model  of  experiential  learning 
throughout  the  state. 

Tlie  STS  program  has  generated  strong  interest  from  m.any  academic  and  non-acadcmic  partners.  W e arc  exploring 
relationships  with  public  school  systems  and  with  leaders  in  private  industry  to  provide: 

• A pre-college  program  to  create  work  experience  and  training  in  technology  for  high  school  students. 

• A similar  work  program  focused  on  the  two  year  technical  college  in  Milwaukee 

• Summer  internships  and  potential  scholarships  for  the  STS  student  employees  at  UWM 

• Meaningful  industry  relationships  for  students  leading  to  employment  upon  graduation. 

We  have  met  with  enthusiasm  and  willingness  by  all  participants  to  work  together  on  these  new  relationships.  We 
look  forward  to  a greatly  enhanced  student  empowerment  program  when  the  new  partnerships  are  in  place. 

Internet  Connection 

For  additional  information,  please  visit  the  STS  web  site  at:  www.uwm.edu/IMT/STS 
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Abstract:  This  paper  is  a report  of  an  ongoing  longitudinal  evaluation  assessing  the  effect  of 
level  of  technology  training  on  teachers'  attitudes  and  comfort  toward  using  and  integrating 
technology  into  the  curriculum.  Multivariate  and  univariate  analyses  were  used  on  an  initial 
sample  of  54  K-12  public  and  private  school  teachers  from  a Northeast  state  undergoing  two 
levels  of  technology  training  who  participated  as  part  of  a Technology  Literacy  Challenge 
Grant.  Initial  findings  support  and  extend  previous  research  that  shows  teachers  become  more 
comfortable  with  using  and  integrating  technology,  and  possess  more  positive  attitudes  toward 
technology,  as  the  extent  of  training  increases. 


Introduction 

Although  over  8.5  million  computers  are  being  used  for  instruction  in  the  U.S.,  and  85%  of  K-12 
teachers  have  access  to  computers  and  other  technology  (Survey  finds.. .,  1995),  only  half  of  all  teachers  report 
using  computers  for  instruction  (Hunt  & Bohlin,  1995).  Social  learning  theory  (Bandura,  1977)  predicts  that 
failure  of  teachers  to  use  computers  will  vicariously  teach  students  not  to  use  computers.  This  lack  of  computer 
use  may  be  related  to  attitudes  (e.g.,  anxiety)  toward  technology  (Pina  & Harris,  1993).  Although  of  all 
people  who  use  computers  experience  some  degree  of  “computerphobia”  (Rosen  & Maguire,  1990),  when 
teachers  experience  it,  this  interferes  with  their  ability  to  integrate  technology  into  the  curriculum  (Hunt  & 
Bohlin,  1995).  The  strongest  predictor  of  computerphobia  is  experience  (Rosen  & Weil,  1995).  Limited 
research  has  shown  anxiety  towards  computers  decreases  after  training  and  practice  (Cambre  & Cook,  1987), 
yet  only  29%  of  teachers  report  having  more  than  five  hours  of  technology  training  in  the  past  year 
(Technology  Counts  '99,  1999). 


The  Study 

The  purpose  of  this  study  was  to  investigate  the  effect  of  level  of  technology  training  on  teachers*  attitudes 
toward,  confidence  using,  and  comfort  with  integrating  technology.  Attitudes  were  represented  by  three  constructs: 
anxiety,  avoidance,  and  fear  of  stereotyping.  Confidence  was  measured  in  three  areas:  software,  hardware,  and 
communication  tools.  Comfort  with  integration  was  assessed  by  participants  indicating  one  of  sbc  stages  (awareness, 
learning  the  process,  understanding  and  application,  familiarity  and  confidence,  adaptation  to  other  contexts,  and 
creative  application  to  new  contexts).  Training  occurred  at  two  levels  (Level  I;  5 days  traming  on  using  Power  Point  or 
HyperStudio  in  the  classroom:  Level  II;  Level  I training  plus  3 additional  days  training  in  Web  page  design  for 
curriculum  use).  Participants  were  54  public  and  private  school  educators  participating  in  a regional  Technology 
Challenge  Grant  training  program  in  a Northeast  state.  Twenty-six  educators  experienced  Level  I training  while  28 
educators  experienced  Level  II  training.  Over  300  educators  are  expected  to  be  trained  at  both  levels  by  the  end  of  this 
grant  year.  An  end-of-year  paper/pencil  survey  was  administered  to  the  first  cohorts  of  educators  experiencing  both 
Level  I and  Level  II  training  so  far  this  school  year.  Ten  items  measured  attitudes  toward  technology  (anxiety, 
avoidance,  and  fear  of  stereotyping)  on  a six-point,  Likert-type  scale  (l=strongly  agree,  6=strongly  disagree).  Eighteen 
items  measured  confidence  in  using  technology  (hardware,  software,  communication  tools)  on  a sbc-point  scale 
(l=extremely  confident,  6=not  at  all  confident).  The  Stages  of  Adoption  Scale  (Knezek  & Christensen,  1999) 
measured  comfort  with  using  and  integrating  technology  into  the  curriculum. 
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Findings 


A multivariate  analysis  of  variance  (MANOVA)  was  used  to  analyze  the  data.  The  fixed,  categorical 
independent  variable  was  level  of  technology  training  (Level  I and  Level  II).  The  continuous,  random 
dependent  variables  were  confidence  with  using  technology  (software,  hardware,  and  communication  tools), 
and  attitudes  toward  technology  (anxiety,  fear  of  stereotyping,  and  avoidance).  Findings  indicated  no 
significant  difference  for  level  of  training  on  confidence  in  using  technology  (F=1.26;  df==3,37;  p>.05). 
However,  a significant  difference  was  found  for  the  effect  of  level  of  training  on  the  construct  of  attitudes 
toward  technology  (F=2.83;  df=3,50;  p<.05).  Examination  of  the  univariate  Fs  indicated  a significant  portion 
of  the  construct  was  avoidance  (F=6.78;  df=3,50;  p<.05)  and  anxiety  (F=6.61;  df=3,50;  p<.05),  accounting  for 
15%  of  the  total  variance  (Wilkes  !ambda=.85).  A univariate  analysis  of  variance  (ANOVA)  was  used  to 
analyze  the  effect  of  the  fixed,  categorical  independent  variable  (level  of  training)  on  the  continuous,  random 
dependent  variable  of  comfort  using  and  integrating  technology  (six  stages:  awareness,  learning  the  process, 
understanding  and  application,  familiarity  and  confidence,  adaptation  to  other  contexts,  and  creative  application 
to  new  contexts).  A significant  difference  was  found  for  level  of  training  (F=5.00,  df=l,50,  p<.05)  on  stage  of 
comfort.  Strength  of  association  showed  that  nine  percent  of  the  variability  in  level  of  comfort  could  be 
accounted  for  by  level  of  training.  Those  with  more  training  were  more  comfortable  with  integration. 


Conclusion 

These  findings  support  previous  research  that  shows  teachers  who  receive  training  in  using  and 
integrating  technology  into  the  curriculum  experience  less  anxiety  toward,  and  more  comfort  with,  that 
technology.  This  study  adds  to  existing  literature  by  showing  that  additional  training  with  more  advanced 
technological  software  further  decreases  computer  anxiety  and  increases  comfort  with  integrating  technology 
above  and  beyond  less,  and  lower-level,  training.  If  teachers  are  expected  to  integrate  technology  into  the 
curriculum  and  model  appropriate  technology  utilization  behavior  for  their  students,  they  need  more  training 
than  the  levels  currently  being  received  by  most  educators. 
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Abstract:  The  Vocational  Education  and  Training  (VET)  sector  in  Australia  is  coming  under 
increasing  pressure,  both  nationally  and  internationally,  to  deliver  courses  in  flexible  ways  that 
incorporate  the  latest  in  information  and  communications  technologies  such  as  the  Internet  and 
the  World  Wide  Web.  LearnScope  is  a national  initiative  funded  by  the  Australian  National 
Training  Authority  (ANTA).  Its  aim  is  to  provide  professional  development  support  to 
individuals  and  work  groups  in  order  to  develop  skills  and  capabilities  for  the  online  environment 

A preliminary  study  of  LearnScope ’s  Virtual  Learning  Community  (VLC)  was  undertaken  using 
a questionnaire  based  survey  plus  analysis  of  transcripts  of  forum  discussions  and  published 
stories.  The  results  highlight  the  complexity  of  the  role  of  providing  professional  development 
for  establishing  online  learning  communities.  The  major  areas  of  Technical  Skills  and  Computer 
Literacy,  Writing  and  Communication  Skills,  issues  of  Interactivity,  and  the  development  of 
Self-related  attributes  are  identified  as  important  if  the  VET  sector  is  to  build  a critical  mass  of 
creative,  capable  instructors  to  work  in  the  online  environment. 


The  Changing  Context  of  Vocational  Education  and  Training 

As  the  geographical  boundaries  of  Vocational  Education  and  Training  (VET)  in  Australia  become  blurred,  both 
nationally  and  internationally,  the  competitive  edge  of  providers.  Registered  Training  Organisations  (RTO’s), 
will  not  only  be  the  courses  and  training  programs  they  offer  but  the  quality  of  instructor  support  and  service 
provided  to  the  learner.  Information  and  communications  technologies  (ICT)  such  as  the  Internet  and  the  World 
Wide  Web  has  seen  online  interactivity  emerge  as  a critical  factor  in  delivering  learning  materials  and 
supporting  learners  and  instructors.  A series  of  Australian  National  Training  Authority  (ANTA)  reports  (1994, 
97  and  98)  provide  details  to  developments  in  this  area. 

Traditionally  VET  practitioners,  in  particular  those  in  Technical  and  Further  Education  (TAPE)  Colleges,  have 
been  provided  with  professional  development  to  update  their  skills.  In  recent  years  this  has  been  through  work 
based  learning  using  a range  of  processes  such  as  mentoring,  coaching,  workshops,  discussion  groups, 
conferences  and  site  visits.  In  the  area  of  ICT,  professional  development  programs  have  primarily  focussed  on 
technical  skills  development,  that  is,  learning  how  to  use  the  technology  without  focusing  on  the  ‘deeper’  level 
of  critically  understanding  the  benefits  and  restrictions  of  using  these  tools  in  the  learning  environment. 

LearnScope  and  its  Virtual  Learning  Community 

LearnScope  is  a national  professional  development  initiative  that  is  available  to  work  teams  and  individuals 
within  RTO’s.  It  is  focussed  on  the  development  of  skills  to  deliver  training  in  increasingly  flexible  ways  using 
new  ICT’s.  The  LearnScope  home  page  provides  more  details:  http://www.leamscope.anta.gov.au/cover.html 

For  the  individual  instructors  it  is  LeamScope’s  Virtual  Learning  Community  (VLC)  that  provides  a range 
of  activities  in  several  zones.  These  activities  include: 

■ About  LearnScope  - an  information  about  the  national  LearnScope  project,  guidelines  and  application 
forms  and  contact  people  within  the  project. 

■ Key  Issues  - an  information  area  which  provides  clustered  groupings  of  key  issues  relating  to  flexible 
delivery.  This  includes  resources,  text,  access  to  software,  people  or  web  sites. 

■ Project  Gallery  - the  area  where  LearnScope  teams  publish  their  plans,  interim  and  final  reports. 

■ Expert  Spruik  - an  area  to  introduce  key  people  who  will  contribute  to  the  LearnScope  VET  community. 


■ RantZone  - an  area  for  people  to  practice  their  publishing  skills,  request  information  or  make  a statement. 

■ Evaluation  - an  area  for  the  Learn  Scope’s  project  evaluator  to  submit  information  about  evaluation  issues 

■ What 's  New  - a frequently  changing  area  which  provides  a summary  of  key  information  for  all  users. 

■ Forums  - an  area  for  both  public  and  private  discussions. 

■ Chat  Room  - an  area  for  planned  synchronous  discussions. 

While  the  VLC  is  primarily  for  the  support  of  Learn  Scope  project  teams,  anyone  interested  in  the  application  of 
new  learning  technologies  to  achieve  more  flexible  learning  and  professional  development  within  the  vocational 
education  and  training  sector  is  welcome  to  register  and  participate. 

The  Survey:  Methodology  and  Sample 

Data  was  gathered  in  1999  during  two  rounds  of  LcamScope  projects  and  in  two  formats:  a questionnaire,  and 
forum  discussions  and  published  stories  (Weathereley,  2000).  A total  of  58  questionnaires  were  distributed  and 
33  were  returned  giving  a response  rate  of  57%.  A total  of  153  postings  to  5 forums  (average  14,  range  10  to  21 
participants  per  forum)  as  well  as  336  published  stories  were  also  collected.  These  data  was  then  used  to: 

(i)  Identify  the  reactions,  feelings  and  thoughts  to  using  a virtual  learning  community  in  the  context  of  the 
national  VET  system  and  to  gain  insight  as  to  the  level  of  experience  of  users. 

(ii)  Identify  how  VET  practitioners  used  an  asynchronous  forum  area  set  up  specifically  for  learning. 

Findings  and  Recommendations 

It  is  recommended  that  for  instructors  or  professional  development  planners  who  are  engaging  in  developing 
online  learning  plans  that  the  following  areas  of  skills  development  and  capabilities  must  be  addressed: 

(i)  Technical  Skills  and  Computer  Literacy:  To  achieve  the  best  possible  learning  environment  it  is  necessary 
to  go  beyond  the  simplistic  operational  understanding  of  ICT.  It  is  important  to  have  the  capability  to  make 
critical  judgements  about  its  appropriateness  and  to  develop  ideas  about  teaching  and  learning  within  the 
environment.  The  challenge  for  the  national  VET  system  is  getting  more  people  involved  in  a professional 
community  which  as  Spitzer  and  Wedding  (1995)  says  “...requires  the  active  and  willing  participation  of 
teachers  in  the  process  of  examining,  reflecting  on,  experimenting  with,  and  ultimately  changing  the  way  they 
practice  - in  the  context  of  a professional  community  that  allows  them  to  enrich  their  understanding  of  subject 
matter  and  to  consider  issues  about  how  students  learn.” 

(ii)  Writing  Skill  and  Communication  Style:  For  online  learning  communities  to  be  successful,  the  writing 
and  language  skill  must  be  conducive  to  the  audiences  ability  so  that  group  interaction,  sharing  and  participation 
can  lead  to  an  environment  based  on  relationship  development. 

(iii)  Interactivity 

(iv)  Understanding  the  range  of  interactive  possibilities  and  promoting  an  environment  which  encourages  active 
interaction  online  between  people  is  vital.  Understanding  the  significance  of  the  human  medium  and  the  levels 
of  interactivity  will  provide  teachers  with  the  potential  to  make  critical  judgements  about  student  learning. 

(v)  Self-related  attributes:  Developing  self-related  attributes  is  significant  in  professional  development  for  the 
online  learning  environment.  Teachers  need  to  identify,  develop  and  model  these  attributes.  Self-related 
attributes  should  also  be  an  important  aspect  of  learner  development. 

(vi)  Teaching  and  Learning:  Knowledge  building  in  order  to  gain  maximum  benefits  of  an  online  community 
and  to  provide  a quality  learning  environment,  requires  a range  of  strategies.  Based  on  the  constructivist  model 
where  learning  builds  on  previous  professional  and  personal  experiences,  the  importance  of  linking  practice 
with  discussion,  dialogue  and  debate  in  this  new  learning  field  is  critical.  Professional  development  activities 
need  to  incorporate  pedagogical  dimensions.  These  recommendations  must  be  incorporated  into  instructor’s 
professional  development  plans  otherwise  a quality  online  learning  environment  will  not  be  achieved. 
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Abstract:  The  purpose  of  this  study  is  to  explore  students’  perceptions  of  the  use  of  World  Wide  Web  (WWW)  technology  as  an 
instructional  resource  specifically  customized  to  support  teaching  and  learning  practices  on  campus-based  classes.  While  there  are  many 
efforts  to  design  meaningful  learning  environments  there  is  very  little  current  instructional  design  literature  on  fostering  and  support  of 
Web-based  environments.  This  study  examined  college  students’  perspectives  on  the  usefulness  of  specific  Web  site  functions  used  for 
teaching  and  learning  purposes,  in  an  effort  to  optimize  the  use  of  Web-based  instructional  support  resources. 

Review  of  Relevant  Literature 

The  1999  Campus  Computing  Project’s  (CCP)  national  annual  survey  of  information  technology  in  higher 
education  reported  that  many  higher  education  institutions  are  providing  more  services  via  the  WWW  (CCP,  1999). 

For  example,  more  than  half  (69.5%)  of  these  institutions  provide  online  undergraduate  applications  on  their  Web 
sites.  More  than  three-fourths  (77.3%)  make  the  course  catalog  available  online.  A quarter  (25%)  of  the  institutions 
make  library-based  course  reserves  readings  available  online  and  almost  a half  (46.5%)  offer  one  or  more  full 
college  courses  online  via  the  Internet  and  the  WWW.  Similarly,  the  percentages  of  college  courses  using  the 
WWW  in  the  syllabus  rose  from  7%  in  1995  to  28%  in  1999  (CCP  1996).  This  number  quadrupled  by  1999 
meaning  that  more  than  one  in  every  four  classes  has  a Web  page.  In  fact,  compared  to  e-mail  and  general  use  of 
Internet  resources,  use  of  Web  pages  has  shown  the  highest  rate  of  increase  over  the  six-year  period.  E-mail  had 
already  been  in  use  in  1994  rising  to  54%  by  1999  up  from  44%  in  1998,  and  20%  in  1995.  Khan  (1997)  provided  a 
definition  of  Web-based  instruction  used  for  the  purpose  of  this  study. 

Web-based  instruction  (WBI)  is  a hypermedia-based  instructional  program  which  utilizes  the 
attributes  and  resources  of  the  World  Wide  Web  to  create  a meaningful  learning  environment 
where  learning  is  fostered  and  supported  [italics  added],  (p.  6) 

At  a university  level,  the  Syracuse  University  Faculty  Syllabus  Web  site  available  at  http://sYllabus.svr.edu  is  an 
example  of  a growing  list  of  campus-based  course  Web  sites.  For  the  purposes  of  this  study,  a preliminary  analysis 
of  175  Syracuse  University  Web  sites  was  conducted  during  the  fall  1998  semester. 

Theoretical  Framework 

A conceptual  framework  was  developed  to  ground  and  guide  the  research  conducted  in  this  study.  There  is  evidence 
from  research  findings  that  certain  principles  exist  in  most  successful  teaching  and  learning  practices  (Chickering  and 
Gamson,  1987,  1991).  Second,  there  is  evidence  in  the  literature  that  the  application  of  instructional  design  principles  to  the 
teaching  and  learning  practices  leads  to  more  effective  learning  (Gagne’  1975,  Gagne’,  Briggs,  & Wager,  1992).  One  of  the 
important  measures  of  effective  instruction  is  how  students  feel  about  the  instructional  process  (Dick  & Reiser,  1 989).  Based 
on  the  above  premises,  one  can  make  the  assumption  that  a well-designed  instructional  Web  site  can  support  and  facilitate 
good  teaching  and  learning  practices  and  therefore  can  be  perceived  as  a valuable  instructional  resource  for  both  faculty  and 
students. 

Purpose  of  Study  and  Research  Questions 

The  purpose  of  this  study  was  to  examine  the  opinions  and  perceptions  of  college  students'  use  of  the  course  Web 
site  as  an  instructional  resource  for  classroom-based  teaching  at  a private  university  in  the  northeast  of  the  United  States.  The 
focus  of  the  study  was  on  the  identification  of  Web  site  functions  that  students’  perceived  as  supporting  and  enhancing  of 
their  learning  experiences  by  exploring  students’  perceptions  of  the  usefulness  of  the  various  Web  site  functions.  The 
following  question  was  posited:  How  useful  did  students  perceive  their  course  Web  sites? 


Method 


142  primarily  undergraduate  students  enrolled  in  nine  courses  that  employ  Web  sites  as  course  resources  responded 
to  a questionnaire  on  their  use  of  their  respective  course  Web  sites.  Initially,  all  Web  sites  were  first  screened  using 
an  instrument  developed  by  the  researcher  to  evaluate  the  Web  sites’  instructional  quality  and  design  based  on 
principles  of  instructional  design.  On  the  student  Web  site  perception  questionnaire  (SWPS),  functions  pertaining  to 
the  Usefulness  dimension  were  construct  validated  using  confirmatory  factor  analysis.  Two  indexes  were 
constructed  for  the  Usefulness  dimension  using  items  that  loaded  (>0.60)  on  these  two  factors:  GUIDE  and 
PREPARE.  The  first  represents  practice,  elicitation,  and  feedback  functions.  The  second  represents  functions  that 
facilitate  recall  of  prior  learning  and  enhance  retention  and  transfer  through  communication,  enrichment  and 
remediation.  All  functions  are  derived  from  Gagne',  Briggs,  and  Wagers’  (1992)  nine  events  of  instruction  and 
Chickering  and  Ehrmann’s  (1996)  principles  of  good  teaching  and  ieaiiiiiig  pi'actices  using  technology  as  lever. 


Results 


Results  showed  that  most  functions  were  useful  with  a mean  score  for  the  PREPARE  index  (n=132, 
M=1.87,  SD=0.74,  scale  0-4  where  4=highly  useful)  and  a higher  one  for  GUIDE  (n=131,  M=2.20,  SD=0.71,  scale 
0-4  where  4=highly  useful). 

The  PREPARE  index  consisted  of  functions  including:  (a)  the  use  of  pictures,  tables,  diagrams,  etc.  to 
recall  or  present  new  information  (n=  78,  M=2.79,  SD=1.00);  (b)  the  additional  links  and  information  on  the  course 
Web  site  for  further  study  (n=  96,  M=2.43,  SD=1.16);  (c)  the  links  to  review/pre-requisite  material  to  help  recall 
(n=91,  M=1.87,  SD=0’74),  and  (d)  overall  usefulness  of  the  course  Web  site  (n=132,  M=2.63,  SD=1.13). 

The  GUIDE  index  consisted  of  functions  including:  (a)  the  opportunity  to  ask  questions  online  (e-mail, 
listservs,  hypermail,  etc.  (n=  112,  M=2.74,  SD=1.00);  (b)  posting  graded  assignments/Home  Work  on  the  Web 
(n=64,  M=2.66,  SD=1.30);  (c)  online  practice  assignments  (n=47,  M=2.64,  SD=0.85);  (d)  online  feedback  on 
various  assignments  (n=  73,  M=2.60,  SD=1.00);  (e)  situations,  case  studies,  formulas,  problems,  simulations,  etc. 
posted  on  the  course  Web  site  to  improve  learning  (n=  65,  M=2.52,  SD=0.99);  (f)  online  self-assessment  quizzes  (n= 
49,  M=2.51,  SD=1 .23);  and  (g)  instructions  on  how  to  navigate  the  Web  site  and  download  software,  get  an  account, 
subscribe  to  a listserv  (n=  100,  M=2.30,  SD=1.10). 

Discussion 

Key  findings  from  the  study  are  indicated  below  using  Chickering  and  Ehrmann's  (1996)  seven  principles  of  good 
teaching  and  learning  practices  for  undergraduate  education  and  Gagne’,  Briggs,  and  Wager’s  (1992)  nine  events  of 
instruction  as  the  framework  for  discussion.  Findings  for  the  PREPARE  index  related  to  assisting  students  to  recall 
information  and  to  transfer  new  knowledge  were  supported  by  functions  representing  two  principles:  (a)  Good 
practice  gives  prompt  feedback  and  (b)  Good  practice  encourages  contacts  between  students  and  faculty  in  that  the 
most  useful  functions  were  those  that  enabled  students  to  communicate  with  their  instructors  and  to  receive  timely 
feedback.  Electronic  mail,  computer  conferencing,  and  the  WWW  increase  opportunities  for  students  and  faculty  to 
interact  and  communicate  much  more  efficiently  than  before.  In  addition,  the  nature  of  communication  becomes 
more  thoughtful,  deliberate,  convenient  and  safe.  Moreover,  the  negative  aspects  of  face- to  face  confrontation  are 
lessened.  Findings  from  the  GUIDE  index  indicated  that  Gagne’s  et.  al.  (1992)  instructional  events  pertaining  to 
presentation,  practice,  feedback,  and  elicitation  of  new  knowledge  and  skills  were  overall  most  useful  functions. 
The  multimedia  capabilities  of  the  Web  allow  for  an  almost  seamless  integration  of  email  capabilities,  video  clips, 
sound,  images,  and  animation  thereby  allowing  for  multiple  modes  of  communication  through  feedback  and 
assessments.  In  conclusion,  in  terms  of  theoretical  implications,  the  study  supported  the  usability  of  Gagne  s events 
of  instruction  in  particular  the  ones  related  to  presentation,  practice,  elicitation,  and  feedback,  as  a viable  means  of 
designing  course  Web  sites  to  support  teaching  and  learning  practices.  The  study  also  provided  empirical  findings 
to  support  Chickering  and  Ehrmann’s  (1996)  principles  of  good  teaching  and  learning  practices  in  particular  the  ones 
related  to  prompt  feedback  and  student-faculty  interactions.  It  is  recommended  that  course  Web  site  design  could  be 
optimized  to  include  more  instructional  functions  such  as  elicitation  of  responses,  practice,  feedback,  and  increased 
student-faculty  interactions  using  the  interactive  and  visual  and  auditory  display  capabilities  of  the  medium. 
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Abstract:  This  work  in  progress  short  paper  is  a report  on  subjects'  evaluations  of  a 
web-based  language  course's  motivational  aspects  tluough  WebMAC,  Website 
Motivational  Analysis  Checklist,  developed  by  Small  and  Amone  (1998).  The  subjects 
were  26  adult  students  who  participated  in  a basic  online  English  course  for  Internet 
users,  focused  on  informal  asynchronous  and  synchronous  writing  and  communication 
skills,  called  Surfing  and  Learning.  This  course  is  designed  by  teachers  and  researchers 
of  the  EDULANG  research  group.  The  instrument  measured  the  level  of  four  motivation 
factors  which  provided  the  results  for  two  dimensions  within  an  Expectancy-Value 
framework:  Value  and  Expectancy  for  Success.  The  evaluation  results  showed  a high 
level  of  Value  dimension  and  Expectancy  for  i^wcce^^,  although  factors  linked  to 
Expectancy  for  Success,  such  as  Organization  and  Ease  of  Use  showed  more  variation 
in  scores  than  those  linked  to  Value,  such  as  Meaning.  Although  WebMAC  seems  to  be 
sensitive  to  student  motivation,  other  kinds  of  data  are  required  to  a complete  course 
evaluation.  It  will  continue  to  be  used  for  the  ongoing  evaluation  of  Surfing  and 
Learning  and  other  courses  offered  by  EDULANG. 


Introduction 

Most  researchers  agree  that  motivation  is  a difficult  construct  to  work  with  (Weiner,  1990). 
Wlodkowski  (1999)  says  that  beyond  the  general  understanding  of  motivation  as  a concept  that  explains 
why  people  think  and  behave  as  they  do,  "any  specific  discussion  of  the  meaning  of  motivation  brings  in  a 
cornucopia  of  differing  assumptions  and  terminology"(p.l).  According  to  Domyei  (1998)  motivation  is  a 
"process  whereby  a certain  amount  of  instigation  force  arises,  initiates  action,  and  persists  as  long  as  no 
other  force  comes  into  play  to  weaken  it  and  thereby  terminate  action,  or  until  the  planned  outcome  has 
been  reached."  (p.l  18).  He  also  agrees  with  the  idea  that  motivation  is  a multifaceted  factor  and  no  theory 
has  yet  been  able  to  represent  it  in  its  total  complexity.  Despite  difficulties,  instructional  designers 
continue  to  seek  motivational  frameworks  in  order  to  achieve  a better  design  in  tecnology  based  learning 
environments  such  as  the  WWW  (Cornell  & Martin,  1997).  Duchastel  (1997)  elects  two  main  models. 
One  is  Keller's  ARCS  model,  considering  four  factors  in  motivation  to  learn:  (a)ttention,  (r)elevance, 
(c)onfidence,  and  (s)atisfaction.  The  other  one  is  Malone's  CFC  framework  for  instrinsically  motivated 
inscruction,  which  involves  three  factors:  (c)hallenge,  (f)antasy,  and  (c)uriosity.  ARCS  guided 
interventions  reduced  the  drop  out  rate,  by  half,  of  a Master's  degree  distance  course  intended  to  prepare 
adult  students  for  careers  in  the  areas  of  instructional  design,  job  performance  improvement,  etc. 
(Chyung,  Winiecki  and  Fenner,  1999).  Also  influenced  by  the  ARCS  model,  the  Website  Motivational 
Analysis  Checklist  - WebMAC  (Small  & Amone,  1998,  1999)  was  designed  to  assess  the  motivational 
quality  of  a website.  Small  (1997)  claims  that,  although  the  ARCS  model  is  rooted  in  a number  of 
motivational  theories,  its  main  basis  is  the  expectancy-value  framework  in  which  motivation  to  behave  is 
the  sum  of  individual's  expectancy  for  success  in  a given  task  plus  the  value  attributed  to  success  in  the 
task.  Website  quality  is  measured  within  expectancy-value  framework:  the  user  is  motivated  to  remain  in 
the  website  if  it  has  value  and  if  the  user  expects  to  be  successful  in  the  website  environment.  Domyei 
(1998),  surveying  motivation  in  second  and  foreign  language  learning  (L2)  research  shows  that 
researchers  have  adopted  and  empirically  validated  some  concepts  originally  introduced  by  motivational 
approaches  in  mainstream  psychology,  such  as  expectancy-value  theories,  goal  theories  and  self- 
determination  theory.  Therefore,  WebMAC  was  chosen  here  to  evaluate  motivational  aspects  in  a L2 
web  based  instructional  design.  This  evaluation  is  part  of  a major  research  project  in  which  the  analysis  of 
interactive  flow  is  also  considered  (see  Collins'  paper). 


Methodology 


The  subjects  were  26  students,  male  and  female,  age  29  to  64  , who  participated  in  a web-based 
language  course  in  1999.  The  course.  Surfing  and  Learning  (http://cogeae.uol.com.br/sal)  offers  a basic 
English  for  adult  Internet  users,  focusing  on  informal  asynchronous  and  synchronous  writing  and 
communication  skills.  It  was  designed  by  English  language  teachers,  members  of  EDULANG  research 
group,  under  the  coordination  of  Heloisa  Collins,  in  the  Graduate  Program  in  Applied  Linguistics  of  the 
Catholic  University  of  Sao  Paulo  (LAEL/PUCSP).  WebMAC  versions  3.1  and  4.0  were  translated  into 
Brazilian  Portuguese  and  edited  into  a HTML  format,  so  that  the  students  answered  the  checklist 
anonymously  at  the  end  of  the  course  and  submitted  it  through  a CGI  implemented  web  form.  The  answers 
were  scored  according  to  WebMAC  scoring  guidelines.  Both  versions  were  used  and  the  scores  obtained 
from  3.1  version  were  adjusted  to  version  4.0  factors.  WebMAC  is  composed  of  assertions  or  list  items  to 
be  rated  by  an  evaluator  on  a 4-point  level  of  agreement  scale.  Tlie  32  items  in  version  4.0  measured  4 
factors:  1)  Stimulating  ; 2)  Meaningful;  3)  Organized  and  4)  Easy  to  use.  The  sum  of  first  two  equals  the 
Value  dimension  level;  the  sum  of  the  other  two  equals  the  Expectation  for  Success  dimension  level.  Zero 
to  8 points  for  a factor  means  the  site  needs  much  improvement  on  this  factor;  9 to  16,  it  needs  some 
improvement,  17  to  24  means  the  site  is  highly  motivating.  Version  4.0  contains  four  additional  unscored 
items:  two  yes/no  assertions  and  two  open  questions.  Most  of  list  items  are  the  same  in  both  versions. 
However,  they  are  not  related  to  the  same  factors.  In  both  versions,  each  subject  provides  two  main  scores 
that  are  plotted  on  a bidimensional  grid:  1)  Low  Value  to  High  Value  and  2)  Low  Expectation  for  Success 
to  High  Expectation  for  Success.  The  score  position  reflects  the  motivational  status  of  the  site  in  terms  of 
Value  and  Expectation  for  Success,  (fig.l)[figures  at  http://www.anise.f2s.com/anise-edmedia.html] 

Preliminary  findings 

The  results  (fig.  1)  show  the  scores  on  the  average  to  high-level  area  of  Expectation  for  Success 
and  Value,  Only  two  students  showed  low  Expectation  for  Success  and  only  one  gave  low  scores  to  V alue 
dimension  factors.  The  individuals'  scores  under  each  of  the  four  factors  (fig.2  ) show  less  variation  for 
the  Meaning  factor  than  Organized  and  Easy  to  Use  factors.  The  answers  to  WebMAC  open  questions 
presented  in  4.0  version  revealed  some  students'  satisfactions  and  dissatisfactions.  Despite  high  levels  of 
Value  and  Expectation  for  Success,  the  amplitude  of  scores  in  each  of  the  four  factors  suggests  that  the 
course  can  be  improved  in  Organization  and  Ease  of  Use  factors.  The  interaction  in  communication  areas 
of  the  course  was  pointed  out  by  students  as  a particularly  strong  aspect  of  the  course  design.  The 
complaints  about  tools  and  dictionaries  seem  to  confirm  comparatively  lower  scores  of  Organization  and 
Ease  of  Use  factors.  Future  work  should  consider  the  use  of  WebMAC  during  the  course.  These  results 
can  be  discussed  in  terms  of  guidelines  for  motivating  language  learners  (Domyei,  1998).  Although  the 
present  instrument  seems  to  be  sensitive  to  student  motivation,  other  kinds  of  data  are  required  to  complete 
the  course  evaluation,  such  as  the  interaction  flow  analyses.  The  ongoing  evaluation  of  Surfing  and  Learning 
and  other  courses  offered  by  EDULANG,  such  as  Reading  and  Listening  Comprehension  for  public  high 
school  teachers,  will  continue  to  use  WebMAC. 
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Abstract:  UWired  created  the  web-based  Catalyst  initiative  to  support  innovation  in  teaching 
through  technology.  To  scale  beyond  early  adopters,  Catalyst  has  developed  a new  strategy 
centered  on  collaborative  partnerships  with  campus  teaching  practitioners — learning,  technology, 
and  teaching  research  centers,  libraries,  and  departments — to  disseminate  innovative  uses  of 
technology  in  teaching  through  the  Catalyst  website. 


The  Problem  of  Supporting  Innovative  Teaching  with  Technology 

In  1994,  three  top-level  administrators  at  the  University  of  Washington  were  charged  by  the  Provost  to  “do 
something  about  technology.”  Collaborating  with  faculty,  librarians,  technologists,  and  students  they  quickly  moved 
beyond  a small  pilot,  rounding  up  partners  and  gathering  resources  from  wherever  they  could  to  focus  on  bringing 
technology  into  the  service  of  teaching  and  learning.  Today  the  organization  they  formed — UWired — has  a 
permanent  budget,  a staff  of  eight  professionals  and  over  60  students,  and  has  extended  services  and  support  to 
thousands  of  students  and  faculty. 

As  higher  education  institutions  have  scrambled  to  keep  pace  with  rapid  changes  in  information  technology,  many 
campuses  have  established  faculty  technology  centers,  offering  workshops  and  training  to  interested  educators  and 
perhaps  even  supporting  faculty  technology  projects  to  transform  particular  courses.  These  efforts  have  generally 
targeted  "early  adopter"  faculty  who  are  eager  and  willing  to  teach  themselves  how  to  use  and  implement  new 
technologies.  Resources  are  thus  devoted  under  the  assumption  that  early  adopters  will  play  the  role  of  Johnny 
Apple-Seed,  spreading  innovation  throughout  their  home  departments.  Indeed,  UWired  inaugurated  its  Center  for 
Teaching,  Learning  and  Technology  (CTLT)  to  foster  and  support  instructional  innovation  in  this  manner. 

For  the  most  part,  however,  this  strategy  has  had  a negligible  impact  on  educational  transformation  through 
technology.  Beyond  the  few  early  adopters  are  legions  of  "wary  adopters"  who  look  for  easy  ways  to  improve  their 
teaching  through  technology  but  are  unwilling  to  match  the  time  commitment  of  the  “early  adopters.”  The  classic 
early  adopter  is  drawn  to  technology  for  its  own  sake,  reveling  in  the  "gee  whiz  factor"  of  new  technologies,  but 
"wary  adopters"  are  pragmatic.  Representing  the  mainstream  of  higher  education  faculty,  "wary  adopters"  will  use 
technology  only  when  it  can  be  demonstrated  that  doing  so  will  add  value  to  their  teaching  without  significantly 
reducing  time  already  allocated  to  teaching,  research,  and  service  activities.  Needless  to  say,  the  CTLT  found  it 


quite  difficult  to  scale  up  support  services  while  also  ensuring  that  these  services  meet  the  often  different  needs  of 
"wary  adopters.” 

Creating  a Catalyst  for  Innovative  Teaching  with  Technology 

The  need  to  expand  support  to  a much  wider  audience — and  to  do  so  without  a major  infusion  of  new  staff  or 
relocation  to  a larger  facility — was  clear.  How  to  do  so  was  not.  In  fact,  the  CTLT  faced  four  major  obstacles  posed 
by  the  needs  of  "wary  adopters":  (1)  the  absence  of  a coherent  framework  for  faculty  Web  publishing,  (2)  the 
absence  of  well  explained,  compelling  examples  of  enhanced  teaching  and  learning  through  the  Web,  (3)  the 
absence  of  convenient,  in-person  support  located  in  departments,  and  (4)  the  absence  of  a clear  incentive  for  faculty 
to  explore  the  use  of  technology  in  their  teaching.  After  an  intense  period  of  reflection  and  wide  ranging 
conversations  with  campus  stakeholders  and  faculty  focus  groups,  the  CTLT’s  leadership  team  developed  a new 
framework  for  faculty  support.  The  framework  depended  upon  an  instructional  support  Web  site  that  would  provide 
faculty  with  anytime-anywhere  resources  and  also  serve  as  the  foundation  for  redesigned  instructional  workshops 
and  one-on-one  consulting. 

The  Web  site.  Catalyst,  was  launched  in  1 999  to  provide  this  support  while  placing  pedagogy  and  student  learning  at 
the  center  of  the  discussion  about  technology.  Catalyst  is  based  upon  four  assumptions:  (1)  faculty  want  just-in-time 
learning  and  support;  (2)  they  prefer  to  learn  at  their  own  pace,  in  their  local  environment;  (3)  the  Web  is  or  will 
become  the  vehicle  of  choice  for  just-in-time  information  and  learning;  and  (4)  distributed  support  personnel  are 
best  suited  to  make  the  critical  decisions  about  local  infrastructure  and  local  support.  Given  this,  the  key  function 
for  Catalyst — the  value  add — is  in  capturing,  focusing  and  disseminating  ideas,  resources,  and  tools  that  allow  both 
faculty  members  and  local  support  personnel  to  make  innovative  use  of  new  technologies  in  teaching  and  learning 
with  a minimum  of  duplicative  effort. 

Scaling  Innovation  through  Partnerships  and  Collaboration 

Through  Catalyst’,  UWired  can  easily  identify  promising  practices  from  within  the  campus  and  then  use  them  to 
address  enterprise-wide  educational  needs.  As  an  easily  navigable  clearinghouse  for  information  on  the  use  of  the 
Web  in  teaching  and  learning,  the  site  has  met  faculty  needs  and  drawn  repeat  visits,  creating  a center  of  gravity  for 
innovation  in  teaching  with  technology  that  far  exceeds  the  physical  reach  of  UWired  and  the  CTLT.  Catalyst  has 
four  basic  types  of  content;  (1)  profiles  of  educators  and  programs  that  provide  a vehicle  to  share  ideas  and 
experiences  with  technology,  humanizing  its  use  and,  hopefully  diffusing  teaching  innovation;  (2)  guides  to 
instructional  methods  and  technology  tasks  that  provide  a map  to  a wide  range  of  individual  documents  geared 
towards  improving  learning;  (3)  dynamic  content  and  frequently  updated  information  on  news  and  events  relevant  to 
teaching  with  technology;  and  (4)  instructional  tools  that  provide  a standard  mechanism  for  faculty  to  create 
interactive,  Web-based  instructional  modules  in  any  computing  environment  using  only  a Web  browser. 

As  envisioned,  departments  and  college  support  personnel  who  have  developed  materials  specific  to  their  local 
computing  environments — for  instance  how-to  documents  on  a particular  Web  editor  or  "one-off  Web-based 
instructional  tools — have  been  eager  to  see  their  work  repackaged  and  made  available  to  a much  larger  audience. 
More,  through  a partnership  with  a university-wide  initiative,  the  Program  for  Educational  Transformation  through 
Technology  (PETTT),  UWired  now  has  the  capacity  to  conduct  research  on  the  science  of  learning  and  disseminate 
information  via  Catalyst,  thus  promoting  the  thoughtful  exploration,  development,  assessment,  and  dissemination  of 
next-generation  technologies  and  strategies  for  teaching  and  learning. 

Catalyst  not  only  extends  UWi red’s  reach,  but  it  also  improves  the  overall  quality  of  faculty  support,  providing 
department  support  personnel  with  a set  of  resources  they  can  draw  upon,  freeing  them  individually  tailor  services  to 
their  faculty.  By  aggressively  profiling  the  efforts  of  campus  educators.  Catalyst  makes  faculty  innovation  visible 
and  contributes  directly  to  the  vitality  of  the  teaching  community  on  campus.  In  its  first  year,  the  Catalyst  Web  site 
logged  half  a million  page  views;  it  now  averages  100,000  page  views  per  month.^  On  the  UW  campus,  over  1000 
faculty  and  instructors  from  departments  ranging  from  Anesthesiology  to  Urban  Horticulture  have  implemented 
Catalyst  tools. 


‘ See  http://depts.washington.edu/catalyst/home.html 
^ See  http://depts.washington.edu/catalyst/2000_report.pdf 
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Abstract:  Social  pressures  are  creating  the  necessity  for  a system  of  lifelong  learning.  Electronic 
distance  education  appears  to  be  the  optimal  system  for  meeting  the  educational  requirements  of 
learners.  While  educational  needs  have  shaped  the  development  of  the  electronic  learning 
environments  there  has  been  little  attention  paid  to  the  physical  and  psychological  health  issues  for 
the  electronic  learner.  Athabasca  University  (AU)  has  used  questionnaire  and  focus  groups  to 
examine  the  implications  for  learners  using  AU's  electronic  environments  for  learning. 


INTRODUCTION 


Distance  education  is  often  identified  as  a panacea  for  adult  learners  as  it  overcomes  geographical  barriers  that  often 
prevent  students  from  attending  universities  to  study.  With  the  advent  of  the  Internet  as  a readily  accessible  medium, 
web-based  courses  have  become  immensely  popular  with  providers  of  distance  education  around  the  world.  It  clear 
by  the  enthusiasm  with  which  computer-based  distance  delivery  has  been  embraced  that  society  is  willing  to  rely  on 
computer  technology  for  learning. 

At  Athabasca  University,  where  the  mandate  is  distance  education  , students  are  distributed  not  only  across  Canada 
where  the  University  is  located,  but  also  across  North  America  and  the  rest  of  the  world.  For  the  past  five  years 
Athabasca  University’s  Centre  for  Computing  Information  Systems  and  Mathematics  (CCIS)  has  offered  20 
undergraduate  Computing  Science  courses  over  the  World  Wide  Web  (WWW).  In  2001  it  will  be  offering  a Master 
of  Computing  and  Information  Systems  over  the  WWW.  The  Center  for  Nursing  and  Health  Studies  (CNHS)  has 
offered  a Master  of  Health  Studies  (MHS)  degree  over  the  WWW  since  September,  1999  and  in  less  than  a year  has 
over  200  graduate  students  enrolled  in  the  program. 

The  needs  and  capabilities  of  the  learners  are  our  first  considerations  in  the  development  of  an  electronic  learning 
environment.  In  our  developments  we  have  focussed  on  involving  the  learners  to  optimize  the  environment  for 
learning  (Holt,  Stauffer  & Jelica,  1999).  However,  until  recently  we  did  not  look  seriously  at  health  related  issues. 
Marcoulides,  Mayes,  and  Wiseman  (1995)  have  suggested  that  the  use  of  computers  in  the  workplace  can  result  in 
anxiety.  Similarly,  technology-based  learning  may  result  in  added  stress  among  learners.  There  are  other 
potentially  negative  implications  for  students  who  rely  on  computers  for  learning.  According  to  Faulkner  (1998) 
these  may  include  psychological  factors  such  as  alienation,  inadequacy,  lack  of  privacy,  loss  of  responsibility  and 
damage  to  self-image.  Furthemore,  Dix  et  all  (1998)  noted  that  physical  problems  include  repetitive  motion  strain, 
postural  fatigue,  eye  fatigue  and  less  frequently,  skin  rash,  reproductive  disorders,  and  cataracts. 


In  order  to  examine  health  issues  related  to  the  use  of  computers  for  learning,  we  decided  to  survey  graduate 
students  in  our  Master  of  Health  Studies  Degree  and  undergraduate  students  in  Computing  Science.  Questionnaire 
data  has  been  collected  on  general  computer-related  habits,  any  health  problems  encountered,  and  their  ergonomic 
awareness  related  to  design  of  workspace,  posture,  usage,  and  exercise.  This  was  a large  exploratory  study  with  a 
questionnaire  of  19  items  distributed  to  1310  CCIS  Students  and  145  CNHS  students).  More  indepth  data  will  be 
gathered  by  focus  groups  which  are  underway —the  results  and  will  be  reported  more  fully  at  EdMedia  2000.  The 
more  interesting  questionnaire  results  analyzed  to  date  are  presented  in  this  session. 

CNHS  students  and  CCIS  students  differed  on  a number  of  dimensions  including  graduate  versus  undergraduate, 
continuous  monthly  enrollment  versus  semester,  discipline,  and  different  WWW-based  learning  platforms.  Since  the 
purpose  of  the  study  is  to  consider  ergonomic  factors  in  the  design  of  electronic  curricula  for  both  CCIS  and  CNHS 
students,  we  have  not  focussed  on  inferential  statistics  and  differences  between  the  groups  but  upon  useful 
descriptive  statistics  within  each  group. 

The  return  rate  was  low  for  CCIS  students  (150  of  1310  questionnaires).  It  was  higher  for  CNHS  students  (41  of  145 
questionnaires).  The  low  return  rate  makes  the  results  more  difficult  to  interpret  as  students  with  health  related 
problems  may  be  more  likely  to  respond  giving  us  a biased  sample.  Nevertheless,  computer  usage  was  reported  to 
be  very  high  among  respondents.  Both  CNHS  students  (80.5%)  and  CCIS  students  (85.3%)  had  high  rates  of 
computer  use  in  general  and  relatively  high  use  for  education  other  than  the  current  courses  (43.9%  and  72.3%). 

The  mean  time  using  computers  was  24.2  hours  per  week  for  CNHS  students  and  32.2  hours  for  CCIS  students. 

There  was  a wide  range  of  usage  but  it  would  appear  these  are  high-use  groups  at  risk  for  computer  related  health 
problems. 

CNHS  students  and  CCIS  students  rated  themselves  respectively  as  somewhat  or  informed  (75.6%  and  63.4%)  about 
ergonomic  issues.  However,  a majority  of  all  students  did  not  make  use  of  special  ergonomically  sound  furniture 
and  at  least  half  of  both  CNHS  students  (53.7  %)  and  CCIS  students  (50.0%)  did  not  take  any  regular  breaks  while 
working  at  the  computer.  As  high  usage  groups  these  students  ought  to  be  more  aware  of  ergonomic  issues  and  in 
ideal  circumstances,  practice  more  ergonomically  sound  computer  usage. 

Both  CNHS  and  CCIS  students  reported  some  physical  problems  related  to  computer  use  (26.6%  and  26.8% 
respectively).  Problems  reported  include  carpal  tunnel  syndrome,  eye  strain,  and  back  strain.  Ironically,  although 
both  groups  reported  low  rates  of  psychological  problems  (9.8  % and  6%),  they  did  report  considerable  levels  of 
anxiety  about  their  use  of  computers.  Interestingly,  the  most  intense  anxiety  was  associated  with  issues  of  privacy 
and  security  with  70.7%  of  the  CNHS  students  and  65.1%  of  the  CCIS  students  reporting  slight  to  high  levels  of 
anxiety  in  this  area.  Other  areas  of  anxiety  included  the  cost  of  computer  technology  (46.3%  and  57.7%);  keeping 
current  with  software  (46.3%  and  40.3%);  loss  of  data  (43.9%  and  60.8%);  keeping  current  with  technology  (34.1% 
and  40.3%;  and  keeping  up  with  the  information  glut  on  the  Internet  (36.6  and  47.7).  High  computer  usage  does 
appear  to  affect  these  students’  social/family  life— 36.6%  of  the  CNHS  students  and  34.7%  had  complaints  from 
family  members  about  their  computer  usage.  This  finding  is  hard  to  interpret  in  lack  of  comparison  with  students 
learning  by  other  methods  of  education  but  it  certainly  warrants  further  attention. 

Of  the  CNHS  students  32.5%  reported  that  the  electronic  courses  aggravated  old  problems,  50.0%  said  they  created 
new  problems,  and  51.3%  had  suggestions  for  course  design.  Of  the  CCIS  students  12.7%  reported  that  the 
electronic  courses  aggravated  old  problems,  16.7%  said  they  created  new  problems,  and  30.7%  of  CCIS  had 
suggestions  for  course  design.  When  students  listed  concerns  most  of  them  were  related  to  computer  use  or  course 
design  rather  than  to  health  related  problems  explicitly  (other  than  perhaps  related  to  stress).  This  could  be  a 
function  of  students  being  so  engrossed  in  their  use  of  computers  for  learning  that  they  are  relatively  unaware  of 
health  problems.  We  are  conducting  further  focus  groups  to  further  investigate  the  factors  and  underlying  situations 
that  contribute  to  students  being  in  undesirable  physical  and  /or  psychological  danger  and  other  ameliorative 
approaches  they  might  use.  On  the  basis  of  findings  to  date  we  are  implementing  a Java/XML  based  printing  utility 
and  highlighting  ergonomic  tips  in  the  course  materials. 


1647 


DESIGNING  SYNCHRONOUS  INTERACTIVE  LEARNING 
TELECONFERENCES:  Digital  Field  Experiences  at  the  Columbus  Zoo,  an 
Informal  Science  Contextual  Setting 


Tamara  J.  Garcia-Barbosa, 
Andrea  K.  Balas, 

Dr.  Barbara  S.  Thomson, 
The  Ohio  State  University 


^ A f\  A TT-.li  ^ r\  A ^ TT*  1 

rups  naiiy  ly^j  in,  nigii 


Columbus,  OH 
garcia-barbosa.  1 @osu.edu 


Abstract:  The  Columbus  Zoo  has  a Distance  Learning  Interactive  Field  Experience 
Teleconferencing  program  (CZDLIFE)  to  provide  a vehicle  to  enhance  standards-based  math  and 
science  instruction  in  Ohio’s  schools.  The  use  of  interactive  technology  in  schools  can  increase  the 
ability  to  use  contextual  settings  in  education  which  add  a level  of  reality  to  classroom  learning. 
Making  connections  through  planned  partnerships  outside  the  traditional  classroom  promotes 
curiosity,  creativity,  and  exploration.  Learning  in  an  informal  setting  helps  students  connect  and 
transfer  their  knowledge  base  as  they  engage  in  new  learning  opportunities.  Zoos  can  provide  an 
excellent  informal  context  to  ground  distance  learning  interactive  field  experiences. 


The  Project 

The  goal  of  the  Columbus  Zoo  Distance  Learning  Interactive  Field  Experience  (CZDLIFE)  is  to  offer  educational 
and  scientific  resources  to  enhance  standards-based  math  and  science  instruction  using  two-way  audio/video 
interactive  technologies.  The  CZDLIFE  provides  an  exciting  and  meaningful  curriculum  that  incorporates  a variety 
of  activities  for  all  students  at  the  distant  site.  The  instructional  design  of  the  project  was  grounded  in  contextual 
teaching  and  learning  theory  and  situated  learning  environments  (Brown  et  al.  1989,  Balas,  2000).  The  design 
emphasizes  the  use  of  several  pedagogical  strategies  blended  with  contextual  experiences  that  promote  real-time 
problem-solving  interaction.  These  experiential  activities  are  designed  to  promote  knowledge  creation  through 
interaction  with  zoo  facilitators  and  other  learners  (Thomson,  B.,  et  al.  1999).  The  field  experience  was  constructed 
using  six  key  areas:  1)  considerations  to  connections  to  the  curriculum,  2)  connection  to  multiple  content  areas,  3) 
inclusion  of  non-formal  educators  in  the  design  phase,  4)  developmentally  appropriate  activities,  5)  professional 
development  activities,  and  6)  follow-up  activities  (Francis,  1 999). 

Research  Problem  and  Question 

The  Columbus  Zoo  has  experienced  a great  deal  of  success  implementing  on-site  and  outreach  educational 
programs.  The  recent  advent  of  the  use  of  two-way  audio/video  interactive  teleconferencing  as  a means  of  sharing 
the  zoo  resources  has  added  a new  and  complex  medium  for  educational  content  delivery.  Interactive 
teleconferencing  is  a unique  medium  that  requires  different  and  diverse  pedagogical  approaches  than  those  regularly 
used  with  on-site  visitors  and  outreach  populations.  The  main  benefit  of  two-way  interactive  teleconferences  is  that 
the  nature  of  the  system  provides  a level  of  intimacy  in  communication  that  is  not  apparent  in  other  forms  of 
distance  learning.  Two-way  real  time  video  transmission  of  information  implies  a new  definition  of  real-world 
context,  in  which  the  learning  environment,  although  video-mediated,  is  indeed  the  actual  environment  with  which 
the  learner  interacts  (Jonassen  et.  al.  1995).  Another  strength  of  the  technology  lies  in  its  synchronous  nature;  the 
teacher  and  learners  experience  parallel  delivery  and  reception  of  information  without  a time  delay. 

In  order  to  add  to  research  based  design,  the  main  question  of  the  study  is  How  do  you  design  effective  standard- 
based  two-way  synchronous  teleconferences  from  an  informal  education  setting? 


Research  Design  and  Procedures 

Background  research  in  the  development  of  interactive  teleconferences  in  contextual  settings  such  as  the  zoo  is 
almost  non-existent.  The  study  involves  the  Columbus  Zoo,  in  Columbus  Ohio,  and  four  distance  learning  sites 
throughout  the  state.  The  four  sites  were  divided  among  three  schools  in  New  Lexington,  Stark  County,  and 
Richfield,  Ohio;  and  one  community-based  after-school  enrichment  program,  which  used  the  Cincinnati  Zoo  as  their 
teleconferencing  facility.  Five  elementary  school  teachers,  one  principal  , four  technology  coordinators  and  seven 
classes  were  used  for  data  collection.  The  student  population  ranged  from  second  to  fifth  grade  with  a combined 
total  of  140  students.  This  population  is  representative  of  Ohio’s  urban,  suburban  and  rural  students. 

Qualitative  data  was  gathered  through  surveys,  interviews,  observations,  field  notes,  and  researcher  reflections. 
Surveys  were  collected  immediately  after  the  CZDLIFE,  and  interviews  were  conducted  with  the  teachers  and 
technology  coordinators  prior  to  and  after  the  CZDLFE  teleconference.  The  qualitative  data  was  interpreted  through 
the  development  of  common  themes  generated  from  the  interviews  and  is  being  synthesized  into  assertions  about  the 
nature  of  the  experience. 

Preliminary  Results 

The  CZDLIFE  teleconferences  use  a view  of  learning  that  encourages  knowledge  creation  rather  than  a knowledge 
delivery.  For  most  learners  having  tactual  contextual  opportunities  is  critical  to  accommodate  for  their  learning 
styles  (Thomson,  B.,  et  al.  1999).  The  CZDLIFE  program  has  been  effective  in  providing  an  opportunity  to: 

1.  Bring  to  the  zoo  a wider  student  audience. 

2.  Help  meet  the  needs  of  students  who  are  unable  to  attend  on-site  programs. 

3.  Provide  real  time  face-to-face  interaction  between  classrooms  and  zoo  educators. 

4.  Involve  outside  speakers  who  would  otherwise  be  unavailable  to  regular  zoo  visitors. 

5.  Provide  an  opportunity  to  “export”  unique  exhibits  such  as  the  coral  reef  or  some  of  the 
endangered  animals  that  can  not  be  used  on  regular  zoo  outreach  programs. 

6.  Allow  “real  time”  2 way  audio  and  video  contact  between  students  and  the  facilitators  at 
the  zoo. 

7.  Incorporate  the  use  of  experiential  learning  activities,  live  action  and  pre-recorded  video 
to  present  interactions  carried  out  at  the  zoo  which  are  not  available  to  the  general  public. 

Implications 

The  results  and  conclusions  of  this  program  help  provide  a better  understanding  of  design  process  for  interactive 
distance  learning  field  experiences  in  informal  science  settings.  The  study  also  serves  as  one  of  the  few  studies  that 
examine  the  outcomes  of  using  two-way  interactive  distance  learning  between  elementary  classrooms  and  an 
contextual  learning  setting. 
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Abstract:  This  presentation  reports  on  our  current  and  on-going  R&D  efforts  towards  the 
design  and  development  of  multimedia,  web-based  courses  suitable  for  use  through  the 
Internet.  We  discuss  the  process  that  has  been  adopted  in  designing  the  interface  and  the 
tools  used.  An  important  issue  for  the  implementation  of  the  courses  is  the  use  of  streaming 
technology  for  the  reliable  transmission  of  multimedia  content  material  over  the  Internet. 
This  technology  allows  us  to  transmit  multimedia  data  in  a more  efficient  way  and  to 
introduce  interactive  elements  using  SMIL  (Synchronized  Multimedia  Integration  Language). 


Introduction 

Until  now,  the  traditional  educational  system  has  assumed  that  all  students  learn  identical  materials 
via  the  same  delivery  mode.  However,  it  is  known  that  each  student’s  intelligence  profile  and  learning  style  is 
unique  and  therefore,  the  traditional  lecture-based  method  of  classroom  instruction  cannot  accommodate  the 
large  number  of  diverse  learning  preferences.  In  this  context,  the  Internet  has  emerged  as  a new  means  for 
meeting  the  needs  of  individual  learning  diversity  and  has  introduced  significant  opportunities  for  the 
improvement  of  the  educational  process. 

However,  although  significant  investments  have  been  made  for  the  technological  support  of  this  new 
educational  environment,  similar  steps  have  not  been  taken  towards  the  creation  of  noteworthy  educational 
software.  Thus,  although  a vast  amount  of  electronic  educational  material  currently  exists  on  the  Internet,  it  is 
usually  in  the  form  of  simple  text  occasionally  combined  with  some  graphics.  Undoubtedly,  there  is  a lack  of 
genuinely  qualitative  educational  content.  This  is  due  mainly  to  the  fact  that  the  current  Internet  infrastructure 
is  incapable  of  transmitting  high-quality  multimedia  content  (a  problem  which  can  partially  be  addressed  by 
the  use  of  streaming  technology).  In  addition,  the  significant  expense  and  time  required  for  the  development  of 
such  material  are  serious  prohibiting  factors. 

This  paper  discusses  our  efforts  towards  the  design  and  development  of  interactive  multimedia,  web-based 
courses  suitable  for  use  through  the  Internet.  The  primary  characteristics  of  the  material  developed  are  the 
following: 

1 . It  is  dynamic  and  interactive. 

2.  It  supports  the  concept  of  collaborative  learning  i.e.  it  offers  opportunities  to  simulate  real  situations  of 
collaboration  among  various  groups  of  users. 

3.  It  can  be  transmitted  efficiently  to  users. 


Designing  the  interface 

The  appropriate  design  of  the  interface  depends  on  the  specific  context  that  it  is  intended  for.  To  this 
purpose,  an  effort  was  made  to  create  an  interface,  which  would  be  used  by  adult  graduate  and  post-graduate 
students  and  would  use  unambiguous  “metaphors”  for  the  effective  and  consistent  delivery  of  specific  concepts 
to  learners.  The  screen  was  divided  into  three  vertical  frames,  the  left  of  which  always  displays  the  contents  of 
the  course.  The  middle  frame  presents  each  individual  course  in  the  form  of  text  while  in  the  third  frame, 
accompanying  graphics,  video  or  animation,  can  be  viewed  by  the  user,  depending  on  the  requirements  of  the 
specific  material  being  worked  on.  It  is  important  to  emphasize  that  that  each  lesson  is  available  in  several 
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different  forms  of  delivery  i.e.  text,  narration,  etc. 


Implementation  of  the  educational  material 

On  a technical  level,  the  implementation  of  the  courses  was  based  on  the  use  of: 

a)  Streaming  technology 

b)  HTML,  JavaScript,  VBScript,  XML  (SMIL),  ASP 

Streaming  technology  for  the  efficient  transmission  of  multimedia  elements 

One  of  the  basic  problems  encountered  during  the  implementation  of  a multimedia,  web-based  course  is 
undoubtedly  the  limited  bandwidth  of  the  Internet.  This  is  perhaps  one  of  the  main  factors  that  justify  the  lack 
of  extremely  high-quality  multimedia  material.  The  greatest  amount  of  educational  software  existing  presently 
on  the  Internet  is  in  the  form  of  simple  text,  frequently  enriched  with  some  graphics.  This  form,  however, 
proves  to  be  inadequate. 

Streaming  technology  addresses  this  problem  in  the  following  way: 

Although,  the  conventional  way  of  transmitting  data  through  the  Internet  requires  that  the  entire  file  (e.g.  a 
video  file)  be  received  before  it  may  be  viewed  by  the  user,  this  is  not  a necessity  when  streaming  technology  is 
the  adopted  method.  In  the  latter  case,  the  information  is  presented  to  the  user  (e.g.  the  video  file  begins  to 
“play”)  even  though  the  remaining  information  is  still  being  received  and  it  becomes  possible  to  view  the 
multimedia  elements  almost  simultaneously  with  the  appearance  of  the  rest  of  the  HTML  document. 

A primary  advantage  of  this  kind  of  technology  is  that  it  is  possible  to  transmit  very  large  multimedia  files  in 
an  efficient  and  reliable  way  thus  resulting  in  the  preservation  of  the  learner’s  concentration. 

Streaming  technology  makes  use  of  the  RTSP  (Real  Time  Streaming  Protocol)  instead  of  the  HTTP  protocol. 
The  product  selected  to  deal  with  the  difficulties  associated  with  the  network  infrastructure  of  the  Internet  was 
the  RealServer  created  by  Real  Networks.  RealServer  makes  it  possible  to  incorporate  high-quality  multimedia 
elements  (RTSP  is  designed  to  stream  clips  that  have  time  lines)  within  a web-based  course.  Whether  the 
learner  has  an  ISDN  line  or  accesses  the  Internet  through  a low-bandwidth  connection,  s(he)  may  still  view  a 
course  that  comprises  a variety  of  multimedia  elements.  There  is  also  a possibility  for  the  dynamic  detection  of 
different  connection  speeds  between  the  client  and  the  server  so  as  to  achieve  improved  transmission. 

Emphasis  should  be  given  to  the  fact  that  it  is  possible  to  include  additional  interactive  elements  within  the 
educational  material  (sound,  video,  animation)  through  the  use  of  the  SMIL  language. 

Interactivity 

An  additional  item  of  concern  was  that  the  educational  material  should  be  enriched  with  a variety  of 
interactive  elements.  To  this  extent,  SMIL  (Synchronized  Multimedia  Integration  Language)  presents  notable 
interactive  characteristics. 

Based  on  XML,  this  language  reacts  upon  the  multimedia  elements  of  the  course,  which  are  transmitted 
through  the  use  of  streaming  technology. 

The  main  reasons  that  make  SMIL  so  important  are: 

a)  It  can  integrate  and  co-ordinate  many  diverse  types  of  multimedia  information.  We  can  capture  events  that 
are  generated  from  the  interaction  between  the  learner  and  the  multimedia  element.  This  is  achieved  through 
the  use  of  callback  methods. 

c)  It  can  be  considered  as  an  “open”,  platform-independent  technology,  which  is  based  on  W3C  XML. 
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Abstract:  Based  on  case-studies  of  edutainment  Multimedia  programmes  this  study  is  exploring 
the  affordances  of  the  contextual  elements  of  narrative  in  relation  to  interactive  Multimedia 
programmes,  the  possible  cognitive  costs  that  lack  of  canonical  narrative  elements  may  have  for 
the  user  and  the  particular  possibilities  they  may  offer  in  a learning  environment  in  which 
meaning,  experience  and  motivation  for  further  exploration  are  key  elements. 


Narrative  and  the  user's  construction  of  meaning 

Educational  or  edutainment  Multimedia  programs  are  offering  new  environments  for  learning  and  communication 
(Gjedde  1999a)  but  often  these  programs  are  not  employing  the  canonical  narrative  elements  that  can  be  found  in 
popularization  of  complex  information  in  television,  magazines  or  in  books.  Instead  there  has  been  a natural  interest 
in  exploring  the  non-linear  possibilities  that  the  interactive  media  is  offering.  This  has  been  done  while  letting  go  of 
the  canonical  narrative  elements  and  schemata's  that  typically  has  a beginning  including  a notion  of  time,  place  and 
a protagonist,  a development  and  an  end  (Bartlett  1932  1995;  Labov  1972).  Research  on  the  relationship  between 
narrative  structure  and  cognitive  processes  like  memory  and  comprehension  has  indicated  that  there  tends  to  be 
better  memory  and  comprehension  for  texts  that  employ  a narrative  structure  than  texts  that  do  not  use  a narrative 
structure.(Thomdyke  1977;  Mandler  1984).  Narrative  theories  are  furthermore  pointing  to  its  importance  for 
experience,  identity  and  communication.  (Schank  1990;  Schank  and  Abelson  1995;  Schank  1997). 

The  structuring  and  communicative  elements  of  narrative  are  often  not  used  in  educational  or  edutainment 
Multimedia  programs,  which  are  often  encyclopaedic  in  structure  and  if  using  other  structures  as  a motivating  factor, 
tend  to  use  games  that  often  do  not  have  the  basic  narrative  elements.  They  are  not  embedding  the  information 
offered  in  the  context  and  structure  of  a narrative,  but  it  is  instead  presented  in  an  abstract  cyberspace,  or  in  an 
encyclopedic  structure  that  does  not  provide  the  same  cognitive  and  experiential  level  of  support  for  the  user. 

This  lack  of  narrativity  can  have  consequences  for  the  ability  of  the  user's  to  construct  meaning  from  the  seeming 
chaos  gathered  by  experiencing  information  that  is  presented  out  of  a context  or  a known  and  comprehensible 
universe. 

This  can  be  found  in  the  edutainment/educational  genre,  where  the  user  may  be  particularly  vulnerable  to  the  lack  of 
cognitive  support  for  the  organization  of  the  material,  that  a narrative  structure  may  provide. (Plowman  1996; 
Plowman  1998) 

The  user  may  also  lack  the  involvement,  which  the  narrative  can  create  through  an  experiential  dimension,  through 
the  unfoldment  of  a narrative,  with  the  possibilities  for  emotive  and  identificatory  involvement  and  a narrative  logic 
with  an  explicit  causal  structure.  These  factors  may  be  of  importance  for  the  learner's  motivation  and  construction  of 
meaning,  particularly  the  possibility  for  making  a connection  between  the  information  they  may  gather  through  the 
program  and  their  present  experiences  and  knowledge  structures.  (Sarbin  1986;  Schank  1990;  Gjedde  1998b) 
Previous  research  in  this  area,  has  looked  at  various  CD-ROM's  in  the  edutainment  and  educational  genre 
(Laurillard  and  Taylor  1994;  Plowman  1998).  There  are  indications  that  a non-narrative  or  encyclopeadic  structure 
tends  not  to  support  the  user's  construction  of  knowledge  or  perception  of  global  coherence  to  the  extent  that  the 
narrative  that  may  be  found  in  books  and  television  typically  does. 

An  empirical  research  project  with  an  experimental  design,  the  MENO-project  (Multimedia  in  Education  and 
Narrative  Organisation)  has  experimentally  employed  different  types  of  structure  in  an  interactive  educational 
program.  (Laurillard  and  al.  1995)  (Plowman,  1999  ).  It  was  however  focused  on  the  structural  manipulations  and 
not  on  a narrative  embedding  of  the  material  through  a consistent  main  character  that  provided  a focus  for 
identification,  or  the  point  of  view  of  a narrator. 


This  type  of  narrative  embedding  may  be  found  in  several  successful  popular  science  productions.  An  example  of 
this  is  the  best-selling  novel  by  Jostein  Gaarder  about  the  History  of  philosophy.*  Sophies  World  (Gaarder  1992), 
which  became  the  world's  most  sold  book  in  1995.  A CD-ROM  production  was  made  on  the  basis  of  this  book 
within  the  genre  of  edutainment,  which  was  based  on  some  radically  different  structures.  In  the  CD-ROM  version 
some  of  the  canonical  narrative  elements  have  been  replaced  by  a NLP-generator,  which  is  capable  of  having  a 
primitive  dialogue  with  the  user;  and  with  a number  of  game  elements. 

Based  on  a pilot  project  with  a case-study  of  some  users  of  this  CD-ROM  I have  identified  some  potential  problems 
that  will  be  further  explored  in  a study  that  will  involve  other  CD-ROM's  of  the  same  genre  that  are  using  narrative. 
The  results  from  the  pilot  study  are  indicating  that  it  might  have  implications  for  the  user's  experience  when  the 
basic  narrative  element  are  replaced  by  other  elements,  which  do  not  support  the  process  of  narrative  construction, 
by  the  user.  The  narrative  is  seen  as  fundamental  for  the  user's  experience  of  meaning  and  global  coherence.(Bruner 
1990;  Schank  1990;  Schank  and  Abelson  J.1995)  the  narrative  is  furthermore  of  importance  for  the  dimension  of 
experience.(Gerrig  1993). 

Without  the  narrative  which  affords  an  embedding  context  the  user  tends  to  experience  the  options  offered  in  the 
interactivity  as  lacking  in  coherence  and  meaning.  This  must  be  seen  in  relation  to  the  genre  of  edutainment  and  that 
the  target  group's  users,  are  not  supposed  to  have  much  previous  knowledge  about  the  particular  field  that  the  CD- 
ROM  is  presenting. 

The  study  is  being  done  in  a naturalistic  mode,  with  a high  degree  of  ecologic  validity,  using  natural  users  of  the 
CD-ROM's  with  a mobile  usability  lab.  This  set  up  allows  for  a video-recording  to  be  done  with  a video  of  the  user 
as  well  as  a documentation  of  the  activity  on  the  screen  through  a scan-converter.  The  study  uses  a qualitative 
methodology  that  involves  user's  narratives  about  their  experiences  with  the  program  and  their  construction  of 
meaning  from  it.  Using  both  video-observation  and  qualitative  interviews  as  well  as  a questionaire  allows  for  a 
comparison  between  the  different  sets  of  data.. 

The  focus  of  the  project  is  to  explore  the  user's  experience  through  narrative  and  how  it  translates  into  a potential 
greater  sense  of  motivation  and  possibly  cognitive  capability  for  construction  of  meaning,  in  relation  to  the  non- 
linearity of  the  media. 


The  aims  of  the  project: 

• To  provide  a greater  understanding  for  the  qualitative  aspects  of  narrative  in  relation  to  interactive  Multimedia 
and  the  particular  possibilities  they  offer  in  a learning  environment  in  which  meaning,  experience  and 
motivation  are  key  elements. 

• To  explore  the  relationship  between  the  contextual  elements  of  a narrative  . In  particular  these  which  are  offered 
by  the  elements  that  constitute  a beginning  in  a canonical  narrative  (Labov  1967.1997;  Mandler  1984)  - and  the 
cognitive  implications  for  the  user  in  terms  of  experience,  understanding  and  interest  for  the  subject  and 
motivation  for  further  explorations  into  the  subject. 

• To  look  at  possible  connections  between  the  narrative  elements  in  the  interactive  program  and  the  user's  process 
of  narrative  construction  and  focus  on  the  relationship  between  the  narrative  organization  of  the  material  and 
user's  level  of  processing.(Laurillard  et  al.  1984,  1997) 

• To  explore  gender  specific  preferences  in  the  relation  to  narrativity  (Gjedde  1998b)  and  multimedia.(Adam 
1998). 

• To  inform  the  producers  of  educational  multimedia  about  the  implications  for  using  narrative  structure  and 
other  narrative  elements  in  relation  to  specific  groups. 
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Abstract:  The  aim  of  the  current  project  is  to  design,  develop  and  implement  a teaching  strategies 
model  and  a multimedia  template  which  crosses  over  curricular  boundaries  and  responds  to  a wide 
variety  of  learning  styles  of  students.  The  Conversational  Framework  model  as  proposed  by 
Laurillard  formed  a starting  point  in  examining  the  place  of  a multimedia  template  in  Academic 
Learning  while  the  design  of  the  template  has  been  guided  by  the  need  to  establish  reusable 
resource  materials.  Key  characteristics  of  such  a template  were  deemed  to  be  accessibility, 
cohesiveness,  supportiveness,  flexibility  and  quality. 


Background 

According  to  Laurillard,  the  teaching  - learning  process  is  an  interaction  between  teacher  and  student  operating  at 
two  intermingling  levels: 


Level  1 - Discursive  Level  - the  level  of  theory 

4 stages  to  reach  consensus: 

Level  2 - Interactive  level  - the  level  of  practice 
The  way  the  student  acts  in  the  world 

1.  Teacher  articulates  the  subject  matter 

1.  Teacher  sets  task 

2.  Student  joins  dialogue  - asks  questions,  practices  etc 

2.  Student  acts 

3.  Teacher  re-expresses  their  point  to  clarify  and 
elaborate.  * Teacher  uses  student  understanding  to 
adapt  interactive  activities  and  elaborate  theory  to  those 
appropriate  to  student  needs 

3.  World  responds  - includes  teacher,  other  students, 
experts  etc.  * Teacher  uses  student  understanding  to 
adapt  interactive  activities  and  elaborate  theory  to  those 
appropriate  to  student  needs 

4.  Student  has  another  attempt  at  representing  the  theory 

4.  Student  modifies  action  to  improve  their  performance 
( adaptation  and  reflection). 

The  teaching  model  is  designed  to  provide  a framework  for  the  teacher  and  student  for  all  steps  in  both  levels  whilst 
the  multimedia  matrix  plays  a role  in  only  part  of  the  total  process.  A balanced  mix  of  multimedia  resources,  face 
to  face  interaction,  online  communication  and  practical  activity  could  form  the  components  of  a complete  learning 
experience. 

Teaching  Plans 

It  is  the  intent  of  the  designers  that  the  multimedia  template  act  as  a guiding  mechanism  for  the  teacher  in  order  to 
facilitate  the  mapping  and  selection  process  for  the  teaching  sequence.  Through  thorough  analysis  of  the  learning 
objectives,  the  resource  selection,  interaction  planning  and  assessment  strategies  are  developed  and  a mapping  of 


656 


strategies  is  undertaken.  Once  all  aspects  of  the  teaching  mix  are  determined,  then  the  multimedia  elements  are 
produced  and  assembled  and  supporting  technologies  are  weaved  into  the  virtual  learning  environment. 

A typical  analysis  by  a teacher  in  planning  the  learning  experiences  may  be  as  follows: 

* Concept  Map  is  produced  showing  the  interconnecting  curriculum  and  content  elements 

* Identification  of  feasible  interactions  that  best  achieve  learning  outcomes 

* Development  of  plans  for  each  interaction  eg  multimedia  matrix,  face  to  face  discussion,  practical  excursion  etc 

* Topic  in  Context  material  is  assembled  and  Themes  and  associated  Activity  sequence  is  determined  and  designed 

* Media  Resources  are  gathered  for  supporting  problem  solving  as  required  by  activities 

* Computer  tools  are  assembled  for  use  by  students  in  carrying  out  activities 

* Delivery  proceeds  with  monitoring  and  feedback  loops  built  in  to  the  interactions  - Evaluation  data  is  gathered  and 
used  in  review  cf  the  model  and  matrix. 

In  teaching  academic  staff  to  utilise  the  matrix  tool,  the  above  sequence  forms  the  basis  for  a workshop.  This 
enables  the  intertwining  of  a pedagogical  and  a technological  focus  and  is  punctuated  by  a management  theme  ie  all 
three  focuses  become  a part  of  the  teaching  plan. 

The  Multimedia  Matrix 

The  matrix  template  consists  of  a number  of  "cells"  which  may  be  populated  with  content  or  resources  which  are 
required  for  the  student  to  carry  out  their  study  of  a particular  topic.  These  are  as  follows: 

A)  Topic  in  Context  - an  overview  of  the  topic  and  its  place  in  the  whole  curriculum.  The  relevance  of  the  topic  to 
the  profession  is  also  a key  aspect. 

B)  Topic  resources  - these  are  not  directly  linked  to  any  one  theme  or  activity  but  rather  are  more  global  to  the  topic 
eg  Extended  readings  or  digressions  may  be  placed  in  the  resource  area 

C)  Themes  - In  order  to  create  a single  environment  for  students  that  is  self  contained  but  has  interconnecting  parts, 
the  template  allows  for  the  creation  of  several  themes.  A concept  map  highlights  the  links  between  themes. 

D)  Activities  - these  are  the  problems  and  instructions  that  guide  students  through  the  resources  and  learning 
experiences  designed  for  them. 

E)  Tools  - These  include  Glossaries,  Concept  Map  generators.  Text  editors.  Analysis  tools  and  a launch  pad  for 
applications  that  are  common  place  such  as  word  processors  and  spreadsheets. 

F)  Media  - Video,  Audio,  Graphics  and  Text  elements  can  be  used  in  the  template  - these  are  dropped  into  any 
matrix  and  become  a part  of  the  application. 

Progress  to  date 

The  use  of  the  matrix  in  teaching  the  concepts  of  movement  through  the  specific  idea  of  "gait"  has  been  planned.  A 
plan  for  the  main  phases  of  teaching  the  sequence  has  been  established  and  each  matrix  in  a sequence  multimedia 
support  applications  is  being  developed.  These  include  Biomechanical,  Bioenergetics,  Diversity  and  Neurologic 
perspectives  on  gait.  Rather  than  examining  gait  in  each  of  these  aspects  separately,  the  planned  sequence  moves 
from  an  exploratory  view  through  each  of  these  perpectives  in  order  to  establish  core  understanding  and  then 
evolution  of  ideas  both  quantitative  and  qualitative  from  a combined  view.  To  illustrate,  a typical  activity  may 
require  a student  to  make  a series  of  observations  that  will  lead  to  understanding  of  both  diversity  and  biomechanical 
principles  and  their  interrelation.  Use  of  a common  visual  resource  in  supporting  such  an  activity  ie  the  same  video 
sequence,  also  reinforces  this  cross  curricular  notion.  The  template  itself  is  in  BETA  phase  and  has  been  used  to 
assemble  the  first  of  these  applications. 

The  staff  development  program  that  is  essential  in  establishing  the  model  for  achieving  maximum  effectiveness  of 
implementation  of  the  Conversational  Framework  model  has  begun  and  a network  of  colleagues  from  one  faculty 
group  who  are  in  close  proximity,  has  been  set  up.  This  is  a key  feature  for  staff  development  in  an  area  that 
requires  complete  rethinking  and  restructuring  of  teaching  and  its  planning.  Collegiality  and  support  cannot  be 
underestimated  in  their  power  to  catalyse  best  practice  teaching.  We  envisage  that  the  notion  of  a learning  matrix 
both  in  terms  of  the  multimedia  application  itself  and  of  the  curriculum  structure  will  provide  a clearer  view  for  both 
students  and  teachers.  In  this  way  the  path  through  the  study  of  a discipline  may  be  made  more  transparent  and 
flexible.  In  supporting  a student-centred  approach  to  teaching,  the  initial  effort  has  been  significant,  however  the 
revolution  that  will  follow  will  show  energy  levels  that  pale  these  efforts  into  insignificance. 
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Abstract:  The  purpose  of  this  study  is  to  investigate  which  experiences  of  adult  learners 
in  an  online  cooperative  learning  environment  contribute  to  the  expected  evolution  from 
an  aggregate  of  strangers  to  a group  capable  of  high  performance.  There  is  a clear  need 
for  a thorough  qualitative  study  focusing  on  the  development  of  online  relationships 
required  for  successful  cooperative  learning  groups.  . The  interview,  observation,  and 
journaling  methods  will  provide  information  to  address  impression  formation,  social 
presence,  group  development  and  the  conditions  of  ambiguity  and  detraining.  In  addition, 
the  results  of  this  study  will  help  provide  a palette  of  best  practices  for  designers  of  online 
learning  environments  utilizing  similar  instructional  strategies. 


Introduction 

Providing  opportunities  for  students  to  take  distance  education  courses  is  becoming  commonplace 
in  higher  education.  It  should  come  as  no  surprise  that  this  growth  fosters  change  and  with  change  comes 
resistance  and  criticism.  Yet,  perhaps  it  is  time  to  listen  to  that  criticism.  Historian  David  Noble,  professor 
at  York  University,  wrote  a series  of  articles  criticizing  what  he  characterized  as  the  unmindful  haste  by 
higher  education  institutions  to  adopt  the  Internet  and  World  Wide  Web  for  course  delivery  creating 
nothing  more  than  a technologically-slick  updated  version  of  the  correspondence  course  (summarized  in 
Noble,  1998).  Nobles’  primary  fear  is  that  we  are  creating  courses  devoid  of  any  true  interaction  between 
the  student  and  instmctor. 

Course  developers  and  instmctors  recognize  the  importance  of  designing  interactions  into  courses 
offered  by  distance  education  means.  The  importance  and  validity  of  these  interactions  is  amplified  in 
numerous  studies  in  which  students  reported  that  their  individual  success  was  due,  in  large  part,  to  a high- 
level  of  interaction,  typically  learner-instructor  (Eastmond,  1995;  Wegerif,  1998).  Likewise,  many  learning 
theorists  do  describe  the  integral  role  that  interaction  plays  in  the  level  of  learning  attained  by  students  (see 
Reeves  & Reeves,  1996,  for  review).  Most  studies  on  communication  using  computer-mediated 
communication  (CMC)  or  asynchronous  learning  networks  (ALN)  have  examined  interactions  resulting 
from  tasks  requiring  some  type  of  online  discussion  (Eastmond,  1995;  Hiltz,  1994;  Wegerif,  1998)  Yet, 
very  few  studies  have  looked  at  the  social  aspects  of  group  processes  in  online  environments  using 
cooperative  learning  techniques. 

Reports  from  distance  learning  students  in  collaborative  or  cooperative  learning  groups  using 
computer-mediated  communication  (CMC),  specifically,  asynchronous  learning  networks  (ALN),  indicate 
that  there  is  a crucial  moment  in  their  experience  in  which  they  cross  a threshold  from  outsider  to  insider  or 
from  newcomer  to  old-timer  (Eastmond  1994;  Wegerif,  1998).  Crossing  this  threshold  is  crucial  for 
increased  positive  attitude  leading  to  continuation  of  the  course  and  higher  achievement.  Failure  to  cross 
this  threshold  often  results  in  the  student  dropping  out  of  the  course.  How  does  a collection  of  strangers 
move  from  being  an  aggregate  of  people  to  a group  capable  of  high  performance  using  computer-mediated 
communication  (CMC),  specifically  asynchronous  learning  networks  (ALN),  as  the  sole  means  of 
communication? 

It  stands  to  reason  that  understanding  interpersonal  communication  across  asynchronous  learning 
networks  is  crucial  to  designing  and  implementing  effective  cooperative  learning.  Results  from  laboratory 
research  led  to  a set  of  interpersonal  CMC  categorized  as  the  “Cues-Filtered-Out  Approach”  (see  Walther, 
1994,  for  review).  Essentially  those  theories  postulated  that  the  reduction  of  nonverbal  cues  leads  to 
communication  that  is  task  oriented  and  impersonal,  resulting  in  more  equal  communication  among  group 
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members  but  the  possibility  of  very  negative  interactions  (‘flaming’).  Initially,  Walther  (1992)  stated  that 
the  communication  process  observed  within  CMC  follows  that  found  in  face-to-face  groups  only  the  time 
required  for  certain  behaviors  to  emerge  is  of  much  longer  duration.  In  addition,  an  individual’s  perception 
of  anticipated  future  interaction  with  group  members  leads  to  more  social  relational  behavior.  His  most 
recent  work  showed  that  computer-mediated  communication  can  run  the  gamut  from  impersonal  to 
interpersonal,  and  finally  to  what  he  calls  hyperpersonal  (Walther,  1 996). 

There  are  major  discrepancies  for  prescriptions  on  how  to  establish  and  manage  groups  and 
cooperative  groups,  whether  face-to-face  or  online.  For  example,  the  need  for  an  individual  to  identify  with 
the  group  as  a prerequisite  for  intimacy,  affection  and  social  orientation  (Bales,  1950)  appears  to  be 
contrary  to  the  identification  of  heterogeneous  groups  as  the  most  successful  for  cooperative  learning 
(Johnson,  Johnson,  & Holubec,  1994).  Furthermore,  there  are  conflicting  theoretical  approaches  and  even 
haziness  in  single  theories  about  the  ongoing  development  of  thiriving  groups  in  CMC.  Specifically,  there 
are  conflicting  indications  of  how  one  might  go  about  creating,  facilitating,  and  managing  online  groups 
using  cooperative  learning  communicating  via  ALN/CMC.  How  can  these  discrepancies  be  resolved? 


Proposed  Research 

This  study  examines  the  key  aspects  to  consider  when  using  cooperative  group  instructional 
strategies  for  distance  learning  mediated  by  text-based  Internet  communication.  Based  on  components 
derived  from  research  on  small  group  development,  cooperative  learning,  and  relational  behavior, 
instructional  strategies  are  recommended  and  a set  of  research  questions  are  proposed. 

Understanding  the  cooperative  group  processes  when  using  CMC  that  provide  for  the 
development  of  successful  interpersonal  communication  leading  to  close  interpersonal  relationships  will  1) 
provide  information  for  instructional  designers  and  instructors  of  online  courses  to  guide  their  design  and  2) 
delivery  efforts  in  terms  of  group  composition,  group  activities,  and  facilitation  strategies,  and  guide  design 
of  better  interfaces  and  software  to  enhance  effective  communication  interaction. 
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Abstract:  This  paper  describes  a research  project  on  primary  school  English  teaching  and 
teacher  training  in  Inner  Mongolia  P.R. China  where  the  financial  situations  of  the  schools  and 
pupils  vary,  and  English  teachers’  levels  differ  greatly.  The  research  group  has  developed  a 
course,  which  has  a set  of  kernel  textbooks,  and  there  are  eight  different  combinations.  This 
article  describes  why  certain  media  are  chosen  and  how  they  are  used  in  order  to  make  sure 
that  teachers  can  be  trained  properly  and  the  pupils  can  get  the  most  possible  benefit. 


Introduction: 

There  is  no  doubt  the  use  of  technologies  in  learning  is  gathering  pace  around  the  world.  The  use  of  the 
multimedia  as  leaming/teaching  is  changing  the  way  the  world  thinks  and  acts.  Multimedia  has  the  potential  to 
make  teaching  and  learning  more  effective  but  can  we  use  them  everywhere?  This  article  describes  the  research 
project  on  primary  school  English  teaching  and  teacher  training  in  Inner  Mongolia  and  why  certain  media  are 
chosen  and  how  they  are  used.  Future  plan  is  mentioned  and  suggestions  for  improvement  are  longed  for. 


Analysis  and  schemes: 

The  demand  for  learning  English  is  greater  and  greater  in  this  fast  developing  world.  This  research  project  is 
aimed  to  meet  the  need  of  this  demand.  To  plan  for  flexibility  and  adaptation  right  from  the  start  is  very 
important,  if  we  aim  to  get  good  learning  results  for  a long  period  of  time  (Griffiths,  Heppell,  Millwood, 
&Mladenova,  1994).  The  following  aspects  are  what  we  considered  when  we  tried  to  work  out  a proper  plan: 

There  are  not  enough  English  teachers  in  Inner  Mongolia.  Some  schools  may  have  one;  some  other  schools  may 
have  more.  The  teachers'  English  levels  and  teaching  experience  vary;  Inner  Mongolia  is  a region  with  different 
nationalities.  The  main  languages  are  Mongolian  and  mandarin  which  have  sharp  differences.  Mongolian  pupils 
learn  Mongolian  and  mandarin  in  primary  school,  so  they  can  only  learn  English  for  one  or  two  years  in  primary 
schools  to  make  sure  that  they  may  not  be  overloaded;  Some  schools  may  have  tape  recorders,  others  may  have 
overhead  projectors.  Video  player,  etc;  locations:  Inner  Mongolia  is  a vast  area.  Primary  schools  scatter  all  over 
the  area.  To  train  all  the  teachers  face  to  face  is  impossible;  Some  pupils  cannot  afford  expensive  textbooks  and 
tapes.  Some  schools  cannot  afford  expensive  teaching  aids;  Although  Inner  Mongolia  is  a backward  region,  it  is 
also  changing  rapidly.  The  numbers  of  English  teachers  in  each  school  will  change,  the  facilities  in  schools  will 
change,  the  home  appliances  will  change;  How  can  we  use  different  media  for  the  pupils  and  teachers  in  the 
most  appropriate  way?  (Collis  & Deboer,  1998;  Griffiths  et  al.,  1994) 

After  all  the  above  aspects  besides  teaching  methodology  being  considered,  certain  media  are  used  in  the  way  as 
the  following:  Kernel  textbooks  and  supplementary  textbooks  have  been  developed  to  make  sure  that  the  pupils 
who  can  only  study  one  or  two  years  may  have  basic  knowledge  of  English  before  they  go  to  middle  schools 
and  other  pupils  who  have  the  opportunity  to  learn  more  in  primary  school  may  well  informed.  Eight  teaching 
plans  with  scientifically  related  teaching  materials  have  been  designed  to  make  sure  that  all  the  pupils  can 
benefit  most  efficiently.  The  textbooks  are  designed  black  and  white  and  special  revision  exercises  are  designed 
to  make  pupils  paint  the  textbooks  with  colorful  pencils  by  themselves.  Therefore  the  textbooks  not  only  attract 


pupils  attention  more  and  the  pupils  may  remember  more  of  the  teaching  materials  but  at  the  same  time  the  price 
of  the  textbooks  is  cut  down;  Two  sets  of  tapes  are  designed.  One  set  is  that  one  tape  is  offered  each  term  to  help 
the  pupils  to  learn  at  home  and  to  keep  a low  price  for  those  pupils  who  cannot  afford  two  tapes.  The  other  set  is 
that  two  tapes  are  offered  each  term.  This  is  to  make  sure  that  there  are  enough  feedbacks  and  interactive 
exercises  to  do,  and  it  makes  preview  and  review  convenient.  At  the  same  time,  these  tapes  can  also  be 
convenient  for  teachers  to  prepare  their  lessons.  A special  tape  is  designed  only  for  teachers  to  check  pupils 
assignment.  In  this  way  pupils  do  not  have  to  buy  that  tape  and  teachers  will  not  have  trouble  with  the  questions 
and  answers.  Transparent  sheets  for  overhead  projectors:  Two  sets  of  transparent  sheets  are  designed.  One  is 
black  and  white;  the  other  is  with  color  to  meet  the  need  of  different  schools.  In  this  way,  teachers  may  have 
teaching  aids  even  in  schools  with  limited  budgets.  We  made  videotapes  for  pupils  and  teacher  training  so  that 
teachers  from  remote  areas  can  also  have  the  opportunity  to  watch  other  teachers'  teaching  and  understand  the 
theories  of  teaching.  Schools  with  video  players  can  play  the  videotapes  to  their  pupils  and  pupils  with  video 
players  at  home  can  watch  the  videotapes  to  learn  better. 

Learning  should  be  immediate  feedback  as  many  educators  have  already  recognized.  However,  the  above  media 
which  have  been  used  are  quite  limited  in  the  aspect  of  interactive  activities.  This  means  that  a lot  of  potentials 
from  the  teachers  and  pupils  have  not  been  cultivated.  What  should  we  do  to  accomplish  our  goal  then?  In  order 
to  work  out  our  future  plan,  let's  first  see  what  will  happen  in  this  part  of  world  in  the  near  future? 

The  future  plan: 

In  the  near  future  more  and  more  schools  in  Inner  Mongolia  will  also  have  computers.  Since  Internet  connection 
in  China  is  still  quite  expensive,  most  of  the  families  in  Inner  Mongolia  will  not  be  able  to  get  an  access  very 
soon.  But  the  perspective  of  using  Internet  in  schools  is  great.  A significant  number  of  schools  will  have  the 
capability  to  interact  online.  It  means  that  teaching  materials  designed  to  be  used  with  computers  may  help  more 
and  more  teachers  and  pupils  to  teach  and  learn  more  effectively.  The  key  to  successful  interactivity,  is  not 
simply  offering  educational  access  to  preexisting  networked  materials.  Rather,  a more  valuable  role  is  to  offer  a 
collaborative  environment  where  users  can  create,  and  share  information  of  their  own  with  other  users.  Ifs  an 
opportunity  that  many  teachers  won't  let  slip  by  them.  It  means  that  we're  reaching  a point  where  teachers  can't 
be  competitive  in  other  words,  prepare  today's  children  for  the  university  and  the  modem  workplace  without  the 
Web.  Therefore,  our  future  plan  is  divided  into  three  steps: 

1 . To  put  the  teachers  books  on  Internet  first  and  build  a forum  for  teachers  to  share  their  teaching  experience. 
In  this  way,  teachers  will  be  well  informed  and  share  their  experience  in  teaching,  and  with  the  latest 
version  of  teachers’  books,  they  can  always  try  to  teach  their  pupils  better. 

2.  To  put  teaching  materials  with  more  interactive  activities  on  CDs.  In  this  way  schools  with  computers  and 
pupils  with  computers  at  home  can  use  these  materials  to  teach  and  learn  more  efficiently. 

3.  To  establish  on  line  courses  for  pupils  to  learn.  Besides  textbooks  with  interactive  activities,  a reading 
series  can  be  developed.  In  this  way,  pupils  with  internet  access  not  only  can  learn  the  textbooks  well  but 
can  also  learn  by  their  own  pace  to  develop  their  abilities  in  reading,  listening  , speaking  and  writing. 

The  above  are  our  plan  for  the  near  future.  We  are  eager  to  ask  you  the  specialists  in  the  conference  to  offer  us 
suggestions  in  order  to  make  this  ideal  into  reality.  All  your  advice  would  be  greatly  appreciated. 
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Abstract:  Developers  of  educational  material  like  web-based  training  and  online  courses  face  well  known 
aspects  of  classic  software  development  processes:  Reusability  of  course  units;  Supporting  cooperation  and 
resource  sharing  beuveen  the  members  of  the  design  process,  like  content  authors,  designers,  ruultiriicdia 
producers  and  quality  managers  and  Quality  management.  Therefor,  software  engineering  techniques  like 
object-oriented  analysis  (00 A)  based  on  a central  database  to  be  used  in  iterative  design  processes  will  also 
be  useful  for  the  development  of  educational  material.  This  considerations  are  the  background  for  our  work 
and  experiences  in  the  project  Virtual  University  as  described  in  this  paper. 


INTRODUCTION 

The  Project  “Virtual  University  of  Applied  Science” 

This  paper  reports  work  done  in  the  project  “Virtual  University  of  Applied  Sciences”  (period  of  duration  1998-2003,  see 
http://vfh.de).  It  aims  at  establishing  a location  independent  university  with  a curriculum  for  computer  science  of 
multimedia  systems  and  for  business  engineering  (Bachelor,  Master).The  authors  of  this  paper  are  involved  in  the 
production  of  web-based  courses,  in  the  design  of  user-adequate  learning  spaces  and  in  the  support  of  the  design  process. 
Their  focus  is  on  usability  recommendations  and  quality  management  for  the  course  material  during  the  development 
process.  Other  aspects  like  teaching  strategies,  learning  processes  or  technical  issues  concerning  the  course  production  are 
supervised  by  other  dedicated  consulting  groups  within  this  project. 

Adressed  Problems 

During  the  first  months  of  course  production  for  the  virtual  university  several  problems  got  obvious: 

• The  lack  of  a appropriate  context  of  use  description. 

Practice  showed  that  the  developers  of  course  material  had  only  an  incomplete  mental  model  about  the  needs  and 
abilities  of  the  intended  user  groups,  like  previous  knowledge,  learning  style,  preferences  in  teaching  mode,  motivation 
and  other  information  that  would  be  important  to  adapt  the  teaching  strategies  and  delivery  of  hypermedia  structure  and 
content  accordingly  (Kritzenberger  2000).  The  mental  models  about  this  user  group  and  the  assigned  attributes  even 
differed  during  development  phases. 

• No  stable  base  for  the  quality  management 

Quality  management  (e.g.  of  learning  processes,  functional  correctness,  usability)  suffers  from  a lack  of  basic 
information  about  design  and  implementation  requirements. 

• Consistency  of  content  information  and  re-use  of  contents 

Another  problem  is  that  the  content  or  the  structure  of  the  course  units  may  get  inconsistent  during  workflow.  While 
the  content  author  has  her  or  his  version  (e.g.  a linear  MS-Word-Document),  the  hypermedia  producer  has  to  transfer 
content  and  structure  to  web  formats  (HTML  and  sometimes  JAVA  Add-Ons).  Changes  in  the  linear  version  have  to  be 
manually  transferred  to  the  hypermedia  version.  This  is  a source  of  errors  and  often  prevents  maintainance,  because  of 
the  extra  effort  it  will  take.  Keeping  such  a training  unit  up  to  date  often  requires  to  start  all  over  again  (e.g.  with 
updating  the  Word-Document)  and  repeat  former  development  steps. 

DESCRIPTION 

The  main  idea  is  to  overcome  the  problems  mentioned  above  by  introducing  a central  information  base  which  is  used  by  all 
participants  during  the  design  process:  content  author,  designer,  producer  and  quality  manager  (see  figure  I).  Using  a 
simple  relational  database  combined  with  a powerful  web  frontend  seems  to  be  adequate.  This  database  should  include  all 
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the  context  information  as  well  as  the  content  of  the  teaching  units.  In  a first  step  these  informations  are  only  presented  to 
the  involved  persons  but  no  contents  are  automatically  generated. 

For  database  organization  object  oriented  techniques  are  used.  Classes  of  information  like  “user  attributes”,  “organisational 
requirements”,  “content”,  etc.  are  identified  and  then  may  be  freely  combined  into  “views”.  These  views  are  role  dependent, 
for  example  the  content  author  may  concentrate  on  the  analysis  data  and  the  contents.  Whereas  the  producer,  who  has  to 
implement  the  learning  unit,  may  need  additional  design  rationales  added  by  the  designer.  A quality  manager  can  base  his 
evaluation  of  the  system  on  the  requirements  identified  and  documented  during  the  analysis  phase. 


As  the  size  and  the  number  of  the  learning  units 
is  difficult  to  handle,  techniques  of  the  OOA  are 
used:  abstraction,  inheritence  and 

generalization.  If  for  instance  many  different 
user  groups  are  to  be  considered,  they  may  be 
ordered  hierarchically:  “All  users”  “users  of 
the  teaching  unit”  “A  special  group  within 
these  user  group”.  Attributes  of  the  most 
general  object  (“All  users”)  are  inherited  by  all 
following  objects  and  then,  e.g.  the  designer’s 
view  includes  all  attributes,  from  the  general  to 
the  specific  ones.  OOA  techniques,  like 
underspecification  and  refinement  make  the 
handling  of  large  object  sets  easier  and  allow  all 
participants  to  start  with  rather  raw  data  and  to 
refine  them  during  the  lifecycle  of  the  developed 
system  using  OOA  techniques  based  on  a 
database  has  several  advantages.  The  database 
supports  the  complete  lifecycle  of  the  course  unit 
and  makes  all  information  and  design  rationales 
available  again  for  maintaining  or  updating  the 
course.  As  the  contents  are  HTML-based  they 
can  be  included  into  the  courses  with  less 
manual  effort.  Furthermore,  the  connection  of 
each  unit  to  the  position  and  role  in  the  database 
is  kept,  nection  to  the  database.  Additionally 
for  each  production  phase  there  is  appropriate 
additional  information  available,  like  design 
rationales  and  the  related  context  informations. 
Updating  a content  in  the  database 
automatically  updates  the  course  and  avoids 
inconsistencies. 

Current  and  future  work 

Currently  two  major  tasks  are  to  be  solved:  First 
the  different  object  classes  (content  and  context 
objects)  and  views  needed  for  the  educational 
systems  must  be  identified,  validated  and 
implemented.  Second  the  database  tool  has  to  be 
implemented  and  tested. 

As  the  described  method  is  similar  to  a special  OOA  technique  currently  under  development  in  our  institute  as  well 
(project  “TAToo”)  see  (Herczeg  1999),  we  reuse  implementation  resources  from  that  project.  Actually  the  database  frontend 
and  the  needed  JAVA/JDBC/JSP-Interface  will  be  the  same  in  both  projects. 
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Figure  1:  An  iterative  development  process  for  educational 
systems  (eg.  web  based  training  units).  Please  note  that  all  these 
steps  are  to  be  handled  iterativly,  that  means  the  whole  flow 
must  be  done  several  times. 
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Abstract:  In  an  Internet  environment,  users'  expectations  are  diverse  due  to  its  on-line 
characteristics.  Internet  can  be  assessed  by  any  user  of  any  age,  group,  race,  nationality,  and  gender 
all  over  the  world  at  any  time  and  at  any  place.  Considering  these,  web  site  designers  must  adhere 
to  certain  design  standards  and  properly  evaluate  their  sites  to  ensure  its  usability  and  effectiveness. 
Despite  abundance  of  studies  on  web  usability,  very  few  of  them  focus  on  the  political  web  sites. 
Using  political  web  sites  in  Malaysia  as  an  empirical  base,  this  research  intends  to  identify  factors 
affecting  the  use  of  political  web  sites;  and  then  based  on  the  analysis,  develop  a framework  for 
evaluating  the  usability  of  political  web  sites.  The  main  purpose  is  to  contribute  towards  the 
improvement  of  cyberdemocracy  in  the  information  age. 


Introduction 

The  central  focus  of  the  debate  on  the  success  of  a democratic  political  system  is  people  participation  in  election, 
political  discussion,  and  the  process  of  governance.  In  order  to  participate,  people  should  possess  sound  knowledge 
about  political  and  social  issues  affecting  them.  Traditionally,  people  are  mostly  dependent  on  mass  media  mainly 
newspapers,  magazines,  television  and  radio  as  sources  of  political  information.  Indeed,  mass  media  has  been  very 
influential  in  shaping  public  opinion  about  political  and  social  issues  in  any  democratic  country.  Today,  the  advent 
of  the  Internet  has  revitalized  the  whole  concept  of  communication  media.  Ideally,  it  does  not  only  provide  an 
alternative  source  of  political  information,  but  it  can  also  be  used  as  an  effective  political  communication  medium 
between  and  among  citizens,  public  leaders  and  political  parties.  This  development  certainly  provides  opportunities 
for  the  enhancement  of  democracy  whose  main  emphasis  is  on  citizens’  freedom  to  actively  participate  in  politics  by 
debating  and  exchanging  views  on  key  social  and  political  issues. 


Cyberdemocracy  and  Current  Practice 

There  is  little  agreement  on  the  exact  definition  of  cyberdemocracy  because  of  its  uncommon  usage.  Clift  (1997) 
states  that  cyberdemocracy  is  about  the  use  of  online  communication  tools  for  many-to-many  civic  discussions 
among  citizens  and  public  leaders.  It  is  based  on  the  belief  that  opens  communication  and  participation  is  the 
foundation  of  democracy.  According  to  Alexander  and  Grabbs  (1998),  cyberdemocracy  refers  to  the  optimized  use 
of  internet-based  communication  technologies  by  government  agencies,  interest  groups  and  non-profit  organizations 
to  promote  public  participation  in  the  process  of  governance.  In  general,  cyberdemocracy  is  defined  as  the  use  of  the 
Internet  technology  such  as  World  Wide  Web  (WWW),  File  Transfer  Protocols  (FTP),  electronic  mail  (email), 
conference  facilities  and  net  chatting  for  dissemination  of  information  and  exchange  of  ideas  between  and  among 
government,  political  parties  and  public  leaders  with  the  general  public.  There  are  a number  of  examples  of  efforts 
carried  out  by  government,  political  parties  and  voluntary  bodies  to  establish  cyberdemocracy.  Minnesota  e- 
democracy  project  (http://www.e-democracy.org),  established  in  1994  by  a group  of  volunteers,  held  the  first  online 
debate  via  email  among  candidates  for  governor  and  United  States  Senate  and  launched  the  MN-POLITICS  email 
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discussion  forum.  Another  example  is  United  Kingdom  Citizen  Online  Democracy  (http://www.democracy.org.uk), 
a non-partisan  effort,  that  began  work  before  the  1997  national  election.  It  hosted  a number  of  topical  events  on  such 
topics  as  European  monetary  union  efforts  and  online  delivery  of  government  services,  and  it  held  an  all-party 
debate  during  the  election. 


Web  Usability  and  Research  Focus 

Usability  is  basically  a very  broad  concept  in  system  design  and  difficult  to  define.  In  this  research,  usability  refers 
to  how  easy,  effective  and  useful  it  is  for  a person  who  uses  a political  web  site  to  achieve  their  goals.  According  to 
Nielsen  and  Mack  (1994),  and  Shneiderman  (1998),  there  are  5 main  aspects  of  usability:  ease  of  use,  efficiency, 
satisfaction,  rate  of  users'  error  and  retention  over  time.  The  primary  focus  of  this  research  will  be  on  the  usability 
issues  of  web  sites  used  as  political  communication  medium.  These  issues  are  grouped  into  seven  main  factors: 
Screen  design.  Content,  Accessibility,  Navigation,  Media  Use,  Interactivity,  Consistency  - abbreviated  by  the 
researcher  as  SCANMIC.  These  are  the  generic  factors  of  web  usability  based  on  an  analysis  of  current  web  design 
guidelines  and  literature. 


Research  Question 

In  an  Internet  environment,  users'  expectations  are  diverse  due  to  its  on-line  characteristics.  Internet  can  be  assessed 
by  any  user  of  any  age,  group,  race,  nationality,  and  gender  all  over  the  world  at  any  time  and  at  any  place. 
Considering  these,  web  site  designers  must  adhere  to  certain  design  standards  and  properly  evaluate  their  sites  to 
ensure  its  usability  and  effectiveness.  Despite  abundance  of  studies  on  web  usability,  very  few  of  them  focuses  on 
the  political  web  sites.  Yet,  usability  has  become  one  of  the  most  important  aspects  that  determine  the  success  of 
web  sites  regardless  of  their  category.  With  this  in  mind,  this  research  will  address  three  major  questions; 

a)  What  are  factors  that  affect  the  usability  of  political  web  sites? 

b)  How  to  evaluate  the  usability  of  political  parties'  web  sites? 

c)  What  is  the  level  of  the  usability  of  web  sites  in  current  cyberdemocracy  efforts  in  Malaysia? 


Conclusion 

The  above  research  questions  will  be  used  to  achieve  the  following  objectives: 

a)  To  identify  factors  that  affect  the  usability  of  political  parties'  web  sites. 

b)  To  design  a framework  for  evaluating  the  usability  of  political  web  sites.  The  framework  is  designed  for  both 
technical  and  non-technical  people  who  are  involved  in  political  web  site  development. 

c)  To  test  the  applicability  of  proposed  framework. 

d)  To  propose  a guideline  for  developing  a usable  political  web  site. 
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Abstract:  Biggs’  (1996)  proposed  a pedagogical  model  where  “performances  of  understanding” 
specified  in  course  objectives  are  used  to  systematically  align  teaching  methods  and  assessment 
practices.  The  model  was  applied  in  a plant  physiological  ecology  course  delivered  at  a distance 
over  the  web.  Students  and  instructor  thought  the  course  format  helped  learning  and  that  the 
assessment  model,  the  SOLO  Taxonomy,  was  good.  Everyone  also  concluded  that  the  course 
workload  was  too  great  in  spite  of  ample  feedback  to  discourage  such  excess.  The  course  may  be 
viewed  at:  http://www.biology.ualberta.ca/courses.hp/bot43 1 .hp/bot43  lhp.html 


Evidence  based  good  teaching  practice  seeks  to  enhance  “deep  learning”  by  emphasizing  active  learning 
strategies,  high  expectations,  time  on  task,  prompt  feedback,  respect  for  diverse  ways  of  learning  and  student-student 
and  instructor-student  interaction.  Deep  learning  is  supported  by  a constructivist  pedagogy  that  argues  that  it  is  the 
learner’s  activities  that  aid  their  construction  of  meaning  to  bring  about  learning. 

Biggs  (1996)  has  articulated  a “Constructive  Alignment”  model  that  is  the  marriage  of  two  ideas: 
constructivism  as  a framework  with  the  learner's  activities  as  central  in  creating  meaning  and  the  instructional 
designers  emphasis  on  the  relationship  between  the  learning  objectives  and  the  targets  for  assessing  student 
performance.  In  his  model,  the  "performances  of  understanding"  nominated  in  the  objectives  are  used  by  the 
instructional  designer  to  systematically  align  the  teaching  methods  and  the  assessment  activities. 

Biggs  and  Collis  (1982)  previously  developed  a ‘Structure  of  Observed  Learning  Outcomes  (SOLO) 
Taxonomy’  that  analyzes  the  cognitive  level  of  students’  written  work.  It  identifies  five  categories: 

1 ) prestructural  - no  understanding  of  the  topic, 

2)  unistructural  - nominal  understanding  of  one,  or  a few,  points, 

3)  multistructural  - several  aspects  understood  but  used  separately, 

4)  relational  - components  integrated  into  coherent  wholes, 

5)  extended  abstract  - the  integrated  whole  is  reconceptualized  at  a higher  level  of  abstraction. 

The  taxonomy  provides  a scheme  that  served  as  a basis  for  giving  feedback  to  students  on  their  written  work. 

The  Constructive  Alignment  model  was  applied  in  a distance  delivered  web  based  course  for  senior 
undergraduate  and  graduate  students.  For  each  week  of  the  course,  the  course  web  site  gave  details  of  the  learning 
objectives,  required  reading,  optional  supplemental  reading  and  web  hyperlinks  to  resource  sites.  The  web  site  also 
provided  an  extensive  background  on  the  pedagogical  principles  behind  the  course,  the  nature  of  the  SOLO 
Taxonomy,  sample  responses  to  a problem  and  their  assessment  according  to  the  Taxonomy,  and  advice  on 
preparing  portfolios  of  evidence  that  the  learning  objectives  had  been  achieved. 

Students  were  invited  to  assemble  their  own  portfolios  of  evidence  that  they  had  achieved  the  learning 
objectives.  Portfolios  were  submitted  by  e-mail  and  assessment  feedback  was  provided  in  the  terminology  of  the 
SOLO  Taxonomy.  Students  had  access  to  a password  protected  conferencing  system  that  allowed  them  to  interact 
with  each  other,  the  teaching  assistant  and  the  instructor  in  a virtual,  asynchronous  group  environment.  An  on-line 
chat  room  was  also  available  through  the  conference  system.  Private  communication  between  participants  was 
available  through  e-mail. 

The  course  opened  with  a telephone  conference  to  allow  the  students  to  meet  each  other  and  interact  with 
the  teaching  assistant  and  instructor.  The  students  and  the  teaching  assistant  then  moved  to  a computer  lab  where  all 


-Cl 

i.DOO 


students  had  the  opportunity  to  demonstrate  that  they  were  familiar  with  the  computer  and  software  skills  required  to 
complete  the  course  work. 

Student  assessment  in  the  course  was  based  on  the  submission  of  three  portfolios  of  evidence  of  meeting 
the  learning  objectives  through  the  term.  Take  home  mid-term  and  final  exams  were  also  given.  These  exams  were 
integrative  of  the  material  in  the  portfolios  and  gave  the  students  considerable  discretion  in  selecting  the  context  in 
which  the  questions  were  answered.  Exam  feedback  was  again  based  on  the  SOLO  Taxonomy.  As  the  eight 
students  in  the  course  had  no  previous  experience  of  distance  education  or  portfolio  assessment,  they  were  invited  to 
maintain  a reflective  journal  that  was  read  three  times  during  the  course  by  the  instructor.  Marks  for  keeping  the 
journal  were  given  on  a credit/no-credit  basis. 

At  the  end  of  the  course  students  were  asked  to  evaluate  the  course  and  their  learning  experience. 
Evaluations  were  submitted  to  the  TA  who  cut  and  pasted  the  anonymous  student  comments  into  an  e-mail  message 
to  the  instructor. 

The  evaluations  showed  that  the  students  thought  the  course  format  helped  learning,  they  developed 
independent  learning,  time  management  and  communication  technology  skills,  that  the  web  materials  on  the  SOLO 
Taxonomy  were  good,  that  the  web  material  on  the  course  philosophy  was  good  but  only  in  retrospect  and  that  the 
learning  journal  material  was  very  helpful. 

In  the  future  the  students  thought  that  workload  in  the  course  should  be  reduced,  feedback  increased,  course 
notes  provided,  more  texts  should  be  placed  on  reserve,  a lab  should  be  included  and  a TA  with  a background  in 
physiology  would  be  preferable. 

Students  thought  that  future  offerings  of  the  course  should  retain  the  chat  room,  group  camaraderie, 
learning  objectives,  the  SOLO  Taxonomy,  and  the  instructor’s  understanding. 

The  instructor’s  evaluation  concluded  there  should  be  a reduced  workload,  more  frequent  feedback,  greater 
use  of  an  on-line  seminar  facility  and  continued  use  of  the  SOLO  Taxonomy  and  learning  journals. 

Workload  was  an  issue  for  everyone.  Despite  the  feedback  given  on  their  portfolios,  the  volume  of 
evidence  provided  on  each  of  the  three  submission  dates  increased  steadily.  Students  were  spending  more  time 
assembling  evidence  and  the  instructor  was  spending  more  time  assessing  it.  Conversations  in  the  chat  room 
emphasized  the  concern  that  everyone  shared  and  a phone  conference  was  organized  to  discuss  the  problem  in  real 
time.  Despite  encouragement  by  the  instructor  to  pay  attention  to  the  quality  of  the  portfolio  contents  rather  than  the 
quantity,  the  problem  persisted. 

Providing  more  regular  feedback  would  have  spaced  out  the  assessment  workload  for  the  instructor  but  it  is 
not  clear  that  it  would  have  helped  reduce  the  pressure  the  students  placed  on  themselves  to  submit  a large  mass  of 
evidence  of  their  learning.  The  conferencing  tool  (WebTeach)  had  a seminar  facility  where  students  could  have 
posted  questions  for  their  peers  relating  to  the  learning  objectives.  Responses  to  questions  posted  by  their  peers 
were  not  readable  until  participants  had  submitted  their  own  answer.  This  facility  remained  unused  although  it 
might  have  been  a vehicle  to  model  the  depth  of  material  that  was  reasonable  in  a portfolio.  It  is  also  possible  that 
students  would  see  use  of  the  seminar  facility  as  even  more  work. 

Everyone  involved  in  the  course  was  satisfied  with  the  SOLO  Taxonomy  as  a framework  for  assessment.  It 
allowed  the  students  to  align  their  portfolios  to  the  learning  objectives  and  the  assessment  criteria.  Students  found 
preparing  a learning  journal  to  be  useful.  Evidence  of  concern  over  workload  was  recorded  in  the  journals  along 
with  recognition  of  the  feedback  they  were  getting  on  what  was  anticipated.  However,  recognition  of  the  issue  and  a 
discussion  of  solutions  did  not  solve  the  problem. 

The  learning  objectives  were  well  received  by  the  students.  However,  some  continued  to  think  that  course 
notes  would  have  helped.  It  seems  that  even  these  senior  students  were  uncomfortable  in  assuming  a large  degree  of 
control  over  their  own  learning.  It  is  worth  noting  that  some  students  commented  that  the  material  on  the  pedagogy 
behind  the  course  was  only  useful  “in  retrospect”.  If  these  various  opinions  were  expressed  by  the  same  set  of 
individuals,  it  might  suggest  that  more  quickly  coming  to  terms  with  the  pedagogical  background  might  have 
compensated  for  the  lack  of  the  course  notes. 

In  conclusion,  Biggs’  pedagogical  model  of  Constructive  Alignment  is  suitable  for  application  in  a web 
environment. 

References 

Biggs,  J.,  & Collis,  K.F.  (1982).  Evaluating  the  quality  of  learning:  the  SOLO  taxonomy.  New  York:  Academic 
Press. 

Biggs,  J.  (1996).  Enhancing  teaching  through  constructive  alignment.  Higher  Education  32,  347-364 


■»''»  r* 

i.  Q O / 


A Multimedia  Resources  Bank  for  Teaching  and  Learning 


Vincent  Hung,  Jacky  Pow 
Centre  for  Learning,  Teaching  and  Supervision 
Hong  Kong  Institute  of  Education,  Hong  Kong 
vhung@ied.edu.hk , jpow@ied.edu.hk 


Winnie  So 

Department  of  Science 

Hong  Kong  Institute  of  Education,  Hong  Kong 
wiso@ied.edu.hk 


Abstract:  This  paper  introduces  an  attempt  to  make  use  of  Information  Technology 
to  facilitate  teaching  and  learning  in  a teacher  education  institute.  A multimedia 
video  bank  was  being  developed  to  provide  with  teacher  educators  and  student 
teachers  necessary  video  clips  diat  could  be  made  use  to  produce  computer-  assisted 
instruction  programs  that  fit  the  local  context.  All  the  videos  are  intentionally  to  do 
without  any  narration  or  background  music  to  allow  flexibility  for  the  subject 
teachers  to  present  their  materials  and  to  better  communicate  with  the  students. 


Background 

In  the  First  Policy  Address  of  the  Chief  Executive  of  the  Hong  Kong  Special  Administrative  Region 
(HKSAR)  Government,  a firm  devotion  of  the  Government  to  develop  information  technology  (IT)  in 
education  was  first  revealed.  However  the  education  community  in  HK  is  still  striving  to  seek  the 
direction  of  IT  in  education  that  matches  the  local  context,  a unique  culture  of  the  East  and  the  West. 
Most  of  them,  especially  the  primary  school  teachers  are  puzzled  simply  because  they  do  not  know 
how  to  start  with  teaching  with  IT. 


With  the  aim  to  facilitate  teaching  and  learning  with  IT  that  fits  the  local  context,  this  project  attempts 
to  build  up  a multimedia  resources  bank  (MMRB)  for  teacher  educators  and  student  teachers  in  the 
Hong  Kong  Institute  of  Education  (a  publicly-funded  teacher  education  institute)  to  develop  their  own 
computer- assisted  instruction  (CAI)  programs  or  to  use  as  part  of  the  teaching  materials.  It  works 
towards  a direction  to  encourage  and  promote  innovative  teaching  approaches  with  an  effective  use  of 
IT  in  classrooms. 

Multimedia  Resources  Bank  in  General  Studies 

Despite  of  the  resources  put  in  acquiring  computer  hardware  and  software,  there  is  hardly  any  quality 
interactive  CAI  program  in  “General  Studies”  being  developed  because  there  are  difficulties  facing  by 
the  primary  school  teachers: 

[1]  The  heavy  workload  does  not  allow  them  to  have  location  shooting  and  video  editing  that  are  both 
needed  in  developing  the  CAI  programs  in  “General  Studies”. 

[2]  It  is  not  worthwhile  for  the  primary  school  teachers  to  heavily  involve  in  video  production,  as  it  is 
not  their  profession  in  this  aspect. 

However,  there  are  enthusiastic  primary  school  teachers  who  are  willing  to  produce  their  own 
multimedia  teaching  materials  provided  that  there  is  enough  support  (Pow,  So  and  Hung,  2000).  With 
this  in  mind,  we  therefore  tried  to  build  a MMRB  in  “General  Studies”  for  the  primary  schools  teachers 
and  let  them  to  make  use  of  the  resources  to  tailor  make  their  own  CAI  programs  to  facilitate  teaching 
and  learning.  With  the  cooperation  between  a lecturer  from  Department  of  Science  and  education 
technology  professionals  from  the  Centre  for  Learning,  Teaching  and  Supervision  in  Hong  Kong 
Institute  of  Education,  the  team  has  produced  about  a hundred  video  clips  according  to  the  syllabus  of 
the  “General  Studies”  to  be  put  on  the  MMRB.  We  chose  “General  Studies”  as  the  starting  point  for 
MMRB  as  it  is  a new  subject  that  integrates  science,  social  study  and  health  education  in  primary 
school  curriculum,  and  hence  at  hand  resources  are  still  lacking. 


Objective 


The  content  of  the  video  clips  covers  a broad  area  of  topics  from  Hong  Kong  community  services, 
transportation,  housing,  etc.  to  scientific  experiments  such  as  the  property  of  magnet,  heating,  etc.  The 
videos  are  classified,  edited  into  a reasonable  length  and  stored  onto  a video  server  for  easy  retrieval 
and  viewing.  All  the  videos  are  intentionally  to  do  without  any  narration  or  background  music  since  we 
would  like  to  allow  flexibility  for  the  subject  teachers  to  present  their  materials  and  to  better 
communicate  with  the  students.  Although  it  is  the  Institute  that  owns  the  copyrights,  staff  and  students 
can  use  it  freely  in  their  own  works  or  teaching.  Staff  and  students  can  instantly  view  the  video  clips  of 
their  choice  at  any  computer  that  is  linked  up  with  the  Institute’s  Intranet  or  download  these  media 
clips  for  their  own  use  later  on.  This  MMRB  is  expected  to  be  able  to  save  up  valuable  time  and 
resources  for  those  who  want  to  try  out  teaching  with  IT  but  do  not  have  enough  time  and  resources  to 
produce  the  media  on  their  own.  It  is  hoped  that  this  could  contribute  partly  to  promote  innovative  use 
of  IT  in  teaching  and  learning,  not  Just  within  our  Institute  but  also  to  the  education  community  at 
large. 
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Abstract;  The  purpose  of  this  study  is  to  examine  the  effects  of  learner’s  PBL 
authenticity  perception  on  achievement  and  motivation.  There  are  two  major 
research  questions.  First,  is  there  a difference  on  learner’s  achievement  between  high 
and  low  perception  level  of  authenticity?  Second,  is  there  a difference  on  learner’s 
motivation  between  high  and  low  perception  level  of  authenticity?  The  results  show 
the  following:  (1)  There  is  a significant  difference  in  the  learner’s  achievement 
between  low  and  high  level  of  authenticity  perception  (t=-3.42,  p<.05)  (2)  There  is 
a significant  difference  in  learner’s  motivation  (t=-3.34,  p<.05). 


Introduction 

In  recent  years,  with  the  advent  of  the  Information  Society,  the  focus  of  educational  paradigm  has  been 
shifting  from  ’’teaching”  to  ’’learning”.  This  signifies  a move  to  a learner-centered  education  where  the 
student  becomes  an  active  participant  in  the  learning  process.  Korea's  educational  reform  committee  has 
established  a goal  to  build  a learner-centered  educational  environment  to  prepare  Korea’s  students  to  be 
competitive  in  21*‘  centuiy. 

“Web’s  Cool”  program  has  been  planned  since  1998  to  support  the  learner-centered  education  system 
in  Korea.  Subject  areas  such  as  language  arts,  math,  science  and  ESL(English  as  a Second  Language) 
for  middle  school  students  have  been  developed  on  the  Web.  Since  the  Web  was  considered  to  be  an 
effective  medium  for  learner-centered  approach  by  applying  the  constructivistic  principles, 
PBL(Problem  Based  Learning)  strategy  were  applied  to  provide  students  opportunities  to  participate  in 
the  learning  process  actively. 

While  the  Web’s  Cool  has  been  developed,  we  wanted  to  investigate  the  fact  that  whether  the  problem 
we  were  using  as  an  authentic  task  has  any  effect  on  learner’s  motivation  and  achievement.  In  order  to 
measure  the  authenticity  of  the  task,  the  expert’s  as  well  as  student’s  perception  level  of  the  authenticity 
toward  the  task  has  been  scored  by  using  a 5 point  scale.  Two  tasks  were  selected  as  high  and  low 
authenticity  level  of  perception  by  both  10  experts  and  10  students  to  investigate  the  effects. 

The  purpose  of  this  study  is  to  examine  the  effects  of  the  authenticity  perception  on  learner’s  motivation 
and  achievement.  There  are  two  major  research  problems.  First,  is  there  a difference  on  learner’s 
motivation  between  high  and  low  perception  level  of  authenticity?  Second,  is  there  a difference  on 
learner’s  achievement  between  high  and  low  perception  level  of  authenticity? 

Research  Method 


0 


Subjects 


Subjects  for  this  study  are  5 1 students  of  the  middle  school  students.  Two  learning  tasks  perceived  low 

and  high  authenticity  are  selected. 

Results 

The  results  are  as  follows: 

(1)  There  is  a significant  difference  in  the  learner's  achievement  in  general  between  low  and  high  level 
of  authenticity  perception  (t=-3.42,  p<  .05).  A significant  difference  was  found  in  the  learner's 
authentic  achievement  between  low  and  high  level  of  authenticity  perception  (t=-4.54,  p<  .05). 
However,  no  significant  difference  was  found  in  the  learner's  logical  achievement  between  low  and 
high  level  of  authenticity  perception  (t=-1.75,  p<  .05). 


Achievement 

Authenticity  perception  level 

Mean 

SD 

t 

P 

Authentic  Achievement 

High 

1.58 

.77 

-4.54* 

.00 

Low 

1.05 

.88 

Logical  Achievement 

High 

1.69 

1.01 

-1.75 

.09 

Low 

1.34 

1.03 

Total  Achievement 

High 

3.27 

1.56 

-3.42* 

.00 

Low 

2.42 

1.99 

(*:p<.05) 


(2)  There  is  a significant  difference  in  learner's  motivation  of  a total  (t=-3.34,  p<.05),  attention(t=-3.16, 
p<.05),  relevance(t=-2.55,  p<.05),  confidence(t=-2.85,  p<.05)  and  satisfaction(t=-2.34,  p<.05) 
between  low  and  high  level  of  authenticity  perception. 


Motivation 

Authenticity  perception  level 

Mean 

SD 

t 

P 

Attention 

High 

3.68 

.57 

-3.16* 

.003 

Low 

3.44 

.52 

Relevance 

High 

3.64 

.50 

-2.55* 

.014 

Low 

3.47 

.53 

Confidence 

High 

3.47 

.42 

-2.85* 

.006 

Low 

3.29 

.41 

Satisfaction 

High 

3.40 

.46 

-2.34* 

.023 

Low 

3.25 

.45 

Total 

High 

3.45 

.42 

-3.34* 

.001 

Low 

3.27 

.40 

(*:p<.05) 


Conclusions 

Based  on  the  results  of  the  study,  following  suggestions  were  made:  (1)  Measuring  scale  of  authenticity 
of  the  learning  task  has  to  be  further  validated  through  many  different  situations  and  various  subjects 
(2)  Authentic  assessment  of  the  authentic  learning  environment  has  to  be  studied  further. 
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Abstract:  Wireless  Application  Protocol  (WAP)  provides  a universal  open  standard  for  bringing 
Internet  and  Intranet  content  and  advanced  Value  Added  Services  to  mobile  phones,  PDA’s  and 
other  wireless  devices  [1].  The  production  of  WAP  applications  and  content  requires  new  software 
professionals  on  the  area.  In  this  paper  we  describe  our  experiences  from  the  multimedia  course 
where  students  developed  WAP  applications  as  their  project  work[5]. 


Background 

WAP  is  an  industry  standard  whose  purpose  is  to  achieve  universal  mobile  access  to  Internet  based 
information  services.  The  application  is  independent  of  the  underlying  digital  wireless  network  technology  [1]. 
WAP  applications  are  developed  with  Wireless  Markup  Language  (WML)  and  WML  Script.  [2].  WML  applications 
can  be  developed  with  special  toolkits,  for  example  Nokia  WAP  Development  Toolkit , Ericsson  WAP  development 
kit  or  UP.PHONE  development  environment . These  toolkits  include  smart  editors  and  debuggers,  phone  simulators 
and  help  system.  However,  its  is  possible  to  develop  your  WML  applications  using  only  a simple  ASCII  editor.  The 
students  developed  simple  WAP  applications  in  the  laboratory  work  and  implemented  a larger  project  work  in  the 
end  of  the  course. 

Wireless  Markup  Language  SpeciHcation  and  WML  Script 

The  Wireless  Markup  Language  specification  describes  the  markup  language,  WML,  including  the  document 
semantics.  Document  Type  Definition  (DTD),  and  encoding  extensions.  WML  is  a markup  language  based  on 
Extended  Markup  Language  (XML)  and  is  intended  for  use  in  specifying  content  and  user  interfaces  for  narrowband 
devices,  including  cellular  phones  and  pagers.  All  information  in  WML  is  organized  into  a collection  of  cards  and 
decks  [3].  WML  Script  is  a scripting  language  that  makes  it  possible  to  implement  dynamical  elements  in  the  WML 
application. 

Student's  WAP  projects 

The  students  were  required  to  design  and  implement  a real  WAP  software  and  document  the  code  in  the  end  of 
the  course.  The  topic  of  the  project  was  free  and  many  of  the  students  chose  to  implement  a game  in  WAP  phone.  In 
this  chapter  we  will  present  three  WAP  games,  namely  Moonlanding  game,  Lucky  Wheel,  and  Jackpot.  The  idea  of 
the  Moonlanding  game  is  to  land  to  the  surface  of  the  moon  by  controlling  the  fuel  consumption  of  the  rocket  (Fig. 
3).  In  the  Lucky  Wheel  game  your  mission  is  to  find  out  a secret  word  or  words  by  guessing  the  missing  characters. 
Jackpot  is  the  WAP  implementation  of  the  old  casino  game  (Fig.  3).  Some  of  these  applications  are  being  developed 
furthermore  and  will  be  published  in  the  commercial  WAP  service  by  the  national  mobile  phone  operator. 

The  first  step  in  the  project  was  the  learning  of  the  WAP  development  environment.  The  Nokia  Wap  software 
toolkit  was  used  in  the  projects.  In  general,  programming  in  WML  and  WMLS  is  similar  than  programming  in 
almost  any  other  environment.  It  is  possible  to  use  the  toolkit  as  a stand-alone  environment  or  with  Nokia  WAP 
gateway  in  order  to  simulate  real  wireless  connections  [6].  In  our  case,  it  was  not  required  for  the  students  to  test 
their  software  in  real  phones.  During  the  development  of  their  software  the  students  find  out  several  problems.  One 
of  the  most  significant  problem  was  the  differences  between  the  real  phone  and  the  simulated  phone.  Of  course  the 
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toolkit  itself  was  beta-level  software  and  was  not  very  reliable.  In  addition  the  real  WAP  phone  does  not  flillfill  the 
WAP  specification  [4].  For  example,  the  Nokia  7110  WAP  phone  does  not  support  table  feature  in  the  WML 
content  and  the  WAP  applications  can  use  only  a little  memory  - the  memory  size  in  the  phone  is  appx.  2 KB. 

Figure  3.  The  moonlanding  game  and  the  Jackpot. 


Positive  aspects  in  the  project  were  that  the  project  was  very  motivating,  the  simulator  makes  it  possible  to  develop 
software  without  the  real  phone,  it  was  easy  to  learn  to  program  with  WML  and  WMLS  because  they  are  similar 
than  HTML  and  JavaScript  and  students  used  actively  WAP  discussion  groups  in  internet. 

Conclusions  ^ 

Increasing  global  competition,  digitalization  of  content  the  general  use  of  HTML  and  WML  languages  and  the 
adoption  of  mobile  broadband  data  transfer  will  require  software  specialists  with  new  skills.  In  the  future  content 
producers,  telecommunication  companies,  IT  companies  and  media  companies  need  workers  with  good  knowledge 
technological  convergence  and  good  ability  to  use  the  new  content  producing  technologies  like  WAP.  This  is  a 
demand  for  the  educational  institutes  and  universities.  We  should  provide  up-to-date  technological  education  and  be 
prepared  to  modify  our  curriculum  to  follow  the  latest  technological  innovations.  In  this  paper  we  have  discussed 
about  WAP  as  a new  technology  to  implement  wireless  content  services.  The  students  at  Tampere  University  of 
Technology  studied  the  WAP  technology  and  implemented  simple  WAP  applications  as  their  project  work.  The 
applications  were  developed  using  freely  downloadable  Nokia  WAP  development  toolkit.  The  most  significant 
problem  was  the  differencies  between  the  real  WAP  phone  and  the  simulated  WAP  phone.  From  our  point  of  view 
the  projects  succeeded  well:  some  of  these  applications  are  being  developed  furthermore  and  will  be  published  in  the 
commercial  WAP  service  by  the  national  mobile  phone  operator. 
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This  paper  examines  a project  which  aims  to  improve  dental  clinical  management  through  case 
management  cf  medically  ccmpr^y^niiocd  dentdi  ^diiciud.  xviiiiiig  uic  pasi  icw  yeais^  ii  iias  oecorne 
increasingly  difficult  to  access  suitable  patients,  especially  child  patients,  for  dental  students  and 
consequently  for  them  to  become  sufficiently  experienced  in  treating  patients  with  medical 
conditions.  Multimedia  simulation  provides  a means  to  fill  this  gap  and  allows  interactive  learning  in 
a non-threatening  environment.  This  paper  examines  the  design,  development,  project  management 
and  evaluation  of  the  first  module  - congenital  heart  disease. 

Teaching  dental  students  to  treat  patients  safely  and  efficiently  presents  many  challenges.  The  declining 
availability  of  suitable  patients  attending  the  teaching  clinics  of  the  Royal  Dental  Hospital  of  Melbourne 
prevents  students  from  developing  a range  of  experiences  in  total  patient  management.  A disadvantage  of 
traditional  preclinical  laboratory  teaching  is  that  students  are  not  able  to  integrate  theoretical  and  practical 
skills.  Consequently  there  are  concerns  that  dental  students  are  not  competent  in  combining  preventive  and 
restorative  management  philosophies  while  integrating  diagnosis  and  treatment  planning  (Suvinen,  Messer, 
Franco,  1998).  Multimedia  case  simulations  were  considered  to  be  a viable  alternative  as  they  replicate  the 
dental  clinic  without  requiring  ‘live’  patients.  These  modules  provide  an  opportunity  to  develop  and 
consolidate  the  concept  of  integrated  patient  care. 

During  1999  the  first  module,  congenital  heart  disease  (CHD)  was  designed  and  completed.  During  2000 
further  funding  will  allow  three  other  modules  to  be  developed:  diabetes,  Down  syndrome  and  cleft  lip  and 
palate.  These  medical  conditions  were  chosen  as  they  are  the  most  likely  medical  conditions  to  be 
encountered  and  are  least  understood  by  a dentist  in  general  practice  treating  children.  Each  condition  will 
follow  the  existing  template  developed  for  the  CHD  module  except  for  the  diabetes  module  which  will  use 
a 3-D  Dental  surgery  to  enhance  interactivity.  The  CHD  module  contains  a description  and  review  of  the 
medical  condition,  considerations  of  medical  management  and  dental  management.  The  student  is  then 
presented  with  a case  scenario  of  a child  with  CHD.  The  student  constructs  a treatment  plan  and 
management  approach  after  considering  a range  of  treatment  options.  The  module  series  will  replace  some 
tutorial  and  lecture  content  for  both  4^^  and  5*^  final  year  dental  students.  The  modules  will  not  replace 
valuable  clinical  practice;  but  prior  learning  from  the  modules  will  enhance  students’  ability  to  perform 
clinical  work.  The  modules  will  also  be  used  for  post-graduate  students  and  in  Continuing  Dental 
Education  programs  for  dentists. 

Design  and  Development 

In  designing  and  developing  the  CHD  module,  the  project  team  followed  the  model  utilised  by  the 
Biomedical  Multimedia  Unit  (Keppell,  1998).  The  design  phase  of  the  project  focussed  on  developing  the 
macro-structure  for  the  module  in  the  form  of  a concept  map  (Novak  & Gowin,  1 984)  and  completing  the 
micro-structure  in  the  form  of  a ‘planning  grid’.  The  content  experts  for  the  project  were  experienced 
specialists  and  educators  in  the  field  of  paediatric  dentistry.  They  worked  with  the  instructional  designer  to 
develop  the  concept  map  and  planning  grids  for  the  project.  Figure  1 http://www.mcdfac.unimelb.cdu.au/Ed- 
Media2QQQ/  Outlines  the  resultant  concept  map  which  provided  a scaffold  for  the  development  of  the  planning 
grids.  The  process  used  to  develop  the  concept  map  and  planning  grids  followed  the  Content  Production 
Process  (Keppell,  1999).  A specific  form  of  storyboard  or  ‘planning  grid’  has  been  developed  to  provide  a 
‘communication  tool’  for  interfacing  between  the  instructional  designer,  content  expert  and  graphic 
designer/programmer  (Keppell  & Buschgens,  1995).  The  planning  grid  is  useful  in  streamlining 
communication  and  providing  a common  ground  for  discussing  the  multimedia  project.  The  planning  grid 
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is  analogous  to  an  ‘architectural  blueprint’,  which  can  be  applied  or  ‘engineered’  by  the  graphic 
designer/programmer  (see  Figure  2 http://www.medfac.unimelb.edu.au/Ed-Media2000/).  The  graphic  designers 
developed  the  interface  for  the  congenital  heart  disease  module  and  the  subsequent  modules  (see  Figure  3. 
http://www.medfac.unimelb.edu.au/Ed-Media2000A 


Project  Management  & Evaluation 

The  project  management  focussed  on  carefully  scoping  the  project,  providing  a detailed  planning  grid  to 
the  graphic  designer/programmer  and  establishing  the  “look  and  feel”  of  the  interface.  England  and  Finney 
(1999)  suggest  that  each  project  will  define  its  “own  quality  priorities”  (p.l7).  In  this  project  the  priorities 
of  target  audience,  subject,  budget  and  the  time  available  impacted  on  the  project  management  of  the 
project.  England  and  Finney  (1999)  also  suggest  that  “design  quality  for  media  projects  = content  and 
treatment  agreement”  (p.l7).  These  treatment  parameters  were  carefully  considered  in  the  initial  stages  of 
the  project  conceptualisation.  The  first  module  was  targeted  to  be  completed  within  an  hour  by 
undergraduate  users.  The  macrostructure  for  integrating  the  subsequent  modules  was  also  considered  as  an 
essential  characteristic  of  the  project.  In  order  to  analyse  project  costs  and  as  a means  to  plan  for  the  future 
modules,  the  total  work  hours  were  tracked  for  this  module.  A total  of  390  hours  was  required  to  produce 
the  CHD  module  (see  Table  1 http://www.medfac.unimeib.edu.au/Ed-Media2000A.  The  CHD  module  was  completed 
in  November,  1999  and  was  evaluated  with  a group  of  15  recent  graduates  (interns  and  postgraduate 
students)  at  the  School  of  Dental  Science  in  December  1999,  using  a Likert-style  questionnaire.  Table  2 
http://www.medfac.unimelb.edu.au/Ed-Media2000/  shows  the  distribution  of  responses  to  questions  concerning 
content.  The  majority  of  all  responses  were  either  “strongly  agree”  or  “agree”,  with  most  modal  responses 
being  “strongly  agree”  (Table  3).  httD://www.medfac.unimelb, edu.au/Ed-Media20QQ/  Most  respondents  correctly 
identified  the  deliberately  placed  distractor,  “confusing”.  In  general  appraisal,  the  majority  of  respondents 
rated  the  module  as  “good”  or  “excellent”,  and  “very  easy”  to  operate  and  navigate  (Table  4). 
http://www.medfac.unimelb.edu.au/Ed-Media2000/  Of  interest.  One  respondent  rated  the  module  “very  poor”  and 
“difficult”  on  these  points.  The  module  is  currently  undergoing  beta  testing  at  three  national  and  three 
international  locations,  in  departments  of  pediatric  dentistry. 

Discussion 

Computer-assisted  education  in  dentistry  is  in  its  early  stages  (Yip  and  Barnes,  1999).  Developments  are 
occurring  very  rapidly,  and  educators  must  evaluate  modules  developed  internally  or  purchased  externally 
(Mercer  and  Ralph,  1998).  Sound  instructional  design  indicates  the  goals  of  instruction  must  be  made  clear, 
so  that  users  know  what  is  expected  of  them  in  operating  and  navigating  a module.  This  may  not  have  been 
so  in  the  instance  of  one  respondent.  Nonetheless,  it  is  concluded  that  computer  assisted  instruction  was 
enthusiastically  received  by  most  of  the  recent  graduates.  Experience  in  focus  groups  has  shown  that 
participants  valued  the  tutor  discussions  that  were  held  at  the  end  of  the  module,  which  allowed  extension 
of  learning  and  further  professional  role-modelling. 
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Abstract:  At  the  George  Washington  University  teacher  candidates  in  the  master’s  program 
learn  to  apply  technology  in  teaching  and  learning  within  the  context  of  methods  and 
curriculum  courses  and  school-based  practicum  and  internships.  Teacher  candidates  acquire 
skills  and  understanding  as  they  prepare  presentations,  develop  curriculum  materials,  teach 
lessons,  assess  student  learning,  and  reflect  on  practice.  This  short  paper  describes  the  ways 
in  which  teacher  candidates’  development  of  multimedia  electronic  portfolios  to  assess  their 
ability  to  implement  a variety  of  instructional  models  assists  in  learning  to  use  a variety  of 
hardware  and  software  for  instructional  purposes.  It  is  expected  that  teacher  candidates  will, 
on  some  level,  transfer  their  proficiency  in  using  technology  to  the  secondary  students  with 
whom  they  work  during  their  teaching  internships. 

Why  Electronic  Portfolios? 

Performance  assessment  in  teacher  education  programs  commonly  includes  development  of  a teaching 
portfolio  during  the  teacher  internship.  The  purposes  of  the  portfolio  are  to  document  professional  growth 
(Campbell,  et.  al.  2000)  and  to  cultivate  outstanding  teaching  and  learning  (Wolf,  1996).  Fulfilling  these  purposes 
requires  that  the  portfolio  is  a dynamic,  flexible  documentation  of  student  growth.  Emerging  technologies  facilitate 
creating  electronic  portfolios  that  support  flexibility  rather  than  traditional  portfolios  constructed  in  3-ring  binders 
that  tend  to  be  collections  of  artifacts.  The  electronic  format  offers  several  advantages:  the  ease  of  storage  usually  on 
a computer  hard  drive  or  some  sort  of  removable  media  (floppy  disk.  Zip  disk,  CD);  easy  access  to  information; 
portability  is  enhanced;  multiple  copies  can  be  created  at  low  cost;  the  opportunity  to  add  sound,  pictures,  graphics, 
and  even  video  make  an  electronic  portfolio  more  attractive  and  interactive  than  traditional  style  binders;  in  the 
process  of  developing  portfolios,  students  gain  valuable  technology  skills  as  they  create  and  edit  this  multimedia 
document.  These  skills  can  be  transferred  to  6-12  students  during  the  internship  and  as  the  successful  teacher 
candidate  moves  into  classroom  teaching. 

Development  of  the  Electronic  Portfolio 

A group  of  teacher  candidates  in  the  secondary  education  graduate  program  at  The  George  Washington 
University  are  expanding  their  technology  proficiencies  through  developing  an  electronic  portfolio.  The  primary 
purpose  is  to  provide  teacher  candidates  with  technology  related  skills  that  they  can  transfer  to  middle  and  high 
school  students  and  use  to  support  their  teaching.  A secondary  purpose  is  to  enable  teacher  candidates  to 
demonstrate  their  teaching  competencies  using  an  alternative  method  to  the  binder-style  portfolio  currently  in  use  in 
many  teacher  preparation  programs. 

The  conceptual  framework  used  to  develop  the  portfolios  is  based  on  the  Interstate  New  Teacher 
Assessment  and  Support  Consortium  Standards  (Darling-Hammond,  1992)  and  the  National  Board  for  Professional 
Teaching  Standards  (1991).  The  domains  addressed  include:  1)  commitment  to  students  and  their  learning,  2) 


knowledge  of  the  subjects  taught  and  how  to  teach  those  subjects  to  students,  3)  establishment  and  management  of 
student  learning  in  a positive  environment,  4)  assessment  of  student  progress  and  adapting  instruction  to  improve 
student  learning,  5)  commitment  to  reflective  practices,  and  6)  professionalism.  Several  indicators  are  provided  to 
students  within  each  domain  specifying  ways  in  which  each  is  exemplified  in  teaching  or  in  course  work. 

Those  participating  are  currently  enrolled  (spring  2000)  in  a class  that  teaches  application  of  instructional 
and  classroom  management  models.  The  instructional  models  reviewed  include  Direct  Instruction,  Inductive, 
Inquiry,  Integrative,  and  Cooperative  Learning.  Each  of  the  models  represents  a cognitive  perspective  of  learning 
which  views  learners  as  active  investigators  of  their  environment.  In  these  models,  teachers  are  directly  involved  in 
guiding  learning  through  questioning  and  discussion.  Candidates  construct  a number  of  products  to  show  their 
understanding  of  concepts  and  theory  and  their  ability  to  apply  them  in  a teaching-learning  situation. 

Specifically,  teacher  candidates  learn  how  to  use  various  technologies  as  they  learn  pedagogy.  In  the 
course,  they  will  practice  using  tcchriolog>'  to:  present  information;  research  content;  organize  and  store  data:  track 
and  record  their  development  as  teachers;  and  reflect  on  their  teaching  behaviors  as  a process  for  solving  problems 
and  growing  professionally.  For  example,  they  will  learn  to  use  PowerPoint  in  the  context  of  learning  how  Howard 
Gardner’s  Theory  of  Multiple  Intelligences  applies  to  the  teaching  of  science,  math,  English,  social  studies  or  ESL. 
Each  candidate  will  design  a presentation  that  outlines  how  each  of  the  8 intelligences  is  used  to  frame  curriculum 
development  of  a specific  unit  topic.  Candidates  select  topics  according  to  their  content  specialization.  Each  will 
focus  on  the  utility  of  the  theory  in:  1)  designing  curriculum,  2)  engaging  and  motivating  students,  and  as  3) 
assessing  students’  learning. 

In  demonstrating  their  understanding  of  and  ability  to  apply  instructional  models,  candidates  will  teach 
lessons  they  develop  to  other  members  of  the  class.  They  will  be  videotaped  using  a digital  video  camera.  After 
receiving  feedback  from  peers  and  the  course  instructor,  the  teacher  candidates  will  view  the  video  and  reflect  on  the 
effectiveness  of  their  teaching.  As  a part  of  the  reflective  process,  they  will  edit  their  video  to  capture  those  portions 
that  exemplify  what  they  have  learned  about  their  teaching  and  will  include  this  shortened  version  in  the  electronic 
portfolio.  In  the  process  of  creating  the  portfolio,  teacher  candidates  will  learn  how  to  create  a personal  website. 
The  goal  is  to  expand  their  portfolios  as  they  move  through  the  graduate  program.  Unlike  what  typically  occurs  in 
the  binder  approach  to  portfolios  in  which  students  organize  a collection  of  artifacts  during  the  teaching  internship 
which  illustrate  their  best  work,  the  approach  taken  here  will  concentrate  on  documenting  growth  as  teacher 
candidates  develop  their  knowledge  and  skills  in  both  pedagogy  and  content.  This  means  that  candidates  will  begin 
assembling  their  work  during  their  initial  experiences  in  the  program  and,  the  work  included  in  the  portfolio  will  not 
necessarily  showcase  their  best  products.  Rather,  the  final  product  will  reveal  the  ways  and  extent  to  which  each 
candidate  has  grown  as  a beginning  teacher. 

Conclusion 

One  major  area  of  emphasis  in  the  teacher  preparation  program  at  the  George  Washington  University  is  to 
prepare  teachers  who  will  serve  as  leaders  in  the  schools  where  they  teach.  An  important  aspect  of  that  leadership 
role  is  technology.  As  our  students  develop  their  proficiency  in  this  area,  they  will  be  encouraged  to  explore  the 
connections  to  teaching  in  middle  or  high  school.  The  ways  in  which  technology  use  can  enhance  teaching  and 
learning  is  of  particular  importance.  Specifically,  through  the  development  of  electronic  portfolios,  these  teacher 
candidates  will  consider  how  the  use  of  technology  can  assist  secondary  level  students  in:  learning  content; 
presenting  information;  conducting  research;  and,  reflecting  on  their  development  as  learners. 
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Abstract:  This  paper  is  based  on  experiences  with  the  production  of  web-based  training  (WBT)  courses  for  a 
course  of  studies  in  medical  computer  science.  Course  production  is  regmucd  as  software  development  process, 
which  starts  with  conceptual  models  shaping  the  design  and  defining  the  way  the  software  product  is  presented 
to  the  user.  This  paper  reports  experiences  with  conceptual  design  models  and  shows  the  danger  of  being 
insufficient  or  incorrect. 


INTRODUCTION 

Our  project  “Distance  Education  in  Medical  Computer  Science”  (started  in  January  1999)  aims  at  providing  a 
complete  course  of  studies  for  the  specialization  of  students  in  medical  computer  science,  offered  at  a virtual 
university.  Our  responsibility  is  to  transfer  the  linear  text  documents  (mostly  MS-Word  format)  into  hypermedia 
networks  and  multimedia  courses,  which  means  practicing  reverse  engineering  for  WBT  course  development. 

PROBLEMS  WITH  CONCEPTUALIZING  WBT  COURSES 

The  transfer  of  a text  document  to  hypermedia  means  transformation  of  format,  gain  of  flexibility,  interaction 
capabilities  and  so  on.  Hypermedia  and  multimedia  courses  can  be  used  in  ways  beyond  the  intentions  of  the 
content  author.  On  the  other  side,  people  occupying  other  roles  in  the  development  process,  like  multimedia 
designers,  or  quality  managers  do  not  know  about  the  content  domain  or  didactic  aspects.  The  static  version, 
which  the  content  author  had  in  mind,  has  only  one  dimension  of  presentation.  Accordingly,  the  semantic 
(content)  structure  and  the  didactic  structure  of  the  course  are  intertwined.  Most  authors  are  not  able  to  separate 
both  structures  in  a way  applicable  for  hypermedia  production  and  vice  versa  the  multimedia  experts  are 
normally  no  experts  in  the  semantic  domain  or  in  didactics,  therefore,  they  can  hardly  explain  their  needs  to  the 
content  authors.  This  lack  of  knowledge  in  each  others  domain  results  in  unintended  course  structures, 
unnecessary  restrictions  or  other  mismatches.  A similar  problem  arises  if  the  author  for  example  has  to  start  with 
specifying  a semantic  model  and  then  would  have  to  design  for  different  learner  groups. 

Furthermore,  we  found  in  our  projects,  that  media  designers,  producers,  and  content  authors  often  have 
insufficient,  incomplete,  and  partly  incorrect  models  about  the  learners,  the  task  and  the  situation  of  use.  The 
models  vary  at  different  times  during  the  period  of  course  production. 

COURSE  PRODUCTION  AS  SOFTWARE  DEVELOPMENT  PROCESS 

Our  approach  to  course  production  as  practised  in  our  projects  is  to  regard  the  production  of  hypermedia  and 
multimedia  WBT  courses  as  a software  development  process.  As  for  many  other  software  systems,  it  is  also  true 
for  the  development  of  WBT  courses:  To  realize  a user  and  task  adequate  human-computer-system,  developers 
have  to  take  into  consideration  all  system  components:  user,  task  and  situation  in  order  to  form  a conceptual 
model  that  is  as  adequate  as  possible  [1].  The  conceptual  model  of  the  developers  shape  the  software  product,  in 
our  case  the  WBT  course,  and  therefore,  will  decide  how  usable  and  user-friendly  the  WBT  course  will  be  and 
how  adequately  it  will  support  learning  processes. 

Figure  1 shows  the  different  models  we  have  to  deal  with  in  the  context  of  WBT  course  production.  First  of  all 
there  is  the  developer’s  mental  model  the  design  process  starts  with.  In  the  case  of  WBT  course  development  we 
have  to  deal  with  diverse  developers.  In  our  project  there  is  the  conceptual  models  of  the  content  author,  of  the 
designer,  of  the  HT/MM  producer  and  of  the  quality  manager.  Furthermore,  there  is  the  learner’s  mental  model 
about  the  application  domain,  how  the  WBT  course  is  implemented  and  how  it  supports  learning  processes.  And 
last  but  not  least,  there  is  the  system’s  model,  which  contains  the  way  the  application  domain  is  implemented 
and  sometimes  information  on  the  user’s  interaction  with  the  system.  Each  model  represents  the  complex  domain 
of  interaction  with  WBT  courses  from  a certain  perspective  and  in  a specific  way. 
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Figure  1 : Models  in  the  context  of  WBT  course  production 


Mismatches  between  these  models  lead  to  usability  problems  of  the  software  product.  Above  all,  the  imponance 
of  the  developer’s  conceptual  model  for  shaping  the  software  product  has  been  noticed  for  other  kinds  of 
software  applications  for  a long  time  [2],  [3].  According  to  our  view,  this  holds  true  for  WBT  courses  as  well. 

CONCEPTUAL  MODELS  OF  DEVELOPERS 

A fundamental  problem  for  the  development  process  is,  that  the  developers  need  substantial  and  relevant 
information  about  the  user,  the  task  and  the  situation  of  use.  They  need  this  information  from  the  very  beginning 
of  the  design  process  in  order  to  specify  the  requirements  and  shape  the  WBT  course  adequately.  However,  in 
practice  content  authors  start  course  production  in  the  traditional  way  and  produce  a linear  document  that 
includes  their  conceptual  model  about  domain  knowledge  and  about  teaching  strategies.  This  production 
methods  corresponds  to  the  production  of  traditional  lectures  for  classroom  teaching.  These  courses  will  not 
match  the  situation  of  use  in  the  virtual  university  of  the  future.  Lectures  on  demand  have  to  serve  the  needs  of 
different  user  groups  and  different  situations  of  use.  The  attributes  for  users,  the  task  and  the  situation  of  use  will 
differ  [4]  and  should  have  an  adequate  representation  in  the  conceptual  model  of  all  authors.  A severe  drawback 
with  this  situation  is,  that  it  is  impossible  for  a content  author  (and  also  for  the  other  developers)  to  take  into 
consideration  all  these  attributes  at  one  time. 

In  some  way,  of  course,  the  author  is  aware  of  the  situations  of  use  and  the  complexity  involved.  This  might  be 
the  reason,  why  the  conceptual  model  about  the  user  group  and  the  assigned  attributes  differ  from  phase  to  phase 
of  the  development  process.  This  is  a problem  as  well.  If  the  conceptual  model  of  the  developers  includes  a 
prospective  mental  model  of  the  user  and  if  the  design  is  based  on  this  model,  then  it  will  be  clear  that  varying 
the  conceptual  model  during  the  development  process  will  result  in  an  inconsistent  WBT  course,  which  will  be 
hard  to  understand  and  will  not  support  learning  processes. 

FUTURE  WORK 

The  conceptual  models  of  the  diverse  developers  shape  the  WBT  course  and  determines  the  way  the  educational 
system  is  presented  to  the  user  and  determines  if  it  supports  learning  processes.  Therefore,  it  is  important  for  the 
developers  to  have  sufficient  and  appropriate  knowledge  about  the  user,  the  task  and  the  situation  and  it  is  also 
important  that  the  conceptual  models  of  the  diverse  authors  can  be  communicated  and  complement  each  other 
during  the  design  process. 

With  this  background  we  are  working  on  methods  to  adequately  represent  conceptual  knowledge  for  the  design 
process  [5].  We  also  want  to  offer  ways  to  communicate  relevant  knowledge  components  between  all  developers 
through  the  whole  development  process. 
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Abstract:  The  goal  of  this  work  was  to  develop  a set  of  recommendations  for  the  use  of 
videoconferencing  in  large  classrooms.  The  students’  acceptance  of  synchronous 
videoconferencing,  the  role  of  the  instructors  as  well  as  the  influence  of  videoconferencing  on 
motivation  and  interaaivity  were  in  the  focus  of  this  study. 

Based  on  the  statements  of  1 78  evaluated  students  there  are  three  main  recommendations  that 
can  be  made:  (1)  The  didactic  approach  of  the  course  must  concentrate  on  an  more  active 
involvement  of  the  participants.  (2)  The  instructor  must  be  willing  to  adapt  his  teaching  style  to 
the  integration  of  new  media.  (3)  Technic  has  to  be  so  perfect,  that  it  guarantees  the  first  two 
aspects:  an  undisturbed  transmission  with  possibilities  to  interact  for  all  participants  and 
support  of  the  instructor  to  make  teaching  easy. 


Introduction 

A lot  of  studies  report  about  the  benefits  of  videoconferencing  in  the  context  of  small  classes  with  active  student 
participation.  Videoconferencing  in  small  classes  has  been  shown  to  increase  student  motivation  as  well  as 
interaction  among  students  and  lecturers  often  leading  to  a redefinition  of  teaching  (^ong  others  Abotte,  Dallat, 

Livingston,  & Robinson,  1994;  Bramble  & Martin,  1995).  Experiences  with  videoconferencing  in  the  large  { 

classroom  context  however  differ  at  least  in  part  in  some  of  these  results.  Increased  i^ibition  to  speak  in  front  of 

others  and  lowered  spontaneity  seemed  to  be  real  problems  (Knox,  1997).  Further,'  an  improved  motivation  or 

satisfaction  was  not  found  (Fillion,  Limayen,  & Bouchard,  1999).  The  purpose  of  this  study  was  to  find  out, 

which  aspects  are  the  most  crucial  for  the  use  of  videoconferencing  in  large  classrooihs. 


Method 

Equipement 

The  videoconferencing  facility  used  in  this  evaluation  is  a synchronous  system  based  on  ATM.  Its  theoretical 
bandwidth  of  up  to  155  Mbits/s  allows  the  simultaneous  transmission  of  three  audio-  and  video-streams  with 
high  quality  and  perfect  audio- video  synchronisation  between  two  and  more  connected  sites.  The  system 
provides  full  interaction  possibilities  to  all  participants  at  all  connected  sites. 

The  setting  in  the  examined  lecture  is  a two-point  configuration  consisting  of  the  following  components: 

Remote  classroom:  Three  screens,  projected  by  beamers,  with  1 .)  the  instructor,  2.)  the  PowerPoint  slides  and  3.) 
the  other  class. 

Local  classroom:  Two  screens,  projected  by  beamers,  with  1 .)  the  PowerPoint  slides  and  2.)  the  remote  class.  For 
further  technical  details  see  (Walter  & Hanni,  1998). 

Participants 

The  examined  sample  consisted  of  totally  178  participants  of  a two  year  course  in  pharmaceutical  chemistry  (86 
students  in  the  locat  classroom;  92  students  in  the  remote  classroom).  Gender  representation  was  not  equal  (139 
female  and  39  male). 

Control  data  (normal  course  without  videoconference)  were  collected  from  students  of  both  locations  during 
summer  1999.  The  experimental  data  resulted  from  the  first  half  (13  weeks)  of  the  academic  year  1999/2000. 

The  constructs  and  its  components  consisted  of  items  which  were  measured  using  a five-point  Likert-type  scale 
(l=strongly  disagree  to  5=strongly  agree). 
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Results  and  discussion 


The  following  presented  results  were  based  on  a minimum  level  of  significance  of  p<0.001. 

Acceptance  and  Motivation 

As  expected,  students  in  the  local  classroom  accepted  with  M=3.3  the  videoconference  better  than  remote 
students  (M-2.7).  Furthermore  there  was  a significant  difference  in  the  acceptance  which  is  correlated  with  the 
infrastructure:  While  in  location  1 the  auditorium  used  for  the  transmission  consisted  of  a permanent  equipped 
room,  location  2 had  to  build  up  all  equipment  for  the  lecture  every  week. 

In  contrast  to  studies  who  reported  about  increased  motivation,  the  measured  motivation  construct  (consisting  of 
subject  interest  and  self-efficacy)  remained  the  same  with  versus  without  videoconference.  Nevertheless,  with 
r=0.57,  motivation  was  one  of  the  strongest  predictor  of  the  acceptance. 

Influence  of  the  quality  of  the  transmission 

The  participants  rated  the  quality  of  audio  (M-3.9),  the  level  of  synchronisation  (M==3.7)  as  well  as  the  quality  of 
the  transmitted  videos  (picture  of  the  lecturer  (M-3.8)  and  projection  of  the  slides  (M=4.1))  more  than 
satisfying.  Only  the  video  of  the  remote  auditorium  was  assessed  average  (M-2.9).  The  quality  of  transmission 
seems  to  be  so  good,  that  there  were  no  correlations  greater  than  r=.34  with  the  acceptance  or  the  motivation. 

Role  of  interactivity 

The  largest  impact  on  the  acceptance  of  a videoconference-classroom  exerted  the  possibilities  of  interaction. 
Students  assessed  this  kind  of  lecture  in  comparison  with  a normal  classroom  less  stimulating  for  discussions 
between  them  and  the  lecturer  (M-2.5).  This  variable  had  with  r=.6 1 thew  strongest  correlation  with  acceptance. 
The  hypothesis,  that  students  were  more  inhibited  to  ask  questions  during  the  class,  can  be  accepted.  However, 
the  level  of  inhibition  was  lower  than  expected  (M-3.5).  With  r=-.27  (p<0.002)  the  correlation  between 
inhibition  and  acceptance  never  reached  the  expected  level. 

The  possibilities  of  spontaneous  intervention  were  even  rated  slightly  higher  than  in  the  traditional  classroom 
setting  (M==3.2)  and  did  not  coincide  with  the  students'  actual  hesitance.  This  high  rating  is  most  likely  due  to  the 
lecturers'  efforts  to  integrate  prepared  questions  in  their  slides.  Its  correlation  of  r=.43  with  the  acceptance 
underlined  the  importance  of  the  possibility  to  interrupt  the  speaker  at  any  time. 

Role  of  the  instructors 

The  correlation  of  r=.38  between  the  acceptance  of  videoconferencing  and  the  performance  rating  of  the 
lecturers  (with  M-3.6  both  were  assessed  above  average)  emphasized  the  important  role  of  the  instructors. 

It  is  encouraging  to  note,  that  the  implementation  of  videoconferencing  had  positive  side  effects  on  teaching:  The 
cooperation  of  the  two  involved  professors  lead  to  an  overall  improvement  of  the  curriculum  with  better 
structure,  more  goal  orientation  and  the  integration  of  prepared  questions. 


Recommendations 

Based  on  these  results,  the  following  guidelines  are  proposed  for  the  use  of  videoconference  systems  in  the 
lecture  context  with  groups  consisting  of  more  than  20  participants  on  each  side: 

1. )  Lecturers  have  to  consciously  plan  possibilities  to  interfere  and  to  create  an  atmosphere  that  motivates 
students  to  undertake  a more  active  part. 

2. )  The  instructor  must  be  above  average  and  be  willing  to  accomplish  an  extra  effort. 

3. )  The  technical  infrastructure  must  guarantee  a clear  transmission  of  video  and  audio  as  well  as  the  possibility 
of  interaction  at  anytime  for  all  participants. 
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Abstract:  This  study  is  intended  to  understand  collaborative  learning  in  a context  that  enables 
communication  by  computer  text  messaging.  Particularly,  the  study  seeks  to  determine  the  level  of 
collaboration  in  a classroom  that  uses  computer-mediated  communication  (CMC),  as  well  as  to 
examine  the  relationship  between  the  language  functions  associated  with  CMC  generated  messages 
and  the  collaborative  learning  process.  In  this  study,  "language  functions  refer  to  ways  we  can  use 
language  to  achieve  a communicative  purpose"  (Olsen,  1 992).  It  is  what  people  "can  do  with 
language"  (McDonell,  1992),  or  "what  people  want  to  do  with  the  language"  (Finocchiaro  & 

Brumfit,  1983,  p.l3),  such  as:  requesting  information,  requesting  confirmation,  making  suggestion, 
apologizing,  etc. 

The  study  is  prompted  by  following  questions 

1.  What  are  the  levels  of  collaboration  (in  terms  of  interactivity)  in  the  classroom  that  uses  CMC? 

2.  Are  there  any  patterns  in  people  using  of  language  functions  in  relation  to  the  degree  of  collaboration  (in 
terms  of  interactivity)  in  the  context  of  CMC?  If  yes,  what  are  the  patterns? 

a) .  Are  there  any  patterns  in  people  using  "requesting  information"  in  relation  to  the  degree  of  their 
collaboration  (in  terms  of  interactivity)? 

b) .  Are  there  any  patterns  in  people  using  "requesting  confirmation"  in  relation  to  the  degree  of  their 
collaboration  (in  terms  of  interactivity)? 

Computer-mediated  communication  (CMC),  is  an  effective  electronic  means  of  connecting  geographically  dispersed 
learners  using  computers,  in  order  that  they  may  collectively  participate  in  a learning  experience  where  the 
interaction  is  mediated.  Despite  it  relatively  short  history  as  a distance  medium,  a great  deal  of  interest  has  been 
shown  in  this  technology. 

Courses  offered  by  CMC  are  gaining  favor  in  schools  and  institutions  of  higher  education.  Students  can 
interact  at  a time  and  in  a place  that  is  personally  convenient,  discuss  issues  as  fully  as  desired  without  constraints 
(Henri,  1995).  CMC  promotes  interactive  activities  such  as  active  dialogue  and  sharing  of  information  and 
knowledge,  which  help  develop  a sense  of  community  among  participants  (Burge,  1993),  The  CMC  can  be  used  as 
a powerful  tool  for  group  communication  and  collaborative  learning  (Crook,  1995),  The  collaborative  learning 
environment  promotes  active  mental  engagement  in  the  experience  (Grabowski,  1990), 

Despite  the  large  body  of  literature  on  peer  learning  and  teaching,  there  are  few  acknowledgments  of  how  well  CMC 
may  support  collaborative  learning.  Although  cognitive  and  social  psychology  have  already  provided  much 
information  about  how  students  communicate  and  its  relationship  with  collaborative  learning,  the  context  has 
usually  been  the  traditional  face-to-face  setting  with  its  paralinguistic  cues  to  facilitate  communication.  What  we  do 
not  yet  know  in  published  detail  are  how  people  learn  collaboratively  and  how  this  collaborative  learning  relates  to 
the  communication  in  the  CMC  context.  In  addition,  O'Malley  (1995)  calls  for  studies  of  collaborative  learning  that 
"focus  more  on  the  processes  involved  in  successful  peer  interaction,  rather  than  just  on  learning  outcomes". 

Therefore,  this  study  seeks  to  understand  how  people  learn  collaboratively  and  how  their  collaborative 
learning  relates  to  the  communication  in  the  context  of  CMC.  By  asking  these  research  questions,  I assume  that  if 
we,  as  educators,  know  more  about  the  process  of  collaborative  learning  in  a computer-mediated-communication 
environment,  we  have  a better  chance  of  developing  practical  guidelines  for  facilitating  learning.  In  addition,  the 
information  obtained  from  the  study  may  contribute  to  the  present  understanding  of  students  learning  in  CMC 
environment  and  may  suggest  future  studies  in  the  area. 

For  instance,  numerous  studies  (Johnson  & Johnson,  1989)  dealing  with  collaborative  learning  have 
suggested  that  students  need  appropriate  social  skills  in  order  to  be  successful  in  their  collaborative  learning. 
Johnson  and  Johnson  (1989)  emphasize  that  collaboration  is  the  most  important  and  basic  form  of  human 
interaction,  and  the  skills  of  collaborating  successfully  are  the  most  important  skills  anyone  needs  to  master,  Coelho 
(1992)  explains  that  good  social  skills  can  promote  collaborative  learning  and  some  negative  behaviors  can  shut  off 
future  collaboration.  In  addition,  the  literature  on  collaborative  learning  emphasizes  the  need  to  teach  social  skills. 
Students  need  assistance  in  acquiring  those  social  skills  and  many  of  these  social  skills  can  be  regarded  as 
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communication  skills.  For  example,  Olsen  and  Kagan  (1992)  state  that  ’’social  skills  include  ways  students  interact 
with  each  other  to  achieve  activity  or  task  objectives  (e.g.,  asking  and  explaining)  and  ways  students  interact  as 
teammates  (e.g.,  praising  and  recognizing)”. 

However,  all  these  conclusions  are  based  on  the  works  that  have  been  done  in  traditional  face-to-face 
settings.  Are  the  conclusions  still  true  in  the  CMC  environment?  This  remains  to  be  answered  and  it  raises 
important  questions  in  the  field  of  collaborative  learning.  This  study  will  try  to  understand  how  students  go  about 
communicating  and  learning  in  the  CMC  environment.  Once  we  have  this  understanding,  we  might  then  compare 
the  CMC  learning  strategies  with  those  reported  for  face-to-face  classroom  contexts  in  order  to  search  for  new 
practices  to  facilitate  collaborative  learning  and  better  suited  to  the  CMC  context.  And  only  when  we  have  a better 
understanding  of  computer  mediated  learning,  we  can  make  a better  use  of  CMC. 

The  subject  of  this  study  will  be  a grade  5-6  classroom  in  an  inner  elementary  school  in  Toronto. 
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Introduction  and  Overview 

Two  distinct  fields  of  educational  activity  should  inform  instructional  design  of  courses  using  teaching 
technologies.  These  are  cognitive  theory  and  instructional  theory.  Cognitive  psychologists  tend  to  analyze 
cognitive  structures  and  processes  at  a conceptual  level  without  due  regard  to  how  these  may  be  implemented  in 
the  instructional  setting.  Conversely,  content  designers  and  instructors  do  not  often  recognize  the  value  of 
undertaking  the  analysis  of  content  and  teaching  practices  from  a cognitive  analytic  perspective  (Reusser,  1993). 
We  propose  that  one  means  to  combine  the  best  practices  of  both  fields  in  teaching  and  learning  is  to  ensure  that 
the  instructional  design  process  integrates  both  perspectives.  Three  principles  inform  the  proposed  framework  of 
design. 

• Selected  models  of  instruction  that  best  support  the  teacher’s  purpose  for  mastery  of  content,  and  the 
selection  of  enabling  technologies  that  mirror  the  selected  model  of  instruction  are  combined  in  the  design 
process. 

• Models  of  instruction  and  enabling  technologies  are  selected  according  to  the  nature  of  the  content  to  be 
learned,  the  teaching  styles  and  preferences  of  instructors,  and  the  learners  concerned. 

• Promoting  and  supporting  development  of  the  active  nature  of  the  learner  underscores  the  entire  design 
process. 

The  development  of  this  framework  took  place  through  a recursive  process  of  experimentation  and  teaching  in 
two  different  university  courses.  The  campus  course  uses  concept-mapping  software  as  a teaching  aid  and  as  a 
study  aid  for  students  needing  to  master  the  more  difficult  and  abstract  content  areas  in  the  study  of  German 
Thought  and  Culture.  The  software  used  was  chosen  for  its  ability  to  mirror  the  ‘meaningful  learning’  model  of 
instruction  developed  by  (Ausubel,  Novak  & Hanesian,  1978)  and  (Novak,  1998).  The  online  course. 
Introduction  to  Academic  Writing,  uses  a Web  site  as  the  primary  teaching  tool  and  its  design  mirrors  the  ‘expert 
instructional  scaffolding’  model  of  learning  developed  by  (Hildyard,  1 996).  This  model  supports  the  need  to 
ladder  content  into  instructor  supported  incremental  steps  that  allows  modeling  of  best  writing  practices  through 
interactive  feedback  processes. 


Using  Visual  Organizers  and  Meaningful  Learning  to  Teach  Cultural  History 

Students  unfamiliar  with  cultural  history  are  frequently  presented  with  a huge  number  of  facts  and  dates  that 
have  little  significance  to  them,  encouraging  - if  not  forcing  - them  to  acquire  as  much  as  they  can  through  rote 
learning.  The  nature  of  the  assignments  and  examinations  often  further  reinforces  rote  learning  and  tends  to 
discourage  the  synthesis  and  creative  application  of  the  material.  The  use  of  visual  organizers  coupled  with 
instructional  design  based  on  a model  of  meaningful  learning  permits  the  student  to  see  how  the  information 
communicated  in  the  course  is  interrelated  internally,  and  also  how  it  is  connected  with  students’  prior 
knowledge.  Relationships  and  interrelationships  can  be  seen  and  apprehended  quickly  and  easily  when 
represented  visually. 

Concept  mapping  conveys  information  by  reducing  it  to  concepts  (perceived  regularities  in  objects  or  events) 
and  propositions  (significant  combinations  of  concepts),  which  are  then  structured  hierarchically  and  linked  by 
means  of  lines  representing  the  relationships  between  concepts  and  labeled  accordingly.  Each  concept  is  thus 
already  presented  in  a contextual  relationship  with  other  concepts.  This  device  can  be  used  either  as  a visual  aid 
in  teaching  or  more  actively  as  an  aid  to  learning.  Novak’s  strategy  of  concept  mapping  is  based  on  Ausubel’ s 
principles  of  meaningful  learning  designed  to  promote  retention  and  subsequent  related  learning.  These 
principles  include  the  following: 
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• The  learner  must  have  prior  knowledge  related  to  the  new  information  to  be  learned 

• The  new  material  must  be  relevant  to  other  information  and  must  contain  significant  concepts 

• The  learner  must  choose  to  learn  meaningfully,  rather  than  by  rote. 

We  focus  on  concept  mapping  as  the  most  viable  strategy  to  link  prior  knowledge  with  relevant  new 
conceptual  material,  using  the  CMapTools  software  package  to  support  this  strategy.  The  software  has  the 
features  needed  for  the  strategic  learning  practices  outlined  above;  it  is  downloadable  as  freeware  and  it  is  easy 
to  learn  and  use. 


Using  ‘Expert  Scaffolding’  and  the  Web  to  Teach  Academic  Writing  Skills 

The  purpose  of  this  course  is  to  provide  the  foundation  for  scholarly  writing  that  students  can  build  on  through 
further,  related  coursework  and  through  application  of  their  writing  skills  in  other  university  courses.  Because 
writing  skills  are  best  acquired  sequentially,  the  material  in  this  course  is  scaffolded  into  linked,  incremental 
units  that  build  on  each  other.  Throughout  the  course,  the  instructor  provides  expert  modeling  through  feedback 
as  evaluative  discourse.  Using  WebBoard  as  a vehicle  for  threaded  discussion,  students  interact  with  an 
experienced  writer  who  supports  them  until  skills  are  mastered  at  each  level.  Drawing  from  (Langer  and 
Appleby,  1986),  four  design  principles  for  scaffolding  were  used  to  inform  teaching  practice  and  design  of  the 
Web  site 

• Students  were  encouraged  to  assume  ownership  of  their  writing  tasks 

• The  assignments  were  designed  to  address  students’  incremental  levels  of  knowledge 

• The  assignments  were  designed  to  support  the  sequence  of  skills  acquisition  in  the  course 

• The  instructor  and  tutors  supported  students  in  a facilitative  manner 

Successful  teaching  in  a course  of  this  nature  requires  providing  ongoing  feedback  using  multi-faceted  dialogue 
with  students  through  commentary  on  writing  and  involvement  in  peer  editing  sessions.  Feedback  comes  from 
the  instructor  and  tutors  and  from  other  members  of  the  student  group.  The  course  begins  with  a shared  basis  of 
readings  ~ reading  groups  which  present  their  interpretation  of  texts  - and  continues  with  discussion  groups 
which  assist  writers  to  refine  and  elaborate  their  thoughts,  and  concludes  with  peer  revision  and  editing  groups. 
Thus  evaluation  is  embedded  in  the  design  of  the  course  and  specifically  with  the  design  of  the  assignments.  The 
assignments  and  final  project  include  interactive  exercises,  instructor  and  peer  commentary  and  editing,  and 
online  areas  for  discussing  course  content. 


Conclusion 

To  date,  we  have  successfully  demonstrated  in  two  courses  that  instructional  theory  and  enabling  technologies 
can  be  combined  in  a useful  framework  to  inform  instructional  design  of  a given  course.  The  next  step  is  to 
design  research  studies  that  address  the  impact  of  using  this  framework  on  teaching  and  student  learning. 
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Conceptual  approach 

The  TeleCabri  practices  can  be  articulated  with  the  whole  of  the  experimental  studies  led  these  ten  last  years  on 
remote  teaching  (Dessus  et  al.  97)  in  term  of  situated  learning.  The  teachers  seldom  intervening  in  the 
realization  of  the  computer-based  environments  that  they  use  to  teach  remotely,  the  purpose  of  this  research  is 
to  specify  the  cognitive  influence  (Balacheff  91)  of  the  mediation  of  these  environments  on  the  student/teacher 
interaction.  The  computer-based  didactic  systems,  with  the  constructivist  meaning  of  the  didactic  theory  of  the 
situations  (Brousseau  86),  are  its  object  of  study.  It  is  built  according  to  the  interactionist  approach  in  HCI 
(Pochon  & Grossen  97)  considering  these  systems  as  a semiotic  mediator  by  way  of  the  didactic  concept  of 
computer-based  milieu  (Chevallard  92).  A computer-based  milieu  is  regarded  as  knowledge  carrier  aimed  by 
the  learning.  We  are  particulary  interested  in  the  milieu  which  include  microworlds  or  simulators  characterized 
by  a significant  epistemological  validity  and  a capacity  to  support  a learning  guided  by  the  discovery  (Masseux 
& Michau  96).  It  can  thus  include  a computer-based  tool  identified  like  software  or/and  hardware  support  of  a 
learning.  Here,  we  center  our  study  on  the  environments  including  several  educational  applications.  This  study 
aims  at  specifying  the  strategies  of  didactic  regulation  engaged  by  the  teacher  modelled  before  in  the  theory  of 
the  didactic  situations.  These  strategies  are  analyzed  using  cognitiv  models  (Anderson  et  al.  90)  of 
action/feedback  between  a user  and  an  application  who  are  declined  for  the  student  and  the  teacher.  The  finality 
of  this  research  is  to  develop  tools  which  help  the  teacher  to  diagnose  in  real  time  the  learning  activity  of  the 
student.  It  lies  within  the  scope  of  study  of  computational  didactics  applied  to  mathematics  (Balacheff  & Kaput 
97). 


Functional  integration  of  a platform  to  teach  remotely 

The  functional  integration  of  the  platform  is  based  on  the  capacity  of  a system  to  bring  answers  to  the  specific 
constraints  of  the  students  and  teachers.  The  functionalities  of  the  platform  bring  a technological  response  to  the 
complexity  of  management  of  distributed  human  resources  and  of  which  the  availability  is  unforeseeable. 

The  student/teacher  interaction  through  PCs  articulates  three  types  of  functionalities: 

- functionalities  allowing  an  autonomous  work  of  the  student  (educational  applications) 

- functionalities  of  synchronous  collaboration  (video,  audio,  shared  applications) 

- asynchronous  functionalities  of  interaction  (email,  sending  of  files) 


Cognitive  influence  of  a multiapplication  environment  on  the  remote  student/teacher 
interaction 

With  TeleCabri,  the  student  and  the  teacher  interact  remotely  by  way  of  three  applications,  each  one  considered 
here  as  didactic  milieu  and  each  one  characterized  by  an  epistemological  validity  specific  to  the  stake  of 
learning: 

- an  oral  milieu,  supported  by  the  videocommunication, 

- a milieu  Web  on  the  geometry,  including  a base  of  exercises,  definitions  and  properties. 
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- a milieu  (CabriGeometre)  micro  world  of  direct  interaction  with  geometrical  objects  provides  as  a solving 
board  of  problem. 


To  control  the  student/environment  interaction 

The  regulation  which  the  teacher  carries  out  is  conditioned  by  the  sequential  organization  of  the  various  stages 
of  use  of  the  platform  by  the  student. 

This  sequential  organization  can  be  schematized  as  follows: 

- an  autonomous  stage  of  work  for  the  student  which  precedes  the  stage  by  videocommunication  with  the 
teacher, 

- a stage  of  videocommunication  during  which  the  student  requests  the  assistance  of  the  teacher  to  solve  his 
difficulties  by  sharing  its  applications 

The  regulation  processes  of  the  teachers  on  the  student/environment  interaction 

The  observation  underlines  the  didactic  contribution  of  the  property  multiapplication  of  the  environment.  This 
contribution  is  observed  through  the  diversity  and  the  accuracy  of  the  regulation  processes  carried  out  by  the 
teachers.  Those  try  to  exploit  the  epistemological  complementarity  of  the  three  milieu  within  the  limit  of  their 
own  functional  and  cognitive  control  of  the  environment. 

In  short,  to  control  the  student/environment  interaction,  the  teachers  proceed  according  to  two  types  of 
interaction: 

- co-operative  interactions  on  the  geometrical  objects  with  the  interface  of  CabriGeometre  when  the  teacher 
focuses  himself  on  the  diagnosis  while  the  student  concentrates  on  the  autonomous  experimental  solving  of  the 
problem, 

- collaborative  interactions  to  create,  to  modify  and  to  remove  objects  with  the  interface  of  the  micro  world  when 
the  teacher  wants  to  destabilize  a balance  of  action/feedback  of  the  student  with  the  environment. 

The  process  of  destabilizing  used  by  the  teacher  consists  in  questioning  the  student  on  the  inaccuracies  or  the 
cognitive  disjointednesses  which  arise  from  the  interaction  of  the  student  with  the  three  milieu.  The  regulation 
in  the  oral  milieu  in  term  of  formulation,  reformulation  in  the  linguistic  reference  frame  of  the  Euclidean  theory, 
takes  into  account  the  collaborative  interactions. 
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Abstract 

With  the  advent  of  online  education  the  Universities  of  the  world  are  for  the  first  time 
exposing  their  wares  in  a public  space,  the  Internet,  in  direct  competition  and  comparison 
to  similar  commercial  offerings. 

While  the  argument  for  the  University  offering  as  being  one  of  ‘accredited  academic 
quality’  is  easy  to  win  against  its  unaccredited  training  and  professional  development 
competitors  it  is  not  necessarily  as  easy  to  suggest  the  university  offering  is  always  a 
better  ‘product’. 

Universities  typically  have  little  experience  selling  their  wares  in  a commercial 
environment  where  price,  service  and  immediacy  matter.  It  then  comes  as  no  surprise  that 
when  a mature  adult  learner  is  shopping  around  universities  are  often  being  overlooked 
for  education  alternatives  more  in  tune  with  the  requirements  of  a modem  lifestyle. 

The  Total  Student  Experience  offers  a conceptual  framework  for  the  discussion,  delivery 
and  measure  of  an  online  educational  system  from  the  student’s  perspective.  It  suggests  a 
model  with  four  distinct  layers,  which  individually  and  collectively,  effect  the  learner’s 
overall  satisfaction  with  the  system.  The  four  layers  are  presentation,  function,  education 
and  administration  with  a number  of  key  variables  occurring  within  each  layer? 

This  paper  will  discuss  the  use  of  this  framework  and  the  importance  of  the  underlying 
assumption  that  the  university  ‘product’  in  the  online  environment  will  be  measured  not 
only  by  its  traditional  hallmarks  but  also  by  the  key  elements  of  the  medium  in  which  it  is 
offered  When  that  medium  is  the  Internet  these  elements  are  convenience  and  relevance, 
service  and  value,  and  the  antithesis  of  the  mass  production  era  of  education, 
personalisation. 

The  complete  version  of  this  paper  is  available  at 
http://www.nexted.com/news/papers/tse  mckev.html 
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Abstract:  This  paper  is  on  the  Telematic  Education  (TE),  which  is  not  the  mere 
use  of  the  television  set  as  a teaching  tool/aid. . It  should  be  viewed  and  managed  as 
a strategic  resource.  TE  refers  to  a comprehensive  system  of  flexible  learning.  It 
emphasises  the  use  of  technology  that  will  enhance  the  teaching  and  learning 
environment.  Nurses/lecturers  have  been  confronted  with  the  fast  development  of 
technology  for  many  years,  but  mostly  in  clinical  practice.  The  Department  of 
Nursing  Science  of  the  University  of  South  Africa  is  providing  in  the  learning  needs 
of  professional  nurses  in  certain  rural  areas  of  the  country  by  means  of  TE. 


1.  Introduction: 

Broadcast  television  has  been  a solution  to  educational  requirements 
such  as  the  widely  distributed  campuses  of  some  countries  such  as 
Australia  and  Canada,  or  even  the  widely  distributed  student 
populations  of  universities  (Laurillard.  1993:  113).  To  be  of  any 
value  to  a student  television  education  must  be  interactive  in  nature. 
This  implies  that  student  and  lecturer  must  be  able  to  communicate 
during  a lecture  that  is  being  broadcast  by  means  of  television.  This 
brings  to  mind  the  concept  of  Telematic  Education  (TE),  which  is 
not  the  mere  use  of  the  television  set  as  a teaching  tool/aid. 


2.  University  of  Pretoria  (UP) : 

No  university  is  immune  to  the  impact  of  technological  change. 
Organisations  can  benefit  tremendously  by  exploiting  technological 
change.  It  should  be  viewed  and  managed  as  a strategic  resource. 
The  deployment  of  the  Technology  Plan  of  the  UP  has  already 
borne  fhiit.  The  establishment  of  the  unit  for  Telematic  Teaching 


and  the  subsequent  virtual  campus  initiative  are  direct  benefits  of 
the  Plan. 


3.  Telematic  Education: 


Before  one  should  attempt  to  understand  what  (TE)  is,  it  is 
important  to  know  what  is  meant  by  ‘flexible  learning’.  According 
to  Dr.  Tom  Brown  of  the  Telematic  Education  Department  of  the 
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flexible  learning.  It  emphasises  the  use  of  technology  that  will 
enhance  the  teaching  and  learning  environment.  TE  is  mostly 
delivered  over  a distance.  The  dependent  learner  will  be  more 
directed  by  the  lecturer  whereas  the  autonomous  student  who  are 
capable  of  self-study  will  be  able  to  employ  the  higher 
technological  learning  facilities.  In  the  educational  setting,  all  the 
dynamics  of  the  traditional  classroom  are  preserved,  regardless  of 
the  distance  separating  the  locations”.  This  is  the  ideal  situation  to 
which  finances  and  available  sources  not  always  lend  it. 


4.  Interaction: 

The  structure  of  a typical  teaching  session  must  have  two  parts. 
During  the  first  part  of  the  lecture  the  lecturer  delivers  the  lecture, 
i.e.  passing  on  knowledge  to  the  students.  The  second  part  of  the 
lecture  can  be  in  the  form  of  a discussion/question  session  with  high 
interaction  between  the  lecturer  and  the  students  (Jameson,  Hanlon, 
Buckton,  & Hobsley,1995:l  14).  The  need  for  students  to  have  back 
up  videos/copies  of  broadcast  stems  from  the  fact  that  some 
students  might  not  be  able  to  attend  broadcast  sessions  due  to 
employer  demands,  technical  failure  or  such  like  reasons  (Jameson 
etal.  1995:45). 


5.  The  Nursing  Education  Situation: 

Nurses/lecturers  have  been  confronted  with  the  fast  development  of 
technology  for  many  years,  but  mostly  in  clinical  practice. 
Telematic  education  (TE)  is  employed  in  nursing  education  (NE) 
for  post-basic  programs.  Post-basic  programs  imply  that  people 
who  have  aheady  registered  as  professional  nurses  at  the  South 
Afncan  Nursing  Council  (SANC)  are  continuing  and  furthering 
their  skills  and  knowledge  regarding  certain  specialities  in  the 
profession.  Continued  education  for  post-basic  nursing  requires  a 


distance  education  method  that  will  serve  as  an  academic  solution  to 
professional  nurses  in  rural  areas  of  South  Africa.  The  Department 
of  Nursing  Science  of  the  University  of  South  Africa  is  providing  in 
the  learning  needs  of  professional  nurses  in  certain  rural  areas  of  the 
country.  Jack  Yensen  (1996:213),  who  calls  this  method  of  teaching 
‘telenursing’,  is  of  the  opinion  that  the  two  key  dimensions  are 
distance  and  electronic  mediation. 


6.  The  Lecturer: 

Lecturers  who  teach  by  means  of  telematic  education  must  be 
skilled,  self-confident,  understand  the  uniqueness  of  the  rural 
students  and  should  be  flexible.  They  should  keep  the  concepts  of 
adult  learning  in  mind  and  respect  the  authority  of  experience  that 
many  students  bring  to  the  classroom.  Lecturers  who  have  not 
taught  by  means  of  this  method  need  sufficient  training.  They  do 
not  need  technical  courses,  but  training  on  how  to  adjust  their 
method  of  teaching  as  well  as  teaching  aids  to  that  of  the  telematic 
medium  (Yeaworth.  1996:150).  The  use  technology  influences 
educational  innovation. 


7.  Closing  Remarks: 

The  twenty-first  century  will  see  many  changes,  the  first  of  which 
will  be  the  end  of  extensive  emphasis  on  classroom  teaching. 
Teachers,  students,  and  preceptors  will  come  together  in  dialogues 
to  question,  to  demonstrate,  and  to  participate  actively  in  the 
learning  process.  Nursing  will  hopefully  develop  a ‘world  view’. 
“Remember,  the  information  revolution  will  continue  to  roll  on  - 
with  or  without  you.  So,  get  on  board  and  begin  changing” 
(Simpson.  1997:27). 
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Abstract:  The  greater  prevalence  of  computer  technology  in  the  daily  lives  of  middle  school  students  is 
impacting  their  learning  envrionment.  Students  of  both  genders  and  different  levels  of  socio-economic 
status  are  well  on  their  way  to  becoming  part  of  the  digital  culture.  This  paper  suggests  that  the  gaps  that 
once  existed  with  regard  to  computer  access,  use,  and  perceived  expertise  are  narrowing  significantly. 
These  findings  are  based  upon  survey  data  from  512  middle  school  students  in  three  areas:  (a)  self- 
perception of  computer  skills  and  their  acquisition;  (b)  exposure  to  technology  at  home  and  at  school; 
and  (c)  media  style  and  content  preferences. 


Introduction 

Media  and  digital  technology  are  pervasive  in  our  lives.  The  advent  of  technology  into  the  lives  and  culture 
of  adolescents  is  not  one  of  choice  but  of  certainty.  If  we  understand  the  influences  at  work  among  our  youth,  then 
perhaps  we  can  address  their  educational  needs  in  ways  that  maximize  what  we  already  know  about  how  to  structure 
effective  content.  Are  there  any  signs  of  technology  acculturation  at  work?  Have  computer  presence  and  increased 
proficiency  translated  into  any  appreciable  closing  of  the  previously  documented  digital  divide  with  regard  to  gender 
and  socio-economic  variables  among  middle  school  students? 

The  Study 

Based  on  a series  of  small  focus  groups  with  middle  school  students,  the  authors  compiled  a 68-item 
questionnaire,  both  closed  and  open  answer,  to  cover  students’  preferences  and  practice  using  different  types  of 
media.  Students  answered  the  paper  and  pencil  questionnaire  during  either  a science  or  computer  technology  class 
period.  Typically,  the  questionnaire  took  30  minutes  to  complete. 

Between  October  1998  and  April  1999,  568  middle  school  students  were  surveyed  from  eight  different 
metropolitan-area  public  and  private  middle  schools  covering  four  different  school  districts.  The  students  ranged  in 
age  from  11  to  15  years  (M  = 12.59,  SD  = 0.66).  Students  who  did  not  report  essential  demographic  information 
were  dropped  from  the  analysis  for  a final  sample  of  512  students.  Of  the  final  sample,  43.4%  were  male  (n  = 222) 
and  56.6%  were  female  (n  = 290).  Schools  were  recruited  to  obtain  a diverse  student  population  representative  of 
urban  and  suburban  schools,  as  well  as  all  ranges  of  socio-economic  status  (SES).  The  final  sample  contained  31% 
high  disadvantage  students  (n=158),  33%  middle  disadvantage  students  (n=170),  and  36%  low  disadvantage 
students  (n=184).  Within  each  SES  category,  male  and  females  were  represented  nearly  equally. 

Findings 
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The  digital  divide  separating  the  confidence  levels  of  male  and  female  adolescents  with  regard  to  computer 
use  is  narrowing.  While  some  significant  differences  emerged,  the  patterns  of  computer  use  and  the  purposes  of  use 
were  remarkably  similar  for  males  and  females  in  our  sample.  As  recently  as  1994,  one  study  by  Sakomoto  indicated 
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that  among  fourth  through  sixth  grade  students  considered  ’’heavy  users"  of  computers,  the  ratio  of  boys  to  girls  was 
4 to  1 . Our  results  indicate  that  this  gap  is  closing  as  we  found  no  gender  difference  among  those  who  could  be 
identified  as  heavy  users  of  computers.  Additionally,  when  looking  at  the  use  of  computers  for  identified  school 
subjects,  we  found  similar  levels  of  self-reported  computer  use  between  males  and  females. 

In  contrast  to  previous  research,  both  males  and  females  indicate  that  games  dominate  the  home  use.  Even 
in  the  school  environment,  games  were  one  of  the  prime  uses.  Games  remain  an  important  part  of  adolescents’ 
computer  repertoire.  These  findings,  in  particular,  argue  for  ways  in  which  to  infuse  the  gaming  format  with 
sufficient  intellectual  content  to  make  the  “game”  a valuable  learning  tool.  Some  educators  have  already  seized 
upon  this  notion  and  are  producing  more  substantive  Internet  games  that  incorporate  accepted  learning  theory.  The 
theoretical  framework  of  curriculum  built  around  problem-based  or  case-based  learning  could  lend  itself  to  a gaming 
environment.  Crucial  issues  of  how  intrinsic  the  problem  is  to  the  game  scenario  and  how  well  do  learning 
objectives  translate  into  an  appealing  hypermedia  format  are  all  ones  that  would  need  to  be  explored. 


Conclusions 

While  the  content  of  educational  web  sites  targeted  for  middle  school  students  is  still  an  issue  for  debate, 
the  data  from  this  survey  point  clearly  toward  web  sites  as  an  effective  delivery  mechanism.  Access,  use,  and 
perceived  self-confidence  in  navigating  the  Internet  are  less  constrained  by  gender-related  concerns  than  they  were 
less  than  a decade  ago.  In  addition  to  the  rapid  rise  in  adolescent  female  participation  in  technology,  the  data  also 
echo  the  research  of  others  in  highlighting  the  prominence  of  all  types  of  media  in  the  life  of  adolescents. 

The  recent  Kaiser  survey  (Kaiser  Family  Foundation,  1 999)  found  that  the  average  amount  of  time  each 
day  spent  using  all  types  of  media  for  children  from  8 to  1 8 years  olds  was  6.43  hours.  The  influence  of  all  media  in 
youth’s  life  was  reinforced  by  our  survey  participants’  choices  of  “your  favorite  famous  person.”  Of  the  512 
responses,  74%  were  performers  or  musicians.  The  remaining  types  of  persons  named  could  be  categorized  as  1 8% 
athletes,  3%  historical  figures,  2%  religious  figures,  and  4%  other. 

Media  of  all  types  are  playing  a larger  role  in  children’s  lives;  however,  it  is  mostly  a solitary  experience. 
Our  data  indicate  that  children  are  largely  using  the  Internet  and  computers  at  home  free  of  supervision.  It  also  may 
not  be  the  case  that  the  computer  is  replacing  other  forms  of  media  in  adolescents’  lives,  but  rather  it  is  being 
“added”  to  the  existing  repertoire  among  those  who  are  already  “heavy  users”  of  media. 

One  of  the  challenges  before  us  is  the  creation  of  sufficient  quantity  and  quality  of  digital  materials  to 
attract  and  retain  adolescents’  interest  in  an  unsupervised  environment.  The  materials  must  be  free  of  gender-bias 
and  bridge  the  gulf  that  currently  exists  between  how  kids  learn  and  communicate  on  their  own  and  how  they  are 
taught  in  school.  Can  the  production  of  high-quality,  educationally-grounded,  and  content  significant  materials 
attract  adolescents  to  the  world  of  learning  via  the  Web? 
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Abstract 

This  evaluation  project  examines  the  required  elements  for  a standardized  instrument  for  the  evaluation  of 
educational  technology.  The  project  will  survey  current  users  of  educational  technology  and  seek  common  threads 
and  patterns  in  a successful  evaluation.  This  study  will  attempt  to  answer  this  question:  What  should  we  be 
collecting,  measuring  and  examining  when  evaluating  technology  in  an  educational  setting?  This  project  is  the  first 
step  in  a multi-stage,  two-institution  evaluation  study  that  will  develop  a standardized  evaluation  instrument  for  use 
in  the  evaluation  of  education  technology.  The  project  will  enlist  the  assistance  of  two  post  secondary  faculties, 
clients,  developers  and  students.  Each  group  will  be  surveyed  using  an  online  form.  A representative  sample  from 
each  group  will  be  interviewed  in  a focus  group  setting.  Expected  outcomes  of  this  first  stage  include  the  collection 
of  responses  from  users,  developers  and  clients  of  educational  technology.  The  survey  responses  (n=81)  have 
provided  a unique  opportunity  to  share  and  compare  educational  strategies  and  concepts  between  post  secondary 
institution  faculty,  users  and  developers.  Common  patterns  were  evident  within  and  between  groups  relative  to 
concepts  of  educational  technology  evaluation  and  important  aspects  of  instructional  design. 

Introduction 

This  evaluation  project  will  examine  the  required  elements  for  a standardized  evaluation  instrument  for  educational 
technology  tools  and  applications.  The  project  will  survey  current  users  of  educational  technology  and  seek  common 
threads  and  patterns  amongst  user  groups  for  the  design  of  a successful  evaluation  instrument.  The  goal  of  this 
project  is  to  determine  what  users  and  developers  deem  important,  valuable  and  relevant  with  respect  to  instructional 
design  and  the  evaluation  of  technology  in  the  post  secondary  arena.  The  survey  data  collected  will  be  used  to 
create  a standardized  evaluation  tool.  This  evaluation  tool  will  assist  developers,  instructors,  teachers,  and  learners  in 
the  design,  construction  and  use  of  an  instructional  design  model  and  template  suitable  for  use  in  the  post  secondary 
field  of  education. 


Project  Rational  and  Purpose 

First,  the  research  in  this  area  has  focused  on  the  application  of  technology.  Evaluation  studies  on  educational 
technologies  have  traditionally  focused  on  the  particular  technology  tool  and  it's  impact  on  the  learning 
environment.  While  useful  in  producing  data  for  that  tool  or  application,  generalizability  to  other  technologies  or 
application  is  not  possible  due  to  the  design  of  the  instrument.  Typically  these  instruments  survey  specific  program 
issues  and  fail  to  consider  the  larger  pedagogical  issues  of  technology  (this  link  provides  examples  of  about  50 
instruments  of  evaluation  used  in  educational  technology)  *.  Such  programs  as  the  Annerberg's  Flashlight  Project^ 
and  McMaster  University's  EVNet^  have  been  leaders  in  educational  technology  project,  but  have  failed  to  develop  a 
standardized  instrument,  useful  in  evaluating  all  types  of  projects,  across  multiple  users  in  various  circumstances. 

Secondly,  this>  project  is  the  first  step  in  a multi  stage;  two-institution  evaluation  study  dedicated  to  the  development, 
testing  and  dissemination  of  a standardized  instrument  for  educational  technology  evaluation.  The  first  stage  is  a 
necessary  step  in  determining  what  should  be  evaluated  when  examining  technology  in  the  education  field  of  study. 
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Several  studies  have  identified  survey  instruments  and  other  collection  tools  for  use  in  educational  technology"^ 
While  useful  in  assessing  individual  programs  for  a specific  outcome,  these  types  of  instrument  are  difficult  to 
generalize  to  educational  technology,  regardless  of  platforms,  users  or  programs.  Determining  what  should  be 
evaluated  in  a format  that  can  transferred  across  technologies  and  application  is  a critical  next  step  in  valuing 
educational  technology's'  contribution  to  learning. 

Conclusions 

The  first  step  of  this  multi-stage  evaluation  project  produced  an  extensive  database  of  information  on  what  users, 
developers,  clients,  and  educators  think  about  educational  technology  and  its  application  to  learning.  The  collection 
of  data  ranges  from  attitudes  towards  technology  to  perceptions  of  technology  and  its  affect  on  learning.  It  also 
provided  a resource  in  the  development  of  a standardized  tool  for  evaluation  educational  technology. 

From  this  study  sample,  it  is  clear  that  respondents  agree  on  a number  of  issues  in  terms  of  importance  and  relevance 
to  instructional  technology.  It  is  also  important  to  note  that  there  are  some  differences  between  respondent  groups 
relative  to  the  evaluation  process  for  educational  technology.  This  information  supports  previous  research  in 
educational  technology  that  has  identified  key  concepts  and  factors  of  educational  technology. 

This  first  stage  of  the  project  provided  an  opportunity  to  validate  the  survey  instrument,  by  seeking  some  responses 
to  confirm  that  the  evaluation  of  educational  technology  is  an  important  aspect  of  the  instructional  design  process. 
These  findings  can  be  used  to  development  a survey  instrument  designed  to  evaluate  the  important  constructs  of 
instructional  design  with  the  use  of  technology. 

This  first  stage  of  this  project  provided  a unique  opportunity  to  share  and  compare  educational  strategies  and 
concepts  between  two  post  secondaiy  institutions.  It  is  hoped  that  the  two  institutions  will  share  a common  ground 
with  respect  to  the  development,  application  and  deliveiy  of  educational  technology. 
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Introduction 

This  paper  investigates  how  continuous  access  to  laptop  technology  and  telecommunication  tools  can  promote  a 
student-centered,  inquiry-based,  problem-solving  teaching  practice.  The  purpose  of  the  study  is  to  deepen  our 
understanding  on  how  technology  and  Internet  tools  can  best  be  utilized  in  K-12  classrooms.  We  present  an  Internet- 
based  project  that  utilizes  laptop  computers  in  a sixth  grade  classroom. 


Research  Questions 

We  explore  how  laptop  computers  and  Internet  tools  are  used  by  a sixth  grade  teacher  who  has  extensive  experience 
with  technology  and  multiple  opportunities  for  professional  development.  The  overall  questions  that  guide  this 
research  are  as  follows:  a)  What  are  the  teacher’s  beliefs  about  teaching  and  learning?  b)  What  is  the  nature  of 
students’  assignments?  c)  What  is  the  role  of  the  teacher  and  the  student  in  the  classroom?  d)  How  are  laptops  being 
utilized  in  the  curriculum?  e)  How  is  the  Internet  being  used  in  the  classroom? 


Methodology 

This  study  began  in  Septemberl998  and  takes  place  in  urban  elementary  and  intermediate  schools  in  New  York 
City,  The  primary  data  collection  techniques  that  are  utilized  include  classroom  observations,  teacher  interviews, 
and  document  analysis.  These  activities  have  been  carried  out  during  school  visits  since  the  academic  year  of  1998- 
99,  Participants  for  this  initial  research  effort  include  a sixth  grade  teacher  and  her  students.  All  students  in  the  class 
are  minority  students;  the  teacher  is  a white  female. 

The  classroom  is  part  of  the  Laptop  Project,  which  was  initiated  by  the  school  district  in  1997.  All  students  in  the 
project  have  been  issued  a laptop  computer.  The  classroom  is  also  part  of  the  Eiffel  Project,  a major  effort  of  the 
Institute  for  Learning  Technologies  (ILT)  at  Teachers  College,  Columbia  University  to  assist  schools  and  teachers 
integrate  computers  and  telecommunication  tools  effectively  in  their  classrooms.  ILT  provides  teachers  with 
technological  and  pedagogical  support,  as  well  as  with  opportunities  for  communication  and  collaboration  with  other 
teachers  and  researchers  involved  in  the  Eiffel  Project. 


Project  Design  and  Preliminary  Results 

Mary,  the  sixth  grade  teacher  participating  in  our  experiment,  started  teaching  for  first  time  using  laptop  technology 
during  the  academic  year  of  1997-98.  That  first  experience  provided  her  with  valuable  training  on  the  use  of 
computers  in  teaching.  Mary  was  ready  to  try  new  ideas,  experiment  with  different  projects,  and  go  beyond 
traditional  instruction.  Therefore,  she  volunteered  to  participate  in  the  Reach  the  World  (RTW)  project. 

RTW  (www. reachth eWorld, org)  is  a virtual  expeditionary  venture  that  uses  the  world  circumnavigation  of  Makulu 
II,  a sailboat,  as  an  interactive  tool  for  classroom  teaching  in  the  United  States.  RTW  utilizes  the  Internet  and  other 
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telecommunication  tools  to  bridge  the  student  classroom  with  different  cultures  of  the  world.  The  crew  of  the 
Makulu  II  communicates  with  students  regularly  via  satellite  email.  Since  September  1998,  Mary  accommodated  her 
social  studies  curriculum  to  the  RTW  project.  Soon  after  she  started  participating  in  the  project,  she  became  the  most 
active  teacher.  Some  of  the  lesson  plans  she  developed  are  posted  on  the  RTW  web-site  as  examples  for  other 
teachers. 

Nature  of  Student  Assignments 

Planning  an  Itinerary.  In  this  project  students  were  asked  to  form  groups  and  conduct  research  in  order  to  identify 
possible  sites  that  the  Makulu  II  should  visit  on  their  trip  to  Singapore.  Each  group  researched  different  attractions 
and  recommended  sites  of  interest.  Students  used  travel  brochures  to  locate  tourist  attractions,  maps  to  figure  out 
distances  and  ways  of  transportation  in  Singapore,  and  historical  information  from  the  Internet  to  identify  museums 
of  interest.  When  the  final  itinerary  was  put  together  it  was  sent  out  via  email  to  the  Makulu  II  crew.  The  Makulu  II 
crew  completed  the  tour  and  sent  back  relevant  information  and  pictures  to  the  students  via  email  and  air-mail. 

Comparing  religions  of  the  world.  Students  studied  different  religions  using  information  and  videotapes  that  were 
sent  by  the  Makulu  II  crew.  In  sequence,  students  formed  groups  and  selected  one  religion  to  study  in-depth.  Final 
reports  were  presented  to  other  sixth  grade  students  in  the  school  using  PowerPoint  slides. 

The  Role  of  the  Teacher  and  the  Students 

Teaching  was  organized  around  two  types  of  pedagogies:  Collaborative  knowledge  construction  and  direct  verbal 
instruction.  Direct  instruction  did  not  represent  the  dominant  mean  of  communicating  knowledge  to  students. 
Students  were  rarely  asked  to  memorize  facts.  They  were  instead  asked  to  locate  facts  using  CD-ROMs,  books,  the 
Internet,  and  other  resources.  The  information  was  then  analyzed  in  order  to  complete  projects  and  solve  problems. 

The  teacher  was  mostly  a facilitator  of  learning  and  a guide  on  the  site  ”.  Students  were  given  opportunities  for 
self-directed  inquiry.  In  many  cases  they  were  asked  to  generate  their  own  questions  and  search  for  the  answers. 
Laptops  were  very  helpful  because  they  put  a lot  of  the  resources  into  the  hands  of  the  students. 

Utilization  of  Laptop  Computers  and  Telecommunication  Tools  in  the  Classroom 

Laptops  were  a vehicle  to  support  the  curriculum  and  expand  the  classroom  ’5  walls.  Students  typically  used 
application  programs  such  as  a word  processor  for  typing  and  editing  documents,  databases  for  organizing 
information,  and  graphics  packages  for  developing  multimedia  presentations.  Email  was  also  used  extensively  in  the 
classroom  for  communication  purposes.  Moreover,  all  students  used  First  Class,  an  Intranet  service  that  allowed 
them  to  communicate  with  their  classmates  and  teacher  after  school  hours. 

Laptops  were  effective  in  promoting  collaborative  work  and  problem  solving.  Very  often  students  were  required  to 
work  in  groups  and  create  projects.  In  collaborative  projects,  students  were  required  to  perform  research,  collect 
information,  analyze  it,  synthesize  it,  and  produce  a final  product. 


Conclusions 

This  paper  presented  an  effort  to  integrate  technology  in  the  classroom.  Laptop  computers  and  telecommunication 
tools  were  used  to  provide  students  with  authentic,  meaningful,  and  motivating  experiences.  Learning  became  a 
challenge  and  students  exhibited  increased  levels  of  control  and  creativity.  Successful  use  of  technology  in  the 
classroom  depends,  however,  to  a great  extend,  on  the  teacher.  Therefore,  in  any  implementation  effort,  adequate 
time  must  be  devoted  in  helping  teachers  become  comfortable  with  the  equipment  and  absorb  the  new  teaching 
principles  associated  with  the  use  of  the  new  technologies. 

We  currently  work  with  a number  of  teachers  in  the  New  York  City  area  assisting  them  with  their  technology 
implementation  efforts.  Many  teachers  use  laptop  computers  and  Internet  tools  for  instructional  purposes.  Therefore, 
we  continue  our  research  by  observing  what  is  happening  in  these  classrooms  and  analyzing  changes  in  the 
instructional  beliefs  and  practices  of  the  teachers. 
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Abstract:  Scenario  building  is  a process  that  can  be  effectively  used  to  design  a distance-learning 
program.  It  begins  with  the  identification  of  the  key  stakeholders,  who  are  typically  sludeiils  or 
participants,  course  developers,  teachers  or  facilitators,  support  staff,  and  administration.  Then,  for 
each  perspective,  the  relevant  characteristics  are  identified,  followed  by  a determination  of  what  is 
required  to  meet  the  needs  of  a person  in  that  category.  The  process  also  has  the  benefit  of  being  a 
group  activity  that  can  facilitate  team-building  and  commitment.  We  have  used  this  process  with 
multiple  organizations  and  report  on  these  case  studies. 


Introduction 

Distance  learning  programs  are  designed  in  many  ways:  some  evolve  from  individual  efforts,  some  are  designed 
using  available  technological  capabilities,  and  others,  more  recently,  are  purchased  from  or  co-bran ded  with  e- 
leaming  providers.  Scenario  building  provides  a systematic  process  for  the  group  of  people  involved  in  planning  and 
implementing  a program  to  work  as  a facilitated  team  to  design  a program.  The  process  can  be  used  for  any  type  of 
organization,  including  a university  or  business.  It  involves  a number  of  steps:  determining  who  will  be  involved  in 
developing,  delivering,  supporting,  administrating,  or  taking  courses,  and  prioritizing  these  to  order  the  next  phase. 
Starting  with  the  most  important  perspective,  usually  that  of  the  student  or  participant,  the  relevant  characteristics  of 
members  of  that  category  are  determined  by  the  group.  This  may  include  computer  and  Internet  literacy,  work 
environment,  and  time  commitments.  The  group  then  determines  what  happens  when  a representative  of  that 
category  carries  out  their  role,  i.e.,  taking,  teaching,  or  supporting  a class.  This  results  in  a list,  which  is  then 
prioritized,  of  the  needs  and  requirements  from  that  perspective  for  the  distance  learning  program.  Subsequent 
perspectives  are  usually  easier  to  consider  after  the  first,  and  the  entire  process  leads  to  requirements  analysis  and 
planning  for  the  program.  This  process  has  been  used  for  a number  of  organizations  that  are  planning  distance 
learning  projects,  including  the  University  of  Puerto  Rico. 

The  University  of  Puerto  Rico 

The  Puerto  Rican  government  funded  an  ambitious  distance-learning  program  at  The  University  of  Puerto  Rico.  To 
help  get  the  program  underway,  the  university  called  on  a team  of  EDS  consultants  to  structure  and  support  the 
program's  essential  planning  and  implementation  processes.  The  team  began  with  strategic  planning  to  determine  the 
tasks  and  responsibilities  needed  to  ensure  the  program's  on-schedule  readiness.  During  this  process,  it  became  clear 
that  the  distance-learning  concept  conjured  widely  different  images  for  members  of  the  steering  committee 
depending  on  their  knowledge  of  its  theory  and  practice.  Many  assumptions  were  based  on  the  association  of 
distance  learning  with  the  room -based  videoconferencing  that  dominated  distance  education  prior  to  the  introduction 
of  Web-based  technologies  in  the  mid-1990s.  Some  of  these  images  did  not  account  for  the  level  of  interactivity  and 
engagement  that  students  and  teachers  experience  using  state-of-the-art  distance-learning  technologies  or  failed  to 
accommodate  the  important  differences  between  learning  and  teaching  in  traditional  and  technology-mediated 
settings. 

The  team  proceeded  with  scenario  building,  an  activity  designed  to  help  the  program's  steering  committee 
understand,  envision,  and  specify  technology’s  role  in  the  new  distance-learning  offering.  Scenario  planning  began 
by  determining  the  roles  that  people  will  play  in  the  university's  distance-learning  program;  those  roles  are  student, 
professor,  teaching  assistant,  and  a number  of  support  and  administrative  positions.  The  scenario-planning  work  led 


to  each  role  being  assigned  characteristics  that  are  relevant  to  the  how  a person  in  that  role  operates  in  a distance 
learning  program  or  that  give  the  role  dimension  and  interest.  Characteristics  of  a student,  for  instance,  included  age, 
family/home  situation,  educational  background,  technology  literacy,  Spanish  and  English  literacy,  and  typing  skills. 

The  goal  was  to  agree  upon  the  role  of  technology,  the  specific  types  of  technology,  and  the  support  requirements 
for  technology  in  the  distance-learning  program.  To  achieve  this  goal,  we  developed  a prototypical  member  of  each 
role  in  order  to  envision,  as  a group,  what  a day  in  that  person's  life  would  be  like.  We  carried  this  to  the  level  of 
detail  of  which  relevant  activities  they  were  engaged  in,  where  they  carried  out  the  activities,  and  what  technological 
and  human  support  was  required.  There  were  implications  from  some  of  the  characteristics  of  a role;  to  discuss  the 
activities  of  a person  in  a day,  we  needed,  for  instance,  to  know  if  there  was  access  to  a computer  and,  if  so,  under 
what  conditions  or  with  what  possible  distractions  it  would  be  used.  If  there  was  no  access,  then  a staffed  computer 
room  must  be  available  during  reasonable  hours.  Additionally,  we  considered  the  implications  of  atypical  but 
plausible  cases  for  each  characteristic,  such  as  the  ramifications  if  a student  was  older  and  returning  after  a period 
out  of  school. 

Conclusions 

The  scenario-building  process  uncovered  many  issues,  concerns,  assumptions,  and  previously  unarticulated 
misconceptions,  which  led  to  constructive  discussion  that  culminated  in  a consensus  about  technology’s  role  in  the 
project  and  about  the  people  who  would  manage  and  support  it.  Equipped  with  the  scenarios'  insights,  we  completed 
a thorough  analysis  of  the  program’s  specific  technology  needs,  identifying  and  prioritizing  the  components  of  the 
supporting  technology.  In  subsequent  discussions,  we  continued  to  refer  to  the  prototypical  characters,  and  found 
that  they  continued  to  help  clarify  issues  since  the  group  could  look  at  that  person’s  specific  (imagined)  needs. 
Ultimately,  the  process  of  identifying  each  role’s  relevant  characteristics  and  the  hypothetical  activities  and 
interactions  of  a person  in  that  role  pointed  directly  to  the  criteria  against  which  the  entire  program  will  be 
evaluated,  ensuring  that  the  overall  effort's  success  will  be  measurable  and  more  easily  repeatable. 
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Abstract:  Collaborative  technology  projects  require 
new  types  of  relationships  and  distinctive 
management  strategies.  This  paper  outlines  three 
critical  elements  in  creating  and  continuing  such 
partnerships:  common  vision,  a reward  system,  and 
consensus. 


Technology  ventures  often  demand  collaborative  efforts  which  bring 
together  personnel,  financial  resources,  needs  and  solutions  from  diverse 
entities.  Older  models  of  cooperation  worked  within  top-down  power 
structures;  new  structures  of  collaboration  are  complex  and  require  long- 
term thinking  and  conflict  management.  Furthermore,  sustaining 
collaboration  is  critical  because  the  payoffs  of  such  efforts  are  often  long- 
term. We  identify  here  three  elements  critical  to  building  and  sustaining 
collaboration:  common  vision,  a reward  system,  and  consensus 
operations. 

Common  Vision 

Diverse  groups  who  collaborate  in  technology  ventures  always 
face  semantic  difficulties,  terms  are  defined  differently  and  goals  are 
stated  in  language  only  the  originators  understand.  As  a result,  common 
definitions  of  terms  and  in-depth  discussion  about  problems,  solutions, 
and  common  issues  are  essential  (Winer  1994).  In  this  initial 
establishment  of  common  understanding,  group  members  must  adopt  a 
posture  of  tentative  knowing,  a non-judgmental  stance  which  reserves 
value  judgments  and  allows  for  continuing  conversation.  Although 
individuals  may  be  experts  in  their  domains,  they  need  to  drop  their 
"know  it  all"  perspective  when  they  work  collaboratively.  Rarely  does 
any  entity  get  all  it  wants,  but  group  members  must  be  honest  about  what 
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they  are  willing  to  put  into  a collaborative  effort  as  well  as  what  they 
wish  to  get  out  of  the  collaboration. 

A Reward  System 

Different  entities  have  different  currencies  and  rewards. 
Rewards  must  come  to  group  members  in  the  currency  valuable  to  them 
in  their  system,  be  it  promotion,  tenure,  credit  hours,  test  scores,  or 
bonuses.  Furthermore,  different  stakeholders  define  accountability 
differently,  as  staying  within  budget  or  positive  publicity  (Report  1997). 
As  specific  rewards  are  discussed,  entities  must  spell  out  to  whom  and 
how  they  are  accoimtable  and  how  that  will  affect  their  participation. 

Consensus 

In  diverse  collaborative  groups,  absolute  agreement  is  rare. 
Thus  it  is  essential  to  establish  that  the  group  will  operate  by  consensus 
and  to  establish  a process  for  achieving  consensus  and  resolving 
conflicts.  Even  when  members  do  not  agree  on  an  outcome,  if  they  have 
subscribed  to  a specified  process  which  has  been  followed  consistently, 
they  will  be  able  to  accept  the  result  (Winer  1994).  Once  established, 
conflict  resolution  needs  to  be  called  into  play  quickly  when  problems 
arise.  Difficulties  which  drag  on  are  much  harder  to  deal  with. 

Clearly  collaborative  ventures  in  technology  are  neither  simple  nor  easy. 
However,  the  power  of  successful  collaboration  is  S5mergistic. 
Collaborative  groups  which  are  able  to  sustain  their  efforts  can  achieve  a 
whole  which  is  indeed  greater  than  the  sum  of  the  parts. 
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Abstract:  This  paper  presents  a system  called  Cyberscience'  dedicated  to  distance  education. 

The  learning  interface  of  Cyberscience  includes  a component  consisting  in  activating  an 
interactive  multimedia  simulation  in  which  the  student  can  carry  out  direct  manipulation 
tasks  making  the  simulation  a virtual  laboratory  (vitual  lab  in  short).  A virtual  lab  for  genetic 
study  is  currently  realized.  As  an  intelligent  distance  learning  system,  Cyberscience  includes 
a number  of  intelligent  learning  agents  that  operates  in  the  space  of  the  virtual  lab  in  order  to 
help  students  to  achieve  their  goals.  This  paper  presents  an  overview  of  those  agents. 

Introduction 

There  is  already  a lot  that  has  been  said  about  software  agents,  especially  intelligent  agents’ 
technology.  While  this  is  a really  a current  trend  in  software  engineering.  The  concept  of  intelligent  agents  is 
at  the  same  time  simple  in  principle  and  complex  in  classification  (Jennings  and  Wooldridge,  1998;  Nwana, 
1999)  While  intelligent  agents  systems  are  widely  being  developed,  most  of  the  frameworks  available  simplify 
the  architectural  and  technical  aspects  and  provide  relatively  no  support  to  implement  agent  behavior.  The 
goal  of  this  research  is  to  provide  people  with  a strong  basic  architecture  and  shell  to  handle  the  complex  task 
of  implementing  intelligent  behavior  in  software  agents.  The  tools  provided  must  have  a fairly  generic  aspect 
so  they  can  be  applied  to  a wide  variety  of  problems  and  systems  while  still  being  able  to  adapt  sufficiently  to  a 
problem  to  provide  relevant  intelligent  features.  It  should  be  possible  to  achieve  such  capability  by  building  a 
set  of  general-purpose  agents  that  would  be  easily  customizable  and  reusable. 

A set  of  multiple  agents 

The  first  part  of  building  such  a set  of  agents  is  to  determine  a set  of  feature  that  are  desirable  and 
that  are  sufficiently  high  level  to  be  reusable.  The  current  proposal  includes  a planning  agent,  a decision 
making  agent,  a data  mining  agent,  a rule-based  reasoning  agent,  a data  extracting  agent  and  a case  based 
reasoning  agent.  These  agents  cooperate  in  the  context  of  CyberScience  in  order  to:  create  an  action  plan  for 
the  current  instructional  objective  associated  with  the  selected  course  or  theme,  according  to  the  student  model, 
select  the  relevant  resource  the  student  will  use  to  achieve  the  goal,  update  the  student  state  of  knowledge,  help 
the  student  during  the  learning  process  and  so  on.  In  the  following,  we  describe  each  of  the  agents. 

Data  Mining  Agent.  The  Data  miner  has  a high  probability  of  being  highly  useful  to  many  different 
applications.  Basically,  this  agent’s  purpose  is  to  retrieve  useful  information  from  databases  and  electronic 
documents  both  on  a local  machine,  a network  or  even  the  Internet.  Of  course  it  can  mean  only  search  for 
certain  keywords  but  it  is  also  possible  to  implement  more  complex  and  advanced  intelligent  search 
algorithms.  It  remains  to  be  decided  how  it  is  most  convenient  to  interface  with  such  an  agent.  It  is  also  to  be 
considered  if  such  an  agent  should  be  mobile  or  not  or  even  if  it  should  decide  whether  it  should  move  itself  to 
a different  host  system  to  be  able  to  accomplish  it’s  job  more  efficiently. 

Rule-Based  Reasoning  Agent.  The  rule-based  reasoning  agent  acts  mostly  as  an  expert  system.  Expert 
systems  have  proven  over  time  to  be  efficient  to  a large  number  of  tasks.  The  goal  of  this  agent  is  to  derive 
conclusions  from  a knowledge  base  consisting  of  facts  and  knowledge.  It  needs  an  engine  and  a set  of 
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deductive  rules.  It  needs  an  interface  to  specify  the  content  of  the  knowledge  base.  An  interface  also  needs  to 
be  defined  to  query  the  truth  of  undefined  facts.  The  engine  would  be  most  likely  based  on  the  Rete  algorithm 
that  is  already  well  known  and  extensively  used  and  tested. 

Case-Based  Reasoning  Agent.  The  case-based  reasoning  agent  on  the  other  hand  compares  the  different  cases 
and  makes  generalizations  out  of  them.  When  the  case-based  reasoning  agent  is  invoked  on  a particular 
situation  it  tries  to  find  already  known  cases  that  are  similar  to  the  current  one. 

Planning  Agent.  The  planning  agent  is  software  capable  of  making  plans.  The  goal  is  to  come  up  with  a 
combination  of  actions  to  be  executed  in  the  future  in  order  to  achieve  a desired  behavior.  The  interface  to  the 
planner  agent  is  a bit  more  complicated  than  previously  discussed  agents,  but  this  is  expected  considering  the 
complexity  of  planning  problems.  First  it  needs  a way  to  specify  an  initial  state  and  a goal  specifying  the 
desired  task  to  be  achieved.  It  also  needs  a set  of  specifications  of  the  actions  feasible  by  the  system.  Finally  it 
needs  a planning  strategy,  a starting  plan  and  a set  of  bound  parameters  both  in  time  and  space.  The  agent’s 
engine  would  be  most  likely  to  base  on  SIMPLAN  because  of  its  capacity  to  operate  in  highly  variable 
environments  (Nkambou  and  Kabanza,  1999).  This  agent  will  generate  a plan  of  actions  repairing  the  plan 
given  in  input  that  best  satisfies  the  goal  from  the  initial  state,  given  the  restrictions  implied  by  the  bound 
parameters. 

Decision-Making  Agent.  Finally  the  decision-making  agent  and  the  knowledge  extraction  agents  are  making 
extensive  use  of  previously  discussed  agents  in  this  paper.  They  would  probably  encapsulate  specific  instances 
of  these.  The  decision-making  agent,  as  its  name  implies,  takes  decisions  about  the  different  tasks  to  be  done 
and  how  they  must  be  accomplished  effectively.  It  would  work  by  using  the  rule-based  agent  to  generate  a 
goal  as  of  what  to  do.  A planning  agent  starts  to  search  how  to  best  achieve  the  goal  by  calculating  a new  plan 
while  case  based  reasoning  agent  tries  to  find  a relevant  plan  that  was  previously  used  in  a similar  situation. 

Knowledge  Extraction  Agent.  The  knowledge  extraction  agent  finds  relevant  information  from  different 
digital  data  sources.  Like  the  decision-making  agent,  the  knowledge  extractor  uses  a combination  of  three 
internal  agents.  It  obviously  uses  a Data  Mining  agent  to  retrieve  pieces  of  documents  or  database  information 
related  to  a subject  requested  by  an  internal  rule-based  reasoning  agent.  But  a case-based  reasoning  agent  also 
tries  to  find  similar  cases  as  where  relevant  was  found  to  help  the  data  mining  agent.  Once  it  is  retrieve  the 
information  can  then  be  used  by  the  system. 

Conclusions 

The  development  of  the  set  of  intelligent  agents  involved  in  CyberScience  will  lead  to  a shell  to  easily 
implement  intelligent  learning  agents  systems.  Most  of  the  required  features  for  each  agent  have  already  well 
known  solutions.  There  are  also  numerous  frameworks  that  already  offer  facilities  to  create  agents  and  to  give 
them  mobility  abilities.  It  is  expected  that  a higher  level  of  intelligence  in  behavior  of  the  system  would  be 
achieved  by  close  collaboration  of  these  agents.  It  has  to  be  defined  how  it  should  be  organized  to  provide  a 
generic  functionality  that  would  be  reusable.  Also,  it  must  be  established  how  would  the  specification  for  a 
custom  system  based  on  these  agents  would  take  place.  With  this  shell  the  use  of  intelligent  learning  agents 
system  is  likely  to  widespread.  But  the  first  application  it  is  used  on,  as  a test  case,  is  the  Cyberscience  virtual 
laboratory.  While  it  could  merely  be  used  as  a replacement  for  current  expert  system  to  add  them  a higher  level 
of  intelligence,  adaptability  and  interaction  level,  it  is  also  expected  they  could  be  applied  to  new  breeds  of 
problems  directed  at  a less  specialized  category  of  users.  It  can  be  easily  assumed  how  this  system  could  be 
used  to  create  assistants  or  coaches  in  a context  of  an  Web-based  intelligent  tutoring  system. 
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Abstract 

The  Open  Learning  Agency  in  British  Columbia,  Canada,  is  involved  in  a multi-year 
project  aimed  at  producing  a number  of  courses  that  are  derived  from  structured 
SGML  documents.  Presently,  fourteen  courses  are  being  delivered  on-line  in  33 
School  Districts  throughout  British  Columbia.  This  paper  describes  the  use  of 
structured  documents  for  course  development  with  a focus  on  the  benefits  and 
challenges  of  balancing  technological  and  instructional  design  issues. 


Background 


The  Open  School,  a division  of  the  Open  Learning  Agency,  is  involved  in  a multi-year  project  aimed  at 
producing  a number  of  courses  that  are  derived  from  structured  SGML  documents.  The  goal  is  to  offer 
innovative  and  current  educational  resources  for  students  and  teachers  that  are  directly  linked  to  the 
government’s  curriculum  and  provide  flexibility  through  delivery  of  alternative  formats  within  a distributed 
learning  environment  (e.g.,  for  use  in  a classroom  environment  and  also  within  an  independent  self-paced 
learning  environment).  Fourteen  Grade  1 1 and  12  courses  were  developed  within  a structured  document 
environment  using  SGML  (Standard  Generalized  Mark-Up  Language)  technology  to  meet  Open  School’s 
goal.  These  courses  are  presently  being  delivered  to  about  1200  students  registered  either  in  one  of  the  nine 
Distance  Education  Schools  or  within  one  of  the  regular  classroom-based  schools  in  British  Columbia. 

Open  School  has  traditionally  developed  large  print  -based  courses  with  some  complimentary  wrap  around 
media  such  as  text,  video,  or  audio.  Development  time  from  the  planning  stage  to  delivery  took,  on  average, 
two  years  to  complete.  Once  a course  was  delivered,  it  was  cumbersome  and  time  consuming  to  update  and 
maintain.  Courses  were  seen  as  being  out  of  date,  too  large,  too  much  work,  and  provided  primarily  a 
singular  pathway  for  students  to  learn.  Open  School  had  some  previous  experience  with  developing  courses 
for  on-line  use,  but  realized  the  need  to  change  their  course  development  processes  in  order  to  develop 
more  timely  and  flexible  content.  The  implementation  of  SGML  as  an  underlying  course  development  tool 
for  planning,  development,  production  and  delivery  was  identified  as  a means  to  an  end.  This  technology 
can  facilitate  the  following: 

• creation  of  courses  from  instructional  design  to  deliver, 

• creation  of  assessment  types  and  mechanisms, 

• identification,  association,  and  categorization  of  resources  within, 

• identifies  the  course  architecture, 

• creation  of  custom  views  of  materials  for  different  types  of  learners, 

• storage  and  management  of  parts  and  pieces  of  course  materials, 

• output  to  both  print  and  web,  as  well  as  to  other  media. 

SGML  has  roots  in  the  high  tech  and  publishing  industries  and  is  the  international  standard  for  representing 
electronic  text  in  a mark-up  language.  The  mark-up  provides  a means  for  interpreting  the  text.  SGML  is  not 
dependent  on  any  specific  hardware  or  operating  system  SGML  provides  a consistent  approach  to 
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identifying  the  components  of  a document  and  how  they  relate  to  each  other.  The  process  permits  explicit 
tagging  to  identify  the  components  of  a "course”:  lessons,  learning  outcomes,  glossary  terms,  activities, 
online  resources,  etc.  There  are  an  infinite  number  of  document  structures  that  can  be  defined  depending  on 
the  specific  requirements.  SGML  is  a descriptive  mark-up  language  rather  than  a procedural  mark-up 
language.  A descriptive  mark-up  language  uses  codes,  called  elements,  to  name  and  categorize  parts  of  a 
document.  Procedural  mark-up  language,  like  that  of  HTML,  is  often  used  to  represent  text  in  a certain 
style  like  bold.  SGML  also  includes  a DTD:  Document  Type  Definitions.  The  DTD  defines  the  "allowed" 
elements  and  specifies  the  order  in  which  the  elements  may  or  may  not  appear.  The  authoring  process  must 
produce  documents  that  conform  to  the  specifications  of  the  DTD. 


The  Marriage  of  Instructional  uesign  and  SGML:  The  Benefits 

Open  School,  as  part  of  the  Open  Learning  Agency,  has  a long  history  and  experience  with  implementing 
sound  instructional  design  practices  for  developing  learning  materials.  Fundamental  stages  of  instructional 
design  including  needs  assessment,  learner  analysis,  planning  (review),  development  (prototype,  formative 
evaluation),  production  (evaluation),  delivery,  and  summative  evaluation  - all  have  been  at  the  heart  of 
course  development  practices  for  over  twenty  years.  Various  instructional  technologies  have  been  used  over 
the  years  and  have  often  been  perceived  as  tools  whether  they  were  word  processing  software  programs, 
computer  conferencing  systems,  videos  audiotapes,  and  more  recently  the  World  Wide  Web/Intemet. 

However,  with  the  implementation  of  SGML,  instructional  design  methods  became  more  explicit  as  part  of 
the  structured  process  dictated  by  the  technology.  Open  School’s  SGML  based  courses  have  been 
developed  with  very  explicit  Document-Type-Definitions  that  lay  out  the  architecture  of  courses  from  the 
instructional  design  planning  stage  to  the  delivery  stage.  This  process  has  permitted  course  developers  to 
explicitly  examine  the  contents  of  their  courses  in  a more  detail  manner  that  was  never  before  possible.  A 
course  basically  “grows”  from  an  instructional  design  plan  (e.g.,  consisting  of  learning  profiles,  learning 
outcomes,  resources,  proposed  activities  and  assessment)  into  a fully  authored  final  product  (e.g., 
completed  modules  with  lessons  including  narrative,  instructions,  content,  activities,  and  assessment). 
Because  of  the  structured  information  approach  to  course  development,  “bits  and  pieces”  (e.g.,  granules, 
learning  objects)  of  a course  can  be  re-used  and  re-purposed  for  other  learning  situations  or  outputted  to 
different  formats  (e.g.,  print  or  the  Web).  Developing  14  courses  for  on-line  and  in  the  print  at  the  same 
time  would  not  have  been  possible  without  the  use  of  SGML  as  an  underlying  tool. 


The  Marriage  of  Instructional  Design  and  SGML:  The  Challenges 

Although  there  are  many  benefits  in  the  implementation  of  SGML  as  a course  development  framework. 
Open  School  has  also  faced  a number  of  challenges.  Here  is  a list  of  the  main  challenges: 

• Adopting  a new  technology  that  entailed  a whole  new  infrastructure  (e.g.,  with  new  hardware, 
software,  additional  staff,  extensive  training  for  internal  staff,  new  procedures  to  follow). 

• Reviewing  and  analyzing  existing  processes  and  products  in  order  to  come  up  with  consistent 
structures  and  language  for  different  course  architectures  (e.g.,  deciding  on  what  constitutes  a 
course,  a module,  a lesson,  an  activity,  etc). 

• Designing  content  to  be  used  in  different  learning  situations.  With  a database  as  a backbone, 
SGML  files  can  be  used  by  other  course  developers  to  re-use  existing  content.  There  are  issues 
around  designing  content  to  be  stand-alone  (e.g.,  learning  objects).  How  granular  can  content  be 
(e.g.,  a paragraph  or  a lesson)? 

• Ensuring  that  content  which  is  designed  in  “bits  and  pieces”  (e.g.,  learning  objects)  is  pulled 
together  into  a cohesive  manner  which  provides  an  effective  learning  environment  for  students. 
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Abstract:  There  is  increasing  international  usage  of  the  World  Wide  Web  for  course  delivery.  The 
current  South  African  education  system  is  still  sUuCtui’ed  as  a legacy  of  the  apartheid  system.  The 
South  African  government  currently  promotes  the  uses  of  technology  to  enhance  access  to 
education  and  to  foster  resource-based  learning.  The  authors  report  findings  of  a survey  of  Web- 
based  course  delivery  at  South  African  tertiary  institutions,  including  problems  being  experienced 
by  academics  in  adopting  Web-based  techniques. 

Introduction 

The  Internet  and  the  World  Wide  Web  (WWW)  are  part  of  a shift  away  from  traditional  classroom  based  course 
delivery  methods  {See  for  example  Pilgrim  & Creek,  1998,  p.  189;  Kinzie  et.al.,  1996  p.59).  The  South  African 
situation  does  not  reflect  this  trend.  South  Africa  is  characterized  by  a transforming  education  system  that  aims  to 
address  imbalances  brought  about  by  the  apartheid  era.  Disenfranchised  sectors  of  the  South  African  population 
have  historically  had  limited  access  to  higher  education  (Christie,  1986). 

Background 

The  South  African  Department  of  Education  promotes  the  use  of  technology  (including  the  World  Wide  Web)  to 
enhance  and  widen  access  to  higher  education  and  to  foster  resource-based  learning.  The  Department  of  Education  is 
actively  engaged  in  promoting  technology-enhanced  learning.  It  has  initiated  a five-year  strategic  plan,  through  the 
National  Centre  for  Educational  Technology,  to  build  an  infrastructure  to  support  technology-enhanced  learning 
(Department  of  Education,  South  Afnca).  The  current  social  and  education  systems  are  still  heavily  influenced  by 
inheritances  from  the  apartheid  political  system  before  1994.  For  example,  the  majority  of  South  Africa’s  population 
(mainly  black)  do  not  have  access  to  the  Internet  {Webcheck  National  Survey,  1999).  Potential  university  students 
have  no  experience  in  the  use  of  information  and  communication  technologies  (ICTs).  Currently,  only  0.02%  of 
South  African  schools  have  an  Internet  connection  (IDRC,  2000).  Their  ability  to  use  Web-based  education  is 
limited.  The  lack  of  telecommunications  infrastructure,  combined  with  low  levels  of  technological  literacy  of  the 
population  is  a major  obstacle  to  deployment  of  Web-based  material.. 

The  Survey 

A survey  of  academics  from  fourteen  South  African  tertiary  institutions  was  conducted  between  June  and  September 
1999.  A non-experimental  survey  method,  using  a structured  questionnaire,  was  used  to  investigate  the  following: 
The  extent  of  usage  of  specific  WWW  features  for  course  delivery.  In  particular  the  authors  assessed  to  what  extent 
current  adopters  have  used  the  WWW  to  move  away  from  traditional  teaching  methods.  Specific  features  listed  in 
the  survey  were:  Use  of  Email,  posting  of  syllabi  (dynamic  & static),  detailed  course  information  updated  regularly, 
distribution  of  lecture  notes,  links  to  external  sites,  feedback  mechanisms  to  students,  online  chat-room,  streaming 
video,  multimedia  online  lectures,  animation,  submission  of  online  homework,  testing  and  examinations.  Problems 
that  exist  at  tertiary  institutions  that  are  impeding  the  implementation  of  WWW  technology.  Specifically  the  survey 
examined  areas  of  availability  of  human  and  physical  resources,  organizational  conditions,  and  the  attitude  of 
academics  to  the  use  of  WWW  as  compared  with  traditional  classroom  based  methods. 


Findings 


1.  Current  and  intended  usage  of  the  WWW  by  South  African  academics: 

Figure  1 (see  http://isMdw.ac.za/stqff/research/web/figureLhtm)  indicates  that  current  levels  of  adoption  of  the 
thirteen  WWW  based  activities  listed  in  the  survey  fall  into  two  broad  categories.  The  first  category  reflects  a 
relatively  high  level  of  use.  The  seven  activities  in  this  category  (e.g.  Email,  syllabus  postings,  distribution  of 
lecture  notes)  are  activities  that  are  widely  used  and  are  those  that  support  traditional  styles  of  teaching.  They  do  not 
necessarily  change  the  actual  method  of  classroom-based  delivery.  A possible  reason  for  this  could  be  that  such 
activities  could  be  implemented  at  a low  cost  and  with  no  need  to  spend  a large  amount  of  time  doing  in-depth 
planning  and  design.  The  second  category  reflects  very  low  or  non-existent  levels  of  use  (six  activities).  This 
category  involves  activities  (e.g.  Chatrooms,  multimedia  online  lectures,  online  testing)  that  would  significantly 
move  away  from  traditional  teaching  methods.  These  are  activities  that  require  intensive  planning  and  design,  and 
are  also  dependent  on  a high  availability  of  a technological  infrastructure  supported  by  adequately  trained  staff. 
There  is  an  increased  level  of  usage.  Respondents  also  indicated  an  intention  to  increase  level  of  use  of  activities  that 
fall  into  the  second  category  listed  above.  This  is  a positive  indication,  since  the  activities  that  fall  into  this  category 
are  those  that  will  realise  a paradigm  shift  away  from  traditional  course  delivery  methods  to  a student-centred, 
constructivist  approach. 

2.  Comparison  of  level  of  South  African  tertiary  institutions  with  international  usage 

The  WWW  features  listed  in  the  survey  echoed  those  from  an  international  survey  that  was  conducted  by  an 
American  university  at  the  end  of  1998  (Peffers  & Bloom,  1998).  Figure  Two  (see 

http://is.udw.ac.za/staff/research/web/figure2.htm)  provides  a comparison  between  the  South  African  tertiary 
educators  surveyed  and  those  of  the  international  survey.  The  similarities  in  the  trends  are  noted  in  the  paper. 
However,  it  must  be  noted  that  this  not  imply  that  the  broad  state  of  WWW  adoption  for  course  delivery  in  South 
Africa  is  at  the  same  levels  of  usage  as  our  international  counterparts. 

3.  Existing  difficulties  in  the  implementation  of  Web-based  course  delivery. 

Staffing  problems:  The  results  of  the  survey  suggest  that  academic  staff  do  no  have  the  necessary  skills  to 

implement  Web-based  courses.  The  situation  is  further  compounded  by  the  absence  of  support  staff  needed  to  run 
Web-based  courses.  Organizational  conditions:  The  Survey  suggested  that  the  organizational  conditions  such  as 
time  and  flexibility  in  respect  of  work  commitments  do  not  allow  academics  (especially  if  they  are  novices  at 
implementation  of  Web  technologies)  to  experiment  with  WWW  technologies.  Financial  Restraints:  The  problem 
of  shrinking  financial  resources  is  affecting  service  delivery  at  all  levels  of  education  in  South  Africa.  The  survey 
indicates  that  current  budgets  do  not  allow  them  to  invest  in  Internet  technologies. 

Conclusion 

The  desire  for  on-line  teaching  at  South  African  tertiary  institutions  is  increasing.  The  survey  indicates  that 
academics  acknowledge  the  positive  role  of  the  WWW  in  course  delivery.  Key  recommendations  are:  Adopters  of 
Web  technology  in  the  classroom  need  to  be  encouraged  to  utilise  the  Web  for  more  than  merely  support  of 
traditional  lectures;  Management  at  the  faculty  and  institutional  level  should  be  encouraged  to  create  the  possibility 
for  and  invest  in  adequate  training  of  both  academic  and  support  staff.  Heads  of  academic  units  support  the 
adoption  of  Web  technologies  by  providing  incentives  to  staff.  Tertiary  institutions  should  investigate  possible 
collaboration  with  external  partners  in  the  private  sector  as  a means  of  alleviating  high  costs.  Some  of  the  issues  that 
require  further  research  and  investigation  are:  international  experiences  and  collaboration  with  counterparts; 
Development  of  guidelines  for  designing  Web-based  courses;  the  long  term  effectiveness  of  Web-based  teaching  on 
students;  the  role  of  academics  in  developing  Web-based  courses;  and  user-  friendly  authoring  tools. 
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Abstract:  This  paper  looked  at  the  variation  of  structure  and  participations  for  online  conferences. 
Four  different  structural  models  were  used  in  designing  the  conferences,  which  were  (1)  Open- 
ended  discussions  by  expert,  (2)  Scaffolded  discussion  with  facilitated  follow-up,  (3)  Scaffolded 
discussion  without  facilitated  follow-up,  and  (4)  Open-ended  discussion  without  facilitated  follow- 
up. Data  were  collected  on  frequency  of  messages,  word  count  of  messages,  and  key  word  analysis 
for  themes.  The  findings  will  provide  comparative  and  descriptive  data  related  to  conference 
structures,  participant  variables,  and  levels  of  participation  for  future  research  in  online 
conferencing. 


Use  of  Online  Conferencing  as  a Vehicle  for  Training  Special  Educators 

There  is  evidence  that  materials  designed  to  assist  teachers  in  working  with  students  with  behavioral  problems  are 
currently  in  high  demand.  The  design  and  development  of  the  Internet  as  a delivery  medium  for  such  continuous 
training  in  both  preservice  and  inservice  teacher  education  has  been  documented  in  special  education  (Smith, 
Martin,  & Lloyd,  1998).  A team  of  researchers,  funded  by  a U.S.  Department  of  Education  grant,  has  been  hosting  a 
series  of  online  conferences  for  preservice  and  inservice  educators  under  the  auspices  of  the  Virtual  Resource  Center 
in  Behavioral  Disorders  (Fitzgerald  & Semrau,  1998-2000).  These  virtual  conferences  involve  the  participation  of 
national  experts  in  the  behavioral  disorders  field  for  the  purpose  of  providing  in-depth  discussion  of  topics  and 
social  discourse  regarding  the  application  of  the  information. 

Design  and  Structure  of  the  Virtual  Conferences 

Four  different  structural  models  were  used  in  designing  the  online  conferences.  Each  conference  ran  for  two  weeks, 
but  the  degree  of  openness  and  structure,  involvement  of  the  online  expert,  and  the  use  of  discussion  facilitators 
differed  across  the  conferences. 

Conference  #1:  Open-ended  Discussion  by  Expert.  95  participants;  477  messages. 

One  expert  online  for  two-week  duration  with  all  participants  in  one  large  discussion  group;  no  required  pre- 
conference preparation  or  readings;  messages  not  organized  by  themes  or  available  via  archives. 

Conference  #2:  Scaffolded  Discussion  with  Facilitated  Follow-up.  105  participants;  283  messages. 

One  expert  online  for  first  week  with  all  participants  in  large  group;  participants  grouped  into  four  smaller 
discussion  groups  directed  by  project  facilitators  for  second  week;  required  writing  of  a case  scenario;  suggested 
readings  offered  via  web  site;  messages  organized  by  themes  and  available  on  web  archives. 


Conference  #3:  Scaffolded  Discussion  without  Facilitated  Follow-up.  37  participants;  155  messages. 

One  expert  online  for  first  week  with  all  participants  in  large  group;  participants  continued  discussion  for  second 
week  without  expert  or  facilitators;  pre-conference  self-evaluation  encouraged;  suggested  readings  offered  via  web 
site;  messages  organized  by  themes  and  available  on  web  archives. 

Conference  #4:  Open-ended  Discussion  without  Facilitated  Follow-up. 

Three  national  facilitators  for  first  week  with  all  participants  in  one  large  discussion  group;  participants  continue  for 
second  week  without  national  or  project  facilitators;  no  required  pre-conference  contributions  required;  free  CD- 
ROM  provided  prior  to  conference  containing  suggested  software  and  resources  for  viewing;  messages  organized  by 
themes  and  available  on  web  archives;  greater  mix  of  field  teachers  participating  with  preservice  teachers. 

Preliminary  Findings  & Importance  of  the  Work 

The  findings  reveal  equivalent  levels  of  participation  regardless  of  prior  computer  experience,  teaching  experience, 
access  to  equipment,  typing  skills,  learning  styles,  writing  anxiety,  or  frequency  of  e-mail  and  Internet  use.  Writing 
anxiety  may  play  a role  in  length  of  message  but  not  frequency  of  responding.  Graduate  students  demonstrate 
significantly  higher  levels  of  participation  compared  to  undergraduate  students.  Less  structured  conferences  allow 
participants  to  initiate  more  of  their  own  topics  of  concern.  Instructors  have  an  important  role  in  providing  technical 
assistance.  Instructors  and  course  requirements  impact  participation  and  authenticity  of  involvement  of  preservice 
participants.  Readings  provide  common  ground  for  discussion.  Written  scenarios  can  be  requested  from  participants, 
but  they  emerge  naturally.  Structure  and  facilitation  provide  an  even  level  of  participation  and  ongoing  discussion. 
Messages  must  be  archived  by  threads  for  asynchronous  access. 

This  research  provides  a comparison  of  online  participation  during  conferences  that  differ  in  open-ended  vs. 
scaffolded  features  and  the  use  of  facilitation.  Online  conferencing  appears  to  be  a valid  and  valued  method  of 
providing  preservice  and  inservice  training,  but  the  variables  of  designing  and  running  virtual  conferences  are  more 
of  an  art  than  a science  at  this  stage  of  research.  These  studies  will  provide  comparative,  descriptive  data  related  to 
conference  structure,  participant  variables,  and  levels  of  participation  to  guide  future  research  in  this  mode  of 
instruction. 
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Introduction 

In  medical  education,  Problem-Based  Learning  (PBL)  is  becoming  appreciated  more  and  more.  The  design  and 
implementation  of  a problem-oriented  medical  curriculum,  however,  is  very  time-consuming  for  the  educational 
staff  Case-based,  Web-based  training  systems  can  help  the  educators  to  develop,  implement,  organise  and  reuse 
well-structured  multimedia  cases.  For  students,  such  a system  can  be  used  to  improve  and  test  their  problem-solving 
competence.  With  respect  to  heterogeneous  learning  environments,  the  system  should  be  flexible  and  adaptable. 
CAMPUS  tries  to  meet  these  requirements  by  covering  the  needs  of  different  user  groups  (e.g.,  students,  educators, 
physicians)  in  different  application  scenarios  (e.g.  self-study,  presentation,  learning  groups)  to  get  maximum  benefit 
of  integrated  medical  cases.  With  the  CAMPUS  player  component  (learner’s  front-end),  users  can  work  out  medical 
cases  in  a simulative  - and  therefore  realistic  and  interactive  - way  or  let  the  system  present  a case  in  different 
manners.  Users  can  get  help  via  expert  comments  or  context-sensitive  systematic  knowledge  which  is  available  in 
addition  to  the  case  data.  With  the  CAMPUS  authoring  component,  medical  experts  can  feed  the  system  with  cases 
from  any  medical  field.  CAMPUS,  part  of  a comprehensive  virtual  university  project,  is  going  to  be  integrated  in  a 
reformed  medical  curriculum  approach  at  the  University  of  Heidelberg. 


The  idea  of  CAMPUS  is  to  develop  a case-based,  Web-based  training  system  in  Medicine  including  a broad  variety 
of  usage  modes  with  the  aim  of  minimizing  acceptance  problems  and  maximizing  the  effectiveness  and  efficiency  of 
case-data  input,  which  is  time-consuming  and  still  one  of  the  central  problems  of  case-based  systems. 

To  achieve  such  a functionality,  CAMPUS  provides  the  following  features: 

a)  Integration  of  medical  cases  from  any  medical  field 

b)  Storage  of  case  data  in  a very  detailed  manner  in  a relational  database 

c)  Presentation/treatment  of  cases  in  different  adaptable  ways 

The  degree  of  interactivity/realism  of  a case  presentation/treatment  is  optional.  The  most  interactive  form  is 
‘decision’,  where  the  student  has  to  treat  a case  in  a realistic,  simulative  way  leading  to  a maximum  of  active 
knowledge  processing.  The  user  has  to  ask  questions  of  anamnesis,  order  physical,  lab  or  technical  exams,  make 
diagnostic  decisions  and  propose  a therapy.  In  the  form  ‘compact’  the  student  only  has  to  make  diagnostic 
decisions  and  propose  the  therapy.  The  results  of  the  different  tests  are  shown  en  bloc.  If  a user  is  only 
interested  in  medical  case  data  he  can  choose  the  form  ‘total’,  where  the  whole  case  is  presented  at  once  with  no 
interactivity  at  all. 

With  these  different  forms  of  application  the  user  can  look  at  a case  the  way  he  likes  and  is  able  to.  For 
example,  if  a user  does  not  want  to  do  the  anamnesis  on  his  own  for  any  reason,  he  may  choose  the  form 
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‘compact’.  If,  on  the  other  hand,  he  wants  to  concentrate  his  learning  activities  on  the  physical  exam,  he  may 
choose  the  form  ‘decision’. 

d)  Expert  comments  on  demand 

It  is  very  important  to  explain  certain  things  about  a case,  e.g.,  why  a diagnosis  has  been  made  or  why  a 
particular  exam  has  been  done.  Hence,  for  every  particular  point  within  a case  the  CAMPUS  author  can  define 
an  expert  comment. 

e)  Questions  if  wanted 

Questions  are  - besides  other  things  - an  important  aspect  of  interactivity  and  active  knowledge  processing. 
Thus  the  case  author  can  define  questions  of  different  types  on  every  definable  point  within  a case,  e.g.,  after  or 
before  the  user  gets  the  answer  to  a certain  question  of  the  anamnesis  or  after  or  before  the  whole  physical 
exam.  The  user  can  choose  if  he  wants  to  see  only  case-dependent  questions,  case-dependent  and  case- 
indcpcndcnt  questions  or  no  questions  at  all. 

f)  Distinction  between  case  data  and  systematic  knowledge 

Every  user  has  a different  state  of  knowledge.  Therefore,  every  user  needs  help  at  different  points  when 
working  on  a case.  Hence,  CAMPUS  shows  context-sensitive  systematic  knowledge  only  on  demand.  The 
knowledge  can  be  internal  (integrated  by  CAMPUS  authors)  or  external  (using  external  knowledge  sources 
from  digital  libraries  over  the  internet). 

g)  Detailed  case  search 

Cases  are  searchable  by  defining  symptoms  or  diagnoses.  For  example,  one  could  look  for  cases  with  fever, 
shivering  and  chest  X-ray  abnormal.  Such  a specific  search  is  possible  because  of  the  detailed  database. 

h)  Execution  over  the  intemet/intranet  or  locally 

To  acquire  platform-independence,  permanent  actuality  and  world-wide  access  along  with  all  capabilities  of  a 
programming  language,  CAMPUS  is  implemented  in  Java  as  a Browser  applet.  For  users  at  home,  it  may  be 
better  to  have  a local  application  to  save  time  and  costs.  Therefore,  CAMPUS  is  also  executable  locally  using  a 
local  database. 


In  order  to  avoid  the  structure  and  vocabulary  of  cases  varying  from 
author  to  author,  a case  has  a well-defined  structure  and  has  to  use  a 
defined  vocabulary  for  anamnesis  questions,  exams,  diagnosis  (ICD-10) 
and  therapies.  Every  case  has  the  following  structure:  anamnesis, 
physical  exam,  suspected  diagnosis  followed  by  one  to  many  passes  of  a 
therapy  loop  with  examinations  (physical,  technical,  lab),  working 
diagnosis  and  therapy.  That  means  that,  unlike  many  other  case-based 
training  systems,  CAMPUS  doesn’t  stop  at  the  first  diagnosis  given  by 
the  user  but  considers  a case  in  a more  detailed  and  realistic  way. 

To  give  the  user  an  easy  to  understand  user  interface,  we  implemented  a 
so-called  situated  learning  environment,  where  the  ‘situation’  is  a 
physician’s  room. 


Conclusion 

CAMPUS  implements  a new  dimension  of  flexibility  of  a case-based,  medical  Web-based  training  system.  It  can  be 
used  by  different  user  groups  in  different  learning  environments.  Therefore,  the  effectiveness  and  efficiency  of  the 
time-consuming  case  data  input  has  been  improved.  CAMPUS  offers  the  chance  to  develop  a case-database  with 
well-structured  multimedia  cases  to  be  used  by  students  as  well  as  educational  staff  (especially  in  problem-oriented 
curricula)  who  can  use  and  - more  important  for  the  educators  - reuse  cases  in  several  scenarios. 

A first  prototype  is  available  and  is  going  to  be  integrated  in  the  reformed  medical  curriculum  at  Heidelberg  at  least. 
A comprehensive  evaluation  is  taking  place  in  June/July  2000.  An  international  version  of  CAMPUS  is  planned. 
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Abstract:  To  study  the  effects  of  asynchronous  web-based  collaboration  in  an  existing  video- 
based  distance  educaliun  piogram,  the  authors  structured  coursework  to  enable  evaluation  of 
differences  in  cognitive  and  emotional  aspects  of  learning  between  those  using  web-based 
collaboration  and  those  completing  assignments  through  traditional  independent  methods. 


Educators  have  increasingly  recognized  the  important  role  collaboration  can  play  in  learning.  Asynchronous 
web  conferencing  has  been  identified  as  a unique  tool  in  promoting  such  collaboration,  particularly  when 
applied  in  distance  education  coursework  [Knapczyk,  Chung  & Baik  2000,  Moller  1998,  Duffy  et  al.  1998, 
Bonk  & Cunningham  1998,  Ernest  1995,  McMahon,  O’Neill  & Cunningham  1992].  In  the  past  three  years  we 
have  integrated  web  conferencing  into  our  existing  video-based  distance  education  program  at  Indiana 
University  to  promote  collaboration  and  ownership  among  learners  [Rodes  et  al.  2000,  Rodes,  Knapczyk  & 
Chapman  1999,  Chung,  Rodes  & BCnapczyk  1998]. 

While  we  have  been  pleased  with  the  results  of  web  conferencing  in  our  program,  we  recognize  the  need  to 
evaluate  its  effects  in  verifiable  terms.  In  fact,  although  much  has  been  written  about  the  benefits  of 
asynchronous  web  conferencing,  few  studies  have  been  conducted  to  verify  these  findings.  The  education 
literature  typically  offers  descriptions  of  programs  and  effects,  with  little  controlled  study  or  verification. 
Furthermore,  little  has  been  written  on  the  possibilities  of  web  conferencing  when  integrated  with  other  distance 
education  technologies.  To  address  this  lack,  we  have  been  conducting  a study  in  Spring  2000  to  investigate  the 
effects  of  asynchronous  web-based  conferencing  on  both  cognitive  and  emotional  aspects  of  learning. 

The  Collaborative  Teacher  Education  Project  at  Indiana  University  offers  graduate-level  special  education 
coursework  to  cohort  groups  of  teachers  in  rural  Indiana  communities.  Classes  are  delivered  via  two-way  video 
and  are  supplemented  by  e-mail,  fax  machines,  and  web-based  conferencing  using  the  SiteScape  Forums  web 
conferencing  system  [Knapczyk  et  al.  2000,  BCnapczyk,  Rodes  & Chung  1998].  In  the  Spring  2000  semester  we 
offered  a single  course  on  behavior  management  to  in-service  teachers  at  six  sites. 

For  the  current  study  we  divided  our  course  content  into  four  units.  The  independent  variable  of  our  study  was 
the  use  of  asynchronous  web-based  conferencing.  For  each  unit,  we  developed  two  parallel  sets  of  assignments 
and  activities:  one  set  was  to  be  completed  individually  with  collaboration  taking  place  at  the  learners’ 
discretion;  the  other  set  was  to  be  completed  collaboratively  in  web-based  learner  teams.  During  each  unit, 
learners  at  three  sites  completed  their  work  using  web-based  collaboration,  while  learners  at  the  other  three  sites 
completed  their  work  using  the  “traditional”  assignments. 

To  facilitate  direct  comparison,  during  every  unit  each  web-based  site  was  paired  with  a non- web-based  site  to 
meet  for  a video-based  course  once  a week.  In  this  way  we  ensured  that  the  sites  to  be  compared  received  the 
same  instruction.  After  each  unit  the  roles  assigned  to  the  groups  were  reversed.  For  example,  learners  in  Tell 
City  were  linked  each  week  with  learners  in  Spencer  in  a video  session  originating  from  the  lU-Bloomington 
campus  where  the  instructors  were  located.  For  the  first  unit  of  the  course,  the  Tell  City  group  completed 
assignments  in  web-based  teams,  while  the  Spencer  group  completed  assignments  individually  or  using 
informal  collaboration.  For  the  second  unit  of  the  course,  these  roles  were  switched,  and  so  on.  By  pairing  sites 
in  this  way,  by  switching  the  assigned  roles  at  the  end  of  each  unit,  and  by  repeating  this  full  cycle  two  times 
during  the  semester  we  aimed  to  minimize  the  effects  of  any  differences  among  learner  characteristics  and 
environments  across  the  different  sites. 
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The  dependent  variables  we  have  been  investigating  reflect  both  cognitive  and  emotional  aspects  of  learning. 
We  are  evaluating  the  following  three  elements  of  content  learning  [Bloom  1956]: 

• acquisition— through  product  evaluation  and  content-based  tests 

• application— through  product  evaluation  and  field  reports  from  learners 

• retention— through  follow-up  tests  and  surveys  on  content  issues 

In  the  emotional  domain,  we  are  using  both  immediate  and  follow-up  surveys  to  evaluate  learner  satisfaction, 
which  is  a particularly  important  factor  to  consider  in  working  with  adult  learners  [Galbraith  1998]. 

A specific  grading  rubric  was  used  to  ensure  uniformity  of  evaluation  and  to  be  sure  products  were  assessed  for 
issues  of  applicability  as  well  as  acquisition.  To  facilitate  clear  comparison,  all  surveys,  testing,  and  product 
evaluation  have  been  linked  to  the  four  individual  units  of  the  course.  In  this  way  we  can  compare  the  results  at 
each  study  site  not  only  to  the  linked  cohort  site  using  the  opposite  approach,  but  also  to  the  results  at  that  same 
site  during  other  units,  to  track  any  differences  as  learners  moved  from  web-based  to  non-web-based  activities. 

By  completing  this  study  we  hope  to  have  gathered  reliable  data  on  the  effects  and  the  effectiveness  of  web- 
based  collaboration  when  combined  with  other  distance  education  technologies.  We  anticipate  that  our  findings 
will  contribute  to  an  understanding  of  the  importance  of  collaborative  learning  and  web-based  conferencing  in 
distance  education  programs,  and  for  education  in  general. 
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Abstract;  Edubox®  is  a system,  built  at  the  Open  University  of  The  Netherlands,  that  supports 
educational  innovation  processes  in  higher  education.  The  system  allows  maximal  flexibility  in  that 
it  enables  educational  designers  to  model  an  instructional  system  completely  after  their  ideas.  In 
this  paper  the  phases  of  the  design,  development  and  implementation  of  an  instructional  system  are 
described  with  particular  attention  to  Edubox®. 

Introduction 

Edubox®  is  a system,  built  at  the  Open  University  of  The  Netherlands,  that  supports  educational  innovation 
processes  in  higher  education.  The  system  allows  maximal  flexibility  in  that  it  enables  educational  designers  to 
model  an  instructional  system  completely  after  their  ideas,  unhindered  by  implicit  didactic  beliefs  similar  systems 
often  harbor. 

An  instructional  system  is  here  defined  as  the  assembly  of  'objects’  that  collectively  assist  a student  in  acquiring  a 
set  of  learning  objectives.  At  a more  concrete  level,  an  educational  system  features  such  items  as  people  (e.g. 
teacher,  fellow  students,  experts),  tools  (e.g.  word  processor,  browser),  knowledge  sources  (e.g.  books,  videos,  cd- 
rom's),  and  activities  (e.g.  writing  a paper,  peer-assessing  a fellow  student). 

Design  Stage 

Edubox®  is  able  to  provide  the  eductional  designer  with  such  freedom  and  flexibility  because  it  makes  use  of  a 
didactic  metamodel  embedded  in  an  Educational  Markup  Language  (EML)  - a generic  XML-based  language  - as 
described  by  Koper  (1999). 

In  principle,  everything  the  learner  or  teacher  will  experience  in  Edubox®  has  to  be  modelled  by  the  designer.  In 
contrast  with  traditional  electronic  learning  platforms  it  does  not  make  any  (implicit)  assumptions  (Onstenk  & 
Meijer,1998;  Dabbagh,  Bannan-Ritland  & Silc,  1999)  on  preferred  didactic  scenarios,  thus  providing  the  designer 
with  a didactic  ’’clean  slate". 

When  working  with  Edubox®  the  educational  designer,  needs  to  focus  his  design  activities  on  two  main  questions 
right  from  the  start: 

• What  is  the  structure  of  the  instructional  system  needed  to  support  the  acquisition  of  certain  learning  objectives 
by  students? 

• What  will  be  the  delivery  medium  of  the  designed  instructional  system? 

Decisions  on  the  Instructional  System 

The  instructional  system  to  be  developed  may  be  modelled  at  several  levels:  the  micro-level  (e.g.module),  meso- 
level  (e.g.  course),  and  macro-level  (e.g.  curriculum).  The  designer  has  to  make  up  his  mind  about  the  main  level  for 
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his  design  activities.  Subsequently,  the  designer  has  to  opt  for  a pedagogic  approach,  taking  into  account  several 
ingredients  of  his  instructional  system: 

• educational  objectives  and  prerequisites,  (aimed  at  the  acquisition  of  knowledge,  skills,  attitudes,  competences, 
complex  cognitive  skils)  and  target  group 

• relations  between  the  stated  educational  objectives,  prerequisites  and  the  structure  of  the  instructional  system 

• the  context  in  which  the  education  takes  place  (roles,  sources,  instruments,  on-the-job) 

• situational  experiences  and  domain  specific  information  in  education 

• individual  differences 

• co-operation  and  collaboration 

• communication 

• the  use  of  different  educational  materials 

• assessment 

• the  kind  of  activities  which  have  to  be  performed  to  assure  learning  processes 

• the  learning  process  and  transfer  processes  passes  off 

• stimulating  motivation 

• sequencing  actvities 

Descisions  on  the  Delivery  Medium 

The  educational  designer  has  to  make  a decision  on  how  the  educational  system  will  be  delivered  to  the  learners.  For 
example:  will  the  product  be  a book  or  will  it  be  delivered  on  the  Internet?  Depending  on  this  decision  and  the  level 
of  implementation  of  the  instructional  system,  the  designer  will  have  to  design  a style  sheet  for  the  lay-out  of  a book 
or  an  entire  graphical  user  interface  (G.U.I.). 

Development 

Developing  the  Instructional  System 

In  the  development  phase  the  whole  instructional  system  has  to  be  structured  using  the  Educational  Markup 
Language  (EML).  For  specific  pedagogical  orientations  (e.g.  problem-based  learning,  case-based  learning),  ready 
made  EML-templates  may  be  available.  EML  knows  the  following  main  elements  (Tab.l): 


element^ 

description 

metadata 

all  qeneral  data  concerninq  an  educational  obiect 

roles 

the  different  roles  persons  can  have  within  an  instructional 
activity 

objectives 

(learning)  objectives  which  are  related  to  an  instructional  system 

prerequisites 

prerequisites  (e.g.  in  terms  of  knowledge,  personal 
charactaristics,  instruments  available  to  learner)  related  to  an 
instructional  system 

content 

all  componounds  (defined  within  the  elements  'activity'  and 
'environmenti  ) which  are  conditional  for  the  acquisition  of 
(learning)  objectives 

activity 

a stimulation  of  the  learner  to  perform  certain  behavior 

environment 

componounds  (e.g.  tests,  knowledge  sources,  instruments) 
which  are  conditional  for  the  acquisition  of  (learning)  objectives 

method 

structure,  didactics  and  processes  within  an  instructional  system 

Developing  the  delivery  medium 

In  the  development  phase  the  necessary  features  for  publication  to  the  medium  chosen  during  the  design  phase 
should  be  developed. 

Implementation 

Because  of  the  richness  of  EML  and  the  flexibility  of  Edubox®,  different  designers  may  implement  similar  ideas  in 
very  different  ways. 
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Abstract:  Throughout  history,  technological  advances  have  led  to  scientific  breakthroughs. 
Computer-mediated  learning  environments  have  the  potential  to  function  similarly:  as 
technological  tests  of  educational  theories.  Here,  this  unexplored  research  role,  for  one  such 
learning  environment,  is  illuminated.  The  Generative  Virtual  Classroom,  a web-delivered 
virtual  classroom  for  teacher  education,  is  demonstrated  and  preliminary  research  findings 
with  respect  to  theory  advancement  reported.  Some  emerging  implications  for  computer- 
mediated  learning  and  for  the  generation  of  educational  knowledge  are  discussed. 


Cultural  development  is  littered  with  striking  examples  of  technological  advances  that  have  led  to  scientific 
breakthroughs.  The  successful  building  of  European  cathedrals  provoked  the  development  of  a detailed 
engineering  science  (Bronowski  1973/1992).  In  our  own  time,  pioneering  brain-imaging  technologies  have 
brought  important  new  neuroscientific  insights  (see,  for  example.  Carter  1998  and  McCrone  1999).  Similarly, 
technological  tools  for  learning,  especially  those  designed  on  particular  educational  principles,  ought  to  expose 
the  worth  of  those  principles.  In  action,  such  technologies  may  well  constitute  a potent  and,  as  yet,  unexplored 
research  tool  by  means  of  which  educational  theorising  can  be  advanced.  In  this  work-in-progress  paper,  I 
explore  this  research  role  for  one  such  technological  tool  for  learning:  the  Generative  Virtual  Classroom. 


The  Generative  Virtual  Classroom 

This  paper  begins  by  demonstrating  the  Generative  Virtual  Classroom  itself,  so  as  to  elucidate  its  educational 
principles.  Emerging  from  a series  of  classroom  studies  of  learning  and  learning  to  teach  (in  particular,  Cosgrove 
& Schaverien  1996  and  Schaverien  & Cosgrove  1997),  this  web-delivered  learning  environment  attempts 
sophisticated  computer-mediated  teacher  education  in  elementary  technology-and-science.  Its  design  embodies  a 
generative  view  of  learning  in  which  learning  is  conceived  as  a generating  and  testing  cycle  (after  Minsky  1986, 
Wittrock  1974,  1994  and  Plotkin  1994,1997).  According  to  this  view,  learners  generate  ideas  that  they  then  test 
on  their  value,  keeping  those  that  survive  their  tests  (Schaverien  & Cosgrove  1999,  2000).  In  a nested  pair  of 
virtual  classrooms  (a  virtual  elementary  one  and  a virtual  tertiary  one).  Education  students  can  develop  their 
views  of  learning  by  observing  exemplary  learning  by  young  children  from  an  archive  of  pre-recorded  digitised 
video  excerpts.  They  can  formulate  their  thoughts  about  these  excerpts  and  communicate  with  others  about  them, 
considering  others’  views  of  them,  including,  in  particular,  a generative  view  of  learning.  As  an  environment  in 
which  learners  can  develop  a generative  view  of  learning,  in  ways  consistent  with  such  a theory,  the  Generative 
Virtual  Classroom  is  poised  to  be  a powerful  technological  test  of  this  generative  theory  itself 


Do  Learners  Learn  Generatively  in  the  Generative  Virtual  Classroom? 

Already,  pilot  research  conducted  in  the  Generative  Virtual  Classroom  affirms  the  fruitfulness  of  viewing 
learning  generatively.  For  example,  in  a nine-month  autobiographical  study  of  the  development  of  her  views  of 
learning  in  the  Generative  Virtual  Classroom  (Allard  1 998),  an  Honours  Education  student  was  able  to  establish 
her  conceptual  progression,  for  herself,  and  to  identify  what  she  still  needed  to  understand.  On  analysis,  this 
student’s  learning  was  well  explained  in  generative  terms;  and,  as  well,  on  the  basis  of  her  experience,  minor 
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alterations  were  able  to  be  made  to  the  learning  environment  itself.  Such  studies  suggest  the  value  of  proceeding 
to  larger  scale  tests  of  the  worth  of  this  theory.  Consequently,  a large-scale  collaborative  research  trial  of  the 
Classroom  is  now  in  its  beginning  stages.  In  this  trial,  a full  cohort  of  First  Year  Education  students  (n=~100)  has 
begun  to  work  in  the  Generative  Virtual  Classroom,  yielding  early  data  on  the  nature  of  the  learning  of  a diverse 
range  of  individual  students  over  a semester.  Students’  text  contributions  to  the  Classroom’s  community  database 
and  e-mail  discussion  group  will  be  collated,  described  and  analysed  for  evidence  of  generativity,  in  an  attempt  to 
assess  the  worth  of  such  a theory  in  making  sense  of  what  they  did. 


Implications  for  Computer-mediated  Learning  and  the  Generation  of  Educational 
Knowledge 

Clearly,  the  recent  proliferation  of  technologically  mediated  environments  in  which  learners  can  teach  themselves 
has  brought  new  opportunities  to  explore  the  nature  of  learning.  Even  more  recently,  such  writers,  inter  alia,  as 
Alexander  & McKenzie  1998  and  Hannafin,  Hannafin  & Oliver  1997  have  begun  to  insist  that  these  new  learning 
environments  be  explicitly  grounded  in  educational  principles.  From  these  positions,  it  is  a short  but  significant 
step  to  conceiving  of  computer-mediated  learning  environments  as  instruments  for  the  testing  of  educational 
theories,  an  idea  whose  time  has  come.  Once  crystallised  in  these  terms,  such  a research  program  has  the 
potential  to  redress  long-standing  dissatisfaction  with  the  apparent  lack  of  theoretical  progression  in  Education  in 
particular  (Thagard  1992)  and  in  the  social  sciences  in  general  (Ziman  1978/1991).  The  paper  closes 
speculatively  by  discussing  the  implications  of  such  a re-conceptualisation  for  computer-mediated  learning  and 
for  the  generation  of  educational  knowledge. 
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Abstract:  This  paper  draws  on  data  gathered  during  a year-long  ethnographic  study  of  graduate 
education  classes  in  a 2-way  audio  2-way  video  distance  education  classroom.  Cultural  analysis  of 
transcript  excerpts  indicating  student  reactions  to  their  experiences  in  the  technologically  mediated 
classroom  highlight  the  anxiety  and  confusion  they  experience  in  making  the  shift  from  the 
familiarity  of  traditional  classroom  culture  to  the  uncertainty  of  a new,  often  strange  environment. 
This  transition  phase  in  classroom  cultures,  what  Turner  (1969)  calls  the  liminal  phase,  is  a time  of 
high  anxiety  but  also  of  high  learning.  Our  data  suggests  that  instructors  who  take  on  the  role  of 
technology  guide  and  provide  students  the  opportunity  to  vent  anxiety  and  learn  about  the 
technology,  create  environments  that  significantly  increase  the  comfort  levels  of  students  and 
receive  much  higher  ratings  from  them. 


Introduction 

Distance  learning,  via  web  based  and  two-way  interactive  video  classes  has  changed  the  market  areas  of 
educational  institutions  from  how  far  faculty  are  willing  to  drive  to  how  far  the  Internet  or  phone  lines  will  reach. 
Given  this  new  global  market,  institutions  of  higher  education  are  rushing  to  fill  the  niche.  Students  are  recruited, 
new  learning  environments  are  created  and  faculty  are  suddenly  asked  to  teach  classes  in  front  of  computer  screens 
and  cameras.  Although  the  literature  encourages  the  use  of  instructional  design  principles  when  teaching  this  way 
(Moore  & Kearsley  1 996),  in  the  rush  to  stake  out  new  territory,  such  classes  are  often  thrown  together  with  little 
planning  and  less  understanding. 

According  to  Willis  (1993),  “research  suggests  that  distance  education  and  traditionally  delivered 
instruction  can  be  equally  effective  if  the  distance  educator  puts  adequate  preparation  into  understanding  the  needs 
of  the  student  and  adapting  the  instruction  accordingly”  (p.  22).  But  how  does  one  do  that?  How  can  students  and 
teachers  anticipate  their  needs  in  a substantially  new  learning  environment?  Are  these  new  learning  environments 
traditional  classrooms  with  a few  technological  add-ons  where  the  assumption  can  be  made  that  student’s  needs  are 
little  changed?  Or  are  they  complex  new  environments  that  require  descriptive  and  analytic  illumination  by 
researchers  such  that  student  needs  can  be  assessed  and  then  addressed?  Our  research  suggests  the  latter. 

It  is  our  contention,  that  the  situation  people  face  in  distant  classes  is,  from  a cultural  perspective,  radically 
different  from  the  traditional  classroom.  The  enculturation  process  that  occurs  during  years  of  schooling  solidifies  in 
the  learner  norms  for  the  role  of  the  teacher,  for  methods  of  classroom  communication,  and  for  student  interaction 
that  lasts  a lifetime  (Henry,  1955).  The  entire  dynamic  of  teaching  and  learning,  which  is  deeply  grounded  in 
traditional  classroom  culture,  is  changed  in  complex  and  nuanced  ways  when  the  class  becomes  technologically 
mediated.  The  study  on  which  this  paper  is  based  used  the  approach  of  ethnography  and  the  perspectives  of  cultural 
anthropology  (Geertz  1973)  and  social  interactionism  (Mead  1934)  to  uncover  these  changes  in  classroom  culture 
and  illuminate  changes  in  the  teaching/leaming  dynamic. 

In  this  brief  paper  we  can  do  little  more  than  present  fragments  of  interviews  to  show  how  descriptions  by 
students  of  their  distance  education  experiences  and  an  awareness  of  traditionally  based  student  expectations  can 


inform  a deeper  understanding  of  the  needs  of  students.  This  understanding  then  becomes  the  basis  for  strategies 
that  instructors  can  use  to  help  learners  through  the  cultural  transformation  process  and  to  have  more  successful 
learning  experiences. 


The  Study 

The  study  is  based  on  data  gathered  for  an  ethnographic  case  study  of  graduate  education  classes  during  the 
inaugural  year  of  a 2-way  audio  2-way  video  interactive  distance  learning  classroom.  Courses  included  in  the  study 
varied  significantly  in  teaching  style,  course  objectives,  course  content  and  student  responses.  Traditional 
ethnographic  and  qualitative  research  methods  facilitated  data  gathering  and  analysis  (Agar  1996;  Maxwell  1996). 
Data  gathering  involved  more  than  400  hours  of  participant  observations  in  classrooms,  fomial  intern ews  and 
informal  conversations  with  students  and  faculty,  weekly  e-mail  correspondence,  open  ended  surveys,  focus  group 
interviews,  and  video  of  all  classes.  Thematic  coding,  frequency  counts,  and  frequent  debriefings  between 
ethnographers  L.  and  R,  Schmertzing  contributed  to  interpretation  and  ongoing  analysis  of  the  data  (Spradley  & 
Mann  1975). 


The  Data 

Excerpts  of  student  comments  from  e-mails  and  interview  transcripts  introduce  the  complex  culturally  based  issues 
that  students  face  in  the  interactive  televised  classroom.  All  of  the  following  passages  were  written  by  students  in 
response  to  questions  from  the  researchers  regarding  the  students  first  night  in  the  distance  classroom.  We  will 
present  the  excerpts  in  the  data  section  and  comment  on  them  in  our  discussion. 

When  1 saw  the  room  set  up  I was  in  shock,  actually.  It  was  my  first  interactive  distance 
learning  class  and  1 walked  into  the  room  thinking  "why  did  I sign  up  for  this?"  At  the  sight  of  all 
these  different  video  screens  and  cameras  all  around  the  room,  I didn't  think  that  the  grades  would 
be  the  same.  1 thought  they  would  favor  the  class  that  had  the  most  physical  interaction  with  the 
professor.  1 became  sort  of  jealous  of  the  other  campus  and  resented  the  fact  that  she  (the 
instructor)  was  there.  I felt  like  I was  at  the  main  campus  where  the  professor  should  have  been. 

If  the  other  class  was  at  a branch  campus,  they  should  have  primarily  been  in  the  distance  learning 
aspect  of  it.  My  GPA  is  very  important  to  me.  I feel  that  the  grade  I earn  will  be  affected  by  how 
much  time  I will  have  to  spend  watching  her  on  a monitor.  There  were  so  many  distractions,  it 
was  hard  for  me  to  stay  focused.  Also,  there  was  so  much  zooming  in  and  out  with  the  video 
cameras,  I honestly  got  motion  sickness  and  walked  out  of  class  feeling  sick  to  my  stomach  with  a 
headache.  I did  not  like  it.  The  cameras  were  set  up  in  awkward  angles  and  you  couldn't  see  the 
professor’s  head  when  she  was  talking.  We  got  great  views  of  the  back  of  her  head. 

Many  students  made  similar  comments,  although  the  attitudes  of  others  were  more  like  the  following  who  noticed 
the  difference  but  struggled  to  adjust. 

I was  really  surprised  at  my  reaction.  I’ve  taken  a web-based  class  before  and  I have  seen 
the  equipment  for  this  classroom  before.  I didn’t  think  I would  feel  any  different,  but  I did!  I was 
surprised  at  how  I looked  on  camera  and  feel  that  I will  have  to  work  on  this  so  that  it  doesn’t 
impact  my  participation.  Also,  I forgot  several  times,  where  to  “look”  so  I would  be  facing  the 
remote  group  when  I “spoke”.  I also  found  that  as  the  night  wore  on,  I kind  of  avoided  responding 
to  questions  ‘cause  I felt  like  I was  monopolizing  the  session.  I did  like  being  able  to  “face”  the 
other  group  and  do  not  feel  that  the  experience  effected  my  learning.  I found  myself  “concerned” 
about  the  remote  site  when  we  had  a few  technical  problems  and  am  concerned  about  the  learning 
experience  when  the  instructor  goes  to  the  other  class  and  I am  in  the  remote  site,  having  to  rely  on 
the  technology. 

For  some  students  the  two-way  audio/video  class  they  participated  in  was  an  entirely  positive  experience. 
“After  the  first  night  of  class  I am  totally  satisfied  with  everything.  I especially  enjoy  being  able  to  take  the  class  at 
this  campus  and  am  excited  to  have  the  opportunity  to  be  a part  of  this  class.” 

Other  students  cited  particular  elements  of  the  technologically  mediated  classroom  that  fed  their  anxieties 
and  complicated  their  ability  to  focus  in  the  classroom.  “I’m  just  much  more  aware  of  asking  questions  and  making 
contributions,  because  I have  to  do  this  mechanical  thing.”  The  necessity  to  physically  encounter  technology  in 
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order  to  participate  in  class  confused  students  and  contributed  to  a magnified  concern  for  the  value  of  their  own 
comments.  “You  don’t  have  the  ability  to  just  sit  there  and  say,  ‘Wait  a minute;’  you’re  on  camera  now,  you’re  on 
air-time,  and  so  if  you  have  what  you  think  may  be  a little  question,  you’re  wasting  class  time  [if  you  ask  it].” 


Discussion 

The  variation  in  student  responses  points  toward  the  diversity  and  complexity  in  their  perspectives. 
Students  from  each  class  all  have  different  ideas  about  their  experiences  in  their  new  classroom  environment.  They 
all,  however,  enter  the  classroom  with  an  understanding  of  the  way  a traditional  class  works  and  quickly  confront 
the  differences.  Students  expect  the  classroom  to  look  a certain  way;  they  expect  to  focus  on  a teacher  standing  at 
the  front  of  their  classroom;  they  expect  to  have  easy  means  to  speak  and  he  heard.  When  these  culturally  based 
expectations  are  not  met,  students  are  often  left  with  a sense  of  anxiety  and  feelings  of  uncertainty.  We  found  that 
complex  technologically  mediated  interactions  between  learners,  instructors,  and  content  forced  the  re-construction 
of  the  way  participants  had  previously  “done  class.”  Learners,  caught  between  the  old  way  and  the  new  way,  moved 
through  a liminal  state  (Turner  1969)  to  find  a balance  that  provided  enough  comfort  with  the  environment  so  that 
distractions  were  minimized  and  learning  maximized.  The  student  need  indicated  in  this  paper  is  the  need  to  be 
guided  through  this  liminal  state,  the  need  to  be  helped  through  the  adjustment  phase.  In  such  a context,  the  new 
role  for  the  instructor  is  as  a guide. 

Instructors  that  we  observed  to  be  successful  guides  used  various  strategies  to  speed  up  and  enhance 
students’  adaptation  to  the  new  classroom.  Students  found  balance  more  quickly  when  instructors  recognized, 
articulated,  and  showed  sympathy  for  the  “awkward,”  “uncomfortable,”  “surprising,”  “overwhelming”  state  of  the 
learner.  When  instructors  talked  to  their  students  in  an  “up  front,”  “understanding,”  “sensitive,”  (from  student 
transcripts)  manner  about  the  strangeness  of  having  class  in  the  new  environment,  students  were  more  responsive  to 
instructions  on  how  to  use  the  technology  to  accomplish  class  goals.  When  learners  were  allowed  opportunities  to 
express  their  feelings  about  the  environment  without  fear  of  repercussions,  they  appeared  to  overcome  the  added 
burden  of  “hiding  [their]  anxieties”  about  to  taking  a distance  class  and  became  more  “willing”  to  work  with  the 
new  technologically-mediated  environment.  The  instructor  who  provided  students  the  opportunity  to  “vent”  through 
both  class  discussions  and  online  threaded  discussions  achieved  the  highest  rating  for  classroom  interaction  of  all  the 
instructors  considered  in  this  study  and  did  so  in  the  least  amount  of  time. 

The  style,  manner,  and  attitude  an  instructor  established  the  first  night  of  class  was  also  a key  factor  in  how 
long  it  took  students  to  learn  the  technological  system  and  for  the  ease  and  speed  with  which  they  learned  to 
function  effectively.  First  class  meetings  rated  highly  by  students  were  those  that  were  characterized  as  “fun,” 
“calm,”  and  “slow  enough  to  understand.”  Instructors  who,  in  early  classes,  laid  ground  rules  for  communication, 
clearly  and  completely  explained  the  technology  of  the  classroom,  gave  students  an  opportunity  to  use  the  system  in 
a non-threatening  manner,  and  provided  a thorough  picture  of  what  would  be  expected  throughout  the  semester  built 
bridges  between  participants  and  the  technology.  Once  the  bridge  was  built  and  the  connection  made,  the 
environment  felt  “safer”  and  interaction  took  place  more  easily. 

These  are  a few  of  the  insights  and  strategies  that  have  emerged  from  our  analysis  so  far.  We  expect  that 
our  continued  analysis  will  produce  many  more. 
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Abstract 

Within  the  scope  of  the  development  of  a new  multimedia  education  concept  for  the  teaching 
profession  “Computer  Science”,  the  strengths  and  weaknesses  need  to  be  constituted  already 
during  the  development-phase  and  weaknesses  need  to  be  eliminated  if  necessaiy.  Therefore, 
the  concept  is  examined  within  the  formative  evaluation  on  its  tangibility  and  effectiveness. 
The  newly  developed  education  concept,  which  was  developed  at  the  Gerhard  Mercator 
University  Duisburg  under  usage  of  new  media,  aims  at  creating  new  possibilities  for 
differentiation  and  decentralized  learning  by  a multimedia  teaching  and  learning  concept.  The 
didactical  new  conception  of  the  course  is  supposed  to  take  the  students  different  learning 
prerequisites  into  account.  Thus,  procurement  methods  are  striven  for  that  enable  individual 
preparation,  consultation  and  practice  dependent  on  the  students  educational  background. 


Introduction 

The  teaching  profession  with  the  vocational  field  “Computer  Science”  at  the  Gerhard  Mercator  University 
Duisburg  can  be  attended  by  students  with  different  main  subjects.  Students  can  combine  the  specialized 
professional  field  “Computer  Science”  with  different  main  subjects,  e,g,  mechanical  engineering,  chemistry 
technique  or  textile  and  clothing  technique.  So  a teaching  concept  had  to  be  developed  due  to  the  different  bases 
in  the  information  technical  education  of  the  different  courses.  It  has  to  consider  the  heterogeneous 
foreknowledge  of  the  students  and  offer  according  intensive  procurement  methods. 

The  developed  concept  is  used  in  the  basic  course  “Fundamentals  of  Computer  Science”  as  first  trial  phase.  The 
course  covers  topics  such  as  the  Boolean  algebra  or  switching  algebra,  the  technical  implementation  of  logical 
operations,  which  are  based  on  the  Boolean  algebra  and  finally  the  fundamental  digital  circuits  and  systems  as 
well  as  simple  digital  computers,  which  are  based  on  these  operations. 

The  procurement  of  contents  and  the  conversion  of  the  quoted  objective  requires  the  completion  of  the  course 
forms  so  far  by  means  of  a suitable  concept  on  computer  base  and  the  transfer  by  means  of  multimedia 
technique.  The  interactive,  multimedia  teaching  and  learning  software  is  set  up  in  such  a way  that  the  different 
learning  methods  of  the  students  are  taken  into  account.  Therefore,  different  points  of  entry  are  available  for  the 
software. 

One  entry  point  runs  over  the  content  structure,  here  the  student  can  choose  his  learning  path  in  accordance  with 
the  content  of  the  lecture.  The  course  so  far  included  exercises  for  practicing  and  applying  the  taught  methods. 
This  exercise  part  is  also  integrated  in  a suitable  form  into  the  interactive  learning  concept.  The  second  entry 
point  clarifies  the  interaction  between  the  acquired  knowledge  and  the  latter  practice  on  the  basis  of  a descriptive 
task.  Thus,  a learning  goal  oriented  or  res,  a problem  based  learning  is  enabled.  The  student  receives  an  course  of 
action.  A detailed  description  of  the  concept  is  given  in  (Schwarz,  Hunger,  Werner,  1999). 


Initial  Considerations  for  Evaluation 

Sources  of  information  that  are  available  for  a study  group  are  on  the  one  hand  conservative  sources  such  as 
books,  scripts  and  patterns,  the  instructor  and  laboratory/experiment  device.  Additionally,  fellow  students  also 
count  as  information  source,  since  notes,  sources  of  information  and  interpretations  can  be  exchanged  with  them. 
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New  media  can  take  the  place  of  an  information  source  (book)  and  the  place  of  an  experiment  device 
(simulations).  It  can  also  partly  take  over  the  role  of  a teacher,  by  answering  standard  questions  and  checking 
exercise  results,  as  far  as  the  exercises  are  machine  controllable.  Furthermore,  new  media  creates  more  extensive 
communication  possibilities  (Dweck,  1 988). 

The  entire  focus  during  the  introduction  of  new  media  in  complex,  problem  based  learning  situations  is 
determined  and  already  distributed  on  different  learning  tasks.  The  attention  can  not  be  increased  by  the 
introduction  of  new  media!  In  the  case  of  new  channels  being  available,  the  individual  must  redistribute  its 
bandwidth  and  thus  its  attention  on  the  available  channels.  A rearrangement  takes  place  on  expense  of  other 
channels.  Refocusing  might  lead  to  a favoring  of  channels  due  to  improved  accessibility  of  information  through 
new  media,  which  enables  individuals  to  receive  information  more  easily  and  thus  the  attention  on  other 
channels  is  encouraged.  We  can  speak  of  a positive  modification  of  the  social  behavior  induced  by  introducing 
new  media,  if  both  possibilities  deal  with  the  channels  which  the  individual  use  to  interact  with  other  members 
of  the  group. 


Formative  Evaluation 

The  concept  is  evaluated  in  several  steps.  In  the  first  step,  the  teaching  and  learning  processes  are  evaluated  by 
observing  and  interviewing  the  participants.  The  experiences  of  students  from  different  subgroups  are  of 
importance  and  thus  raised  in  particular.  The  members  of  the  group  are  interviewed  by  using  a questionnaire. 
This  questionnaire  ranks  important  data  through  accordingly  formulated  questions  on  socio-metric  matters  for 
the  4 partitions  of  the  theory  model  (economy  of  attention,  social  behavior,  motivation  control,  accessibility  of 
information).  The  behavior  and  movements  of  the  user  is  recorded  at  the  same  time.  Thus,  it  is  to  be  determined, 
whether  the  mapped  mental  model  is  applicable  on  the  learning  group  and  whether  weaknesses  in  the  navigation 
and  comprehensibility  can  be  discovered.  The  results  flow  directly  into  the  development.  Furthermore,  an  expert 
workshop  is  set  up  in  order  to  determine  suitable  scenarios  of  different  emphasis.  The  expert  workshop  primarily 
consists  of  professors  of  the  respective  field. 

The  evaluation  leads  to  the  following  results: 

70  participants  were  interviewed,  among  them  where  25%  that  use  MODULO  regularly.  The  participants  hardly 
use  MODULO  for  stringent  reading,  but  rather  for  the  direct  search  of  information.  This  functions  well  to 
outstanding;  students  indicated  that  finding  information  under  usage  of  MODULO  is  substantially  better.  The 
links  between  exercise  and  lecture  were  considered  as  helpful.  The  complex  scenarios,  which  are  supposed  to 
clarity  the  application  of  knowledge  in  vocational  practice,  are  considered  less  helpful  for  understanding  the 
material.  Most  students  rather  find  the  scenarios  helpful  at  ranking  the  material  into  vocational  practice. 

In  a further  step,  a summative  evaluation  will  take  place  after  final  completion  in  order  to  elevate  the  acquired 
knowledge  of  numerous  years  and  using  it  for  advancements  of  the  concept. 


Summary  and  Prospect 

In  this  article  a new  concept  for  supporting  the  theory  of  the  teaching  profession  course  ’’Computer  Science”  and 
an  appending  formative  evaluation  was  presented.  The  concept  aims  at  media-technical  editing  of  the 
disposition,  the  development  of  a training  concept  and  the  development  and  implementation  of  a WEB-database 
with  dynamic  access.  This  concept  holds  the  advantage  of  being  able  to  go  into  different  needs,  previous 
knowledge  and  learning  methods  of  students,  due  its  modular  structure.  It  enables  a self-controlled  learning, 
promotes  media  competence  and  offers  demanding  possibilities  of  acting. 

The  first  evaluation  which  accompanied  the  development  of  the  system  had  positive  results  and  constructive 
criticism.  The  entire  system  is  to  be  completed  by  September  2000,  so  that  the  summative  evaluation  can  be 
started  in  the  winter  semester  2000/01. 
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Abstract:  CSULA  was  charged  with  assisting  the  NSA  in  the  research,  development, 
documentation  and  the  necessary  expertise  to  convert  a traditional  face-to-face  platform 
course  to  a multimedia  web-based  course.  We  converted  a platform-based  course  taught  by  the 
NSA  called  "EEO  100:  Equal  Employment  Opportunity  Law  and  Diversity  Training  for 
Selection  Boards."  To  accomplish  the  task,  an  interdisciplinary  team  of  graduate  and 
undergraduate  students  from  Instructional  Technology,  Art  Education,  Computer  Science,  and 
Design  collaborated  on  producing  the  website. 


Constructivism 

Recent  studies  in  education  have  promoted  the  constructivist  approach  to  learning  “...where  students 
develop  their  knowledge  through  team  collaboration,  discuss  different  interpretations  of  a problem,  and 
negotiate  and  synthesize  ideas  drawing  from  various  disciplines”  (Boyer  & Semrau,  1995,  14).  Technology  can 
play  a significant  role  in  this  process  by  having  students  become  more  proactive  in  their  own  learning  and  more 
independent  in  their  search  for  knowledge.  Students  using  technology  have  access  to  ongoing  real  world 
problems,  on-line  dialog  with  scientists,  and  access  to  the  latest  technological  advances.  Ultimately,  students 
will  be  engaged  in  their  learning,  see  relevance  in  it,  and  take  on  an  ownership  in  it.  To  meet  the  needs  and 
interests  of  students,  education  must  emphasize  problem  solving  and  application  of  information,  instead  of 
memorization  of  facts.  Roberts  et  al  state,  “...students  need  to  be  able  to  find  and  use  relevant  information,  share 
and  discuss  data  and  ideas,  and  collaborate  on  problem  solving”  (1990,  116). 

Year  One  of  Project 

July  of  1998,  California  State  University,  Los  Angeles  (CSULA)  began  a 3-year  project  sponsored  by  the 
National  Security  Agency  (NSA)  entitled  “NSA  Partnership  with  Cal  State  L.A.:  Courseware  Conversion  and 
Evaluation  Project.” 

CSULA  was  charged  with  assisting  the  NSA  in  the  research,  development,  documentation  and  the 
necessary  expertise  to  convert  a traditional  face-to-face  course  to  a multimedia  web-based  course.  CSULA 
students  were  selected  to  work  on  the  project  with  a focus  on  collaborative  group  work  and  a constructivist 
approach  to  learning.  An  interdisciplinary  team  of  graduate  and  undergraduate  students  from  Instructional 
Technology,  Art  Education,  Computer  Science,  and  Design  were  brought  together  under  the  direction  of  two 
faculty  members. 

In  this  project  we  are  converting  to  the  web  a platform-based  course  taught  by  the  NSA  called  “EEO  100: 
Equal  Employment  Opportunity  Law  and  Diversity  Training  for  Selection  Boards.”  EEOlOO  is  directed  to 
employees  who  serve  or  who  may  serve  on  boards,  panels,  or  committees  charged  with  the  responsibility  of 
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rating  workers  for  selection  purposes.  The  course  is  designed  to  provide  participants  with  an  understanding  of 
the  extent  to  which  their  rating  activities  are  subject  to  applicable  civil  rights  laws,  and  to  appreciate  the  value 
and  contributions  of  every  candidate.  Students  after  taking  this  course  will  be  able  to: 

1 . Understand  the  requirement  for  EEO  100 

2.  Understand  legal  rights  and  responsibilities  for  fair  and  equal  treatment  in  the  decision-making  process 

3.  Gain  a common  understanding  of  “diversity” 

4.  Recognize  “blind  spots”  and  conditioning  from  society,  and  how  these  factors  impact  decisions 

5.  Provide  a hands-on  opportunity  to  improve  the  decision-making  process 

6.  Appreciate  the  value  and  contributions  of  every  individual 

Training  was  provided  to  the  CSULA  student  team  on  the  basics  of  htm!,  Adobe  Photoshop,  Premiere,  and 
Fagemiii  in  order  tor  them  to  be  able  to  undertake  the  task  of  courseware  conversion  to  the  web.  They  were  also 
trained  on  the  use  of  WebCT  for  developing  the  website,  uploading  of  web  pages  to  website,  and  designing  of 
web  pages. . WebCT  is  an  online  Internet  authoring  tool  for  web-based  courses.  The  students  were  guided 
regarding  the  overall  layout  and  screen  designs.  For  a foundation  in  working  collaboratively  in  building  a 
website  the  students  first  designed  a demonstration  site  on  streaming  media — RealAudio  and  RealVideo.  The 
demonstration  site  was  developed  for  sharing  our  research  and  for  communication  among  the  students  and 
project  faculty.  Focus  was  placed  on  developing  an  educational  website  demonstrating  various  applications  of 
streaming  video  and  audio  in  education  as  well  as  being  a source  of  research  data  and  instruction  for  creating 
streaming  video  and  audio  for  web-based  training. 

The  students  investigated  the  hardware  and  software  requirements  for  streaming  media,  selected  examples 
of  streaming  video,  and  html  coded  web  pages  documenting  their  gathered  data  about  streaming  media  and  its 
role  in  web-based  training.  The  WebCT  course  site  also  offered  additional  built-in  communication  features 
including  e-mail,  chat  rooms,  a bulletin  board,  and  white  board.  Students  used  the  bulletin  board  to  provide  help 
to  each  other  and  to  assess  each  other's  web  pages. 

For  the  foundational  first  part  of  our  collaborative  work,  the  students  researched  and  structured  their 
content  on  streaming  media  into  the  following  sub-areas: 

1 . Introduction  to  Streaming  Media 

2.  How  Does  It  Work— Streaming  Video? 

3.  How  Does  It  Work-Streaming  Audio? 

4.  Aesthetics  of  Streaming  Video 

5.  Aesthetics  of  Streaming  Audio 

6.  Educational  Applications  of  Streaming  Media 

7.  Our  Examples  of  Streaming  Video 

8.  Project  Report  1998-99 

9.  About  Us 

Each  student  was  charged  with  the  design,  structure,  research,  and  development  of  one  content  sub-area. 

Year  Two  of  Project 

During  the  fall,  1999  we  moved  onto  converting  the  EEO  100  course  content  into  a web-based  course. 
We  followed  the  pattern  established  in  the  prior  year  where  each  student  was  assigned  full  responsibility  for  the 
research,  design,  development,  and  coding  of  a topic.  The  EEO  100  content  was  subdivided  into  the  following 
main  topic  areas:  Current  requirements,  additional  requirements,  definitions  of  discrimination,  disparate  impact, 
harassment,  affirmative  action  review,  and  tools.  Each  student  studied  and  learned  the  EEO  100  course  content 
and  was  then  individually  tested  on  it.  In  the  process  of  understanding  the  content,  they  realized  that  it  would  be 
necessary  to  rewrite  and  revise  the  content,  as  well  as  integrate  multimedia  to  accommodate  a web-based 
learning  environment.  Next  they  html  coded  the  EEO  100  course  content  and  thus  begun  the  creation  of  the 
web-based  course.  One  of  the  students  developed  the  screen  design  incorporating  American  Disabilities 
Association  (ADA)  guidelines. 
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They  developed  new  test  items  for  the  content.  They  researched  websites  and  selected  appropriate  ones 
related  to  the  content  and  created  external  links  to  these  websites.  We  designed  draft  layout  for  structuring  the 
menu  and  content.  And,  we  investigated,  selected,  and  adapted  a quiz  format  for  use  in  this  website  to  set  up 
pop  quizzes  that  were  integrated  within  content  areas. 

In  the  winter  of  2000,  the  students  analyzed  a video  that  was  a tape  of  an  original  face-to-face 
traditional  approach  to  teaching  the  class,  which  was  provided  by  the  NS  A.  They  made  a list  of  the  video  clips 
matching  the  content  they  were  responsible  for.  Before  we  started  digitizing  the  content,  we  inspected  the 
length  and  filesize  of  each  video  clip  to  determine  how  to  reduce  the  size  of  very  large  clips.  We  came  up  with 
several  solutions  which  included  digitizing  only  the  audio  for  certain  clips  and  accompanying  the  audio  with 
still  diagrams  of  key  points  and  cutting  up  the  longer  clips  in  to  shorter  segments.  Next  the  students  captured 
the  video  using  Adobe  Premiere  and  tlie  Osprey  100  video  card  in  a Del!  Precision  401  computer.  They  edited 
down  the  clips  using  Adobe  Premiere.  The  audio  was  captured  and  edited  with  SoundForge.  Next,  the  clips 
were  streamed  with  RealProducer  and  html  coded  into  their  relevant  web  pages  of  content. 

To  polish  off  the  interface  design,  we  brought  on  an  art  student  who  was  charged  with  developing  an 
easy  to  use  menu  of  topics  and  navigational  system. 

In  the  spring  of  2000,  it  is  our  intent  to  synthesize  various  course  components  (content,  video,  test 
items)  into  a final  course  website  and  to  design  visual  learning  aids  (diagrams,  charts,  and  pictures)  and  tools 
(glossary,  law  books,  and  court  cases)  to  enhance  learning  of  the  course  content.  We  are  preparing  to  design  and 
incorporate  simulations  into  the  website.  NS  A has  been  encouraging  and  supportive  of  this  idea  as  an  extended 
project. 

Conclusion 

The  new  web-based  version  of  this  course  will  make  the  content  accessible  to  more  NS  A employees 
located  in  various  places  throughout  the  world.  Plus,  the  online  version  will  include  interactive  pop  quizzes 
with  immediate  scoring  and  feedback  as  well  as  streaming  media  lectures  to  complement  the  course  content, 
which  is  presented  as  text  on  the  web  pages. 

Through  this  project  the  students  became  constructively  involved  in  their  own  learning  and  have 
acquired  in-depth  experiences  in  collaborative  learning.  The  students  became  empowered  to  be  creators  of  their 
own  curricular  materials  and  web  pages  instead  of  being  passive  viewers  of  others’.  In  this  project  all  of 
Bloom’s  higher  level  taxonomies  are  implemented.  Students  are  analyzing  websites,  synthesizing  criteria  that 
they  have  researched,  applying  their  book  readings,  comparing  and  contrasting  their  criteria,  designing  and 
producing  their  own  web  pages,  and  more. 
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Abstract:  The  convergence  of  modem  data  acquisition  technologies  with  the  Web’s  interactivity, 
connectivity  and  multimedia  capabilities  presents  an  exceptional  opportunity  for  distance  education  in  the 
physical  sciences.  Web-mediated  access  and  control  of  laboratory  equipment  can  improve  utilization  of 
expensive  and  specialized  instruments,  facilitate  collaborative  data  sharing  and  analysis,  and  provide 
essential  practical  experience  in  physical  science  courses  delivered  at  a distance. 

This  paper  describes  a remotely  controlled  experiment  for  determining  the  rates  of  fast  chemical  reactions. 
The  experiment  is  not  a simulation;  it  involves  actual  equipment  controlled  in  real  time  ffom  remote  locations 
on  the  Web.  The  experiment  is  the  first  in  a series  designed  to  provide  a pedagogically  on-line  laboratory 
experience  for  Web-delivered  general  chemistry  courses.  Students  use  the  experiment's  Web  interface  to 
collect  data,  to  obtain  interactive  technical  support  and  background  information,  and  to  display  and  analyze 
results.  Each  experiment  is  designed  to  encourage  sharing  of  data  and  collaboration  with  users  at  other 
institutions,  providing  students  with  a valuable  first  look  at  work  in  a distributed  laboratory  environment. 


Introduction 

Chemistry  is  an  experimental  subject.  Chemists  construct  knowledge  by  systematically  examining  quantitative  data 
for  patterns.  Patterns  suggest  hypotheses,  which  in  turn  form  the  basis  for  theories.  Theories  are  then  critically 
evaluated  in  the  light  of  new  experimental  data.  This  is  the  only  sanctioned  approach  for  progress  in  science. 
Pedagogy  that  is  not  founded  on  the  interplay  between  theory  and  experiment  poorly  serves  students  by  not 
involving  them  in  the  essential  process  of  science. 

The  experimental  nature  of  chemistry  presents  severe  challenges  for  teaching  the  subject  at  a distance.  Many  current 
online  chemistry  courses  either  have  no  laboratory  experience  at  all  or  use  simple  kits  or  'kitchen  chemistry' 
experiments  that  can  be  performed  with  household  materials.  Obtaining  accurate,  quantitative  data  ffom  which 
relevant  and  interesting  conclusions  can  be  drawn  is  difficult  with  such  simple  equipment. 

Online  chemistry  courses  often  make  heavy  use  of  simulations.  While  simulations  can  be  quite  effective,  they  are 
theoretical  constructs  that  cannot  substitute  for  practical  experience.  For  example,  students  regarded  the  output  of 
simulations  in  a statistical  analysis  experiment  at  Oxford  University  as  "pretend"  data  (Cartwright,  1998).  They 
sometimes  failed  to  carry  over  the  lessons  learned  ffom  analyzing  simulated  data  to  data  collected  in  actual 
experiments.  Concerns  about  the  pedagogical  quality  of  the  simulations  were  addressed  by  the  development  of  a 
simple,  economical  "optical  rig"  (Cartwright,  1999),  which  allows  students  to  collect  and  analyze  actual  data  over 
the  Internet. 

This  project  addresses  the  problem  of  practical  experience  in  Web-delivered  courses  by  providing  students  with 
remote  access  and  control  of  real  equipment.  A remote-control  reaction  kinetics  experiment  suitable  for  chemistry 
students  at  the  undergraduate  or  advanced  secondary  level  is  described.  The  apparatus  monitors  the  course  of  a fast 
chemical  reaction  spectrophotometrically  using  a simple  continuous-flow  method.  Students  can  use  the  data  they 
collect  to  determine  the  rate  laws  for  the  reaction  under  a variety  of  conditions.  By  collaboratively  analyzing  the 
results  of  a large  group  of  experiments  performed  by  peers,  students  can  propose  mechanisms  for  the  reaction.  While 
the  experiment  is  conceptually  quite  simple,  it  involves  specialized  apparatus  that  is  unavailable  in  most 
undergraduate  laboratories.  Rapid  data  acquisition  can  allow  many  users  to  perform  experiments  in  real  time,  and 


nearly  spontaneously. 


Method 


The  experiment  is  an  adaptation  of  a classic  continuous  flow  method  for  studying  the  rates  of  fast  reactions  (Dalziel, 
1953;  Roughton  & Chance,  1963).  The  apparatus  can  be  used  to  determine  the  rate  of  any  sufficiently  fast  reaction 
that  produces  a colored  product,  although  practical  considerations  limit  the  choice  of  reaction(Shoemaker,  1974), 
Pilot  investigations  studied  the  formation  of  FeSCN^^  (an  orange  complex  that  absorbs  at  blue  light  at  455  nm)  and 
“Prussian  blue”,  an  intensely  colored  complex  used  in  blueprinting,  paints  and  inks  (Sharpe,  1976). 


A schematic  diagram  of  the  apparatus  is  shov.Ti  in  (Fig.  1). 
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schematic  for  fast  reaction  kinetics  experiment 


Reactant  solutions  are  forced  from 
large  reservoirs  into  a T-shaped 
chamber,  where  they  mix  completely 
in  less  than  1 millisecond.  The 
reacting  mixture  then  travels  down  a 
long  capillary  tube.  The  product  of 
the  reaction  is  intensely  colored  and 
strongly  absorbs  a characteristic 
wavelength  of  light.  Product 
concentration  rises  as  the  mixture 
moves  along  the  tube.  Seven  light 
sensors  placed  at  fixed  positions 
along  the  tube  monitor  the  increase 
in  concentration.  Monochromatic 
light  is  passed  through  the  capillary 
at  each  position  using  a fiber-optic 
cable.  High  sensitivity  photocells 
placed  on  the  opposite  side  of  the 
tube  record  the  intensity  of 
transmitted  light  at  each  point;  the 
intensity  drop  caused  by  absorption 
of  the  light  by  the  complex  is  simply 
related  to  the  concentration.  A flow 
rate  sensor  allows  the  position  of 
each  of  the  spectrophotometers  to  be 
associated  with  a reaction  time. 


Data  from  the  sensors  is  acquired  by  “sanzone”,  a 500-MHz  PC  equipped  with  a National  Instruments  data 
acquisition  card.  A Lab  View  program  collects,  labels,  and  writes  the  absorbance,  temperature,  and  flow  rate  data  to 
an  "outgoing”  queue.  The  program  then  checks  for  requests  in  an  "incoming"  queue.  If  one  exists,  a new  run  is 
started.  Otherwise,  the  system  is  flushed  and  the  valves  are  closed  to  avoid  unnecessary  waste  of  reagent  solutions. 


A Webcam  collects  pictures  of  the  apparatus  during  the  run.  The  camera  view  includes  the  reservoir  manometer  and 
a digital  thermometer,  so  it  is  used  for  data  collection  as  well  as  for  helping  students  monitor  the  progress  of  the 
reaction.  The  Webcam  is  attached  to  a second  PC  to  avoid  contention. 


A Unix  Web  server  (Marie)  mounts  Sanzone ’s  hard  drive  across  the  local  network.  Perl  scripts  running  on  Marie 
fetch  data  from  the  outgoing  queue,  interpret  it,  and  prepare  the  results  for  display. 

Students  perform  the  experiments  via  a "smart"  instrument  panel  served  from  Marie,  displayed  in  a standard 
browser.  The  instrument  panel  recreates  the  actual  look  and  feel  of  the  remote  experiment.  Operators  immediately 
see  the  results  of  their  actions  via  the  Webcam,  as  well  as  on  a rich  data  display.  The  panel's  panic  button  launches  a 
conferencing  connection  with  the  system’s  caretaker  via  Netmeeting  during  working  hours  and  by  email  at  other 
times.  The  panel  monitors  and  interprets  student  activity  to  diagnose  and  correct  conceptual  problems  and  to  prevent 


actions  that  could  damage  or  tie  up  the  apparatus  for  long  periods  of  time. 

The  instrument  panel  front  end  is  implemented  in  JavaScript  and  Macromedia  Flash.  When  students  submit  a run 
from  the  instrument  panel,  backend  Perl  scripts  label  and  process  the  request  and  write  it  to  an  "incoming"  queue  on 
Sanzone  and  the  expected  time  of  completion  for  the  job  is  displayed  on  the  panel.  Jobs  complete  within  a few 
seconds  when  the  incoming  queue  is  empty.  Queues  are  intelligently  managed  by  consecutively  scheduling  jobs 
with  similar  operating  parameters  and  by  assigning  lower  priority  to  non-local  IP  addresses  (to  prevent  ‘walk-in’ 
users  from  monopolizing  the  experiment). 

Future  versions  of  the  experiment  will  allow  students  to  manipulate  additional  parameters,  such  as  reagent 
concentrations,  pH,  temperature,  ionic  strength,  and  flow  rate.  Data  from  different  groups  can  then  be  pooled  and 
analyzed.  Students  will  then  use  these  results  to  evaluate  proposed  mechanisms  for  the  reaction.  Further 
developments  will  be  outlined  on  the  project  home  page  at  http://antoine.frostburg.edu/chem/senese/inetchemset. 
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Abstract:  Distance-Learning  based  on  WWW  has  become  a trend  for  the  development  of 
education.  But  the  distance-learning  site  based  on  WWW  is  static,  the  designers  of  the 
courses  don’t  know  whether  the  design  of  courses  is  rational  or  consistent  with  the  teaching 
law.  While  there  is  a lot  of  students’  information  accumulated  in  the  web  site,  and  the 
information  is  useful  for  our  course  designers.  In  this  study,  we  present  a distance-learning 
model  based  on  Web  Mining,  which  can  take  advantage  of  those  students’  information 
accumulated  in  the  web  site. 


l«introduction 

Distance-learning  based  on  WWW  has  become  a trend  for  the  development  of  educadon.  The  available 
distance-learning  model  based  on  WWW  is  discussed  in  (Xuejun  Li  1999).  But  recently  the  distance-learning  site 
based  on  WWW  is  static,  and  the  designers  of  the  courses  don’t  know  whether  the  design  of  courses  is  rational 
and  consistent  with  the  teaching  law.  While  there  is  a lot  of  students’  information  accumulated  in  the  web  sites, 
such  as  students’  access  patterns  and  registration  and  communication  data  information,  and  these  information  is 
useful  for  our  course  designers  and  teachers. 

In  this  study,  we  present  a distance-learning  model  based  on  Web  Mining,  which  can  take  full 
advantage  of  those  students’  information  accumulated  in  the  web  sites. 


2.A  Distance-Learning  Based  on  Web  Mining 

The  model  integrates  the  Web  Mining  technology  into  the  available  model  base  on  WWW.  It’s 
architecture  is  shown  (Fig.l).  In  this  model,  we  add  the  Web  Mining  module  into  the  available  model.  The  Web 
Mining  model  is  located  in  the  server-side  and  performs  in  the  server-side. 

According  to  our  special  system  for  distance  learning,  the  Web  Mining  process  as  shown  in  (Fig.4). 
The  Web  Mining  module  has  three  phases(Robert  cooley  et  al.  1998):  Preprocessing(Yilin  yang  et  al.l999), 
mining  process  and  pattern  analysis. 

3«Conclusion 

our  model  has  following  points:  The  course  designers  can  reconstruct  a Web  sites  in  order  to  better 
serve  the  needs  of  students  of  a site  according  to  these  information  such  as  the  students’  access  patterns  and 
page  access  frequency  statistics;  it  can  provide  individualised  page  or  different  course  content  for  individual 
student;  and  it  can  provide  teachers  the  students’  feedback,  and  teachers  can  redesign  their  teaching  plans,too. 
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Figure  1.  A Distance-Learning  Model  Based  on  Web  Mining 
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Introduction 

The  aim  of  this  paper  is  to  illustrate  and  demonstrate  the  value  of  a user-centered  method  for  CALL 
systems  development.  We  accept  the  argument  of  Hubbard  (1996),  Levy  (1997)  and  other  the  deficiencies  of 
current  CALL  system  can  be  attributed  to  the  lack  of  a systematic  methodological  theory  guiding  their  design, 
Accordingly,  a CALL  design  method  has  been  developed  which  draws  on  recent  developments  in  the  field  of 
human-computer  interaction  regarding  scenario-based  design  and  prototyping  in  relate  to  the  user-centered 
design  of  computer-based  learning  material. 


Scenario-Based  Prototyping 

The  main  features  of  the  method  include  the  use  of  scenarios  to  identity  user  requirements  (Carroll, 
1995),  the  development  of  prototypes  embodying  key  design  issues,  and  a series  of  formative  workshops  to 
evaluate  the  prototypes.  The  overall  results  confirm  the  general  validity  of  the  user-centered,  scenario-based 
methodological  approach. 

There  were  three  cycles  of  prototyping  process  in  this  research.  Each  cycle  was  based  on  a set  of  User 
Satisfaction  Scenarios.  The  scenarios  were  used  to  build  prototypes  in  two  different  ways:  user  dissatisfaction 
scenarios  were  used  to  define  existing  problems  that  have  been  found  from  the  current  CALL  products;  and 
user  satisfaction  scenarios  were  used  to  clarify  possible  problems  that  could  be  found  in  the  CALL  product  to 
be  built.  A template  for  scenarios  is  proposed  for  the  educational  environment.  The  template  include  the  user 
environment  (i.e.,  age,  school,  class,  user's  conception  toward  CALL  system,  motivation  of  using  CALL 
software,  software  type),  learning  environment  (i.e.,  classroom,  self-studying,  group- studying),  and  narrative 
user  interaction  scenario  to  define  features  user  satisfied/dissatisfied.  Overall,  this  paper  is  to  suggest  the 
method  of  use  of  scenarios  explicitly  for  designing  CALL  systems,  not  to  show  the  results  of  prototypes  which 
have  been  constructed  and  evaluated 

Developing  prototypes  is  an  integral  part  of  iterative  user-centered  design  process  because  it  enables 
designers  to  try  out  their  ideas  with  real  users  and  to  gather  feedback.  There  were  three  major  cycles  of 
prototyping  process  with  different  prototypes  embodied  different  design  issues,  which  is  based  on  user- 
satisfaction  scenarios.  The  main  design  issue  of  the  first  prototype  systems  was  to  find  how  user-control  could 
affect  to  achieve  high  quality  CALL  systems  in  terms  of  both  educational  and  technical  value.  Two 
comparative  systems  were  built  for  this  initial  stage:  one  was  a high  user-control  (learner-based)  system,  and 
the  other  was  a low  user-control  (teacher-oriented)  system.  According  to  the  feedback  from  a series  of 
formative  user  workshops,  it  has  been  found  that  the  effectiveness  of  user-control  were  dependent  on  learners' 


experience  in  CALL  systems  and  their  learning  environments.  Based  on  the  feedback  from  the  first 
workshops,  the  user  satisfaction  scenarios  were  revised  to  find  the  design  issues  for  the  second  prototype. 

The  second  prototype  system  was  a hyperlink  system  between  three  different  systems:  Tutorial  system 
for  listening  skills  with  low  user-control,  Main  Lesson  system  for  practicing  listening  skills  with  high  user- 
control  for  experienced  users,  and  a sub-system  called  Speaking  Dictionary  to  provide  learners  linguistic  help. 
The  user  response  about  the  hyperlink  system  was  that  it  would  be  better  to  have  a different  user-control  by 
learning  stages  concerning  user  experiences.  Users  also  showed  positive  opinions  about  the  user  satisfaction 
scenarios  because  they  agreed  that  scenarios  make  it  easy  to  visualize  the  real-world  environment  and  to 
comprehend  users’  perspective  toward  CALL  systems.  On  the  other  hand,  users  responded  that  learners  need 
more  support  to  control  their  learning  process  more  effectively. 

Therefore,  the  third  prototype  constructed  was  focused  to  investigate  the  effectiveness  of  Teaching 
Agent,  to  support  learners’  learning  process  with  respects  of  HCl  aspects  and  educational  achievement.  At  this 
stage,  we  argue  that  TA  should  have  domain  knowledge  (i.e.  English),  Training  knowledge  (i.e,  listening 
skills),  and  Specific  pupil  knowledge.  Here,  the  specific  pupil  knowledge  is  the  most  significant  to  advise 
learners  what  to  do  next.  While  the  knowledge  of  domain  and  training  are  static  and  independent  on  student  or 
teacher,  the  knowledge  of  specific  pupil  depends  on  followings:  how  well  is  the  learner  doing?;  what  listening 
skills  have  been  used?;  what  personality  does  the  learner  have?;  and  what  is  the  learning  goals?  The  TA 
system  called  Magic  Mirror  has  been  developed  to  apply  those  aspects  based  on  the  revised  user  satisfaction 
scenarios,  in  which  learner  personality  and  learning  goal  of  specific  pupil  knowledge  were  not  considered.  The 
result  from  the  third  workshop  with  teachers  was  positive  towards  the  concept  of  TA  in  general,  but  some 
teachers  questioned  how  TA  could  act  as  a substitute  for  a teacher.  On  the  other  hand,  learners  showed  more 
positive  attitude  towards  the  TA  because  they  compared  the  TA  with  Help  functions  in  other  CALL  system, 
not  with  their  teachers.  The  conclusion  is  that  TA  could  make  CALL  systems  more  effective  if  all  the  aspects 
of  the  specific  pupil  knowledge  are  regarded  on  the  development. 


Conclusion 

The  essential  elements  of  this  user-centered  method  for  designing  CALL  systems  are  scenarios, 
prototypes,  and  formative  workshops.  Prototyping  helps  designers  to  make  user-oriented  decisions  by  eliciting 
information  from  the  formative  evaluations  of  users,  and  scenarios  are  useful  to  articulating  new  design 
concepts  like  TA.  In  HCI  practice,  scenarios  of  use  are  increasingly  being  used  to  help  set  overall  design 
objectives  for  guiding  various  aspects  of  the  design  process.  Carroll  (1995)  argues  that  if  we  believe  the 
scenario  approach  to  be  a valuable  enhancement  of  current  development  practice,  we  must  ask  how  the  use  of 
scenarios  in  system  development  can  be  evoked,  supported,  and  brought  to  bear  on  various  activities  within  the 
system  development  cycle.  Overall,  this  user-centered  method  is  very  effective  regarding  the  following  aspects. 
Firstly,  Scenarios  are  used  as  a bridge  between  designers  and  users  to  define  learners'  or  teachers’  needs  and 
learning  environment  in  order  to  achieve  learning  goals.  Secondly,  prototypes  are  used  to  apply  design  issues 
and  to  demonstrate  the  expected  learning  environment  concerning  pedagogical  value.  Finally,  user  workshops 
are  held  to  investigate  user  satisfaction  to  evaluate  design  issues  and  principles  in  terms  of  both  technical  and 
educational  value. 


References 

Carroll,  J.  M.  (1995).  Scenario-based  design:  Envisioning  work  and  technology  in  system  development.  NY: 
John  Wiley  & Sons,  Inc. 

Hubbard,  Philip  L.  (1996).  Elements  of  CALL  Methodology:  Development,  Evaluation,  and  Implementation. 
In  M.C.  Pennington  (Eds.)  The  Power  of  CALL  (pp.  15-32pp).  NY : Athelstan 

Levy,  Mike  (1997)  Computer-Assisted  Language  Leaning:  Context  and  Conceptualization.  Oxford:  Clarendon 


1737 


Flexibility  and  Facilities 
In  Children’s  Electronic  Textbooks 


Norshuhada  Shiratuddin 

Department  of  Information  Science,  University  of  Strathclyde,  Glasgow,  UK 
shuhada@dis.strath.ac.uk,  shuhada@uum.edu.my 

Monica  Landoni,  PhD 

Department  of  Information  Science,  Universit}^  of  Strathclyde,  Glasgow,  UK 
monica@dis.strath.ac.uk 


Abstract:  Design  of  electronic  books  should  take  into  account  the  diversity  of  learning  styles, 
intelligences  and  preferences  of  each  user.  This  project  main  aim  is  to  provide  high  level  of 
electronic  book  interaction  and  flexibility  by  matching  different  categories  of  electronic  pages 
(graphic  page,  talking  page,  hypermedia  page  and  web  page)  to  the  different  learning  styles.  Each 
page  exhibits  programs  and  activities  that  attempt  to  cater  for  each  learning  style.  The  same  content 
in  each  page  will  be  presented  in  four  different  presentation  modes.  In  addition  to  the  technical 
system  development,  a conceptual  model  of  children’s  electronic  textbook  will  also  be  proposed. 
This  model  is  based  on  Howard  Gardner’s  theory  of  Multiple  Intelligences  and  Philip  Barker’s 
generalization  of  electronic  books  model. 


Introduction  and  definition  of  electronic  book 

Educational  environment  stresses  the  importance  of  reading  materials:  textbooks,  reference  books,  encyclopaedias, 
magazines  and  newspapers  as  media  of  knowledge  carriers.  Hence  converting  printed  publications  especially 
textbook  into  interactive  electronic  form  should  prove  to  be  extremely  useful  and  helpful.  There  are  various 
definitions  of  electronic  book  (e-book).  Bonime  et  al.  (1998)  described  any  kind  of  information  ranging  from  a CD- 
ROM  title  to  an  online  interactive  database  including  collection  of  web  pages  as  electronic  book.  Barker  (1998) 
defined  e-book  as  essentially  a computer-based  information  storage  that  embeds  a book  metaphor.  More  precisely 
(Barker,  1999),  e-book  is  usually  considered  to  be  composed  of  a collection  of  reactive  pages  of  electronic 
information  that  are  organised  in  a thematic  way  and  that  exhibit  many  of  the  characteristic  features  and  properties 
of  a conventional  book.  Barker  categorises  e-book  into  ten  types  depending  on  the  types  of  information  embedded: 
textbooks,  picture  books,  talking  books,  moving  picture  books,  multimedia  books,  polymedia  books,  hypermedia 
books,  intelligent  books,  telemedia  books  and  cyberbooks.  While  Barker  categorises  ten  types  of  e-book,  Landoni  et 
al.  (1993)  classified  e-book  according  to  three  different  criteria:  books  which  are  portable  (e.g.  personal  digital 
assistant),  books  which  preserve  the  logical  structure  (i.e.  chapters,  sections  and  subsections)  and  lastly,  books  that 
support  both  the  physical  and  logical  aspects  of  a paper  book.  In  context  of  this  project,  definition  of  an  e-book  will 
be  defined  when  the  study  is  completed. 


Research  Question 

Children  learn  in  different  ways.  There  are  many  theories  on  how  people  learn  and  one  example  is  the  Multiple 
Intelligences  (MI)  theory  proposed  by  Howard  Gardner  (1993).  This  theory  states  that  there  exist  seven  intelligences 
(thus  seven  learning  styles):  verbal/linguistic,  logical-mathematical,  visual/spatial,  bodily-kinesthetic,  musical, 
interpersonal  and  intrapersonal  (to  date,  two  more  intelligences  have  been  added).  Based  on  this  theory,  Armstrong 
(1994)  wrote  a book  on  how  to  apply  this  theory  into  the  classroom  environment.  He  identifies  that  each  child  could 
learn  in  any  one  of  these  ways  or  through  a combination  of  several  ways.  With  these  different  learning  styles,  design 
of  e-books  should  take  into  account  these  diversity  and  attitudes  of  each  user  particularly  young  children.  Thus,  one 
question  can  be  extracted:  how  to  give  young  children  better  flexibility  so  as  to  cater  their  differences? 


Proposed  Solution:  Giving  E-Book  Flexibility 


There  are  many  methods  to  give  flexibility  in  e-books.  One  example  is  by  providing  tools  or  facilities,  which  can  be 
used  by  users  while  consulting  the  e-book.  Landoni  et.  al  (1993)  in  her  project  provided  users  with  facilities  such  as 
personalisation  (highlighting,  bookmarking,  note  taking),  searching  through  TOC  table  of  content  and  index,  and 
printing.  Other  examples  include  providing  system  adaptivity,  supporting  for  group-based  learning,  considering  the 
likelihood  that  teachers  will  use  tlie  program  in  class  and  including  different  levels  of  contents  in  the  books. 

It  is  the  intention  of  this  on  going  project  to  suggest  that  another  method  to  achieve  flexibility  is  through  flexible 
presentation  modes  that  include  programs  and  activities  which  match  each  learning  style.  The  following  gives  few 
examples  on  how  to  match  MI  theory  to  the  existing  e-book  model: 

(a)  Vcrbal/linguistic  - Is  matched  with  e-book  concept  of  kypeirnedia  through  programs  such  as  letting  users 
create  essays,  read  aloud  and  include  storytelling. 

(b)  Visual/spatial  - Is  matched  with  e-book  concept  of  graphic  page  through  programs  such  as  letting  users  draw 
and  paint  and  allowing  users  to  read  and  see  information  as  graphics,  maps,  charts  and  diagrams. 

(c)  Musical  “*  Is  matched  with  e-book  concept  of  talking  page  through  programs  such  as  combining  stories  with 
songs  and  letting  users  sing  along. 

(d)  Interpersonal  - Is  matched  with  e-book  concept  of  web  page  by  including  programs,  which  need  two  or  more 
users  at  the  same  time  and  group  project  or  discussion. 

Proposed  electronic  book  model 

Page  = Contents  + [Object  A + Object  B + Object  C + Object  D] 

Object  A = Graphic  page  + {Program  G1  | Program  G2  | . ..|  Program  Gn} 

Object  B = Talking  page  + (Program  T1  | Program  T2  | .. .|  Program  Tn} 

Object  C = Hypermedia  page  + (Program  HI  | Program  H2  | .. .|  Program  Hn} 

Object  D = Web  page  + (Program  W1  | Program  W2  | . ..|  Program  Wn} 


Conclusion 

In  conclusion,  our  e-book  shall  exhibit  four  different  concepts  and  include  a number  of  presentation  modes  and 
styles,  in  order  to  provide  readers  with  a high  level  of  flexibility.  Users  can  choose  between  the  four  different 
presentation  modes  when  he/she  opens  any  page.  Upon  completion,  this  project  will  be  used  to  prove  these 
assumptions: 

Hl:It  is  possible  to  address  diverse  learning  styles  by  providing  users  with  flexible  presentation  modes  of  each  page. 
H2:These  flexible  presentation  modes  are  probably  more  appropriate  to  use  and  as  such  give  e-book  high  level  of 
flexibility. 

H3:It  is  then  possible  to  suggest  that  due  to  this  high  level  of  e-book  flexibility,  users  are  more  likely  to  prefer  this 
type  of  e-book. 

H4:If  H1,H2  and  H3  are  proven,  then  we  could  postulate  that  an  additional  desirable  feature  for  e-book  would  be  to 
present  contents  by  mixing  many  presentation  modes. 
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Abstract:  The  Internet  is  rnaking  inroads  into  almost  all  quarters  of  our  society.  Education 
is  no  exception  to  that,  nor  should  it.  Exciting  new  ways  of  building  content,  of  delivering 
it,  of  interacting  with  our  students,  indeed  of  managing  our  educational  'industry'  can  be 
seen  to  evolve.  Even  if  one  may  not  wholeheartedly  support  all  these  innovations, 
particularly  for  all  those  with  an  inquisitive  mind  it  is  inspiring  to  see  education  being 
rethought  in  so  many  and  fundamental  ways.  Or  is  it? 

The  Internet  revolution  clearly  is  a technological  revolution.  It  pushes  developments  in  various  fields,  education 
being  no  exception  to  that.  However,  are  the  new  technologies  used  to  innovate  education,  that  is,  to  bring  about 
a genuine  transformation;  or  do  they  just  substitute  older  technologies  for  new  ones? 

An  educational  model  that  is  widely  implemented  these  days  - although  it  doesn’t  alway  go  by  this  name  - is  that 
of  the  extended  classroom.  As  the  name  suggests,  an  extended  classroom  takes  the  ordinary  classroom  as  its 
starting  point  and  extends  it,  mainly  spatially.  So  the  dominant  didactic  model  here  still  is  a teacher  who 
addresses  a group  of  students;  students  and  teacher  are  simultaneously  present;  the  teacher  takes  the  lead; 
students  may  ask  questions  but  those  only  serve  to  clarify  the  exposition  of  the  teacher;  students  seldom  interfere 
with  the  choice  of  content,  which  is  the  teacher’s  exclusive  realm;  students  hardly  ever  have  a say  in  the  didactics 
used,  this  also  is  the  teacher's  prerogative.  In  the  extended  classroom,  teacher  and  students  may  be  in  different 
locations,  the  distance  being  bridged  by  television,  videoconferencing,  audiographics  and  similar  technologies. 
Put  extremely,  with  the  extended  classroom  one  aims  to  mimick  a regular  classroom  to  the  largest  possible 
extent.  Hence  the  call  for  ever  larger  bandwith,  for  systems  that  allow  for  ’natural’  interactions  between  the 
teacher  and  the  remote  classroom,  etc. 

However  cleverly  designed  our  communication  systems,  however  large  our  bandwidth,  extended  classrooms  will 
always  be  second  best,  a substitution  for  the  real  thing,  only  useful  as  a means  to  reach  out  to  the  remote  student, 
perhaps  to  serve  larger  groups  of  students  and  achieve  some  economy  of  scale.  Lecture  notes  that  are  being  put 
on  the  web  as  a service  to  the  students,  or  e-mail  and  newsgroups  that  deliver  additional  support  only  serve  to 
underline  the  main  conclusion:  in  the  extended  classroom  technology  is  used  to  substitute  or  augment  existing 
teaching  modalities. 

Obviously,  there  is  nothing  inherently  wrong  with  this.  Technology  is  a means  to  an  end  and  should  be  used 
accordingly,  that  is,  how  we  see  fit.  However,  if  it  is  our  lack  of  imagination  or  if  it  is  the  rigidity  of  our 
institutions  that  prevent  us  from  using  technology  in  more  creative  ways,  then  we  should  not  rest  content.  We 
should  reexamine  our  educational  practices  and  ask  ourselves  to  what  extent  they  were  limited  by  existing 
technologies  and  to  what  extent  the  new  technologies  empower  us  to  innovate.  And  indeed,  it  is  our  claim  that 
we  only  stand  at  the  beginnings  of  truly  innovative  educational  practices  and  genuinely  new  didactic  principles. 

Distributed  learning  is  a term  that  is  often  used  in  this  context.  In  distributed  learning  systems,  there  are  no 
classrooms  anymore,  no  teachers  offering  students  prepackaged  chunks  of  education  carefully  aranged  to 
conform  to  the  needs  and  capacities  of  the  average  student.  Students  now  occupy  center  stage,  assembling  and 
arranging  content  that  fits  their  specific  needs;  they  take  it  in  when  and  where  it  suits  them  best,  perhaps  in  a 
didactic  style  that  optimally  matches  their  learning  style.  The  role  of  the  teacher  now  is  not  communicating 
knowledge,  but  to  make  it  available,  to  empower  students  via  intakes  and  assessments  and  help  them  assimilate 
knowledge,  to  offer  custom  support.  In  distributed  learning,  technological  innovation  is  not  the  push  behind 
educational  innovation,  as  is  the  case  in  the  extended  classroom.  Rather,  the  converse  holds  true,  educational 
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innovation  pushing  technology.  And  indeed,  distributed  learning  requires  specific  technologies  that  extend 
beyond  the  mere  facilitation  of  communication.  Technologies  are  needed  for  creating  learner  centered  content, 
for  the  delivery  of  such  content,  for  flexible  intake  and  assessment  procedures,  for  portfolio  management,  etc.  In 
our  presentation  we  will  discuss  these  technologies  at  some  length. 
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This  paper  describes  a study  aimed  at  investigating  the  impact  of  a technology-enabled  training  program  for  Project 
Executives  (PE's)  in  the  IBM  Corporation.  The  program  is  focused  on  a mentoring  approach  to  learning  whereby 
PE’s  considered  to  be  experts  in  specific  skills  coach  less  experienced  colleagues  in  the  development  of  critical 
skills.  Since  job  demands  prevent  ongoing  face-to-face  mentoring  relationships,  technology  which  enables 
collaboration  at  a distance  is  implemented  to  support  the  mentoring  relationship.  Of  interest  in  this  study  is  the 
effectiveness  of  instructional  design  approach,  the  impact  of  the  technology-enabled  learning  environment  on 
sustaining  mentoring  relationships,  and  the  nature  of  online  discussions  between  experts  and  students. 

IBM  PE's  are  responsible  for  managing  large  information  technology  contracts.  These  professionals  deliver  on 
commitments  to  customers,  ensure  customer  satisfaction,  secure  business  opportunities,  and  mentor  other  project 
executives.  Their  role  of  mentor  is  critical  to  the  professional  development  of  less  experienced  colleagues.  While 
traditional  face-to-face  mentoring  is  desired  and  encouraged,  job  demands  often  prevent  mentors  and  students  from 
sustaining  the  type  of  relationships  needed  for  the  transfer  of  critical  skills. 

In  the  current  study,  a combination  of  face-to-face  instruction  and  a technology  enabled  learning  environment  is 
used  to  develop  and  sustain  mentoring  relationships  among  expert  PE's  and  less  experienced  colleagues.  The  face- 
to-face  instruction  includes  a two  day  workshop  consisting  of  five  sessions  led  by  the  expert  PE’s.  The  purpose  of 
the  workshop  sessions  is  to  1)  initiate  a mentoring  relationship  among  expert  PE’s  and  a group  of  colleagues  and  2) 
to  begin  the  mentoring  process  which  includes  the  transfer  of  a specific  set  of  problem-solving  skills. 

At  the  conclusion  of  the  workshop  participants  are  introduced  to  e-mentor,  a Lotus  Notes  discussion  database 
which  can  be  accessed  using  a Lotus  Notes  Client  or  a Web  browser.  The  discussion  database  is  used  to  provide  an 
online  learning  environment  in  which  the  mentor-protege  relationship  is  preserved.  Mentors  post  mini-lessons  on 
topics  relevant  to  the  development  of  complex  problem-solving  skills  needed  in  customer  situations.  Participants 
respond  to  the  lessons  in  discussion  format,  post  questions  and  collaborate  with  one  another  in  solving  customer 
problems.  Mentors  are  available  to  assist  participants  with  appropriate  strategies  for  addressing  customer  problems. 

The  instructional  design  approach  used  in  the  workshop  and  online  is  based  on  cognitive  apprenticeship,  an 
instructional  approach  developed  by  Collins,  Brown  and  Newman  (1989).  Cognitive  apprenticeship  focuses  on  the 
development  of  higher  level  thinking  skills  such  as  problem-solving.  Because  of  its  focus  on  the  teaching  of 
cognitive  and  metacognitive  knowledge,  cognitive  apprenticeship  may  be  a more  appropriate  instructional  design 
model  for  online  mentoring  than  more  traditional  instructional  designs  such  as  a lecture  approach.  Whereas  text  and 
lecture-based  instructional  design  models  focus  on  teaching  concepts,  facts,  and  procedures  in  non-situated 
environments,  cognitive  apprenticeship  focuses  on  teaching  the  processes  and  strategies  used  in  expertise  and  how 
this  knowledge  is  used  to  solve  real-world  problems.  Snyder  (2000)  demonstrated  the  effectiveness  of  cognitive 
apprenticeship  as  an  instructional  design  approach  for  teaching  technical  skills  in  an  online  environment. 

Collins,  Brown,  and  Newman  (1989)  and  Collins  (1991)  identify  four  aspects  of  cognitive  apprenticeship 
learning:  content,  instructional  methods,  sequencing  of  instruction,  and  sociology.  Content  refers  to  the 
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different  types  of  knowledge  required  for  expertise  and  includes  domain  knowledge  and  strategic  knowledge. 

Domain  knowledge  consists  of  concepts,  facts,  and  procedures;  strategic  knowledge  refers  to 

knowledge  which  underlies  an  expert's  ability  to  make  use  of  concepts,  facts,  and  procedures  to  solve  problems. 

Instructional  methods  are  the  learning  activities  used  during  instruction  to  help  students  construct,  use,  manage, 
and  acquire  new  knowledge  (Collins,  Brown,  & Newman,  1989).  Seven  instructional  methods  are  recommended. 
These  methods  include  teachers  "modeling"  skills,  "coaching"  learners  as  they  attempt  to  mimic  expert  skills, 
providing  "scaffolding"  in  the  form  of  support  for  learners,  and  gradually  "fading"  support,  as  learners  become  more 
proficient.  Learners  are  also  encouraged  to  "articulate"  their  knowledge,  "reflect"  on  their  problem-solving 
processes,  and  "explore"  new  approaches  to  problem  solve  on  their  own. 

Sequencing  involves  the  staging  of  learning  whereby  tasks  are  presented  in  increasing  complexity  and  diversity 
so  that  students  develop  a broad  understanding  of  the  domain  of  expertise.  Sociology  deals  with  the  authenticity  of 
the  learning  environment.  Technological,  social,  time,  and  motivational  characteristics  of  real-world  situations  are 
designed  into  the  learning  environment  so  that  students  will  learn  when,  where,  and  how  the  knowledge  applies  to 
other  situations  (Collins,  Brown,  & Newman,  1989). 

The  PE  workshop  in  this  study  and  the  e-mentor  database  are  designed  to  incorporate  the  four  aspects  of 
cognitive  apprenticeship  learning.  In  the  workshop,  learning  takes  place  in  the  context  of  solving  a variety  of  real- 
world  complex  customer  problems.  Mentors  begin  sessions  with  an  introduction  a complex  customer  problem. 

Facts  and  concepts  about  the  customer  environment  are  presented  (domain  knowledge)  and  followed  by  modeling  of 
the  strategies  and  processes  used  to  solve  the  problem.  During  this  activity,  mentors  use  think-aloud  protocols  to 
model  the  strategic  thinking  required  to  solve  the  problem.  Next,  tips  and  techniques  (scaffolds)  for  remembering 
the  processes  and  strategies  are  provided  to  students.  Students  are  then  assigned  to  small  teams  of  six  and  work 
together  to  develop  a solution  to  an  existing  customer  problem.  The  problems  used  in  this  exercise  are  real-world 
and  are  brought  to  the  workshop  by  students.  Students  are  required  to  articulate  the  processes  and  strategies  used  to 
address  the  customer  problem  and  then  briefly  explore  alternative  approaches  to  problem-solving.  At  the  conclusion 
of  this  exercise,  mentors  model  their  thinking  with  respect  to  solving  the  team  problems.  When  the  two-day 
workshop  ends,  the  mentor-  protege  relationship  is  maintained  using  the  Lotus  Notes  discussion  database.  Workshop 
participants  use  the  database  to  request  help  from  colleagues  and  mentors  regarding  problems  experienced  in  the 
workplace.  Mentors  use  of  the  cognitive  apprenticeship  method  when  responding  in  the  online  environment. 

The  results  of  this  study  to  date  have  been  successful  in  that  this  combination  of  face-to-face  and  online 
mentoring  has  enabled  a community  of  IT  executives  who  are  geographically  dispersed  to  form  working 
relationships  and  share  knowledge  on  a regular  basis.  This  model  has  also  provided  participants  with  an  environment 
for  continuous  learning  on  a just-in-time  basis,  convenient  to  their  demanding  work  schedules.  The  instructional 
design  approach  has  been  notably  successful  in  providing  an  effective  way  for  PE’s  to  learn  new  skills.  Participants 
in  the  workshop  quickly  adopted  the  habit  of  thinking  about  problem-solving  from  a strategic  perspective  (that  is, 
identifying  the  processes  and  strategies  needed  to  solve  customer  problems  and  using  reflection  and  exploration 
techniques  before  identifying  a solution).  This  learning  approach  has  also  provided  a common  ground  for  PE’s  to 
work  together  to  solve  customer  problems.  Results  of  a survey  administered  to  participants  of  the  workshop 
indicated  high  ratings  with  respect  to  the  methods  fostering  working  relationships  among  PE’s,  the  value  of  the 
content  of  the  workshop,  its  structure  (instructional  design  approach)  and  the  value  of  the  e-mentor  database  in 
providing  a continuous  learning  environment  which  facilitates  collaboration. 
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Abstract:  This  work-in-progress  paper  describes  a pedagogical  approach  to  designing  an  online 
collaborative  learning  environment  that  scaffolds  learners  in  becoming  better  collaborative  learners. 
This  environment  is  based  on  Collaborative  Problem  Solving  (CPS)  instructional  theory.  The  paper 
will  overview  the  need  for  the  environment,  and  the  design  principles  used  to  design  and  develop  it. 


Need 


As  online  education  becomes  more  prevalent  as  a system  for  distance  education,  a need  arises  for  an  online 
structure  that  supports  instructors  and  learners  engaging  in  small  group  collaborative  problem  solving  or  project 
work  at  a distance  from  one  another.  Particularly,  there  is  an  increasing  need  for  online  learning  environments  that 
support  effective,  efficient  collaborative  processes,  such  as  those  used  by  expert  group  problem  solvers. 

When  online  environments  impose  restrictive  structures  or  do  not  account  fully  for  these  processes,  they 
may  hinder  the  flow  of  learning,  and  curtail  effective,  efficient  problem  solving.  Most  online  environments  that 
attempt  to  use  collaboration  as  a learning  strategy  incorporate  a smorgasbord  of  communications  technologies  that 
are  not  structured  according  to  research  into  the  most  effective  ways  to  scaffold  effective  collaborative  interactions. 
Furthermore,  the  applications  included  in  these  systems  (such  as  email,  threaded  discussions,  and  file  storage)  are 
generally  lower-level  communication  tools  as  opposed  to  tools  that  could  support  robust  collaboration.  Because 
they  do  not  use  a guiding  theory  that  is  pedagogically  based,  the  resulting  interaction  often  do  not  adequately 
address  the  social,  intellectual,  and  logistical  complexity  of  effective  collaborative  learning.  This  is  partially  due  to 
the  fact  that  until  recently,  no  comprehesive  theory  capable  of  support  effective  online  collaboration  existed. 

Specifically,  the  weakness  of  these  approaches  is  their  primary  reliance  on  a document-based  paradigm 
(such  as  Lotus  QuickPlace  or  Hyperwave),  conferencing-based  paradigm  (such  as  Microsoft  NetMeeting  or  Lotus 
Sametime),  or  room-based  paradigm  (such  as  BioMOO  or  Diversity  University)  (Miao,  Fleschutz,  & Zentel  1999). 
For  the  design  to  fully  support  a cohesive  collaborative  learning  approach,  the  strengths  of  each  of  these  approaches 
must  be  synthesized  into  a context-based  system  that  allows  for  a more  flexible  combination  of  people,  places,  tools, 
documents,  and  interactions  than  these  other  types  of  systems  afford,  thus  providing  robust  support  for  the 
communication,  collaboration,  and  coordination  necessary  to  learn  effectively  as  a small  group. 


Proposed  Solution 

We  propose  to  design,  develop,  implement,  and  evaluate  an  online  collaborative  learning  environment  that 
both  supports  pedagogically  sound  instruction  and  provides  appropriate  scaffolding  for  moving  learners  from  novice 


to  expert  collaborative  learners.  It  will  be  based  on  Collaborative  Problem  Solving  (CPS),  an  instructional  theory 
designed  for  effective,  efficient  small-group  problem  solving  (Nelson  1999).  This  theory  outlines  a detailed 
collaborative  learning  process  derived  from  a series  of  carefully  documented  case  studies  of  expert  learning  groups 
integrated  with  findings  from  current  research  in  areas  such  as  small  group  dynamics,  cooperative  learning,  and 
problem-based  learning.  The  goal  of  this  theory  is  to  provide  step-by-step  scaffolding  to  both  instructors  and 
learners  that  will  help  learners  make  the  difficult  transition  from  novice  to  expert  collaborative  learners. 

Unlike  existing  tools  that  merely  provide  a collaboration  platform,  this  online  system  will  be  designed  to 
serve  as  a robust  teaching  and  learning  environment  that  will  provide  guidance  for  both  instructors  (in  designing 
their  instruction  and  in  appropriately  monitoring,  supporting,  and  assessing  the  learners’  progress)  and  learners  (in 
developing  effective  collaborative  skills  while  learning  at  a distance).  This  learning  environment  will  be  capable  of 
scaling  to  any  number  of  small  groups  and  will  support  instruction  in  a variety  of  disciplines  desiring  to  engage 
learners  in  project-  or  problem-based  learning  around  complex,  real-world  problems.  Its  stracunc  wiii  afford 
support  for  every  phase  of  effective  small  group  collaboration,  from  building  readiness  to  providing  closure,  by 
providing  all  the  collaborative  learning  spaces,  tools,  and  resources  necessary  to  do  so,  such  as  “just-in-time 
instruction”  for  learners  as  they  progress  through  the  given  project  or  problem  scenario. 


Project  Goals 

The  goal  is  to  create  an  online  learning  environment  that  has  the  following  features  and  characteristics: 

• Teaches  and  supports  the  nine  phases,  not  just  isolated  activities,  of  Nelson’s  effective  collaborative 
problem  solving  process,  which  includes:  building  readiness,  forming  and  norming  the  groups,  determining 
a preliminary  problem  definition,  defining  and  assigning  roles,  engaging  in  an  iterative  collaborative 
problem-solving  process,  finalizing  the  solution  or  project,  synthesizing  and  reflecting,  assessing  products 
and  processes,  and  providing  closure  to  the  learning  experience, 

• Employs  proven  pedagogical  theory  and  arrays  tools  around  this  theoretical  structure, 

• Responds  to  the  needs  of  the  users  by  being  flexible,  customizable,  and  reconfigurable, 

• Runs  entirely  on  open  standards-based  Internet  protocols, 

• Enables  distributed  groups  to  learn  and  work  together  at  least  as  effectively  as  co-located  groups, 

• Provides  guidance  for  both  instructors  and  learners  in  effective  collaborative  learning,  and 

• Archives  the  work  of  small  groups  for  both  the  group’s  reference  and  for  research  analysis. 


Project  Design 

Our  design  approach  is  to  translate  each  element  of  CPS  theory  into  a corresponding  aspect  or  structure  of 
the  environment,  such  that  it  both  implicitly  and  explicitly  supports  effective,  efficient  collaborative  problem  solving 
philosophy  and  methods.  From  this  blueprint,  we  will  work  with  a software  designer  to  create  a data  model.  From 
the  data  model,  the  software  designers  will  create  a database-driven,  internet-based  collaborative  environment  that 
will  scale  to  any  number  of  small  teams.  Simultaneously,  we  will  begin  a paper  prototyping  phase  to  determine  the 
initial  visual  design,  working  with  a target  user  pool  of  instructors  and  students  at  the  two  distance  education 
entities.  We  will  then  employ  a rapid  prototyping  method  to  improve  our  visual  design  and  address  the  usability 
issues  of  instructors  and  learners  testing  alpha  versions  of  the  system.  Once  we  have  a beta  quality  environment,  we 
will  conduct  pilot  studies  with  instructors  and  learners  in  diverse  content  areas  and  geographical  locations.  Finally, 
we  will  evaluate  case  studies  of  the  system  fully  implemented  in  a variety  of  learning,  work,  and  research  settings 
and  use  the  data  to  refine  a final  version  that  we  will  release  for  use  on  the  local  servers  of  any  interested  partners. 
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Abstract:  This  paper  describes  the  implementation  of  a pilot  project  involving  the  formation 
of  networked  technology  learning  communities  designed  to  nurture  the  growth  and  sharing  of 
expertise  in  meaningful  uses  and  integration  of  technologies  into  teaching  and  learning  among 
college  faculty  members,  student  teacher  candidates  and  classroom  teachers.  The  paper 
focuses  on  issues  related  to  the  establishment  of  technology  learning  communities  and 
includes  brief  studies  of  selected  members  of  the  learning  communities. 


Introduction 

Recent  studies  (Trotter,  1997;  U.S.  Department  of  Education,  1998)  indicate  that  a large  majority  of 
classroom  teachers  in  the  U.  S.  continue  to  feel  inadequately  prepared  to  integrate  technology  into  classroom 
instruction.  This  work-in-progress  paper  describes  the  implementation  of  a pilot  project  that  focuses  on  ways  to 
address  this  problem.  The  project  involves  the  formation  of  networked  technology  learning  communities 
designed  to  nurture  the  growth  and  sharing  of  technological  expertise  among  college  faculty  members,  teacher 
candidates  and  classroom  teachers  in  the  integration  of  technologies  into  teaching.  The  project  studies  potential 
for  such  communities  to  enlarge  the  number  of  technologically  proficient  current  and  future  teachers  by 
increasing  the  number  of  college  faculty  members  who  model  the  use  of  technologies  in  their  teaching,  the 
number  of  teacher  candidates  exposed  to  technology-proficient  college  and  classroom  teachers,  and  the  pool  of 
technologically  proficient  classroom  mentor  teachers.  Ultimately,  the  goal  for  this  project  is  to  support  a cycle 
of  increased  meaningful  integration  of  technology  into  teaching. 


Technology  Learning  Communities 

Technology  learning  communities  provide  “a  culture  of  adult  learning  and  mutual  support” 
(McKenzie,  1999,  p.  71)  that  enable  individuals  with  different  skill  levels  to  assist  each  other  in  the  learning  of 
ways  to  integrate  technologies  meaningfully  into  teaching.  Technology  learning  by  members  occurs  in  both 
informal  and  formal  settings,  with  support  and  expertise  provided  when  requested  by  members  of  the 
communities.  These  communities  sustain  the  different  learning  styles  of  the  members.  The  strength  of  a 
learning  community  lies  in  the  emphasis  on  members  teaching  each  other  what  needs  to  be  learned  and  at  a 
pace  that  meets  individual  needs  and  preferences  (McKenzie,  1999).  Emphasis  in  the  pilot  learning 
communities  is  on  supporting  projects  that  have  a direct  impact  on  teaching  and  learning. 

Inherent  in  the  formation  of  learning  communities  with  membership  from  diverse  groups,  i.e.  college 
faculty,  college  students,  and  classroom  teachers,  are  many  challenges.  If  a learning  community  is  to  be 
successful,  membership  must  be  “flat  and  non-hierarchical”  (McKenzie,  1999)  in  the  sense  that  all  members  are 
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equal  partners.  Ideally,  the  learning  communities  should  have  an  impact  on  education  that  radiates  beyond  the 
communities.  The  focus  of  each  member  of  a community  must  be  on  ways  to  use  technology  to  improve 
teaching,  not  simply  on  the  development  of  individual  technological  expertise.  Members  of  a learning 
community  must  accept  the  responsibility  to  take  the  acquired  skills  and  knowledge  back  to  their  circle  of 
professionals.  Each  member  must  be  willing  to  build  on  acquired  skills  and  knowledge,  even  after  the  pilot 
project  ends. 

Each  ten  member  learning  community  in  this  project  is  composed  of  college  faculty  members,  teacher 
candidates,  and  preK-5  teachers,  with  varying  degrees  of  technological  expertise.  Criteria  for  membership  were 
established  prior  to  the  formation  of  the  learning  communities.  Student  members  were  selected  based  on 
meeting  those  criteria  and  documented  faculty  support  for  their  membership.  Faculty  members  and  K-5  teachers 
meeting  the  criteria  and  having  a range  of  technological  expertise  were  identified  and  invited  by  project  leaders 
to  participate  in  the  project.  The  members  of  the  current  learning  communities  appear  to  have  successfully 
established  a non-hierarchical  relationship  partly  due  to  the  selection  process.  Each  member  of  the  learning 
communities  is  equipped  with  a laptop  computer  and  Internet  access  to  facilitate  collaboration. 

In  the  initial  meeting  of  the  learning  communities,  members  were  surveyed  on  their  levels  of 
technology  expertise  and  discussed  their  learning  expectations.  This  information  assisted  the  coordinator  of  the 
learning  communities  in  planning  group  learning  sessions.  In  addition  to  these  face-to-face  meetings,  members 
of  the  communities  interact  using  email  and  a discussion  list.  The  discussion  list,  TechTalk,  provides  a 
mechanism  for  members  to  share  and  request  resources,  and  to  obtain  fast  response  to  technology-related 
inquiries.  With  support  from  learning  community  members,  the  college  faculty  members  are  increasing 
technology  use  in  the  courses  they  teach.  The  teacher  candidates  have  the  opportunity  to  assist  in  the 
implementation  of  technology-based  lessons,  to  design  lessons  and  materials  to  be  used  during  student  teaching 
with  preK-12  students,  and  to  work  with  the  college  faculty  members  as  they  integrate  technology  into  courses. 
The  classroom  teachers  are  sharing  and  expanding  their  expertise  in  integrating  technology  into  the  preK-5 
curriculum. 

Members  of  the  learning  communities  have  formed  partnerships  as  reflected  in  interviews  done  with 
one  student  and  one  faculty  member  in  the  communities.  Faculty  Member  A is  a beginning  technology  user, 
though  she  is  knowledgeable  about  the  Internet.  As  a college  histoiy  professor,  she  wants  to  explore  the  use  of 
the  Internet  not  as  an  electronic  library  reflected  in  the  web  sites  that  she  includes  in  her  syllabus  bibliography, 
but  as  a vehicle  to  encourage  her  education  students  to  think  critically  about  events  in  history.  She  finds  the 
learning  communities  helpful  in  learning  about  social  studies  issues  important  to  elementary  teachers  but 
receives  the  most  help  from  one-on-one  tutoring  from  individual  members.  She  is  especially  appreciative  of  the 
help  from  Student  B,  an  above  average  technology  user,  who  acquired  many  of  her  skills  at  the  college.  In 
addition  to  tutoring  Faculty  Member  A,  Student  B is  helping  two  classroom  teachers  in  her  learning  community 
to  teach  their  students  HyperStudio.  She  finds  that  the  learning  communities  aid  her  goal  of  being  a teacher 
because  through  this  collaboration  she  is  spending  time  in  a classroom  setting  working  with  children. 

Evaluation  of  the  progress  being  made  by  members  of  the  learning  communities  in  meeting  the  initial 
goals  of  this  pilot  project  is  encouraging.  This  project  is  providing  the  groundwork  for  modification  and 
expansion  of  future  technology  learning  communities. 


References 

McKenzie,  J.  (1999).  Howf  teachers  learn  technology  best.  Bellingham,  WA:  FNO  Press. 

Trotter,  A.  (1997).  Training  called  key  to  enhancing  use  of  computers,  poll  finds.  [On-line].  Available: 
http://www.edweek.org/ew/vol- 1 6/29jost.hl  6. 

U.S.  Department  of  Education,  National  Center  for  Educational  Statistics  (1998).  Teacher  survey  on 
professional  development  and  training. 

Acknowledgements 

This  project  is  supported  in  part  by  a U.S.  Department  of  Education  grant  (P342A990151).  However,  the 
contents  of  this  paper  do  not  necessarily  represent  the  policy  of  the  Department  of  Education,  nor  endorsement 
by  the  Federal  Government. 


Four  Challenges  for  TeleTeachers  in  Rural  Schools 

Ken  Stevens,  Memorial  University  of  Newfoundland,  Canada 

The  introduction  of  teleleaming  in  schools  in  Canada,  as  in  other  developed  coimtries, 
has  been  particularly  noticeable  in  rural  areas  and  has  been  influenced  by  declining 
school  emolments.  There  are  four  challenges  common  to  the  technological  enhancement 
of  rural  schools:  the  building  of  appropriate  infrastructures,  the  location  of  appropriate 
technologies,  the  development  of  pedagogy  for  teleteaching  and  the  need  for  on-going 
professional  development. 

Challenge  One:  Building  Appropriate  Structures  (Digital  Intranets) 

By  constructing  web-sites  both  teachers  and  students  in  a rural  school  can  enable  people 
located  almost  anywhere  to  connect  with  them  and  learn  about  their  programs  and 
activities.  In  Canada  regional  and  national  educational  networks  link  classes  in  schools 
on  dispersed  sites.  Stem-Net  in  Newfoimdland  and  Labrador  in  eastern  Canada  is  the 
provincial  arm  of  the  federal  SchoolNet  System.  These  federated  electronic  structures 
enable  teachers  to  extend  the  curriculum  available  to  learners  in  dispersed  and,  often, 
very  isolated  sites.  Schools  in  rural  Canada  are  developing  open  and  flexible  academic 
and  administrative  structures  as  they  electronically  link  with  one  another.  These  new 
structures  exist  within  the  traditional  and,  by  comparison,  closed  school  systems  in  each 
school  district.  In  the  open  educational  structure  of  teleteaching  within  digital  intranets, 
constructed  to  link  schools  for  the  delivery  of  specific  courses  for  some  students, 
participating  institutions  academically  and  administratively  interface  for  that  part  of  the 
school  day  during  which  classes  are  being  taught.  This  is  a very  different  educational 
structure  from  the  traditional  and,  by  comparison,  closed  educational  enviroiunent  of  the 
autonomous  school  with  its  own  teachers  and  its  own  students.  In  Canadian  networks 
such  as  Stem-net,  each  site  becomes  an  inter-dependent  part  of  a virtual  school,  without 
which  it  could  not  provide  all  the  courses  that  local  people  are  increasingly  accessing. 

Challenge  Two:  Finding  Appropriate  Technology 

As  schools  become  teaching  and  learning  sites  within  digital  Intranets,  there  is  increasing 
demand  for  adequate  technological  support  and  co-ordination.  Few  teachers  have  the 
skills  to  provide  adequate  technological  support  for  the  increasing  numbers  of  computers 
and  computer  users  in  their  schools.  The  result  is  that  the  emerging  teaching  and  learning 
infrastructure  in  many  places  is  often  very  fragile.  An  essential  aspect  of  the  development 
of  open  electronic  classes  is  the  coordination  of  both  hardware  and  software  between 
schools.  Without  coordinated  technology,  schools  cannot  fully  participate  in  electronic 
networks.  However,  the  purchase  of  appropriate  hardware  and  software  is  a matter  of 
confusion  for  many  principals,  teachers  and  School  Boards  who  seek  support  and  advice. 
Many  rural  schools  with  open  electronic  classes  realize  that  the  successfiil  administration 
of  a network  requires  local  technical  support.  Unless  adequate  technical  support  systems 
can  be  established,  electronic  networked  classes  could  be  curtailed  by  teachers  who  could 
argue,  with  justification,  that  there  is  insufficient  back-up  to  justify  their  investment  in 
teleleaming.  The  need  to  have  people  in  schools,  or  available  to  schools,  to  maintain 
information  technology  hardware  and  software  is,  for  many  teachers,  a matter  of  growing 
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importance.  In  geographically  isolated  communities  the  assurance  of  adequate  technical 
backup  is,  however,  often  particularly  difficult  to  provide  for  teachers  and  principals.  In 
schools  in  Newfoundland  and  Labrador,  the  majority  of  which  are  designated  rural, 
teleleaming  often  depends  on  the  goodwill  of  one  or  two  teachers  with  enough 
knowledge  to  keep  computers  going  when  they  mal-function. 

Challenge  Three:  Pedagogy  for  TeleTeaching 

In  the  changing  technological  environment,  teachers  often  have  a choice  of  teaching  face 
to  face  and,  for  part  of  a school  day,  providing  or  receiving  instruction  for  their  students 
at  a distance.  This  can  be  done  in  synchronous  or  asynchronous  mode,  with  a range  of 
technologies,  and  with  a range  of  learners.  The  use  of  information  technologies  in  schools 
has  been  vigorously  promoted  by  the  federal  agency.  Industry  Canada  (Information 
Highway  Advisory  Council,  1995;  1997).  Students  often  have  more  independence  in 
managing  their  learning  in  open  electronic  classes  but  most  have  to  be  assisted  by 
teachers  in  the  setting  of  goals,  the  meeting  of  deadlines  and  in  evaluating  their  progress 
(Stevens,  1996).  Teachers  are  effective  in  open  electronic  classes  if  they  can  be  flexible 
in  ways  that  they  allow  students  to  participate  in  on-line  lessons.  Strategies  and  protocols 
for  on-line  teaching  have  to  be  developed  between  participating  schools  if  all  students  are 
to  be  able  to  fully  participate.  The  introduction  of  a rural  school  to  an  open  electronic 
network  considerably  improves  its  resource  base  for  both  teachers  and  learners,  but  does 
not  solve  all  of  its  problems.  It  is  often  difficult  to  coordinate  the  timetables  of  networked 
schools  and  a considerable  measure  of  inter-institutional  and  intra-institutional 
cooperation  is  required  to  develop  detailed  and  effective  plans  for  collaboration.  The 
need  for  rural  networked  schools  to  have  a close  relationship  with  the  suppliers  of 
technology  has  become  increasingly  apparent  to  schools,  school  boards  and  to  technology 
retailers.  Rural  schools  with  teleleaming  infrastmctures  require  expert  advice  and  support 
from  technology  suppliers  at  the  network  rather  than  at  the  individual  school  level.  There 
are  several  immediate  pedagogical  challenges  to  be  considered  for  effective  teaching  in  a 
digital  Intranet: 

1 . Teachers  have  to  learn  to  teach  from  one  site  to  another.  This  is  fundamental  to  the 
success  of  teleteaching.  Teaching  face  to  face  and  on-line  are  different  skills. 

2.  Teachers  have  to  learn  to  teach  collaboratively  with  colleagues  from  multiple  sites. 

3.  Teachers  have  to  judge  when  it  is  appropriate  to  teach  on-line  and  when  it  is 
appropriate  to  teach  students  in  traditional  face-to-face  ways.  These  judgements  have 
to  be  defended  on  the  basis  of  sound  pedagogy. 

Challenge  Four:  Professional  Development  for  TeleTeaching 

A difficulty  many  teachers  face  in  Canadian  schools  is  simply  finding  the  time  in  their 
busy  days  to  learn  about  information  and  communication  technologies.  It  is  impossible 
for  some  teachers  to  do  this,  let  alone  reflect  on  how  new  technologies  can  be  integrated 
into  their  classrooms  and  their  professional  lives.  There  is  not  only  a widespread  need  for 
professional  development  for  teleteaching;  there  is  a need  for  it  to  be  provided  on  a 
continuous  basis.  A second  challenge  facing  many  teachers  is  finding  effective  ways  of 
integrating  information  and  commxmication  technologies,  including  the  Internet,  into 
teaching  and  learning.  It  is  not  difficult  to  add  information  and  communication 
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technologies  to  any  classroom  but  it  requires  considerable  planning  to  integrate 
technological  developments  so  that  both  teaching  and  learning  are  enhanced.  A third 
challenge  for  the  professional  development  of  teachers  is  to  be  able  to  justify,  on  the  basis 
of  professional  judgements,  why  the  Internet  and  other  technologies  should  be  used  in 
teaching  and  learning  at  all. 

Conclusion 

The  introduction  of  inter-school  electronic  networks  has  added  a new  dimension  to  rural 
education  in  Canada  and  in  other  developed  countries.  In  Canada  and  in  many  other 
places,  teachers  and  researchers  are  seeking  appropriate  ways  of  using  existing  as  well  as 
emerging  technologies  to  enhance  teaching  and  learning.  The  teachers  and  researchers 
who  are  collaborating  in  the  development  of  new  electronic  structures  for  delivering 
education  to  dispersed,  rural  sites  in  Atlantic  Canada  are  very  conscious  of  being 
pioneers.  Both  realize  there  is  a lot  to  learn  to  about  both  teaching  and  learning  in  this 
new  environment. 
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Abstract:  The  paper  presents  an  experimental  validation  of  learning  effectiveness  of  three  different 
types  of  group  interaction  for  computer  mediated  collaborative  learning  and  problem  solving.  The 
assumption  underlying  this  research  is  that  the  form  in  which  knowledge  is  shared  in  collaborative 
learning  is  a substantial  for  cognitive  construction  and  reconstruction.  The  experiment  revealed  that 
the  learning  effectiveness  is  influenced  significantly  by  the  mode  of  group  interaction  that  depends 
on  the  extend  to  which  students  share  their  learning  not  only  as  results  but  also  as  a process  of 
knowledge  acquisition  and  creation  by  a direct  interaction. 

Theoretical  background  and  research  question 

Rapidly  changing  technologies  generate  new  requirements  for  creating  computer  environments  that  support  learning 
in  the  broad  context  of  global  networking  communities.  Different  theoretical  perspectives  emphasise  collaboration 
as  a successful  and  powerful  activity  for  learning  and  problem  solving. 

Collaboration  creates  an  extension  of  the  internal  cognition  of  the  personality  in  outside  world  (Perkins,  1993). 
According  Salomon  (1993)  the  individuals’  and  the  shared  group  cognitions  interact  with  one  another  in  a spiral-like 
fashion  whereby  individual  inputs,  through  students’  collaborative  activities,  affect  the  nature  of  the  joint,  shared 
system,  which  in  turn  affects  their  cognitions.  It  is  emphasised  that  reconstruction  of  individual  cognition  requires  a 
profound  and  mutual  understanding  of  collaborators’  perspectives  and  shared  interpretations  of  the  problem.  Only 
the  knowledge  that  is  meaningful  for  individuals  could  be  internalised. 

The  assumption  underlying  this  research  is  that  the  form  in  which  knowledge  is  shared  in  collaborative  learning  is  a 
substantial  for  cognitive  construction  and  reconstruction.  The  main  research  question  is:  Is  the  mode  of  group 
interaction  a predictor  of  learning  effectiveness  in  a computer  based  collaborative  problem  solving  design? 


Collaborative  scenarios 

Three  scenarios  of  problem  solving  group  interaction  have  been  designed,  based  on  the  assumption  that  knowledge 

exists  in  three  forms:  as  individual  mental  constructs  and  precepts,  as  knowledge  in  action,  and  as  knowledge 

embodied  in  artefacts  (Hedlund  8l  Nonaka,  1994) 

1.  Distributed  interaction^  Group  members  work  autonomously  and  produce  intermediate  artefacts  embodying 
their  knowledge  and  visions,  which  are  passed  to  the  other  members.  This  circuit  is  repeated  until  all  group 
members  reach  a common  vision  of  the  problem.  The  process  of  knowledge  acquisition,  creation  and 
internalisation  is  individual. 

2.  Moderated  interaction^  Interaction  process  is  facilitated  by  a group  moderator  (the  role  is  taken  by  one  of  the 
group  members)  adjusting  individually  produced  artefacts  until  a common  group  vision  is  reached.  The 
representations  of  individual  cognitive  structures  are  not  directly  accessible  but  group  members  are  involved  in 
the  process  of  negotiation  of  meanings  and  ideas  that  take  place  between  them  and  the  group  moderator. 

3.  Shared  interaction.  Group  members  interact  directly  by  synchronous  activity  and  common  efforts  to  solve  the 
problem  as  a group.  They  share  their  knowledge  in  action.  Knowledge  is  communicated  in  the  process  of  its 
acquisition  and  creation  and  is  not  mediated  by  intermediate  artefacts.  Collaborative  actions  of  students  are  the 
individual  inputs  toward  shared  cognition. 

Experimental  validation 
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The  learning  effectiveness  of  the  three  modes  of  group  interaction  was  tested  experimentally.  The  experimental 
design  was  a random  assignment  pre-test  post  test  control  group.  The  independent  variable  was  the  mode  of  group 
interaction  with  three  levels  - “Distributed”  “Moderated”  and  “Shared”. 

The  dependent  variable  was  learning  effectiveness  operationalised  as  individual  learning  achievements  in  terms  of: 
fluency  (number  of  concepts  representing  individual  understanding  of  the  problem),  flexibility  (variety  of  concepts 
and  ideas),  knowledge  acquisition  (number  of  new  concepts  acquired),  individual-to-group  transfer  (individuals’ 
inputs  in  the  group  solution),  group-to-individual  transfer  (concepts  derived  from  the  group  solution), 
reconfiguration  (changes  in  the  concepts’  structure),  retention  (contra  indicated  by  the  reduced  concepts),  and 
creativity  (new  concepts  and  ideas  generated  individually  after  the  collaboration). 

Twenty-six  students  (6  groups)  of  University  of  Twente,  Faculty  of  Educational  Science  and  Technology,  enrolled 
in  “Linear  and  Hypermedia”  course,  were  selected  as  experimental  subjects  and  were  randomly  assigned  to  the  three 
collaborative  scenarios.  The  group  task  was  formulated  as  an  open-ended  problem.  Concept  mapping  technique  was 
chosen  for  representation  and  communication  of  knowledge  because  of  its  effectiveness  as  a tool  for  extemalisation 
and  representation  of  cognition.  (Stoyanov  & Kommers,  1999).  Data  was  analysed  applying  one-way  ANOVA 
procedure  by  SPSS  9.0  statistical  package. 

Results  and  Discussion 

The  results  are  significantly  predictive  for  the  learning  effectiveness  of  the  three  group  interaction  scenarios.  The 
mode  of  group  interaction  influences  significantly  the  knowledge  fluency  (Sig.  = .047),  concepts  acquisition  (Sig.  == 
.045),  and  group-to-individual  transfer  (Sig.  = .004).  On  these  criteria  students  in  Shared  groups  scored  significantly 
higher  than  students  in  Moderated  and  Distributed  groups. 

All  three  scenarios  enabled  students  to  present  and  incorporate  their  knowledge  in  the  group  process.  No  significant 
difference  was  found  on  individual-to-group  transfer. 

Shared  mode  was  highly  beneficial  for  acquiring  knowledge  both  from  each  other  and  from  taking  new  perspectives 
in  exploring  the  problem  space.  A significant  part  of  the  knowledge  that  was  elaborated  in  group  session  were 
incorporated  as  a meaningful  part  of  students’  cognitive  structures  and  transferred  to  individual  cognitions. 

Students  working  in  Moderated  mode  incorporated  less  new  concepts  in  their  personal  cognitive  structure  than 
students  in  Shared  mode.  Only  those  aspects  of  the  group  solution  that  were  developed  by  the  personal  participation 
of  a particular  student  were  internalised  in  his/her  cognition. 

Regardless  that  students  in  Distributed  mode  had  a broad  access  to  all  other  students’  knowledge  representation  they 
were  not  able  to  gain  enough  from  the  externalised  experience  of  the  others. 

Both  the  Shared  and  Moderated  mode  proved  their  potential  for  reconstruction  of  the  individual  cognition 
represented  mainly  by  reshaping  of  map  spatial  configuration.  Significant  interdependence  between  common  group 
problem  solution  and  the  individual  outputs  was  found.  Students  working  within  Distributed  mode  resist  on  their 
prior  conceptual  structure. 

No  significant  difference  was  found  on  criteria  of  knowledge  flexibility.  The  three  types  of  groups  were  able  to 
explore  the  problem  space  from  different  perspectives  and  within  different  conceptual  paradigms. 

Surprisingly,  the  individual  creativity  was  not  influenced  by  the  mode  of  group  interaction. 

Nevertheless  no  significant  difference  was  found  (Sig.  = .345),  it  should  be  stated  that  students  working  in  Shared 
mode  showed  a lower  level  of  retention  of  their  initial  concepts  than  the  students  in  the  other  two  modes. 

In  summary,  the  experiment  revealed  that  the  learning  effectiveness  is  influenced  significantly  by  the  mode  of  group 
interaction.  In  general.  Shared  interaction  scenario  proved  to  be  the  most  effective  in  collaborative  learning  and 
problem  solving.  These  leads  to  the  conclusion  that  the  learning  effectiveness  depends  on  the  extend  to  which 
students  share  their  learning  not  only  as  results  but  also  as  a process  of  knowledge  acquisition  and  creation  by  a 
direct  interaction. 
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Abstract:  The  use  of  databases  involves  assigning  resources  to  users  with  changing  access  rights. 
How  will  the  information  be  presented  to  the  right  users?  How  can  resources  be  reused  for  different 
audiences  and  how  should  access  rights  be  provided.  Who  has  the  right  to  change,  who  can  read  it 
and  who  is  owner  of  the  objects  in  a resource  base? 


Introduction 

The  faculty  of  Educational  Science  and  Technology  started  a faculty  wide  implementation  of  Teleleaming  with  the 
TeleTop  project  in  1997.  The  aim  of  the  project  was  to  support  teachers  and  students  with  a web  based  database 
driven  environment.  Two  years  later  the  entire  university,  including  ten  faculties,  is  ready  to  the  implement  the 
Teletop  system.  How  reuse  of  material  should  be  provided  is  a point  of  research  because  content  management  can 
make  the  use  of  objects  more  efficient. 

The  problem  of  assigning  resources  to  people 

Scaling  up  a course  management  system  means  that  more  information  will  be  stored.  Reusing  includes  retrieving 
stored  information.  Finding  relevant  material  and  assigning  this  material  to  the  right  users  is  a problem  because  of 
the  large  amount  of  objects  and  the  large  scale  of  users.  A second  point  of  attention  is  the  use  of  access  rights. 
Access  rights  are  used  to  give  users  rights  to  read  or  change  objects.  Using  databases  includes  controlled  access  to 
different  objects  for  different  users. 

Objects: 

Every  object  has  users  that  can  read  or  use  the  object,  users  who  are  able  to  change  or  create  the  object  and  users 
that  can  actual  delete  an  object.  These  rights  are  set  in  the  access  control  list  and  every  database  uses  some  sort  of 
access  control  list  to  control  access  objects  in  a database.  These  objects  can  consist  of  (elements  of)  learning 
material,  course  descriptions,  assignments,  multimedia  fragments,  HTML  pages  or  any  other  type  of  digital  content. 
Controlling  access  to  information  is  needed  so  that  copyrights  or  confidential  material  can  be  stored  in  a protected 
environment;  i.e.  an  environment  that  is  not  accessible  for  unauthorized  users. 

Users: 

Users  are  mostly  unaware  of  these  access  control  lists,  because  software  developers  make  these  environments  as 
easy  as  possible  by  not  showing  objects  that  users  are  not  authorized  to  see.  The  problem  in  most  cases  is  that 
objects  may  be  accessible  for  one  user,  but  not  for  another.  Resources  available  for  teachers  may  not  be  accessible 
for  students.  Assigning  students  to  these  resources  may  result  in  irrelevant  links  for  the  students.  This  makes  reuse 
for  different  users  and  audiences  a difficult  issue.  This  problem  has  not  earlier  been  identified  because  most  web- 
based  environments  are  still  pilots  or  prototypes.  A faculty  wide  implementation  includes  a large  number  of  users 
who  can  create  objects  and  even  more  users  that  should  be  able  to  use  these  objects.  Dealing  with  a large  scale  of 
objects  and  really  reusing  these  in  database  driven  courses  is  a new  development. 


Solution  for  large  numbers  of  objects 

Maintaining  access  control  for  every  single  object  in  a very  large  resource  base  is  not  manageable.  Managing  many 
(more  than  10.000)  users  over  many  (more  than  10.000)  objects  is  a realistic  example  in  our  situation  at  the 
University  of  Twente.  Combining  a number  of  related  objects  in  a course  is  a solution  to  make  control  access  rights 
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manageable.  In  this  way  it  is  possible  to  assign  a set  of  objects  to  several  predefined  user  groups  comprised  of 
teachers  or  students.  One  group  can  be  assigned  the  role  of  author  and  other  groups  as  readers.  Copying  objects  to 
new  courses  can  involve  reuse  of  objects.  This  method  has  been  sufficient  until  now,  because  in  most  cases 
instructors  wanted  to  reuse  only  the  material  created  by  themselves.  Reuse  as  described  gives  the  possibility  to  make 
changes  to  objects  for  the  new  audience. 

Making  objects  more  flexible 

Considering  the  results  of  reuse  we  recently  developed  a new  model  for  access  rights.  This  model  is  based  on  roles 
that  are  assigned  to  user  groups.  A user  group  is  a list  of  people  who  can  access  the  same  functionality  in  a database. 
So  the  access  control  of  every  object  in  a course  depends  on  the  role  a user  has.  Three  roles  where  identified: 
Teacher,  student  and  administrator.  Using  roles  makes  reuse  possible  by  employing  an  abstract  access  level  that  can 
be  used  over  courses  and  provides  a consistent  control  of  rights.  Objects  can  be  used  in  every  course  without 
changing  access  control  lists  of  involved  courses.  The  development  of  a resource  browser  is  in  progress.  In  the  first 
place  reuse  of  objects  within  a course  is  being  researched,  in  the  second  place  exchange  of  objects  between  courses 
will  be  researched. 

Challenge,  future  plans 

In  the  next  years  reuse  of  objects  at  the  university  level  should  be  supported.  Objects  should  be  reused  within 
courses,  between  courses  and  faculties.  Even  different  universities  should  be  able  to  reuse  objects  in  courses. 
Therefore,  interfaces  should  be  developed  that  support  users  selecting  the  right  resources.  Also,  access  right  models 
have  to  be  developed.  Another  objective  of  our  research  should  be  directed  at  the  ownership  of  objects;  i.e. 
copyrights.  Ownership  can  be  seen  as  part  of  the  access  rights  involved  with  objects.  Is  a user  who  creates  an  object 
the  owner  and  is  this  user  the  only  one  who  may  reuse  or  add  or  edit  this  object?  Who  owns  such  copyrights?  All 
these  points  need  further  research.  In  this  respect  it  is  important  to  note  that  a number  of  organizations  involving 
standards  for  learning  technology  and  related  issues  are  aware  of  these  problems  and  are  trying  to  find  solutions,  but 
until  now  there  is  not  a standard  way  to  handle  large  scale  resource  bases.  Examples  of  these  organizations  are  the 
Instructional  Management  Systems  project  (IMS)’,  the  Alliance  of  Remote  Instructional  Authoring  and  Distribution 
Networks  for  Europe  (ARIADNE)^,  the  IEEE  1484P  Learning  Technology  Systems  Architecture  (LTSA)^, 
Advanced  Distributed  Learning  (ADL)'’  and  the  Aviation  Industry  CBT  Committee  (AICC)^. 


‘ IMS:  http://www.imsproject.org 
^ ARIADNE:  http://ariadne.unil.ch 
^ LTSA:  http://www.manta.ieee.org/pl484 
^ ADL:  http://www.adlnet.org 
^ A ICC:  http://www.aicc.org 
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Abstract:  As  the  Internet  brings  new  means  of  presenting  information  to  learners,  the  need  for 
meaningful  assessment  increases.  We  describe  the  design  and  development  of  a computer  tool  to 
facilitate  a particular  fonn  of  assessment  that  can  be  meaningful  in  online  learning  contexts.  The 
tool  combines  a web  browser  with  an  annotation  facility,  a database,  and  special  means  for 
visualizing  assessment  data  captured  with  the  tool. 

Introduction: 

As  educators  explore  the  many  possibilities  for  delivering  instruction  online  via  the  Internet,  they  tend  to  focus 
primarily  upon  delivering  presentations  of  information.  The  forms  of  presentation  include  web  pages  and  web  sites, 
text,  images,  audio  clips,  video  clips,  queries  to  databases,  and  Java  applets  that  run  simulations.  In  addition  to  the 
information  itself,  the  Internet  also  enables  the  online  gathering  and  experience  of  groups  of  people  who  might  (or 
might  not)  be  physically  distant  from  one  another.  The  online  communities  of  learners  can  be  formed  such  that  the 
members  are  all  at  a very  similar  level  of  expertise  in  the  subject  they  are  studying.  What  is  largely  missing  from 
these  innovations,  however,  is  a form  of  educational  assessment  that  is  well  adapted  to  the  new  patterns  of  learning. 

Assessment  is  sometimes  regarded  as  the  unpleasant  part  of  education,  in  part  because  traditional  testing  is 
sometimes  distasteful  to  students  and  because  assessment  is  typically  considered  less  important  to  the  presentation 
of  information.  However,  studies  of  assessment  in  support  of  learning  (rather  than  in  support  of  administration  or  of 
college  admission,  for  example)  have  lead  to  recommendations  for  alternative  assessment  methods  such  as 
performance  assessment,  authentic  assessment,  portfolio-based  assessment,  and  continuous  assessment  (see 
Tanimoto,  1998).  These  methods  offer  the  possibility  to  make  assessment  a more  "natural"  part  of  the  learning 
process,  and  to  permit  learning  to  occur  with  less  stress  and  in  ways  more  in  line  with  workplace  goals.  Yet  these 
methods  are  often  difficult  to  implement  because  they  may  require  more  teacher  training,  more  teachers’  time  spent 
on  assessment,  and  because  their  results  may  seem  to  be  more  subjective  than  the  results  of  standardized  tests.  We 
have  developed  a software  tool  for  assessment  to  help  us  explore  one  approach  to  assessing  online  learning.  The 
tool,  called  INF  ACT,  permits  the  assessment  to  be  performed  unobtrusively  in  a manner  reminiscent  of 
psychologists'  observations  from  behind  the  glass  wall. 

Pedagogical  Approach 

Our  approach  to  assessment  takes  place  within  a particular  pedagogical  methodology.  We  have  applied  the 
methodology  in  two  subject  areas:  high-school  physics  and  undergraduate  statistics.  Students  in  the  class  are 
divided  into  groups  of  4-6  students  each.  An  instructional  unit  lasts  one  week.  On  Monday,  a problem  that  requires 
conceptual  thinking  is  posed  by  the  teacher.  Each  student  in  the  class  is  told  to  make  her/his  own  interpretation  of 
the  problem  and  suggestion  for  solving  it  and  to  write  and  post  a newsgroup  entry  using  a specially  modified  version 
of  the  HyperNews  software  from  the  University  of  Illinois.  The  posting  is  due  by  12  noon  on  Tuesday.  This  initial 
post  is  invisible  to  other  students  until  a virtual  curtain  is  lifted  shortly  after  noon  on  Tuesday,  at  which  time  it 
becomes  visible  to  the  other  members  of  the  student's  group.  The  students  are  then  required  to  respond  to  each 
other's  postings  in  an  online  discussion,  and  they  are  told  to  come  up  with  a group  solution  by  Friday.  Over  the 
course  of  the  week,  their  postings  accumulate,  creating  an  archive  which  feeds  our  assessment  tool,  INFACT. 


INFACT  stands  for  "Integrated  Networked  Facet-Based  Assessment  Capture  Tool."  It  is  used  by  teachers  to  view 
and  annotate  student  newsgroup  postings  in  such  a way  as  to  build  up  a database  of  assessment  information  that  can 
be  used  to  track  student  and  group  progress  in  understanding  the  concepts  of  the  instructional  unit.  INFACT  makes 
use  of  catalogs  of  standard  misconceptions  that  students  commonly  have  about  a subject.  Each  misconception  is 
called  a facet,  and  facets  are  grouped  into  clusters  such  that  all  facets  in  a cluster  represent  misconceptions  about  the 
same  concept.  A facet  catalog  must  be  compiled  in  advance  of  using  INFACT  for  assessment.  However,  it  is 
possible  to  refine  a facet  catalog  in  the  course  of  assessment.  INFACT  makes  the  assumption  that  facets  can  be 
linearly  ordered  by  their  "level  of  expertise"  (see  Minstrell  1992  and  Hunt  and  Minstrell  1994  for  details). 

The  Design  of  INFACT 

INFACT  comprises  the  following  components:  an  integrated  web  browser,  a database  (currently  an  MSQL 
database),  dialogs  for  creating  and  editing  assessment  items,  and  a visualization  component.  The  browser  component 
permits  the  teacher  to  load  and  view  HyperNews  postings  in  essentially  the  same  form  that  the  students  see  when 
they  read  and  post.  This  component  also  supports  creating  selections  of  text  within  the  document  being  viewed. 
The  INFACT  software  determines  (insofar  as  possible  given  the  limitations  of  available  browser  software)  the  start 
and  end  positions  within  the  document  where  the  selection  occurs.  Selections  of  this  form  can  then  be  annotated. 

Annotations  of  two  types  are  pennitted:  facet  annotations  and  free  annotations.  A facet  annotation  represents  an 
atomic  assessment  item,  and  it  consists  of  the  text  selection,  a reference  to  the  document  in  which  the  text  occurs, 
identification  of  the  student  who  authored  the  text,  identification  of  the  assessor,  reference  a particular  facet  for 
which  this  text  provides  evidence,  a certainty  value  given  by  the  assessor,  the  date  of  authoring  and  the  date  of 
assessment.  A free  annotation  is  simply  a comment  associated  with  the  selection.  The  assessor  can  check  a box 
associated  with  each  annotation  to  make  it  visible  to  the  student.  The  database  is  used  to  keep  track  of  the 
annotations  and  to  make  assessment  items  retrievable  by  student,  by  facet  cluster,  by  date,  etc.  The  database  also 
holds  the  facet  catalog.  The  visualization  portion  on  INFACT  permits  the  user  (usually  the  teacher)  to  call  up 
assessment  data  by  student,  by  group,  by  date,  or  by  facet  cluster.  The  user  can  also  view  graphs  that  help  to  reveal 
trends  in  the  data.  Two  types  of  graphs  supported  by  INFACT  are  (a)  histograms  of  facet  frequencies,  and  (b)  facet 
transition  diagrams.  The  latter  show  how  one  or  more  students’  misconceptions  about  a concept  change  over  time. 

As  a teacher  reads  student  postings  with  INFACT  and  makes  annotations,  s/he  can  use  INFACT  to  send  feedback  to 
the  students.  INFACT  supports  two  means  of  feedback:  direct  email  and  student-visible  annotations.  Direct  email 
permits  the  teacher  to  send  a message  to  a student  in  the  context  of  an  assessment  item.  This  can  be  used  to  get 
clarification  from  the  student  about  something  not  clear  from  the  posting.  It  can  also  be  used  to  commend  the 
student  or  to  make  a suggestion.  However,  its  main  value  over  email  from  ordinary  mail  programs  is  the  automatic 
association  created  between  the  message  and  the  assessment  context  in  which  it  originates.  INFACT  makes  it 
possible  for  the  teacher  to  include  a multiple-choice  question  in  the  message  that  when  answered  online,  provides 
additional  evidence  in  favor  of  one  facet  or  another  within  the  facet  cluster  associated  with  the  assessment  item.  For 
a screen  shot  of  INFACT,  see  http://www.cs.washington.edu/research/metip/crlt/infact-l.gif 
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Introduction 

The  primary  aim  of  educational  software  is  to  help  students  learn.  Therefore,  evaluation  of  this  kind  of 
software  is  crucial.  Its  aim  is  to  test  the  functionality  and  usability  of  the  design  and  to  identify  and  rectify 
any  problems  (Dix  et  al.  1993).  In  particular,  formative  evaluation  is  one  of  the  most  critical  steps  in  the 
development  of  learning  materials  because  it  helps  the  designer  improve  the  cost-effectiveness  of  the 
software  and  this  increases  the  likelihood  that  the  final  product  will  achieve  its  stated  goals  (Chou  1999).  In 
this  paper,  we  present  and  discuss  the  evaluation  of  an  intelligent  tutoring  system  for  the  passive  voice  of 
the  English  language  using  the  CIAO!  framework  (Jones  et  al.  1999). 

The  Passive  Voice  Tutor 

The  tutoring  system  that  we  developed  and  evaluated  is  called  Passive  Voice  Tutor  (Virvou  & Maras 
1999a;  1999b).  The  Passive  Voice  Tutor  complies  with  the  architecture  of  most  ITSs,  which  consists  of  the 
domain  knowledge,  the  student  modeller,  the  advice  generator  and  the  user  interface  (Wenger  1987). 

In  particular,  the  domain  knowledge  of  the  Passive  Voice  Tutor  comprises  knowledge  about  how  to 
convert  a sentence  from  active  to  passive  voice  and  vice  versa.  It  also  provides  the  knowledge  needed  for 
the  dynamic  construction  of  new  exercise  sentences.  Finally,  it  comprises  knowledge  about  the  semantic 
relation  between  the  words  used  for  the  construction  of  new  sentences. 

The  student  modeller  checks  the  student’s  answer  and  in  case  of  error,  it  performs  error  diagnosis.  It 
contains  a bug  library  of  the  most  common  student  errors  and  generic  mal-rules  that  students  often  apply 
when  they  have  misconceptions.  Another  responsibility  of  the  student  modeller  is  to  keep  a history  record 
concerning  each  student’s  progress  in  learning.  This  record  is  used  in  cases  were  ambiguity  arises,  so  that 
the  system  attributes  a particular  error  to  a category  of  errors  that  the  student  seems  to  be  prone  to. 

The  advice  generator  is  responsible  for  responding  in  the  most  appropriate  way  to  a student’s  error,  by 
informing  the  user  about  the  cause  of  the  error  and  by  showing  to  him/her  the  relevant  part  of  the  theory. 
Finally,  the  user  interface  of  the  Passive  Voice  Tutor,  is  a multimedia  user  interface,  which  involves 
animations,  sounds  and  a limited  form  of  natural  language  so  that  it  can  attract  the  student’s  interest. 

The  Evaluation 

The  CIAO!  framework  outlines  three  dimensions  to  evaluate:  (i)  context;  (ii)  interactions;  and  (iii) 
attitudes  and  outcomes.  One  important  aspect  of  context  is  the  reason  why  CAL  is  adopted  in  the  first  place, 
i.e.  the  underlying  rationale  for  its  development  and  use;  different  rationales  require  different  evaluation 
approaches.  According  to  the  framework,  the  reason  for  looking  at  students’  interactions  with  the  software 
is  in  order  to  understand  more  about  their  learning  processes;  such  interactions  can  provide  protocol  data 
for  later  analysis.  Finally,  at  the  “outcomes”  stage,  information  from  a variety  of  sources  needs  to  be  used 
including,  pre  and  post-achievement  tests,  interviews  and  questionnaires  with  students  and  tutors.  The 
overall  emphasis  of  this  framework  is  on  educational  issues. 

There  are  many  other  evaluation  techniques,  such  as  the  JIGSAW  model  (Squire  & Preece  1996)  or  the 
‘set  of  learning  with  software  heuristics’  (Squires  & Preece  1999),  which  also  address  the  problem  of 
integrating  both  usability  and  learning  issues  in  the  evaluation  of  educational  software.  In  the  JIGSAW 
model,  the  evaluation  is  performed  in  three  levels.  In  Level  1,  the  subtasks  of  the  learning  and  operational 
tasks  are  considered  independently  of  each  other.  As  we  move  from  Level  1 to  Level  2,  integration  within 
the  learning  and  the  operational  tasks  is  considered.  At  Level  3 integration  between  the  learning  and 
operational  tasks  is  considered.  On  the  other  hand,  the  set  of  ‘learning  with  software’  heuristics,  are  an 
adaptation  of  the  “usability  heuristics”  presented  in  Nielsen  (1994),  so  as  to  relate  them  to  socio- 
constructivist criteria  for  learning.  These  heuristics  include  the  need  to  consider  appropriate  levels  of 
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learner  control,  the  need  for  the  prevention  of  peripheral  cognitive  errors  and  the  need  for  strategies  for  the 
cognitive  error  recognition,  diagnosis  and  recovery  cycle.  The  CIAO!  framework,  seemed  to  be  a more 
suitable  method,  in  the  sense  that  it  also  evaluated  the  students’  use  of  the  resource  package. 

For  the  evaluation  along  the  dimensions  of  the  CIAO!  framework,  presented  above,  we  have  done  the 
following: 

i)  Context:  The  underlying  rationale  for  the  development  and  use  of  the  Passive  Voice  Tutor  is  to  help 
students  during  the  process  of  learning.  Hence,  one  of  the  primary  aims  of  the  system  is  to  address 
individual  learners’  misconceptions  and  help  them  correct  their  errors.  Evaluation  in  terms  of  the 
context  has  been  addressed  by  questionnaires  to  human  tutors  before  their  students  had  ever  used  the 
system.  This  was  done  so  that  the  domain  knowledge  and  student  modeller  could  be  evaluated.  In 
particular,  tutors  were  given  a set  of  erroneous  answers  to  questions  relating  to  the  domain,  which  were 
generated  using  the  system’s  domain  knowledge  and  bug  library.  For  each  question,  each  tutor  had  to 
provide  an  explanation  as  to  what  s/he  thought  the  underlying  misconception  of  the  error  was.  Each 
human  tutor’s  explanation  was  compared  to  the  Passive  Voice  Tutor’s  explanation.  A.  satisfactory' 
degree  of  compatibility  was  found  (45%).  However,  the  comparisons  also  showed  that  the  explanations 
could  be  improved  at  a later  version  of  the  Passive  Voice  Tutor. 

ii)  Interactions:  On  a second  phase,  tutors  were  asked  to  evaluate  the  system,  after  their  students  had 
interacted  with  the  Passive  Voice  Tutor.  Protocol  data  was  collected  through  the  individual  long  term 
user  model  of  each  student.  This  kind  of  protocol  data  was  first  automatically  analysed  by  the  student 
modeller  and  resulted  in  information  that  concerned  the  student’s  individual  way  of  learning.  Then, 
human  tutors  were  also  given  the  same  set  of  students’  protocols  and  were  asked  to  analyse  them. 
Finally  the  tutors’  analysis  was  compared  to  the  system’s  analysis.  To  a large  extent,  human  tutors  had 
similar  beliefs  to  the  system’s  beliefs  about  the  students  who  took  part  in  this  evaluation  stage. 

iii)  Attitudes  and  Outcomes:  In  the  case  of  the  Passive  Voice  Tutor,  at  the  outcomes  stage,  questionnaires 
were  given  to  students  and  tutors.  Students’  questions  included  the  following:  “Did  you  consider  the 
system  responses  to  your  errors  helpful  for  understanding  your  mistakes?”  Tutors  were  also  asked  to 
evaluate  the  outcome  of  the  use  of  the  system,  after  their  students  had  interacted  with  the  Passive  Voice 
Tutor.  Therefore,  the  teachers  were  asked  to  provide  information  about  whether  the  software  helped 
them  to  understand  better  the  difficulties  that  their  students  had  in  learning  the  passive  voice  of  the 
English  language.  Tutors  also  seemed  quite  satisfied  by  the  system’s  student  modelling  component. 
Finally,  the  progress  statistics  of  each  student  that  had  been  recorded  in  his/her  long  term  student  model 
was  also  used  at  this  stage  of  the  evaluation.  The  analysis  of  this  kind  of  data  showed  that  the  majority 
of  students  (57%)  made  a progress  in  their  learning,  whereas  only  43%  did  not  seem  to  have  any 
progress  in  their  learning  of  the  passive  voice.  This  is  an  encouraging  result  in  terms  of  the  educational 
effectiveness  of  the  system. 

Conclusions 

The  CIAO!  framework  for  evaluating  educational  software  has  provided  insight  for  several  aspects  of 
the  software  that  needed  evaluation  at  several  stages;  e.g.  before  any  student  had  interacted  with  the 
software  and  after.  In  this  way,  both  learning  effects  and  usability  issues  could  be  evaluated.  The  results  of 
the  evaluation  showed  that  the  Passive  Voice  Tutor  was  quite  satisfactory  but  there  is  scope  for 
improvement  in  terms  of  the  explanations  provided  by  the  system. 
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Introduction 

Web-based  education  has  the  obvious  benefit  of  allowing  platform  indeperident  access  to  easily  updated 
teaching  material.  Recently,  a large  number  of  educational  applications  have  been  delivered  through  the 
World  Wide  Web.  Unfortunately,  most  of  these  systems  are  static,  they  do  not  maintain  a student  model 
and  thus  they  are  unable  to  provide  individualised  tutoring.  Nonetheless,  the  future  of  web-based  education 
seems  promising  since  researchers  in  the  field  of  Intelligent  Tutoring  Systems  have  made  quite  successful 
attempts  to  either  move  existing  ITSs  to  the  WWW  or  build  from  scratch  web-based  ITSs  (Eliot  et  al., 
1997;  Stem  et  al.,  1997).  However,  the  development  of  an  ITS  requires  the  involvement  of  a large  number 
of  people,  including  experts  of  the  specific  domain,  instructors  and  programmers.  A way  to  overcome 
these  problems  may  be  the  development  of  authoring  tools,  which  will  help  construct  cost-effective  and 
reusable  ITSs  in  various  domains.  One  such  authoring  tool  for  ITSs  is  WEAR  (WEb-based  authoring  tool 
for  Algebra  Related  domains).  WEAR  incorporates  knowledge  about  the  constmction  of  exercises  and  a 
mechanism  for  student  error  diagnosis  that  is  applicable  to  many  domains  that  can  be  “described”  by 
algebraic  equations  (Virvou  & Moundridou,  1 999). 

WEAR  uses  a talking  head  in  its  interface  with  the  students.  Animated  characters  have  often  been  used 
in  the  interfaces  of  systems  (Rist  et  al.,  1997;  Stone  & Lester,  1996).  Such  an  interface  makes  a system 
more  appealing  and  attractive  to  the  user  and  if  talking  about  an  educational  application  it  may  also 
promote  the  learning  objectives.  Walker  et  al.  (1994)  investigated  subjects’  responses  to  a synthesised 
talking  head  displayed  on  a computer  screen  in  the  context  of  a questionnaire  study.  Their  findings 
showed  that  compared  to  subjects  who  answered  questions  presented  via  text  display  on  a screen,  subjects 
who  answered  the  same  questions  spoken  by  a talking  head  spent  more  time,  made  fewer  mistakes,  and 
wrote  more  comments.  In  this  paper  we  describe  how  the  talking  head  is  used  in  the  whole  educational 
application. 

Description  of  the  System  and  its  Interface 

WEAR  aims  to  be  useful  to  teachers  and  students  of  domains  that  make  use  of  algebraic  equations. 
Such  domains  could  be  chemistry,  economics,  physics  etc.  In  particular  the  tool  accepts  input  from  a 
human  instructor  about  a specific  equation-related  domain  (e.g.  physics).  This  input  consists  of  knowledge 
about  variables,  units  of  measure,  formulae  and  their  relation.  When  the  human  instructor  wishes  to  create 
exercises  s/he  is  guided  by  the  system  through  a step  by  step  procedure.  At  each  step  of  this  procedure  the 
instructor  specifies  values  for  some  parameters  needed  to  construct  an  exercise.  Such  parameters  could  be 
for  example  what  is  given  and  what  is  asked  in  the  exercise  to  be  constructed.  After  the  completion  of  this 
procedure  the  tool  constructs  the  full  problem  text  and  provides  consistency  checks  that  help  the  instructor 
verify  its  completeness  and  correctness. 

WEAR  assigns  to  each  student  a level  of  knowledge  according  to  his/her  past  performance  in  solving 
problems  with  the  tool.  The  tool  suggests  each  student  to  try  the  problems  corresponding  to  his/her  level  of 
knowledge.  When  a student  attempts  to  solve  an  exercise  the  system  provides  an  environment  where  the 
student  gives  the  solution  step  by  step.  The  system  compares  the  student’s  solution  to  its  own.  The 
system’s  solution  is  generated  by  the  domain  knowledge  about  algebraic  equations  and  about  the  specific 
domain  in  which  the  exercise  belongs  (e.g.  economics).  While  the  student  is  in  the  process  of  solving  the 
exercise  the  system  monitors  his/her  actions.  If  the  student  makes  a mistake,  the  diagnostic  component  of 
the  system  will  attempt  to  diagnose  the  cause  of  it. 

When  interacting  with  the  students  WEAR  responds  through  a talking  head,  representing  in  some 
cases  the  instructor  and  in  some  others  a co-student.  The  talking  head  component  of  the  system  uses 
speech  synthesis  to  automatically  produce  speech  output  from  text  using  MBROLA,  a freely  available 
speech  synthesiser  (http://tcts.fpms.ac.be/synthesis/mbrola.html).  The  talking  head  renders  the  interface 
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quite  attractive  to  students  through  the  sound  of  speech.  Moreover,  since  WEAR  is  an  authoring  tool  for 
ITSs  there  are  a lot  messages  that  are  dynamically  formed  during  the  execution  of  the  application. 
Therefore,  the  authoring  tool  could  not  use  pre-stored  material  for  the  speech  feature  but  rather  a speech 
synthesiser. 

In  the  case  when  the  talking  head  represents  the  instructor  it  guides  the  student  to  the  environment, 
recommends  what  problem  to  solve  next  and  reads  the  problem  text  from  the  database  of  the  authoring 
tool.  Since  WEAR  is  an  authoring  tool,  new  exercises  are  continuously  added  to  its  database.  When  the 
student  begins  to  solve  the  problem,  s/he  may  choose  to  solve  it  either  with  a simulated  “co-student”  or 
with  the  “instructor”.  These  two  different  choices  are  only  on  the  level  of  the  user  interface.  This  means 
that  they  use  the  same  underlying  reasoning  abilities  from  the  diagnostic  component  of  WEAR, 

If  the  student  selects  the  mode  of  the  “co-student”  then  the  talking  head  provides  very  friendly 
messages  as  a peer  to  the  student.  This  simulated  student  is  responsible  for  providing  positive  feedback 
when  the  student’s  actions  are  correct  and  for  pointing  out  the  student’s  underlying  misconception  in  case 
of  an  erroneous  action.  The  information  conceining  which  actions  are  considered  correct  or  not  and  also 
the  messages  that  the  simulated  student  should  say  are  provided  by  the  diagnostic  component  of  WEAR. 
The  talking  head  as  a simulated  student  is  aimed  at  increasing  the  student’s  attention  and  possibly 
collaboration  attitude  (although  in  a limited  form).  Indeed  as  VanLehn  et  al,  (1994)  have  pointed  out  peer 
learning  even  in  a setting  where  the  other  peer  is  a simulated  student  may  increase  students’  collaboration 
skills. 

If  the  student  selects  the  mode  of  the  “instructor”  then  the  talking  head  provides  messages  similar  to 
the  “co-student”  but  they  are  more  formal  and  the  diagnosis  of  misconceptions  goes  one  step  further  to 
resolve  ambiguities  using  the  long  term  student  model.  Indeed,  there  may  be  cases  where  a student’s 
erroneous  action  may  be  attributed  to  more  than  one  misconception.  In  such  cases,  the  diagnostic 
component  first  consults  the  long  term  student  model  and  then  through  the  talking  face  asks  the  student  a 
question  to  determine  his/her  underlying  misconception  and  resolve  the  ambiguity.  The  benefit  of  directly 
asking  the  student  is  twofold:  firstly,  the  system  may  find  out  the  real  reason  for  the  erroneous  action  and 
provide  appropriate  feedback  and  secondly,  the  student  through  explaining  why  s/he  acted  in  that  way 
gains  more  knowledge  and  understanding.  It  is  a common  finding  in  many  researches  that  explaining 
things  either  to  oneself  or  to  another  student  helps  one’s  understanding  (Webb,  1989;  Pressley  et  al., 
1992). 

Conclusions 

In  this  paper  we  described  the  work  in  progress  of  the  development  of  a web-based  authoring  tool  for 
Intelligent  Tutoring  Systems.  The  tool,  called  WEAR,  aims  to  be  used  by  teachers  and  students  working  in 
domains  that  make  use  of  algebraic  equations.  In  this  paper  we  focussed  on  the  interface  of  the  ITSs  that 
are  generated  by  WEAR:  animated  talking  heads  are  used  to  simulate  both  the  instructor  and  a co-leamer 
of  the  student  solving  a problem.  In  that  way,  instruction  and  feedback  are  more  “human-like”  resulting  in 
friendlier  and  more  appealing  ITSs.  Furthermore,  the  students  interacting  with  WEAR,  may  gain  better 
understanding  of  the  domain  being  taught  and  also  improve  their  ability  to  collaborate  with  others; 
benefits  that  arise  from  the  peer  learning  situation  in  which  the  students  are  involved. 
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Introduction 

Intelligent  Tuioring  Systems  have  traditionally  been  very  good  at  providing  dynamic  aspects  to 
the  educational  system’s  reasoning.  This  is  usually  due  to  the  student  modelling  component  that  attempts 
to  follow  the  student’s  way  of  learning  taking  into  account  possible  misconceptions  that  the  student  may 
have.  Other  important  dynamic  features  of  ITSs  are  due  to  the  advice  generator  that  models  the  tutoring 
strategy  by  which  the  student  is  going  to  be  taught  the  domain. 

However,  ITSs  have  often  been  criticised  that  they  represent  immensely  complex  approaches  to 
building  learning  environments  (Boyle,  1997).  A solution  to  this  problem  may  be  authoring  tools  that 
provide  the  facility  of  building  multiple  ITSs.  In  particular,  as  Murray  (1999)  points  out  ITS  authoring 
tools  may  give  the  instructional  designer  a combination  of  facilities  to  produce  visually  appealing, 
interactive  screens  and  a deep  representation  of  content  and  pedagogy. 

This  paper  describes  an  authoring  tool  for  ITSs  that  incorporates  a domain-independent  reasoning 
mechanism.  The  reasoning  mechanism  is  used  for  the  student  modelling  and  the  tutoring  strategy  of  the 
resulting  ITS. 


Related  Work 

The  domain-independent  reasoning  mechanism  of  the  authoring  tool  described  in  this  paper,  is 
based  on  an  adaptation  of  a cognitive  theory  called  Human  Plausible  Reasoning  (Collins  & Michalski, 
1989).  Human  Plausible  Reasoning  theory  (henceforth  referred  to  as  HPR)  is  a domain-independent  theory 
originally  based  on  a corpus  of  people’s  answers  to  everyday  questions.  HPR  formalises  the  plausible 
inferences  based  on  similarities,  dissimilarities,  generalisations  and  specialisations  that  people  often  use  to 
make  plausible  guesses  about  matters  that  they  only  know  partially.  These  inferences  may  lead  to  either 
correct  or  incorrect  guesses;  in  any  case  these  guesses  are  plausible. 

This  theory  can  be  used  as  a domain-independent  generator  of  plausible  human  errors  or  as  a way 
of  reasoning  for  open  questions.  Indeed  in  previous  research  we  adapted  and  used  HPR  in  the  user 
modelling  component  of  an  intelligent  help  system  for  UNIX  users  (Virvou,  1998;  Virvou  & DuBoulay, 
1999)  and  in  an  intelligent  learning  environment  for  novice  users  of  a GUI  (Virvou  & Stavrianou,  1999; 
Virvou  & Kabassi,  2000).  In  the  present  research  we  adapted  and  used  HPR  for  the  student  modelling  of 
multiple  domains,  such  as  geography,  biology  etc.  and  as  a tutoring  strategy  for  teaching  students  how  to 
deal  plausibly  with  open  questions.  In  addition  we  have  provided  the  facility  to  human  instructors  to 
construct  the  knowledge  - base  of  the  domain  to  be  taught  to  students. 


Operation  of  the  Authoring  Tool 

The  initial  input  to  the  authoring  tool  is  a description  of  factual  knowledge  of  a specific  domain 
given  by  a human  teacher.  Then  the  tool  constructs  a knowledge  base  concerning  the  specific  domain  in 
the  form  of  hierarchies,  which  is  compatible  with  HPR.  The  human  teacher  may  also  construct  tests  that 
consist  of  questions  concerning  the  factual  knowledge  of  the  domain. 

The  resulting  ITS  may  be  used  by  students  who  can  be  shown  facts  from  the  knowledge  base.  In 
addition  students  may  test  their  knowledge  by  answering  the  questions  that  the  human  teacher  has  given. 
The  student  modelling  component  examines  the  correctness  of  each  student’s  knowledge  and  the 
soundness  of  the  reasoning  s/he  has  used.  Information  about  each  student  is  kept  in  a long  term  student 
model.  This  kind  of  information  may  be  used  to  measure  the  student’s  progress  and  record  persistent 
misconceptions  and  weaknesses  of  the  particular  student. 


The  tutoring  component  of  the  resulting  ITS  poses  questions  to  the  student  who  is  asked  to  give 
an  answer  and  an  explanation  for  this  answer.  For  example,  in  the  domain  of  geography  the  tutor  may 
pose  the  question:  “Does  Italy  produce  lemons?”  Then  the  student  may  answer  either  “Yes”  or  “No”  and 
give  an  explanation  for  his/her  answer.  An  explanation  could  be  “Yes,  because  I know  it  does”.  If  the 
student  does  not  possess  the  factual  knowledge  needed  for  this  answer  s/he  may  say:  “My  guess  is  yes.  I 
know  Spain  produces  lemons;  Spain  is  similar  to  Italy  in  terms  of  agricultural  products.  Therefore,  Italy 
probably  produces  lemons”.  In  this  case,  both  the  answer  and  the  reasoning  are  examined  by  the  ITS. 
There  may  be  cases  where  the  student’s  answer  may  be  incorrect  but  the  reasoning  reveals  that  the  student 
has  a good  knowledge  of  geography  and  uses  it  in  a plausible  way. 

In  particular,  there  may  be  several  cases  of  combinations  in  terms  of  the  correctness  of  the 
guessed  answer  and  the  correctness  of  the  reasoning.  One  case  is  both  the  guessed  answer  and  the 
reasoning  used  to  be  correct.  Another  case  is  the  guessed  answer  to  be  wrong  but  the  reasoning  used  to  be 
correct.  In  this  case  the  tutor  informs  the  student  about  the  mistake  but  admits  that  the  reasoning  that  the 
student  used  was  plausible.  A third  case  is  the  guessed  answer  to  be  correct  but  the  reasoning  used  to  be 
incorrect.  In  this  case  the  tutor  informs  the  student  that  the  answer  was  correct  by  chance.  There  are  also 
cases  where  the  reasoning  used  may  be  correct  but  the  factual  knowledge  of  relevant  topics  may  be 
incorrect.  For  example,  if  the  student  was  asked  whether  Italy  produced  coffee  and  answered  “My  guess  is 
yes  because  Spain  produces  coffee  and  Spain  is  similar  to  Italy  in  agricultural  products”.  In  this  example 
Spain  is  similar  to  Italy  but  it  does  not  produce  coffee. 

The  learning  environments  that  result  from  this  authoring  tool  are  more  open  than  tutoring 
systems  which  do  not  have  any  reasoning  abilities  about  correct  and  incorrect  knowledge.  For  example, 
Ohlsson  (1993)  suggests  the  analysis  of  epistemic  activities  (arguing,  describing,  explaining,  predicting, 
etc.)  to  be  more  relevant  for  higher  order  learning  of  declarative  knowledge  than  the  study  of  goal-oriented 
action.  In  addition,  Andriessen  & Sandberg  (1999)  point  out  that  the  process  of  becoming  an  expert  in  a 
certain  domain  should  no  longer  be  solely  viewed  as  the  acquisition  of  a representation  of  correct 
knowledge;  the  knowledge  to  be  acquired  should  flexibly  manage  open  problems.  In  this  sense  the 
authoring  tool  provides  support  in  the  student’s  learning  of  factual  knowledge  and  the  reasoning  that  may 
manage  this  knowledge. 


Conclusion 

In  this  paper  we  described  the  work  in  progress  of  an  ITS  authoring  tool  that  incorporates  a 
domain-independent  reasoning  mechanism.  The  reasoning  mechanism  is  based  on  HPR,  which  is  a 
cognitive  theory  about  how  people  make  plausible  guesses.  In  this  way,  the  resulting  ITSs  may  provide 
quite  open  learning  environments  for  the  students  to  practice  their  factual  knowledge  on  a certain  topic, 
factual  knowledge  on  possibly  relevant  topics  and  the  reasoning  they  use  to  deal  with  open  problems  by 
combining  factual  knowledge  from  relevant  topics. 

The  authoring  tool  is  in  the  process  of  being  evaluated  in  the  domain  of  geography  and  biology. 
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Abstract;  This  paper  reports  on  the  results  of  the  project  "Open  and  Distance  Learning 
Network  in  Information  Technology  and  Microelectronics  - EDIT"  run  by  University 
"Politehnica"  of  Bucharest,  Technical  University  of  Cluj,  Technical  University  "Gh.  Asachi" 
of  Iasi,  University  of  Craiova,  Romania  and  IBM  Global  Services,  Education&Training,  La 
Hulpe,  Belgium.  The  paper  focuses  on  the  description  of  the  main  objectives,  technical 
infrastructure,  course  materials,  impact/target  groups,  type  of  learning  environment  and 
further  developments.  A distributed  web-based  learning  environment  has  been  developed 
within  the  EDIT  project. 


Objectives 

The  main  objective  of  EDIT  Project  was  to  develop  the  first  Romanian  academic  open  and  distance 
education  network  aimed  at  training  the  students  and  academics  from  the  four  partners  universities.  In  the 
long  run  the  network  will  extend  to  link  the  main  technical  universities  in  Romania,  including  researchers  and 
experts  from  other  institutions.  The  project  is  financial  supported  by  the  Romanian  National  Higher  Education 
Funding  Council  for  a period  of  three  years. 

The  project's  specific  objectives  are:  1)  to  setup  and  to  develop  the  technical  infrastructure;  2)  to 
design  and  to  implement  some  good  examples  of  education/training  course  materials  in  the  field  of  IT  and 
Microelectronics;  3)  to  deliver  the  designed  courses  to  our  students/customers;  4)  to  find  the  best  suited 
learning  environments  for  a given  topic/target  group;  5)  to  support  itself  through  contracts  with  different 
customers  at  the  end  of  the  three  years  period. 


The  Technical  Infrastructure 

Because  of  the  unique  distance  learning  environment  of  the  Web  and  the  availability  in  all  Romanian 
universities  the  EDIT  Project  focuses  on  web-based  learning  environments. 


The  Design  and  Implementation  of  Course  Materials 

In  1998,  EDIT  Distance  Learning  Network  starts  to  deliver  his  first  course  on  Modeling  and 
Simulation  of  Semiconductor  Processes  and  Devices  (MSSPD)  for  the  MSc  students  from  Department  of 
Electronics  and  Telecommunications  (Drondoe  et  al.  1998)  using  an  in-house  designed  web-based  learning 
environment:  EDIT  Learning  Environment  - ELE.  Presently,  we  offer  four  courses  for  Department  of 
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Electronics  and  Telecommunications:  Electron,  Devices  and  Circuits,  MSSPD,  Optimization  Methodologies 
and  Particle  Simulation  in  Nanostructures  (Drondoe  et  al.  1999). 


Impact/Target  Groups 

The  courses  are  open  for  undergraduate  and  graduate  students,  academics  and  researchers  that  can 
choose  between  different  level  of  complexity  depending  on  their  previous  knowledge  and  personal  goals. 

The  Type  of  Learning  Environment 

Depending  on  the  topic  and/or  target  group  the  chosen  learning  environment  used  for  implementation 
of  the  course  material  was  either  ELE  and/or  two  well-known  distance  learning  commercial  products  (i.e. 
LOTUS  Learning  Space  and  IBM  Distance  Learning  Space/DLS). 

The  main  features  of  ELE  are  (see  Zolti  et  al.  1999):  www  dedicated  arhitecture,  distributed  way  of 
working  both  in  design  and  exploitation,  high  degree  of  reusability,  integrated  communication  system  (e-mail 
and  virtual  classrooms),  web-based  publishing  system,  web-based  voting  system  and  secure  web  access  to  the 
database. 


Sustainability 

The  sustainability  of  the  project  is  envisaged  through  contracts  with  online  continuing  education 
courses  on  demand  dedicated  to  enhancement  of  adults'  knowledge/skills  and/or  to  professional  reorientation 
of  the  unemployed  people.  A special  attention  is  also  given  to  distance  training  of  the  work-force  in  the 
corporate  sector. 

Conclusions  and  Further  Development 

A distributed  web-based  learning  network  has  been  presented.  At  the  basis  of  this  implementation  lies 
WWW  dedicated  architectures.  This  has  allowed  us  to  reach  a high  level  of  reusability  and  also  cost- 
effectiveness.  Currently,  there  is  much  work  in  progress  to  implement  more  courses  within  EDIT  Distance 
Learning  Network  and  to  further  develop  the  ELE  by  establishing  some  efficient  authentication  methods  that 
would  make  awarding  certificates  more  legitimate. 
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This  short  paper  describes  dissertation  work  in  progress  investigating  the  cognitive 
processes  and  social  practices  involved  in  the  creation  of  computer  based  learning 
environments  (CBLEs).  Unfortunately,  knowledge  regarding  the  actual  design  process  is 
very  limited.  Formal  instructional  design  prescriptions  (Dick  & Carey  1985;  Gagne 
Briggs  & Wager  1988)  and  design  principles  based  on  cognitive  psychology  (Duffy  & 
Cunningham  1996;  Wasson,  1996)  do  exist  but  it  is  not  clear  how  these  prescriptions  and 
design  principles  are  used  in  ‘real  life’.  Research  into  the  cognitive  processes  involved  in 
instructional  design  tasks  is  limited.  Studies  have  examined  such  things  as  the  extent  and 
nature  of  alternative  designs  and  stopping  rules  (Kerr,  1983),  expert-novice  differences  in 
problem  understanding  and  problem  solution  (Rowland,  1992)  and,  expert-novice 
differences  in  declarative  and  procedural  knowledge  and  representations  of  design  (Perez 
& Emery,  1995).  There  are  very  few  studies  of  the  design  process  involved  in  the 
creation  of  CBLEs.  The  lack  of  empirical  work  regarding  the  design  processes  and 
practices  involved  in  the  creation  of  CBLEs  is  surprising  given  the  important 
contributions  this  type  of  knowledge  could  bring  to  future  design  activities.  Studies  of  the 
process  could  indicate  where  typical  difficulties  exist  and  suggest  strategies  to  overcome 
them.  Additionally,  it  would  be  interesting  to  observe  the  effect  of  the  application  of  one 
of  design  principles  on  the  actual  design  process. 

Theoretical  Perspective 

The  theoretical  perspective  adopted  in  this  study  combines  traditional  human  problem- 
solving (Newell  & Simon,  1972)  perspective  with  an  activity  theory  (Engstrom,  1989) 
one.  Problem-solving  research  focuses  on  the  internal  mental  process  involved  in 
problem-solving  and  has  been  used  to  investigate  instructional  design  problem-solving 
process  but  it  has  not  been  used  rigorously  to  study  designers  of  computer  material.  On 
the  other  hand,  activity  theory  investigates  how  social  systems  influence  human  practice. 
Activity  theory  has  not  been  used  to  study  the  design  processes  involved  in  the  creation 
of  computer  based  material  but  has  been  used  to  study  other  computer  design  activities. 
Research  that  combines  human-problem  solving  and  activity  perspectives  provides  a 
theoretical  perspective  that  recognizes  the  cognitive  and  social  dimensions  of  design 
activities  (Bracewell  & Witte  1997;  Cobb,  1994;  Portes  1996). 
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Questions  and  Method 

(1)  What  problem-solving  processes  are  involved  in  developing  computer-based  learning 
environments  (human-problem  solving  perspective) 

(2)  What  social  practices  are  involved  in  the  creation  of  developing  computer-based 
learning  environments  (activity  theory  perspective) 

A case  study  of  a design  team  at  a local  multimedia  development  firm  was  conducted 
over  several  weeks.  Participants  included  one  subject  matter  expert,  an  editors  and  a 
graphic  artists.  Data  consists  of  participant  observation  field  notes  collected  over  several 
weeks,  interviews  and  think-aloud  protocols. 

Results 

Preliminary  results  indicate  that  the  problem-solving  activity  of  team  members  is  goal 
directed  and  involves  establishing  numerous  local  plans  and  frequent  evaluations.  It  is 
clear  that  a concurrent  analysis  that  combines  problem-solving  and  activity  theory  is  a 
very  promising  approach.  It  provides  data  that  allows  researchers  to  analyse  and  interpret 
the  relationship  between  internal  mental  processes  and  social  context. 
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Abstract:  This  paper  describes  a case  study  on  the  use  of  comrnunity-based  low 

powered  FM  radio  that  has  been  undertaken  in  Uganda  as  part  of  the  work  of  the 
Commonwealth  of  Learning.  It  describes  a media  solution  that  employs  low-powered 
FM  radio  as  an  effective  method  of  delivering  education  and  information  to  a rural 
community.  It  also  describes  the  inhibitors  to  proliferation  of  community  radio  stations 
and  reflects  on  the  future  directions  for  the  use  of  radio  as  a delivery  system  for 
education. 


Introduction 

**All  available  instruments  and  channels  of  information,  communications,  and  social  action  could  be  used 
to  help  convey  essential  knowledge  and  inform  and  educate  people  on  social  issues.  In  addition  to  the 
traditional  means,  libraries,  television,  radio  and  other  media  can  be  mobilized  to  realise  their  potential 
towards  meeting  basic  education  need  of  all”.  Final  Report  World  Conference  for  All:  Meeting  Basic 
Learning  Needs,  Jomtien,  Thailand,  1990. 

Why  Community  Radio? 

Community  radio  is  an  immensely  powerful  technology  for  the  delivery  of  education  with  enormous 
potential  reach  globally.  Creating  opportunities  for  communities  to  utilise  this  delivery  system  will  enable 
disadvantaged  groups  to  engage  in  a development  agenda,  sensitive  to  their  needs  and  aspirations.  In  order 
to  serve  the  underprivileged  and  rural  poor,  mass  media  such  as  radio  must  create  conditions  and 
mechanisms  that  provide  people  with  genuine  access  to  useful  information.  Such  mechanisms  will  offer 
ways  in  which  people  can  express  their  sentiments,  opinions,  views,  dreams  and  aspirations,  their  fears  and 
insecurities,  their  strengths  and  capabilities,  and,  of  course,  their  ideas  for  development. 

High  illiteracy  rates  and  low  levels  of  schooling  among  disadvantaged  groups,  especially  women,  in  many 
developing  countries  continues  to  limit  their  ability  to  lift  themselves  out  of  poverty.  The  existing 
educational  system  has  shown  itself  to  be  unable  to  respond  to  the  massive  demand  for  increased  education. 
This  is  especially  true  in  many  poverty-stricken  countries  with  respect  to  meeting  the  massive  education 
needs  of  the  rural  poor.  Consequently,  disadvantaged  groups  continue  to  be  denied  access  to  information, 
knowledge,  skills  and  technology  transfer. 

The  answer  is  to  deploy  Distance  Education  techniques  and  delivery  systems  such  as  radio  and  television 
based  at  the  community  level  to  address  directly  local  issues  and  needs. 

In  order  to  empower  disadvantaged  groups  as  equal  partners  in  development,  the 
limitations  of  formal  and  non-formal  education  are  now  being  challenged.  Urgent  and 
new  ways  to  achieve  mass  education,  that  can  be  both  efficient  and  effective,  are  being 
sought.  In  this  context,  radio,  an  effective  telecommunications  medium,  was  proposed  at 
the  Jomtien,  Thailand  UNESCO  Education  of  All  conference  in  1990  as  one  massive 
solution.  Radio  can  cut  across  geographic  and  cultural  boundaries.  Given  its  availability. 
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accessibility,  cost-effectiveness  and  power,  radio  represents  a practical  and  creative 
medium  for  facilitating  mass  education  in  a rural  setting.  However  ten  years  on  since 
Jomtien,  radio  still  continues  to  be  an  undemtilized  technology  in  education.  This  is 
especially  surprising  because  from  a learner’s  point  of  view,  radio  is  user  friendly, 
accessible  and  a well-established  medium.  From  an  educational  provider’s  point  of  view 
it  is  easy  to  set  up,  produce  and  broadcast  programmes.  After  almost  hundred  years  of 
broadcasting  history  most  nations  of  the  world  have  more  than  a respectable  level  of 
engineering  skills  and  broadcasting  talent  to  apply  the  technology  in  education.  In  the  last 
ten  years,  radio  has  been  greatly  enhanced  by  the  emergence  of  new  technologies,  which 
have  opened  up  new  opportunities  for  a variety  of  foim.s  of  deliver^'  and  access  for  both 
broadcaster  and  listener.  For  example,  portable,  low  cost  FM  transmitting  stations  have 
been  developed  and  digital  radio  systems  that  transmit  via  satellite  and/or  cellular  are 
being  implemented  in  many  parts  of  the  globe.  Internet  streaming  audio  software 
technology  has  emerged  recently  to  allow  a global  audience  to  listen  the  news  from  a 
distant  country.  And  windup  and  solar  radios  have  been  developed  thus  freeing  radios 
from  expensive  power  sources.  COL  is  addressing  the  issue  of  education  for  all  with 
creative  media  solutions  that  fall  under  the  banner  of  an  initiative  called  The 
Commonwealth  of  Learning  Media  Empowerment  (COLME). 

COLME  - A Template  for  a Media  Projects 

COLME  is  building  creative  media  models  for  delivery  of  information  and  training  in  the 
area  of  non-formal  and  formal  education. 


COLME  strives  to: 

• Provide  new  skills  in  the  use  of  technology  for  the  disadvantaged 

• Provide  media  models  that  will  stress  local  participation  and  transfer  of 
knowledge  and  skills 

• Provide  opportunities  for  disadvantaged  groups  to  participate  and  benefit  from 
new  technology  and  media  based  initiatives 

• Create  a capacity  for  dialogue  among  government  sectors,  institutions  and 
different  interest  groups 

• Create  a research  base  and  body  of  knowledge  that  can  be  utilised  by 
Commonwealth  governments,  organisation  and  communities  as  models  for  media 
and  technology  based  initiatives. 

Media  projects  in  radio  stations  and  projection,  video  and  audio  production,  and 
computing  solutions  have  all  been  part  of  the  work  of  COLME  in  the  past  several  years. 
The  focus  here  is  a portable  FM  radio  station  that  has  been  used  in  several  developing 
countries  of  the  Commonwealth  such  as  Uganda,  Namibia  and  South  Africa. 


Overview  of  the  Station 


Radio  is  a very  powerful  technology  that  can  allow  large  sectors  of  the  population  to  be 
reached  with  information,  quickly  and  economically.  Due  to  national  broadcast 
regulations  in  many  countries,  conununity  radio  stations  have  not  developed.  Also  the 
cost  of  transmitters,  infrastructures,  and  equipment,  has  placed  most  potential  conununity 
broadcasters  at  a disadvantage,  especially  those  in  the  remote  rural  areas.  There  is  a 
distinct  information  gap  to  the  rural  comers  of  some  countries  resulting  from  the  lack  of 
service  by  national  broadcasters  who  in  some  cases  have  weak  or  non-existent  signal 
coverage.  Under  COLME,  portable  FM  radio  systems  have  been  tested  and  implemented 
as  part  of  media  project  work  over  the  past  three  years.  The  station  configuration  that 
was  first  developed  has,  with  input  and  data  gathered  from  COLME  initiatives  in  the 
field,  have  aided  the  manufacturer  in  altering  the  station  to  address  the  each  community’s 
need.  The  station  configurations  range  in  price  from  three  for  five  thousand  dollars  US 
including  all  elements,  antenna,  transmitter,  console,  mixer,  microphones  and  CD  and 
tape  decks  (Figure  1).  The  stations  can  mn  from  12  V DC  or  120/240  AC. 


Figure  1 - Pictured  is  a station  in  its  watertight  carrying  case  (on  the  consul  starting  on  the  left  top  the 
gooseneck  microphone,  below  is  the  mixer,  top  right  are  two  tape  decks,  below  are  two  CD  decks).  The 
transmitter  and  power  supply,  not  pictured,  are  housed  under  the  consul.  The  consul  is  removed from  the 
carrying  case  when  in  operation  (see  Figure  4 for  operational  mode). 

Keys  Elements  to  Success 

There  are  number  technological  factors  that  are  important  in  the  initial  needs  analysis 
before  a station  can  be  considered.  First  the  physical  landscape  must  be  conducive  to  an 
FM  signal  to  reach  the  intended  target  audience  especially  if  rebroadcast  of  the  origin 
station  signal  is  not  possible  due  to  cost  or  licensing  regulations.  If  the  landscape  is 
mountainous  then  there  will  be  difficulty  in  the  signal  reaching  a large  radius  of  users. 
Secondly,  the  station  target  audience  must  have  radios  or  access  to  radios.  Thirdly,  there 
must  a situation  where  there  is  a steady  flow  of  content  and  a regular  broadcast  schedule. 
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Fourthly,  the  station  must  be  targeted  to  the  local  users  so  that  they  can  directly  relate  to 
the  content,  language,  and  situations  discussed. 

In  the  feasibility  stage  before  station  implementation  certain  conditions  must  exist  to  improve  the  element 
of  sustainability.  In-country  stakeholders  are  identified  for  each  of  the  stations.  Their  role  is  to  insure 
infrastructure  is  in  place  for  FM  radio  and  that  all  licensing  and  issues  pertaining  to  community 
broadcasting  are  dealt  with. 

Another  important  factor  is  that  the  broadcasts  are  in  languages  that  are  used  daily  in  the  local  community 
level.  The  national  or  regional  stations  do  not  have  the  capacity  to  aim  linguistically  or  at  the  level  of 
information  detail  for  rural  community  issues.  Community-based  stations  can  be  effective  if  well  managed 
in  providing  information  and  training  directly  to  the  community.  In  the  case  of  the  COLME  installed 
community  station  in  Uganda,  it  was  imperative  that  the  station  be  able,  by  law,  to  rebroadcast  Radio 
Uganda  in  the  event  of  important  political  announcements.  Therefore,  among  the  technological  upgrades 
in  the  design  of  the  station,  in  addition  to  the  interface  for  telephone  calls,  extra  microphone  inputs  for 
group  discussions,  and  a more  powerful  transmitter,  a facility  for  radio  rebroadcast  of  the  national 
government  station,  (in  Uganda,  Radio  Uganda)  and  international  broadcasters  (such  as  the  BBC)  was 
implemented. 

The  overriding  factor  to  the  success  of  these  stations  has  been  the  proper  community 
access  and  ownership,  which  was  paramount  in  the  initial  project  design.  If  the  station  is 
or  becomes  an  integral  part  of  the  voice  of  the  community  and  local  interest  groups  have 
an  equal  say  in  the  information  that  is  disseminated  via  the  station,  then  there  is  a lesser 
risk  of  failure  in  the  long-term  sustainability  of  the  station.  This  can  be  achieved  with 
good  station  management  that  works  with  community  leaders  and  committees  consisting 
of  both  political  and  community  leaders. 

The  local  stakeholders,  with  the  aid  of  COLME,  will  provide  on  going  evaluation  of  the 
stations  via  listener  surveys  and  media  expert  evaluation.  Workshops  will  be  given  in 
production  and  survey  techniques  that  will  aid  broadcasters  with  improving  programming 
to  suit  the  needs  of  the  community.  Local  broadcasters  will  be  tapped  to  train  in 
advanced  broadcasting  techniques  and  programme  development  that  will  improve 
community  radio  personnel. 

A Solar  Station  on  the  Move  - A COLME  Case  Study 

Apac,  Uganda  is  located  in  the  northern  region  of  Uganda.  This  COLME  project  was  a cooperative  effort 
with  the  Minister  of  State  and  Tourism,  The  Right  Hon.  Akaki,  to  work  with  community  leaders  to 
implement  an  FM  radio  station  in  the  Apac  region.  The  COLME  feasibility  study  revealed  several 
limitations  with  the  electrical  infrastructure,  which  was  not  reliable.  This  was  a result  of  load  sharing 
throughout  the  country  (Apac  would  not  receive  power  for  several  days).  The  power  was  also  not  usable 
for  electronic  equipment  due  to  the  dramatic  power  fluctuations.  Therefore,  it  was  decided  that  in  order  to 
maintain  a reliable  broadcasting  schedule  and  develop  the  station  as  a center  point  to  community  activities 
by  different  groups.  Radio  Apac  would  be  operated  entirely  by  solar  power.  This  would  free  the  project 
from  the  constraints  of  electrical  situation  and  the  tariffs  associated  with  it.  A configuration  was 
determined,  in  consultation  with  a solar  distributor  in  Kampala,  to  allow  the  station  to  stay  operational 
during  the  eighteen-hour  broadcast  day.  Eight  solar  panels  and  seven  deep  cycle  batteries  were  installed  at 
the  station,  which  now  provide  lighting  and  all  the  station  power  requirements  for  daily  broadcasting  (see 
Figures  2 to  4).  The  lifespan  of  solar  installations  is  over  a decade  with  low  maintenance  costs. 

A committee  was  organized  and  a station  manager  appointed.  This  person  works  directly  with  the 
community  to  develop  programming  and  allow  the  development  of  community  involvement.  The  station 


has  a rebroadcast  facility  incorporated  for  programmes  from  the  national  broadcaster  as  well  as  the  BBC 
and  WorldSpace.  A further  radius  of  listeners  will  be  taken  in  Spring  2000  with  a retransmission  unit  that 
will  allow  the  signal  to  take  in  other  populated  areas.  It  is  anticipated  that  the  total  potential  listening 
audience  will  be  over  one  million.  A VHF/UHF  radio  system  will  also  be  implemented  to  act  as  a 
community  telephone  that  will  interface  into  the  FM  radio  system.  This  will  allow  the  broadcasters  to  out 
to  the  community  and  speak  directly  to  groups  live  on  air  thus  eliminating  the  issue  of  travel  to  the  station 
by  people  such  as  farmers  and  medical  officers.  Links  have  been  made  to  educational  programmers  in 
South  Africa  to  supply  educational  programmes  to  the  station.  The  station  has  been  used  by  the 
government  for  health  related  issues  and  announcements.  There  is  also  an  element  of  income  generating  by 
the  station  with  the  interest  of  local  businesses  in  advertising  their  services. 


Figure  2 - Receiving  the  solar  panels  at  the  station  site 


Figure  3 - Installing  the  panels 
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Figure  4 ~ Radio  Apac,  92.9  FM  on  air,  powered  by  the  sun 

Conclusion 

Radio  is  an  effective  system  for  delivery  of  education  to  large  numbers  of  people.  It  facilitates  information 
exchange  at  the  community  level,  acting  as  a “community  telephone”  and  can  be  effective  in  literacy  and 
formal/non  formal  education.  Analogue  systems  for  radio  will  be  supplanted  by  digital  broadcasting  in  the 
coming  decade,  however  digital  radio  will  pose  issues  including  cost  of  radio  receivers  and  renewal  of 
broadcasting  infrastructure.  Analogue  radio  systems,  such  as  the  portable  solution  that  COL  and  others 
have  utilised  in  community  FM  radio  initiatives,  can  be  effective  in  delivering  education  to  the  masses 
without  the  high  infrastructure  costs  associated  with  radio  broadcasting.  With  community  broadcasting  not 
only  can  broadcasters  focus  on  addressing  local  needs  through  their  own  produced  programming,  but  also 
have  the  choice  among  a tremendous  variety  of  quality  educational  content  that  is  available  via  rebroadcast 
from  national  and  international  sources  whether  it  is  delivered  via  satellite  or  via  the  Internet. 
Rebroadcasting  also  should  be  balanced  with  the  needs  of  the  local  community  and  the  provision  of 
appropriate  and  relevant  programming  content. 


There  is  a marriage  between  the  digital  and  the  FM  analogue  systems  that  is  taking  place.  The  convergence 
also  includes  Internet  streamed  audio  based  broadcasters  that  can  effectively  be  employed  by  the 
community  FM  station  in  a rebroadcast  mode.  Will  we  be  able  to  say  in  ten  years  that  radio's  potential  for 
educational  delivery  to  millions  of  disadvantaged  groups  has  finally  been  realised?  With  the  many  varied 
formulas  for  convergence  of  digital  and  analogue  technology  and  the  vast  selection  of  content  and  tools  to 
create  original  culturally  sensitive  material  for  education  at  the  community  level,  we  state  clearly  - yes.  But 
will  the  bodies  that  regulate  frequencies  for  community  radio  initiatives  reform  regulations  to  reflect  the 
current  technological  developments  and  pressing  need  for  mass  media  to  meet  the  goal  for  education  for  all 
in  the  next  ten  years?  We  can  only  hope.  The  past  ten  years  and  the  failure  of  Jomtien  is  a heavy  burden  to 
bear.  The  next  ten  should  see  the  harnessing  of  radio,  analogue,  and  more  so  digital,  as  the  powerhouse  for 
delivery  of  education.  Governments  should  be  prepared  to  adjust  broadcasting  regulations  to  adhere  to 
technological  developments  and  realities,  and  also  consider  community  based  mass  media  delivery  as  an 
effective  solution  for  improving  a nation's  human  resource  development  towards  the  goal  of  education  for 
all. 
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ABSTRACT  The  authors  will  describe  the  programs  used  in  the  preparation  of  faculty  who  are  participating 
in  an  Web-based  Integrated  Curriculum  in  Engineering  (ICE).  The  programs  are  a blend  of  efforts  between 
faculty  and  the  Educational  Technology  Team. 


The  ICE  program  is  a comprehensive  effort  between  engineering  faculty,  humanities  faculty,  physics  and 
mathematics  faculty.  The  ICE  program  includes  specifically  designed  sections  of  the  foundation  courses 
required  of  all  engineering  students:  the  calculus  sequence,  the  physics  sequence,  the  humanities  sequence,  social 
science,  and  introductory  engineering  courses.  What  differentiates  the  ICE  program  from  traditional  engineering 
curricula  is  that  all  courses  incorporate  active  and  collaborative  learning  and  a reliance  on  computer  and  web 
technology.  In  addition,  the  ICE  program  promotes  team  learning,  team  design  projects,  and  a well-documented 
series  of  assessment  practices. 

Background  of  ICE: 

The  motivation  behind  this  program  includes: 

■ high  attrition  rates  in  calculus,  physics,  and  chemistry; 

■ perceptions  of  the  faculty  that  these  students  do  not  read  and  write  well; 

■ a lack  of  comprehension  of  the  building  nature  of  the  foundation  courses  and  their  relationship  to  the 
engineering  curriculum; 

■ the  lack  of  continuous  use  of  common  computer  technology; 

■ failure  of  students  to  adjust  to  the  university  environment, 

■ and  the  isolation  that  many  students  feel  in  such  programs  of  study. 

The  Integrated  Curriculum  for  Engineers  was  introduced  in  1997  to  a volunteer  group  of  65  entering  freshmen 
selected  from  a pool  of  1 30  initial  volunteers.  In  succeeding  years,  similar  sized  classes  have  volunteered  to 
enter  this  program. 

This  paper  describes  the  programs  specifically  designed  by  the  Educational  Technology  Team  and  the  programs 
put  together  to  teach  faculty  how  to  use  active  and  cooperative  learning  in  the  various  courses  which  are  included 
in  ICE. 


Description  of  ICE  methodology 


Computer  and  web  technology  are  crucial  to  the  implementation  of  the  ICE  program.  Students  in  this 
program  must  have  laptops  for  use  in  each  course.  Classrooms  have  been  wired  so  students  have  access  to 
the  campus  network  and  access  to  the  Internet.  Common  software  in  use  includes  Maple,  Microsoft  Office, 
Bentley  Microstation,  and  Internet  access.  All  courses  are  web-based  so  that  students  have  access  to  each 
course  in  the  program,  to  reference  materials,  tutorials,  reviews  for  exams,  and  to  each  faculty  member  and  to 
their  peers.  Forums,  emails,  chat  rooms  and  links  to  other  sites  are  all  available  to  every  student.  Students 
can  plug  their  laptops  into  any  campus  port  as  well  as  their  dorm  room.  The  authors  will  discuss  examples  of 
daily  classroom  usage  of  a web-course  and  we  will  comment  on  the  active  and  collaborative  nature  of  the 
courses  by  showing  how  these  are  incorporated. 

Another  crucial  component  of  the  ICE  program  is  the  use  of  cooperative  and  collaborative  learning  in  each 
course.  Thus,  in  addition  to  restructuring  the  content,  all  courses  have  been  redesigned  for  an  active  and 
collaborative  learning  environment.  Each  course  assigns  team  exercises,  daily  or  weekly,  which  are  to  be  done 
both  in  and  outside  of  the  classroom.  In  any  of  these  courses,  it  is  common  to  ask  a team  to  present  their  work  to 
their  peers  at  the  end  of  a daily  class.  In  addition,  the  mathematics  faculty  designs  external  team  problems  that 
arc  reported  in  class  about  a week  later.  For  each  such  team  problem  it  is  required  that  the  problem  be  written 
using  a word  processor  and  the  report  is  graded  for  composition  and  exposition  as  well  as  the  mathematics. 

Class  presentations  are  oral  and  each  team  receives  a grade  based  upon  accuracy,  style  and  clarity  of  exposition. 

Students  in  the  program  are  assigned  to  a team  of  four.  A team  is  selected  based  on  the  class  rank  condition.  A 
team  stays  together  through  all  the  courses  during  that  semester.  The  engineering  faculty  devote  the  first  week  of 
the  introductory  engineering  course  to  team  building  skills  and  continue  to  monitor  the  effectiveness  of  the  teams 
throughout  the  semester.  This  engineering  course  also  serves  as  a student  success  course  based  on  the  University 
101  model  and  include  topics  such  as  time  management,  study  skills  as  well  as  adjustment  and  orientation  to  the 
university. 


Programs  of  the  Educational  Technology  Team 

Most  of  the  faculty  who  teach  at  Embry-Riddle  use  word  processing,  spreadsheets,  and  email.  A few  of  the  ICE 
faculty  had  some  acquaintance  with  use  of  the  web,  but  none  had  ever  put  a course  up  on  the  web  and  many  were 
apprehensive  about  trying  A template  was  designed  specifically  for  the  web-based  portion  of  the  ICE  program, 
and  faculty  members  were  provided  with  training  on  how  to  develop  and  post  materials  in  the  template. 

Because  the  ICE  faculty  were  technically  proficient  in  specialized  areas  and  working  with  a web  template 
specific  to  their  program,  a customized  training  program  was  developed  for  them  by  the  university  Educational 
Technology  Team.  A description  of  the  ICE  web  template  and  training  program  and  ongoing  support  for  the 
ICE  faculty  will  be  presented  in  this  talk 

Programs  used  to  acquaint  faculty  with  Active  and  Cooperative  Learning. 

When  faculty  think  of  teaching,  they  think  of  lecturing  - the  me-  centered  model  of  education.  Active  learning  is 
participatory  - an  idea  or  theme  is  selected,  the  students  think  about  it,  promote  their  own  understanding,  and 
share  it  with  the  class.  Collaborative  learning  is  group  based  - in  our  case,  team-based.  It  requires  the  team  to 
work  together  in  and  out  of  class  on  assignments. 


Assessment 

Assessment  of  these  training  programs  include  informal  and  formal  components.  In  this  presentation,  the  authors 
will  comment  on  each,  including  what  worked  and  what  did  not.  In  addition,  we  will  comment  upon  some  of  the 
measures  used  by  faculty  in  the  ICE  program  itself  These  include  attitudinal  surveys  about  use  of  teams,  use  of 
active  learning,  use  of  computer  technology,  use  of  computer  software,  use  of  web-based  courses,  adjustments  to 
the  university,  and  a variety  of  other  topics.  Assessment  of  use  of  web-based  materials  is  done  by  formal 
assessment  at  the  end  of  the  term.  All  aspects  of  assessment  will  be  shared  with  the  audience. 
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Abstract:  The  FutureBoard  system  has  been  developed  to  support  the  collaborative  learning  process,  especially 
design  activities  in  Project  Based  Learning. 

Two  kinds  of  board  system  have  been  created:  one  is  a Personal  Board  for  supporting  the  personal  reflective 
thinking  process  based  on  a wireless  note  PC,  and  the  other  is  a Collaboration  Board  for  supporting  collaborative 
learning  and  the  creative  process  based  on  a 50  inch  monitor  with  pen  input  support. 


Project  Based  Learning  and  Design  Activities 

In  Japan,  the  national  curriculum  standards  will  be  changed  in  2002,  and  Project-based  learning  known  as  “The 
period  of  integrated  study”  will  be  introduced  to  elementary  and  secondary  schools.  During  the  process  of  project- 
based  learning,  design  activities  such  as  making  books  and  posters  are  critical  because  they  give  students  the  chance 
to  reflect  on  their  ideas.  Computers  have  been  used  for  the  design  activities  in  project-based  learning,  however  they 
are  limited  to  student’s  individual  expression  when  using  graphic  software  and  DTP  software. 

We  believe  computers  can  render  the  process  more  dynamic  when  designing  activities  such  as  in  collaborative 
design  for  the  social  construction  of  ideas,  by  many  professional  designers  do. 

Design  Process  of  Professional  Designers 

We  have  analyzed  the  design  process  of  professional  interface  designers  (a  university  professor  and  two 
professional  designers  in  a private  corporation)  in  order  to  clarify  points  to  support  the  collaborative  design  process. 

1 . Using  Tracing  Paper 

All  used  tracing  paper  in  the  collaborative  design  process.  One  designer  drew  an  idea  on  paper,  while  the  other  put 
tracing  paper  on  it,  revised  and  added  new  ideas  to  previous  ideas. 

2.  Fast  and  Interactive  Process 

This  process  was  fast-paced  and  interactive.  They  sketched  ideas  in  a few  seconds  and  talked  like  a “This  is  better,” 
pointing  the  revised  idea. 

3.  Using  Personal  Spaces 

Sometimes,  Personal  spaces  were  used  such  as  on  a small  memo  or  a comer  of  the  tracing  paper  to  dot  down  their 
ideas. 

Outline  of  the  FutureBoard  System 

Following  this  analysis,  we  made  design  policies  for  the  new  system  that  supports  the  collaborative  design  process. 
We  named  it  “FutureBoard”. 

1 . Tracing  Paper  Function 

FutureBoard  contains  8 layers  of  tracing  paper  wherein  the  level  of  transparency  can  be  modified.  The  lowest  layer 
is  called  the  “Master  Layer”,  a special  layer  for  guiding  design  activities  such  as  Maps,  Axes,  Tables,  etc. 


2.  Pen  Input  and  Drawing  Tool 

A user  can  employ  the  pen  input  (or  his/her  finger)  similar  to  ordinary  paper  resulting  in  the  interaction  being  much 
quicker.  Moreover,  the  software  becomes  not  just  a paint  tool  but  a drawing  tool,  so  that  each  object  drawn  can  be 
moved  or  deleted. 

3.  Personal  Board  and  Collaboration  Board 

Two  kinds  of  board  system  have  been  developed:  one  is  a Personal  Board  for  supporting  the  personal  reflective 
thinking  process  based  on  a wireless  note  PC,  and  the  other  is  a Collaboration  Board  for  supporting  collaborative 
learning  and  the  creative  process  based  on  a 50  inch  monitor  with  pen  input  support. 

Personal  Board  and  Collaboration  Board 

The  hardware  of  the  Collaboration  Board  system  (CB)  consists  of  a PC  and  a 50  inch  projection  ivioiiilor  with  pen 
input  support  and  a wireless  LAN  station. 

The  software  of  CB  is  a simple  drawing  programme  with  7 half  transparent  layers.  The  user  can  control  the 
visible/invisible  aspects  of  the  specific  layers  as  well  as  control  the  5 levels  of  transparency.  All  operations  are 
assigned  to  the  icons,  thereby  eliminating  the  menu  bar. 

The  hardware  of  the  Personal  Board  (PB)  system  consists  of  a pen-based  PC  and  a wireless  LAN  card. 

The  software  of  PB  is  a subset  of  the  Collaboration  Board  that  has  two  layers  and  is  optimized  for  SVGA  screen. 
(The  CB  is  of  XGA  size) 

Initially,  students  will  use  the  PB,  and  then  send  the  work  to  the  CB’s  layer.  With  the  CB,  they  can  lay  ideas  on  the 
other  half  of  the  transparent  layers.  And  CB's  data  can  be  downloaded  to  a PB  using  the  wireless  LAN  system. 


FIGURE  1:  Jr.HS  students  using  Personal  Board  and  Collaboration  Board 

Evaluation 

The  FutureBoard  system  was  evaluated  from  November  to  December  of  1999  at  Utase  Junior  High  School  in  Chiba 
Prefecture,  JAPAN.  The  system  was  used  in  project-based  learning  about  “Makuhari  Bay  Town”  where  the  students 
live.  To  meet  the  goals  of  the  curriculum.  Small  groups  were  formed.  The  students  explored  and  researched  many 
aspects  of  their  town,  such  as  the  traffic  system  and  road  safety,  the  soundscape  of  the  town.  They  reported  the  result 
of  their  research  on  the  Personal  Board  and  transferred  the  data  to  the  Collaboration  Board.  Using  Collaboration 
board.  They  discovered  new  ideas  from  the  overlapped  information  and  made  final  presentations  to  an  audience 
composed  of  classmates,  teachers,  municipal  officials  and  parents. 

We  are  now  evaluating  how  these  tools  supported  the  collaborative  process  by  protocol  analysis  and  interviewing. 
The  main  points  of  the  evaluation  center  around  how  8 half-transparent  layers  and  drawing  tools  work  as  a cognitive 
resource  in  collaborative  process. 
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Abstract:  In  this  paper  we  describe  how  elementary  school  students  used  physical 
manipulatives  in  conjunction  with  the  digital  interface  of  educational  software  for  geometry. 
The  blending  of  physical  manipulatives  and  digital  interface  may  help  them  to  overcome  the 
limits  of  the  representation  and  interaction  modalities  of  the  digital  interface. 


Introduction 

Our  study  used  SuperTangrams  (Sedighian  & Westrom  1997),  a program  developed  by  the  EGEMS  group  at  the 
University  of  British  Columbia,  We  introduced  manipulatives  that  students  could  use  while  playing  with  the 
software.  The  idea  was  to  use  manipulatives  in  conjunction  with  the  software  in  order  to  overcome  the 
representational  problems  of  the  interface,  Manipulatives  allow  the  students  to  experience  what  they  are  doing  on 
the  screen  in  the  real  world, 

SuperTangrams,  SuperTangrams  is  a computer-based  learning  environment  for  two-dimensional 
transformational  geometry.  Learning  of  this  mathematical  domain  is  situated  in  the  context  of  the  traditional 
tangram  puzzles  game  activity.  The  player  is  challenged  to  put  together  several  geometric  shapes  to  fill  a given 
outline.  To  move  the  shapes  on  the  screen  the  student  selects  and  applies  to  them  one  of  three  transformations: 
translation,  rotation,  and  reflection. 

Subjects  and  Settings,  The  study  involved  approximately  180  fifth  grade  students  at  a College  Station,  Texas, 
elementary  school.  The  task  of  each  student  was  to  play  with  SuperTangrams  to  solve  the  tangram  puzzles.  The 
student  was  free  to  use  the  tools  that  we  placed  on  the  side  of  his  computer.  Data  collection  techniques  included 
class  observations,  video-recordings,  scratch  paper  used,  pre-tests  and  post-tests,  questionnaires  and  log  files  of 
computer  sessions. 


Use  of  Physical  Manipulatives 

First  set  of  manipulatives 

During  a first  study  session  (two  weeks),  we  provided  the  students  with  the  following  manipulatives: 

1)  a set  of  tangram  plastic  shapes,  2)  a reflection  mirror,  3)  a colored  grid  board  with  a rotating  stick, 

TTie  board  had  two  different  drawings  on  it:  a line  for  reflection  and  a circle  divided  in  16  sectors  of  equal  size. 
This  is  consistent  with  the  software  that  allows  the  minimal  angle  to  be  1/16*  of  a circle,  A stick  is  connected  to 
a hub  at  the  center  of  the  circle  and  can  rotate  360°,  The  idea  was  that  the  students  would  place  a shape  on  the 
circle  and  then  rotate  it  by  pushing  against  it  with  the  stick.  We  intended  for  them  to  use  the  reflection  mirror 
with  the  line  drawn  on  the  grid  board  to  find  the  reflection  line. 

Rotation,  One  common  use  of  the  shapes  was  to  put  them  on  the  screen  and  animate  them.  A video  recording 
shows  a girl  using  them  for  rotation.  She  started  superimposing  one  plastic  shape  on  the  digital  shape  on  the 
screen,  and  then  she  moved  the  shape  along  the  arc  of  rotation  trying  to  move  and  rotate  it  at  the  same  time.  In 
this  operation  she  paid  attention  to  the  position  of  the  shape  on  the  screen,  but  not  to  the  orientation,  or  angle  of 
rotation.  The  students  who  used  the  grid  positioned  the  plastic  shape  on  the  circular  drawing,  trying  to  position 
the  shape  in  the  same  orientation  as  the  one  on  the  screen.  Then  they  would  push  the  shape  with  the  stick  until 
the  shape  could  reach  the  orientation  of  the  target.  They  would  count  the  number  of  sectors  the  stick  moved  and 


move  the  digital  handle  accordingly.  Some  students  felt  comfortable  with  the  board,  while  some  others  preferred 
to  bring  the  shapes  directly  to  the  screen. 

Reflection.  Students  did  not  use  the  mirror  on  the  grid  board  with  the  plastic  shapes  as  we  had  expected.  Instead, 
they  placed  the  mirror  directly  on  the  screen  and  looked  for  the  reflected  image  of  the  digital  shape.  The  first 
operation  involved  was  to  position  the  mirror  until  they  could  see  the  reflected  image  superimposed  on  the  target. 
Once  the  position  was  found,  they  had  to  play  with  the  digital  handles  to  position  the  reflection  line  along  the 
position  found  for  the  mirror. 


New  M anipulatives 

The  use  the  students  made  of  the  manipulatives  surprised  us,  since  they  preferred  to  use  them  on  the  screen, 
merging  the  physical  operations  with  the  digital  ones.  Thus,  for  the  second  session,  with  a second  group  of 
students,  we  modified  the  manipulatives  to  make  it  easier  for  the  students  to  use  them  on  the  screen.  We 
eliminated  the  grid  board  and  built  a rotation  tool  consisting  of  two  popsicle  sticks  joined  together  at  one  end  in 
such  a way  that  they  could  rotate,  and  placed  some  sticky  paste  on  the  other  ends.  The  popsicle  stick  had  more 
success  than  the  grid.  The  students  would  stick  the  chosen  shape  at  the  end  of  one  stick,  and  position  the  two 
overlying  sticks  directly  on  the  digital  radius  of  rotation,  with  the  plastic  shape  matching  the  digital  shape.  Then 
they  would  rotate  the  stick  holding  the  shape  and  keep  the  empty  stick  in  the  initial  position.  When  the  plastic 
shape  reached  the  target,  the  opening  of  the  two  popsicle  sticks  represented  the  sought  after  angle.  In  this  case 
they  could  hold  the  right  "angle"  in  their  hand.  Finally,  they  would  recreate  the  digital  angle  playing  with 
handles,  holding  the  physical  one  as  a model. 


Student  reactions 

The  second  set  of  tools  had  more  success  than  the  first.  It  was  more  natural  for  the  children  to  bring  the  tools  to 
the  screen,  and  they  could  physically  experiment  with  the  transformations.  In  the  questionnaires  we  asked 
children  if  it  was  good  to  have  manipulatives,  and  if  they  helped  them  learn.  Here  are  some  of  the  answers: 

*'They  helped  because  they  were  3D;"  "they  helped  you  to  learn  the  movement;"  "all  tools  because  they 
helped  you  think  ahead;"  "you  need  some  reference  to  look  at;"  "because  it  is  important  to  have  stuff  you 
can  work  with  in  your  hand;"  "it  helps  understand. " 


Conclusions  And  Future  Work 

Software  running  on  a computer  is  something  abstract:  when  the  student  drags  a handle  with  the  mouse 
something  happens  on  the  screen.  In  educational  software  for  geometry  it  is  important  that  the  student  learns  to 
relate  what  happens  on  the  screen  to  the  real  word.  Our  preliminary  findings  indicate  that  physical  manipulatives 
can  be  a valuable  addition  to  educational  software.  We  noticed  how  the  use  of  manipulatives  helped  students 
overcome  limits  of  the  representation  and  of  the  types  of  interaction  allowed  by  the  interface.  For  example, 
additional  information  on  the  screen  whose  purpose  was  to  help  the  students,  such  as  additional  lines,  actually 
confused  them,  causing  them  to  miss  important  concepts.  Using  a physical  representation  of  those  concepts,  such 
as  an  angle  between  two  popsicle  sticks  joined  together,  helped  them  to  focus  on  the  main  aspects  of  the 
transformation  without  being  lost  in  representation  details.  During  our  study  we  collected  log  files  and  data 
relative  to  pre-test  and  post-test.  The  statistical  analysis  of  this  data  will  determine  the  effect  on  learning  of  the 
different  tools. 
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Contemporary  education  needs  to  be  connected  to  the  very  world  it  purports  to  study.  In  this  context,  media  archives  and 
documents  are  value-added  material  that  can  support  both  traditional  pedagogical  practices  and  media  literacy  related  activities. 
Public  broadcasting  archives  and  related  documents  can  play  an  edifying  role  in  offering  various  sources  through  which  to  grasp 
a host  of  current  and  historical  issues.  Recognizing  the  educational  value  of  its  media  assets,  Radio-Canada  is  working  with 
schools,  colleges,  and  universities  in  exploring  a variety  of  pedagogical  scenarios  in  which  media  content  can  complement 
traditional  study  material.  This  presentation  will  illustrate  how  Canada’s  public  broadcaster  can,  by  means  of  forging  partnerships 
with  various  organizations  (multimedia,  cultural,  or  educational),  and  repurposing  of  media  archives,  play  an  active  role  in  the 
development  of  educational  Web  content  as  part  of  a constructionist  approach  to  learning. 
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Learning  Code 


Suzanne  Rhodes,  Human  Code,  USA 


Who  We  Are 

The  Learning  CodeTM  development  team  extends  beyond  the  walls  of  our  studios  to  include  the  clients  we 
work  with  and  the  learners  we  work  for  as  well  as  a passionate  and  compassionate  team  of  education 
specialists  which  includes  the  following  professionals: 

• Educators 

Interaction  Specialists 

Producers 

Artists 

Technologists 

What  We  Do 

At  Human  Code,  our  mission  is  to  humanize  technology  to  fundamentally  transform  the  way  people  learn, 
work  and  play.  Human  Code's  Learning  Technology  Group  meets  this  challenge  by  designing  spectacular 
learning  environments. 

Our  approach  affords  the  Learning  Technology  Group  with  the  discipline  to  manage  the  complexity  of 
tasks  that  characterize  each  design  stage,  as  well  as  the  flexibility  to  provide  appropriate  consideration  to 
the  six  interrelated  components  that  define  each  of  our  spectacular  learning  environments.  These 
components  include: 

Setting 

Community 

Content 

Pedagogy 

Tools  and  Technologies 
Assessment 

How  We  Do  It 

Derived  from  years  of  best  practices  and  research  in  the  field  of  learning  technologies,  our  Learning 
CodeTM  approach  to  educational  environment  design  is  an  iterative  process  that  motivates  our  team  of 
specialists  through  the  following  comprehensive  design  stages: 

Envision 

Design 

Produce 

Implement 

Integrated,  On-going  Reflection 

Who  Benefits? 

Targeted  learners 

Learning  Code  TM  engages  the  learner  in  an  experience  that  is  personal,  relevant  and  meaningful.  Our 
environments  engender  a love  of  learning,  content  mastery,  higher-order  thinking,  and  inspire  life-long 
learning. 

Clients  and  Educational  Partners 
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Learning  Code’s  collaborative  nature  transforms  clients  into  development  associates.  Skillfully  managed 
projects,  delivered  on-time  and  on-budget  meet  the  needs,  goals,  and  expectations  of  our  client  partners. 

Human  Code 

Each  project  affords  our  company  teams  with  heightened  knowledge,  skills,  and  abilities  that  inform  future 
designs  and  techniques.  The  result  is  continually  evolving  processes,  products,  and  services. 

About  Human  Code 

Human  Code  is  a leading  developer  of  interactive  experiences  for  e-commerce,  games,  learning  systems 
and  marketing  communications.  The  company’s  approach  combines  creativity,  consulting  and  technology 
to  create  user-centered  solutions  that  change  the  way  people  learn,  work  and  play.  Human  Code's  creativity 
and  client-semce  focus  are  maintained  through  the  company’s  Studio  Network,  which  includes  Austin 
Studios,  Presage  Studios  in  San  Rafael,  CA,  Human  Code  Japan  in  Tokyo,  and  Human  Code  Shanghai. 

The  company  was  founded  by  industrial  designers  in  Austin,  Texas  in  1993,  and  received  venture 
investments  from  Austin  Ventures  and  Applied  Technology  in  1995  and  1999.  Additional  information  on 
Human  Code  can  be  found  at  www.humancode.com. 

Human  Code’s  Learning  Technologies  Group  is  emerging  as  a market  leader  in  creating  spectacular 
learning  environments.  From  a statewide  initiative  in  Idaho  to  programs  that  provide  entry-level  technology 
workers  with  training  in  hard  and  soft  skills.  Human  Code  is  becoming  known  as  a true  total  solutions 
provider  for  educational  technologies. 


Teaching  and  learning  in  a virtual  campus: 

The  model  of  the  Uni  vers  itatOberta  de  Catalunya 


Albert  Sangra 
e-Education  Laboratory 
Universitat  Oberta  de  Catalunya 
asangra@campus.uoc.es 

Lourdes  Guardia 

Head  of  Instructional  Design  Unit 
Universitat  Oberta  de  Catalunya 
lguardia@campus.uoc.es 


By  combining  the  latest  technologies  with  both  a solid  pedagogical  foundation  and  continuous  evaluation,  the 
Universitat  Oberta  de  Catalunya  has  built  a Virtual  Campus:  an  electronic  communications  environment  that 
provides  the  means  to  effectively  support  the  tutoring  of  students,  and  the  provision  of  university  resources  and 
social  activities. 

It  has  presently  a total  of  13.000  registered  students.  Eight  main  degree  programs  are  offered,  as  well  as  Master 
programs,  postgraduate  courses  and  a wide  offer  of  continuous  education  programs. 

The  organization  is  also  working  on  the  Metacampus,  which  mission  is  to  demonstrate  the  global,  virtual  co- 
operation of  universities;  a concept  known  as  the  flexible  university  which  is  achieved  by  interconnecting  the 
partners  virtual  services  and  infrastructures. 

It  will  be  able  to  capture,  register,  invoice,  teach,  assess  and  accredit  courses  and  provide  all  the  administrative 
services  in  a global  scale,  using  the  Internet  as  the  only  medium  to  support  all  these  educational  processes. 


Deploying  Web-Based  Assessments 

Eric  Shepherd 


Learn  the  principles  of  creating  and  administering  Web-based  tests  and  assessments  using  Question  Mark''’” 
Perception^”  software.  Find  out  how  to  use  Perception’s  authoring  wizard  to  create  questions  in  a variety  of 
formats:  multiple  choice,  multiple  response,  hot  spot,  text,  numeric,  selection,  and  matrix  questions  as  well 
as  explanation  screens  that  can  include  text,  graphics,  and  multimedia.  Participants  will  also  learn  how  to 
create  multiple  question  banks  from  which  to  assemble  tests,  surveys,  and  questionnaires.  Other  topics 
include  how  to  set  up  interactive  feedback,  create  adaptive  tests,  analyze  test  results,  and  create  reports.  . 
Example  questions  will  be  presented  from  various  applications  including  competency  testing,  employee 
recruitment,  customer  satisfaction  questionnaires,  study  aids,  diagnostic  tests,  skiiis  assessments,  product 
knowledge  exams,  course  evaluations,  and  certifications.  The  presentation  will  cover  such  issues  as  test 
security,  linking  with  learning  management  systems,  and  the  use  of  Perception  Secure  Browser  for  high- 
stakes  tests. 
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The  LeamingStation.com 

The  LeamingStation.com  is  dedicated  to  providing  a realistic  and  cost-effective  way  to  manage 
technology  and  make  it  available  to  the  teachers,  students,  and  administrators  who  will  transform 
the  educational  process  to  meet  the  expectations  of  the  information  economy. 

The  LeamingStation.com  is  the  leading  and  largest  Application  Service  Provider  (ASP)  and 
portal  in  the  educational  market,  providing  online  access  to  educational  content,  tools,  and 
applications,  on  a subscription  basis.  With  The  LeamingStation.com,  a managed  and  maintained 
library  of  applications  is  made  available  to  PCs,  Macintosh  systems,  and  thin-client  devices 
across  the  school’s  network  and  over  the  Internet.  The  LeamingStation.com  Chief  of  Innovation 
and  Co-Founder  Jim  Pennington  states,  ^'Schools  are  extremely  excited  to  hear  that  they  can  get 
out  of  the  software  management  business,  and  focus  on  using  software  rather  than  delivering  it,  ** 
says  Pennington,  ''and  the  fact  that  they  can  make  good  use  of  their  installed  base  of  cross- 
platform computers  has  a significant  impact  on  their  ability  to  make  technology  accessible  to 
more  students  and  teachers. " 

The  Education  Market  Opportunity 

Technology  is  now  accepted  as  an  integral  part  of  education  in  Canada.  As  awareness  of  the 
importance  of  preparing  students  for  the  information  age  grows,  school  boards  are  turning  an 
increasing  amount  of  attention  to  this  process.  Educational  spending  now  accounts  for  8%  of 
Canada’s  budget,  and  technology  is  fast  becoming  a priority,  including  significant  investment  in 
hardware,  software,  and  cabling.  In  the  1997  - 1998  school  year,  $230  million  was  invested  in 
bringing  computer  technology  into  Canadian  classrooms. 

Despite  a high  level  of  spending,  most  schools  are  equipped  with  a collection  of  computers 
ranging  from  Apple  II  to  x386  to  Pentium-based  computers.  This  presents  an  implementation 
nightmare.  It  is  currently  impossible  for  schools  to  incorporate  new  software  applications  into 
their  curriculum  without  consistently  upgrading  their  hardware,  a situation  which  is  neither 
desirable  nor  cost-efficient.  Regular  overhauls  of  computer  equipment  is  financially  impossible 
for  schools  with  limited  budgets.  This  issue  is  probably  the  single  largest  barrier  to  institutional 
technology  integration. 

The  Leamingstation.com  presents  a solution  to  this  problem.  It  has  the  potential  to  completely 
reorganize  instmctional  computing  as  we  know  it.  It  would  allow  schools  to  run  brand  new 
applications,  as  well  as  provide  web  access  and  e-mail,  while  removing  the  financial  burden  of 
having  to  constantly  upgrade  hardware.  It  would  also  allow  schools  to  use  equipment  donated 
through  programs  such  as  the  Computers  for  Schools  Program,  which  provides  schools  with 
recycled  business  and  government  computers. 

What  is  thin-client  computing? 

Citrix  Systems,  the  world  leader  in  server-based  computing,  reports  over  2,000,000  ports  in  place 
supporting  over  10,000,000  users  today.  The  idea  behind  the  thin  client  is  simple:  concentrate 
computing  power,  storage,  applications,  and  data  on  "servers"  (powerful  computers)  and  provide 
users  with  a simple  "client"  computer  that  is  easy  to  install  and  maintenance-free.  Users  connect 
to  The  LeamingStation.com  servers  over  the  Internet  to  process  applications,  access  files,  print, 
and  perform  all  other 

services  available  to  ordinary  computers.  Since  the  applications  and  data  are  kept  on  a server, 
only  keystrokes,  mouse  clicks  and  screen  updates  travel  the  network,  so  just  a fraction  of  the 
normal  bandwidth  is  required  to  run  applications.  Thin-client  delivery  has  the  flexibility  to 
support  any  device — from  full-featured  computers  like  PCs,  Macintosh  systems,  and  UNIX 
workstations,  to  Windows-based  terminals  and  hand-held  information  appliances. 


How  do  schools  benefit  from  using  an  Application  Service  Provider  (ASP)? 

• Reliability  and  Ease  of  Use 

ASPs  allows  teachers  and  students  to  spend  more  time  working  on  technology  projects  such  as 
accessing  information  on  the  Internet  and  using  educational  software — rather  than  spending  class 
time  fixing  and  upgrading  PCs  or  adjusting  software  programs  to  suit  each  individual 
workstation.  Users  simply  log  on  and  access  their  applications  and  data. 

• Simplified,  Centralized  Management 

The  LeamingStation.com’ s servers  are  managed  by  skilled  network  administrators, 
removing  this  responsibility  from  educators.  Updates  and  additions  are  made  only  once 
at  the  server  and  are  immediately  available  to  all  users. 

• Lower  Cost  of  Technology 

Zona  Research,  a technology  research  firm,  estimates  that  organizations  will  save  54-57%  of  their 
systems  administration  costs  over  a period  of  5 years  with  a thin-client  computing  approach. 
Using  the  services  of  an  ASP  further  reduces  costs  by  eliminating  software  upgrades  and 
inefficient  purchasing  volumes. 

• Security 

Vital  data  and  applications  may  be  kept  on  The  LeamingStation.com  servers  creating  a 
higher  level  of  security.  Sensitive  applications  and  data  are  made  available  to  authorized 
users  only  via  password  protection.  Data  is  effectively  secured  and  backed-up  in  one  central 
location  either  on  a local  customer  server  or  at  The  LeamingStation.com. 

• Universal  Access 

Since  applications  and  data  are  stored  on  the  server,  access  is  available  to  students  and 
faculty  from  any  machine  at  any  location  and  at  any  time  via  an  Internet  connection  and 
the  desktop  appearance  is  always  consistent. 

Strategic  Alliances 

The  LeamingStation.com  has  created  a network  of  strategic  partners  supporting  thin-client 
delivery  of  applications  to  the  education  market.  The  LeamingStation.com  is  continually  looking 
for  new  strategic  partners  to  enhance  it’s  offering  to  education  customers. 

Software  Publishers  and  Online  Content — The  LeamingStation.com  has  strategic 
agreements  to  offer  products  from: 

• Microsoft 

• The  Learning  Company 

• Sunburst  Communications 

• Saratoga  Group 

• Infonautics 

• Schepp-Tumer  Productions 

• CBT  Training 

• Lotus 

• Learning  Outfitters 

• Steck  Vaughn 

• MediaSeek 

• Webivore 

• Fog  Cutter 

• SkillsBank 

• And  many  others 

Technology  Partners — The  LeamingStation.com  has  strategic  agreements  with: 


• Citrix 

• Sun  Microsystems 

• Empower  Corporation 

• MCNC 

• Wyse  Technology 

• Computer  Network  Power 

• Boundless  Technologies 

• Boca  Research 

• OZ  New  Media 

Server  Access  Accounts 

The  LeamiiigSlalion.com  customers  use  Server  Access  Accounts  (SAA)  that  allow  access  to  the 
LeamingStation.com  server  farm  via  the  Internet.  Users  have  access  to  a basic  package  of 
applications  and  services  that  include: 

• Reference. Center,  a customized  version  of  Infonau tic’s  Electronic  Library 

• Adventure  Online,  real  world-based  core  learning  materials 

• Star  Office  Suite,  an  online  suite  of  productivity  tools 

• Facilitator,  conferencing  tools  for  collaborative  learning 

• Internet  filtering  and  vims  protection  utilities 

Additional  products  and  services  can  be  added  as  desired  including: 

• Microsoft  Office  or  Works 

• SkillsBank  or  Cornerstone  basic  skills  software 

• Classroom  Planner,  a comprehensive  teacher  productivity  tool 

• Webivore,  a directory  of  educationally  reviewed  web  content 

• Explorasource,  a curriculum  correlation  tool  for  teachers. 

• Many,  many  more 

Schools  require  a Local  Area  Network  (LAN)  to  provide  users  access  to  The 
LeamingStation.com.  A T-1  line,  or  a fast  broadband  connection  like  high  speed  cable 
modems,  is  recommended  for  school  site  access,  while  an  individual  account  can  gain 
access  over  a modem.  SAA’s  are  for  concurrent  usage,  so  schools  only  need  to  purchase  as  many 
accounts  as  they  will  use  at  one  time.  Access  is  24  hours  a day,  seven  days  a week,  allowing 
teachers,  students  and  parents  to  gain  access  from  home  and  after  hours  as  well. 

Total  Cost  of  Ow^nership 

Typically,  educational  technology  purchase  decisions  are  made  on  the  basis  of  acquisition  cost 
and  compatibility.  As  technology  usage  becomes  more  pervasive,  educators  will  need  to  pay 
more  attention  to  the  Total  Cost  of  Ownership  (TCO)  which  includes  support  and  maintenance 
costs,  reliability,  and  lifecycle  replacement  costs.  And  in  these  areas,  thin  clients  continue  to  out 
perform  new  PCs  in  cost  effectiveness  by  a wide  margin. 

Zona  Research,  a technology  research  firm,  estimates  that  organizations  will  save  54-57%  of  their 
systems  administration  costs  over  a period  of  5 years  with  a thin-client  computing  approach.  And 
this  cost  savings  is  in  addition  to  the  lower  cost  thin-client  devices  and  ability  to  leverage  your 
current  computing  infrastructure — hardware,  applications,  networks  and  training — to  extend  the 
reach  of  your  thin-client  network. 

A study  by  the  Gartner  Group  released  in  June  1999  estimated  that  thin-client  applications  require 
80  percent  less  management  than  traditional  PC  deployment  (Computer  Reseller  News  Online, 
6/1/99). 
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Here  are  some  details  of  the  Gartner  Group  Study  for  annual  PC  cost  of  ownership: 

• Actual  cost  of  the  PC  Three-year  depreciation  $677 

• Non-Labor  Expenses  Software,  network  connectivity,  etc.  $1677 

• Technical  Support  Service,  Maintenance,  Help  desk  $1089 

• Network  Capital  Cabling,  Servers,  etc.  $585 

• Network  Support  Management,  virus  control,  etc.  $544 

• End  User  Admin  Backups,  upgrades,  productivity  loss  $981  from  downtime,  etc. 

Total  Cost  of  Ownership  $6507 

Thin-Client  Computing  Cost  Savings 

While  schools  typicdlly  cannot  provide  the  type  of  support  thal  is  described  above,  the 
information  serves  to  illustrate  the  cost  savings  that  thin  client  computing  provides.  In  fact,  the 
lack  of  resources  available  to  support  PCs  makes  an  even  stronger  case  for  thin-client  devices  in 
education.  The  cost  savings  of  thin-client  computing  in  education  are  just  starting  to  be 
quantified: 

• Thin  client  devices  can  have  an  effective  life  of  5 - 10  years 

• Your  current  PC’s  can  be  easily  utilized  as  Thin  client  devices 

• Software  is  a fixed  annual  expense  without  hidden  version  upgrade  expenses 

• Network  support  can  be  provided  centrally  on  servers  instead  of  desktops 

• End  users  have  a common,  easy-to-leam  desktop  environment 

• Downtime  loss  from  viruses,  hardware  failures  and  vandalism  are  all  but  eliminated 

• Universal  access  enhances  productivity  and  leverages  the  technology  to  more  users 


Campus  Notebook  Computer  Programmes 


Craig  Blurton,  University  of  Hong  Kong,  China 


Notebook  computer  programmes  are  proliferating  as  many  universities  move  from  "corporate"  ownership 
of  computing  resources  to  individual  ownership.  Benefits  of  such  programmes  have  been  widely  touted  by 
universities  and  commercial  notebook  computer  vendors  including  enhancement  of  universities  marketing 
position,  decreased  IT  expenses  because  of  cost-sharing  between  institutions  and  students,  extended 
learning  opportunities,  and  enhanced  IT  skills. 


But  how  each  individual  programme  is  implemented  is  highly  contextualized  on  local  goals,  needs,  and 
circumstances.  This  special  interest  group  meeting  will  give  representatives  from  tertiary  institutions  an 
opportunity  to  interact,  share  success  stories,  identify  common  concerns,  and  perhaps  develop  collaborative 
efforts  around  the  issues  of  "assured  access"  and  "mobile  computing." 


Issues  that  may  be  discussed  include: 


* Planning  and  implementing  such  programmes. 

* Multiple  or  single  vendor  solutions 

* Cost-effectiveness 

* Bridging  the  gap  between  academia  and  business 

* Managing  vendor  relationships 

* Equity  of  implementation 

* Staff  support  and  development 

* Barriers  to  success 

* Programme  evaluation 


Providing  Quality  Distance  Education  in  an  Outcomes-Oriented  Program 


Dr.  Ginny  Engman 
Fort  Lewis  College 
52  Hesperus  Hall 
Durango,  Colorado  81301.. 
engman_v@FORTLEWIS.EDU 


Abstract:  rroviding  On-line  programs  for  pre-service  teachers  has  presented  many  issues. 

Assessing  the  effectiveness  of  one’s  teaching  in  the  classroom  is  difficult  under  the  best  of 
circumstances;  doing  so  through  distance  education  is  even  more  challenging.  The  program  at  Fort 
Lewis  College  has  addressed  these  issues  and,  while  resolving  many  of  them,  has  developed  some 
concerns  that  require  further  study. 


Quality  Distance  Education  in  an  Outcomes-Oriented  Program 

The  development  of  an  on-line  program  in  elementary  and  early  childhood  licensure  has  raised  many  issues.  It  has 
also  caused  some  dissension  among  the  faculty.  The  program  consists  of  all  courses  in  the  traditional  program  being 
offered  through  Top  Class  and  Web  CT.  Currently  two  faculty  (of  the  10  full-time  faculty)  have  been  responsible 
for  the  majority  of  the  program  delivery.  There  has  been  an  increase  in  the  number  of  students  who  have  opted  for 
this  avenue  since  its  inception  three  years  ago.  Students  are  admitted  to  the  program  in  the  same  way  that  students 
are  admitted  to  the  on-campus  program;  minimum  gpa  of  2.5,  proof  of  having  worked  with  children,  pass  a state 
basic  skills  test,  pass  and  oral  exam,  and  adequate  ACT/SAT  score. 


Causes  For  Concern 

There  are  three  major  causes  for  concern.  First,  after  an  initial  orientation  meeting  on  campus,  the  faculty  have  little 
or  no  face-to-face  contact  with  the  students.  They  must  rely  on  the  written  evaluations  and  comments  of  the  public 
school  teachers  with  whom  the  students  are  placed.  Historically,  these  assessments  have  been  somewhat  skewed 
toward  the  positive.  In  fact,  the  teachers  are  willing  to  be  more  objective  if  they  do  not  have  to  put  their  observation 
in  writing.  However,  for  legal  purposes  we  need  to  keep  a “paper  trail”.  The  problem  arises  when  we  face  a 
disconnect  between  what  the  teachers  are  writing  and  the  actual  effectiveness  of  the  student  (teacher  candidate).  It 
comes  to  a head  at  the  student  teaching  experience — after  the  student  has  finished  all  course  work  and  pre-student 
teaching  practica.  The  dilemma  we  face  is  that  the  students  are  out  of  our  service  are  (hence,  the  term  distance 
education)  so  that  it  is  not  possible  for  our  faculty  to  observe  and  evaluate  them  on-site,  when  predicted  teacher  How 
do  we  balance  this  with  our  commitment  to  preparing  qualified,  caring  and  competent  teachers? 

Second,  besides  the  teacher  education  courses,  there  is  a need  for  courses  outside  the  department  to  be  delivered  on- 
line. There  are  auxiliary  courses  in  Exercise  Science,  Art  and  Music  that  our  students  need  as  part  of  their  licensure 
program.  Additionally,  if  the  are  seeking  a degree  in  Arts  and  Sciences,  they  will  need  to  take  classes  in  they  major. 
There  has  been  much  resistance  on  the  part  of  faculty  to  develop  an  electronic  delivery  of  their  courses.  How  can 
we  break  through  the  barrier  that  keeps  faculty  from  taking  a risk  in  the  area  of  distance  education? 

Third,  and  lastly,  until  recently,  the  upper  level  administration  has  been  reluctant  (not  necessarily  resistant)  to  invest 
significantly  in  distance  education.  There  is  a skepticism  about  the  ability  to  have  high  quality  and  require  high 
standards  in  an  electronically  delivered  program.  How  can  we  educate  decision-makers  so  that  we  can  get  their  buy- 
in  for  distance  education? 
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How  to  Prepare  Teachers  for  Integrating  Multimedia  and  Technology 

into  K-12  Classrooms 


Barbara  G.  Foster,  Ed.D.  - Spalding  University:  Louisville,  KY  USA  bfoster@spalding.edu 


Abstract:  To  meet  the  needs  of  diverse  learners,  teachers  must  be  able  to  integrate  multimedia 
and  technology  into  K-12  classrooms.  Participants  will  walk  away  from  this  session  with 
practical  assignments  and  constructivist  activities  to  enhance  professional  productivity  and 
support  instruction,  successful  assessment  strategies  that  promote  positive  attitudes  for 
facilitating  the  lifelong  learning  of  various  technologies  by  teachers  and  students,  and  samples 
of  student  work.  Topics  will  include:  knowledge  of  basic  multimedia  terminology, 
applications,  and  technologies;  development  of  basic  skills  in  operating  multimedia  hardware 
to  support  instruction  and  learning;  development  of  basic  skills  in  using  and  integrating  the 
Internet  and  multimedia  applications  into  the  curriculum  to  meet  the  needs  of  learners  at 
different  age  levels  and  in  different  subject  areas;  experience  in  making  presentations  using 
multimedia  technologies;  experience  in  designing  and  planning  instruction  that  integrates 
multimedia  technologies  into  the  curriculum;  and  awareness  of  the  copyright  laws  as  they 
relate  to  educational  technology. 
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Educational  Technology  and  Context:  an  exploration  of  values,  roles, 
and  concerns  from  Lecturers,  tutors  and  instructors. 


Alison  Hudson 

Learning  and  Teaching  Institute, 
Sheffield  Hallam  University 
Adsetts  Centre,  Howard  Street 
Sheffield,  UK,  SI  IWB 
a.r.hudson@shu.ac.uk 


John  Steel 

Learning  and  Teaching  Institute 
Sheffield  Hallam  University 
Adsetts  Centre,  Howard  Stree 
Sheffield, UK,  SI  IWB 
i.a.steel@shu.ac.uk 


Introduction 

Educational  and  instructional  technologies  are  increasingly  becoming  part  of  learning  and 
teaching  in  higher  education  institutions.  There  is  no  doubt  therefore  that  the  adoption  of 
such  technologies  in  this  context  impacts  upon  many  areas  of  university  life.  It  could  be 
argued  however,  that  in  addition  to  significant  benefits  to  the  institution  and  its  students, 
the  increasing  use  of  educational  technologies  raise  a number  problematic  issues  and 
areas  of  uncertainty. 

This  special  interest  discussion  therefore  seeks  to  get  to  grips  with  a number  of  key  issues 
around  the  role  of  educational  technology  in  the  learning  and  teaching  equation  from  the 
perspective  of  the  lecturer/tutor/instructor.  It  is  considered  therefore  that  by  generating 
discussion  around  a number  of  important  areas  of  interest  to  educational  practitioners, 
some  improved  awareness  of  obstacles  as  well  as  examples  of  best  practice  can  be  made. 
From  this  starting  point,  greater  understanding  and  progression  may  be  encouraged  or 
developed. 
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Applications  and  categorization  of  software-based  scaffolding 


Koos  Winnips,  University  of  Twente,  Netherlands;  Catherine  McLoughlin,  University  of  New  England, 

Australia 


The  increasing  popularity  of  computer-based  learning  environments,  combined  with  a need  for  time- 
flexible  and  self-reliant  learning,  has  caused  an  increase  in  the  demand  for  scaffolding  embedded  in 
instructional  software  environments  (see  for  example  Acovelli  & Gamble,  1997;  Guzdial  & Kehoe,  1998; 
Tabak  & Reiser,  1997;  Rada  & Yazdani,  1998).  The  aim  of  software-based  scaffolding  is  to  provide  some 
of  the  same  kinds  of  support  a teacher  could  provide  in  a clas.sroom  setting,  but  now  in  a computer-based 
learning  environment.  In  this  setting  it  is  assumed  that  face-to-face  contact  between  student  and  teacher  is 
reduced  or  impossible.  Software  scaffolding  can  be  defined  by  the  following  three  characteristics  (Guzdial, 
1995,  Zhao,  1997,  Winnips,  1998): 

- Modeling:  the  desired  behavior  is  modeled  by  providing  a kind  of  structure,  communicating  what  is 
desired,  or  presentation  of  an  expert  model. 

- Support  is  given  to  the  learner  so  that  the  learner  can  perform  a task  independently. 

- Fading  takes  place  so  that  students  become  self-reliant. 

While  many  applications  and  examples  of  software  scaffolding  are  now  known,  a categorization  of  these 
examples  has  not  emerged  yet.  Applications  of  scaffolding  can  be  categorized  according  to  who  regulates 
the  scaffolding  (teacher,  peer-student,  computer,  self),  technology  used,  pedagogy  used,  or  according  to  its 
intended  learning  outcome.  An  example  of  a categorization,  based  on  types  of  support  given  is  found 
below: 

- Providing  examples:  ideally  these  examples  should  not  only  focus  on  products,  but  also  on  a process. 
Providing  examples  links  to  the  concept  of  modeling  as  Bandura  (1965)  described  it,  where  a teacher  would 
serve  as  a role  model. 

- Helping  students,  by  giving  away  (physical)  parts  of  the  solution.  This  can  serve  to  help  students  with  a 
solution,  or  the  help  may  just  serve  to  support  students  on  very  time-consuming  tasks  that  are  not 
absolutely  necessary  for  learning  (Acovelli  & Gamble,  1997). 

- Providing  a model  for  design,  or  a structure  to  design  in.  For  example  providing  design  guidelines  (Collis 
& Winnips,  1998). 

- Cueing^inting:  helping  students  with  a solution  by  providing  a hint  or  cue  to  a possible  path  of  the 
solution  (Acovelli  & Gamble,  1 997). 

- Coaching  comments  (Jonassen,  1998).  These  comments  are  intended  for  motivation,  providing  feedback 
and  advice  on  performance,  and  provoking  reflection. 

- Asking  questions,  pointing  out  weaknesses,  asking  for  a motivation,  in  order  to  enhance  reflection. 

- Metacognitive  support:  stating  why  the  above  types  of  support  are  given,  in  order  to  model  the  type  of 
metacognition  that  experts  would  use. 

- Providing  a timeline,  with  fixed  dates  and  goals  built  in.  This  structure  could  be  present,  to  help  students 
appearring  to  be  very  goal  directed  to  build  in  multiple  evaluation  moments  into  the  actual  experience  of 
studying. 

Aim  of  this  SIG  discussion  is  to  reach  consensus  about  a number  of  categories  for  scaffolding  and  to 
construct  guidelines  about  the  relationship  of  learning  outcomes  to  particular  forms  of  scaffolding.  This  in 
order  to  provide  structure  for  future  research  and  to  be  able  to  better  link  previous  experiences  in  software 
scaffolding  to  future  experiments. 
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20 
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WORKSHOP  OBJECTIVES 

By  the  completion  of  the  workshop  participants  should  be  able  to; 

1.  understand  the  processes,  phases  and  activities  involved  in  planning  online  courses  and 
learning  resources 

2.  develop  a persuasive  proposal  for  project  funding  including  concept,  rationale,  team 
formation,  learner  profile,  timeline  and  budget 

3.  locate  potential  sources  of  project  funding 

4.  understand  the  pedagogic  principles  of  designing  online  courses  and  learning  resources 

5.  identify  project  learning  and  teaching  strategies 

6.  develop  the  content  for  online  courses  and  learning  resources 

7.  understand  the  legal  issues  involved  with  the  design  of  online  courses  and  learning 


INTENDED  AUDIENCE/EXPERTISE  LEVEL 

This  tutorial  is  for  faculty  and  staff  developers  who  want  to  fund,  design,  develop,  implement 
and  evaluate  online  courses  and  multiple  media  learning  resources.  The  tutorial  is  set  at  an 
introductory  and  moderate  level. 


resources 
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ABSTRACT 


In  this  tutorial  we  discuss  and  commence  the  crucial  planning  tasks  required  to 
successfully  fund,  design,  develop,  implement  and  evaluate  online  courses  and  learning 
resources.  The  tutorial  will  be  of  most  benefit  to  participants  who  come  with  a specific 
project  idea  they  want  to  develop  further. 

We  discuss  a learner-centred  project  framework  that  consists  of  proposal,  media 
production  and  educational  design  processes,  phases  and  activities.  You  will  use  the 
framework  to  start  developing  a persuasive  proposal  including  the  project  concept, 
rationale,  objectives,  team  fcrmaticn  and  learner  profile.  You  then  use  the  framework  to 
develop  sound  project  timelines  and  a justifiable  budget.  Potential  sources  of  project 
funding  are  identified. 

Design  principles  for  online  courses  and  learning  resources  are  discussed  based  on  the 
instructional  design  process.  Bigg's  concept  of  'alignment',  and  Laurillard's 
'conversational  model'  of  the  learning  and  teaching  process.  You  will  commence  identifying 
your  project's  learning  and  teaching  strategies.  We  discuss  methods  of  developing  the 
content  for  your  project  and  the  relevant  legal  issues. 


TUTORIAL  OUTLINE 

1.  Introduction  - participant's  projects 

2.  Learner-centered  project  proposal,  media  production  & educational  design  processes, 
phases  and  activities 

3.  Activity  ^1:  [develop  pro Ject  concept,  rationale,  objectives,  team  and  learner  profile 

4.  Timelines  for  online  educational  projects 

5.  Activity  ^2:  Develop  pro Ject  timeline 

6.  Budgeting  for  online  educational  projects 

7.  Activity  #3'  Develop  project  budget 

8.  Funding  sources 

9.  Design  principles  for  online  courses  and  learning  resources 

10.  Activity  Identify  pro Ject  learning  and  teaching  strategies 

11.  Developing  educational  content 


12. 


Relevant  legal  issues 


13.  Plenary  Discussion 

14.  Conclusion  and  tutorial  evaluation 


PRESENTER'S  QUALIFICATIONS  & EXPERIENCE: 

Andrew  Litchfield  lectures  in  Learning  and  Teaching  at  the  Centre  for  Professional 
Development,  AAocquarie  University.  He  is  responsible  for  supporting  Macquarie  University 
staff  in  course  design,  the  use  of  educational  technology,  and  the  planning,  design  and 
management  of  teaching  innovations  and  learning  resources.  His  academic  experience  also 
includes  lectureships  in  Media  Communications  and  in  Educational  Technology. 

Andrew's  media  company.  Positive  Image,  has  produced  numerous  large-scale  educational 
media  projects.  He  has  extensive  project  management  and  media  design  experience  and  has 
produced  four  award-winning  projects  funded  by  the  Australian  Government  as  'Projects  of 
National  Significance'. 
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Research  & evaluation  of  online  systems  for  teaching  & learning 

Anne  A'Herran,  James  Cook  University,  Australia 

Piecemeal  adoption  of  methods  of  online  educational  delivery  can  duplicate  effort  and  waste  an  institution's 
resources.  Early  enthusiasm  for  online  delivery  of  courses  soon  fades  without  strategic  initiatives  on  the  institution's 
part.  In  August  1999  a Web  Educational  Development  Advisor  was  appointed  at  JCU  to  research  and  evaluate 
systems  for  online  teaching  and  learning,  with  a view  to  integrating  the  selected  system  university  wide.  That 
research  and  evaluation  is  the  subject  of  this  paper. 

The  relationship  between  learners'  personality  types  to  their  performance  in  computer- 
mediated  distance  education 

Justin  Ahn,  Fairfield  University,  USA;  MiLee  Ahn,  Hanyang  University,  South  Korea 

Many  distance  education  programs  tend  to  be  focused  on  programs  rather  than  on  individuals  and  their 
characteristics  with  the  goal  of  improving  the  learners'  performance.  Moic  lesearch  into  the  effects  of  distance 
education  should  be  oriented  toward  the  individual  learners.  Little  is  known  about  the  personal  factors  that  promote 
or  inhibit  success  in  distance  education  environments.  This  study  examined  the  relationship  between  learners' 
personality  types  and  their  performance  in  computer-mediated  distance  education.  This  study  found  that  Perceiving 
types  posted  fewer  but  longer  messages,  Judging  types  posted  more  but  shorter  messages.  Feeling  types  perceived 
that  they  had  gained  knowledge  more  than  other  types,  and  Sensing  types  were  less  satisfied  with  the  CMC 
experience.  There  were  no  significant  relationships  between  personality  types  to  grades,  level  of  small  group 
collaboration,  and  level  of  leadership  influence.  Students  indicated  that  the  leadership  changed  from  topic  to  topic 
and  learning  in  CMC  can  be  achieved  without  a leader.  They  used  others'  frequency  and  quality  of  postings,  writing 
skills,  and  feedback  to  postings  as  a factor  in  determining  the  leadership. 

Tutoring  Skills  for  Instructors  in  Distance  Delivery 

Mohamed  Ally,  Athabasca  University,  Canada 

The  growth  in  use  of  the  Internet  and  other  telecommunications  technologies  in  education  is  changing  the  role  of 
the  instructor  from  that  of  a provider  of  information  to  that  of  a tutor.  This  new  role  of  the  instructor  requires 
different  skills  to  work  with  students  in  a virtual  and  distributed  environment.  This  session  will  present  the  tutoring 
skills  that  are  required  by  tutors  to  function  in  a distance  delivery  mode. 

Development  of  an  Introductory  Financial  Accounting  Text  in  Print-based  and  Electronic 
Muitimedia  Environments 

David  Annand,  School  of  Business,  Athabasca  University,  Canada 

Athabasca  University  is  Canada’s  largest  distance-based,  open  university.  Students  presently  registering  in  the 
University’s  introductory  financial  accounting  course  are  provided  with  a paper-based  textbook,  solutions  manual, 
study  g;uide,  and  assignment  manual.  These  materials  are  all  developed  and  printed  in-house.  The  800-page  textbook 
was  recently  converted  to  on-screen  presentation  format.  The  amount  of  material  was  sigmficantly  reduced  by 
moving  discussion  cases  to  an  on-line  instructors’  manual.  The  material  was  edited  for  on-screen  use  by  altering 
backgrounds,  screen  breaks,  and  page  size.  The  files  were  then  converted  into  Authorware  Professional®. 
Information  presentation  was  streamlined  and  interactive  elements  were  incorporated.  Feedback  for  selected  text 
problems  was  inserted  at  appropriate  points.  Audio/visual  segments,  two  computer  simulations,  and  navigation  and 
index  systems  were  developed.  The  final  product  will  be  packaged  on  a CD-ROM,  included  with  each  print-based 
course  package  sent  to  students,  and  incorporated  into  the  University’s  Virtual  Teaching  and  Learning  (ViTAL) 
electronic  environment. 

The  Biotechnology  Project:  A Case  Study  on  Integrating  the  Use  of  The  Internet  for 
Research  and  On-Line  Communication  in  a Brazilian  School 

Cristiana  Assumpgao,  Columbia  University,  USA 

This  project  was  developed  in  a model  private  school  in  Brazil,  Colegio  Bandeirantes,  having  the  reputation  for  its 
high  academic  level  and  vision  for  the  future.  Using  an  advanced  topic  in  Biology,  students  were  challenged  to  go 
beyond  normal  academic  activities,  acquiring  skills  to  integrate  technology  and  become  lifelong  learners.  They  have 
had  the  opportunity  to  interact  with  researchers  in  the  field  of  Biotechnology,  as  well  as  with  peers  in  New  York. 
Students  have  learned  to  use  the  Internet  as  a research  tool  as  well  as  a place  for  collaboration  and  content  building. 
This  is  a study  of  the  role  of  technology  in  empowering  students  to  become  active  learners  and  researchers,  as  well 
as  adaptation  to  these  new  methods  of  communication.  Interesting  patterns  have  emerged  on  how  students 
communicate  online  versus  face  to  face.  The  study  continues  on  how  to  create  pedagogically  sound  best  practices 
for  technology  integration. 


utilization  of  BBS  for  ESL  class 

Hideyuki  Baba,  Keio  Gijuku  Girls  High  School,  Japan 

We  introduced  the  usage  of  a BBS  system  into  ESL  learning  to  enhance  English  reading  and  writing  competence  of 
our  high  school  students.  The  BBS  system  can  enable  our  students  to  learn  English  in  a totally  relaxed  and 
comfortable  way.  They  are  expected  to  find  an  English  file  by  using  a Web  browxer  and  write  down  their  reaction 
on  a page  designed  in  the  BBS.  Students  are  also  expected  to  name  the  URL  on  the  top  of  the  reaction  writing  so 
that  we  can  easily  trace  back  what  kind  of  file  our  students  have  read  and  what  kind  of  viewpoint  they  have  taken. 
The  BBS  system  itself  is  not  so  complicated.  The  program  was  originally  created  by  a Japanese  college  student  and 
made  to  adapt  our  learning  goals  later.  The  good  points  of  this  system  is  (1)  it  is  quite  easy  to  grasp  the  several 
stages  of  developing  process  of  our  studets'  English  learning  only  by  giving  a look  at  the  writing  page,  (2)  students 
can  read  and  write  English  in  a non-stressed  way,  (3)  the  English  the  sudents  are  likely  to  write  is  not  so  intimate  or 
personal,  which  is  one  of  the  disadvantages  of  "pen-pal,  or  mail-pal"  type  of  writings  (4)  anyone  concerned  can  write 
comments  to  the  writings  so  that  a kind  of  dialogue  is  possible  among  the  people  inside  the  system. 

The  eMINTS  Project:  Enhancing  Missouri's  Instructional  Networked  Teaching  Strategies  - 
- Promising  Developments  and  Projected  Outcomes 

Adam  Bickford,  OSEDA/University  of  Missouri  Extension,  USA;  Bill  Elder,  OSEDA/University  of  Missouri  Extension, 
USA;  Barbara  Hammer,  OSEDA/University  of  Missouri  Extension,  USA;  Patrick  McGinty,  OSEDA/University  of 
Missouri  Extension,  USA;  Priscilla  McKinley,  OSEDA/University  of  Missouri  Extension,  USA;  Susan  Mitchell, 
OSEDA/University  of  Missouri  Extension,  USA 

The  eMINTS  Project  is  an  ongoing  cooperative  initiative  between  the  Missouri  Department  of  Elementary  and 
Secondary  Education  (MoDESE),  the  Missouri  Research  and  Education  Network  (MOREnet)  and  98  elementary 
classrooms  in  44  school  districts.  eMINTS  classrooms  are  provided  with  an  extensive  technological  infrastructure: 
high-speed  internet  connectivity,  enough  student  computers  to  provide  one  computer  for  every  two  students,  a high 
capacity  teacher  workstation  with  video  conferencing  capabilities,  and  a computer  projector  and  a Smart  Board.  In 
addition  to  the  computing  infrastructure,  participating  teachers  receive  extensive  training  in  constructivist,  inquiry- 
based  teaching  methods  which  are  supported  by  the  University  of  Missouri  College  of  Education,  which  provides 
full-time  assistance  in  locating  and  using  on-line  instructional  resources.  Results  from  the  first  wave  of  this  project, 
the  MINTs  Project,  have  demonstrated  that  well  trained  teachers  working  in  technologically  rich  classrooms,  can 
focus  student  interest  and  improve  student  scores  on  measures  of  academic  achievement. 

LEZI:  a Tool  for  Easy  Development  of  Interactive  Video  for  Education 

Mario  A.  Bochicchio,  University  of  Lecce,  Italy;  Roberto  Paiano,  University  of  Lecce,  Italy;  Paolo  Paolini,  Politecnico 
di  Milano,  Italy;  Elisabetta  Andreassi,  University  of  Lecce,  Italy;  Tiziana  Montanaro,  University  of  Lecce,  Italy 

The  design  and  implementation  of  complex  multimedia  application  is  an  extensive  effort,  requiring  time,  technical 
skills  and  sizable  budgets.  In  few  cases,  therefore,  in  educational  environment  it's  possible  to  realize  complete 
hypermedia  applications.  It's  true,  nevertheless,  that  within  several  academic  institutions  there  is  the  capability  of 
developing  high  quality  educational  content,  traditionally  delivered  as  paper,  while  it  could  be  more  efficiently 
delivered  in  multimedia  format.  Our  Laboratory  is  devoted  to  development  of  Multimedia  applications,  some  of 
them  concerning  educational  subjects  in  collaboration  with  several  professors  from  the  Humanities.  Many  of  them 
asked  for  developing  their  own  "cultural"  application,  but  may  be  they  don't  have  the  budget,  human  resources  or 
technical  skills  to  carry  on  the  job.  With  this  requirements  in  mind  we  conceived  LEZI,  a tool  that  requires  a 
minimal  computer  expertise  in  order  to  develop  in  a quick  manner  educational  multimedia  application  based  on 
interactive  video  and  electronic  documents. 

New  Views  on  Community  and  Collaboration:  Building  and  Sharing  Dynamic  Worlds  in 
Cyberspace 

Paula  Bonta,  LCSI,  Canada;  Richard  Borovoy,  MIT  Media  Lab,  USA;  Susan  Einhorn,  Consultant,  Canada;  Brian 
Silverman,  MIT  Media  Lab,  USA 

Abstract:  The  Internet  has  the  potential  to  provide  an  unprecedented  dynamic,  interactive  learning  environment  - 
one  in  which  students  from  around  the  globe  can  collaborate  in  the  construction  of  exciting  new  worlds.  Despite 
the  Internet's  capabilities,  there  is  a lack  of  applications  geared  towards  fulfilling  this  need.  This  poster  session  will 
present  a prototype,  the  Kids  Internet  Construction  Kit  (KICK),  that  allows  K-12  students  to  create  and  share 
dynamic  worlds  on  the  Internet. 


i 


806 


Reeves  Ten-Dimensional  Model:  Application  for  the  Design  and  Development  of  Web- 
Based  Learning  Scenario  on  Learning  Theories 

Caroline  Brassard,  Universite  du  Quebec  a Chicoutimi,  Canada;  Jacqueline  Bourdeau,  Universite  du  Quebec  a 
Chicoutimi,  Canada;  Pauline  Minier,  Universite  du  Quebec  a Chicoutimi,  Canada 

This  poster  presentation  covers  the  design  and  development  of  a media-based  learning  scenario  and  Web 
environment  with  a view  to  implementing  a pedagogical  design  that  integrates  new  technologies.  The  scenario 
responds  to  concerns  regarding  pedagogical  innovation  in  "Teachers  Training".  It  tries  to  facilitate  the  development 
of  the  skills  required  within  the  larger  framework  of  courses  where  theoretical  content  is  high  and  cooperative 
learning  methods  must  be  supported.  The  focus  was  therefore  placed  on  the  ability  to  "make  links"  and  on  the 
socio-constructivist  approach.  The  results  showed  the  importance  of  design,  the  need  for  cognitive  strategies,  the 
inherent  pedagogical  value  of  cooperative  learning  and  developing  the  ability  to  "make  links".  The  creation  of  links 
can  go  beyond  the  context  to  encompass  any  training  session  that  contains  a wide  range  of  tlieofics  auu  a broad 
body  of  knowledge.  It  appears  that  the  need  for  a design  that  produces  the  most  appropriate  overall  educational 
formula  may  lead  to  proven  pedagogical  effectiveness. 

The  Creative  Networkithe  development  of  a collaborative  learning  community  for  mature 
professionals 

Helen  Brown,  University  of  Central  England,  Birmingham  Institute  of  Art  and  Design,  England 

The  hypothesis  underlying  the  formation  of  the  Creative  Network  is  that  a radical  approach  to  Continuing 
Professional  Development  in  the  U.K.  is  needed  to  prepare  those  employed  in  the  Creative  Industries  for: 

1.  the  portfolio  career 

2.  short  term  contracts 

3.  rapidly  changing  technologies/ materials 

4.  structural  changes  in  the  companies,  flatter  hierarchies,  featuring  multi-skilling  and  cross  functional  teams 

5.  e-business  (e-commerce  and  e-volution) 

The  Creative  Network  web  site  is  the  gateway  to  a learning  community.  There  is  no  standard  course  or  assessment. 
Instead  the  Creative  Network  tutors  work  with  each  individual  learner  or  company  to  analyse  their  development 
needs  and  structure  a training  plan.  The  Creative  Network  aims  to  facilitate  deep  learning  based  upon  a spirit  of 
facilitation  which  helps  learners  to  contextualise  new  learning  and  construct  personal  relevance  and  meaning. 

Breathing  New  Life  Into  The  Classroom 

Lawrence  Bundy,  Nebraska  Department  of  Education,  USA 

Two  Nebraska  projects,  funded  by  grants  from  the  United  States  Department  of  Education,  will  work  together  to 
prepare  teachers  to  use  technology  in  the  classroom.  The  Connections  Project,  a five-year  Education  Technology 
Innovation  Challenge  grant,  works  with  in-service  teachers.  Introductory,  intensive  five-day  summer  workshops 
focus  on  classroom  uses  of  technology,  brain-compatible  teaching  strategies,  life  skills,  multiple  learning  styles,  and 
integrated  curriculum  models.  On-going  staff  development  follows  the  workshops  at  local  sites  throughout  the 
academic  year.  The  teachers  create  curriculum  units  incorporating  the  strategies  they  have  learned  and  post  them  on 
the  project  Web  site — http://ois.unomaha.edu/connections.  They  also  create  CD-ROMs  of  the  workshop 
presentations,  which  will  be  distributed  to  over  6,000  teachers.  The  Catalyst  Project,  a three-year  Preparing 
Tomorrow’s  Teachers  to  use  Technology  (PTTT)  grant,  involves  seventeen  higher  education  institutions  that  are 
engaged  in  active  systemic  changes  to  prepare  pre-service  teachers  to  use  educational  technology.  The  two  projects 
will  work  together  to  identify  and  develop  a cadre  of  K-12  teachers  who  are  leaders  in  the  use  of  educational 
technology.  This  cadre  will  act  as  resources  in  the  higher  education  classroom  and  as  cooperating  teachers  for 
teacher  education  programs. 

Improving  Collaboration  in  Face-to-Face  Groups  using  Technology 

Robert  W.  Cavenagh,  Jr.,  Dickinson  College,  USA 

Abstract:  Face-to-face  collaborative  work  is  an  essential  tenet  of  much  contemporary  teaching,  especially  within 
constructivist  models.  The  practical  use  of  technology  by  small  intact  groups  has  received  relatively  little  research 
attention.  Observation  of  learners  in  existing  settings  suggests  that  commonplace  computing  facilities  may  limit 
effective  collaborative  work  by  more  than  two  individuals.  The  design  of  facilities  and  strategies  specifically  for 
collaboration  has  been  demonstrated  to  improve  the  effectiveness  of  such  work.  This  PosterDemo  reviews 
applicable  research,  presents  design  and  cost  criteria  for  and  photographic  examples  of  purpose-built  facilities,  and 
delineates  pedagogic  strategies  found  useful,  including  both  assignment  and  assessment  strategies.  It  summarizes 
findings  and  future  topics  to  explore.  It  also  simulates  the  look  and  feel  of  one  such  collaborative  facility. 


Educational  Applications  of  Hypervideo 

Teresa  Chambel,  University  of  Lisbon,  Portuga 

Video  clips  can  gready  enhance  the  authendcity  of  a computer  based  learning  environment.  Something 
constructivists  have  been  strongly  arguing  for.  Broadcast  television  does  not  afford  composition  or  the  time  to 
reflect.  But,  television  and  video,  when  properly  constructed,  can  be  a powerful  tool  for  reflection.  If  the  user  can 
select  what  is  to  be  seen  and  control  the  pace  of  the  material,  and  it  is  easy  to  go  back  and  forth,  to  stop,  to  make 
annotations,  to  compare  and  to  relate  to  other  materials.  Effective  reflection  requires  some  structure  and 
organization,  the  main  issues  in  hypermedia.  Hypervideo  refers  to  the  integration  of  video  in  truly  hypermedia 
documents,  taking  into  account  its  spatial  and  temporal  dimensions,  and  defining  the  semantic  and  mechanisms  to 
link  and  navigate  video  and  other  media.  In  this  presentation,  different  mechanisms  for  the  integration  and 
navigation  of  video  in  educational  applications  are  demonstrated.  These  were  developed  using  HTIMEL,  an 
extension  to  HTML  and  existing  Web  tools,  being  developed  and  used  in  the  Unibase  project  on  distance  learning. 

Coiiaboration  between  T&T3  Project  Team  and  ETU/CeLTS  at  HKIEd  in  Promoting 
Teaching  Skills  and  IT  for  HE 

Thomas  YH  Chan,  Lingnan  University,  Hong  Kong,  China;  Julianne  WY  Wong,  Hong  Kong  Institute  of  Education, 

Hong  Kong,  China;  Elson  Szeto,  Hong  Kong  Institute  of  Education,  Hong  Kong,  China 

We  are  presenting  the  fruit  of  our  collaboration  in  the  context  of  the  T&T  Project  — a publicly-funded  inter- 
institutional  collaborative  initiative  for  all  academic  and  professional  staff  in  Hong  Kong's  HE  institutions.  Project 
mission  is  to  pull  together  human  and  instructional  resources  from  all  institutions  in  a collaborative  effort  in 
promoting,  fostering  and  advancing  quality  teaching  and  learning.  We  are  going  to  share  with  conference 
participants  the  following: 

• Designing  and  conducting  sessions  on  IT  in  teaching  and  learning  for  academic  and  professional  staff 
from  all  tertiary  institutions  in  Hong  Kong,  in  particular  on: 

• formatting  and  layout  of  text 

• digital  graphics 

• perfecting  presentation  skills 

• other  digital  media  for  enhancing  teaching  and  learning 

• Designing  and  producing  for  colleagues'  use  self-instructional  multimedia  packages  for  enabling  IT  self- 
sufficiency  (in  CD-ROM  format) 

• Initiating  audio  and  video  streaming  for  the  “T&T  Project  Website"  and  related  on-line  multimedia 
packages 

Teaching  Symbol  Recognition  with  Interactive  Java  GUI 

Li  Chao,  University  of  Houston- Victoria,  USA 

Abstract:  The  ease  of  learning  Java,  its  reliability,  and  its  portability  make  this  programming  language  more  and 
more  popular  among  students  and  faculty.  It  has  gained  much  attention  through  its  power  of  multithreading, 
networking,  and  computer  graphics.  As  an  assistant  in  teaching  computer  vision  such  as  symbol  recognition,  Java 
has  the  flexibility  needed  to  develop  classroom  demonstrations  which  are  accessible  through  Internet.  The  objective 
of  this  paper  is  to  present  an  interactive  Java  based  GUI  to  assist  the  teaching  of  symbol  recognition. 

Kabisa:  A computer-based  training  program  for  diagnostic  reasoning  in  tropical 
medicine 

Geraldine  Clarebout,  Univerisity  of  Leuven,  Belgium;  Jan  Elen,  University  of  Leuven,  Belgium;  Joost  Lowyck, 
Univerisity  of  Leuven,  Belgium;  Jef  Van  den  Ende,  Institute  of  Tropical  Medicine,  Belgium;  Stefano  Lagans  Italy 

A shift  took  place  in  educational  goals  from  knowing  everything  in  a certain  domain,  to  knowing  how  to  deal  with 
complex  problems.  The  reasoning  process  has  become  more  important  than  the  amount  of  information  to 
memorize.  In  medical  education  the  same  evolution  took  place.  A computer-based  training  program  that  is 
developed  to  guide  and  help  to  develop  diagnostic  reasoning  skills  in  tropical  medicine  is  KABISA,  In  this  poster, 
the  program  be  presented  and  all  its  functionalities,  as  well  as  a user-test  that  was  performed.  This  test  was  done 
to  identify  whether  KABISA  really  contributes  to  the  development  of  reasoning  skills  in  tropical  medicine.  The 
main  aim  of  this  test  is  to  determine  what  has  to  be  changed  to  the  program  to  provide  students  with  an  efficient 
learning  environment  to  learn  diagnostic  reasoning  and  to  optimize  the  program.  Use  was  made  of  a thinking  aloud 
method  and  log  file  analysis. 


WEB-LIT:  Web-Based  Support  for  Learning  Theory  and  Instructional  Design 

Ken  Clinkenbeard,  University  of  Bergen,  Norway;  J.  Michael  Spector,  University  of  Bergen,  Norway 

The  primary  objectives  of  our  project  include  creating  an  open  and  flexible  web  site  to  host  materials  on  learning 
theory  and  instructional  design.  We  will  find  and  evaluate  existing  web-based  materials  on  these  topics  which  reflect 
a wide  variety  of  perspectives,  applications  and  lessons  learned  and  organize  the  site  so  that  materials  can  be  posted 
and  easily  accessed.  Interactivity  also  plays  a key  role.  Opportunities  are  provided  for  users  to  comment  on  existing 
materials  and/or  submit  new  materials.  Our  goal  is  not  to  create  an  online  course  in  the  area  of  instructional  design 
of  learning  theory,  but  to  augment  the  design  of  these  courses  by  offering  support  resources.  A secondary  goal  of 
this  site  is  to  create  a web-based  set  of  resources  that  can  provide  a source  of  evaluation  data  on  the  use  of  web- 
based  materials  in  graduate  education. 

Realism  and  Credibility  in  a Simulation-Based  Virtual  Physics  Laboratory  (VPLab):  An 
Empirical  Study 

Marc  Couture,  Tele-universite,  Canada;  Alexandre  Francis,  Universite  de  Montreal,  Canada 

Realism  and  effectiveness  of  computer  simulation-based  learning  or  training  environments  have  been  examined  in 
several  studies.  It  was  shown  that  under  certain  conditions,  simulations  can  be  as  efficient  as  real  experiments,  and 
that  increased  realism  may  result  in  gains  in  'practical  appreciation'.  However,  few  have  investigated  the  relationship 
between  realism  and  credibility,  or  between  credibility  and  effectiveness.  The  VPLab  is  a simulation-based  learning 
environment  featuring  many  characteristics  and  constraints  normally  associated  with  real  experiments.  These 
include  uncertainty  in  measurement,  random  fluctuation  of  parameters,  and  limitations  in  user  control  over  the 
simulation.  This  approach  distinguishes  the  VPLab  from  most  existing  simulation-based  laboratories.  We  will 
present  first  results  of  an  experimentation  with  first-year  university  science  students,  in  which  we  sought  to  identify 
the  factors,  most  notably  those  associated  with  our  approach  of  realism,  that  may  enhance  the  credibility  of  such  an 
environment  and/or  its  perceived  relevance  as  a tool  for  learning  laboratory  skills  as  well  as  science  concepts. 

A Web-based  Tutorial  on  Optical  Communications 

Ignacio  de  Miguel,  University  of  Valladolid,  Spain;  Carlos  Jesus  Fuertes,  University  of  Valladolid,  Spain;  Abel  Prieto, 
University  of  Valladolid,  Spain;  Patricia  Fernandez,  University  of  Valladolid,  Spain;  Miguel  Lopez,  University  of 
Valladolid,  Spain;  Fernando  Gonzalez,  University  of  Valladolid,  Spain;  Juan  Carlos  Aguado,  University  of  Valladolid, 
Spain;  Ruben  M.  Lorenzo,  University  of  Valladolid,  Spain;  Evaristo  J.  Abril,  University  of  Valladolid,  Spain 

We  are  developing  a tutorial  on  optical  communications  in  order  to  increase  the  interest  of  the  students  in  this  area 
and  to  clear  up  some  concepts.  The  tutorial  is  divided  into  four  didactical  units;  Propagation  of  Signals; 
Components  and  Devices;  Optical  Communication  Systems;  and  Optical  Communication  Networks.  Within  each 
unit,  basic  information  is  provided  as  well  as  simulators  and  animations.  The  students  can  also  perform  self- 
evaluations  to  check  their  progress.  The  self-evaluations  are  randomly  generated  from  the  contents  of  a database 
according  to  the  students'  preferences  (topics,  difficulty  and  kind  of  items:  questions  and/or  short  problems).  The 
tutorial  is  being  developed  at  http://pesquera.tel.uva.es/tutorial.  The  utilization  of  web  technologies  allows  the 
students  to  access  the  tutorial  from  their  homes,  and  we  hope  it  will  facilitate  the  collaboration  with  other  groups 
working  in  these  topics.  This  work  is  partially  supported  by  Consejeria  de  Educacion  y Cultura  (Junta  de  Castilla  y 
Leon). 

Posting  Visual  Cues  on  Threaded-Message  Boards  in  Science  Curricula:  Enhancing 
Transfer  of  Theory  Knowledge  to  Clinical  Practice 

Gregory  A.  DeBourgh,  University  of  San  Francisco,  USA 

Results  of  a pilot  study  (16  baccalaureate  nursing  students)  suggest  posting  thematic,  integrated  photographs, 
graphic,  and  text  data  on  asynchronous,  threaded-message  boards  facilitates  transfer  of  theory  knowledge  to  clinical 
practice  and  supports  the  acquisition  of  skills  in  clinical  reasoning  (pattern  recognition,  critical  analysis,  problem 
solving,  and  identification  of  therapeutic  interventions).  Visual  cues  provide  powerful  representations  of  reality  that 
enable  students  to  recognize  manifestations  and  patterns  of  clinical  pathologies.  Text-based  learning  prompts 
stimulate  reflective  thinldng  and  collaborative  problem  solving.  Used  synergistically,  they  create  clinical  context, 
authentic  complexity,  and  scaffold  learning.  Provided  both  words  and  pictures  during  instruction,  learners  encode 
information  in  such  a manner  that  recall  is  facilitated  and  transfer  of  knowledge  enhanced.  Students  report  on-line 
interaction  results  in  the  positive  experience  of  belonging  to  a professional  community  during  learning,  affords 
support  and  opportunities  for  collaboration,  and  increases  access  to  the  instructor  to  clarify  "accuracy  of  learning" 
and  provide  feedback.  Using  visually-enriched  on-line  conferencing  is  an  enjoyable,  convenient,  and  satisfying  way 
to  amplify  and  enhance  learning,  and  "brings  the  course  content  alive". 
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The  «BacVert»  or  «Blue  Box»  - A Knowledge  Recycling  Bin  for  Students 

Celine  Desjardins,  Centre  de  recherche  LICEF,  Canada;  Efoe  Wallace,  Centre  de  recherche  LICEF,  Canada;  Claude 
Ricciardi-Rigault,  Centre  de  recherche  LICEF,  Canada 

While  it  has  become  common  to  provide  students  with  a list  of  documents  to  peruse  on  the  Internet,  information 
gathering,  net-surfing  and  annotation  all  have  an  exploratory,  individual  and  work-in-progress  quality.  Sharing 
insights  gained  through  these  activities,  especially  in  distance  education,  is  complicated  and  requires  highly 
structured  settings.  The  INF9002  «BacVero>  web-based  dispositif  at  Tde-universite  supports  such  sharing.  The 
natural  language  parsing  performed  by  Nomino  provides  flexible  automatic  structuration,  both  of  the  core  Internet 
material  proposed  by  the  teacher  to  which  the  students  gradually  add  their  findings,  and  of  the  comments  about 
these  documents  produced  by  the  students.  By  allowing  a group  of  students  to  constitute  an  evolutive  information 
base,  to  comment  on  it  and  to  compare  notes,  the  1NF9002  «BacVero>  constitutes  a step  towards  tackling  the 
complexity  and  heterogeneity  of  the  information  environment,  while  supporting  exchanges  among  peers  about  the 
material  at  hand. 

Building  Global  Information  Communities:  the  University  of  Iowa  Center  for  Electronic 
Resources  in  African  Studies  (CERAS) 

Barbara  Dewey,  University  of  Iowa  Libraries,  USA 

This  poster  session  provides  a case  study  of  the  issues  and  lessons  learned  regarding  development  of  the  University 
of  Iowa’s  Center  for  Electronic  Resources  in  African  Studies  (CERAS)  (http://sdrc.lib.uiowa.edu/ceras/).  The 
Center  is  a “virtual”  space  for  creating,  disseminating,  and  accessing  scholarly  electronic  resources  in  text, 
multimedia,  and  interactive  format,  and  a “virtual”  place  for  further  development  of  electronic  resources  (textual 
and  multimedia-based)  pertaining  to  Africa  created  by  scholars  from  the  United  States  as  well  as  from  Africa  and 
other  parts  of  the  world  (http://sdrc.lib.uiowa.edu/ceras/).  CERAS  is  part  of  the  University  Libraries’  Scholarly 
Digital  Resources  Center  supporting  students  and  faculty  engaged  in  African  Studies  research  and  teaching  at  the 
University  of  Iowa,  and  scholars  nationally  and  internationally.  CERAS  is  also  a venue  for  U.S.  and  African  scholars 
for  creating  and  disseminating  African  Studies  scholarship  and  information  resources  in  electronic  format.  Several 
CERAS  initiatives  have  focused  on  image  creation,  particularly  in  the  field  of  African  art,  a strength  at  the 
University  of  Iowa.  CERAS  will  serve  as  a venue  for  digitizing  projects  related  to  Africana  scholarship  (i.e.  museum 
collections,  selected  non-copyrighted  collections,  archival  materials,  and  documents).  Other  goals  include  bringing 
high  quality  African  Studies  scholarly  electronic  collections  and  web  sites  currently  in  existence  into  a user-friendly 
digital  library  web  page;  providing  a venue  for  “publishing”  original  materials,  including  pre-prints,  (articles,  books, 
conference  proceedings)  as  well  as  creating  significant  thematic  textual  and  multimedia  collections  of  materials  in 
electronic  format;  and  developing  an  electronic  international  community  of  Africana  scholars  where  dialogue  about 
issues,  research,  and  projects  can  take  place.  This  latter  feature  is  intended  to  provide  a broader  opportunity  for 
Africana  scholars  to  post  drafts  of  research  papers  and  projects  for  critique  and  discussion  by  other  Africanists  from 
around  the  world.  The  University  of  Iowa  Libraries  is  seeking  collaborative  projects  and  partnerships  with 
institutions  and  Africana  scholars  from  around  the  world. 

Introducing  a Distance  Learning  version  of  a Postgraduate  Program  on  Networking  in 
Argentina 

Dean  Javier  Diaz,  University  of  La  Plata,  Argentine;  Maria  Alejandra  Osorio,  University  of  La  Plata,  Argentine;  Ana 
Paola  Amadeo,  University  of  La  Plata,  Argentine 

The  great  popularity  of  the  Internet  and  the  widespread  use  of  the  World  Wide  Web  are  promoting  a new 
methodology  for  distance  learning.  The  main  advantage  of  the  Internet  based  courses  is  possibility  to  promote  a bi- 
directional communication  between  the  students  and  the  professor  and  to  highly,  motivate  the  student  due  to  an 
adequate  feedback.  This  poster  describe  the  introduction  of  Internet  tools  (such  as  www,  e-mail,  IRC,  FTP,  mailing 
list)  in  the  distance  learning  postgraduate  program  of  the  ‘Magister  de  Redes’  (http:/ / www.linti.unlp.edu.ar/master) 
of  the  UNLP  . The  ongoing  experience  was  successful  due  to  certain  facts  that  are  stated  in  the  poster  which  also 
mentions  some  drawbacks  and  the  strategy  for  overcoming  them.  The  distance-learning  version  of  the  program 
started  in  1998  as  way  to  reduce  the  obstacles  for  he  students  (professionals  from  the  engineering  and  the  computer 
science  field  in  the  networking  area),  mainly  the  traveling  expenses  and  the  time  it  takes  due  the  large  geography  of 
our  country  and  the  long  distances. 

The  Bones  Of  The  Skull:  Creating  Anatomical  Models  With  Quicktime  Vr 

Marilyn  Dispensa,  The  University  of  Iowa,  USA;  Jim  Duncan,  The  University  of  Iowa,  USA;  Jerry  Moon,  The 
University  of  Iowa,  USA 

This  electronic  demonstration  will  provide  an  overview  of  QuickTime  Virtual  Reality  (QTVR)  and  its  utility  in  the 
development  of  virtual  anatomical  models  which  can  be  used  alone  or  integrated  into  computer-based-learning 
(CBL)  software.  The  Bones  of  the  Skull:  A 3-D  Learning  Tool  is  a CD-ROM-based  module  designed  to  help 
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college-level  anatomy  students  learn  the  bones  and  important  landmarks  of  the  skull.  The  software  has  two  main 
features:  1.  A collection  of  Quicktime  VR  movies  that  allow  the  user  to  rotate  and  view  the  skull  and  its  bones  in 
three  dimensions.  2.  An  interactive  "textbook"  that  contains  high  quality  2-D  images,  descriptive  text  and  many 
embedded  activities  to  encourage  interaction  with  the  content.  This  presentation  will  demonstrate  the  product  and 
provide  an  overview  of  QuickTime  VR,  including  technical  requirements  and  hardware/software  costs.  The  Bones 
of  the  Skull  won  the  1999  Sandoz/Slice  of  Life  Student  Software  Award. 

The  impact  of  establishing  a virtual  university:  a case  study  at  MUST 

Nomusa  DIodIo,  National  University  of  Science  and  Technology,  Zimbabwe 

The  purpose  of  establishing  any  university  is  to  cater  for  the  educational  needs  of  the  surrounding  community.  In 
developing  countries  universities  fail  to  cope  with  large  numbers  of  applicants.  The  reason  is  that  the  number  of 
universities  in  such  countries  is  limited  due  partly  to  historical  circumstances  and  partly  due  to  financial  constraints. 
The  exercise  of  building  brick  and  mortar  universities  and  acquiring  skilled  staff  to  run  them  is  expensive. 

A Distributed  Maritime  Simulation  Training  Environment  based  on  HLA 

Mirko  Dobermann,  Computer  Graphics  Center  ZGDV,  Germany;  Harro  Kucharzewski,  MarineSoft  GmbH,  Germany 

A new  approach  for  advanced  training  systems  is  the  integration  of  computer  simulation  into  such  applications, 
allowing  students  to  experiment  in  complex  scenarios.  The  presentation  introduces  a distributed  simulation-based 
training  scenario  , generated  in  a distributed  environment,  which  that  offers  extended  functionality  for  interaction 
and  demonstration  within  a maritime  training  scenarioapplication.  The  main  objectives  of  the  solution  are  threefold: 
to  set  up  and  establish  a distributed  simulation-based  training  environment,  to  provide  real-time  interaction 
functionality  during  run-time,  as  well  asand  to  integrate  already  available  simulators  that  are  currently  on  the  market. 
The  infrastructure  for  this  simulation  is  based  on  HLA/RTI,  while  which  already  offers  functionality  to  couple 
embedded  applications  with  networked  simulation  applications.  The  Computer  Graphics  Center  in  Rostock 
(ZGDV)  currently  focuses  on  maritime  scenarios  related  to  competency  training  for  crews  aboard  and  ashore.  The 
development  of  the  simulation  environment  is  part  of  a research  project  in  collaboration  with  regional  enterprises  in 
the  Maritime  sector. 

Human  Anatomy  Predissection  Lecture-On-Demand  at  The  National  University  of 
Singapore 

Gilles  Doiron,  National  University  of  Singapore,  Singapore 

The  NUS  Integrated  Virtual  Learning  Environment  (IVLE)  was  developed  so  that  staff  and  students  could  use  this 
information  technology  infrastructure  to  communicate,  exchange  documents  and  information,  discuss,  chat,  and 
access  custom  learning  materials  and  course  related  web  sites.  IVLE  also  enabled  the  university  to  consider  new 
pedagogical  approaches,  which  would  utilise  its  campus-wide  broadband  access  to  meet  specific  teaching  and 
learning  needs.  A "lecture-on-demand"  (LoD)  delivery  was  seen  as  a viable  tool  that  would  allow  students  to  take 
more  responsibility  for  their  learning  and  enable  them  to  have  greater  control  over  their  time  schedule.  With  the 
collaboration  of  the  Faculty  of  Medicine,  the  Center  for  Instructional  Technology  and  the  Centre  for  Development 
of  Teaching  & Learning,  a prototype  anatomy  pre-dissection  LoD,  "The  Abdominal  Wall  & Inguinal  Canal",  was 
produced.  This  paper  examines  the  design  and  development  issues  addressed  in  building  the  prototype.  Field  test 
data  on  the  technical  reliability,  and  ratings,  and  comments  from  student  feedback  are  presented. 

ExploraGraph  and  CINEMA 

Aude  Dufresne,  Univ.  of  Montreal,  Canada;  Claire  Isabelle,  Univ.  of  Moncton,  Canada;  Roger  Nkambou,  Univ.of 
Montreal,  Canada;  Van  Laporte,  Univ.  of  Montreal,  Canada;  Frank  Ferront,  Univ.  of  Montreal,  Canada 

The  ExploraGraph  interface  was  designed  to  facilitate  interaction  in  the  context  of  distant  learning.  It  was 
developed  as  an  ^ternative  to  simple  web  interaction,  in  order  to  increase  flexibility,  visibility,  and  structure  in  the 
learning  environment.  It  may  be  used  as  a front  end  to  existing  courses  on  the  web  and  to  support  learner  in  them. 
The  ExploraGraph  Navigator  makes  it  possible  to  navigate  through  conceptual  graphs  with  automatic  arrangement 
of  elements,  zoom  and  "fish  eye"  effects.  Each  node  of  the  graph  may  have  a description  attached  to  it  and  may 
give  direct  access  to  an  application,  a document  or  an  Internet  site.  Graphic  structures  may  thus  be  used  to 
represent  the  organization  of  tools,  activities,  concepts  or  documents.  The  Navigator  offers  each  user  a tool  to 
specify  his  goals  and  the  system  can  support  him,  using  multiple  modalities:  Hypertext,  graphical  cues,  Ms  Agents 
avatars,  voice,  visual  demonstrations  and  force  feedback  guiding. 
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Future  Special  Education  Teachers*  Abilities  to  Integrate  New  Technology  Into  Teaching 
Reading  Comprehension 

Catherine  Dumoulin,  Universite  du  Quebec  a Chicoutimi,  Canada;  Jacqueline  Bourdeau,  Universite  du  Quebec  a 
Chicoutimi,  Canada 

This  study  questions  future  teachers'  abilities  to  integrate  new  technology  into  their  teaching.  A pedagogical  scenario 
based  on  a cognitive  approach  was  proposed  to  graduating  students  of  the  bachelor  of  Special  Education  program 
in  order  to  teach  Grade  3 students  with  reading  comprehension  difficulties.  This  pedagogical  scenario  integrated 
new  technology  via  the  "Village  Prologue",  a computerized  environment.  Future  teachers  were  found  to  organize 
pedagogical  activities,  create  learning  situations  suitable  for  teaching  reading  comprehension,  help  motivate  children 
to  read,  and  fmally,  guide  children  in  their  learning  processes  within  the  computerized  environment.  Furthermore, 
the  pedagogical  scenario  helped  in  motivating  future  teachers  to  include  new  technology  in  their  future  teaching. 
Tlic  computerized  environment  also  permitted  the  fumre  teachers  to  adapt  their  teaching  to  the  specific  neeus  or 
children  with  reading  comprehension  difficulties. 

School  Festival  on  the  Internet  - Project-based  and  cooperative  learning  - 

Hironori  Egi,  Keio  University,  JAPAN;  Yoshitomo  Tsutsui,  Kamogata  High  school,  JAPAN;  Yuuki  Nishimura,  Keio 
University,  JAPAN;  Keiichirou  Ishibashi,  Keio  University,  JAPAN 

We  report  a case  of  project-based  and  cooperative  learning  activities.  School  Festival  on  the  Internet,  Okayama 
1999  (SF99)  in  Japan.  Three  public  high  schools  in  Okayama,  Keio  University  and  Kurashiki  University  of  Science 
and  the  Arts  participated  in  SF99.  The  goal  of  SF99  was  to  put  the  school  festivals  held  at  each  of  the  high  school 
on  the  web  so  that  the  students  efforts  could  be  widely  appreciated.  School  festivals  are  considered  as  an 
educational  event  in  Japan,  and  we  implemented  SF99  as  a project-based  and  cooperative  activity.  The  festival 
provides  students  with  opportunities  to  present  achievements  of  their  research  or  their  other  interests.  Video 
streams  and  online  communication  through  a Bulletin  Board  System  (BBS)  were  the  main  applications  used  to 
achieve  the  end.  SF99  is  one  of  the  few  examples  were  extra-curricular  activities  such  as  school  festivals  were 
digitalized.  Information-based  learning  and  extra-curricular  activity  have  similar  goals.  We  also  analyzed  the 
achievements  of  SF99  and  discussed  the  educational  effect  of  information-based  learning  using  the  Internet. 

The  ParlEuNet-project:  Problems  with  the  validation  of  socio-constructivist  design 
principles  in  ecological  settings 

Jan  Elen,  University  of  Leuven,  Belgium;  Geraldine  Clarebout,  University  of  Leuven,  Belgium 

Instructional  design  aims  at  generating  indications  about  optimal  relationships  between  learner-related  and 
instruction-related  variables  in  view  of  the  attainment  of  instructional  and/or  learning  goals.  In  this  poster,  a study 
is  presented  in  which  it  was  started  from  socio-constructivist  view  on  learning  and  instruction  to  identify  a number 
of  design  principles.  These  principles  were  used  to  design  and  develop  a concrete  and  innovative  rich  technological 
leaning  environment,  the  ParlEuNet  learning  environment.  The  main  features  of  the  this  environment  can  be 
described  as  follows: 

• A variety  of  technologies 

• A problem-based  learning  environment  where  use  is  made  of  ill-structured  tasks 

• International  collaboration 

It  was  hypothesized  that  such  an  environment  would  have  an  influence  on  certain  learning  environment.  In  this 
poster  the  environment  and  the  results  of  the  research  will  be  presented,  which  indicate  that  there  is  an  influence  of 
the  learning  environment,  however  in  a different  direction  than  was  expected. 

Examining  Medical  Students’  Attitudes  and  Learning  Experiences  in  BioWorld 

Sonia  Faremo,  McGill  University,  Canada;  Susanne  Lajoie,  McGill  University,  Canada;  David  Fleiszer,  Faculty  of 
Medicine,  McGill  University,  Canada 

This  poster  presents  BioWorld  (Lajoie  et  al.,  1998),  a computer-based  learning  environment  (CBLE)  that  engages 
learners  in  realistic  problem  solving  as  they  attempt  to  diagnose  medical  cases.  Its  design  is  consistent  with  both 
problem-based  learning  (PBL)  and  cognitive  principles  of  instruction.  Solving  a BioWorld  case  involves  interpreting 
case  history  information,  ordering  diagnostic  tests,  developing  diagnostic  hypotheses,  and  reflecting  on  one’s  own 
performance.  As  part  of  a larger  study,  the  data  for  this  poster  consisted  of  measures  of  the  attitudes  and  learning 
experiences  of  medical  students  using  BioWorld.  The  experimental  procedure  involved  having  medical  students 
inividually  diagnose  a set  of  BioWorld  cases  and  complete  a questionnaire  concerning  their  experiences.  Students 
rated  several  aspects  of  the  system  (level  of  interest,  (difficulty,  utility  of  learning  activities,  etc.).  The  results  have 
implications  for  the  design  of  CBLEs  for  complex  domains.  They  also  suggest  that  BioWorld  is  an  effective  learning 
environment. 
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CyberARTS:  A Integrated  Arts  Curriculum 

Tito  Faria,  Don  Mills  Collegiate  Insititute,  Canada;  Sholom  Eisenstat,  Don  Mills  Collegiate  Institute,  Canada 

The  CyberARTS  program  is  offered  at  a number  of  school  sites  in  Toronto.  CyberARTS  uses  a constructivist 
methodology  to  deliver  visual  integrated  arts,  media  communication,  computer  technology  and  now  Gr.  9 
Geography  credits.  Students  have  constant  access  to  labs  of  Macs  and  PCs  and  a range  of  peripherals.  Teams  of 
teachers  work  coUaboratively  creating  and  delivering  the  curriculum.  Senior  students  complete  the  program  with  a 
co-op  experience.  Their  placements  include:  Alias  Wavefront,  Web  Feat,  Apple,  Microsoft,  the  Design  Exchange 
and  Nelvana.  Our  partners  include:  Waterloo  University,  Apple,  Kodak,  Rogers  Cable  and  Softimage.  Our  senior 
class  was  involved  in  designing  and  producing  the  first  edition  of ‘‘RE:  DESIGN”,  a web  “zine”  about  design  for 
teachers  and  students  by  students,  www.dxnet.net.  Currently  a number  of  senior  co-op  students  have  placements  at 
the  Design  Exchange  where  they  are  working  on  the  website  for  Toronto’s  Olympic  bid.  Our  program  has  won  an 
Apple  School  of  Distinction  Award  and  some  of  its  teachers  have  won  a 1999  Prime  Minister’.^  Award  for 
Achievement.  We  will  focus  on: 

• problem  oriented  curriculum  projects  and  units 

• subject  integration  and  authentic  learning  assessment  techniques 

• real-world  connections  and  business  partners 

Managing  Electronic  Resources  - Auto-registration  for  Distance  Learning 

Richard  Fasse,  Rochester  Institute  of  Technology,  US;  Damon  Betlow,  Rochester  Institute  of  Technology,  US;  Randy 
Overbeck,  Rochester  Institute  of  Technology,  US 

Electronic  resources  used  in  distance  learning  are  often  restricted  to  registered  students  through  login  ids  and 
passwords.  As  electronic  resources  proliferate,  so  do  the  problems  with  managing  these  accounts  for  both  students 
and  administrators.  One  option  is  a form  of  auto-registration  with  “course  keys”  that  allows  users  to  select  their 
own  login  id  and  password,  but  restricts  registration  to  only  those  who  received  the  “course  key”  in  a handout  or 
email. 

Interactive  Training  Materials  for  Early  Childhood  Educators:  Cases,  Tools,  and 
Reflection 

Gail  Fitzgerald,  University  of  Missouri-Columbia,  USA;  Louis  Semrau,  Arkansas  State  University,  USA 

Materials  to  assist  early  childhood  educators  in  working  with  young  children  with  behavioral  problems  are  currently 
in  high  demand.  Multimedia  programs  offer  an  effective  method  for  providing  instruction  based  on  authentic  cases 
for  developing  necessary  knowledge  and  skills.  In  the  multimedia  program,  "Trisha,”  users  can  seek  professional 
knowledge  from  an  information  database,  watch  Trisha  in  multiple  settings,  interview  her  teachers  and  other  care 
providers,  read  case  records,  listen  to  expert  commentators  discuss  theoretical  approaches  and  care 
recommendations,  and  carry  out  a series  of  real-world  activities  related  to  Trisha's  needs.  Following  problem- 
solving activities,  users  engage  in  a series  of  reflective  prompts  to  review  their  decisions  based  on  best  practices.  The 
program  contains  electronic  performance  support  tools  that  can  be  used  within  the  program  as  well  as  in  actual  job 
situations.  "Trisha"  is  designed  for  implementation  with  workers  in  childcare  settings  and  preservice  courses. 
Additional  information  is  available  at  http://www.coe.missouri.edu/~vrcbd/ . 

Integrating  computer  ethics  into  the  computer  science  and  computer  engineering 
curricula 

John  Fodor,  Educational  Media  Resources,  Inc.,  USA 

This  demonstration  will  show  how  to  integrate  computer  ethics  into  the  computer  science  and  computer 
engineering  curricula  by  using  the  interactive  CD-ROM  Understanding  Computer  Ethics.  (UCE  was  made  possible 
by  grants  from  the  National  Science  Foundation  and  has  won  AXIEM,  Communicator  and  International  CINDY 
Awards.)  We  will  discuss  such  topics  as  computer  privacy,  computer  security,  ownership  of  intellectual  property, 
software  piracy,  hacking  and  professional  responsibility.  We  will  examine  ways  of  developing  skills,  sensitivities,  and 
understandings  to  "human  value"  issues  and  problems  raised  by  computing  and  information  technologies,  including 
ways  to  elicit  more  reflective  performance  understandings.  We  will  analyze  four  ways  specific  ways  of  integrating 
computer  ethics  into  the  computer  science  and  computer  engineering  curricula: 

1)  as  a stand-alone  class, 

2)  incorporating  computer  ethics  into  existing  cs  & ce  classes, 

3)  as  part  of  a capstone  senior  project,  and 

4)  as  an  independent  study  class. 
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Development  and  Implementation  of  Computerized  Instructional  Materials:  The  Mismatch 

Between  Formal  and  Informal  Technology  Support  Structures  in  a Medical  College 

Cynthia  Frank,  The  University  of  Arizona,  U.SA 

This  study  focused  on  the  development  and  implementation  of  instructional  computer  materials  in  the  everyday 
teaching  environment  of  a southwestern  U.S.  medical  college.  What  happened  between  the  time  an  instructor 
voluntarily  decided  to  implement  computer  technologies  in  a course  and  the  time  a student  could  actually  use  the 
technologies?  The  existence  of  formal  technology  support  structures  led  to  the  assumption  that  instructors  would 
utilize  professional  help.  This  turned  out  not  to  be  the  case.  The  majority  of  instructors  used  informal  support 
structures  to  develop  and  implement  their  materials.  Much  of  the  work  was  done  by  students  who  had  no  training 
in  instructional  design,  multimedia  programming,  or  graphic  design.  This  preliminary  finding  indicates  a 
fundamental  mismatch  between  the  type  of  support  professors  selected  and  the  type  of  support  offered  by  the 
institution. 

Putting  Curriculum  in  Controi:  Interactive  Courseware  Deveiopment  with  the  TechDisc 

Instructionai  Design  Template 

Traig  Friedrich,  Sencore,  Inc.,  United  States;  Bill  Wann,  Sencore,  Inc.,  United  States;  Karen  Korth,  University  of 

South  Dakota,  United  States;  Mike  Hoadiey,  University  of  South  Dakota,  United  States;  Dan  Eastmond,  University  of 

South  Dakota,  United  States 

The  Technology  for  Training  and  Development  division  at  the  University  of  South  Dakota  has  joined  forces  with 
Sencore,  Inc.,  a multimedia  application  provider,  in  the  development  of  an  innovative  multimedia  authoring  tool. 
Designed  specifically  to  apply  recognized  instructional  design  techniques  to  "drag-and-drop”  multimedia 
development,  this  tool  puts  interactive  courseware  authoring  into  the  hands  of  the  classroom  educator.  This 
approach  to  multimedia  application  development  benefits  everyone  involved  in  K-12  education  by  enhancing  the 
classroom  teacher's  control  of  course  curriculum  and  specific  course  content.  During  this  demonstration  of  the 
TechDisc  Instructional  Design  Template,  participants  are  welcome  to  engage  in  open  discussions  of  the  tool,  trends 
in  multimedia  project  authoring,  and  other  issues  surrounding  interactive  courseware  development. 

Using  Virtual  Reality  to  Assist  Air  Travelers  With  Disabilities 

Clark  Germann,  Metropolitan  State  College  of  Denver,  USA;  Jane  Broida,  Metropolitan  State  College  of  Denver,  USA 

Getting  to  the  airport,  checking  our  luggage,  going  through  security,  boarding  the  plane— these  are  all  tasks  most  of 
us  take  for  granted  each  time  we  fly.  But  to  persons  with  physical  disabilities  who  use  a wheel  chair,  the  above  tasks 
can  be  difficult  and  intimidating.  No  two  airports  are  exactly  the  same,  and  procedures  for  air  travelers  with 
disabilities  differ  with  each  air  carrier.  To  help  ease  the  anxiety,  faculty  and  students  at  Metropolitan  State  College  of 
Denver  have  used  computer  technology  to  duplicate  one  major  airport  into  virtual  reality.  Now,  future  air  travelers 
can  preview  the  experience  first  on  the  computer  prior  to  going  to  the  airport  in  person.  With  135,000  people  in 
wheelchairs  in  Colorado  alone,  provision  of  resources  for  air  travel  is  especially  pertinent.  Survey  research  and 
anecdotal  evidence  has  shown  that  these  virtual  representations  can  be  helpful  in  reducing  anxiety  in  persons  with 
disabilities  by  providing  a preview  of  a location  prior  to  visiting  it  in  person. 

Gender  and  Race:  The  Accuracy  of  Internet  Clip  Art 

Carol  Gilley,  University  of  Arkansas,  USA;  Elaine  Terrell,  University  of  Arkansas,  USA 

Abstract:  Computers  are  being  used  to  create  various  print  media  as  well  as  multimedia.  Some  questions  are  being 
raised  in  developing  these  media  materials.  Is  there  a supply  of  free  clip  art  on  the  Internet  that  represents  the  real 
world?  Do  viewers  see  themselves  in  clip  art  used  in  print  and  computer-based  multimedia?  This  study  explores  the 
issues  of  equity  and  stereotyping  with  regard  to  the  images  in  free  Internet  clip  art  by  investigating  the  following  two 
hypothesis.  1)  There  is  no  significant  difference  between  the  proportion  of  men,  women,  boys,  and  girls  represented 
in  free  Internet  clip  art  and  the  proportion  of  men,  women,  boys,  and  girls  in  the  general  population.  2)  There  is  no 
significant  difference  between  the  proportion  of  White,  Blacks,  and  other  minorities  represented  in  free  Internet  clip 
art  and  the  proportion  of  White,  Blacks,  and  other  minorities  in  the  general  population. 

How  To  Be  Innovative  Designing  Educational  And  Interactive  Environments  For  Children 

Gloria  Elena  Gomez  Escobar,  University  of  Los  Andes,  Colombia 

LUDOMATICA  PROJECT  looks  to  effectively  attend  children's  educational  needs.  It  seeks  to  help  them  to 
creatively  think  and  act,  validating  their  rights  to  high-quality  education  and  to  actively  participate  as  social  change 
agents.  This  validation  is  accomplished  through  innovations  in  formal,  non-formal  and  informal  education,  using 
Information  and  Telecommunication  Technologies  within  a non-conventional  pedagogical.  We  want  to  develop  the 
intellectual  potential  and  cognitive  abilities  of  children  who  live  in  high-risk  situations,  taking  advantage  of  their 
adverse  learning  to  stimulate  creative  potentials,  communicative  resources  and  have  them  discover  their  own 
adventurous  spirit  and  creative  capabilities.  The  interactive  game  THE  FANTASTIC  CITY  is  one  of  the 
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environments  created  for  these  purposes.  This  is  a digital  multimedia  microworld  that  motivates  and  develops 
children’s  fantasy  and  creativity  by  means  of  enigmas  and  puzzles,  seeking  to  generate  cognitive  and  affective 
imbalances.  This  interactive  environment  leads  children  to  action.  It  promotes  curiosity  and  experiential  learning;  it 
allows  speculating,  discovering,  learning  from  mistakes;  and  it  stimulates  the  desire  to  learn. 

Teaching  Dynamics  of  Dairy  Herd  Health  and  Management 

Claudia  Haferkamp-Wise,  Cornell  University,  USA;  Heather  G.  Allore,  Cornell  University,  USA;  Yijo  T.  Grohn,  Cornell 
University,  USA;  Lorin  D.  Warnick,  Cornell  University,  USA 

Cornell  University  offers  "Dynamics  of  Dairy  Herd  Health  and  Management" 
(http://courseinfo.cit.cornell.edu/courses/vetmed745)  - a newly  designed  web-based  distribution  course  to  its  on- 
campus  veterinary  students  and  to  students  at  9 off-campus  test  sites  within  the  United  States  and  abroad.  This 
course  uses  a computer-based  instructional  delivery  system  (DairySim)  to  teach  basic  concepts  of  epidemiology  and 
economics  of  dsky  cattle  disorders,  experimental  design,  and  statistics.  The  courseware  provides  ovo  simulation 
models  - SIMHEALTH  and  SIIMIMAST  - with  an  interactive  user-friendly  interface  and  output  analysis  - thus 
making  research  tools  accessible  to  a broader  audience.  DairySim  and  its  accompanying  user  guides  are  available  on- 
line. Lectures  can  be  attended  in  person  on-campus  or  viewed  via  the  internet.  Students  can  print  handouts  for  use 
in  computer  laboratory  sessions,  take  on-line  quizzes  and  download  homework  assignments.  Completed 
assignments  are  handed  in  by  uploading  the  documents.  Instructor-student  and  student-student  communication  for 
off-campus  students  takes  place  via  email  and  an  on-line  discussion  board. 

A Comprehensive  Model  for  Improving  Technology  in  Teacher  Education 

Robert  (Bob)  Hannafin,  College  of  William  and  Mary,  US;  Robert  (Bob)  Hanny,  College  of  William  and  Mary,  US. 

This  paper  identifies  the  professional  development  model  developed  at  the  College  of  William  and  Mary  to  close  the 
gaps  between  its  strategic  vision  for  the  teacher  education  program  and  its  capacity.  The  College's  vision  is  to 
produce  teachers  who  seamlessly  integrate  technology  by  designing  and  assessing  authentic  student-centered 
learning  activities.  Three  areas  were  identified  as  requiring  parallel  effort:  faculty  development,  institutional 
development,  and  clinical  faculty/PDS  development.  Goals  were  established  that  addressed  capacity  shortfall  and 
advance  the  program  to  the  point  where  significant  innovation  could  occur.  Both  our  students  and  faculty  needed 
to  master  both  basic  technology  skills  and  the  subtler  art  of  practicing  creative  integration  strategies.  It  also  required 
placing  our  students  with  teachers  who  are  competent  technology  users  and  encouraging  in  technologically-adequate 
classrooms.  Institutional  changes  were  necessary  on  two  fronts:  upgrading  the  technology  available  in  School 
classrooms,  and  instituting  incentives  to  entice  faculty  to  use  and  integrate  technology. 

Web-Based  Tools  For  Courses  On  Transport  Phenomena 

Bernardo  Hernandez-Morales,  Universidad  Nacional  Autonoma  de  Mexico,  Mexico;  Rafael  Femandez-Flores, 
Universidad  Nacional  Autonoma  de  Mexico,  Mexico;  Jorge  Tellez-Martinez,  Universidad  Nacional  Autonoma  de 
Mexico,  Mexico 

Even  though  multimedia  applications  have  been  extensively  used  in  many  courses,  the  field  of  transport  phenom- 
ena has  received  comparatively  less  attention.  Thus,  we  are  developing  interactive  tools,  to  be  accessed  through 
Internet,  to  solve  problems  observed  when  traditional  teaching  techniques  are  used.  The  material  consists  of 
modules  to  solve  typical  problems  in  transport  phenomena.  Each  module  is  a Webpage  that  includes  the  theo- 
retical background  (with  hyperlinks  to  all  course  material)  and  an  interactive  worksheet  to  solve  a particular 
problem.  The  worksheets  are  built  with  Java  applets  (JDK  1.1.  compatible)  and  may  may  be  used  in  any  of  three 
modes:  example  mode  (fixed  set  of  values),  training  mode  (variable  set  of  values),  and  exploration  mode  (range  of 
values  of  a key  independent  variable).  Thus,  unlike  problems  found  in  traditional  textbooks,  the  student  may 
explore  a range  of  values  for  key  independent  variables  to  establish  its  influence  on  the  dependent  variables. 

Shaknoma:  A collaborative  tool  for  shared  knowledge  management 

Oriel  Herrera,  Pontificia  Universidad  Catolica  de  Chile,  Chile;  Sergio  Ochoa,  Pontificia  Universidad  Catolica  de  Chile, 
Chile;  Vidal  Rodriguez,  Pontificia  Universidad  Catolica  de  Chile,  Chile;  David  Fuller,  Pontificia  Universidad  Catolica 
de  Chile,  Chile 

Nowadays,  the  amount  of  scientific  knowledge  produced  is  overwhelming  and  it  is  clear  that  it  will  grow  even  more. 
Shaknoma  (Shared  Knowledge  Manager)  is  a tool  useful  to  make  available  and  share  the  people’s  individual 
knowledge.  The  activities  carried  out  on  this  tool  respond  to  a model  that  ensure  the  input,  validation,  use  and 
evolution  of  the  knowledge.  The  process  is  implemented  through  four  major  components:  the  shared  repository, 
the  management  model  (see  Herrera  et  al.  2000),  the  representation  language  of  the  knowledge,  and  the 
collaborative  environment.  The  knowledge  management  model  provides  decision-making  mechanisms,  which 
mainly  includes  discussion  and  voting  activities.  As  a way  of  structuring  and  graphic  representation,  Shaknoma  uses 
a language  based  on  conceptual  maps  (Novak  1984).  The  information  repository  does  not  store  only  knowledge,  but 
also  the  information  of  its  use  and  evolution. 
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Teachers  and  Computer  Technology  Training 

David  Hofmeister,  Central  Missouri  State  University,  USA;  Gerry  Peterson,  Central  Missouri  State  University,  USA 

The  poster  session  provides  an  overview  of  the  processes  and  materials  used  to  advance  the  integration  of 
computer  technology  into  student  learning  in  K-12  schools. 

Results  and  conclusions  from  a Learning  Software  Design  Competition 

Brad  Hokanson,  University  Of  Minnesota,  USA;  Simon  Hooper,  University  Of  Minnesota,  USA;  Paul  Bernhardt, 
University  Of  Minneosta,  USA 

The  University  of  Minnesota's  Design  Institute  is  sponsoring  a competition  to  recognize  the  design  of  innovative 
educational  software.  The  competition  was  open  to  professional  and  not-for-profit  educational  software  designers 
in  such  areas  as  higher  education,  K-12  education  and  commercial  training.  Prizes  will  be  given  in  each  category, 
and  entries  for  the  competitions  were  due  February  15,  2000.  Winners  of  the  competition  were  announced  April  10, 
2000.  One  of  the  competition's  goals  is  to  esmblish  a bank  of  exemplar/  educational  software  tliat  can  be  examined 
by  designers  and  students  of  educational  software  design.  To  facilitate  this  goal,  winning  designs  (and  others 
selected  for  inclusion)  will  be  compiled  and  made  available  to  the  public.  We  will  present  the  results  of  the 
competition  in  a workshop  format  at  ED-MEDIA  2000.  The  intended  audience  is  educators  and  practitioners  in 
the  field  of  educational  technology. 

Constructive  Hypertext  for  Learning 

Hsi-chi  Huang,  The  Ohio  State  University,  USA 

This  study  focuses  on  teachers  as  designers  using  constructive  h)^ertext  and  teachers'  perspectives  on  the  use  of  the 
World  Wide  Web  in  educational  contexts  and  its  implications.  Constructive  hypertext  encourages  the  participants 
not  only  to  be  users  of  hypertext,  but  also  to  be  designers  of  h}^ertext.  The  study  describes  the  participants’ 
experiences  as  designers  and  their  thoughts  on  designing  h}^ertext  for  learning.  In  designing  Web-based  hypertext 
learning  environments,  educators  can  make  use  of  the  Web  from  three  approaches:  the  Web  as  a resource  for 
information,  the  Web  as  a medium  of  expression-representation,  and  the  Web  as  a space  for  dialogue.  The 
distribution  perspective  takes  advantage  of  the  Web  as  an  information  network.  The  expression-representation 
perspective  uses  the  Web  as  a space  for  publishing,  and  expressing  ideas  and  creativity.  The  Web  as  a space  for 
dialogue  has  the  capacity  to  encourage  and  embrace  different  voices.  To  participate  in  the  dialogue  is  to  create 
voices.  The  dialogue  space  provides  possibilities  for  students  to  converse  and  exchange  learning  experiences. 

Internet  Infrastructure  with  ATM  for  Small  Countries;  Case  Studies 

Necdet  ICIL,  Eastern  Mediterranean  University,  TURKIYE 

Rapid  development  on  communication  technologies,  become  easy  to  support  QoS  (Quality  of  Services)  application 
on  Internet.  As  a case  study.  North  Cyprus  which  is  a small  country  with  5 small  cities,  develop  a project  to  connect 
all  cities  with  ATM  BackBone.  The  purposes  of  the  project  is  to  connect  all  the  local  government  offices  to  the 
cities  node  and  to  use  all  multimedia  and  information  facilities  on  the  BackBone.  All  governmental  documentation 
will  be  send  electronically  between  the  offices.  Virtual  meeting  will  be  possible  by  using  video  conferencing 
facilities.  Also  all  schools  will  connect  to  the  Ministry  of  Education  to  use  educational  facilities  which  are  prepared 
by  the  trainers. 

NACSIS-ILL  Distance  Training  System  in  IDLE  Distance  Learning  Project 

Tomo’o  Inoue,  National  Institute  of  Informatics,  Japan;  Haruki  Ueno,  National  Institute  of  Informatics,  Japan 

We  have  started  the  distance  learning  project  named  IDLE  (Integrated  Distance  Learning  Environment)  in  1998. 
The  primary  motivation  of  beginning  the  project  is  to  meet  the  social  needs.  There  are  not  many  practical  and 
sustainable  DL  systems  in  Japan,  while  on  the  other  hand  DL  is  already  recognized  as  a natural  form  of  getting 
education  in  the  US.  Current  core  product  of  IDLE  is  NACSIS-ILL  Distance  Training  System,  a WWW  based 
system  for  training  NACSIS-ILL  operators.  NACSIS-ILL  is  a system  that  our  institute  offers  to  nearly  700 
university  libraries  to  support  exchange  of  information  for  the  inter-library  loan  service.  Our  institute  also  offers 
NACSIS-ILL  Training  Courses  to  train  the  staff  in  the  inter-library  loan  sections  of  the  participating  libraries.  We 
have  put  emphasis  on  its  practicality  in  the  project  and  applied  DL  to  the  training  course.  The  system  will  be  in 
actual  use  this  year. 
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Linking  Active  Learning  to  Web-based  instruction:  Students  Teaching  Students  through 
Multimedia  Productions 

James  S.  Javenkoski,  Department  of  Food  Science  and  Human  Nutrition,  University  of  Illinois  at  Urbana-Champaign, 
USA;  Elizabeth  F.  Reutter,  Department  of  Food  Science  and  Human  Nutrition,  University  of  Illinois  at  Urbana- 
Champaign,  USA;  James  E.  Painter,  Department  of  Food  Science  and  Human  Nutrition,  University  of  Illinois  at 
Urbana-Champaign,  USA 

Instructors  (academic  teachers  and  corporate  trainers)  recognize  the  value  of  video  as  an  educational  medium. 
Video  provides  learners  (students  and  trainees)  with  opportunities  to  observe  phenomenological  examples  that 
reinforce  the  concepts  that  are  verbally  described  during  classroom  and  laboratory  sessions  and  on-the-job-training. 
Until  recendy,  educational  video  production  was  a multi-step,  multi-component  process  requiring  at  the  minimum  a 
camera  or  VCR,  a computer  (wi^  a high  speed  processor,  very  large  hard  drive,  and  video  editing  and  processing 
software  instaJledV  and  a high  speed  serial  I/O  connection  to  transfer  the  video  data  between  hardware  devices. 
The  new  Apple©  iMac™  DV  desktop  computer  and  iMovie'^^  software  offer  a simple,  accessible,  and  affordable 
solution  to  instructors  who  wish  to  create  continuous  media  (audio  and  video)  for  use  in  their  courses.  Our 
demonstration  will  showcase  the  QuickTime  streaming  videos  produced  in  UIUC’s  FSHN  199:  Business  Etiquette 
and  Protocol  and  FSHN  355:  Fine  Dining  Management  courses. 

Using  Tailored,  Interactive  Soap  Operas  for  Breast  Cancer  Education  of  High-Risk 
Hispanic  Women 

Maria  Jibaja,  Baylor  College  of  Medicine,  USA;  Paul  Kingery,  Hamilton  Fish  Institute,  USA;  Nancy  Neff,  Baylor 
College  of  Medicine,  USA;  Quentin  Smith,  Baylor  College  of  Medicine,  USA;  Jennifer  Bowman,  USA;  J.  David 
Holcomb,  USA 

While  Hispanic  women  have  lower  rates  of  breast  cancer  than  other  groups,  among  women  from  all  ethnic  groups, 
they  are  the  least  likely  to  undergo  screening  exams.  This  study  evaluated  a culturally  sensitive  and  linguistically 
appropriate  tailored  computer-based  educational  program  on  breast  cancer  early  detection  aimed  at  high-risk 
Hispanic  women.  Spanish-speaking  Hispanic  women  from  an  inner-city  community  health  clinic  were  recruited  and 
randomly  assigned  to  either  a computer  intervention  with  an  interactive  soap-opera  format  (n=118)  or  a 
comparison  group  (n=60).  True-false  pre-  and  post-tests  were  used  to  identify  any  change  in  breast  cancer  related 
knowledge/beliefs.  Both  younger  (18-40  y.o.)  and  older  (41-65  y.o.)  women  in  the  intervention  group  increased 
significantly  their  knowledge/beliefs  as  compared  to  the  younger  and  older  women  in  the  comparison  group 
(p<.05).  Computer-based  tailored  and  interactive  soap  operas  that  are  linguistically  and  culturally  appropriate  are 
effective  in  increasing  breast  cancer  screening  knowledge /beliefs  in  high-risk  Spanish-speaking  Hispanic  women. 

webStract  - a Distributed  Tool  for  Collaborative,  Project-driven  Learning  on  the  Internet 

Werner  B.  Joerg,  NetEssence  / U of  Alberta,  USA 

webStract  is  a combined  software  tool  that  supports  the  construction  and  delivery  of  Internet  based  courseware.  It 
embodies  a project-driven  paradigm  - a problem  based  learning  approach  guided  by  the  needs  of  student  projects.  It 
allows  content  providers  to  make  effective  use  of  the  vast  knowledge  source  of  the  World  Wide  Web  and  to  present 
the  students  with  qualified  material  in  a structured  manner.  It  takes  advantage  also  of  the  interaction  capabilities  of 
the  Internet,  to  enable  focused  synchronous  and  asynchronous  communication  among  distributed  teams  and  team 
members.  Following  a brief  introduction  of  the  motivation  and  the  learning  paradigm,  our  presentation  centers  on 
the  three  principal  areas  of  service  covered  by  webStract:  knowledge  qualification  and  structuring  tools  for  the 
content  provider,  collaboration  and  management  tools  for  the  teams,  and  knowledge  management  tools  for  the 
individual  student.  webStract  is  currently  being  used  in  experimental  courses.  We’ll  report  feedback  from  those 
experiments. 

Digital  Learning  System:  Web's  Cool 

Myunghee  Kang,  Ewha  Womans  University,  Korea 

Web's  Cool  (can  also  be  pronounced  Web  School)  program  with  SRL(Self-Regulated  Learning  principles, 
PBL(Problem  Based  Learning  approaches  and  Keller's  ARCS  model  for  motivation  has  been  designed  by  the 
author  and  developed  by  Samsung  Electronics  Company  in  Korea.  Now,  all  the  contents  and  study  modules  are 
completed  in  CD-ROM  and  the  total  service  including  Q&A  and  real  tutorial  help  is  now  available  on  the  UNITED 
web  service. 
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Learning  Amidst  A Sea  Of  information  in  The  New  Miiiennium 

Dan  Kauwell,  University  of  Illinois,  Urbana-Champaign,  USA;  Jim  Levin,  University  of  Illinois,  Urbana-Champaign, 

USA;  Daniel  Schiff,  University  of  Illinois,  Urbana-Champaign,  USA;  Young-Jin  Lee,  University  of  Illinois,  Urbana- 
Champaign,  USA 

Visualization  can  improve  our  ability  to  access  information  and  construct  knowledge.  Information  visualization  is 
particularly  well  suited  for  the  location  and  analysis  of  information  found  on  the  Internet  and,  for  the  construction 
of  knowledge  from  that  information.  Our  research  has  led  us  to  develop  VisIT,  a tool  for  the  visualization  of 
Internet  based  information  and  the  analysis  of  that  information.  Instead  of  lengthy  lists  of  search  results,  the  user  is 
presented  with  a graphical,  spatial  representation  of  the  search  space.  Now  the  user  can  *’see”  the  hits  returned  by 
the  search  engine  as  well  as  other  pages  from  the  same  site.  When  any  of  the  pages  are  clicked,  the  appropriate  page 
is  displayed  in  the  browser  window.  Furthermore,  VisIT  facilitates  the  knowledge  construction  process  by  allowing 
its  graphical  displays  to  be  edited,  saved  and  re-opened  later. 

An@tomedia:  A New  Approach  TO  Medical  Education  Developments  In  Anatomy 

David  M.  Kennedy,  Monash  University,  Australia;  Norm  Eizenberg,  The  University  of  Melbourne,  Australia;  Chris 
Briggs,  The  University  of  Melbourne,  Australia;  Ivica  Grkovic,  The  University  of  Melbourne,  Australia;  Priscilla  Barker, 
The  University  of  Melbourne,  Australia 

Evaluations  have  indicated  that  students  find  An@tomedia  very  engaging  (Kennedy,  Eizenberg,  & Kennedy,  In  this 
volume).  An@tomedia  supports  students  actively  in  their  learning  tasks,  fostering  a deeper  understanding  of  human 
structure,  with  a stronger  basis  for  clinical  diagnosis  and  procedures.  The  new  medical  course  has  focused  less  on 
traditional  teaching,  instead  emphasising  on  focussed  dissection,  problem-based  and  self-directed  learning. 
An@tomedia  provides  multiple  perspectives  of  the  human  body.  Each  module  is  independent  and  of  equal 
importance.  The  modules  are: 

1 . Back, 

2.  Abdomen, 

3.  Thorax, 

4.  Neck, 

5.  Upper  Limb, 

6.  Lower  Limb, 

7.  Head,  and 

8.  Pelvis. 

The  first  3 of  the  8 modules,  the  Back,  Abdomen,  and  Thorax  have  been  completed,  each  organised  into  4 major 
perspectives. 

1.  'Dissection’  includes  practical  (including  emergency)  procedures. 

2.  'Imaging'  incorporates  sectional  and  endoscopic  anatomy. 

3.  'Regions'  incorporates  surface  and  functional  anatomy. 

4.  'Systems'  incorporates  conceptual  and  clinical  anatomy. 

References 

Kennedy,  D.  M.,  Eizenberg,  N.  & Kennedy,  G.  (In  this  volume).  An  evaluation  of  interactive  multimedia  designed 
to  support  problem-based  learning  in  medicine.  In  ED-MEDIA  & ED-TELECOM  2000,  Proceedings  of  the 
12th  World  Conference  on  Educational  Multimedia  and  Hypermedia  & World  Conference  on  Educational 
Telecommunications.  Montreal,  Canada:  Association  for  the  Advancement  of  Computing  in  Education. 

Offering  Online  Degree  Programs:  A Case  Study  Issues,  Challenges,  Successes,  and 
Lessons  Learned 

Gerard  Kickul,  University  of  St.  Francis,  USA;  Laurel  Jeris,  University  of  St.  Francis,  USA;  Michael  LaRocco, 

University  of  St.  Francis,  USA 

Offering  degree  programs  online  offers  an  attractive  option  in  the  higher  education  environment.  As  schools 
confront  this  opportunity,  famdiar  issues  of  quality,  access,  participation,  retention,  and  assessment  take  on  new 
meaning.  Online  delivery  systems  parallel  traditional  classrooms,  including  discussion  and  assignment  areas,  chat 
rooms,  and  online  assessments.  After  experiencing  high  student  dropout  rates  the  University  focused  on  several 
solutions  to  reduce  its  student  dropout  rate.  These  included  online  support  systems  with  a centralized  approach  to 
addressing  faculty  and  student  technical  concerns,  complaints,  and  requests  for  assistance.  A new  assessment 
procedure  consists  of  a two-phase  process,  a pre-course  assessment  and  a more  traditional  assessment  occurring 
during  the  final  three  weeks  of  the  course.  Finally,  feedback  from  faculty  on  the  various  instructional  design 
challenges  including  learning  curve,  time  commitment,  recommended  class  size,  ethical  issues,  adaptability  of 
subject  matter  content,  and  practical  suggestions  for  reconceptualizing  syllabi  and  assignments  for  online  learning 
will  be  included. 
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Developing  a Distance  Education  Infrastructure 

Gerard  Kickul,  University  of  St.  Francis,  USA;  Mark  Snodgrass,  University  of  St.  Francis,  USA 

Development  of  distance  learning  infrastructure  is  a multifaceted  project,  from  the  desktop  to  the  far  reaches  of  the 
Internet.  Attention  must  be  focused  on  the  outgoing  data  and  information,  as  well  as,  incoming  data  and 
information.  There  are  four  basic  areas  of  concern  when  designing  an  infrastructure  to  facilitate  Internet  distance 
education  solutions:  hardware,  software,  support,  and  security.  Providing  the  proper  hardware  to  developers  and 
support  personnel  is  crucial  in  creating  a productive  and  efficient  work  environment.  Reliable  and  efficient  Internet 
access  is  essential,  with  24-hour  availability  and  minimal  down  time.  Software  components  for  development  and 
support  are  key  ingredients  to  success.  From  a student  and  faculty  standpoint,  the  course  software  must  be  user 
friendly  with  a high  degree  of  stability.  Support  must  be  considered  from  both  a human  and  a technological 
standpoint.  Essential  to  any  online  endeavor  is  secure  access  and  distance  learning  demands  the  highest  levels  of 
security. 

Conteni  Management  for  Web  Based  Learning 

Thomas  Kleinberger,  TECMATH  AG,  Germany;  Paul  Muller,  University  of  Kaiserslautern,  Germany 

With  the  help  of  the  newest  information  and  communication  technology  all  the  needs  of  modern  educational 
systems  can  be  met  for  the  first  time  in  one  tool:  the  computer.  Computer  based  solutions  provide  the  technological 
support  necessary  for  the  whole  teaching  and  learning  process.  The  greatest  advantage  of  this  tool  is  the  possible 
combination  of  all  its  usable  instruments,  their  flexibility  and  configurability.  With  new  technologies  and  new  media 
types  it  will  be  possible  to  support  all  kinds  of  training,  self-study  and  continuing  education.  It  is  clear  that  these 
new  methods  and  technologies  require  new  methods  and  solutions  for  the  tasks  content  creation,  delivery  and 
archiving.  Especially  the  use  of  new  media  sets  high  requirements  on  these  tasks.  This  article  describes  how  content 
management  systems,  a technology  already  used  in  other  kinds  of  business,  can  support  these  tasks  in  web  based 
teaching  and  learning.  A definition  of  content  management  is  made  and  refined  for  the  area  of  web  based  learning. 
The  main  building  blocks  and  features  of  content  management  systems  are  described  and  the  improvements  for 
web  based  learning  are  highlighted. 

Computer-Mediated  Communication  for  Distance  Education:  Deveioping  and  Teaching  a 
Second  Language  Course  in  Academic  Reading 

Esther  Klein-Wohl,  Open  University  of  Israel,  Israel 

The  Open  University  of  Israel  is  searching  for  ways  to  improve  its  distance  education  methods.  Taking  advantage  of 
the  flexibility  of  computer-mediated  communication  (CMC),  we  will  describe  the  application  of  CMC  to  academic 
reading.  We  chose  e-mail  support,  which  allows  students  to  work  on  texts,  and  to  receive  feedback  and. help  when 
needed.  Reading  - a mode  of  language  use,  and  distance  education  - a mode  of  language  instruction,  feature  striking 
parallels:  the  reader  (in  “real  life’*)  and  the  learner  (when  learning  to  read)  each  functions  in  isolation  when 
interacting  with  a text.  Discourse  is  enacted  at  a distance,  a disassociated  first  person  (the  author  or  the  instructor)  is 
actively  present,  and  no  reciprocity  is  manifest,  within  the  interactive  context.  Therefore,  reading  instruction  and 
distance  education  seem  very  well-suited.  Because  CMC  is  particularly  learner-centered,  it  seemed  appropriate  to 
adopt  this  more  motivating  medium  of  instruction  to  our  traditional  reading  program. 

The  Major  Promise  of  Distance  Education  Is  On  Campus 

W.  R.  (Bill)  Klemm,  Texas  A&M  University,  USA 

Distance  Education  (DE)  is  reforming  on-campus  instruction.  DE’s  impact  extends  beyond  the  mere  use  of 
technology  to  put  more  razzle-dazzle  into  classroom  presentations  and  beyond  the  advantages  of  assisted  learning 
via  computer  simulations,  drills,  and  animations.  DE  should  force  teachers  to  re-examine  teaching  philosophies, 
style,  and  tactics.  Distance  Education  shows  that  learning  need  not  depend  on  classroom  lecturing.  Indeed,  we 
should  now  re -consider  the  whole  purpose  of  the  classroom  environment.  DE  prompts  us  to  re-examine 
accrediting  criteria  of  "contact  hours"  and  "credit  hours."  Teachers,  now  freed  from  the  lecturing  straight- jacket, 
can  explore  creative  ways  to  show  students  how  to  find,  comprehend,  manage,  integrate,  and  apply  information. 
DE  reveals  the  value  and  need  in  teaching  for  more  interpersonal  communication,  between  teachers  and  students 
and  among  students.  I see  a revival  of  interest  among  teachers  in  group  work.  Specific  strategies  for  this  new  kind 
of  classroom  teaching  are  illustrated  in  the  poster. 

Interactive  Ophthalmic  Pathology  Tutor 

Gordon  K.  Klintworth,  Duke  University  Medical  Center,  USA;  Anthony  N.  Benson,  Duke  University  Medical  Center, 
USA;  Ann  L.  Bushyhead,  Duke  University  Medical  Center,  USA 

A CD-ROM  interactive,  instructional  course  on  ophthalmic  pathology  and  relevant  anatomy  with  hundreds  of  high 
quality  images  has  been  developed  for  users  with  different  knowledge  backgrounds. 
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Cognitive  idea  Processor  — Modified  Mandai-Art 

Kagemasa  Kozuki,  Konami  Co,  Japan;  Atsushi  Tsubokura,  Osaka  Electro  Communiation  Univ.,  Japan;  Shogo 
Harima,  Osaka  Electro  Communiation  Univ.,  Japan;  Noboru  Ashida,  Osaka  Electro  Communiation  Univ.,  Japan; 
Katsuhide  Tsushima,  Osaka  Electro  Communiation  Univ.,  Japan 

The  new  idea  processor  called  modified  Mandal-Art  which  can  assist  a user  in  his  associative  thinking  between 
given  item  have  been  developed  in  which  a user  can  communicate  with  the  system  using  eye  movement  , finger- 
hand  motion  and  voice.  A user  of  this  idea  processor  is  forced  to  associate  by  seeing  only  nine  items  on  the  matrix 
called  Mandala  on  CRT  at  a time.  We  have  measured  this  information  retrieve  behavior  from  item  pool  of  our 
testee  putting  on  eye  mark  recorder  and  data  gloves.  Human  associative  thinking  in  his  problem  solving  is 
accelerated  by  using  this  cognitive  idea  processor.  This  type  of  idea  processor  may  brings  us  innovation  of 
educational  style  on  the  computer  by  using  the  student  model  which  take  account  the  real  time  associative  behavior 
of  a learner. 

Dilemma  of  Inquiry  and  Reflection  Through  and  With  Technology:  New  Directions  for 
Research  in  Post-Secondary  Teaching  - 

Olga  Kritskaya,  Michigan  State  University,  USA;  Tony  Clay.  Michigan  State  University,  USA 

The  study  explored  the  issues  of  pedagogy  for  teaching  reflective  inquiry  in  the  undergraduate  program  in 
Educational  Psychology.  The  analysis  focused  on  the  role  of  the  learners’  imagination  and  the  use  of  instructional 
hyper-media  texts  in  construction  of  meanings  associated  with  the  course  content.  Particular  attention  was  paid  to 
the  development  of  student  design  ideas  that  they  use  for  the  arrangement  of  images,  which  reflect,  as  a special 
form  of  narrative,  on  their  understanding  of  the  disciplinary  concepts,  personalities  and  life  experiences.  The  data 
illuminate  the  instructional  conditions  when  the  student's  act  of  arranging  information  becomes  an  act  of  insight. 
The  computer-based  student  projects  (Teacher's  Portfolio,  Multi-Media  Gallery),  along  with  the  instructional  hyper- 
media texts,  reveal  a critical  instructional  shift  in  the  pedagogy  of  teaching  reflective  thinking- from  structure  to 
process,  from  enforcing  students'  competence  to  engaging  them  into  performance-that  fosters  a deeper 
understanding  of  the  dialectics  of  socio-cultural  processes. 

Assessing  and  Training  Admunct  Faculty  with  Technology:  Enhancing  their  Classroom 
Learning 

Karen  Krupar,  Metropolitan  State  College  of  Denver,  USA 

Increasing  numbers  of  adjunct  faulty  at  all  major  institutions  of  higher  education  has  made  it  imperative  that 
institutions  examine  the  technology  competencies  of  this  large  contingent  of  faculty  who  now  instruct  40%  of  the 
courses  on  the  undergraduate  level.  Very  Little  effort  is  being  made  to  provide  development  or  training  for  these 
faculty  in  technology  applications  that  would  keep  their  students  current  in  the  rapidly  altering  global  world  of  the 
21st  century.  Metropolitan  State  College  of  Denver  reviewed  their  650  adjunct  faulty  and  found  many  interesting 
factors  that  will  be  reported  during  this  poster  session.  It  is  evident  that  adjunct  faculty  need  assistance  in  both 
learning  specific  technology  programs  and  in  applying  the  technology  into  their  course  work. 

Design  an  Enhanced  Virtual  Experiment  Environment  Using  Science  Process  Skills  on 
WWW 

Li-Ping  Kuo,  Chung  Yuan  Christian  University,  Taiwan,  R.O.C;  Da-Xian  Dong,  Chung  Yuan  Christian  University, 
Taiwan,  R.O.C;  Chang-Kai  Hsu,  Chung  Yuan  Christian  University,  Taiwan,  R.O.C;  Jia-Sheng  Heh,  Chung  Yuan 
Christian  University,  Taiwan,  R.O.C 

This  paper  proposes  a methodolog}'  of  applying  a proper  distance  learning  system  following  Seven  Steps  of 
Problem  Solving  and  Science  Process  Skills.  To  complete  this  process  of  distance  learning,  this  paper  also  su^ests 
some  tools  for  student  undergoing  all  the  process  needed  in  the  problem  solving.  Virtual  Experiment  Environment 
and  Experiment  Record  are  presented  for  some  new  notions  in  this  paper.  In  Virtual  Experiment  Environment, 
Visual  Lab  is  a new  idea  to  operate  experiment  in  Internet  and  show  the  reality  of  experiment.  Experiment  Record 
solves  the  difficult  of  implementing  sheet  and  chart  in  HTML  document.  An  environment  for  teacher  design 
Science  Process  Skills  is  also  presented  in  this  paper. 

Analysis  of  messages  on  the  Only  One  Earth  Club;  TV-based  collaborative  learning  site 
on  the  Internet 

Haruo  Kurokami,  Kanazawa  University,  Japan;  Tatsuya  Horita,  Toyama  University,  Japan;  Yuhei  Yamauchi,  Ibaraki 
University,  Japan 

Messages  sent  to  our  BBS  for  collaborative  learning  can  be  classified  to  10  categories.  Each  of  these  categories  has 
some  types  of  interactions  among  participants  and  site-staffs.  The  types  of  interactions  are  promotion  of  in-depth 
learning,  discussion,  and  question  & answers. 
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Aged  and  Disability  Care  training;  A CD-ROM  based  project 

Mark  Laidler,  RMIT  UNIVERSITY,  Australia 

This  demonstration  has  been  developed  to  highlight  one  solution  for  delivering  high  quality  work-based  training  in 
a multimedia  environment. 

Civilization  in  the  21st  Century 

David  F.  Lancy,  Utah  State  University,  USA;  David  DeBry,  Utah  State  University,  USA;  Megan  Andrew-Hobbs,  Utah 

State  University,  USA 

We  will  report  on  the  evolution  of  an  on-line  course.  The  Civilization/Humanities  course  had  its  origins  in  the 
reform  of  the  university’s  General  Education  curriculum  in  1994-95,  It  was  one  of  several  classes  created  to  replace 
existing  requirements.  The  reform  effort  was  designed  to  create  interdisciplinary  classes  that  would  put  the 
emphasis  on  universal  aspects  of  inquiry  rather  than  the  narrow  focus  of  typical  introductory  classes.  Other 
expected  features  of  these  new  courses  were  an  emphasis  on  'vriting  and  the  integrarion  of  technology.  Trie  inidai 
Civilizadon /Humanities  class  met  these  criteria  and  was,  by  several  measures,  quite  successful.  In  developing  his 
version  of  the  class,  Dr,  Lancy  digidzed  his  ver^'  large  slide  library  and  students  were  thus  able  to  more  readily 
access  the  coUecdon  for  study  and  review.  Another  milestone  occurred  in  1998  when  the  library  adopted  an 
Electronic  Reserve  policy.  Forced  to  incorporate  this  system  into  his  class,  Lancy  went  further  and  adopted  many  of 
the  built-in  features  of  ERes”  such  as  the  "Bulletin  Board."  Civilizadon/  Humanides  was  moving  towards  the  new 
millennium  as  it  were.  It  became  clear  in  the  second  year  of  implemendng  the  new  Gen  Ed  program  (referred  to  as 
University  Studies)  that  not  enough  faculty  were  signing  on  to  teach  the  new  courses.  A botdeneck  emerged  which 
provided  further  incendves  to  adapt  the  few  classes  that  had  been  developed  to  meet  the  new  criteria  for  delivery  to 
a larger  audience.  The  most  recent  stage  in  this  evoludonary  process  we  will  report  on  is  the  transformadon  of  the 
Civilizadon  course  from  a (primarily)  classroom  to  (primarily)  on-line  delivery.  In  concluding  we  will  generalize 
about  this  evoludonary  process  from  cases  gathered  across  the  curriculum  on  our  campus. 

Wired  for  Learning 

Donna  Landin,  West  Virginia  Deaprtment  of  Education,  United  States;  Roberta  Taylor,  IBM,  United  States;  Lynn 

Blaney,  Wheeling  Park  High  School,  United  States;  Susan  Alkire,  Romney  Junior  High  School,  United  States 

The  Reinvendng  Educadon  project  was  established  under  a $2  million  dollar  grant  from  IBM  to  the  West  Virginia 
Department  of  Educadon,  Its  purpose  is  to  define  and  validate  criteria  for  creating  instrucdonal  plans  that  use  the 
power  of  the  Internet  to  address  the  West  Virginia  Instrucdonal  Goals  and  Objecdves  and  improve  student 
achievement  and  learning,  A Criteria  for  Excellence  was  created,  then  employed  by  a group  of  pilot  teachers  to 
develop  lesson  plans  that  would  be  peer  reviewed,  validated  by  field  testing,  observed  during  classroom 
implementadon,  and  repeatedly  revised.  The  resuldng  lesson  plans  have  resulted  in  significant  learning  improvement 
and  have  been  placed  in  the  Best  Pracdces  database  and  shared  with  aU  teachers  in  West  Virginia,  Instrucdonal 
plans  have  been  developed  for  K-12  classrooms  in  the  areas  of  math,  language  arts,  social  studies  and  science  by 
teams  of  teachers,  pre-service  teachers  and  college  instructors.  On  line  professional  development  facilitates 
implementadon  of  the  instrucdonal-coUaboradve  environment. 

Redesigning  An  Individualized  Paper-Based  Course 

Stephane  Lavoie,  SOFAD,  Canada;  Jo-Ann  Stanton,  SOFAD,  Canada 

Web-based  courses  that  are  an  adaptadon  of  a paper-based  version  rarely  take  full  advantage  of  the  Internet  and 
related  technologies.  Taking  this  into  account,  we  developed  a successful  interacdve  web  version  of  a French 
grammar  course.  This  poster  session  will  show  how  "Du  fran9ais  sans  fautes"  was  reengineered  to  avoid  the  pitfaUs 
of  shovelware  using  an  Oracle  database  created  with  Visual  Basic  applicadons.  We  will  offer  an  insight  into  the  most 
challenging  aspect  of  the  process:  the  development  of  interacdve  exercices.  We  will  focus  on  how  we  worked 
around  technical  constraints  in  order  to  maintain  our  pedagogical  goals.  More  than  1500  students  have  enrolled  in 
the  course  since  it  went  online  in  September  1998.  You  may  access  "Du  fran^ais  sans  fautes"  at 
http://www.dfsf  com. 

Team-paced  versus  Self-paced:  The  Effects  of  Educational  Game  Design  on 

Collaboration,  Learning  and  Attitude  towards  Information  Technology 

Edith  Law,  University  of  British  Columbia,  Canada;  Maria  Klawe,  University  of  British  Columbia,  Canada;  Cristina 

Conati,  University  of  British  Columbia,  Canada;  John  Meech,  National  Research  Council,  Canada 

Avalanche  is  a muld-player  game  where  players  must  cooperate  to  achieve  a common  goal.  Using  Avalanche,  a pilot 
study  was  conducted  to  invesdgate  (a)  the  effects  of  cooperadve  (team-paced)  and  independent  (self-paced)  learning 
on  communication  patterns,  performance,  learning  and  attitude  of  the  players,  and  (b)  whether  there  exists  any 
gender  differences  in  how  boys  and  girls  interact  with  the  computer  and  their  teammates  in  a cooperative  gaming 
environment.  16  elementary  school  children,  in  same-gender  groups  of  four,  played  Avalanche  for  two-hour 


sessions.  Results  surest  that  although  team-paced  learning  foster  a positive,  cooperative  team  dynamics,  self-paced 
learning  has  a more  significant  effect  on  learning  itself.  In  addition,  boys  and  girls  react  to  the  two  versions  of  the 
game  differendy.  In  particular,  the  self-paced  female  group  demonstrates  more  cooperative  and  help  seeking/giving 
behavior  than  the  team-paced  female  group,  while  the  reverse  was  observed  among  the  boys. 

Assured  Quality  Strategies  in  Web-based  Accredited  Programs 

Judy  Lee,  University  of  Central  Florida,  USA 

Distance  learning  poses  a challenge  to  voluntary  accreditation.  Quality  assurance  in  emerging  technology  mediated 
distance  learning  is  a central  issue  for  accredited  programs.  The  Council  for  Higher  Education  Accreditation  report 
(CHEA,  1998)  defines  "quality  assurance"  as  the  means  by  which  the  institutions  or  providers  set  their  accredited 
programs  goals  and  measure  results  against  those  goals  in  distance  learning.  Regional  and  specialized  accrediting 
organizations  are  currently  engaged  in  assuring  quality  in  distance  learning  as  part  of  their  ongoing  review  of 
institutions  and  programs.  "Distance  learning  is  seen  by  many  as  transformadve  vehicle  for  increasing  the  pace  of 
change  and  reform  in  higher  education.  For  these  and  other  reasons,  analysis  of  quality  assurance  is  an  essential 
topic  for  national,  state  and  institutional  policy  development  (CHEA,  1998)."  Areas  of  interest  regarding  Quality 
Assurance  Strategies  include  course  and  program  concerns  and  needs,  faculty  and  student  concerns  and  needs, 
technical  needs,  and  administradve  support. 

Reference 

Council  for  Higher  Educadon  Accreditadon  (1998,  April)  . Assuring  quality  in  distance  learning  . Washington,  DC: 
The  Insdtute  for  Higher  Educadon  Policy 

Training  Students  to  Be  Self-Learners  in  Educational  Hypermedia  and  Technology 
Courses 

Amy  S.  C.  Leh,  California  State  University  San  Bernardino,  USA; 

Modern  technology  is  changing  our  educadon  paradigm.  Due  to  technology  advancement,  instructors  can  no  longer 
play  the  role  of  "informadon-giver",  rather  have  to  become  a facilitator.  Facing  tremendous  amount  of  technology 
informadon,  students  in  Educadonal  Technology  need  to  feel  comfortable  of  learning  technologies  by  themselves. 
The  presenter  employed  a variety  of  instrucdonal  strategies  in  her  graduate  courses  to  train  her  students  to  be 
technology  self-learners.  In  the  courses,  her  students  searched  for  Internet  tools  useful  for  instrucdon,  selected 
computer  applicadons  that  they  would  like  to  study,  learned  the  selected  tools  in  a team,  and  trained  other  class 
members  the  use  of  the  tools  they  learned.  In  this  poster  session,  the  author  will  present  teaching  strategies, 
instrucdonal  acdvides,  and  course  assignments  that  are  related  to  the  training.  She  will  also  report  students’  feedback 
on  this  kind  of  training. 

Effective  Use  of  Client-Server  Software  and  Teaching  Strategies  in  Online  Courses 

Amy  S.  C.  Leh,  California  State  University  San  Bernardino,  USA;  Laura  Howzell  Young,  California  State  University 
San  Bernardino,  USA 

In  the  field  of  education,  many  professors  are  incorporating  online  teaching  into  traditional  courses.  What  is  the 
impact?  The  authors  received  a university  internal  grant  to  convert  two  education  courses — an  instructional 
technology  course  and  a course  in  adolescent  development — to  include  online  components.  They  report  the 
benefits  and  challenges  of  using  WebCT  (course  tools)  with  a variety  of  teaching  methodologies.  The  poster  session 
reflects  results  of  a pre-  and  post-  sur\^ey  used  in  both  courses  to  determine  teacher  attitudes  toward  online 
coursework.  Students  in  both  courses  appreciated  the  flexible  schedule  of  online  work  and  time  saved  by  not  having 
to  commute.  The  majority  of  teachers  in  both  classes  had  not  previously  taken  online  courses;  many  liked  the 
threaded  group  discussions  and  online  testing,  but  a few  who  did  not  believe  at  the  outset  that  on-line  teaching  is 
beneficial  continued  to  reject  it.  For  most  of  the  participants,  technology  improved  learning  and  achievement. 

Stepnet:  quality  newspapers  and  web-based  learning  in  the  Netherlands 

Jan  Lepeltak,  Meulenhoff  Educatief,  The  Netherlands 

Stepnet  is  a web-based  newspaper-project  in  the  Netherlands.  Due  to  some  educational  reforms  students  in  the 
upper  grades  of  the  Dutch  secondary  schools  are  to  study  partly  individually,  which  implies  that  teachers  are  obliged 
to  provide  them  with  proper  assignments.  The  results  of  these  are  part  of  the  students’  final  examination.  The  use 
of  ICT  should  also  be  part  of  their  activities  and  a relation  with  daily  (world-)  news-events  is  strongly 
recommended.  In  order  to  facilitate  the  above  aproach  for  teachers  and  students,  PCM,  a Dutch  publishing 
company  consisting  of  newspaper  and  educational  publishing  houses,  developed  a web-site.  This  site  contains  a D- 
base  filled  with  assignments  related  to  the  contents  of  two  daily  national  quality  newspapers.  Every  three  weeks 
during  the  annual  schoolterm  a special  magazine  is  published.  Schools  will  pay  a fee  and  will  receive  a licence  in 
return,  which  entitles  all  participating  students  and  teachers  to  receive  a copy  of  this  magazine  called  the  “@krant”, 
as  well  as  access  to  the  assignment  D-base  and  the  @krant  D-base  containing  the  newspaper  articles.  In  addition, 
teachers  will  receive  a half  a day  on-the-job  training  about  working  with  Stepnet. 


Custom-Authored  Hypermedia  Exercises  for  Use  in  College  Classrooms 

Amy  Lobben,  Central  Michigan  University,  USA 

Continuously  improving  technology  now  allows  educators  new  opportunities  to  move  beyond  traditional  lecture 
style  methods  to  create  custom-authored  multimedia,  hypermedia,  and  animation  presentations  for  use  in  classroom 
demonstrations  or  as  computer-run  student  exercises.  The  development  of  new  software  programs  and  faster,  more 
powerful  computers,  is  beginning  to  change  the  creation  process  of  dynamic  presentations  (multimedia, 
hypermedia,  and  animation).  The  once  time  consuming,  programming  intensive,  and  complicated  methods  of 
authoring  such  presentations  have  become  manageable  tasks  through  the  use  of  improving  computer  programs, 
many  of  which  are  easy  to  learn  and  use.  A series  of  computer-administered  lessons  have  been  created  using 
Macromedia  Director  and  Freehand.  Each  interactive  lesson  is  designed  to  explain  a specific  topic  and  includes 
animation,  graphics,  photographs,  quizzes,  while  maintaining  stringent  design  guidelines.  Applications  such  as  these 
could  have  profound  effects  in  large  college  classrooms,  where  traditional  lecture-style  approaches  limit  the 
individual,  active  learning  by  students. 

Groupware  Tools:  Web  File  System  Allows  Students  to  Access  Information  Anywhere, 
Anytime 

Tyrone  Lobo,  SiteScape,  Inc.,  Canada 

This  poster/demo  is  based  on  SiteScape's  Web  File  System  and  will  explore  how  the  Web  File  System  allows  users 
to  securely  store  and  manage  large  volumes  of  information  entirely  over  the  Web,  giving  students  the  freedom  to 
access  information  from  any  industry-standard  web  browser. 

A Web-based  Hypermedia  Pedagogicai  Course 

Alla  N.  Makarova,  St-Petersburg  State  Pedagogical  Univeristy,  Russia 

The  success  of  any  hypermedia  distance-learning  course  strongly  depends  on  its  psychological  and  pedagogical 
aspects.  A study  was  made  of  the  practical  realization  of  some  common  pedagogical/psychological  criteria  in  a 
Web-based  hypermedia  course.  As  a result  of  this  study  a template  was  developed  which  includes  these  criteria.  This 
template  may  be  included  as  a building  block  for  many  different  hypermedia  courses  devoted  to  a particular  subject. 
Some  open  problems  with  the  template  are  discussed  and  analyzed. 

Info  Pursuits 

Kerrie  Manning,  Leichhardt  High  School,  Australia;  Rosemary  Ward,  UTS,  Australia;  Tony  Ward,  Award  Consulting 
Enterprises,  Australia 

Info  Pursuits  is  an  interactive  CD-ROM  game  designed  to  teach  and  practice  Library  Information  Skills.  Students 
will  answer  Trivial  Pursuits  style  questions  and  will  be  offered  Information/Research  Skills  hints  and  suggestions  as 
part  of  the  feedback.  These  will  include  such  things  as  techniques  for  finding  resources,  using  encyclopaedias, 
dictionaries,  non-print  resources,  Internet,  statistical  resources,  telephone  books,  indexes,  directories,  interviews  etc. 
Positive  experience  in  using  these  resources  may  stimulate  students  to  use  them  again  in  a different  research  task. 
The  same  skills  are  practiced  many  times  with  different  resources  and  in  different  contexts,  to  facilitate  transfer.  The 
questions  will  be  arranged  in  teaching  area  groups,  to  be  used  by  teachers  in  ordinary  lessons  within  the  standard 
curriculum,  or  as  extension  work  within  standard  curriculum  areas.  Many  questions  will  practise  general  skills,  but 
each  curriculum  area  will  provide  experience  with  subject-specific  resources. 

Content  Area  Integration:  A Step  Toward  Emiratisation 

Reo  H.  McBride,  Dubai  Women's  College,  United  Arab  Emirates;  Michael  Ford,  Dubai  Women's  College,  United  Arab 
Emirates;  David  Thomson,  Dubai  Women's  College,  United  Arab  Emirates;  Raymond  Yarsley,  Dubai  Women's 
College,  United  Arab  Emirates;  Laila  Hawker,  Dubai  Women's  College,  United  Arab  Emirates 

The  Higher  Colleges  of  Technology  (HCT)  is  dedicated  to  the  delivery  of  technical  and  professional  programs  of 
the  highest  quality  to  the  citizens  of  the  United  Arab  Emirates  (U.A.E.).  This  is  the  goal  of  “Emiratisation”:  an 
affirmative  action  policy  of  employing  nationals  in  larger  numbers  in  the  workforce,  which  is  currently  dominated 
by  ex-patriot  workers.  TLe  authors  of  this  article  state  that  the  most  effective  way  to  better  prepare  our  students  in 
fulfilling  their  roles  in  the  task  of  emiratisation,  is  to  integrate  the  skills  taught  in  English,  Math  and  Basic 
Computing  in  the  classroom,  and  NOT  teach  them  as  completely  separate  entities.  In  essence,  integrating  the  skills 
taught  in  English,  Math  and  Basic  Computing  develops  in  students  greater  linguistic  abilities,  technical  skills, 
intellectual  capacities,  and  leadership  potential.  This  study  describes  efforts  made  toward  accomplishing  such  a 
daunting  yet  absolutely  necessary  task  for  college  students  in  the  U.A.E. 

High  Tech  Classrooms-Going  The  Distance  with  Distance  Education 

Patricia  McNames,  Indiana  University  Southeast,  USA 

The  rapidly  evolving  paradigm  of  interactive  distance  education  is  changing  the  learning  environment  on  and  off 
campuses  and  schools  nationwide.  Educators  are  suddenly  surrounded  by  not  only  the  opportunities  of  distance 
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education,  but  also  by  its  demands.  Thus,  when  educators  begin  using  these  boundary-spanning  technologies  to 
enhance  the  learning  process,  it  does  dramatically  require  teachers  and  students  to  change  what  they  do.  The 
purpose  of  this  paper  and  poster/demo  is  to  serve  as  a resource  for  educators  who  may  be  considering 
incorporating  distance  education  approaches  into  their  courses.  In  cyberspace,  time  and  place  are  fourth 
dimensional.  The  four  dimensions  that  will  be  discussed  in  this  presentation  are  1)  defining  the  appropriate  distance 
education  system,  2)  designing  the  interactive  course,  3)  developing  a virtual  learning  community,  and  4)  dealing 
with  technical  difficulties. 

An  on-line  introduction  to  quantitative  methods 

Moira  McPherson,  Lakehead  University,  Canada;  William  Montelpare,  Lakehead  University,  Canada 

This  poster/demo  describes  the  development,  implementation,  and  evaluation  of  an  "on-line”  introduction  to  an 
undergraduate  course  in  quantitative  methods.  The  virtual  curriculum  modules  developed  for  the  course  provided  a 
forum  in  which  information,  knowledge,  expertise,  and  questions  could  be  explored  and  updated  throughout  the 
course.  Working  asynchronously,  the  user  completed  six  assignments.  Questions  which  arose  during  the  user's 
session  were  passed  to  "experts"  that  provided  feedback,  electronically.  The  "question  - feedback"  loop  was  a 
dynamic  component  of  the  organizational  structure  and  used  different  modes  of  communication,  including  e-mail, 
electronic  conference  rooms,  bulletin  boards,  and  on-line  help  pages.  The  assignments  were  submitted  electronically 
to  teaching  assistants  who  evaluated  the  submissions  and  provided  feedback  directly  to  the  users.  The  inclusion  of 
direct  feedback  communication  between  experts  and  users  reduced  the  impersonal  characteristics  that  might  arise 
within  an  asynchronous  learning-environment. 

Principles  Of  Cognitive  Evaluation  For  An  Educational  CD-ROM  For  History 

Christina  Metaxaki-Kossionides,  University  of  Thrace,  Greece;  Elevtheria  Gonida,  University  of  Thrace,  Greece; 
Stavroula  Lialiou,  University  of  Athens,  Greece;  Georgios  Kouroupetroglou,  University  of  Athens,  Greece 

The  evaluation  of  CD-ROMs  and  multimedia  products  used  in  educational  environments  ,is  a major  topic.  One  of 
the  erasing  problems  is  the  formation  of  a set  of  pedagogical  principles  or  concepts  and  their  technological 
implementation.  We  present  the  formation  of  a set  of  cognitive  principles.  Those  are  included  in  the  set  of 
eviuation  , concerning  a CD-ROM  developed  by  us.  The  topic  was  a history  lesson  about  the  Homer  period.  We 
have  selected  two  types  of  principles  which  , together  with  the  ones  for  software  development,  should  form  the  set 
for  the  formative  evaluation.  The  one  sub  - set  included  the  pedagogical  requirements  for  the  lesson  of  history.  The 
other  sub  - set  included  the  principles  delivered  by  the  cognitive  scientists.  These  last  ones  were  developed  for  the 
specific  topic.  They  mainly  refer  to  the  nature  and  processes  of  learning  and  instruction.  The  set  of  the  cognitive 
requirements  were  added  explicitly  as  fields  for  a database  for  software  evaluation. 

Authoring  Multimedia,  Designing  Animations  for  Physics  Education 

Donald  J.  Metz,  University  of  Winnipeg,  Canada 

Computer  animations  used  in  high  school  physics  instruction  are  demonstrated.  The  programs  are  written  using 
Asymetrix  Multimedia  Toolbook  and  are  designed  to  address  the  specific  needs  of  the  students  and  their  learning 
environment. 

Dynamic  Communication  Layer  Between  Virtual  Laboratory  and  Intelligent  Agents 

Kaufmann  Meudja,  UniversitT  de  Sherbrooke,  Canada;  Roger  Nkambou,  UniversitT  de  Sherbrooke,  Canada 

In  the  last  few  Years,  virtual  laboratory  has  become  an  important  part  in  online  courses.  Some  of  those  virtual  labs 
involve  intelligent  behaviour.  However,  this  intelligent  behaviour  is  usually  hard-coded  in  the  virtual  lab  and 
therefore,  limit  the  possibility  of  using  reusable  intelligent  agents  that  are  completely  separate  from  the  virtual  lab 
and  which  can  control  most  users  actions  in  the  virtual  laboratory  environment.  We  propose  a way  to  separate 
intelligent  behaviour  from  the  virtual  lab  by  the  implementation  of  a communication  layer  between  the  virtual 
laboratory  and  intelligent  learning  agents. 

Design,  Delivery,  and  Evaluation  of  Online  Courses:  A Primer 

Jerold  Miller,  United  States  International  University,  USA 

Online  learning  environments  can  be  highly  interactive  and  student-centered.  Incorporating  the  contributions  and 
experiences  of  each  member  of  a learning  community  into  the  coursework  encourages  students  to  contribute  more 
of  themselves  to  the  course  and  allows  them  to  learn  more  from  their  teachers  and  peers.  Creation  of  functional 
online  learning  environments  is  contingent  upon  the  effective  design,  delivery,  and  evaluation  of  online  courses. 
This  session  will  address  the  design  of  online  courses  and  adaptation  of  traditional  courses  to  the  online  delivery 
mode.  Participants  will  understand  appropriate  online  delivery  methods,  explore  teacher-centered  vs.  student- 
centered  learning,  understand  effective  strategies  for  communication  online,  know  how  to  adapt  content  and 
curriculum  for  online  teaching,  discover  new  assessment  strategies  for  distance  learning,  and  learn  how  to  create  an 
online  community  of  learners. 


Discovering  xyAlgebra;  Intelligent  Interactive  Internet  Instruction 

John  Miller,  The  City  College  of  CUNY,  USA 

Abstract:  Passive  activities  such  as  watching  presentations,  listening  to  explanations  of  general  principles  and 
watching  experts  solve  sample  problems  are  helpful,  but  peripheral,  to  the  mathematical  learning  process.  For 
students  the  indispensable  step  is  solving  problems  for  themselves.  Yet  most  commercial  mathematics  software  still 
concentrates  on  presentations  and  sample  problems,  while  sending  students  off  line  to  do  practice  problems  on 
paper  without  interactive  support.  Answers  are  either  multiple  choice  or  limited  to  a single  simplified  final  step. 
Early  Internet  courses  are  even  less  interactive.  In  contrast,  students  using  xyAlgebra  can  enter  each  step  of  each 
problem  solution.  They  enjoy  intelligent  support  at  every  step  as  xyAlgebra^s  suggested  solution  strate^  changes  in 
response  to  their  steps  in  simplifying  expressions,  solving  equations  and  even  in  setting  up  and  solving  verbal 
problems.  The  next  version  of  xyAlgebra  will  support  instruction  over  the  Internet,  yet  the  entire  package  can  be 
downloaded  without  cost  at  mathO.sci.ccny.cuny.edu/xyalgebra. 

Problem-based  Learning  and  Flash  4.0:  An  Experiment  in  Science  Education 

Leslie  Miller,  Rice  University,  USA;  Janice  Mayes,  Rice  University,  USA;  Donna  Smith,  Rice  University,  USA 

How  does  one  design  learning  resources  that  will  reach  large  audiences,  capture  the  imagination,  and  compete  with 
the  game  environments  that  are  popular  among  of  middle  school  students?  The  Reconstructors  is  a new  episodic 
adventure  series,  designed  for  the  Web,  with  a substantive  educational  message.  See  http:/ / reconstructors.rice.edu 
where  a student  enters  a futuristic  world  in  which  he  or  she  assumes  the  role  of  a "reconstructor,"  In  the  interactive 
mystery  format,  a student  learns  science  and  history  relevant  to  the  discovery  and  use  of  opiates.  Over  the  course  of 
four  episodes  or  missions,  students  "solve  the  problem."  The  concepts  of  neurotransmission,  the  neurobiology 
underlying  drug  addiction,  drug  tolerance,  and  analgesia,  as  well  as  the  history  of  opium  use  are  presented, 

On-Line  Education  Using  Video  Broadcasting  Delivered  from  Perth  Campus,  Algonquin 

College 

Maike  Luiken  Miller,  Algonquin  College,  Canada;  Dave  Osborne,  Algonquin  College,  Canada;  Ian  McCormick, 

Algonquin  College,  Canada 

As  part  of  the  LCN  (Lanark  Communications  Network)  TAP  project,  Algonquin  College  at  Perth  Campus  engaged 
in  a video  broadcast/conferencing  pilot  to  deliver  a course,  "The  History  and  Philosophy  of  Architectural 
Conservation"  (ARC9001),  simultaneously  in  2 modes:  lectures  with  electronic  slides  and  traditional  black  board 
work  and,  in  parallel,  video  broadcasts  (IN@SEC  VP  Broadcaster)  with  e-mail  feedback.  This  pilot  demonstrated 
successfully  that  on-line  co-delivery  allows  to  teach  students  simultaneously  at  different  locations  employing  video, 
audio  and  text  via  a network  using  relatively  inexpensive  broadcasting/conferencing  software  with  minimal  set-up 
requirements  at  the  clients'  site.  Course  delivery  improved.  Students  enjoyed  the  choice  of  delivery  modes  to  suit 
their  learning  styles.  Professor  appreciated  more  time  for  interacting  with  the  students.  Re-broadcasting  at  a later 
time  possible.  The  delivery  model  - independent  of  the  specific  broadcasting  tool  - provides  a means  to  reach 
remote  connected  students  everywhere  - be  it  rural  or  elsewhere  around  the  world. 

TLC  : Teachers’  And  Learners’  Collaborascope  - A Platform  For  Analysing  And 

Evaluating  Online  Education 

Michelle  Montgomery  Masters,  University  of  Glasgow,  UK;  Stewart  Macneill,  University  of  Glasgow,  UK;  Prof 

Malcolm  Atkinson,  University  of  Glasgow,  UK 

Millions  is  being  spent  on  installing  networks,  computers  and  assembling  distance  learning  material  and  ICT 
(information  and  communications  technology)  software.  Yet  the  network  technology  is  not  being  used  to  assemble 
databases  which  researchers,  managers  and  politicians  can  "data  mine"  to  guide  their  investment.  In  the  TLC  project 
we  are  pioneering  techniques  for  assembling  and  querying  educational  data.  Our  goal  is  to  initiate  research  into  the 
best  methods  of  rapidly  assessing  the  effectiveness  of  resources  deployed  and  then  interpreting  the  data  to  improve 
our  educational  performance.  We  are  not  trying  to  reinvent  the  wheel  and  produce  yet  another  online  learning 
environment  like  WebCT  or  Virtual-U.  Instead  we  are  budding  tools  which  will  work  in  collaboration  with  these 
existing  learning  environments  and  provide  teachers  and  students  with  valuable  information  about  the  usage  of  their 
learning  environment.  The  TLC  project  aims  to  utilise  the  emerging  standards  such  as  those  from  the  IMS  project. 

The  Third  Dimension  For  World  Of  Knowledge:  An  Acceptable  Way  To  Get  In 

Mikhail  Morozov,  Mari  State  Thechnical  University,  Russia;  Aleksander  Markov,  Mari  State  Thechnical  University, 

Russia 

Educational  CD-ROMs  should  be  as  interesting  and  attractive  for  children  as  computer  games.  The  approach 
offered  by  the  authors  allows  to  reduce  the  existing  gap  between  multimedia  for  education  and  for  entertainment.  It 
is  achieved  due  to  the  unusual  design  of  3D  interface  and  by  using  a special  technique  of  development.  As  a general 
metaphor  for  organisation  of  multimedia  environment  the  metaphor  of  theatre  with  aU  appropriate  attributes  was 
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chosen:  the  stage  with  visual  educational  content,  side-  scenes,  a curtain,  etc.  For  presentation  of  educational 
material  in  addition  to  traditional  multimedia  components,  the  new  integrated  forms  are  used.  Because  of 
complexity  and  richness  of  information  of  the  CD-ROM  model,  its  realization  is  based  on  graph  of  program 
transitions  and  application  of  script  paradigm,  which  is  accepted  for  the  description  of  structure  of  the  educational 
and  navigating  information.  These  solutions  can  be  recommended  for  creation  of  a new  generation  of  "three- 
dimensional”  educational  multimedia  CD-ROMs, 

Using  a Web  Site  to  Increase  Student  Motivation  to  Write  and  Enhance  the 
ReadingAA^riting  Relationship 

Susan  Nelson,  Elm  Street  Elementary  School,  USA;  Harrison  Yang,  State  University  of  New  York  at  Oswego,  USA 

This  action  research  project  is  an  example  of  how  Internet  technology  is  being  used  in  an  elementary  classroom  as  a 
means  to  increase  student  achievement.  The  project  explores  the  relationship  between  writing  and  reading  and  its 
affects  on  early  literacy  development.  It  also  explores  the  concept  of  motivating  students  to  write  by  publishing 
student  work  for  real  life  audiences.  The  project  uses  the  current  technology  of  an  Internet  web  site  as  a source  for 
publishing  student  writing.  It  explores  the  effects  that  this  type  of  publication  has  on  student  attitude,  motivation  to 
write,  and  literacy  development. 

TELSI,  Web-Based  Software  For  Collaborative  Learning 

Graciela  Nielsen,  Norwegian  State  Center  for  Adult  Education,  Norway;  Boo  Hever,  Majomas  Vuxengymnasium, 
Sweden;  Aase  Steinmetz,  Undervisningsministerium,  Denmark 

The  TELSI  software  has  been  developed  with  the  support  of  the  European  Union  under  the  SIMULAB-project. 
SIMULAB  is  a concept  for  Web-based  collaborative  tasks.  TELSI  has  been  created  to  cater  for  the  needs  of  this 
type  of  activity,  and  is  a flexible  and  user-friendly  environment. 

Deveiopment  of  Speech  input  System  for  Web-based  Courseware 

Ryuichi  Nisimura,  Nara  Institute  of  Science  and  Technology , Japan;  Shoji  Kajita,  Nagoya  University,  Japan;  Kazuya 
Takeda,  Nagoya  University,  Japan;  Fumitada  Itakura,  Nagoya  University,  Japan 

We  proposes  WebSPEAC(Web  SPEech  Acquisition  for  Courseware)  System  that  provides  a speech-input  function 
for  Web-based  online  education  system.  WebSPEAC  has  the  following  three  features;  (1)  easy  to  maintain  the 
system  by  the  use  of  a simple  speech  input  program  in  Web  browsers,  while  the  other  speech  processing  are 
performed  on  the  Web  server,  (2)  a natural  interactivity  between  Web  server  and  users  by  using  the  server  push 
techonology,  (3)  a simple  installation  and  no  needs  of  external  programs  on  client  machines.  To  show  the 
effectiveness,  an  Web-based  speech  analysis  system,  a speech  recognition  system  and  a student  identification  system 
are  developed  as  the  examples  of  WebSPEAC  System.  By  using  the  speech  analysis  system,  it  is  clarified  that 
WebSPEAC  System  can  significantly  reduce  task  steps,  error  steps  and  elasped  time  for  speech-input  task, 
compared  with  the  conventional  method  (file  upload). 

The  Instructional  Portal  Project 

Kevin  Oliver,  Virginia  Tech,  USA 

The  Instructional  Portal  is  a clearinghouse  of  information,  resources,  tools,  and  services  at  Virginia  Tech,  designed 
to  facilitate  practical  instructional  design  among  faculty.  Faculty  can  increasingly  adopt  and  easily  utilize  educational 
technologies  in  their  courses.  While  some  faculty  use  technology  to  modify  and  improve  teaching  and  learning 
practices,  too  many  rely  on  tools  to  simply  make  their  existing  teaching  practices  more  efficient.  The  portal  is 
designed  to  help  faculty  integrate  technology  more  effectively  by  applying  three  instructional  design  components. 
The  analysis  component  outlines  strategies  and  instruments  to  account  for  environmental  and  learner 
characteristics,  and  provides  information  about  structuring  course  content.  The  strategy  component  describes 
instructional  methods  and  models  that  utilize  technology  effectively,  and  provides  teaching  modules  covering 
popular  web-based  instruction  and  interface  design  issues.  The  evaluation  component  describes  evaluation  methods 
and  models  and  provides  tools  to  help  faculty  revise  and  improve  their  technology  innovations.  Visit  the  portal  at: 
http://www,edtech.vt.edu/edtech/portal/index.html 

Integrating  and  Adapting  Multimedia  Resources 

Mary  Panko,  UNITEC  Institute  of  Technology,  New  Zealand 

In  UNiTEC  students  enrolled  on  an  Automotive  Trades  course  have  access  to  a series  of  32  CDX  disks,  entitled 
The  Automotive  Trainee  Series  that  provides  them  with  a learning  resource  in  addition  to  their  basic  theoretical  and 
practical  teaching.  This  study  attempted  to: 

• discover  what  beneficial  learning  effects  the  students  perceived  as  being  currently  provided  by  the 
multimedia  component; 

• examine  the  extent  to  which  the  tutors  integrated  and  adapted  the  multimedia  material  as  part  of  their 
teaching;  and 
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• compare  patterns  of  current  disk  utilization  with  those  of  the  previous  year. 

Results  showed  that  although  both  staff  and  students  thought  the  multimedia  material  valuable,  its  overall  utilization 
by  students  had  diminished  over  time.  The  reason  for  this  appears  to  be  the  lack  of  integration  of  the  resource  by 
tutors  into  their  teaching.  Furthermore,  tutors  had  generally  not  adapted  the  material  for  their  own  needs  although 
where  workbooks  had  been  provided,  this  seemed  to  increase  the  students’  satisfaction. 

DISCETECH-BIMBOTECH:  An  Experimental  Project  For  Introducing  Innovative 
Technology  Within  The  Italian  School  System 

Paolo  Paolini,  Politecnico  di  Milano,  Italy;  Pierluigi  Della  Vigna,  Politecnico  di  Milano,  Italy;  Francesca  Collina, 
Politecnico  di  Milano,  Italy;  Paolo  Magatti,  Politecnico  di  Milano,  Italy;  Silvia  Moiola,  Politecnico  di  Milano,  Italy 

Since  1996  Politecnico  di  Milano  has  launched  a program  for  experimenting,  within  the  Italian  school  system,  the 
use  of  advanced  technology  for  teaching  and  learning.  Two  projects  (in  Como  and  Lecce)  have  involved  more  than 
700  teachers  and  more  than  8.000  pupils,  being  the  largest  of  this  kind  in  Italy,  both  in  High  Schools  (Discetech 
project)  and  grammar  schools  (Bimbotech).  The  project  emphasizes  the  role  of  the  teachers,  who  are  trained  on 
multimedia  and  Internet  technology  first.  Secondly  they  are  “guided”  and  assisted  in  shaping  up  specific  activities^ 
with  usage  of  either  multimedia  CD-ROM's  or  Internet  Sites.  A large  collection  of  CD-ROM's  has  been  created  and 
several  hundreds  educational  Websites,  on  different  subjects,  have  been  classified.  The  project  has  been  very 
successful  with  the  teachers  (many  of  them  have  never  used  advanced  technology  before)  and  the  pupils,  the 
learning  and  the  interest  of  which  has  been  evaluated  as  sharply  improved. 

Learning  Experiences  in  New  Learning  Environments 

Heli  Pekkala,  University  of  Jyvaskyla,  Finland 

One  of  the  outcomes  of  a departmental  action  research  project  at  our  language  centre  is  a modernised  Learning 
Centre,  where  students  and  staff  of  the  university  have  access  to  learning  materials  in  over  30  languages.  The 
philosophy  and  pedagogical  framework  underlying  the  operations  of  the  centre  is  to  support  and  encourage 
students  in  developing  their  learning  skills  and  self-directiveness  for  continuous  language  learning  and  for  making 
effective  use  of  new  learning  environments  . This  poster  presents  the  Learning  Centre  (interactive  studios,  writing 
studio,  multimedia  studio  etc.)  and  reports  student  experiences  of  learning  a new  language  independently.  One  part 
of  this  learner  training  course  Learning  to  Learn  Languages  was  a self-access  project  during  which  students  were 
requested  to  reflect  upon  the  kinds  of  qualities  and  procedures  that  self-directed  language  learning  requires  from  a 
learner.  They  also  evaluated  what  areas  of  language  were  possible  to  access  without  external  structuring  and 
teaching. 

Microsoft  Office  as  a Learning  and  Teaching  Tool 

Craig  Place,  Acadia  University,  Canada;  Michael  Shaw,  Acadia  University,  Canada 

Many  useful  features  in  popular  software  applications  often  go  unnoticed  by  many  users.  Often  thought  of  as  a 
business  application,  Microsoft  Office  is  one  such  suite  of  applications  that  is  often  overlooked  as  a tool  for  use  in 
the  teaching  and  learning  environment  at  most  grades,  and  at  the  university  and  college  level.  With  just  Word,  Excel 
and  PowerPoint,  MS  Office  can  be  used  as  a tool  to  develop  the  basic  understanding  and  expertise  for  a number  of 
other  applications  including  photo-editors,  drawing  and  animation  programs,  multimedia  authoring  and  web- 
development  tools.  Used  as  a stepping  stone  to  more  advanced  applications  in  this  way,  the  interface  of  each  of 
these  3 applications  can  be  simplified  to  be  useful  at  a number  of  different  grade  levels,  or  levels  of  expertise.  Using 
a combination  of  examples  and  hands-on  exercises  the  workshop  will  illustrate  to  the  participants  how  MS  Office 
can  be  used  to  easily  and  effectively  create  multimedia  hyper-documents,  dynamic  learning-objects,  student  review 
exercises  and  interactive  non-linear  presentations.  Specific  examples  will  be  the  use  of  PowerPoint  as  a concept- 
mapping tool  and  as  a multimedia  authoring  and  animation  tool.  Using  Excel  to  assist  in  language  learning  and 
create  dynamic  graphs  for  interactive  what-if  scenarios,  and  using  Word  to  develop  hypermedia  templates  and 
collaborative  writing  documents.  Several  examples  of  the  usefulness  and  creation  of  macros  will  also  be  shown. 

Students'  expectations  and  perceptions  of  efficacy  for  self-regulated  learning  in 
mediated  learning  environments;  Partial  results  of  an  evaluation 

Laura  Helena  Porras,  Universidad  de  las  Americas,  Puebla,  Mexico 

This  paper  discusses  partial  results  of  an  evaluation  applied  to  an  educational  teleconferencing  system  in  a Mexican 
' higher-education  institution.  Courses  offered  through  the  system  are  based  on  a pedagogical  model,  which 
emphasizes  the  active  role  of  the  learner  as  responsible  of  her  own  learning.  Therefore,  perceptions  of  the  self,  and 
self-regulatory  processes  become  key  components  in  this  learning  environment.  Conclusions  here  presented 
correspond  only  to:  (i)  students'  expectancies,  (ii)  perceptions  of  efficacy  for  self- regulated  learning,  and  perceptions 
of  the  course,  and  of  the  teacher  and  learner’s  work.  Results  indicate  students'  expectancies  focus  on  ’’learning  more 
than  in  face-to  face  situations”,  followed  by  "using  technology",  and  "having  freedom  and  initiative”.  Perceptions  of 
efficacy  for  self-regulated  learning  were  high  overall.  Nevertheless,  analyses  of  individual  items  identify  specific  areas 
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of  improvement.  Qualitative  data  on  perceptions  of  the  course,  and  teacher’s  and  learner’s  work  suggest  that  these 
courses  are  more  demanding  than  the  face-to-face  ones  they  are  taking.  Students’  responses  include  attitudinal  and 
cognitive  adjustments  needed  for  these  learning  environments.  These  results  have  led  to  improvements,  which  are 
presently  being  applied. 

An  Online  Model  For  Precalculus 

Hari  Pulapaka,  Valdosta  State  University,  USA;  Denise  Reid,  Valdosta  State  University,  USA 

Some  of  the  greatest  challenges  to  designing  and  delivering  a completely  online  mathematics  course  are  effective 
and  efficient  uses  of  the  leading  technologies  that  are  available  for  this  purpose.  The  authors  have  just  completed 
designing  the  first  version  of  a precalculus  course  that  is  being  delivered  through  a combination  of  synchronous  and 
asynchronous  environments.  Precalculus  was  chosen  over  other  mathematics  courses  for  several  reasons.  The 
course  has  been  planned  as  a model  online  course  for  any  mathematics  course  (especially  at  the  early  levels).  Future 
versions  of  this  course  and  related  courses  will  undoubtedly  incorporate  additional  technologies  and  components  in 
ail  effort  to  improve  student  learning,  interactivity,  and  end-user  feasibility.  Future  plans  for  the  course  include 
development  and  refeinement  of  the  unfinished  sections  of  the  course  and  tighter  integration  with  graphing 
calculator  and/or  mathematical  software. 

IMAGUS:  An  Environment  To  Design  Video  Applications  For  Digital  Libraries 

Andre  Luis  Alice  Raabe,  Pontificia  Universidade  Catolica  do  Rio  Grande  do  Sul,  Brasil;  Lucia  Maria  Martins  Giraffa, 
Pontificia  Universidade  Catolica  do  Rio  Grande  do  Sul,  Brasil 

This  poster  presents  the  description  of  the  INIAGUS  environment.  IMAGUS  has  tools  to  built  video  applications 
supported  by  Internet.  This  environment  has  being  developed  in  order  to  contribute  for  the  construction  of  the 
PUCRS  digital  library  collection. 

Art,  Poetry,  and  Instructional  Media 

Thomas  Reinartz,  Rosemount  High  School,  USA;  Brad  Hokanson,  University  of  Minnesota,  USA 

In  many  high  school  English  courses,  students  are  trained  to  seek  out  proper  form  and  work  toward  clarity  in  their 
writing  assignments.  They  are  taught  to  write  using  complete  sentences  and  complete  thoughts,  fix  their 
punctuation  and  grammar,  and  strive  for  polish  in  all  written  work.  That  is  what  English  is  aU  about.  On  one  level, 
we  can  be  pleased  with  their  concern.  Plowever,  this  quest  for  excellence  and  perfection  seems  to  interfere  with  the 
poetry  writing  process.  Many  successful  poems  lack  complete  sentences,  defy  traditional  uses  of  punctuation,  and 
contain  word  combinations  that  require  readers  to  make  inferential  connections,  often  sacrificing  the  writing  clarity 
that  English  teachers  work  so  hard  teach.  We  have  taught  them  to  be  concise,  clear,  and  correct  in  all  writing 
endeavors,  which  in  most  cases,  is  sound  advice.  Writing  poetry,  however,  demands  skills  that  require  us  to  play 
with  words  and  dwell  in  areas  where  word  juxtapositions  and  combinations  are  limited  only  by  one's  imagination. 
The  many  choices  in  this  "zone"  of  possibility  are  frighteningly  endless,  so  it  is  no  wonder  students  resist  entering. 
They  may  get  lost.  To  help  foster  an  awareness  about  word  choice  and  imagery,  we  developed  an  interactive 
program  to  generate  "found"  poetry  with  student  input  and  used  it  in  a twelve  week  American  literature  class  taught 
by  T.  J.  Reinartz,  one  of  the  authors.  The  program  evolved  from  discussion  between  the  authors  regarding  the  use 
of  computers  to  inspire  poetry  writing.  A multi  modal  qualitative  research  study  was  undertaken  involving 
classroom  observation,  video  taping,  survey  questions  and  analysis  of  poetry  segments.  The  findings  of  that  research 
and  analysis  will  be  presented,  with  broader  observations  of  the  use  of  computers  'in  the  service'  of  poetry,  and  their 
appropriate  use  in  the  high  school  classroom. 

Students  as  Producers:  Changing  the  Way  we  Teach  and  Learn 

Gail  Ring,  University  of  Florida,  USA;  Sebastian  Foti,  University  of  Florida,  USA;  Melissa  McCallister,  University  of 
Florida,  USA;  Tamara  Pearson,  University  of  Florida,  USA;  Ebraheem  Alkazeemi,  University  of  Florida,  USA 

Educational  technology  courses  at  the  University  of  Florida  are  modeling  the  concept  of  students  as  producers. 
Through  student-based  projects  such  as  an  online  support  center,  educational  software,  student  portfolios,  and 
documentary-type  CD-roms  our  students  are  producing  educational  products.  These  ‘real’  activities  are  improving 
our  courses  as  students  learn  by  doing,  thereby  creating  a constructivist  learning  environment.  Relevant,  contextual 
projects  such  as  these  provide  students  with  an  increased  sense  of  empowerment  and  ownership.  In  our 
presentation  we  will  share  how  the  traditional  roles  of  instructor  and  learner  may  no  longer  appropriate. 

The  GPB  project;  A successful  PHD  seminar  using  Videoconference  in  real  time  and  a 
list  to  communicatebetween  the  classes 

Claire  Roberge,  University  of  Quebec  in  Montreal,  Canada 

This  winter  2000,  a seminar  was  offered  at  the  PHD  level  in  the  departement  of  Communication  at  the  University 
of  Quebec  in  Montreal.Three  teachers  participated,  one  from  Montreal,  one  from  Paris  and  one  from  Grenoble, 
Seven  sessions  were  offered  in  videoconference  to  all  students  enrolled.  A website  and  a list  were  added  tools 
offered  to  the  students  to  communicate  between  the  sessions.  The  sessions  in  videoconference  were  in  real  time. 


Teachers  and  students  made  presentations  and  periods  of  questions  were  planned  at  the  end  of  each  of  those 
presentarions.The  experience  which  will  be  renewed  next  winter  was  a success.  Technology  was  used  wisely,  the 
teachers  made  sure  the  environment  available  enhance  the  learning  process  of  their  students.The  differences  in 
cultural  habitus  and  in  the  uses  of  technology  were  taken  in  consideration. An  evaluation  was  made  taking  seriously 
all  factors. 

Multimedia  Training  , Virtual  Instrumentation  and  Remote  Laboratory  a New  Approach  to 
the  Electronic  Courseware 

Nicoletta  Sala,  Academy  of  arch.  University  of  Italian  Switzerland,  Switzerland 

This  work  describes  a project  developed  at  the  Department  of  Electronic  Politecnico  of  Torino  in  the  field  of 
multimedia  technologies  in  educational  process.  The  goal  was  to  allow  the  students  to  carry  out  a pre- training 
activity  outside  the  laboratory  and  possibly  at  home;  each  student  could  thus  individually  adequate  the  learning  rate 
to  his  own  capabilities.  After  this  prc- training  phase,  students  who  enter  the  laboratory  require  reduced  assistance 
and  less  time  to  complete  the  training  activity.  This  approach  can  reduce  the  qualified  assistance  that  is  not  easily 
found.  For  this  reason  several  hypermedia  mod-ules  have  been  developed  (using  Multimedia  Toolbook)  to  assist  the 
students  to  acquire  the  fundamentals  of  the  basic  electronic  instrumentation.  The  subjects  developed  are: 

• fundamentals  of  the  analog  and  digital  oscilloscope,  which  includes  practical  exercises; 

• fundamentals  of  the  analog  and  digital  voltmeters  and  their  operating  theory; 

• the  IEEE488  standard  interface  for  programmable  instrumentation; 

• the  logic  analyzer; 

• the  spectrum  analyzer. 

Virtual  instruments  are  implemented  in  order  to  allow  simple  simulations  of  the  real  instruments  during  the  self- 
training phase  A client-server  system  has  been  designed  in  order  to  allow  the  students  to  operate  on  a remote 
laboratory  for  experimental  training.  The  idea  is  to  design  a laboratory  that  is  simultaneously  and  remotely  accessible 
to  several  students,  who  concurrently  share  the  same  instrumentation,  but  not  necessarily  the  same  experiment.  The 
instrumentation  and  the  other  hardware  resources  are  accessible  in  a sort  of  time  sharing  process,  which  is  managed 
by  a server  and  transparent  to  the  user.  The  virtual  laboratory  architecture  is  composed  of  two  kinds  of  subsystems: 
the  measurement  Server  and  the  measurement  clients. 

Strategies  for  Teaching  and  Learning;  Lessons  Gained  from  an  Ethnography  of  Graduate 
Classes  in  an  Interactive  Distance  Learning  Studio 

Lorraine  C.  Schmertzing,  University  of  West  Florida,  USA;  Richard  W.  Schmertzing,  Valdosta  State  University,  USA 

This  poster  session  is  based  on  data  collected  during  a one-year  ethnographic  case  study  of  graduate  education 
classes  during  the  inaugural  year  of  a 2-way  audio /2-way  video  interactive  distance  learning  classroom  at  a regional 
university  in  the  southern  United  States.  Cultural  anthropology  and  symbolic  interactionism  acted  as  guide  to 
uncover  and  describe  the  processes  through  which  adult  learners  transformed  traditional  classroom  culture  as  they 
implicitly  redefined  that  culture  during  their  initial  exposure  to  a specific  distance  learning  environment.  This  poster 
session  addresses  the  often  un-addressed  complexities  inherent  in  re-defining  ones  learning  environment  so  that 
effective  learning  can  occur  and  offers  strategies  for  teaching  and  learning  based  on  these  complexities.  Strategies 
include  ideas  for  the  first  night  of  class,  cultural  awareness  activities,  feedback  forums,  and  rules  of  engagement. 

Use  of  Telecommunications  Technology  in  Radiation  Accident  Simulation 

Domenic  Screnci,  Boston  University  School  of  Medicine,  USA;  Kirsten  Levy,  Boston  University  School  of  Medicine, 
USA;  Erwin  Hirsch,  Boston  University  School  of  Medicine,  USA;  Richard  Aghababian,  University  of  Massachusetts 
Medical  School,  USA;  Tracey  Russo,  Boston  University  School  of  Medicine,  USA 

Radiation  accidents  are  rare  but  increasing  in  frequency.  Partners  in  a regional  project  are  providing  training  on 
radiation  accident  preparedness  in  Eastern  Europe  and  the  New  Independent  States.  Recently,  the  partners  devised 
a radiation  accident  exercise  to  simulate  radiation  exposure  and  test  inter-regional  cooperation  among  eight  nations. 
A variety  of  multimedia,  communication  and  distance  learning  technologies  was  used  to  execute  the  exercise.  This 
poster  describes: 

• Objectives 

• Design 

• Procedures 

• Communication  Plan 

• Educational  Technologies 

The  exercise  demonstrates  a live  response  to  a simulated  radiation  accident;  the  ability  to  exchange  information 
among  peers,  request  resources  and  notify  appropriate  agencies;  and  the  use  of  communication  technologies  to 
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debrief  the  experience.  Also  demonstrated  is  the  use  of  distance  learning  technology  to  cultivate  regional 
harmonization  of  response  in  a field  in  which  preparedness  is  essential  but  practical  exercises  lacking. 

Teaching  data  analysis  using  thin  client  technology  for  remote  access 

Robert  Seidman,  San  Diego  State  University,  USA;  K.  Michael  Peddecord,  San  Diego  State  University,  USA 

This  poster  describes  implementation  and  outcomes  of  using  thin  client  technology  to  provide  remote  access  in  a 
data  analysis  course.  All  software  applications  run  on  the  course  server,  so  no  downloading  of  data  or  software  is 
required.  Results  from  statistical  analyses  are  transmitted  to  students’  client  machines.  Students  were  able  to  launch 
applications  and  complete  assignments  from  any  location  that  had  Internet  access.  In  addition  to  7/24  access, 
students  were  able  to  use  software  they  did  not  own  and  large  data  files  they  did  not  have  on  their  own  computers. 
While  they  generally  expressed  satisfaction  with  this  remote  access  capability,  drawbacks  included  initial 
programming  time  and  cost  of  the  server  software,  lack  of  familiarity  with  operating  in  a networked  environment, 
frustration  with  server-side  disconnects  or  ISP  interruptions,  and  occasional  inability  to  print  at  home  because  the 
server  software  does  not  include  all  old  printer  drivers. 

Multimedia  Guide  to  Fractal  Geometry 

Vladimir  Shlyk,  Belarusian  State  Pedagogical  University,  Belarus 

Fractal  geometry  has  changed  our  view  of  natural  processes  and  geometric  forms.  The  Guide  is  a software  kit 
supporting  a university  course  on  this  science.  It  contains  an  introductory  electronic  textbook,  annotated  list  of 
publications,  favorite  Internet  links  and  provides  facilities  to  run  programs.  One  program  of  the  project  visualizes 
the  mechanism  of  birth  of  classical  and  IFS  fractals.  A student  can  grow  fractals  by  himself  and  explore  their 
features  by  trying  different  generators  and  mappings.  Another  program  demonstrates  connection  between  the 
Mandelbrot  and  Julia  sets.  It  throws  a bridge  from  fractal  geometry  to  nonlinear  dynamics  and  chaos  theory  by 
showing  different  variants  of  behaviour  inherent  in  underlying  quadratic  process.  Having  advantages  over  analogous 
programs,  it  is  already  being  used  at  several  universities.  To  create  the  most  complete  educational  software  on 
fractal  geometry  we  would  be  pleased  to  collaborate  with  other  specialists  and  enthusiasts. 

Project-Based  Learning  + Multimedia:  Adding  Value  to  Students'  Education 

Michael  Simkins,  Silicon  Valley  Network,  USA;  William  R.  Penuel,  SRI  International,  USA 

’’Does  technology  really  make  a difference  in  student  learning?"  Definitely,  according  to  the  Challenge  2000 
Multimedia  Project,  one  of  the  original  Technology  Innovation  Challenge  Grants  funded  by  the  U.S.  Department  of 
Education.  The  Project  has  demonstrated  significant  educational  results  when  technology  investment  includes 
teacher  training,  technical  support  staffing,  and  curriculum  materials  development.  In  a five-year  evaluation  of  the 
$6.2  million  project  in  California's  Silicon  Valley,  SRI  evaluators  found  that  technology-using  students  surpassed 
their  non-technology-using  peers  in  developing  some  of  the  most  critical  skills  for  the  new  economy  of  the  21st 
century.  These  skill  advantages  included  better  communication,  teamwork  and  problem-solving.  Researchers  also 
found  important  positive  changes  in  teachers  classroom  practices  that  support  a constructivist  approach  to  learning. 
In  addition,  these  same  students  equaled  their  non-technology-using  peers  in  learning  the  basic  skills  measured  in 
standardized  tests.  In  short,  students  in  technolog)^-using  classes  not  only  "got  the  basics,"  they  got  more. 

The  Search  for  the  Skunk  Ape:  An  Information  Literacy  WebQuest 

Charlotte  Slater,  The  Walden  Institute,  USA;  Pamela  Sawallis,  Florida  Gulf  Coast  University,  USA 

A web-based  instructional  resource  for  distance  and  on-campus  learners  uses  elements  of  goal  based  scenarios 
(Schank,  1994)  and  WebQuests  (Dodge,  1997)  to  provide  a "fun,"  story-based  environment  for  learning 
information  literacy  skills.  This  project  represents  a parmership  between  instructional  technology  and  library 
services  at  a state  university. 

Online  Learning  in  a 3-D  Robotics  Workshop:  The  NASA  ROVer  Ranch 

Stephanie  Smith,  NASA/LinCom,  USA 

The  ROVer  Ranch  is  an  online  environment  based  on  NASA’s  mission  as  it  relates  to  robotics  engineering. 
Students  are  situated  where  they  learn  about  robots  and  then  build  and  run  simulated  robots  in  a 3-D  virtual  world. 
The  robotics  workshop  has  all  the  components  to  build  a "soft"  version  of  a robot  and  the  tools  needed  to  instruct 
the  robot  to  complete  its  mission.  Student  engineers  or  teams  select  a mission,  review  the  mission  goals  then  design 
a robot  with  the  appropriate  attributes  (power  source,  navigation  planning,  tools,  etc.)  based  on  the  mission 
objectives.  Once  the  robot  is  constructed,  students  plan  a navigation  path  and  then  place  the  robot  in  the  simulation 
to  execute  the  mission.  This  simplified  design  and  programming  exercise  is  an  interactive  method  to  learn  science 
and  math  skills  both  individually  and  in  a team  setting  as  students  share  their  experience  in  journals  and  team 
message  boards. 
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OATS:  Operational  Amplifier  Tutorial  Simulation;  a dynamic  presentation 

Sarah  Sniderman,  Concordia  University,  Canada;  Gary  Boyd,  Concordia  University,  Canada;  Geza  Joos,  Concordia 

University,  Canada 

Abstract:  Operational  Amplifiers  are  among  the  most  important  components  used  in  analog  electronic  devices. 
They  are  complex  and  initially  counter-intuitive  in  behaviour.  Electrical  engineering  and  electrotechnology  students 
frequendy  need  special  tutorial  help  to  understand  them  and  configure  them  to  work  properly  in  various  circuits. 
The  main  advantage  of  simuladon  programs  for  higher  education  in  technical  fields  is  that,  when  carefully  designed, 
they  can  promote  higher  level  instructional  goals  of  prime  meta-cognitive  value,  especially  those  involving  the 
autonomous  active  construction  and  exploration  of  knowledge  at  a pace  appropriate  for  the  learners.  Note, 
however,  that  such  free  exploration  of  knowledge  at  a responsive  pace  is  not  a quality  inherent  in  multimedia 
instruction.  High-level  interactivity  involves  more  than  guided  clicking  through  branched  instruction.  A truer 
understanding  requires  a learning  environment  which  accommodates  the  testing,  falsifying,  validating,  and 
elaboration  of  understandings. 

The  Munchhausen-Trick:  Learning  Internet  by  Internet 

Ralph  Sontag,  TU  Chemnitz,  Germany;  Dietrich  Thie,  TU  Chemnitz,  Germany 

The  Chemnitz  University  of  Technology  is  the  first  one  in  Germany,  which  is  able  to  offer  an  internet-based  study 
about  information  and  communication  technology.  The  participants  of  the  courses  are  able  to  collect  fundamental 
and  applicable  knowledge  about  internet-related  technologies  using  it  - a problem  comparable  to  the  baron  of 
Munchhausen,  which  drags  himself  at  his  tuft  out  of  a marsh.  After  two  introducing  lessons  the  communication 
between  learners  and  teachers  is  stricly  internet-based.  Stepwise  the  students  are  qualified  to  understand  and 
maintain  complex  situations  in  management  of  ip-networks  and  servers.  The  knowledge  can  be  proofed  in  a remote 
managed,  but  actually  existing  laboratory.  Moreover  we  had  to  consider  the  very  heterogenous  working  times  of  our 
participants.  We  prefer  asynchronous  communication  with  short  turn-around-times  in  conjunction  with  online-tools 
like  "virtual  seminars",  tests  or  experiments.  The  consequently  platform-independent  project  uses  open  protocols 
and  standardized  document  formats  like  HTML  or  XML. 

In  defence  of  ‘shovelware’:  A powerful  tool  In  staff  development  for  online  teaching  and 

learning 

Heather  Sparrow,  Edith  Cowen  University,  Australia;  Jan  Herrington,  Edith  Cowen  University,  Australia;  Tony 

Herrington,  Edith  Cowen  University,  Australia 

A significant  challenge  for  universities  creating  effective  online  learning  environments  is  educating  and  empowering 
teaching  staff.  Those  who  are  not  confident  using  technology  are  frequently  unconvinced  about  the  benefits  and  are 
concerned  that  it  will  force  unwanted  changes  on  their  current  practices.  This  poster  outlines  a project  which 
introduced  a large  number  of  variously  experienced  academic  staff  to  online  teaching,  whilst  maintaining  or 
enhancing  effective  learning.  The  project  included  the  simple  conversion  of  print-based  external  units  to  an  online 
mode,  as  one  of  several  strategies  of  development.  Whilst  this  process  is  often  referred  to  derogatively  as 
shovelware,  some  of  the  positive  and  immediate  outcomes  achieved  will  be  described,  and  challenges  will  be  made 
to  extremely  negative  attitudes  towards  such  conversions. 

An  Evaluation  of  Video  Compression  Schemes  for  Teleteaching  Applications 

Ognjen  Stanovcic,  Open  Systems  AG  Zurich,  Switzerland;  Thomas  Walter,  Swiss  Federal  Institute  of  Technology 

Zurich,  Switzerland;  Bernhard  Plattner,  Swiss  Federal  Institute  of  Technology  Zurich,  Switzerland 

Synchronous,  interactive  teleteaching  is  demanding  with  respect  to  required  network  bandwidth  and  short  end-to- 
end  delays.  An  uncompressed  digital  video,  e.g.,  requires  a data  rate  of  261  MBit/ s.  Video  compression  is  a method 
to  reduce  required  network  bandwidth.  Efficient  video  compression  is  almost  lossless  with  respect  to  perceived 
video  quality  at  the  receiving  site,  but  it  is  time  intensive.  In  support  of  interactivity,  however,  response  times  must 
be  short  to  allow  participants  to  audio-visually  communicate  like  being  in  the  same  room.  This  implies  that  the  end- 
to-end  communication  delay  should  be  less  than  150  ms.  This  requirement  puts  a hard  constraint  on  video 
compression  since  compression  time  is  only  one  factor  that  adds  to  the  overall  end-to-end  delay.  We  present  a 
framework  for  the  evaluation  of  video  compression  schemes.  We  define  evaluation  criteria  such  as  compression 
delay  and  compression  efficiency.  The  video  compression  schemes  evaluated  are  Motion  JPEG,  MPEG,  H.261  and 
H.263. 

Multibook:  making  web  based  learning  resources  personal 

Achim  Steinacker,  TU  Darmstadt,  Germany;  Cornelia  Seeberg,  GMD  IPSi,  Germany;  Ralf  Steinmetz,  GMD IPSI, 

Germany 

Considering  the  amount  of  knowledge  accumulated  on  the  Web,  the  discussion  of  an  effective  use  of  the  web 
resources  becomes  urgent.  The  main  question  is  how  to  find  the  relevant  information  . In  our  web  based  teaching 
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system  Multibook  we  use  modulaiized  learning  content  stored  in  a knowledge  base  and  describe  these  modules  such 
that  the  combination  of  modules  can  be  matched  to  the  users  preferences,  represented  in  a user  profile.  In  this 
demo  we  will  show  our  knowledge  base  containing  both  the  actual  multimedia  learning  resource  and  the  metadata 
in  form  of  a terminological  ontology,  attributes  for  the  resources  and  relations  between  the  resources.  Then  we 
sketch  the  user  profile  and  finally  we  show  how  these  components  are  matched  together 

Math  and  Science  Curriculum  Revision:  A Collaborative  Approach  to  Improving 
Preservice  Teachers'  Use  of  Technology  Knowledge  and  Instructional  Skills. 

David  Stokes,  Westminster  College,  USA;  Carolyn  Jenkins,  Westminster  College,  USA;  Lorel  Huhnke,  Westminster 
College,  USA;  Gothard  Grey,  Westminster  College,  USA;  Cheryl  Manning,  Bryant  Intermediate  School,  USA 

Reports  on  a Preparing  Tomorrow's  Teachers  to  use  Technology  Capacity  Grant  project,  a consortium  comprising 
Bryant  Intermediate  School,  and  Westminster  College,  both  of  Salt  Lake  City,  Utah,  The  consortium  participants 


instructional  uses  of  technology  in  education.  Skills  in  the  application  of  communication,  presentations,  and  CLE 
technologies  were  explored.  Teaching  units  integrating  sound  curriculum  with  technologies  including:  video- 
microscopes, CBL  probes,  graphing  calculators,  ExceU,  PowerPoint,  various  software  programs,  and  the  Internet 
were  developed.  Outcomes  suggest  improved  teaching  with  technology  skills,  better  real-world  collaborative  inquiry, 
greater  focus  upon  multiple  learning  styles;  and  gains  in  extending  the  power  of  scientific  inquiry  into  mathematical 
and  scientific  reasoning. 


Integrating  pedagogy,  content  and  technology  into  your  curriculum 

David  Stokes,  Westminster  College,  USA;  Wanda  Carrasquillo-Gomez,  Westminster  College,  USA 

Perhaps  there  are  fewer  more  daunting  tasks  to  teacher  educators  than  to  find  the  time,  means,  and  information  to 
begin  to  more  fully  integrate  technology  as  an  instructional  and  learning  tool  into  their  courses.  Over  the  past  year 
the  authors  conducted  a thorough  curricular  revision  of  technology  courses  offered  in  the  Master  of  Education 
program  at  Westminster  College  in  Salt  Lake  City.  Courses  were  redesigned  focusing  curriculum  upon  technology  as 
a communication  rather  than  a presentation  medium.  Outcomes  suggest  this  approach  is  beneficial  to  generating 
increased  understanding  of  the  role  technology  can  play  in  teaching  and  learning,  and  in  strengthening  key  linkages 
between  constructivist  theory  and  teaching  practice. 

SMILE  Maker  - An  Intelligent  Learning  Environment  for  Problem  Solving 

Svetoslav  Stoyanov,  University  of  Twente,  The  Netherlands 

This  poster  introduces  SMILE  Maker  - a web-based  tool  supporting  ill-structured  problem  solving  activities  of 
students  involved  in  problem-based  learning.  SMILE  Maker  stands  for  Solution,  Mapping,  Intelligent  Learning 
Environment,  It  is  both  a problem  solving  and  a learning  tool.  As  a problem  solving  tool  it  proposes  a method  for 
creative  problem  solving  based  on  the  synergy  of  some  mapping  approaches  like  concept  mapping,  mind  mapping, 
cognitive  mapping,  flowscapping,  and  process  mapping,  and  a systematic  creative  problem  solving  methodology. 
The  method  facilitates  elicitation  of  tacit  knowledge,  an  effective  organization  of  the  explicit  knowledge,  and 
avoidance  of  some  negative  individual  problem  solving  syndromes  like  analysis-paralysis,  functional  fixedness, 
premature  judgment,  and  etc.  As  a learning  method  it  provides  an  individualized  learning  environment  for  studying 
and  applying  this  method.  It  adapts  learning  to  four  personal  constructs  such  as  learning  styles,  problem  solving 
styles,  learning  locus  of  control  and  prior  knowledge. 

An  Adaptive  Web-based  Course  in  Financial  Engineering  with  Dynamic  Assessment 

Alberto  Suarez,  Universidad  Autonoma  de  Madrid,  Spain;  Estrella  Pulido,  Universidad  Autonoma  de  Madrid,  Spain; 
Rosa  Carro,  Universidad  Autonoma  de  Madrid,  Spain 

The  increasing  relevance  of  interdisciplinary  skills  calls  for  the  design  and  implementation  of  novel  strategies  to 
address  the  problem  of  teaching  a corpus  of  knowledge  to  a heterogeneous  audience.  We  have  chosen  a problem  in 
Gfmancial  engineering,  the  valuation  of  European  options,  whose  mastery  requires  the  interplay  of  mathematical, 
computational  and  financial  tools.  The  target  audience  in  financial  engineering  courses  is  generally  composed 
students  o^f  either  financial  background  or  from  a scientific,  non-fmancial  area  (mathematics,  computer  science, 
physics).  The  course  is  designed  within  the  framework  of  the  TANGOW  system 
(http://www.ii.uam.es/esp/investigacion/tangow/present.html),  which  provides  the  necessary  mechanisms  to 
construct  a web-based  course  whose  contents  are  dynamically  generated  according  to  the  student's  profile  and 
actions.  The  system  includes  the  possibility  of  dynamic  assessment,  which  involves  testing  not  only  the  student's 
ability  to  understand  static  knowledge,  but  also  his/her  ability  to  actively  design  procedures  (programming  analytical 
formulas,  realization  of  simulations)  to  achieve  the  prescribed  goal. 
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Building  Learner  and  Teacher  Autonomy  for  New  Learning  Environments 

Leena  Subra,  University  of  Jyvaskyla,  Finland;  Liisa  Kallio,  University  of  Jyvaskyla,  Finland;  Ulla  Lautiainen, 

University  of  Jyvaskyla,  Finland 

The  poster  reports  on  the  experiences  of  our  departmental  action  research  project  aiming  to  explore  the  use  of  ICT- 
enhanced  methods  in  promoting  both  learner  and  teacher  autonomy/self-directedness  in  discipline-based  language 
learning  and  instruction.  The  action  research  approach  involves  continuous  staff  development,  needs  identification 
and  aims-setting,  teacher  and  learner  self-assessment,  learner  training,  and  development  of  a variety  of  learning  tasks 
suitable  for  self-directed  learning  in  both  authentic  and  virtual  learning  environments.  It  also  involves  an  attempt  of 
creating  a set  of  ICT-enhanced  course  formats  which  - without  significant,  time-consuming  and  technically 
demanding  modifications  - could  be  used  in  teaching  and  learning  languages.  The  poster  describes  pilot  courses 
which  use  different  approaches,  materials  and  programmes,  and  are  aimed  at  different  kinds  of  learners.  They  cover 
a variety  of  possibilities  and  combinations  but  remain  within  the  same  pedagogical  framework,  since  learner  and 
teacher  autonomy  are  the  basic  guidelines  in  all  the  pilot  courses. 

Development  of  a Computer  Aided  Cooperative  Classroom  Environment 

Akira  Suganuma,  Kyushu  University,  Japan;  Ryunosuke  Fujimoto,  Kyushu  University,  Japan;  Yutaka  Tsutsumi, 

Kumamoto  Gakuen  University,  Japan 

In  the  educational  domain,  a popularization  of  computers  and  the  Internet  enable  us  to  hold  lectures  using  Web 
contents  as  a teaching  material.  We  propose  CACCE,  Computer  Aided  Cooperative  Classroom  Environment, 
which  is  an  WWW-based  supporting  system  for  a classroom  teaching.  CACCE  consists  of  a server-software  and  a 
client-software,  that  is  to  say,  a teacher's  browser  and  a student's  browser.  These  browsers  are  connected  together 
by  a socket  connection,  sends  and  receives  some  messages  such  as  a URL  and  the  CACCE's  command.  There  are 
three  main  features  of  CACCE.  First,  CACCE  automatically  refreshes  student’s  browsers.  Secondly,  CACCE 
displays  a cooperated  mark  and  text  on  these  browsers.  Finally,  student's  browsers  automatically  report  an  URL  of 
page  shown  by  them  to  teacher's  browser.  This  information  is  used  as  one  of  the  guidelines  for  teaching.  It  is 
concluded  that  CACCE  is  not  only  a tool  that  broadcasts  the  document  on  a teacher’s  browser  to  students,  but  also 
a cooperative  environment. 

Using  Knowledge  Models  in  Intelligent  Tutoring  Systems 

Stephen  G.  Taylor,  Champlain  Regional  College,  Canada 

The  Virtual  Tutee  system  will  be  introduced  as  a prototype  of  an  intelligent  tutoring  system.  The  user  of  this  system 
adopts  the  role  of  the  tutor  and  studies  photosynthesis  by  tutoring  a virtual  tutee  residing  within  the  software.  The 
system  was  produced  as  a simulation  based  on  a model  of  the  tutoring  learning  situation  called  the  Knowledge 
Modelling  Approach,  which  will  also  be  introduced.  Fifty  college  students  participated  in  a test  of  the  system. 

An  Evaluation  of  Interactive  Studio  Physics  I 

Holly  Traver,  Rensselaer  Polytechnic  Institute,  USA;  Michael  Kalsher,  Rensselaer  Polytechnic  Institute,  USA;  Karen 

Cummings,  Rensselaer  Polytechnic  Institute,  USA;  Keith  Hmieleski,  Rensselaer  Polytechnic  Institute,  USA 

Rensselaer  Polytechnic  Institute  has  implemented  an  innovative  physics  course  (’’Studio  Physics")  that  promotes 
collaborative  interaction.  Students  work  in  groups  of  2-4  on  computers  connected  to  the  local  network  and 
Internet.  Class  time  is  spent  on  short  lecture  and  group  exercises.  This  research  study  evaluated  Studio  Physics  in 
terms  of  grades  and  alumni  responses.  An  examination  of  course  grades  from  students  who  took  the  first  courses  of 
Studio  Physics  yielded  no  significant  difference  between  traditional  lecture  and  studio  courses.  However,  as  the 
studio  course  has  evolved,  learning  has  improved.  Alumni  were  contacted  and  asked  for  their  opinions  on  all  types 
of  courses,  including  studio.  Alumni  indicated  working  coUaboratively  with  others  helped  them  to  improve 
communication,  interpersonal,  and  computer  skills,  as  well  as  the  abilities  to  work  with  others,  learn  new  material, 
and  perform  at  their  current  job.  This  finding  is  important  for  work  organizations,  since  much  of  work  today  is 
group-based. 

Grasp  of  Few  body  element's  correlative  operation  using  Multi  points  measure  data- 

Compair  with  2 kind  selection- 

Atsushi  Tsubokura,  Osaka  Electro-Communiation  Univ.,  Japan;  Kiyoshi  Masui,  Osaka  Electro-Communication  Univ., 

Japan;  Noboru  Ashida,  Osaka  Electro-Communication  Univ.,  Japan;  Kagemasa  Kozuki,  Konami  Co.,  Japan; 

Katsuhide  Tsushima,  Osaka  Electro-Communication  Univ.,  Japan 

Expert  Operator  has  any  skill  for  machine  operation.But  novice  has  a litde  skiU.The  novice  do  and  error  operate 
when  try  need  to  new  operation  skill. Our  target  is  to  trace  the  Operator’s  skill  learning  process  and  the  task 
identificate  for  the  skill  learned.The  newer  operation  skill  based  another  already  learned  operation  skill.These 
Operator's  skil  was  changing  during  Try  Operating.The  changed  operation  can  show  operation  timming  and  used 
skill  for  operation.  We  used  few  body's  measure  data  and  identificate  his  operation  using  symbolic  way.Measure 
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device  is  Eye  Mark  Recorder, 3D  position  or  degress  sencer(for  Arm  Motion), Data  Grove.This  paper  propose  the 
symbolic  way  for  Multi  Points  Measure  data's  correlation  and  single  measure  data’s  characteristic,  and  evaluate  the 
measure  data  using  these  way  .This  paper  is  introduction  of  these  reserch. 

Offline  projects  for  cross-cultural  competence  improvement 

Victoria  Tuziukova,  Rostov  State  Pedagogical  University,  Russia;  Irina  Rozina,  Rostov  State  Pedagogical  University, 
Russia;  Cyndy  Woods,  Arizona  state  University,  USA 

Russian  and  U.S  faculty  members  from  the  Department  of  Foreign  Languages,  Rostov  State  Pedagogical  University 
- RSPU,  Russia  and  Arizona  State  University,  Phoenix,  USA,  developed  off-line  writing  projects  designed  to 
increase  language  proficiency,  cross-cultural  awareness,  computer  and  networking  skills.  Targeting  EFL  students 
(English  as  a Foreign  Language),  the  project  has  demonstrated  that  students  not  only  increase  English  language 
proficiency,  but  students  learn  more  about  the  way  of  life,  basic  values,  and  cultural  background  of  the  people, 
whose  language  they  study. 

Analyzing  Artists'  Interaction  with  an  Artificial  Reality 

Elpida  Tzafestas,  National  Technical  University  of  Athens,  Greece 

Our  long  term  goal  is  to  understand  how  humans  buUd  models  of  biological  phenomena.  To  this  end,  we  are 
studying  the  relation  between  an  artificial  reality  and  the  model  of  it  that  an  observer  deduces.  The  study  is  currently 
based  on  the  observation  of  how  Digital  Art  students  interact  with  "PainterAjits",  an  educational  simulation 
software  that  is  used  for  laboratory  work  on  intelligent  and  complex  systems.  The  software  is  a special-purpose  ant 
population  demo  that  tries  to  convey  and  teach  regulation  principles  using  graphical  means.  Artists  have  shown  a 
high  motivation  to  experiment  with  the  system,  partly  because  they  tend  to  regard  it  as  a simple  abstract  art  tool. 
They  manage  to  create  a functional  model  of  complexity,  without  really  understanding  the  underlying  models.  Few 
experiments  with  users  of  different  background,  have  shown  that  humans,  independently  of  background 
knowledge,  tend  to  fmd  complex  top-down  explanations  to  apparently  complex  phenomena. 

Supporting  Virtual  Classrooms  through  Extranet  technology:  the  Eurydices  system 

Kostas  Bovilas,  University  of  Patras,  Greece;  Aspasia  Kanta,University  of  Patras,  Greece;  Nikos  Piskopos,  University 
of  Patras,  Greece;  Bill  Vassiliadis,  University  of  Patras,  Greece;  Athanassios  Tsakalidis,  University  of  Patras,  Greece; 
John  Tsaknakis,  University  of  Patras,  Greece;  Evangelos  Sakkopoulos,  University  of  Patras,  Greece;  Christos 
Makris,  University  of  Patras,  Greece 

This  work  describes  how  collaborative,  extranet-based  services  can  be  used  to  teach  virtual  classes  to 
geographically-dispersed  academic  participants.  Eurydices  is  an  extranet-based  training  application,  whose  aim  is  the 
efficient  management  of  the  great  variety  of  users  and  courses  provided  by  the  Greek  Universities  Network 
(GUNet). 

A Web-based  educational  tool  for  solving  equations 

Maria  Virvou,  University  of  Piraeus,  Greece;  Maria  Moundridou,  University  of  Piraeus,  Greece;  Michael  Loizos, 
University  of  Piraeus,  Greece;  Athanasios  Papachristou,  University  of  Piraeus,  Greece;  Nikolaos  Polyzotis,  University 
of  Piraeus,  Greece 

This  paper  describes  a web-based  intelligent  authoring  tool  for  constructing  problems  in  the  area  of  equation 
solving.  The  main  objective  of  this  tool  is  to  be  useful  to  teachers  and  students  of  high  school  level  in  the  area  of 
algebraic  equations.  TTie  teacher  using  this  tool  is  able  to  provide  the  information  needed  to  construct  an  exercise 
through  a user-friendly  interface  on  the  web.  Every  exercise  that  has  been  constructed  is  immediately  available  to 
students.  The  tool  also  monitors  students  closely  while  they  are  solving  the  exercises  through  its  student  modelling 
component.  The  aspects  we  consider  important  in  this  tool  are: 

• The  authoring  capabilities  it  embodies.  Teachers  can  continuously  add  new  exercises  and  be  assisted  at 
their  construction.  Therefore  the  tool  can  be  very  useful  for  the  construction  of  student  assignments 
and/or  exam  tests. 

• It  is  a Web-based  tool.  Therefore  it  can  be  used  as  a support  tool  for  distance  learning.  It  can  also  be  used 
in  classrooms  and/or  home  as  homework,  exam  tests,  student  practice  or  tutorial. 

• The  tool  has  ITS  features  such  as  the  diagnostic  capability.  In  this  sense  the  tool  can  help  the  teacher  to 
assess  his/her  students'  progress  and  level  of  knowledge.  It  can  also  be  very  helpful  at  marking  the 
assignments  and/or  exam  tests.  In  addition  it  can  be  very  useful  to  students  in  two  ways.  First  for  the  self- 
assessment  of  their  skills  and  second  for  the  practice  they  can  have  with  the  provision  of  individualised 
feedback. 
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Constructing  a Learning  Environment  for  Knowledge  Advancement 

Feng-Kwei  Wang,  University  of  Missouri  - Columbia,  USA 

The  purpose  of  this  research  is  to  introduce  the  use  of  technology  to  construct  a learning  environment  for 
knowledge  advancement  through  three  levels  of  knowledge  — know- what,  know-how,  and  know-why.  Know-what 
is  concept-based  domain  content  which  is  prepackaged  and  generalized.  Know-how  translates  know-what  into 
action-oriented  skills.  Know- why  permits  one  to  renew  know-what  and  know-how  and  thus  to  solve  larger  and 
more  complex  problems.  Different  levels  of  knowledge  require  different  learning  methods  and  entail  different 
learning  activities.  The  most  efficient  approach  to  acquiring  know-what  is  through  comprehension  and  feedback. 
The  process-based  know-how  can  be  best  achieved  through  practices  in  simulation  and  case  studies.  Know-why  can 
only  be  built  over  time  through  personal  encounters  with  many  different  problems  and  solutions  in  real-world 
projects.  This  poster  session  will  depict  the  theoretical  grounds  of  this  knowledge  advancement  framework  and 
show  how  technology  is  employed  to  construct  a learning  environment  based  on  this  framework. 

The  Use  of  Cooperative  and  Collaborative  learning  in  a Web-based  Integrated  Curriculum 

for  the  First  Two  Years  of  an  Engineering  Program  of  Study 

John  Watret,  Embry-Riddle  Aeronautical  University,  USA;  Charles  Martin,  Embry-Riddle  Aeronautical  University, 

USA 

The  authors  will  describe  and  present  the  results  of  an  innovative  Integrated  Curriculum  in  Engineering  (ICE) 
designed  for  the  first  two  years  of  an  engineering  program.  Entering  freshmen  engineering  student  volunteers  are 
placed  into  specifically  designed  sections  of  the  foundation  courses  required  of  all  engineering  students:  the  calculus 
sequence,  the  physics  sequence,  the  humanities  sequence,  social  science,  and  introductory  engineering  courses. 
What  differentiates  the  ICE  program  from  traditional  engineering  curricula  is  that  all  courses  incorporate 
cooperative  and  collaborative  learning  and  a reliance  in  computer  and  web  technology.  In  addition,  the  ICE 
program  promotes  team  learning,  team  design  projects,  and  a weU-documented  series  of  assessment  practices. 
Although  our  program  was  written  for  an  engineering  curriculum,  the  underlying  philosophy  is  applicable  to  a wide 
spectrum  of  programs  of  study  as  well  as  the  general  education  component  of  many  degrees. 

Distance  Learning  For  Government  Agencies  / Online  Academies 

Joseph  Wiiczak,  American  West  Enterprise,  USA 

Government  agencies  are  turning  to  online  presentation  formats  for  computer  based  instruction  related  to  distance 
learning.  CBT  lends  itself  well  to  innovative  visual  curriculum  and  realtime  information  sharing.  The  educational  / 
training  implications  that  combine  computer  technology,  the  internet,  email,  and  telecommunications  are 
enormously  untapped,  CD  Rom,  online  student  / instructor  interaction  through  chat  rooms,  or  both  are  utilized. 
Material  is  presented  in  text  supported  with  voice  overs,  and  graphics.  While  writing  screen  text,  voice  over  scripts 
should  be  written  simultaneously,  and  graphics  last.  Lectures  can  be  conducted  online  supported  by  video 
telecommunications,  A final  online  exam  is  provided  at  the  completion  of  each  course  with  follow  up  sessions 
scheduled  depending  on  course  content.  Presentations  are  suitable  for  newly  enrolled  student  course  material  as  well 
as  recertification  or  ongoing  certification  requirements  of  the  particular  agency. 

The  Management  of  the  Telecommunications  Function:  The  Impact  on  Organizational 

Support,  Planning  and  Training  Quality 

John  Willems,  Eastern  Illinois  University,  USA;  Karen  Ketler,  Eastern  Illinois  University,  USA 

This  article  reports  the  results  of  a survey  of  marketing  managers  in  order  to  obtain  information  from  the  end  user 
about  the  impact  of  the  management  structure  of  the  telecommunications  function  on  1)  the  importance  of  the 
function  within  the  organization,  2)  the  formalization  of  planning  practices  and  3)  the  quality  of  training  on  thirty 
issues.  While  the  information  systems  manager  was  the  most  common  form  of  management  of  the 
telecommunication  function,  the  organizations  with  a distinct  telecommunications  manager  reported  greater 
satisfaction  with  top  management  support  and  a more  formalized  telecommunications  planning  and  implementation 
process  than  the  other  organizations.  The  respondents  from  the  organizations  with  a telecommunications  manager 
also  indicated  better  telecommunications  training  (although  still  somewhat  unsatisfactory)  than  the  other 
respondents.  This  was  especially  true  in  the  managerial  issues  with  a long  term  focus  such  as  the  use  of 
telecommunications  to  gain  a competitive  advantage,  and  managing  innovation  and  technology. 

Online  Learning  Communities;  Vehicles  for  Collaboration  in 

Stephanie  Woolley,  University  of  Colorado  at  Denver,  United  States;  Stacey  Ludwig-Hardman,  University  of  Colorado 

at  Denver,  United  States 

Learning  communities  are  environments  that  encourage  mutual  exchange  between  community  members  to  support 
their  individual  and  collective  learning.  Learning  communities  are  founded  on  the  social  negotiation  of  meaning. 
Collaboration  is  the  key  tenet  of  constructivism  and  small-group  theory  (Springer  Springer,  Stanne,  & Donovan, 


1999;  Grabinger,  1995;  Duffy  & Jonassen,  1992)  that  explains  the  social  component  of  learning  and  demonstrates 
that  “conceptual  growth  comes  from  sharing  perspectives  and  modifying  our  internal  representations  in  response  to 
that  sharing'  (Grabinger,  1995,  p.  669).  Online  learning  communities  provide  the  means  for  achieving  “shared 
creation”  and  “shared  understanding”  (Shrage,  1991,  p.  40).  They  are  an  excellent  tool  for  integrating  collaboration 
in  online  learning  environments.  The  authors’  poster  demonstration  defines  collaboration  and  learning  communities 
and  addresses  their  theoretical  foundations.  The  authors  will  discuss  the  benefits  of  collaboration  and  learning 
communities  in  online  learning  environments  and  provide  a case  study  of  a new  online  learning  community  at 
Western  Governors  University. 

The  Construction  of  World-Wide-Web  Resource  for  Chinese  Medicine  and  Acupuncture 

Fan  Wu,  China  Medical  College,  Taiwan;  Shao-Fu  Huang,  China  Medical  College,  Taiwan 

As  the  information  technolog)^  makes  great  strides  in  recent  years,  Internet  has  irresistibly  reached  into  every  aspect 
of  our  daily  life.  With  its  unique  capabilir/  of  well-integrating  text,  Lmage  and  video,  World-Wide-Web  has  been 
pervasive  in  the  dissemination  of  medical  discovery,  the  promotion  of  medical  education  and  the  application  of 
clinical  medicine.  The  issue  of  capitalizing  on  the  state-of-the-art  web  technology  to  build  Chinese  medicine  and 
acupuncture  multi-media  web  resource  is  well  addressed  in  this  thesis.  It  is  organized  around: 

1.  The  largest  Chinese  medicine  news  web  site  domestically. 

2.  The  Chinese-medicine-centric  high  resolution  multi-media  Video  on  Demand  in  the  field  of  Chinese 
medicine. 

3.  The  electronic  journal  on  Chinese  medicine  in  the  field  of  Chinese  medicine. 

4.  The  virtual  classroom  with  complete  features  on  Chinese  Medicine  and  group-learning  systems. 

In  addition,  aU  the  relevant  web  sites  on  Chinese  medicine  will  be  listed  and  linked  for  the  benefit  of  the  public. 
Integrated  with  Chinese  medicine  database,  this  set  of  medical  resource  helps  to  present  teachers  and  researchers 
with  an  informative  environment  surrounded  by  Chinese  medicine  data.  And  it  gives  students  the  chance  of 
individual  learning  and  the  space  for  the  development  for  voluntary  acquisitiveness  for  knowledge  as  well  as  paves 
the  way  for  academic  exchanges  and  pervasion  of  the  information  on  Chinese  medicine. 

Automatic  Generation  of  the  Optimal  Tutorial-Plan  in  Adaptive  Educational  Hypermedia 
System 

Haiyan  Xu,  University  of  Science  & Technology  of  China,  P.R.China;  XueHai  Zhou,  University  of  Science  & 

Technology  of  China,  P.R.China;  Zhenxi  Zhao,  University  of  Science  & Technology  of  China,  P.R.China 

Adaptive  hypermedia  system  provides  a good  framework  for  web-based  education.  An  ideal  adaptive  educational 
hypermedia  system  adapts  to  different  learners  during  the  whole  period  of  the  study.  Aiming  at  the  automatic 
generation  of  the  adaptive  tutorial-plan,  we  simply  introduce  the  architecture  of  the  KDAEHS:  an  adaptive 
educational  hypermedia  system  based  on  structural  computing,  propose  the  concept  of  knowledge  structure  graph 
represented  by  AND/OR  graph  and  then  discusses  the  algorithms  used  to  obtain  optimal  tutorial  plan.  Combined 
with  the  pedagogical  strategies  specified  by  the  instructors,  KDAEHS  generates  the  optimal  tutorial-plan  which 
adapts  to  different  learners  based  on  the  knowledge  structure  graph  and  the  learners’  knowledge  level.  Every 
tutorial-plan  is  well  suit  to  the  individual  learner  and  can  help  the  learner  to  fulfill  the  study  goal  efficiently. 

Virtual  Reality  in  astronomy  teaching 

Yoav  Yair,  CET  / The  Open  University,  Israel;  Rachel  Mintz,  CET  / Tel-Aviv  University,  Israel;  Shai  Litvak,  CET  / Tel- 
Aviv  University,  Israel 

A new  3D  model  of  the  solar  system  with  virtual  reality  (VR)  features.  It  is  based  on  powerful  scientific  visualization 
techniques  and  can  be  used  as  an  effective  aide  in  astronomy  teaching  is  presented.  The  model  allows  for  a powerful 
learning  experience,  and  facilitates  the  mental  construction  of  three-dimensional  space,  where  objects  are  varied  and 
different,  but  share  common  features  and  obey  the  same  physical  principles.  The  learner  “enters”  a virtual  model  of 
the  physical  world,  journeys  through  it,  zooms  in  or  out  as  he  wishes,  changes  his  view  point  and  perspective,  as  the 
virtual  world  continues  to  ^^behave”  and  operate  in  its  usual  manner.  The  new  view  helps  to  overcome  the  inherent 
geocentric  view  and  ensures  the  transition  to  a scientific,  heliocentric  view  of  the  solar  system. 

Uncovering  cognitive  processes  in  discourse  synthesis  using  hypermedia 

Shu  Ching  Yang,  National  Sun  Yat-Sen  University,  Taiwan  ROC 

An  exploratory  study  was  undertaken  to  uncover  the  cognitive  process  of  discourse  synthesis  within  a hypermedia 
learning  environment.  The  focus  was  on  learners’  discourse  synthesis  behavior  and  the  problem  solving  strategies 
they  employed  using  the  Perseus  database.  The  data  for  the  study  was  transcribed  from  tape  recordings  of  the 
’thinking  aloud’  protocols  as  the  learners  attempted  to  work  on  their  tasks.  All  protocols  were  transcribed  verbatim 
and  categorized  according  to  each  strategies  content.  A functional  taxonomy  were  developed  to  characterize  the 
cognitive  operations  and  thinking  processes  of  the  learners’  interactions  with  Perseus.  The  proposed  taxonomy 
characterizes  their  cognitive  engagement  within  hypermedia  from  a broader  context.  It  is  a multifaceted  construct 
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which  entails  thick  cognition  with  many  highly  correlated  factors.  The  study  is  concluded  with  practical  and 
theoretical  concerns  emerging  out  of  this  focus  on  discourse  synthesis  within  hypermedia. 

Integrating  Technology  Into  the  Curriculum 

Rosanne  Yost,  University  of  South  Dakota,  USA;  Ray  Thompson,  University  of  South  Dakota,  USA 

This  session  describes  a teacher  education  course  designed  to  help  K-12  educators  integrate  technology  into  their 
curriculums.  The  Technolog)^  for  Training  and  Development  (TTD)  Division  at  the  University  of  South  Dakota  has 
developed  the  course  by  using  the  same  curriculum  planning  process  that  the  course  is  designed  to  teach.  The 
overall  goal  of  TTD  782:  Technology  Integration,  is  to  help  teachers  develop  an  understanding  of  the  potential  for 
technology  to  transform  school  curriculum  so  that  it  is  student  centered  and  future  orientated.  As  the  professors 
have  modeled  the  curriculum  planning  process  they  teach  to  their  students  they  have  themselves  become  better  at 
using  technology.  The  benefits  to  both  teacher  educators  and  classroom  teachers  have  included  classrooms  where 
active  learning  occurs  and  where  students  gain  some  control  over  there  own  learning,  A variety  of  learning  styles 
and  needs  are  now  being  met  as  teachers  and  students  learn  "how  to  learn"  together. 

Using  Web  Resources  to  Enhance  the  Interdisciplinary  Nature  of  Freshman  Preceptorial 

Course 

Kesheng  Yu,  Union  College,  USA 

The  purpose  of  this  poster  is  to  demonstrate  the  design  and  development  of  the  web-based  set  of  materials 
designed  to  help  instructors  become  familiar  with  topics  or  readings  in  which  the  preceptors’  group  has  little 
expertise.  The  focus  of  Freshman  Preceptorial  is  on  cultural  difference  and  on  contemporary  issues  seen  in 
historical  or  philosophical  perspective.  Three  Freshman  Preceptorial  projects  were  funded  in  1998  and  1999.  While 
two  of  these  projects  are  still  in  the  progress,  the  Resources  for  Teaching  the  Bible  are  finished.  This  website  is 
designed  for  faculty  teaching  and  students  taking  Freshman  Preceptorial,  though  others  are  certainly  welcome  to  use 
it.  VC^ile  the  demonstration  of  the  design  and  development  of  the  web  resources,  the  effect  of  web  resources  on 
Freshman  Preceptorial  course  and  student  attitude  toward  the  use  of  the  web  resources  would  be  discussed  as  well. 
The  URL  for  the  web  site:  http://www.union.edu/RESOURCES/curriculum/fpbible/ 

Development  of  an  Virtual  University  Online  Course  Using  an  Interprofessional  Approach 

to  Teach  Qualitative  Research 

Cheryl  Zaccagnini,  Shippensburg  University,  US;  Denise  Anderson,  Shippensburg  University,  US;  Kent  Chrisman, 

Shippensburg  University,  US 

Using  an  interprofessional  approach,  a qualitative  research  course  was  designed  and  taught  as  a virtual  university 
course.  Professors  from  the  disciplines  of  early  childhood,  special  education,  and  social  work  developed  a qualitative 
research  online  course  to  utilize  virtual  university  resources  to  reach  graduate  and  undergraduate  students  at  distant 
education  sites.  It  was  developed  to  implement  the  course  as  an  interactive  research  course  with  web-based 
programming  and  video  conferencing  sites  to  enhance  the  quality  of  the  course  and  to  provide  sufficient  web-based 
content  but  most  importantly  to  create  an  atmosphere  of  communication  and  connectedness  with  the  professors 
and  students  at  other  sites.  Syllabi  was  developed  using  a team  teaching  approach  with  access  to  the  professors  via 
video  conferencing,  e-mail,  and  web-based  instruction.  Formative  and  summative  evaluation  results  indicate  that 
teaching  qualitative  research  through  virtual  university  resources  can  be  an  effective  alternative. 

Going  the  DistancerOffering  Design  Curriculum  in  the  University  of  Utah’s  Distance 

Learning  MFA  in  Directing/Theatre  Education 

David  Zemmels,  University  of  Utah,  USA 

In  the  fall  of  1999,  the  University  of  Utah’s  Department  of  Theatre,  in  collaboration  with  Sundance 
Institute/Sundance  Theatre  Lab  admitted  the  first  cohort  to  its  pilot  distance  learning  MFA  in  Directing/Theatre 
Education.  The  intent  of  this  session  is  to  provide  an  overview  of  the  pilot  program — its  underpinnings,  its 
infrastructure — and  examine  one  solution  to  offering  a theatrical  design  course  within  the  parameters  set  by  the 
program.  Demonstration  and  examples  of  assignments  and  delivery  methods  will  help  conference  participants 
envision  ways  to  use  distance  learning  technologies  as  part  of  program  development  at  their  home  sites. 

Merging  Fine  and  Performing  Art  with  Digital  Technology:An  Exploration  of  the 

University  of  Utah's  Arts  Technology  Certificate  Program 

David  Zemmels,  University  of  Utah,  USA 

Successfully  integrating  traditional  forms  of  artistic  expression  with  digital  technology  is  an  ongoing  challenge  in  the 
fine  and  performing  arts.  In  the  21st  Century,  digital  art  forms  will  likely  become  even  more  important.  Computer 
technology  has  become  a pervasive  and  necessary  aspect  of  the  arts,  whether  you  are  a music  therapist,  art  historian, 
actor,  musician,  designer,  photographer,  or  painter.  This  past  faU,  the  University  of  Utah’s  College  of  Fine  Arts 
began  the  Arts  Technology  Certificate  Program,  an  interdisciplinary  pilot  program  offering  classes  in  digital 


1337 


BEST  COPY  AVAILABLE 


technology  as  a tool  for  artistic  expression  across  all  art  forms.  The  intent  of  this  session  is  to  provide  an  overview 
of  the  pilot  program  and  the  methods  and  strategies  used  to  implement  it,  and  to  showcase  representative  examples 
of  student  work.  Demonstration  participants  will  also  explore  ways  to  integrate  digital  technology  curriculum  as  part 
of  program  development  at  their  home  sites. 

Comprehensive  Examinations  via  e-mail 

David  Zimmerman,  James  Madison  University,  USA 

This  paper  will  examine  the  electronic  administration  of  the  written  comprehensive  examination  for  the  Masters 
Degree  in  School  Library  Media  Services  from  James  Madison  University,  Prior  to  the  fall  semester,  1998,  all 
students  traveled  to  the  main  campus  in  Harrisonburg,  Virginia  to  take  the  three  hour  exam.  Beginning  in 
November,  1998,  four  students  were  sent  the  exam  via  e-mail  and  returned  their  responses  via  the  same  method. 
The  following  paper  will  include  a comparison  of  the  two  methods  of  administration,  the  results  of  a seven-item 
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